What is a Chinese room?

The Chinese room argument makes no sense to me. I cant see how its different from how young children understand and learn language.

My 2 year old sometimes unmistakable start counting when playing. (Countdown for lift off) Most numbers are gibberish but often he says a real number in the midst of it. He clearly is just copying and does not understand what counting is. At some point though he will not only count correctly but he will also be able to answer math questions. At what point does he “understand” at what point would you consider that chatgpt “understands”  There was this old tv programm where some then ai experts discussed the chinese room but they used a chinese restaurant for a more realistic setting. This ended with “So if i walk into a chinese restaurant, pick sm out on the chinese menu and can answer anything the waiter may ask, in chinese. Do i know or understand chinese? I remember the parties agreeing to disagree at that point.

My gripe with the Chinese room is that Searle argues that his inability to understand Chinese means the program doesn't understand Chinese, but I could say the same thing about the human body.

The neurons that operate your vocal chords have no idea what they're saying, nor the ones in your hands any idea what they're writing, yet they can speak and write exactly because your brain tells them what to do. Your brain is exactly like that book as far as your mouth and hand neurons are concerned.

They don't need to understand language at all for your brain to be able to understand it and give instructions based on that understanding.

My only argument is at what point does an algorithm become sufficiently advanced that it is indistinguishable from a conscious being?

Because at the end of the day, most of what a brain does is information processing based on what it has previously learnt, and that's exactly what the algorithm is doing based on training data. A sufficient enough algorithm should surely be able to replicate understanding.

Sure, that isn't ChatGPT as we know it, as you can tell from its sometimes very zany responses that while it understands what words are valid responses, it doesn't understand what the words themselves mean, but we should reach that at some point, no?

Well mostly the flaw is people assigning the test abilities it was never intended. Like testing intelligence. Turing outright as first thing in the paper presenting "imitation game" noted moving away from testing intelligence, since he didn't know to do that. Even on the realm of "testing intelligent kind of behavior" well more like human like behavior and human being here proxy for intelligent, it was mostly an academic research idea. Not a concrete test meant to be some milestone.

If the meaning of the words ‘machine’ and ‘think’ are to be found by examining how they are commonly useit is difficult to escape the conclusion that the meaning and the answer to the question, ‘Can machines think?’ is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.

Turing wanted a way to step away from stuff like "thinking" and "intelligence" directly and then proposed "imitation game" mostly to the rest of the academia as way to develop computer systemics more towards "intelligent behavior". It was mostly like "hey we need some goal to have as a goal to have something to move towards with these intelligence things. This isn't intelligence, but it might be usefull goal or tool for development work". Since without some goal/project/aim to have project don't advance. So it was "how about we try to develop a thing, that can beat this imitation game. Wouldn't that be good stepping stone. Then we can move to the actual serious stuff. Just an idea".

However since this academic "thinking out aloud spitballing ideas" was uttered by the Alan Turing, it became the Turing Test and everyone started taking it way too seriously. Specially outside academia. Who yes did play the imitation game with their programs as it was intended as research and development tool.

exemplified by for example this little exerpt of "not trying to do anything too complete and ground breaking here":

In any case there is no intention to investigate here the theory of the game, and it will be assumed that the best strategy is to try to provide answers that would naturally be given by a man

It is pretty literally "I had a thought". Turin makes no claims of machine beating the game having any significance other than "machine beat this game I came up with, neat". There is no argument of if machine beats imitation game, then X or then it means Y is reached.

Rest of the paper is actually about objections to the core idea of "it could ever be possible for machine to think" and even as such said imitation game is kinda lead in or introduction to Turing's treatise various objections of various "it would be impossible for machine to think" arguments. Starting with theological argument of "only human soul can think. Hence no animal or machine can think." .... since it was 1950's.

I don’t understand how Chinese room is a valuable argument. To me, while the person inside the room doesn’t understand Chinese, the system room-person-instructions does. You don’t argue that you don’t understand your language because none of your individual neurons understand it.

I don’t claim that chatGPT “understands” the language, I just don’t think that this argument applies in general.