Imagine asking a librarian "What was happening in Los Angeles in the Summer of 1989?" and that person fetching you ... That's modern LLMs in a nutshell.

I agree, but I think you're still being too generous to LLMs. A librarian who fetched all those things would at least understand the question. An LLM is just trying to generate words that might logically follow the words you used.

IMO, one of the key ideas with the Chinese Room is that there's an assumption that the computer / book in the Chinese Room experiment has infinite capacity in some way. So, no matter what symbols are passed to it, it can come up with an appropriate response. But, obviously, while LLMs are incredibly huge, they can never be infinite. As a result, they can often be "fooled" when they're given input that semantically similar to a meme, joke or logic puzzle. The vast majority of the training data that matches the input is the meme, or joke, or logic puzzle. LLMs can't reason so they can't distinguish between "this is just a rephrasing of that meme" and "this is similar to that meme but distinct in an important way".

[–] jsomae@lemmy.ml 0 points 1 week ago (11 children)

Can you explain the difference between understanding the question and generating the words that might logically follow? I'm aware that it's essentially a more powerful version of how auto-correct works, but why should we assume that shows some lack of understanding at a deep level somehow?

[–] merc@sh.itjust.works 1 points 1 week ago (5 children)

Can you explain the difference between understanding the question and generating the words that might logically follow?

I mean, it's pretty obvious. Take someone like Rowan Atkinson whose death has been misreported multiple times. If you ask a computer system "Is Rowan Atkinson Dead?" you want it to understand the question and give you a yes/no response based on actual facts in its database. A well designed program would know to prioritize recent reports as being more authoritative than older ones. It would know which sources to trust, and which not to trust.

An LLM will just generate text that is statistically likely to follow the question. Because there have been many hoaxes about his death, it might use that as a basis and generate a response indicating he's dead. But, because those hoaxes have also been debunked many times, it might use that as a basis instead and generate a response indicating that he's alive.

So, if he really did just die and it was reported in reliable fact-checked news sources, the LLM might say "No, Rowan Atkinson is alive, his death was reported via a viral video, but that video was a hoax."

but why should we assume that shows some lack of understanding

Because we know what "understanding" is, and that it isn't simply finding words that are likely to appear following the chain of words up to that point.

[–] KeenFlame@feddit.nu 1 points 1 week ago

Just if you were a hater that would be cool with me. I don't like "ai" either. The explanations you give are misleading at best. It's embarrassing. You fail to realise the fact that NOBODY KNOWS why or how they work. It's just extreme folly to pretend you know these things. It's been observed to reason novel ideas which is why it is confusing for scientists that work with them why it happens. It's not just data lookup. You think entire Web and history of man fits in 8 gb? You are just educating people with just your basic rage filled opinion, not actual answers. You are angry at the discovery, we get that. You don't believe in it. Ok. But don't say you know what it does and how, or what openai does behind its closed doors. It's just embarrassing. We are working on papers to try to explain the emergent phenomenon we discovered in neural nets that make it seem like it can reason and output mostly correct answers to difficult questions. It's not in the "data" and it looks for it. You could just start learning if you want to be an educator in the field.

load more comments (4 replies)

load more comments (9 replies)