The next comment is so peak tech hubris to me.

It's "just" predicting the next token so it means nothing

This form of argument should raise red flags for everyone. It is an argument against the possibility of emergence, that a sufficient number of simple systems cannot give rise to more complex ones. Human beings are “just” a collection of cells. Calculators are “just” a stupid electric circuit.

The fact is, putting basic components together is the only way we know how to make things. We can use those smaller component to make a more complex thing to accomplish a more complex task. And emergence is everywhere in nature as well.

This is the part of the AGI Discourse I hate because anyone can approach this with aesthetic and analogy from any field at all to make any argument about AI and its just mind-grating.

This form of argument should raise red flags for everyone. It is an argument against the possibility of emergence, that a sufficient number of simple systems cannot give rise to more complex ones. Human beings are “just” a collection of cells. Calculators are “just” a stupid electric circuit.

I've never seen a non-sequitur more non. The argument is that predicting the next term is categorically not what language is. That is, it's not that there is nothing emerging, but that what is emerging is just straight up not language.

The fact is, putting basic components together is the only way we know how to make things. We can use those smaller component to make a more complex thing to accomplish a more complex task. And emergence is everywhere in nature as well.

"Look! This person thinks predicting the next token is not consciousness. I bet they must also not believe that humans are made of cells, or that many small things can make complex thing. I bet they also believe the soul exists and lives in the pineal gland just like old NON-SCIENCE PEOPLE."

the concept of intelligence testing is so central to rats (and therefore to a big portion of HN’s poster base via cultural osmosis) that when folks like this lose their faith in GI, they tend to abandon the site as a whole

Succumbing to AIcoholism here

hah, that's a worthy contender

Basically: AI is (potentially?) useful, but LLMs require substantially more data than a human brain to do what they do, which is limited at best - and often less able for generalised cases than a well-defined physics model. The ideas aren't even new, having their roots in theoretical approaches from the 1940s and applied approaches from the 1980s, but they have a lot more training data and processing power now, which makes it seem more impressive. Even if all of the data in the universe was present, this would not lead to AGI because LLMs can't figure out the "why".

But I don't think there's anything new asserted in that article if you're familiar with the space and the promptfans will dismiss it anyway.

Yeah, it's worth examining. I didn't find any good takeaways, but I feel that they stated their case in a citation-supported manner; it looks like a decent article to throw at folks who claim that LLMs are intelligent.

to be honest I’m in the same boat. it’s tempting but I don’t know if I have the fortitude this week to actually engage with it