The models and methods are improving. Especially through tool use (Internet search, MCP, using programming languages) the model output improves a lot. Reasoning models are allowed to admit mistakes (during thinking) "wait, that's wrong" (in normal conversation they will never say that if you don't point it at the mistake). Otherwise they basically predict tokens, the inference engine selects one and they go with what was selected.

It's a bit like you remember some wrong information (Mandela effect), you're confident it's correct so you don't double check and go with it. They usually don't even know how confident they are, they have no introspection.

In software development scenarios LLMs, due to their high "compression", often hallucinate (misremember) methods and parameters that don't exist in APIs or in different APIs or they don't know about new versions of the API. Many of those errors are catched when the code doesn't compile or unit tests fail but some of them stay (e.g. if the model created the unit tests and they don't test what they're supposed to test).

Also a bit like humans the models often don't have the whole codebase in the context so they make assumptions about the rest. Since they have no introspection they often don't double check if those assumptions are correct.

In case of frontend design they often can't "see" the output or at least not in the way way we do. They don't really know if something looks "good" or not (depending on their training).

Verification with other agents can help but fundamentally these agents have the same issues. It's a workaround.

I'm actually not sure if the bubble will pop. I believe LLMs can be useful in some fields but they're but the path to AGI. They're also way too resource intensive and used in a lot of situations where it's dangerous or doesn't make sense.

However, it's not that the AI researchers don't know about the limitations. They trying to work around the issues of the LLMs with some success for years now and they kind of have to because they kind of work and bring publicity. Behind the scenes the AI craze also brought money for research into different directions with other fundamentals. E.g. JEPA, world models, diffusion models, logic based models, energy based models, small recursive models and a lot of optimizations to make things faster and cheaper to compute.

The bubble could pop if one of the major companies does something stupid and their stock tanks but as long money is pumped in there, there is also actually progress in new fundamentals and if they are developed before the bubble popped we might get "real" AI or AGI.

[–] pelespirit@sh.itjust.works 3 points 19 hours ago* (last edited 19 hours ago) (1 children)

I was trying to see what AI says this time about the bubble and it was kind of interesting. This is from Gemini btw:

Question: But Google is switching to AI search only and everyone hates AI that I know. Isn't that going to be a problem?

Yes, the intense public backlash against AI search is a major risk for Google, but the company is trapped in what economists call the "Innovator's Dilemma"—they must force AI search on users to protect their monopoly, even if people hate it. Google's sweeping AI Search overhaul has sparked heavy criticism over inaccurate answers, cluttered layouts, and a ruined web ecosystem. However, understanding why they are doing it reveals the corporate survival strategy at play.

The Problem: Why Everyone is Frustrated

The Accuracy Deficit: Users are highly vocal about AI "hallucinations". When people want hard facts, a summarized paragraph that is only "mostly accurate" is a massive step backward from a trusted link.

The "Web-Laundering" Backlash: Independent publishers, bloggers, and creators are furious. Google's AI scrapes their data, answers the question directly on the page, and deprives the original creator of website clicks. This threatens to destroy the very content ecosystem Google relies on.

The "Web-Clean" Workaround: The dislike of these features is prominent enough that a growing number of tech-savvy users are actively using URL filters like &udm=14 to strip AI elements and force Google back to the classic "ten blue links" format.

Why Google is Doing It Anyway

Despite the clear user pushback, Google's leadership is deliberately choosing to cannibalize its own classic search experience for two critical reasons:

The Threat of Total Obsolescence: If Google does not build an AI "answer engine," users might completely migrate to OpenAI, Perplexity, or ChatGPT for their everyday queries. Google would rather you be annoyed by their AI than fall in love with a competitor's AI.

The "Silent Majority" Metric: While power users and creators complain loudly on platforms like Reddit and X, Google's internal data shows that the general public is adapting. At the recent conference, Google VP Elizabeth Reid revealed that AI Mode usage has been doubling every quarter, quietly crossing over 1 billion monthly active users. For basic, casual queries, the passive majority takes the trade-off of quick answers over clicking links.

Why Google Can Afford the Backlash (For Now)

A full-blown consumer revolt is unlikely to break Google anytime soon due to their massive structural defense mechanisms:

Unshakable Monopoly: As of early 2026, Google still commands over 90% of the global search market share. Bing sits at a distant 5%. Because consumers have no true equivalent alternative for mass searching, Google can weather significant user dissatisfaction without losing its user base.

Default Ecosystems: Google pays billions annually to remain the default search engine on Apple iPhones and Safari browsers. Unless consumers manually dive into their settings to change their default search provider, they will continue to use Google by default.

[–] affenlehrer@feddit.org 1 points 15 hours ago

Understanding questions and summarizing information is a field where LLMs are quite good at. If they actually look up the sources and "read" the websites they are often able to give good answers. If they don't use tools and just answer from what they "remember", the information often contains hallucinations.

So from a user perspective I think search will get better for specific questions.

However, traffic to websites and all the things the LLMs omits are lost. If the LLMs gives you the answer you don't learn about the author of the information, the design of the website, the nuanced and maybe thoughtful story the author built around the information and all the other stuff the author put there.