It's the same reason for people that get most of their news and information from influencers and podcasts that are usually scummy. Confidently incorrect people getting information from confidently incorrect sources. Psychologically the human brain is more likely to believe something if it's said with confidence and accuracy comes second.
this post was submitted on 19 Jul 2026
91 points (97.9% liked)
Hacker News
5186 readers
347 users here now
Posts from the RSS Feed of HackerNews.
The feed sometimes contains ads and posts that have been removed by the mod team at HN.
founded 2 years ago
MODERATORS
Well your confidence convinced me.
The headline seems to not be correct. Based on the abstract:
We engineered the questions so that AI advice was wrong, separating AI use from its accuracy.
The research doesn't really seem to test if people using AI are more or less correct, only how it affects their judgement.
"Importantly, our goal is not to estimate the overall accuracy of AI assistants, but to isolate the behavioral effect of making an answer available when that answer is systematically unreliable. Therefore, we use Step 3.5 Flash. Because the advice participants could obtain was usually wrong...."
The following Monday morning the research team returned to find all their systems had been replaced with Temple OS.
Well, at least they weren't as confident as they were inaccurate?! Silver linings or something.
How do you sum those two statistics, the surety of the AI model doubles the user's confidence in the given answers but natural usage of that information will prove it incorrect three times as often so the cumulative effect is a net reduction in the overall perceived reliability of the returned answer.
That would be over time and experience. They seemed to be controlling for things like that and focusing on delivery and it's immediate effects. They even used an intentionally bad model to skew the answers towards being incorrect, so I doubt people were notified when they were wrong.