Hacker News

My theory? Its the abstract that trips it up. I don’t know much about neural networks or RAG searching, but the abstract is information dense in a short amount of space. My guess, an LLM scans through a document, finds areas of the document that has the largest quantity of information relevant to the prompt and then branches out until its confident enough in answering the question. While great for most written text, I think the LLM starts with the abstract, gets way too much confidence that it knows enough and then begins spouting bullshit. In my opinion it explains why it creates fictitious citations. Probably sees a citation in an abstract and then conjures up a title from the information in the abstract.

My only evidence is that I try using LLMs to find information quickly in very dense datasheets for electronic components and most times it will rattle off information found in the component overview section rather than telling me about information found in the application, or electronic characteristics section.