130
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 29 Jun 2024
130 points (91.1% liked)
ChatGPT
8667 readers
2 users here now
Unofficial ChatGPT community to discuss anything ChatGPT
founded 1 year ago
MODERATORS
This wasn't an intentional feature; they're actually trying to train it with fine-tuning to add this as an ability. It's one area that highlights the difference between it imitating the text it's been seeing, instead of actually understanding what it's saying -- since most of its training data is of the form "(ask a question) (response to question)" overwhelmingly more often than "(ask a question) (say you don't know, the end)", it is trying to be a good imitator and do the same, and come up with some plausible nonsense even if it doesn't know the answer.
Part of the problem is fine tuning is very shallow, and that a contributing issue for claiming to be right when it isn't is the pretraining on a bunch of training data of people online claiming to be right when they aren't.
Yeah. It is fairly weird to me that it’s such a common thing to do to take the raw output of the LLM and send that to the user, and to try use fine-tuning to get that raw output to look some way that you want.
To me it is obvious that something like having the LLM emit a little JSON block which includes some field which covers “how sure are you that this is actually true” or something, is more flexible and simpler and cheaper and works better.
But what do I know
The problem is that they are prone to making up why they are correct too.
There's various techniques to try and identify and correct hallucinations, but they all increase the cost and none are a silver bullet.
But the rate at which it occurs decreased with the jump in pretrained models, and will likely decrease further with the next jump too.