I find they are good for creative tasks. Picture and music generation, but also ideas - say, give me 10 possible character names for a devious butler in a 1930s murder mystery novel.

But yes, terrible for facts, even rudimentary ones. I get so many errors with this approach its effectively useless.

However, I can see on narrower training data, say genetics, this might be less of a problem.