AI - Artificial intelligence

I feel like diffusion LLMs would get this better.

After “position 5,” an autoregressive LLM has one chance, one pass, to get the next token right instead of another bullet point. And if it randomly picks another bullet point because the temperature is at 1 or whatever, the whole answer is hosed.

Not that OpenAI would ever do that. They just want to deep fry autoregressive transformers more and more instead of, you know, trying something actually interesting.