Fuck AI

Yes, you do know the boundaries of AI. It is purely matrix multiplication: its output distribution is just as intelligible as the distribution of rolls of a dice. We receive a probability distribution for the next token given a sequence of tokens. This is demonstrable; search for softmax online.

To fairly equate a dice roll event to a model prompt event we must understand the technicalities. To say you have a 20 sided die, is equivalent to saying you have a specific model’s architecture and value of every parameter, in the context of qualifying event determinism.

If you can assume your die is fair, and 20 sided, that is an equivalent assumption about a model as to saying it’s llama-3.1-8B-instruct. That is, you do know the specific model weights, corresponding to a functional relationship between input and output which is deterministic. That is, if you know the model weights, which is equivalent to knowing whether a die is fair and n-sided, you can deterministically predict the output of a model as you can deterministically predict which number on a die will land

You’re making specific, technical errors about the mathematical basis of language modeling, and equating things fallaciously to a similar deterministic event.

Despite this, your intuition is right: we can’t perceptually predict the output of a model as we can’t perceptually predict what number will result from a die roll