Given a thousand years, you could look at the patterns and produce text similar to what someone who writes Siamese would write, but there’s still no coherency because you cannot connect the meaning behind any of the words.

That doesn’t necessarily mean your outputs are useless though, someone who does read Siamese can have you generate outputs until you print out something they can infer a coherent thought from, but you’re fundamentally unable to be trained to do that yourself.

You're comparing an LLM to something similar to the infinite monkey theorem. In your analogy, you should consider that someone who knows perfect Siamese is giving me feedback to optimize and improve my outputs, even I don't really know the meaning of anything.

While an LLM may not have a conscience to evaluate if its output is coherent, it can identify patterns and relationships from its training and can generate text that is still appears coherent to human readers.