This is an older look at some of the foundational work that went into today's LLMs. It links to a tool that you can use to explore the associations:
https://lamyiowce.github.io/word2viz/
I think it's hard to say that language models don't "know" things based on even older results like this, even if they "know" things in a completely different way than humans do.