Forget about defining learning. Just look at what "language" means in the dictionary. There are definitions that put an emphasis on using language to communicate thoughts or feelings, which LLMs don't do.

Which definition of language is Chomsky referring to?

I'm not trying to be picky, but I think the answer might help determine whether or not LLMs are proving Chomsky wrong (which he might be, I don't know!)