The risk existed but did it plague X11?

Yes, and it still does. Practically every X11 installation is vulnerable.

(If you're nitpicking my use of the word plagued, though, note that I am talking about the vulnerability, not the exploit.)

I never heard about any app logging keystrokes and sending theme somewhere.

That's because of a variety of external factors, including:

• X11 desktops aren't common enough to be priority malware targets, yet.
• People who run only open-source software typically get it from trustworthy channels, like their OS distro's package repository.
• Devices likely to attract malware, such as game consoles and mobile phones, have avoided X11. (Android phones and Steam Deck are examples.) This is no accident; lack of app isolation was a factor in that decision.

I don’t think normal uses had to worry about it.

We've been lucky so far, in that our circumstances have kept us mostly safe. However: Linux malware is on the rise. Commercial games, both on their own and through anti-cheat systems, are making opaque software more common on our desktops. Flathub is working on paid apps, which could likewise create malware opportunities that weren't there before. The Epic Game Store has already been caught collecting data from other apps, so the intent is clearly present already.

It's generally just a matter of time before exploitable systems become exploited systems. We would do well to close the door on unauthorized key logging, clipboard snooping, screen scraping, and input injection.