Is it?

I'm in the UK, and I don't think the British (or any other European country) particularly gives a fuck about how any of those countries do. Ask most people in the UK who America's best gymnast was and most couldn't tell you.

What people want to see is the best, which to do so needs involvement from the largest countries (the three named). If any of these countries is banned from these sports, chances are that any funding for them is killed pretty quickly, and suddenly those great athletes aren't world-beaters any more.