I used to have this problem in my quest for a silent PC: make the loudest thing real quiet - say a passive watercooling rig with no fan and just a quiet aquarium pump, eliminating the CPU fan - and the next loudest becomes "unbearable", fix that one - special watercooled graphics card, connected to the passive watercooling circuit - and the next loudest one becomes "too noisy". Fix that one - new quiet power source with extra large slower and quieter fan and, you guessed it, it's the next loudest noise source that gets on your nerves (in this case, it was harddisks, as all this was many years ago).

Ultimatelly the solution was thinking "out of the box" (in more ways than one) - good quality noise reduction earphones.