Fair use is determined in a court. If somebody sues you, you can't just say "Nah actually it's fair use" and then not show up to court.

The C&D letter wasn't a lawsuit yet, but a warning that one would be coming. The mod team had the choice of complying or going to court, which costs time and money. In court, even if they ended up winning, it's not guaranteed that the dev team would be granted legal fees. Atop that, who wants to spend the next few months to years stressing on a court fight?

It's an unfortunately lopsided situation where a C&D is enough to make most small time projects fold at the prospect of even having to go to court.