We do wash them, I clean mine by boiling water in them, scraping any stubborn bits with a wooden spatula, rinsing it out under running water and wiping them down with a clean towel and heating the pan again to evaporate any remaining water. No microbials will survive being boiled and then heated again, anything stuck to the pan dissolves away in boiling water and a clean towel will wipe away anything else. After that I add a few drops of oil and wipe down the still hot surface with the thinnest possible coating of oil.

Seasoning for cast iron doesn't mean holding onto previous flavors. It definitely shouldn't taste like last night's dinner. Seasoning in the context of cast iron is the build up of thin layers of polymerized oils from heating them up in a clean pan that forms a durable protective finish that is incredibly non-stick.

So more accurately parallel your underwear example how cast iron is cleaned, if you took your underwear, boiled the hell out of them, used something to give them a scrub, rinsed them out well and then heat dried them.