I don't think the association is about how we presently treat swine. After all, nobody sees that. I think it's part of our inherited schema from back when people commonly kept pigs. For a long time, pigs were fed human excrement. They were sometimes kept in a pit directly under the latrine. That's what a lot of people did with their waste: they fed it to pigs and then they ate the pigs. The circle of life?!
Obviously this doesn't change anything but I just want to point out that the stigma goes a lot deeper and farther back than an analysis of the present day can tell you.