Attacking people is still upsetting even if they don't get hurt. There are many ways to harass people without hurting them, and I'd consider surprise schrodinger shanking one of them. I don't know if I'd call that "evil" per se, but I'd definitely call it an asshole move.

Personally as a DM I wouldn't make the sword evil, but I might make it so eventually it would repel the grasp of the Paladin who used it so flippantly, rather than as a warrior of good.