Your intention isn’t really in play in this situation imo. You behave in a certain way and people around you behave a certain way in reaction and so on. Usually when you need to explain that you didn’t intend to be hurtful or rude it’s accompanied by and I’ll do my best to learn from this and take a different approach next time. Why? Because it’s so common to hurt people unintentionally! You might think “why should I adapt to them” and the answer is just that those are the norms that exist where you live. As you’ve discovered, functioning outside those norms causes general discomfort for both parties (they feel hurt by the bluntness and you feel annoyed that they’re making something out of nothing). To eliminate the discomfort, you adhere to the norm. It’s also fine to refuse to adapt, but I think it’s nice to understand that that’s what you’re doing.

Of course, your family and friends should also understand (if you’ve spoken with them about it) that, given your brain anatomy (or whatever), you don’t share their instincts in some situations, so it’s a bit harder to learn the norms. For example, if staring doesn’t make you feel awkward, you might not instinctually avert your gaze when you find yourself staring like someone who feels awkward in that situation would.

Idk your life, but something practical you might consider is moving somewhere where niceties are less of a norm. They say Germans don’t like small talk! But a much easier option might be to just play along and add “would you mind” before things even if it feels stupid.