I'm a white guy, but can speak pretty decent Spanish. Oftentimes back when I was working retail, if someone was struggling with English, I wouldn't even bother pulling out the Spanish because like you said, it often would have the opposite effect and make them upset. Basically my rule was I only used it if the customer could speak no English at all, and was looking for an interpreter, and there wasn't any Latino coworkers around to help them. It sucks because I learned the language wanting to be able to use it. I did one time catch an unassuming abuela tell her daughter that I was hot. I couldn't resist n said gracias.... If you could see her face man.

I'm in Florida and it's amazing to me that people here still seem to think Spanish is a secret code, and will insult people thinking they won't understand. Never assume. Absolutely ANYONE here can be Spanish of some variety - I have seen genteel white grandmas conversing in Spanish then perfect English too, white looking frat boys, black people, it's more difficult to "look Hispanic" here. Out west is different I think. I literally had no idea there was racism about Hispanic people until I visited California as a teenager because they were the "white people" in my city - the mayor, the rich people, plenty of them Cuban or other Hispanic.