That's common if you don't know a language too well. There is the variant that you learned, and since you don't know more, you think that this variant has no accent and all the other variants (that you didn't learn and thus are hard to understand) you think have accents.

Only once you spent significant time with multiple accents will you be able to pick up the differences.