For me it's two things:

1. In a dream I seem to have some ability to force myself to "wake up" if I really want to. I haven't seriously believed I could do that when I'm awake, but who knows - maybe being awake is just a much more convincing dream.
2. Sometimes when I'm dreaming there are weird enough things happening that I do suspect that I'm dreaming.

Just count your fingers, if you're dreaming there will be way more than 10 of them

I know when I’m dreaming. Like if I question my state of awareness I can tell that I’m dreaming.

I've heard a theory that when you are asleep, your brain is just kind of firing off random thoughts and memories. And when you remember your dreams, it's more like the conscious part of your brain weaving some of those recent thoughts together into a narrative after the fact, in the moments as you are waking up. So you don't know you're dreaming because knowing that you were dreaming doesn't fit into the story that your brain makes up.

At least for me, this fits very well with my experience; I hardly ever remember my dreams unless I wake up suddenly like from an alarm (I don't usually use an alarm). It does not fit as well for people who claim to have more lucid dreams. Maybe their brains just do dreams differently. Or maybe the parts of their brains that make up the stories is more used to making up stories where they are lucid dreaming. Or maybe this whole theory is wrong; brains are weird and there's a lot we don't know.

Because of the way that it is.