There are a lot of good answers here already, but I'll try to attack the question from a new angle.

Firstly, yes: they experience an attractive force from the nucleus, and would in principle have their lowest possible potential energy if they were located exactly in the nucleus. An equilibrium state is the state with lowest energy, so why aren't they exactly in the nucleus?

Consider that an electrons position and speed cannot be exactly defined at the same time (uncertainty principle). So an electron with an exact position could have any speed. If you compute the expectation value of a particles kinetic energy, when the particle can have any speed, you'll find that it's divergent (goes to infinity).

So: Because an electron with an exactly defined position must have infinite kinetic energy, the equilibrium state cannot be an electron with an exactly defined position, and so cannot be an electron exactly in the nucleus. So what do we do?

We have to make the electrons position "diffuse". Of course, that means it is no longer exactly inside the nucleus, so it gains some potential energy, but on the other hand it can move more slowly and has lower kinetic energy.

The equilibrium state is the state we find where the trade off between kinetic and potential energy gives us the lowest total energy, which is described as a 1s orbital. The electron is "diffuse" enough to have a relatively low kinetic energy, and "localised" enough to have a relatively low potential energy, giving as low total energy as possible.

Once you start adding more electrons you need to start taking Pauli exclusion into account, so I won't go there, but the same manner of thinking still essentially holds up.