One way to resolve this is to have some kind of multiverse theory where you don't travel back in time to your universe, but to a narrow slection of parallel universes that are also shifted slightly so that it spits you out in an analogous location to your initial departure.
Same place relative to what?
It's space-time, not space and time. Moving backwards in one moves you backwards in the other.
I always wondered about this
This is not how general relativity works at all. If your coordinates are set to a spot on Earth (or some spot relative to Earth), you will appear at that same point. Spacetime exhibits something called diffeomorphism covariance which is a fancy way of saying you'd have to go out of your way in the dumbest way possible to get this outcome.