The issue for me is that, let’s say that one day a magical device does allow time travel, it would also have to do teleportation relative to some point since everything is in motion. What would all of time and space use for a relative location? Can’t be the earth or sun since those are moving. Even our galactic cluster is moving. So if you went back in time a year, the earth wouldn’t be at the same point in space even though it’s done a full revolution. I’m no space scientist but I don’t imagine we have a solution.

More pragmatically, time travel for a casual party would be risky because you’re carrying germs many generations apart. Time travelers would wear full-body suits or risk dramatically altering history. They could not drink or eat anything.

Okay yes but there's also a bit of dissonance, if I learned that Vlad the impaler had a time travel party to prove it and I had the capability to do so, I wouldn't give a shit, would go see him and also take a gun just to keep myself safe. Safe. But it's somebody from hundreds of years ago, why do I care about any of that?

Sure. This statement seems harsh but I'm playing devil's advocate here, the further you get away from the situation, the less it really matters