My setup is different from yours, but maybe it'll be a reference for something you could do.
I use WireGuard. I forward my dns over the tunnel. The dns servers are my home dns servers (pihole). Those servers resolve my service hostnames to their LAN addresses. Those LAN addresses are resolvable when I'm connected with WireGuard. This means I use the same IP addresses and same host names over vpn or locally on the LAN.