There are 2 reverse proxies involved. One is Nginx which is used to front both the Lemmy UI and the Lemmy backend. That's what the 'proxy' container in the docker compose file is for. It seems to be a required component of the application stack as different request types to the same host FQDN are sent to different backends ('upstreams' in network speak). You could use Caddy here instead if you wanted, which is the point of this page: https://join-lemmy.org/docs/administration/caddy.html. However, that config doesn't work for the latest version of Caddy (you'll get an error about stuff being outside of the site block).

The other one (could either be Nginx again or Caddy or anything else you want instead) is to front the whole thing and provide TLS termination using Letsencrypt. This bit is explained here: https://join-lemmy.org/docs/administration/install_docker.html#reverse-proxy--webserver