I heard about this awhile back, but only got around to trying it now. Junk Store is a newer Decky Plugin that allows you to view and install any of your Epic games store games right from game mode. You could already install epic games through Heroic and some other tools in desktop mode, but this is one allows easier installation (in my opinion), has full controller support for the library management UI, and will install the new games so that they show up like native steam games in your library. Overall I was really impressed by how well it worked, and wanted to recommend it.

To install it, first install the plugin from the Decky store. When you open the plugin, it will show you some different screens, and you need to go to the dependency screen and install all dependencies needed (it will do this for you, you just need to click one button). After that, close and reopen the plugin and it will show you an empty library and ask you to log into Epic. Once you're logged in, that library screen should show all your Epic games, and you can select any of them to install. After the game is installed, it will show up in the "Non Steam" tab of your library.

The plugin mentions GOG support, but it's not live yet. Also the junk store has an open plugins system that will hopefully allow other game libraries to all be accessed this way as well.