I believe this is mostly due to Android clearing the app from system memory. I have had some improvement by exempting Voyager from power management.
On my device the steps are Settings -> Battery -> App Battery Usage -> Voyager -> expand Allow background usage -> check Allow background usage and select Unrestricted
(The exact steps may be different on your device)
This should keep Android from clearing the app memory when you briefly open another app, but it could increase battery usage. You will want to remember to always close the app when you are done using it



