It depends. It's viable if you just need a phone with several open source applications (non-Android) and are fine with that. But if you need Android app compatibility it's probably going to be harder or more inconvenient to do, though I haven't checked the status in recent time. And then there's this evil thing called Google Play Integrity (essentially DRM restricting which apps can run on which OS) which is a problem even for non-proprietary Androids, so you probably won't have any chance if you're dependent on such an app (thankfully it's rare but as we all know stupid ideas tend to become annoyingly popular).

Main problem, as usual, is that Android and iOS have become such big and popular "platforms" for mobile apps that establishing a "third" platform for app developers is basically impossible (also remember what happened to Windows Phone OS, they were late to the market and failed spectacularly to catch up. Of course in this case it's open source so it can grow regardless of user numbers, but still, it's hard to catch up when lots of great Android apps were already developed specifically for Android). So you can only hope that Android app compatibility grows mature enough to be close to 100% compatible, so that you can also run almost all Android apps on your mainline Linux mobile OS. Then you're not "limited" anymore. (At least if you consider it "limited" when you can't run Android apps. Which most probably consider to be "limited").

So I think it's less about the hardware and OS/UI (I think they work fine these days) and more about the available apps.

[My main daily driver phone is a GrapheneOS (Android) and I have a Pinephone with Linux for playing around in WiFi at home only]