Yes, shared secret based, but not a big deal because it is machine generated and unique per account. The 'server has your credential' is only a problem if the credential is reused across services. If you have access to read TOTP secrets from the server, you probably don't need those TOTP secrets to further compromise the service.

But webauthn/passkey is a better approach. Properly managed SSH keys are good too, but folks aren't too happy about how ssh keys are commonly pretty lax. Client certificates similarly would have worked, but never took off. Similar story for smartcards.