How gender dysphoria manifests differs from individual to individual, people experience it in different way, particularly in regards to what someone struggles with: Someone may feel more dysphoric in regards to their face; others in regards to their chest;... And yeah, yearning for different genitals is definitely something that meets the criteria for gender dysphoria.
Does that make you trans, nonbinary, something else? That's something nobody can tell you, because we do not know how strong your feelings are, how much they define you and what you make out of them. That's why the trans community usually avoids telling other people if they are trans or not - or deny anyone that label who claims it.
If you wish to be trans nonbinary, then you are. If you think, these terms do not fit you, then you are not.
Personally I struggled with similar feelings when I was younger. Fast forward and I am now fully transitioning, because I realized it was "not just that." I wanted it all. But that's my way to deal with my feelings. You may chose whatever path suits you and on that path you can try out what this thing or that and cast them off again if they do not feel right. It's a journey of self-discovery and - for better or worse - it's all up to you to discover.