Untitled... referring to your "gender" is like referring to your character. It's not something that has relevance except to you and those around you.
We have only ever needed to know "sex" and that is where the harm has come in, by confusing the two. Sports, safe spaces, medical treatments, harassment, people having life-changing operations to make them sterile and so on.
Maybe if we started using the words transman and transwoman the right way round, it might cause less disruption, offence, confusion...