Sounds to me like you could identify with being gender-queer. I too am female and since I was ten years old had people accuse me of being transgender or secretly having a tiny penis.

Below is some advice from an internet stranger that has helped me:

Way back in the '80s, an extremely common and very effective way to slander someone was to allege that they were homosexual. I don't mean calling them names like 'dyke' or 'fag', which can often be interpreted (especially nowadays) as nonliteral, but making explicit allegations that they literally engaged in sexual acts with persons of their same sex. In the '80s, this kind of allegation was very damaging socially. (And, often, in other ways, as very few laws existed anywhere to protect anyone from antigay bias.)

By the eary '90s, when the modern gay rights movement started really moving, a new paradigm emerged, in which out gay people were essentially immune to this -- there's clearly no point in accusing someone of being gay if they're already out as gay -- while closeted gay people lived in terror of it. That terror included many straight people, in what was jokingly called at the time, 'homomephobia' -- the fear that other people think you're gay. All this might seem odd to you if you're not old enough to remember it, but remember that at that time, 'gay people' were a very discrete slice of the population, and you were either 'in' or 'out' or 'suspected' or 'clear' (above suspicion). Socially, gayness was still very much stigmatised in most parts of the culture, to the point that gay people pretty much exclusively socialised only with other gay people. (It's no coincidence that this was the period when when gay establishments were at their apex.)

Come the late '90s, so many people were coming out at such a rapid pace that a very important paradigm shift occurred: "Who cares?" It was no longer effective to accuse people of being gay, because who cares? Only bigots, the new paradigm said. To accuse someone of being gay was either to admit a newly unacceptable bigotry, or to risk being identified as closeted and terrified yourself, at a time when there was increasingly less reason for anyone to worry about it. People who did this were seen as being backwards in one way or another, and the social habit quickly abated when it came to be more denigrating to the accuser than the accused.

That moment in social history has not yet arrived for gender diversity, however. We live in a time when there is still significant (though rapidly declining) social stigma to being trans, or at least not fully and definitively one clear and unambiguous gender. In only another ten or twenty years, people will look back on our time the way we now look back on the '80s and early '90s, and for exactly the same reasons.

You are being subjected to the exact same social attack as gay people (or merely gay-like people) were twenty years ago and more, but now in respect to gender instead of sexuality. It's not any less stupid for it, and this too will eventually abate in our society. Just not today.

My point in all that is to clarify why your strategy should be one of: "Who cares?" Why would it matter, if it was true? The old tactic of accusing people of being gay worked because it was presumed to matter if it was true. Later, that tactic stopped working once enough people realised that it doesn't matter. More pointedly, it stopped working when gay people themselves realised that, much earlier than the population at large did. It stopped working when gay people (or those merely accused of it) turned the accusation around: "So what? Why would it matter? Why do you care? What's wrong with you, that it would matter to you? Why is it important for you that other people know what you know or think or suspect about someone else? Does your obsession have something to do with you? Do you think anyone else cares?"

Everything you're seeing is nothing but petty social games played by petty people who for whatever reason feel they have something to prove. Hard as it is, your best possible strategy is to put your nose in the air and ignore them -- or at worst, sniff derisively at them for their pettiness, which is beneath you.

In the past, accusing someone of being gay worked to hurt them because being gay was bad. It stopped working when being gay stopped being bad. If this tactic works against you, it's only because you either accept the general paradigm that being trans is bad or you fear that others accept that and will judge you in ways that may detrimental to you. Is that actually true, though?

Let's look at this objectively. Right now, in most of the U.S. you're at some real risk, in a practical sense, if you're a member of a gender minority. Statistically, most U.S. citizens do not live in areas where that status is protected by law. Meaning, you can legally be fired, evicted, or expelled even on the suspicion of being trans. But in some other areas (such as where I live), the opposite is true: You can't legally be discriminated against on that basis, and those who try to hurt others on such allegations may themselves be in violation of law. Your first move should be to clearly establish what the law provides where you are. (Outside of states like mine, some specific counties and cities provide protections where states do not.)

Reading your post,
How much does all of this really matter?