The inherent flaw in your argument is simply this: you think it's an opinion being expressed at this event. Being gay is not an opinion; it's not a belief system; and it's certainly not a 'topic' for you to 'debate'. No one is trying, as you put it, to change your mind on being heterosexual.
I see nothing in your original response to the guy asking you to attend the event that could be labeled homophobic. That said, a school is much like a small town in that it's somewhat of a microcosm-- almost everyone knows almost everyone-- and based on the arguments you make for you disapproval of the event, I find it likely you were called a homophobe for more than just this one interaction with this one person. Your insistence on putting terms such as gay rights and LGBT community in quotation marks undermines the respect you claim to have for "them".