I just wondered if the reason, or part of the reason, is that you aren't attracted enough to men to trigger a rise in your libido in their presence. How anxiety would affect how you feel when faced with a situation like that I couldn't say. Regardless of the complexity of your sexuality, it probably wouldn't be a bad thing to investigate the anxiety etc... even if you work out how you feel, that's still a hurdle to get over.