All I can tell you that is supportive is that I was always told the same thing by family and friends growing up and into young adulthood. I'm too sensitive. Not mainly by men.. just.. everyone.

The worst is the defensiveness for me. Wether its tears or anger or aloofness. It's all the result of feeling misunderstood/stupid/defensive.. for me.. I'm still working on it.

To answer your question.. is it a bad thing? Well...
too much or too little of something can be bad. Life is a balancing act sometimes.

Seeing as how multiple people around you are noticing your level of sensitivity.. it would be wise to at least acknowledge and reflect on that. I often feel we get signs around us to guide us, even if it is simple cause and effect. Perhaps consider such signs regarding your sensitivity.

I'd also like to note that referring to it as sensitivity it very broad... you should look at the way you are reacting towards people more critically if you want answers or change.