Maybe these people are simply curious and just trying to help by suggesting alternatives...

Has anyone actually said they find your presence at work or the study / socializing place annoying, or is that your interpretation?

It's not uncommon to misread certain people...