Why people get annoyed at things that don't affect their routine

I don't get why some people get annoyed when me or anyone else does something and other people get annoyed when it obviously does not affect their routine. Like when I was at college, I always hung out in the main college center where people always gathered to socialize, eat, and even do some homework. Lots of people did that. And I did it a lot as well. But for some reason, there were some people who would ask why I like to hang out there a lot, even though it did not affect their routine. And they appeared annoyed too.

Now, more recently, I may go meet someone and usually I meet them at the college I graduated from since it is convenient. I've had other people ask why I would do that. It is not like I am there long, I always get there a little early since I use public transit and you just never know what may happen, but I am never too early. But there are still people who make it seem like they find it weird or annoying that I do that, even though they don't even go there and it had no affect on their routine. There are several other cases but. These were a couple examples. Anyone get why people do this? I find it annoying. I almost want to say it doesn't affect them so stop acting so annoyed by it. Wish people would mind thei business and let me do what I want or need to do. Anyone get this? How do you deal with it? Just curious.

I've basically come to accept that most everything that I do, including things that are basically good or at least aren't affecting others in any sort of negative way, is likely to be met with criticism from somebody. There are times when I think that some forms of criticism can help us to change things for the better, but I think that it's up to us to determine when questioning or criticism from others might be helpful and when it might be unnecessary. I think that criticism can sometimes help us to make better choices, but I think that it's also the case that even the best choices that we make aren't usually going to please everyone. So, I've found it helpful to accept that the choices that I make aren't going to please everyone. Creating some amount of distance between ourselves and those who routinely find fault with our decisions when they're basically good and not harmful to others can be a good thing, but at the same time, I wouldn't recommend taking avoiding criticism too far, because I think that unnecessary criticism is something that we can always find ourselves faced with, no matter how good the choices we make are.

Are these people actually saying this is annoying to them or could they just legitimately be asking why? To me this actually indicates they are interested. After all, if they were disinterested or didn't care they wouldn't be saying anything at all.

Of course, your reply could always be, "Why do you ask?"

Well they never said they were annoyed but actions speak louder than words and sometimes even their tone of voice can give it away. And yeah I may have to ask why the person is asking next time it happens.

Because some people have nothing better to do.

Well you are right about non-verbal communication being a bigger telling point that words.

I still think it is worth asking. When they give you their answer go after it, if you feel up to it, and glean from that some better understanding. Do you feel okay about self-advocating? It may mean putting someone on the spot.

Makes sense, like politely letting someone know that they are being a bit nosey? Makes sense, I've thought of doing that. In a nice way. Like asking them what it is that I do that bothers them so much. And why. I may do that the next time someone asks.