Maybe they are not taking it as you are not interested in socializing so they don't invite you in the first place. That would hurt me as well as I do personally think social events bring coworkers together. But many times or even at all I am just not interested in the same things. I do small-talk and pretend to be interested. Did they at least try to start talking with you?