I think what you're doing is very all or nothing. People don't talk only about themselves. If you don't talk about anything else and everything you say about yourself is clipped, then your therapist is probably on to something.

How I get out of it is by talking about something other than me. I'll talk about something I went out and did, something I saw, that sort of thing, and avoid talking about myself. My reputation is that I always have a good story to tell but keep my personal life close to the vest.

You say sometimes you try to get involved. Why and when? If it's random, then people will expect you to behave as you normally do, standoffish. If you rebuff people when they approach you, they will feel exactly as you do now and will not subject themselves to it.

I meet with my family at least once a year. It used to be at least twice but I've dropped one. I prep myself mentally ahead of time for it, and pace myself so I can take breaks as I need them throughout, and not get stuck in long conversations with any one person. Because I do that, if I run into any of them throughout the whole rest of the year they consider it a happy accident (go figure) and it's a minute or so of 'nice to see you' and off we go on our merry ways. Because I've put this effort in, if for some reason (I can't imagine) I'd want to meet up with any of them throughout the year, I'd be accepted with open arms. In theory.

How that all works with my avoidance is, I feel obligated and unworthy of disappointing people, which is why I go at all, and I don't talk to myself. I don't think I have extreme social anxiety however. That could be a part of what's going on with you. If it is, check in at that forum for tips on how to get over the hump of interacting at all. If you can't force yourself to talk about something even if it's not yourself, then this plan won't work very well for you.