Probably the worry about not having further invites for saying no is the driving force? I like the thanks for inviting me, I'm so sorry I won't be able to make it this time because of another commitment but keep me in mind if you do this again approach.
I think for me the issue with future considerations is less about the consideration but more about timing of everyone's schedules. I used to meetup with a couple of friends after work for a small time frame before my own work hours expanded. I miss those, yet we're all rather tied up with other life aspects and recreating a different time just hasn't panned itself out.
I think, reflecting on it for myself and considering the social life my own grandmother had had, is the broadening of social circles. So that meetups and doing things never depended on whether one person could make it or not and they kept various activies going. Whether walking together around the block in the neighborhood or setting up cookie swaps or bowling or going to community types of events i.e., baked bean suppers etc there was always a base for her. Isn't that what meetups do these days?

I guess I'm thinking about suggesting broadening your base if you can of course? My gram may have just been lucky?