I've wanted to look into that, but I'm my wife's only caregiver, and I feel too much guilt getting away. Plus I'm so very exhausted that I'd rather just sleep or chillax.

I think there are groups that gather for a purpose, such as (and this is just an example) foraging for wild mushrooms. In these meetups, the social aspect isn't emphasized.

I'd say, "Go for it." When explaining to your husband, describe it as a meeting with a purpose rather than a social event so it might seem less fun / threatening.