I relate. Been off work for over a year but a former coworker told me my work bully died young from cancer. Felt pressured to attend the funeral and my former coworker wanted to catch up over lunch.
I didn't feel I could say no and almost went along. I felt "no" without a plausible excuse would be bulldozed over with "why?!" so I ended up lying about a friend needing me to accompany her to a doctor's appointment.
I feel like I have to justify everything I do or don't do. For example, I would get flak for eating lunch alone, because people felt I was being "unsocial", even after I explained I'm introverted. When I had to attend therapy during working hours (allowed, was my right as long as I made up the time), my boss wouldn't accept "medical appointment", but just had to know why I was seeing a psychologist etc. I told a half truth that it was anxiety and family issues but he had no right to pry and worse, hold my condition against me in work reviews even though I always made deadlines and made up the time. Even though HR didn't even care and accepted it without any questions.