He left town to play games? I'm sorry I'm quite lost atm... Can you not join him when he goes off to do enjoyable things, or is it his form of "me time"?

Really sorry he's being insensitive, that's always a bitter pill to swallow, no matter the circumstance