Well, IDK the semantics, but from what you've written regarding family outings and feeling invisible on Mondays, I assume his wife doesn't know you're a part of her life.

I hate to be the one to break it to you, but in the eyes of logic, his wife wins all the sympathy and empathy points in this scenario.

His wife is caught in the middle of this storm, not you, his wife is the one being betrayed (knowing or unknowingly), a wife he vowed to love and cherish forever.

While you, you knowingly got yourself into this mess, you've agreed to becoming his mistress, you agreed, while his wife has no say...

So I'm not surprized at all that he isn't congratulating you, or stroking your ego for the pain and suffering you're enduring. You expressing your desire for him to do so, has probably put the spotlight on his wife and made him think of how much his wife is or will be suffering...