Now that you've added details, I do think he is gaslighting you.

Personally, I wouldn't be bothered by the masturbation alone. Given the lying, swinger/dating websites, and travel, I'd be worried about an affair/cheating also.