Sounds normal to me. You need a challenge, and you believe M is up to your level, but the relationship is a mess which makes you unhappy, you have to vent your unhappiness in sexual release, J just happens to be there. Tension and failure lead to sex, imagine if you had no angsty feelings and were calm and happy most of the time. Positive feelings probably wouldn't lead you to as many questionable encounters with J as much as the bad feelings do.

I was always interested in the mental dynamic of relationships, and lean towards Freud. Fun read: Who's Been Sleeping in Your Head: The Secret World of Sexual Fantasies [Brett Kahr]