It may sound odd, but perhaps allowing your mind to enjoy the thoughts while with your wife could help you process through all of this? It is the denial of the emotions and desires which create the internal conflict. Thinking things and doing them are not equal.