Ouch, that is a harsh thing to say and not really true... therapy is so much more than a contract. I agree with the others, who have wondered whether he is acting on his own anxieties about how how feels towards you. Because it sounds like a clumsy defensive thing to say to open a session... then to go on and mention a woman he's dating, bit strange.

Good idea to talk to him about it and ask him what he meant and where it came from, sounds like you have an open, honest relationship... although he's probably not going to admit to intense erotic countertransference.