You may be being a bit hypervigilant about this because you have an expectation that he has broken confidence. Possibly overinterpretting things? I don't know. Is talking to him about it really going to help if you've already decided he'll lie to you about it? You will need to vow to trust what response you get from him when you do ask him; otherwise, what is the point?

How bothersome is it if somehow she knows? I've been in a similar situation. I used to see a t in an office where another teacher I worked with happened to work after hours. He clearly put two and two together and I was pretty certain he shared that information with other people I worked with. I just chose not to let it bother me. We were all adults and I wasn't going to get hung up on it. Not quite the same as the t himself breaking confiidences, but similarly uncomfortable.