I think you need to tell your wife that it's very important for her to be open about this conversation. I wouldn't like him either if I were in your shoes. Maybe he isn't looking at you funny. Maybe he wasn't actually looking down her shirt. But that relationship still existed and you have just as much right to feel comfortable in your home as she does. She would not like you bringing over a woman who she felt was shamelessly flirting with you in her house, would she? If she is the kind of person who is actually ok with that, then you just need to press the whole issue of being compassionate and understanding.

Somehow, I just don't believe that everyone in the family knows that they had sex as teens. Why would they know that? It begs the question - did she or he actually go out of their way to TELL them? That just gives me more reason why he should be excluded from things. That's a very strange dynamic. It's not asking too much to say that he should not be invited into your house. It really isn't. If she perceives that to be an issue, then I can't help but think that there are feelings that she isn't sharing (like, maybe she is really ashamed of it or maybe she enjoys those memories?) I definitely don't think this is one of those things you can say "oh no big deal" and expect it to disappear. It really needs to be addressed in a caring and compassionate way.