In a general sense, I think most of us have sexual thoughts that aren't always our partners. Fantasies and such. So in a general sense I don't feel that you have anything to be concerned about.

Now I think the real question is it beyond just fantasizing about that person he saw at the mall or is it how he could get with the other couples you guys hang out with all the time type of thing? There are differences. And if he's just fantasizing about nobody, then it's one thing, but I think it can lead to issues if it's someone he interacts with all the time.