Do you think that you are thinking about this guy too much? Do you think that you are perceiving hostility, where there really is only gossip?

I understand the whole social circle thing, but you can shut down the gossip really easily by not being interested in it. And saying so.

It sounds like this guy might be becoming an obsession for you (that does not mean that you are actively chasing/stalking him....it just means that he and his life are taking up too much room in your head).

From what I have read, he is not interested in you, and is not worth the energy expended on worrying about him, including thinking that you might be "forcing" anything.

I am sure that, as time goes on, he will have less and less effect on you.