Hi Hamster.
In regards to the rationality of his comment and your original post, I have reached a conclusion. It is a, perhaps, outlandish conclusion...I hope you're sitting down for this.
Are you ready?
After much thought, and careful consideration...this guy is, in the highest of my psychological ability and to use a technical term, an asshole.
That being said, this guy is obviously still sore over your "wronging" of him for not filling his self perceived "right" of sex, which is of course utterly ridiculous. That said, he's simply trying to make you feel as bad as he himself feels foolish. A sort of childish revenge for a wrong that never occurred, in a manner of speaking.
It would be best to simply not pay him any mind...he's saying things simply to evoke anger and hurt from you. They're baseless.
Hugs,
Harley