How old is he?

As hard of a concept as it is, it's important on some behaviors to call their bluff. If it is indeed on purpose to get a rise from you, don't engage.

How long ago did you and his father separate? How many years together was he raised under the same roof? It's really important to come out from the break up not feeling like the victim. It will be tough for your son to be looked at with any loathing that you have for his father that seeps through in his direction. That's one memory of mine, that never left. No one is perfect. It can be complimentary when done right.

If your son is returning from visits all wound up and spewing vitriol your way, it might go beyond his father bashing you, it could be the reentry mode effect that I term for my own kids, it's readjusting to surroundings and reality.