Would he really say those things to hurt you? That's what you've got to think. Really think, because it's quite likely he is simply ignorant/unthinking and if that's the case he is only saying those things because he is fond of those memories, not because he is trying to hurt you. You can't change your past but you can accept that you've moved on!

However, if he really is trying to hurt you then I say that's not the kind of person you should be around, let alone listen to! That kind of person wouldn't deserve it, man.