I'll start off by saying, no you're not a horrible person, you did nothing wrong in this situation and it is only naturally for you to feel resentment towards both him and his children. He cheated on you and as you said, they remind you of his ex wife. What he did was wrong and can never really be forgiven, but without forgiveness, only you will end up suffering as you are. You've had the burden of knowing that even while he had suggested to you that he wanted to have a baby with you, and were carrying it, that he was sleeping with his ex and having to carry that around with you will eventually start to make you feel like this. There's nothing worse than betrayal, especially when its with someone you trust and love and the fact that you were carrying his baby makes it all the more hurtful. What he did however, cannot be changed and unless you seek help and advice from a therapist or some form of counselling preferably marriage counselling with him so that this can be discussed and sorted from the core, then it will only get worse and only you will be suffering the consequences. The children on the other hand are also victims in this, they lost their father in this situation and its also only natural for them to want their parents to be together and so talking about happy memories may be their way of trying to encourage this. The fact that you see their mother in them is not their fault and they have also had to lose something from this situation, and so the best option is for you to seek help before this gets any worse.
I hope this helps x