If you're not the kind of person who wants to have a blended family in the future and be a step-mom, then I'd say this isn't the right relationship for you. Honestly, I have no desire to have step children as I know the drama that it brings with the ex and so on and so forth (been through that before, it wasn't fun, and I'm not doing it again).

Its not stupid to leave someone because of their past. Would you stay with a child molester and give the excuse "well that was in the past!". Nope, you wouldn't. These feelings of yours are very valid and I don't think you can just throw the past away, no matter how minor it may seem, as what matters is how it affects you.

You're still in the honeymoon phase I am guessing, but in the future, he will have to divide up much of his time between you and the kids and everything else he has going on in life. I am personally not the type of person who likes to settle for crumbs, so I don't date guys with kids.

Good Luck!