In my opinion, this is just how a recently divorced man/woman is when they are juggling work, kids, and a new relationship.

I married a man who has one child. When we first got together, everything revolved around when his ex would let him see his son. They didn't have a set visitation schedule so he got to see him when his ex was available. She made it very hard on him. He worked 50-60 hours a week. I usually saw him one day a week, possibly two on the weekends IF he didn't have his son. This went on for three months, until he asked me to move in with him. We moved REALLY fast and I don't recommend this. But, in general, seeing each other once or twice a week is pretty healthy for the first few months.

I would see how it goes, and just be happy for now. If, after 3-4 months, he hasn't wanted to see more of you, or change the level of seriousness, start the conversation of where you're headed. If you want to move to the next level (like, meeting the kids?), and he isn't ready, you will know by then what you want to do.

Do you have children?