Gosh, this is so difficult, especially as you now have a baby. Having been in your situation where my ex didn't want to get married (we did after a while but it was my initiative), I would be tempted to say don't ignore this feeling. He once wanted to marry you so he's not against marriage in principle. He may have other concerns on his mind now, like earning enough money to keep you and the baby cared for, but only you know whether that's the case or not. Another factor might be that a sex life often deteriorates after the birth of a baby, due to physical factors (initially) and then often sheer exhaustion, so that may make him doubt. You know what the situation is with regard to the above.

Personally, for me, the lack of desire to get married did feel like a bad sign. It ate away at my self-esteem. We got married in the end because I didn't want the children to be in an unmarried situation, but at the back of my mind I knew he hadn't asked. I do feel that the disinclination to get married in the end was a symbol of my ex's lack of commitment towards me and his lack of effort. Because I wanted marriage, I felt he was withholding something important to me. In effect, he had all the power.

There are a couple of things I'd suggest:

Drop any talk of marriage and get involved in other aspects of life away from your husband. This is a way of making it clear you are not going to chase him on the subject and that your focus is moving away from him. This takes any power out of his hands. If you are showing less interest in him he'll know something has changed. I am not suggesting you be unfaithful or anything, just that you go and busy yourself with something else and be less interested in him. This could have consequences and he might leave if he feels you are no longer paying him attention. The idea is that you don't 'sit at his feet' waiting for him to be kind enough to marry you; you've got more important things to do than sit and worship him. If your focus turns elsewhere, then he'll realise the pressure is off him and also that you are finding happiness elsewhere. It may subtly shift the balance of power.

The other thing I'd suggest (which I doubt you'll want to do with a young baby) is to leave. Tell him you are looking for a relationship with commitment and so you need to opt out of this one. I am not suggesting blackmail. You have to mean it and know that you are breaking up the marriage. This is the option I wish I'd chosen but I was too scared to take this step and didn't feel I could cope at the time. I wish I could have been stronger then though. Giving up on a relationship like this is serious business. You have to consider whether your partner's change of tone about marriage hurts you sufficiently that you will always be in a weaker position to want to take this definitive step. Sometimes self-respect matters more than papering over the cracks. Only you know how you feel.

You, and others, may wonder why I don't suggest talking to him or getting a counsellor involved and explaining your feelings to him. I tried this and I thought once he understood things would change, but they didn't. In the end, it was just humiliating and I was effectively still in the position of persuader. You should not have to persuade him. If he can't offer something so important to you voluntarily then it's only fair to question why you are staying with him.

Good luck. I do know how this feels.