Maybe you were unconsciously trying not to bond because of all of the pain you had in your childhood? It's very common for children who have been abused to develop attachment issues. These can definitely last up until adulthood. You not being able to bond might just be a defensive measure so you don't get hurt, like you have before?
The important thing is that you are beginning to bond now. You can't do anything about the past you can only move forward. Beating yourself up about it won't do you or your child any good.