Unless the mom is threatening to leave you, I cannot see why, with your name listed on the birth certificate and marriage mentioned, fear of losing your child is noted?
What about a paternity test to clear things up?