I think you make perfect sense. The decision to have kids or not is a very personal one and no one has any right to judge you in my opinion.