I don't think you're overthinking. And while we all need some alone time, an entire day every week when you have a young child to care for is a bit much. Is he helping out with the baby much?