Not calling your husband was perfectly reasonable. If your husband is sulking and fussing about that, then he is trying to make something out of nothing. You have a right to reject his criticism of your decision to have breakfast alone with your father.

As a child, you could be made to feel guilty by your mother, even when you were innocent. It sounds like your mother was physically and emotionally abusive. It's understandable that unfair criticism is triggering for you. The child in you craves being treated fairly.

In this world, you will be treated unfairly at times . . . even by your own spouse. As an adult, you have the capacity to analyze the treatment and decide for yourself whether or not you've been wrong. It is not up to your husband to give you your self-esteem. You have to take it. It would be nice if he were emotionally supportive. Sometimes husband's aren't. Sometimes you have to ignore criticism . . . even from your own husband. You are perfectly capable of deciding for yourself whether you were or were not justified in having breakfast alone with your dad. You believe you were, and I happen to agree with you. If your self-esteem depends on never having your husband disapprove of you, then you are doomed to never feeling secure about who you are.

One last thing: from time to time you will be in the wrong. That's okay too. Nobody is right 100% of the time. You don't have to be either. Self-esteem means that sometimes you slip up, and so you forgive yourself and move on. Maybe your mother never forgave anything. You can reject that mindset.