Granite, this just sounds like a part of your lack of trusting. The way that you grew up with the mother, why should you trust anyone? If we can't trust our own mothers, who can we trust?
You will learn eventually to learn to trust if you keep working granite. What you are experiencing right now is part of the process of getting there.