I read somewhere that no one is a good parent to their child at all ages. That is, some are good with infants, some with toddlers, some with pre-teens, etc. Your son is at an age where most everybody struggles with caring. You might find you'll feel more comfortable as he gets older.
Yes, you do need some time away from him. But I found that just getting out with my children, such as taking them to play at a park, to McDonald's for a snack and to their playground, to a library program, etc. helped to settle my nerves. I was really going batty just staying at home with them day after day! I had my moments, too, when I was ready to trade them in, although I love them very much. (They are now 21 and 19, so we all survived!)
If you can get out with at least another mother, too, while you're at it, then you and she can have some adult conversation while they play.
Those of us who were abused with children, I think, do struggle more with parenting. I made some mistakes, but I stayed in therapy and I was able to keep from abusing them like I was abused. My young adults still remember some of the things I did and said to them that weren't cool, but overall I did okay. They are great adults and I am proud of them.
And, yes, he will be going off to pre-school, and regular school later on and will be able to get involved in other outside-the-home activities.
You are a good mother for even worrying about being "an awful mother." It sounds like you are spending quality time with him and that's more than some folks do.
And, you do have your T to talk to, as you get other issues dealt with. I put both my children into therapy, too, knowing that they needed someone to talk to about me!