Thoughts are just thoughts but they can be uncomfortable, and one can suspect them as playing a role one would rather they didn't, as you've found.

I spent many years wanting to smash my fist down my stepmother's throat. Of course, I never did that or anything like it. Once, when my T told me in session she was going to be going away for a few weeks in a month or two, I was great in session but when I left and saw her car, the first thing I thought was that I wanted to slash her tires! That actually made me laugh as there's no way I could do that and it's so far from my "normal" self that I immediately went looking for the trigger and realized it was because I hadn't dealt with her announcement she was going away.

If you work through your issues with the abusers and people from your past, the anger will lessen too. I thought of mine as more of a symptom though and I try to look at the real issue rather than the symptoms as a symptom is just a tool to help us get through things and not something that can be done away with without something else replacing it. I have found I have to get a the core stuff before a symptom will go away.