Hello,

I have problems with this as well and am currently just realizing it's a problem for me at 34 years old.

I believe the anxiety you feel from boundary setting, is tied into sense of self-worth and self-esteem. In my experience, if I have sufficient healthy levels of those things, I find it easier to establish boundaries and assert myself. It's much more natural.
I also find it is easier to maintain them, and when they are tested, it reaffirms them, instead of making me question them. Though this is very much a work in progress for me.
Seems like you have the right idea with this post, in that you're reaffirming them to yourself which is a good thing.

Also, I believe it has to do with how we are wired. Do we have an internal or external locus of control?
Are our choices a result of external, or internal factors is what that means. Here's a bit more info:
Locus of Control and Your Life

Anonymous also has some good points as well with assertiveness training. I plan to look into that as well as I believe it's a source of a lot of my "problems/fears/anxiety."