Hi there!
I also have it. I was diagnosed with Avoidant Personality disorder with some traits of perfectionism.
Of course, having some traits is not the same case as yours.
I used to avoid things when I have doubts on my ability to do things in a perfect way.
You could do some therapy. I think the key is learning to be less severe with yourself. Accept your own boundaries and being moré compasive with yourself.
Are you also inflexible with others?