i can understand the hesitation, but your therapist's feelings/reactions/etc just can't be more important than your own, skys. its difficult, but the only way to break patterns of conflict avoidance is to face it head-on.
sometimes when my clients have been angry at me and hesitant to say so, i give them permission to not look at me while they talk....so, i turn around and they turn around so that we're back to back

and we face each other again when the client feels safe. do you think that's something you might be able to do?