It's so important to be comfortable with your pdoc. Have you talked to him about how you feel? Maybe it is something you can work through. In the end do what is best for you.

BB