It takes time, but i think it would be good for you to talk to him about it. Surprisingly, ive found that most therapists like it when their clients tell them they are frustrated with them.