I don't think it's you at all. What I would be worried about is that he doesn't seem to understand the need for consistence in therapy. I would talk to him about it- maybe that will be enough to resolve it. But if he still doesn't get it, it might be better to keep looking.