I guess I would ask myself if it always feels this way, or if there are also lots of times where I did feel he was getting me? That would help me decide if it was really that he was not a good match for me, or if he did help me sometimes and that my current state might be coloring my current views. If I decided he was not a good match for me, I would start looking elsewhere. Therapy is hard enough - I can't imagine doing it with someone who acted annoyed/tired most of the time!

Good luck!