I might have an opposite view to some but I'm a big believer in trusting your gut feeling. If you are getting a negative impression deep down of what he is saying, it could be that is actually saying it like that. Not all T's are a good fit for the client. You might want to ask him about it, if you think you can be honest with him.
Otherwise trust yourself, does he mostly make you feel comfortable when you are there? Or are you always on edge, thinking about how to word things to not get negative feedback or annoy him (as you write he seems annoyed)? Do you feel you are making progress in your therapy?