"He once told me in an annoyed manner, while he was speaking to me in a particular session that he felt like he was the client, rather than the therapist."

I think it was the therapist's problem and not yours. To me the above would indicate he needed to take a step back and seek his own supervisor or therapist. Poor him to feel like a client.