He's been frustrated with me and once said something I took to be hurtful, but other than that he's cool as a bean. Him being angry at me would be a huge trigger for me and I think he knows that.

I'm getting red flags from your post and personally don't think a T should be yelling at a client.