If you feel you said things you didn't mean, were unclear which led him to misinterpret, or were intentionally hurtful, I would explain or apologize. If not, I would try to let it go.

I accidentally insulted my pdoc once by saying he wasn't a doctor but a pill therapist. I meant: "Doctors are evil. You are not evil, so you're not a doctor. I also can't feel slightly safe here if I remember you're a doctor. You talk with me and prescribe, so you're a pill therapist." Pdoc thought I said: "You're just someone who pushes pills." He said something like "I hope I'm more than that" or "I hope I do more than that" and we (me and the nurse that was accompanying me) explained it was actually a compliment.
And I apologized by email a few weeks ago because I hadn't been very polite during the appointment (raising my voice and interrupting him). In my defense, I was agitated, frustrated and he was scaring and triggering me ('playing doctor' as I call it in my mind, as in, wanting to use an evaluation tool that I've had bad experiences with in the past). His email back simply read "OK!" which was a bit of a disappointment - I'd hoped for something like "no apologies necessary" or "perfectly understandable".