Did you tell the doc about it when you were there, even if you weren't having symptoms at the time, so they could do blood work; if not, I would go back for labwork. It's good to follow up with your doctor to r/o blood sugar issues or other causes to be safe. Zoloft has been known to cause issues of lowering blood sugar so be sure in the interim to have small meals/snacks throughout the day to be safe as well, just in case this is the issue. Even if it is the Zoloft, this does not minimize the symptoms; it is something to be wary of. If you started it recently, it could go away once it gets leveled in your system or it may be an unwanted side effect. I would let the doc know what's going on.

I know I have had those awkward moments in talking with people for other issues. We do what we can and know that it was not intentional. If it's important to you and you see him again, you can always explain how when you saw him last that you were a bit off that day as you were not feeling well and it had nothing to do with him. People generally are very understanding of this.

Take good care of you in the meanwhile and I hope you find answers soon!