Mate, I thought what you did was pretty genius.
I personally wouldn't overly think about the whole situation, but the awkwardness between the both of you next time you see each other would be intense.
Perhaps you should just work on how to keep a nonchalant conversation going and how to maintain it so that it doesn't cause any of your anxious thoughts to flare up.