I always think about consciously perceiving layers that I am subconsciously missing. But I don't know how much layers of social nuances there are that I don't know I don't know about.

Not feeling awkward is more of a mindset in how you think about it than what other people think about you. If you think it is awkward, then that is a problem. If you think they may think it is awkward, that is something else. They may have the same feeling, and blame themselves. Or maybe they didn't perceive any awkwardness whatsoever and you are just overly conscious.