To be honest, I never thought about that really, it probably happened often but I was just in the moment enjoying the hug and not worrying about parts touching.

I think from the way you are very concerned about this, it's not ok with you and you should probably bring it up.