Yes, I would go see a counselor yourself and learn how to feel better about yourself so you do not feel like you have to defend yourself. If you have done nothing wrong, you know that and that's that; what someone else believes is their problem. If your wife is making strange claims about your underwear being bought by another woman, you should be able to laugh as a response since it is so truly strange and wrong.