I understand totally. My husband used to do that to me. It took me a while to get through to him he was not helping. Now he just asks if it a bad day and gives me my space. Communication is the key. It may just take him a while to accept he can't change your mood and make you better.