I agree with Adam ^
I think he was really trying to make you feel better but didn't realize that saying what he did would make you feel worse. I don't think he was trying to make you feel bad on purpose (but i don't know your hubby or your relationship, so if he tends to be abusive, then it may be a different story).
I can see myself saying something similar to myself, actually, during periods of low self esteem, but hearing it come from someone else would hurt me. I think your feelings are justified but you have to communicate to him that saying things like that don't help.