I'd have a serious talk with him, at home, without your priest. His idea that it is okay for him to move the family for a work opportunity, but not you, is ridiculous, and in my mind, deserving of the "whole load of anger" Roadrunner alluded to. However, he simply may not be able to see how unfair he is being.
It is amazing what good communication can do. Unfortunately, however, it requires that both parties listen as well as talk. I hope that your husband will listen! I haven't been following things too carefully, but your husband does stay up with your baby sometimes so that you can sleep, right? He cares about you. I think that you can salvage your marriage.