Everything you are saying about why this man is not emotionally abusive is consistent with being emotionally abused. I encourage you to read more about emotional/verbal/psychological abuse (no real precise definitions among them) which you can find abundantly if search on the web. Sometimes it is harder for people to recognize when a previous relationship has been worse. Just because it's been worse doesn't mean this isn't bad. I think it is painful and harmful to live with someone who uses words as weapons. Abuse isn't limited to threats. I'd encourage you to call your local domestic violence organization. You can start here at the national level; they have online chat: National Domestic Violence Hotline | Get Help Today | 1-800-799-7233

You can also get to local programs from there. I also wonder if any of your financial problems might be due to financial abuse (see the above website)