Hell Yes You Should Be Mad ... !!!
He's being totally disrespectful to you and your relationship!
The only thing you should be dropping is him if he doesn't straighten up and fly right!
Hope you have more respect for yourself than to continue to put up with this type of behavior from him ... or anybody else for that matter!!!