It's enough to have some honest conversation, don't you think? Tell him to watch himself because he upsets you. Then talk it through, find some common ground. Easy solutions are the best solutions IMO.