Sounds like the situation is getting harder to manage. Would it be possible for you to separate from him for a bit? Sounds like you need some space from him and to reevaluate the relationship.