Hang tough! You are not in the least bit in the wrong. His Mom has no business trying to jump in. Your instinct to get away from him is perfectly right. They are both being manipulative, and it isn't right.
You deserve better than what he has done to you. You deserve better than what he is still doing to you. Just keep remembering that. You have nothing to feel guilty about. You are taking appropriate care of yourself.