I'm so sympathetic to this because I am struggling with this as well (I don't want to hijack your thread so I won't go into it here). But it is SO HARD to put your feelings and sense of safety first when you have been taught to put them last. If you don't like the way you feel when you see him, I think you should respect that and reduce contact.

But I also know it is hard to go up against family expectations. Does he still interact with you in an abusive way? If so, consider what you want your kids to see as a model--you tolerating it, or you protecting yourself by staying away.