It is your responsibility to protect your daughter whatever that takes. My recommendation until he gets his behavior together to keep them apart. Her safety needs to come before any repercussions from your wife's family.