Maybe you husband would benefit seeing a Therapist ? It could help him process the relationship he had growing up with them and especially if he’s wanting that validation, your husband likely needs to just process the fact he will never get what he is looking for and move forward with his life , you and your children.

You have every right to not go around them...

My first husband had horrible parents and after I had my child I refused to ever see them again. My husband still saw them every month or so.