It would make sense to me to approach re-establishing contact with his parents slowly too. Staying at a hotel seems like a reasonable compromise. However, since they are his parents, not yours, you can't really dictate to him how he should do it.

I do think that your first responsibility needs to be to protect your children and yourself. You don't have to go, but I can see that you might want to if the kids are going. If he is absolutely insistent that you all go and stay at their house, maybe you should make an escape plan just in case. Maybe you reserve a hotel room and make sure that you have enough money to pay for transportation to it if it is needed?

It's not being unforgiving to make sure you are safe, but since this seems really important to your husband it also makes sense to try to do what you can to accommodate him while at the same time making sure that both you and the kids are OK.