When you married him, you married him alone and not his parents. Regardless of whether you live with them or not, they need to respect your relationship and your space. There are some boundary issues here that need to be addressed before they fester even more.