If you can and your mom is willing, it's probably best to stay put and where you are, building your life from that point.

Have you read anything about "Verbal Judo" or "Non Violent Communication"? There's a way to communicate disengaged and putting your emotions aside, at least in the moment, until it's safe to express them.

It's ok to say, "I'm not going to have this conversation with you."

And as a relationship piece of advice, maybe keep whatever emotional feelings that you have about his family, out of his earshot. The baby lives right there where you both are right now. You are not obligated to move back with his family. Wherever the baby does have their full first 5 years does become state of custody.

Maybe they won't go to the party? And if they do, just disengage from emotions.