First, you can't do a dang thing about other people's reactions! If they want to be immature and get angry when you're busy because "mommy" can't talk, that's their lookout. You're not responsible for them/their problems, even if it's their problems with you :-)

I tell the difference between assertive and good self care versus nasty by the "timing". If it's off-the-cuff with no thought and snarled, it's nasty Giving in to four people until you're frazzled will make you nasty. Being assertive and having the "busy" up and talking one-on-one (maybe make appointments with the others?) is the "best" you can do. You can only do one thing at a time.

Think about that! You LITERALLY can only talk to one person at a time; there is no such thing as multi-tasking, it's just doing things quickly in sequence, but only one thing at a time.

So let them get angry and tell them you were "busy" (don't tell them at what or "excuse" yourself, it's none of their business!) and leave it at that. They'll get over it.