My boyfriend also does this, only he is playing with his phone by himself, ignoring everyone around him. If we are with my family, I ignore it. He is responsible for his own behavior.

If we are with his family, it's a different story. I tell him ahead of time that I am not going to do the hard work of keeping up a conversation with his relatives if he can't be bothered. So when he brings out his phone, I nudge him as a reminder to put it away. I've also told him that if he's going to play on his phone the whole time, I'm not going and he'll have to deal with them alone.