You do have to speak up when someone is violating your boundaries. However, there are times when it is best to keep quiet. Say they are having a political discussion at work and you disagree with most people. If you speak up, it won't change their minds and will cause unnecessary conflict. I am one who speaks my mind too often and unnecessarily, so I've learned that.