Yes...and yes. Sometimes avoidance is the diplomatic approach, and sometimes speaking up is required. It all depends on the context, and the particular individual with whom you are dealing.

I am also learning boundaries at work. I have a thread in the work forum on just that issue. It's very similar to issues you've faced at work.