Not my saying, but one I like.

Once you stop making yourself small, not everyone fits into your life.