I have experienced this also. Manipulative people hate it when I enforce a boundary and I respect myself more than they respect me. I can relate to anxiety re setting boundaries with people.