I think only you can decide if these people have crossed your boundaries too many times and how you feel about it. Sounds like you have given them plenty of chances. And it depends on how draining they are. Sound like to me that you are doing a fine job of standing up for yourself.