Well, you could always say that you are going through some changes right now in your life and that you need to try to change things up a bit socially to see if that helps. If they ask, and it irritates you, you can just say, "No offense, but it's highly personal. I just don't feel comfortable talking about it."

Or you could go with the good old fashioned break-up talk thingy. "I'm sorry, but I can't keep doing this. My life is just too complicated now and I don't want to drag you into the middle of it... It's not you. It's me." Corny, but effectively communicating the point without directly telling them that they make you feel uncomfortable and agitated and you just wish they'd go away.