Friends are those we get to choose (can't choose family) so it's good to be around the "good" ones that make you feel better and inspire you to do your best, etc. There's no place for mere "kindness" in friendship, I don't think; I'm not saying don't be kind to the guy as he follows you around but if you don't want him for a friend (and it sounds like you don't) then you can't really treat him as a friend.

You think you won't have to see him while school is out but inviting him to this thing (that isn't "yours" so if others don't want to hang with him and don't come, that's not bad on their part, anymore than it was good/bad for you to invite him) might mean he gets the "wrong" message from you, you are going beyond kind to saying, "I want to be friends with you" and he may ask you for your phone number, address, suggest hanging out this summer, etc. If he doesn't respond appropriately to kindness, is a "pain" then he has to first learn how to "be" with other people; making it so he's elevated to friend status might be way beyond his league/abilities at the moment and discourage him because he probably knows the others don't care for him and think that he doesn't "deserve" your friendship.