yep, i have experience of a borderline friend getting frightening - you have to set boundaries, otherwise you are unable to have any kind of defence of your self - everybody has to respect certain areas of other people that are off limits, borderline friends included. try to be firm and set boundaries, but attempt to do so in such a way as your friend doesn't become frightened by it - ie don't be aggressive or too defensive, but be firm and clear and strict.

good luck! x