perhaps "I enjoy talking with you casually but I'm not looking for a date".
It sure is awkward sometimes finding out what I want to do in such a situation, and then acting on it.