Honesty should be the basis of all relationships. Be up front with him and tell him you don't mind casual dating, but that's all. Put him in "friend" mode. He'll appreciate your honesty (or not call again). A win-win!