How about having a male friend answering to him? just like he is your boyfriend and politely questions him about what does he want from you.
Can you give you phone to a male friend / brother for a few days and see what happens?