You need to just tell him that phone calls make you uncomfortable. I think giving him hints would make him feel more insecure than if you just opened up and were honest with him.