I only ask for people's numbers when there's a practical reason for it. For example, I like to have my neighbors' phone numbers and I'm very direct in asking for any new neighbor's number immediately. What often makes me feel more comfortable about it is saying something in explanation, like: "Just in case I have an emergency--or you have an emergency and need something." All my neighbors have been more than glad to share their numbers after that, often acknowledging that yeah, it's a matter of safety sometimes. Similarly, when I ask for someone's number, I often joke, "Don't worry, I promise I won't be calling and texting you incessantly or anything. This is just in case something comes up." So maybe having some go-to sentence like that would help you, when you do need or want a number.