My rule of thumb is: I ask three times. If someone doesn't respond or want to do something I leave them alone. Some of my best friends turned me down initially because they were really too busy or didn't feel well or whatever. So be sure to give people a chance. If they tell you no, it doesn't mean they are rejecting you.
And start small, like a cup of coffee.