I doubt it, to be honest... After his reaction, I'd say he's not worth it. "Uh sure whatever?" I mean, c'mon... how's that respectful, polite, tactful, ... or even just nice! Ergh. I [strongly dislike] guys like that. Sorry if this hits a nerve with anyone; I'm certain there are exceptions.

If some random woman gave me her number, I'd be very flattered and thank her, whether I'm interested or not.

I believe the outlook you need is: "I don't care whether he calls or not." obviously you do, but don't let you know that!