Majority of my friends are no where near my height. I actually know someone that is around your height and he hangs out with people that are taller than him. Friendships aren’t based on appearances. It’s based on common interests, and other things like humor and loyalty as an example.
Are you sure it's is has something do with height? You should ask why they never ask you to hang out. You won't know until you ask. Don't make assumptions because it may not be about your height. It could be your personality or attitude? I totally agree what [Spockette] wrote and do believe deep down you have some self-esteem issues.