Sorry, I disagree too... I've always been taller than all my friends growing up, and height or size never was an issue, they just used to moan that I walked too quick. Thinking about other friends I have, one chap who is only a little taller than you has loads and loads of friends, all heights and shapes, and all different backgrounds.

Then ask them if you can join in one weekend or come up with an event that they may like and suggest it to them. You may just have your theory questioned!