It is hard enough for an older, single,male without issues to make friends. Add in low self esteem, low confidence, high sensitivity and it is doubly hard. You're right, making friends is so much easier when you are young. Also making friends is a lot about situation. At work you are with people all day. If you are a mum with school age kids you make friends with other mums. Anywhere where you are with a mix of people a lot of the time it's easier. Even prison haha. Not that i'm advocating that.
I do a lot of things by myself. It would be nice to have company but then being on my own means I can do exactly what I want , when I want and how I want. So there are pros and cons. And I always find people to talk to. You may just have to force yourself to talk to people. Small talk, like a lot of things, oils the wheels of society. Practise in the supermarket but you need to develop a thick skin. prepare to be totally ignored.
It can be quite scary out there and there are a lot off assholes around. I used to belong to a camera club until one guy decided to make fun of my height ( i'm 5'2""). Then one or two others started to the point where it just got too uncomfortable to stay. Their loss as much as mine as I have a pretty good personality and added (I like to think) quite a bit to the club. That is how you must think. You are probably a great guy too.

Lastly, making friends is a two way thing and takes effort. Quite often on here someone says. 'Oh i'm so lonely, I want to make friends with someone'. So I offer. Do they respond. no. Or I hear from someone I haven't heard from in a while. They say ' How are you? I miss talking to you.' So I send a few paragraphs asking what they've been doing , what I've been doing etc etc. Do I hear back from them . No.
Look at meetup.com. Look up local clubs and societies. Look at doing voluntary work. It ain't easy but what's the alternative.