I notice some of the answers you're getting are from the US and thus more US-orientated. Not that it matters much, but do be aware that 'college' means different things in the US and here in the UK.

In the US pretty much all tertiary education (post high school, in other words) is referred to as 'college', including university. Here in the UK 'college' strictly refers to tertiary education institutions where people go to learn vocational trades, like electricians, plumbers, hairdressers, etc. and to do courses which prepare you for Higher Education/University such as AS/A-levels, Access courses, HND/HNDs, etc. and they run courses which are aimed at helping unemployed or vulnerable people get back into work, such as people who've just been in prison, etc. But these institutions cannot award degrees. These have a lot of older adults, but they're mostly made up of 16-19 year olds who have just finished their GCSEs in Year 11 at school. I think in the USA, these are called 'community colleges'.

As for the OP, don't let age hold you back. As an A-level student you will be older than most (if not all) the other students, but so what, you're there for the qualifications and there's no reason you can't befriend them, they probably will not even be able to guess that you're 20. I'm planning on going back to college next year to do an Access course at 20 years old (I'm 19 now) so I can get into university, I was hoping to do one in September but I screwed up on the preliminary maths/literacy tests (anxiety attack, i froze, i couldn't register what i was doing, despite how simple it was) and I had to decline an offer to go back and retake them since I'm unsure if I'll still be in the city next year, my parents have been threatening to throw me out and they're planning on trying to sell the house and moving up North to Leeds.

Hope it goes well, anyway. What are you planning to do after your A-levels, can I ask?