After highschool I ended up 'frying my brain', as well, for five years on and off. I just returned to school this year (I'm 25), and I love it! I'm so glad I decided to go, it was an amazing boost to my self confidence.

I went in without a solid idea of why I was going, but I figured that out and now I couldn't be happier. There are actually a few people that are around my age attending college, I know it isn't 'normal', but that doesn't bother me much, it's not like I look light years older than everyone else.

Also,they have placement tests at the community colleges so you won't be placed in courses that you can't handle, but if you do have trouble just go find a tutor.

Since your in the U.S. I recommend you apply for Fafsa, it's a pain in the butt, but it can really help with the costs of attending. Also, a nice perk of being 24+ is you don't need your parents income on the fafsa forms, so they are a bit easier to fill out.