You're never too old to go back to school, there are many middle aged and even some elderly people who will go back to school and get their degree. I am also a "late bloomer", as in, I didn't get my degree when most people do right after high school (I'll be going back to college in a year and a half, but that is still quite a while off. I'll be 26). 46 is definitely not too old.
I have a mom and a dad. My dad raised us 3 kids by himself while going through mental health issues. We didn't have a lot of money, so extracurricular activities and good healthy food weren't common in our household. The house was often very messy. My mom left us when I was 4, and my siblings were 7 and 2. I have SO much more respect and love for my dad, despite his struggles he did his best and I couldn't ask for more than that. I imagine your children may share my perspective someday about you.
If you ever want to chat/vent, feel free to PM me.