Having ADHD will not destroy your life unless you let it. You may have to come up with creative ways to cope with certain things, but being able to think differently can work to your advantage. In a helping field, like social work, it is important to be intuitive of the feelings of others. Individuals with ADHD are often sensitive to small things about a person that clue us in to their mood. Also with this, you will have a view with those who you are working with that others in your field couldn't understand.
I understand your worries. I had the same worried when I was diagnosed, because I'm pursuing a career as a music therapist. You just have to learn coping strategies and find ways to use it to your advantage!