Hey, Maven...I can certainly understand your dislike of Graphic Design as a major. I'm an art teacher and practicing artist. I worked as a cartoonist for three years after graduating from college...got that job just by chance. Working in any art field, even finding a job therein, is difficult.
No matter how skilled you are in graphic arts, your job will be attending to and pleasing the needs of others. Your talents will be expended toward this monetary goal. Much like a producing machine. Not very rewarding....as an artist. Deadlines, deadlines, deadlines.
I say go for your love! If your heart isn't into the graphic design field, you will continue to be unhappy.
Patty