Maybe you can take a driver's refresh course or something if you don't feel comfortable? I had a girlfriend with multiple eyesight/perception problems with driving but she literally worked at it and trained herself to work with how she knew herself to be; can you work with practicing looking more toward your blind side?
I would not base whether I should drive or not on one bad/freak accident; the plow driver does sound like a jerk; you're not responsible for what he can/cannot see! The person following a person who slams on their brakes and plows into them (sorry for the pun) gets the ticket for a good reason; the person is allowed to put their brakes on, they're not responsible for how close the car behind them is following and whether that car is keeping itself safe. If the plow driver couldn't see, he had no business backing up at all!
I would use your bf's observations of your driving to your advantage, instead of taking his comments "personally" and learn to practice looking out more for problems where you have trouble, see if you can't improve your driving skills all around? My husband use to get angry with me 3-4 cars ago because I would let go of my seat belt and it would retract but hit the window on it's way "up". That made him unhappy and he wanted me to "guide" the retraction with my hand so the metal buckle didn't bang against the glass window. I worked hard, on my own and without saying anything to teach myself to do that and am still quite proud of myself that I succeeded

He never mentioned it but I use my success and what I did as an example for myself and him when there are similar "learning" problems like that?