It's a good thing you're acknowledging this, to start off.

Yes, your behavior may definitely come across as abusive. The reasons you posted sound all very likely. Also, could it be possible that you're afraid he might get into trouble? You said he used to be an alcoholic, after all..

I don't really have an advice, though. Do you see a therapist? That might help with your social struggles. Maybe "couple therapy" could be an option, as well?