I don't think you should tell him your feelings and I don't think you should beat yourself up.

The feelings are likely to fade in time, particularly if you don't beat yourself up over them.

You might want to consider whether you want to be with a man who gets drunk with you and then becomes hostile and uncooperative. I wonder what he might do when drunk to a woman he is actually living with.

You might also want to consider why you listened to his drunken verbal abuse for two hours. I wonder what other options there might have been for you at that time.