I hope you distance yourself from this man. He is not to be trusted and you don’t need someone like this in your life

No you didn’t bring it upon yourself. Sadly there will always be people who will judge and blame victims of assault for how they were dressed or how they behaved or who they invited over etc

He isn’t a stranger so inviting him over is not a crime, sure it’s not advisable to drink heavily for health and safety reasons, but you drinking or inviting a man over is not a permission for him to have sex with you against your will (and taking condom off without a permission)

You were violated. I am very sorry. Please continue working with your therapist. I commend you for sharing with your therapist. Stay strong and keep seeking help