I notice a pattern in this thread repeated on several over the past months and it seems like you're again setting him up to be deemed inappropriate... initiating an interchange and then trying to find his wrongdoing in it, with your excuse for crossing your own lines being that you got yourself stoned so you're not responsible.
What are you hoping to hear? That he's acting irresponsible with you? Do you feel like he's making it too hard for you to distance yourself? I imagine the intimacy is hard?
I've worked with my therapist as late as 1am.
The only person who can really answer your question though of course, would be him.
It's certainly not invading private time, because no one is forced to read emails or texts, nor to give out their number. You can't invade space you've been given permission to occupy unless he sets a new boundary which he hasn't.