Yes, it is abuse. Abusers aren't necessarily abusive all the time, this is why people stay longer than they should in such toxic relationships. They can be nice, caring, charming. This is the game they play to keep their 'prey' bound to them.

And yes, you are overthinking. You are over-analysing the situation. Instead, it is time to take action and leave him. Not saying this is going to be easy but it is the only way.