Hi Katies. (and hi Von),
If you hadnt mentioned your kids I would have said some different sort of things. But from what you said, your children are being hurt by this man.
First and foremost, they deserve your protection, otherwise they are being betrayed.
I dont speak from the moral high ground, but from my own past. I have been involved with a narcissist, I was addcted to him, My child growing up was damaged by his behaviour. You can probably guess my regrets and pain over this.
What ever it takes to protect your children from emotional harm by this person - specially as he's not their father?-

N. are addictive, you get into the situation that reflects thier way of thinking about themselves, that they are the world, and nothing outside them could ever be valuable or nurturing. This is a delusion.

Theres lots of sources of help if you need too.

Its not so much about how good/ bad they are, its about being at a certain developmental stage, but in an adults body this can be dangerous and create a vortex. My mother was an N. What she touched or got close to she polluted and damaged, and worse.

Sorry if this is a bit grim, but first you need to get you and your kids to an emotionally safe place, before you can think clearly and get perspective.
Good luck.

riverx