Hmmm, I am also seeing a man who has mental illness. I think the others are right. You don't owe him anything and should do what makes you feel comfortable.