The client is the one who is supposed to quit therapy, NOT the therapist.

You have insight into what you are doing wrong, so take some time before making decisions, so that you begin to change your behavior that is destructive. Can't you still see him? Tell him what you said here...as I said, it is the client who decides when to leave, NOT the therapist.