I don't think changing the pattern has to be keep on with him or not terminate. It could be, but it could also be a formal ending. Even your message to him doesn't seem to me to be just cutting and running, but trying for a polite break.

This is what I find hard about making decisions like this (my current thoughts about CW, for instance): am I cutting ties too soon, or should I keep giving the other person chances? Change doesn't have to be the exact opposite of what you were doing, there are shades of gray too.