It is ironic that the guy who complained about your leave me alone comment is stonewalling you.

If you stay with him I would expect more such behavior from him in the future.