You did the right thing. Hold firm. He sounds like a con man who doesn't have your best interests at heart despite what he says.