You totally did the right thing. Like I said before, he is probably very immature with a self-image problem and uses you to boost himself. You did the right thing, and you can do so much better!