I also think that he is manipulating you. I would withdraw from him, and consider doing so permanently. He does not sound committed or invested in the relationship.