My thoughts would be that either.......

(1) He started to care for you more than he wanted to.

<font color="red"> "OR" </font>

(2) He has a special someone in his life now - and she is not comfortable with this relationship.