I don't know if he is using you but I'll tell you one thing.....he IS getting all the things he needs. He IS getting all the fun parts without considering your point of view and your needs. This is how I see it. Both of you are in this situation but I see only you limping. The happy moments that you think you are having are not really happy moments, there is a shadow over them and YOU are the one experiencing this shadow.
Maybe it is too early for him to commit? 5 years?? Hmmm.....Maybe you should get up and go. This might shock him enough to FORCE him to think.