Actually, from my perspective it sounds like he wants to be friends, but he's also got a lot of other things going on as well. He may also be picking up on the "I wanna date you" vibe so when you guys hang out, he feels he needs to let things calm back down because he doesn't want to string you out.

Overall, it seems like everyone else (friends, acquaintances, etc) are all pushing something he doesn't seem to feel.