If you want more than a here and there relationship , then your going to have to ask where he feels your relationship is going or not going..
It's hard to ask a question knowing if we don't get the response we want is going to feel devastating, but clearly you want more and its not fair for you to just wait around for the next hook up.
I wish you luck