Ah, then the perfect way to bring it up
Tell him that you liked when he moved closer and now he doesn't, and that you would like it if he could still do that. Perhaps he did it to see if it felt good to you, and needs to hear that.