I don't think that he has a "lack of warmth or empathy", so much as that he doesn't express his connection to you the same way other people do, so you can't read the signals he sends, or he doesn't send signals because he doesn't know you need to receive them. I think you should talk to him, explain your problem--that you can't read his warmth/empathy--and work out a communication strategy so that you can be reassured, and he gets reminded that you need reassurance.

It's just a communication gap, really. You want to communicate with him; he wants to communicate with you. Work together. Pretend you're from different cultures and speak different languages, and find ways to translate. You and he will always be different, and may always have trouble reading each other, but if you learn how to express that you're having trouble reading him, and vice versa, then it can work. Use language as the common ground where you can both meet; language is probably the least ambiguous way to communicate. And make sure you say everything straight out, because he probably isn't that good at getting things you're hinting at. Similarly, he needs to make sure to talk about things he's taking for granted: For example, I've talked to a guy who didn't know his girlfriend wanted to be told he loved her, because he had told her once, assumed she knew the information, and he didn't know that he needed to remind her all the time. Sometimes silly little things like that can get in the way.

But yeah, just talk to him, take on the communication gap together.