That does seem to be unusual for a relatively young man. Although he's not that interested in the sex act, he does seem willing to engage in it with you. It's more the woman who usually gives sex to get love.
Would he be willing to go to a counselor with you? It seems to really be upsetting you, so it would be helpful to hear more about his thinking and background.