I do not think it’s confusing. People crave physical intimacy but don’t always want to be in a relationship. I think this guy is very direct by telling you what he wants. If you don’t want the same thing, then don’t continue with him.
Yes it’s not uncommon to feel sexual attraction toward someone but don’t want to have anything to do with them because sexual attraction doesn’t mean this is the person for you in a long run.