Your gut is already telling you this guy is NOT what you want or need.

I would say dont waste time on him any further.

Casual sex is fine if both people are just wanting that and that alone, But is seems like you want a relationship that goes far beyond just sex hook ups

Stay safe