Gosh Inky...For me, trying tell a woman when/if she should have a physical relationship with a guy is really difficult, since there are so many issues involved. I can't put my "stuff" in your head as far as my decision making goes, ya know? Say I may use my spiritualty, and/or my personal ethics, - mine may be different from yours. So how can I help guide you in something so personal?
One thing I know FOR SURE is this - if you are not absolutely aching with excitement, and anticipation of the moment you two can be together....then it seems like it's it not the right man or not the right time.
Perhaps talking to him and saying, "it's just too soon for me to be comfortable". If he heads for the hills, fine. Heck, even Howard Stern goes by the Three Date Rule! (if no nookie by the 3rd date, he's gone....ya, great loss there!

)
If the sole purpose of this guy's dating life is to get nookie, your heart won't be properly care for, hon. Please protect it, OK?
Emmy