Sure he has lots of good things that confort you and you like so I understand how you feel and I understand your needs. Said that, listen to your guts...they are talking to you out loud. I can feel them in some of your posts in this thread.
Remember that he has a wife, a family to take care of, a life...as you have your life and sometimes, there are too much things against to built a relationship with somebody.

Do you think you and he can stay together only as friends?