Fearing that he might leave is a waste of your time and mental energy.

You will deal with the problem of his leaving you if and only if such a problem arises in the future.

You can prepare yourself to deal with such a problem by diversifying your set of friends and activities,as advised by Payne.

That is all you can do.