I think you should discuss with him your thoughts. You should always respect yourself, and relationships need boundaries to help keep both people faithful.
Birds need to spend time with their master, and be restricted at first, and in good time, they can be let go and they will come back to the master. If the bird doesnt come back to its master, its because the master didnt set boundaries first, and implement them over time.