Sounds to me like he wants an open relationship. Also sounds to me that is not acceptable to you. I would myself consider this a deal breaker for the marriage. Both people have to genuinely want the same things.
Currently I have sexual relations with a man that is married, his wife has her own little sexual relationship with someone else. Even though I thought it would be rather WEIRD at first, it isn't. She made me feel very comfortable and honestly all 4 of us have been in the same room, getting along just fine. Not sexual with all 4 of us or anything like that. Just hanging out.
HOWEVER that is their agreement and they are comfortable with it. Personally in a marriage I am not sure that I could handle it?!? After this experience I might be able to but really I am not sure. It works out great for me as I have promised my oldest son no more men allowed in our lives so I get my sexual satisfaction and don't have to deal with any sort of relationship that involves my children.
A marriage will not work if both people are not wanting the same things out of their marriage, end of story. So I guess that was his way of telling you what he would like and ultimately it sounds like the decision is up to you on if your willing to accept that or not....