You should not "try to keep an open mind". If you want to stay with him, a firm "no" would be easier for him to comprehend and abide by (if possible at all) than "occasionally OK" because that is very ambiguous, unclear and subject to differing interpretations.

Given his predilections, though, most likely your "no" would mean that you and he are not compatible, and I personally do not think that you will lose a lot but dumping him, as TheGal suggested. All I am saying is that IFF any continuous relationship on your terms is possible, a clear, unambiguous rule is better than "occasionally open to".