My oldest sister has Crohn's and my dad had colon cancer, but as far as I know, it hasn't run in the family (so far, anyway). I think it's probably like mental illness -- your kids might have a predisposition, but something has to happen to turn on the gene.

No sense in worrying over what hasn't happened yet -- I am a pro at that myself, however, so I shouldn't be giving advice!