Everybody's family is different. My family is very small and slightly dysfunctional but it's what I've always known. I've noticed I date people with very large, tight-knit families that behave in ways completely different than my own that I do not enjoy.
Everyone's family is different and if you really want to stick with this guy there isn't going to be a way to avoid his family. Have you told him how it makes you feel? Maybe you can explain how much stress it creates for you especially when you haven't been feeling well and just ask to be kept separate his family ordeals as much as possible until you feel better and are able to be put under that kind of stress. Just explain that he is important to you and you know his family is important to him but that it is not a good time for you to be in that kind of environment.