Congratulations on getting married!!!

I would echo what has been said. This is for you and your fiance, and that is that. You are in no way obligated to invite anyone, family or otherwise. If you feel like certain elements of your family would be a negative impact on YOUR big day, you shouldn't feel like it is in any way your "duty" to invite them.

Please have a wonderful wedding and many happy years together, and I wish you my best.