You should start thinking about yourself and break off the engagement.
If your family is more concerned about the possible embarrassment for the family (what is the embarrassment really anyway...this is one of the things that can happen in engagements) than your own well-being, it is time to move on and think about yourself.