but what if the society is wrong?

Don't let me even get started on it. Societies will let apartheids and genocides run under their noses but bent out of shape if somebody likes kinky sex or does drugs that aren't legal and taxed (cause taxed drugs are fine)... or is of different color, creed and doesn't follow arbitrary rules.

If you harm no one (and no, whining of family members about "how dare you to be gay/not become a lawyer/not be proper in my very narrow definition of the word!" doesn't count).