You have to define your own values and find your own happiness. Once you know what they are, violating them will cause emotional distress. So, to answer your question . . . swinging will be mentally harmful if it violates your values and interferes with you obtaining true, long-lasting happiness.