Chloe,

The culprit is the friendship with people who "do not support our relationship (over nothing more than sheer jealousy)", it seems. Unless he dissociates from them fully, you are just being like Sisyphus (citing Albert Camus):

"The gods had condemned Sisyphus to ceaselessly rolling a rock to the top of a mountain, whence the stone would fall back of its own weight. They had thought with some reason that there is no more dreadful punishment than futile and hopeless labor."