I am going to stumble saying this, so please take that into account.

You do have the right to do what you want to. However, this is work and missing the ceremony done by the people who pay you will be a big slap in the face to them. I agree with you, I think you will be hurting yourself if you don't go.

I wish I could help. What do you not like about your work, if that's ok to ask. If not, I respect your choice.