I think, if you pm the group's creator, you can ask them to rescind the invitation and that notification should then go away.

If that doesn't work, you can join the group, then go to your listing of groups you are a member of and click on leave group.

Either way should work I believe.