An invitation is not an obligation. When you are not going to attend, you send your regrets and an explanation or reason why you aren't able to attend is not required. If pressed, "I'm not (or I was not) able to attend" is the only response necessary.