"Guilt is a cognitive or an emotional experience that occurs when a person realises or believes--accurately or not--that they have violated a moral standard, and bear sole responsibility for that violation." http://en.wikipedia.org/wiki/Guilt

I think we carry around guilt only when we don't "bear sole responsibility" and thus, make restitution. It's the dramatic breast-beating without doing anything practical about what we believe.