Hopefully the lithium will help. But it's not a magic pill. Guilt comes from self loathing, which is a common symptom of depression.

But also excessive guilt can have its roots in childhood in a dysfunctional family. Were you guilt tripped a lot as a child? Were you told that your needs weren't important and invalidated? Were you ever told that you were selfish for trying to get your needs met? There are many things that cause a person to suffer from free floating guilt as an adult.