I'd say loneliness is probably the real problem here, when we're lonely we tend to look back on the past more because we want to find a way to not feel alone again so your mind keeps thinking back at a time when you maybe felt less lonely.

Of course, if you could fix your loneliness you probably would have already, so that's not much of a solution.