It seems like maybe you could have some OCD tendencies or other anxiety problem that is causing this... I think that SSRI's can really help with this kind of thing. Have you ever been on meds or considered going to a psych to get some?

I have a close friend who gets irrationally strong worries like that. When she takes the prozac she is prescribed it really helps her not obsess about things so much. Her worries are similar to what you are describing - like it's something she should be concerned over to a degree but when she's not on her meds the little things can start to dominate her thoughts to an irrational degree.

Anyways, I hope that helps... I think you should try to talk to a doctor and get on an SSRI if you are able to (I'm on paxil because I get irrational social fears - a little different but also an irrational fear that I can't control )