It sounds like you have very sensitive hearing, and loud noises are very jarring to your senses and uncomfortable. It makes sense to me that then you'd be afraid of the noises. Especially if it comes out of nowhere or surprises you. Are you in therapy? Maybe you can work with a therapist on this to at least be able to cope with the distress loud noises cause you, and learn not to be afraid of them or the reactions your nervous system has in response. If you're concerned about your hearing, consider seeing a specialist (audiologist, I think).