I think this is something we all contemplate at some time, it's part of the human condition.

When my son was born, and I was mid 20s I suddenly became very aware that one day I would be dead and the awful thought was that I would not be there to care for him - even though in all probability he would be an old man himself by then. It was the enormity of the thought.

I think you should not be afraid of these thoughts, they are a normal part of life and the human condition and may lead to acceptance. As pointed out above, we did not 'know' what it was like not to exist before and that did not hurt us.