This is such an excellent and thought- provoking post. I see your desire to serve in one of two ways:
1) A guilt reflex over not following the program instilled by your religion of origin
2) A genuine yearning to change the pattern of your life and begin to dedicate yourself to spiritual service
Based on what you said, I'm not sure which of these is your truth. Once you decide for yourself if this life of service is what YOU want, I think you will know how to move forward.
If it is what you want, I have been there. I know the experience of changing your life direction at midlife carries with it many frustrations and uncertainties. However ultimately it should lead you toward a life of authenticity.
Happy travels!!!