It sounds like you're putting a lot of expectations and pressure upon this relationship.

Just yesterday you were happy, but worried about what next? Today, you feel as though your work schedules just aren't going to allow a romantic relationship to occur between you. Don't give up so soon! Take some deep, slow breaths and try to let go of that loneliness and paranoia that you're feeling inside.

Out of personal experience, I have firsthand knowledge that no man (or woman) will ever fill that void that you feel within yourself. Try not to focus on the big picture. Like, "I never get ___." and focus instead on moment to moment life.

Human beings are social beings, and do need some interraction to stay sane. Maybe you're going out of your way to be there for those who aren't the right fit with you. Some people will keep sucking you dry, and not giving anything in return. It is time to end those relationships!