I would call it quits the next time you go in. Just say your new job had you start immediately and you aren't able to juggle the schedules.