I would not just quit your job; I would make a business plan and factor your job in, to help finance the start-up, etc. and only quit when the business was well established and looked like it was going to work.

Being in business for yourself is a 24/7 job and extremely stressful and hard work, not just about doing what we enjoy. You will be forced to do what you use to enjoy, long after you may want to stop for the day. It's not all designing and organzing and making the product either, at some point there will be health inspections, licencing and tax issues, and other non-fun stuff unrelated to our specific calling. At first you will not be able to afford to hire accountants, web designers, marketing people and other specialists you will need to become a success and learning/doing those jobs yourself can be frustrating when all you want to do is make the candy.