youOme, I hear a lot of sadness in your post.
Please don't have another baby right now. (How many children do you have?)
A baby won't fix your marriage. There will be more stress and less time for yourselves and each other. And less time to spend on your studies in college.
Please find a good marriage counselor and work on your marriage if you want to stay together. It sounds like you are both wistful for better times in earlier days. This seems like a good thing, that you would both like to recapture the positive about your relationship from before. Good! And sounds like both of you are on the same page. Good! I think it sounds like there is a lot of potential for marriage counseling to help.
You are in college fulltime to get a degree? Great! Stick with it and get that diploma.