Absolutely your choice, sometimes we have to break away from outside influences and do what we feel is right. Others may disagree but their opinions are their business not ours.

Good luck with your new job, I actually know a couple of people who got warehouse posts and love them.