Depression or not, kids are a ton of work, a handful, and yes, the not listening is enough to drive anyone to feeling irritable.
Being a mom, can be frustrating. Is the two weeks on/two weeks off, just enough to toss you into a tailspin, by virtue of having so much peace and quiet, that when they get there, it's an overwhelming change in your routine?