Iam sorry that this happened to you. From what you typed you seem very involved and considerated of the children and there feelings i think these co-workers are just jelous of you in some way. I would just chalk it up to you did the best you could and move on no sense in staying where you obviously not wanted.