Tom has it worse because he can't excape into walls or wherever and any destruction caused by their fighting, the humans are going to blame on him.

Plus, Tom is in the middle since the bulldog is always after him so he gets it from both sides; the frustration of never catching Jerry and Jerry's using the dog/surroundings against him.