"That must have been really difficult for you"

Drives me up the wall! I feel like she is making a big deal out of little things that shouldnt affect me... now reading that I know how it sounds, but still, it annoys me.