Follow up--
If I had left, I am sure I would not have lost my job. But it is likely I would not have gotten paid. I hate, hate, hate confrontation.
Just remembered a quote. It goes something like--A coward dies a thousand deaths; a hero dies but once.

I still cringe when I remember this incident.
The best thing I learned from this is that I still need to work on assertiveness.