A recently hired coworker told me that she considers our boss incompetent and has no respect for incompetency. The first time she said this, I responded that "this was disappointing to hear because I like our boss and I respect what he is doing." The second time she said this, I did not respond at all.

Should I tell my boss about the comments she's making?

Additional background info
1. She joined our team about 3-4 months ago.
2. She tends to be very argumentative and will "grind you down" if you engage with her.
3. I have a sneaking suspicion she's making similar statements to my former boss who was replaced by my new boss.

What I really want to do is just crawl into a corner.