My idea:

First, do not stop making coffee for yourself when you are the first in the office or otherwise. You have every right to do so. she doesn't have to drink it at all. period. just because she's a loud mouth (censored word), doesn't mean you should give up. This is part of why people like her are the way they are.

1. Ignoring her is in no way letting her "walk all over you" in fact it is actually one good way to stop the behavior. See, bullies don't do things to be ignored, they do things for attention. Sure at first she will continue to be loud and obnoxious with you as one of her chosen targets, but in ignoring her, you must not react or even do anything. If you use this approach, basically you have to carry on as if she is not there. do your job, focus on what you're there for, which does not include being her source of entertainment or being her proverbial punching bag, but your job. You will find that eventually she will move on. Unfortunately unless everyone takes the same approach, she will find another victim, but that's not on you, you can only take care of you.

2. NO passive aggression period. the above, what I've said is not passive, it's just going about your business as you hve every right to do, not in any way with the motivation to get back at her or make an aggressive statement to her. Passive aggressiveness will never serve to make things right or better only feed the fire of conflict.

3. Approaching her privately is rational but I'll be honest, it sounds like this is just her default mode of operation and likely one conversation with her no matter how calm and rational you are is going to change her personality or behavior. It's a mature approach I will say but I just dont' want you going this route expecting much.