All of this is why I became a vegetarian who will absolutely not eat anything made of animal dairy (ok, I'm actually allergic to dairy, but if you cut dairy, you cut like 90-95% junk food). Usually I don't eat eggs either, so technically I eat a "vegan diet" most of the time (I don't really follow the whole vegan philosophy though).

Ok, that's the extreme solution, but a few things people can try:
1. Coconut ice cream instead of dairy ice cream. It tastes amazing and is light. And coconut is healthy for you.
2. Eat real chocolate instead of "milk chocolate" (basically eat dairy free chocolate). Sure, chocolate in any form still has calories, but you get less of the "filler" and you pretty much just get bittersweet chocolate and whatever flavoring that's added.
3. Drink/cook with/etc. almond milk (or soy/coconut milk) instead of dairy milk—more calcium, less calories.
4. Drink your coffee black. And if you can't stand that, at least try to slowly cut down on the cream and sugar you use.
5. Substitute tofu or nuts in an entrée for meat as your protein on occasion (or all the time, if that works for you).

I guess my main advice is to eat less meat and animal-based dairy. I ended up cutting it all out, but like I said, that's extreme but it was necessary for me.