I would be careful to separate my emotional feelings of irritation from taking action. The first lesson that I learned from living in many such neighbourhoods is not to make myself a target, unless direct aggression makes it impossible to do otherwise. If you want to take action, then do it with the community police or some such group at your back.

If no one is bothering you directly, then let them be, and have patience. What goes around does eventually come around: that's a second rule of the hood.
It's good that your neighbourhood is going up in the world. At least that means that this one house isn't going to take it down.