I doubt they're going to come get you if they didn't see you give them the middle finger in the first place. Even if they did manage to see it, I still doubt they'll come after you.

Sure, some neighbors are nasty people, but they prefer not to get violent with one another, especially over minuscule things like seeing the middle finger. It's not like you got violent with them and started a physical fight anyways. Plus, neighbors in general don't want to give people an excuse to damage their house or property. They know that getting violent with someone could mean serious repercussions, especially since they are very vulnerable with their house being right next to yours.

Just my opinion, though.