Sounds like the owners wouldn't care to even go look for him? Even if you think they would, it's worth a try right? I think the idea of chasing him away is a good one but you don't have to tell them. Point the hose and let it roar, Also put lemon juice or vinager and water in a spray bottle and spray. You can put little noise traps all around your fence so when he gets near they pop and scare him. Little mouse traps would work as do tin plates, bags of marbles etc.

Or, take the straight road and speak with them. Show them the vet bills and the injured cats and explain that you saw it and could not get to help your kitties in your own yard at the time. You only have females? If so, him being neutered should do the trick. He won't need to show his stuff once he is fixed,