It sounds as though this is a longtime issue, and not an easy one to deal with.
If he should, even in theory, be able to care for himself, then he should be expected to do that. You've done your due diligence. At this point it's probably okay for you to give him some expectations and boundaries, and then to step away from the situation.
Perhaps if things go bad again, rather than stepping in to help immediately, you call APS at that point? See if they'll get involved at a low point instead of brushing it off after the fact?
My feelings about mental health and other services for those who clearly have issues, has become very negative over the past ten years or so. There have been a couple people in my life who were clearly compromised and in need of some kind of help, and yet there seems to not be much assistance in the way of intervention. Rather, the involvement doesn't come until the situation turns catastrophic. That may just be my experience, but it is my experience.
If you choose to step back and distance yourself, that's not only your right, but a healthy thing to do- both for you and him. You can always keep one eye on the situation and report it if it starts looking bad again. But above all, don't feel bad for prioritizing yourself and your family