The real problem with this is he doesn't seem to be a danger to himself or others at this point. The only time he tried to get physical with me was when he had an episode at the house. I spoke to his mom and we agreed if he stayed with his brother, he wouldn't be around me and I couldn't "spy on him" or "whisper to him". He tried to fight me but I blocked everything and didn't go on the offensive, de escalating the situation. I still made him leave.
He mostly plays video games all day and eats, that's about it. When I am around him I engage him in conversation about 2 or 3 times a day mostly to judge his mood.