I see going back to him as playing with fire. You know there is something between the two of you, yet you went back to him, which means part of you still wanted to explore and/or test that. Now you are faced with this slippery slope.
As for this guy, red flags are popping up.