A month ago I met a guy who helped me in a critical situation. After about two weeks after spending looots of time with him and talking about everything in a friendly way - including relationships, where he told me he was a polyamorous person - , we became intimate. It was done in spirit of exploration for me, because I am not usually attracted to men but suddenly felt chemistry with him.

Shortly after I expressed to him that I don't function as a friend with benefits and am not looking for something casual (that's the vibe i got from him) and that it might be better for us to just stay friends. He said he understood but wouldn't mind 'something with me at all' and that it'd be a monogamous dynamic for now cause he isn't seeing anyone atm, 'not like me at least', whatever that means. Yet the vibe I get from him IS totally casual, and i don't see it going anywhere. I really want to communicate that to him and go back to the friends only level but I am afraid. I wish someone esle could just do it instead of me haha.
A part of me also wants to 'give it more time' and see where it goes since it's such a new dynamic AND I definitely caught feelings. However, I do find myself anxious and this situation has been an energy-drain to me.
ANy advice appreciated! I