I think it's okay to do whatever you want as long as everyone involved knows what is going on. When you're honest with people, you give them the ability to make informed decisions.
There are a lot of people in the world into open relationships/polyamory, and all sorts of different sexual scenarios.. Monogamy is just one scenario.
So I think as long as there is open honest communication and consent, whatever you want to do is fine. (:
No matter what people will judge so you just gotta be comfortable with yourself..