On one hand I say if it looks like a duck, quacks like a duck, and walks like a duck its a duck even if you call it a frog. As long as you continue to be happy in this relationship and it works for you then enjoy! If you find yourself compromising who you are and your happiness then I would walk away. Is the relationship honest and open? Are you able to communicate with him about your feelings and desires?