If the relationship is having to be kept in secret then it is already not working and will continue to not work for both you and him. Love needs space to grow and be shared, not kept in an empty box and forgotten about so no one finds out it until sex is wanted. Tread carefully here or end this relationship before it controls you to the point of no end.