Yes I would feel the same about the lack of time. I think you need to sit down with him again organise a regular date nights. Go out somewhere so you are away from the in laws.

Make it the same day/s of the week at first so you get into a routine. It doesn't mean you can't change it and be flexible but it might help to establish a firm routine first to strengthen your relationship.