Are you married? Is the reason he is still dwelling on your history a prelude to a proposal? More to the point, there seems to be a trust issue here.