I personally think that with trust its actually the other way around, you have to do something that causes mistrust before you can prove you are untrustworthy !
So as long as you have been open honest and trustworthy then the problem is his not yours in my head.