I wrote him a letter that said there's no way I could have said no to him touching my shoulder.

We never discussed that.

I might ask for him to touch my shoulder but then he would start rubbing it and there was times it developed into more of a back rub.

I always referred it as him touching my shoulder in letters as that was true but not the whole truth. Will it look like I'm lying?

I never asked him to rub my shoulder or back.

I see a difference and that beyond placing his hand on my shoulder he should have waited for permission to rub it or my back.