I know. I asked him before that what's the difference between someone mattering to you and caring about someone and he said he supposes caring is more of an active feeling. He said after he was wondering if this is what I had meant by that question.
I still couldn't tell him I cared about him though! I went all round the houses and basically implied it, but couldn't say the words.