It's not surprising that he doesn't understand how you feel, since he probably doesn't know how mental illness work. I think you should talk with him and explain how you feel in regards to this. Even if he doesn't understand completely, he can still offer support when you're down