It's stressful enough as our parents age and we consider their mortality.
One thing that stands out in your highlighted response is the wonderment of, but did he understand what you meant when you said what you were trying to find the words to say. To me it comes accross as straightforward. He resides under your sister's roof. It's her home that logic would dictate where you would or would not be welcome.
Out of curiosity as this is utterly eating at you(i've my own hairline triggers surrounding my FOO that consume me to my core), what would your desired outcome from saying that you are not welcome be?