I cared for my mother, in my home, who did have dementia, for a number of years. It is very important to know that nothing you will say or do can make her able to do/remember the things that she cannot, and things will get "worse", this is not behavioral but neurological. It really is best if you can develop a bit of humor (nothing mean) and a lot of patience, let the time go, don't worry too much in the moment, don't worry with your mother present.
If she has a health care power of attorney, it may need to be activated. She will need more assistance as time goes on. Maybe there is a 'senior center' or 'senior friend' or other program/assistance in her area that will not feel like someone coming in from outside. If she is doing 'OK' with you only coming 4 times a year, that is not bad. Maybe she was always anxious, whether that is true or not it is Normal with loss of abilities you once had, with dementia/changes in the brain/abilities) Reassurance is in order, even if it is a bit excessive.
If she doesn't have a power of atty it would be good to do that while she is capable.
I know it is hard but unless she is a danger to herself it is best if she is supported in whatever situation she wants to be in for now. It is not easy, believe me, I do know that. (and a peer support group for caregivers could be a real help for you, a place to let it all hang out and share stories with others...) Best luck!