I'm not sure how it works in Sweden but here in the U.S., specifically Michigan, the term "caseworker" is extremely subjective. I did that job for 10 years and it was quite the experience.
It sounds like your caseworker is more on the "social" side of things versus the programmatic side of things. I worked with those on food assistance, cash assistance, Medicaid, and child day care. Whatever the case, recipients of any type of assistance can be EXTREMELY demanding and it places a LOT of stress on the worker. In no way am I saying you're being unreasonable but you do have to take into consideration a caseworker's load.
I had to manage 400 cases. That's right, 400. All day long my phone would ring off the hook because somebody's power or water got shut off, their benefits didn't appear on their card, they quit their job, got a new job, had a child, moved to another county/state, etc. etc. etc. Our system was designed for failure and there is no way anyone could keep up. I would work 12-hour days (and get paid for eight) just to keep up and even then it was nearly impossible.
If your worker took the time to go on walks with you and show some compassion (at least that's how I read it), consider yourself blessed. Casework is a thankless job. As a caseworker, you seldom hear any praise whatsoever but the second their benefits don't appear on their card or their power gets shut off, you get an earful for some crisis you didn't even create and you'll NEVER be accused of getting things done too quickly.
Just please remember, your caseworker is doing the best she can and she is not there to make a lot of money. Trust me. I wish you the best in all your endeavors but if you want to make your caseworker's day, call her and let her know you appreciate all she does for you and others. You have no idea how much she would appreciate that.
Peace.