Yes it would bother me.

My mum started seeing the same therapist I’d seen several years before, a lot of my issues had been about our relationship. I benefited from therapy and had a positive regard for the therapist. I encouraged my mum to relate to the therapist but she put barriers up and was quite critical of the therapist, ended the sessions early. Not the same as your situation but I did feel uneasy about us having the same therapist, even though I wanted my mum to benefit. It felt like it crossed some sort of boundary through.