I would be very uncomfortable with my husband seeing my therapist too. It really is a conflict of interest, IMO. And important for you to bring up your feelings about it with your therapist. If he's any good, he will understand your feelings and validate them.

If my husband went to my therapist, I would obsess about whether what I tell her matches with what he tells her. I know that I distort things, but I don't know how much or what is really accurate. It would really drive me nuts.