I love that you see colors in the world, and around people. I also love that you're reaching out for support when you need it!!
I know it may sound silly...but I had a thought for you.
If green is your comfort color, and you see blue around the counselor -- try wearing or bringing something yellow to the meeting. The yellow will offset his blue, and make the whole session into something more green.
(I'm quite connected to colors myself.)