Neither of you is ever required to conform to the beliefs of the other, of course, and it is easy for me to remember that fact while also imagining exactly what he has said: "difficult to cope with lack of, or at times, odd spiritual beliefs". You might prefer the idea of "it all seems like brain chemistry to me", but many people believe there is still more either behind or alongside that and your therapist is simply saying he does not know how to offer his best to you in the face of some of your specific beliefs clearly opposed to his.