If I had to decide after 4 or 5 visits to my new T if he is the one I am not sure that I would have decided to stay with him.
I still have troubles adjusting and the past weeks were difficult. But now I finally have the feeling that it was worth it... that he really is the T for me and that I can allow myself to start trusting him...

I guess it really has to do with the kind of issues you have as well and you should always consider that too, when making such a decision.
For me it is generally more difficult to trust or to allow a bond to someone to build up. 4 or 5 weeks are not enough for me and I do know that... So for me that means to just hang in there and give myself enough time until I feel that I can really make a good decision that is based on my true gut feeling and not on my fears.