Better tell him sooner than later. Just tell him that you don't feel the chemistry isn't working well. Then size him up and act accordingly. Congrats on your new job!