Good point, literary lark. Does this person represent a company that you could formally make a complaint to? Good on you for not tolerating his harassment any longer.

Your friend who linked you to him in the first place. ... Are there others from the same source that may be a possibility to help you network?

Time frame. .... Are you able to delay the trip by moving the dates to a later time to allow more time to find a replacement?
Re the "potential agents". Are they linked to this man or are you still able to contact and meet with them. ....just with someone else as your go between?