The tree is his responsibility.

If the tree is a danger and/or diseased, contact your City Aborists. Ultimately, they will remove the tree if such is the case. Otherwise, I'd have it removed and pass along to him the bill.

Do you rent? Contact your landlord and the ombudsman.

In the meantime, build your own fence along the property line. He can't ignore that.