I am wondering what you mean by "You're on the same server so it might not be easy to just drop him." And, you don't owe him anything.