Unfortantly, your going to have to face him at some point in time. My advise would be keep it simple. Set your boundries and keep the FIRM. I've been in a simular situation and it is surely a tuff one. Remember, just cuz your co-workers doesnt mean your friends.