You are not being unreasonable at all. the unreasonable one is your father, in my opinion. You are setting a boundary that he is not respecting.