I think you might feel afraid of rejection and vulnerable or fear letting others down. If they put expectations on you, then you may feel threatened and afraid of how awful it will feel if these fears come true. So as a result, you distance yourself as a way to avoid feeling vulnerable when some says they feel close to you. That's at least my best guess.