I don't know that you have to pull back; just don't move forward to engage. Adults have friendships with people and with friends, you are friends because you enjoy being with them. If you do not enjoy your extended family, you see them during holidays/when it is expected, etc. and that's about it. There is nothing wrong with you for not particularly liking them or wanting to be with them anymore than there is something wrong with them for enjoying who they enjoy.