just me-i'd write a short note to tell your aunt and uncle that you're sorry you missed them on their visit. perhaps they don't understand your abscence for lunch.
it sounds like you have a dysfunctional family similar to mine. my T gave me some good advice. he said, how do you stay healthy in an unhealthy relationship (family in my case)? his answer was keep a healthy distance from them. you don't have to cut them off totally just keep interactions to a minmum and have an exit plan. if you don't you will feel slighted and hurt over and over again. i filled my life and son's life with awesome friends. i had created my own "family". they value me as a friend and care for me warts and all. an example of this-one of my friend's son was my son's best man at his wedding. my son met him through my friendship with the father. i love my family but accept they can be very toxic to me. hope this reply helps. it's not you but your family who are unhealthy. usually there is no logic to the behavior so don't take the blame.be glad you can be out of their arena. know it hurts but some of us are not blessed with a loving family.