If your BF finds her presence awkward, as he might, explain to him that the woman helped you with a referral in the past and that you felt indebted, but that you do not intend to feel that way for life and that you won't invite her again when he is in your company.