What happens when you are the primary invited person. Does she bring additional people into a scenario you thought would otherwise just be the two of you? Do you notice whether these other people are also uncomfortable?

What you might wish to do, next time you get an invitation, is to ask her who will be present. Go ahead and be honest about your comfort zone. Let her know that you would love to get together but that a crowd can be difficult for you. A friend would understand this.