Ack. I hate them and they're aggressive. Can you get to a store that sells wasp spray? That would be the safest way to kill them. If you have a fireplace, they may be building a nest in the chimney. Or in a roof vent, or in an attic. I'd call a pest company to check it out because that's a nest you don't want to mess with.