I agree with telling him, but I'm also bugged that your friend's husband would invite himself over without getting permission first. If you don't want him there, I'd have a conversation with your friend and tell her you're actually not comfortable with it and cancel. Tell her to ask first, if you think you'd ever be OK with it.