That would make things quite a bit worse, that one is a friend of your husband's, IMHO. This is, again, assuming you are not in an open marriage and your husband is and expects you to be monogamous.

In the case of a husband's friend you are potentially ruining your marriage and a friendship. 3 people get hurt in that case (you already know you will feel guilty, you've said so, so you are in that equation) just so you can have some fun.

When I am bad (either up or down) I tend to only destroy one person at a time and end up regretting it massively. I can't imagine the guilt I'd have if I hurt two people any myself at once.