Well aside from me being a bit jealous...

I think your husband is into the threesome idea, and might be hoping to have a nice poly-amorous relationship with you and this girl from work.

You need to let him know that this is NOT okay for you. You are married, you took vows. Even if you did allow this to happen once or twice, that doesn't mean it is okay forever. But he needs to know that is how you feel.

And don't beat yourself up for being "abnormal" or "wrong" - your feelings were valid, there is nothing wrong with you.