This is coming from a Bi female.... Just be honest with her. I was raised baptist, so I know how the guilt thing is. I don't believe it to be wrong anymore. I spent alot of my life miserable.
Talk to her openly about it. I don't really know what to tell her to start the conversation. If she is on here, have her get ahold of me, she can ask me anything she wants.
Right now I'm happy and content in a lesbian relationship, but I've been married before to men that were no where near as understanding as you are.
But also be prepared. In re-discovering myself, I've realized that although I'm bi, I'm primarily gay. I always thought that I was primarily straight. That is something that you may have to face in the future.
Just be honest with her, and be there for her no matter what she discovers about herself in the process.