No one can stop you from having the fantasy if you want it. That's the beauty of a fantasy, no one has to show up for it. However, to me, saying that she would be helping you to create a fantasy if she said she would go for coffee with you (if it were allowed) is a form of rejection. She is saying that it would never happen and that is rejection in my book. Sorry she responded to you in this way. I know how much that hurts.