I can only say that personally, it wouldn't bother me if the person I'm with fantasized about someone else. After all, that's not all that makes up a relationship or even a sexual relationship (not that I have any experience with that but still, just my thoughts :P)
You say you don't want to actually act on these fantasies and you don't truly feel anything for this other person, so was there really any harm done? I think it shows you're a good person for thinking about this and feeling guilty but I also think it's not necessary. Maybe it would help to tell your girlfriend about it? Since I don't know either of you I can't say if that's something that would go over well but it could be something to consider