Your fantasizing about other guys is never about the other person; that's happening in you for your intents and purposes; no one else can "cause" that. I think you probably have trouble with his non-commitment, non-work/study and not knowing where that could put you and he in the future. I can't see being attracted to someone with no interest in anything, not working on their own life and problems? I think you need to confront the differences in your relationship (other than sex) and all the stuff you are trying to use sex and sexual feelings/lack of feelings to avoid discussing.