I think the best place to work this out is in therapy. You just don't have to include him as part of your jealousy issues. If at some point you become comfortable then you can tell him. Best wishes.