It seems like perhaps this would be a good situation to see a counselor or therapist about... not so much to delve into your past, etc. but more to help you sort this whole situation out & decide what to do. It sounds like there's more to this than just the issue of this man's lifestyle (although that's certainly a biggie!) A women's support group might also be beneficial if you could locate one. Since you apparently live in or near L.A., there must be resources available.