Hi
I had the problem as you are describing once. I loved porn and my g/f went nuts over it. We set off for relationship counciling to get it resolved. To her surprise, my g/f was who the women centered on and not me. The advise to her was to stop focusing on it as a problem and instead see it as an opportunity to learn more about each other. Jay, my g/f did exactly that. She sat with me and we looked at the porn together. She pointed out the stuff she hated, the funny stuff, the yukky but interesting stuff and on and on. Our sex life went through the roof and our relationship improved incredibly. I no longer viewed her as a stuck up prude old cow and began to see her as funny, intelligent and sensitive. She began to understand that guys get off on what they see as opposed to what they feel. It was great! Try it. As for the guy trying to take away your self esteem. I think that you are giving it up rather than having it taken from you.
Just some thoughts. Hope it helps