You need a therapist to deal with your feelings about this person.

Your mother is an adult who can make her own decisions.

If you wish to see your mother, invite her somewhere alone when you’re in town. If she isn’t going to agree to not invite him, you will not be seeing her anymore. Do not plan to stay with her when you come to town.