Hello Ballerina: I see this is your first post here on PC. So... welcome to PsychCentral!

I hope you find PC to be of benefit.
I'm sorry you find yourself to be in this most difficult situation.

I wish had some words of wisdom to offer you with regard to this. Unfortunately, short of reporting what has happened to management (which you wrote you cannot do), I don't know what other options there are for you other than to try to find another job (which you wrote you are doing.) Beyond that, I'm afraid what this may come down to is simply telling this man, in no uncertain terms, that your relationship is over & asking him to please simply allow you to do your job. From that perspective, this becomes a matter of personal boundaries.
There has been a lot of news coverage of this sort of thing recently in the U.S. And as a result, hopefully at least, there is some increased sensitivity here to these types of situations. So, at least where I live, you could perhaps go to management & expect to be supported. However I take it, from what you wrote, this may not be the case where you live. That is certainly unfortunate.
Here are links to a selection of articles, from PsychCentral's archives, that may be of some help. This looks like a lot of reading. But the articles are short. So it's not as much as it may appear:
https://blogs.psychcentral.com/psych...dium=popular17
https://blogs.psychcentral.com/women...tional-affair/
https://blogs.psychcentral.com/women...tional-affair/
https://psychcentral.com/blog/12-way...tional-affair/
https://psychcentral.com/lib/what-ar...do-i-get-some/
https://psychcentral.com/lib/the-imp...dium=popular17
https://psychcentral.com/lib/10-way-...er-boundaries/
https://psychcentral.com/blog/7-tips...aries-at-work/
https://psychcentral.com/lib/5-tips-...iveness/?all=1
https://psychcentral.com/blog/when-p...ur-boundaries/
https://blogs.psychcentral.com/imper...dium=popular17
My best wishes to you...