I actually think that I once lived near a tiny illegal Asian brothel in the Bay Area. It was an apartment with blooming flowers in pots outside, very clean, but nobody really stayed overnight - instead, different women appeared at different times, and there were trucks parked during lunch time. I looked up the address online and it popped up in some sort of a SF directory of seedy massage parlors or other some such euphemisms for brothels.

The ladies swept the little yard in front of the apartment very carefully, watered the flowers, were friendly, wore very simple unassuming clothes, and in general would not be recognized as prostitutes the way typical harlots are pictured.