The only thing that is keeping this situation from being a full-blown extramarrital affair, is that he is not reciprocating. Do yourself a favor and put an end to this. An occasional walk with the dog is one thing. A daily,ongoing, situation that entails hours of yours/his time is a horse of another color. Yes, it is your responsibility to think of her...him...and above all, yourself. Every thing that you do.. good, bad, right, wrong or purple impacts your self-esteem. Clearly, your self-esteem is very low at present, or you would not even consider such a debockle. Any movement or thoughts toward infedel, will only serve to hack away at what little esteem that you may have. Love yourself better, you deserve it.