I think it's perfecty ok to take your time and space... you have to take care of yourself! And I don't think you're being harsh tbh. You explained him your situation honestly and that's the best you can do at the moment.
So please, don't feel guilty about it ((xRavenx))