No. You're not being too harsh or too much of a jerk. First and foremost you have to take care of yourself, and that means being responsible about who you allow to drive you to places. You're also allowing him more or less some time to get his act together and focus on himself, as he should be doing (though he might just use the time to continue to behave erratically again). So, while it may feel like you're being cold and uncaring, you're doing the right thing. Take care!