Don't let him use his illness or losing his leg to guilt trip you into doing more than you're comfortable with. My mother backhandedly uses the fact that she has AIDS to keep me from holding her accountable for her actions. I think you should be honest with him and let him know that because of the way he's acted, you want to keep the relationship at a distance for now. And that if he can't respect your boundaries he can have nothing at all.

And yes the card would be a nice gesture but I would leave it at that.