Not really (a scumbag). You weren't prepared. But now he has rung so there's a chance he might ring again today or tomorrow; what will you do if he does? I'd make a plan. Would it be easier for you to accidentally open your door (looking for a newspaper or something :-) when he's going out and wave, maybe call something like, "Did you ring/knock the other day? I wasn't able to come to the door but thought I saw you walking away; did you need something?" Or just open your door and wave, see if he comes over to ask you something then. But you'd be in your doorway, in control.