I think you two need to sit down and have a conversation about your future and where you both see yourselves in a years time. Living together or seeing each other every few days. It sounds like you’ll have a lot of work to do in this relationship before y guys move in together though because the argument will only escalate if you don’t deal with the root cause now.
Lastly, you need to initiate contact and communication as much as he does. It’s not entirely up to him to contact you and nor is it entirely up to you either.