This is something I've struggled with too - could have written every word you did. It's a really awful feeling - wanting to feel connected and like you matter to someone who matters to you mixed with guilt over taking them away from other things in their lives that "matter more than you" mixed with perhaps a small niggling doubt that you truly do matter or even should matter.

My (ex) t told me over and over again that he liked talking to me and that he wouldn't do it if he didn't. I can't say his reassurances always helped but he knew that and asked me to just try to sit with the idea that what he said was authentic and he wouldn't say something he didn't mean.