Oh yeah wanted to add (but had to go just then due to my 5 month old screaming at me for food), that I have been in a FWB type of relationship with an ex before.
Yes, he's using you. He wants the sex with no strings attached. but that doesn't mean you have to stop sleeping with him. I think that you owe it to yourself to eventually find someone who values more than just a casual relationship with you. Just make sure that when something better comes along that you don't pass up the chance to continue being used, because you don't believe you deserve better.
I understand not getting out much, given that I'm a stay at home mom to a 5 month old and a 3 year old. but have you tried a dating site? YOu can use one that I'm thinking of to meet friends and potential partners when you're ready. That is where I met my boyfriend of nearly 2 years now.
Meetup groups for various interests you may have are also good for connecting with those in your area.
but remember, it's quality over quantity when it comes to friends. I learned that the hard way. When you do meet new friends try not to play the "what if" game, but also don't let the desire to people please make you an easy target for use and abuse like I did. There are people IRL and online alike who can sense neediness and a craving for validation, and will take full advantage. Learned that the hard way too. There is such a thing as a healthy balance of watching your back while enjoying a relationship with a friend/partner.