Sounds like a vicious cycle...
You'd fight because you don't feel valued or understood, then be fed up and tell him to leave you alone, and he will show a sliver of interest and you're right back in his arms - going over to his house to "talk it out" then neither of you wants to truly communicate your feelings and pretend everything is ok in person, then you would go home feeling empty because you have no closure whatsoever. No idea where you stand. He still pretends nothing is going on (he isn't being spiteful; he just doesn't want to stir the pot) and you have all this built up emotions inside you, you lash out at him again, and say it's enough... And you're at his house again.

Take some time and really think about how you are feeling so when you go talk to him you can be straightforward and logical, and know exactly what you want to say. Don't yell, don't cry, don't nag - simply state how you feel and give him time to respond, and leave if he needs more time. & I know it's really hard because the attraction is there, but try not to have sex with him. If you're telling him one thing but showing him something else, the man is confused. He doesn't know if his behaviors are good or bad because you're mad at him yet rewarding him with affection.

Don't ever chase a man - make him chase you.
Both of you will be happier that way.
You have the power. They will always come back... Just remember that.