First off, welcome!
When you fight, what do you fight about? This would be helpful in knowing ... to see what you're figthing about and maybe give some help with it.
Do you mean by "not trying" that he doesn't do exactly what he says?
When he comes to your place eventually, I assume he tells you where he's been...sooo...he's not hiding anything?
Maybe a little more information would help...