I don't think it is good for your self esteem if you are just letting him drop by when he has moved out. You have the right to set some boundaries and I think you should.
Let him know you are there for him when he is ready to discuss what is bothering him. Don't let him get away with saying he can't talk about his feelings, that is bs. What kind of a marriage long term will you have if he can't talk about things? Do you want to be married to someone who refuses to try to communicate, you'll always be trying to guess whats going on.
Also, I would stop chasing him, men like the chase, let him come to you. Let him miss you.
Text him you love him and say you are respecting his need for space, and you will be ready to discuss what is bothering him when he is ready. Then ignore him until he comes to you. Don't be a doormat.