You are not alone. I too am an infantilist (though I don't like that word) and do not consider myself a fetishist. It is a comfort thing for me.

If you have never read the article "What Is True Infantilism?" By Kathi Stringer please Google it. It really resonates with me. I read it often. It may resonate with you as well.