I don't think it's cold not to engage in other people's opinion of you or how you live your life.

You've described a world view that may make you unusual in regards to the things you mentioned but not, in my book, "cold".

It's possible you're cold (harsh, unfeeling, insensitive to the feelings of others)...

I'm not asking for answers, share if you like but you may ask yourself:
Do you lack feeling in general? Do you feel numb? Do you feel unconcerned when someone close to you like your wife or child is suffering? If you do something inconsiderate or hurtful wether intentional or not do you feel bad?

Most people I've known, including myself, who believe or have believed that they are alone in this world have been kicked around quit a bit by life or have a traumatic event in their past. Someone once told me I was cold because I was being self protective. I thought kinda funny coming from this person because he was a young alcoholic. I said nothing but wanted to say, " oh just wait in see how you cope once you're sober for awhile".