I do not like small talk either. Mainly because like most of you: I don't care and I do not retain mainstream facts about TV/Movies.
My other problem is that I am virtually incapable initiating conversations with people. Most of the time I end up in an uncomfortable silence.
I wish I had the capability to talk like some of my friends, they are able to talk for hours about nothing! The people involved seem really engaged in these conversations, brief or long.
I have the same issue with phone conversations no matter if it is my best friend, sister ~ whomever! I will do anything to get to the point and get off the phone asap.