I don't collect them but I do like geology and rocks/stones /minerals. I especially like interesting shapes and structures (more than colors). My geology teacher once said that it's a cultural thing - people in the West tend to value colors, in traditional China and Japan they valued shapes.
Not sure if it's true but I find it interesting :-)