It has been my experience that people will torment anyone who dares to be even the slightest bit different. It is in their nature to conform-to completely alternate their very being to better fit what everyone else considers "normal". There is something wrong with those who don't, clearly.
That theory would make a lot more sense if they allowed those targeted to try to fit in as well, I suppose, but they never do.
When I was in grade-school, I was treated like a soggy wad of cooties. All the school was the cool-kid's table and I was not allowed to sit. I couldn't figure out what their reasoning was for teasing me so, and by middle-school I'd decided to stop asking. That is when I entered what I like to call "****-it mode" and when their attitude toward me changed. Of course I was still teased, but it was a different sort of teasing. In high-school, I was treated like some bizarre creature in a circus. They would talk about me in hushed tones. Right next to me, which made no sense. They would come over to say "hello", then rush back to their seat and excitedly exclaim that, holy cow, the weird girl talks. There was a rumor that I was a vampire and I really wish I were joking. They actually thought I was a bloodsucking demon of the night. I blame the then very popular book series "Twilight".

I share your difficulties with mathematics, though it was my teacher, not my peers, who made that fact unbearable. I can remember her yelling at me. "Why can't you get this!?" Division was especially difficult. "Edgar," She called a boy over. "Help her because we can't move on until [I]she[I] gets it." She spat the words out. I felt like complete ****. The only thing I could focus on was keeping my face down so the boy didn't see how badly I was crying. I managed to keep fairly good grades in math throughout school, though I'm really not sure how. I still have difficulty dividing and I can only multiply by single-digits. 30 * 4? Easy, 120. 47 * 28? That involves a lot more. I know 28 is divisible by 4, so 47 * 4 = 188, then 188 * 7 = 1316.
I suppose what I'm saying is: Welcome to the Loser's Table, you've clearly got more than enough experience and I elect you our President.