Do you have a very high basal metabolism ? If so, that means that whatever you eat goes right through you. It sounds like it. Basically, your body is extremely good at using food to produce usable energy. So, it doesn't store anything, like it does in most people. It could be caused by hyperthyroidism. From Wikipedia,

"If there is too much thyroid hormone, every function of the body tends to speed up. Therefore, some of the symptoms of hyperthyroidism may be nervousness, irritability, increased perspiration, heart racing, hand tremors, anxiety, difficulty sleeping, thinning of the skin, fine brittle hair, and muscular weakness—especially in the upper arms and thighs. More frequent bowel movements may occur, but diarrhea is uncommon. Weight loss, sometimes significant, may occur despite a good appetite (though 10% of people with a hyperactive thyroid experience weight gain[7]), vomiting may occur, and, for women, menstrual flow may lighten and menstrual periods may occur less often.[6]"

I would consult a nutritionist, instead of trying extreme things on your own. This definitely sounds like a condition in your body and not something you're doing "wrong". You're likely gonna have to do some tests to determine the exact condition.