Thanks healingme4me, there are many effects explained.
The thing is, one of my female neighbors also avoid drinking coffee, and when I ask her why she doesn't drink, she say that coffee makes her nervous. I didn't wanted to go into details cause she told me that her husband had drug issues before they got children, but maybe she has the same issue as I am. Or maybe I'm just projecting...

Something tells me that coffee may have this influence on behavior (in a after-using-drugs period) cause some of the brain chemistry is still recovering and is sensitive to any changes. I don't know what chemistry exactly, but maybe the same centers that were impacted with other drugs. Just maybe...