One time I was really hypomanic/manic, and a little tiny bit of caffeine sent me shooting through the roof. Most other times, I can drink it like no tomorrow. Out of superstition I used to drink lots of coffee thinking as long as I could stomach it, I wasn't (hypo)manic but I have seen caffeine sensitivity come and go with me. Does nothing on me when I'm super low, does more when I'm elevated, and just randomly moves around the extent of effect it has. Can't give it up though...

I'm no expert, but I think it's good that you recognize something's a little off. That's all that's needed to try to start to work towards feeling more like yourself. No need to be embarrassed (at least I tell myself).