Not a stupid question at all! Obviously something IS bothering you or you wouldn't be responding the way you are. We all have areas of "sensitivity" based on our past.
I'm not saying you're "going off the deep end" by any means, but I'm just wondering if you happen to be in therapy. I'm asking because T's can help people get an understanding about why they are responding the way they are and if their responses are appropriate.
We all do need to set boundaries, as you seem to be suggesting, so we won't allow people to run over us. Is the person not listening to your desires? Not willing to compromise, but insisting on his/her way only and all the time? Somehow not meeting the needs you feel are appropriate? I understand that you are not conscious of anything he/she is doing "wrong," but something is going on. I even see your "anger" mood.
The questions I asked are general "boundary-breaking" concerns. When you can get a handle on WHY you are upset, then you can talk about them to the person involved.
I wish I could help more, but that's all I can think to say not knowing any other details.