Hi dear Missminnie ~ I agree with Lostinbooks ~ I think you need to talk with your partner, and just explain that perhaps your fears are unfounded, you cannot help it and ask him to please be patient with you.
And like "Lost" said, it may be a good idea to get into therapy. After awhile, this fear may grow tiresome for your partner and you don't want to drive him away. Ask your doctor to refer you to a good therapist who can help you to deal with this, and perhaps even stop these fears.
I wish you the very best. Please - let us know how you're doing from time to time, okay? We really DO care about you.

God bless and please take care. Hugs, Lee