Maybe you're shaving a bit too close, leaving a stubble that barely exits your skin properly.
I shave my stubble down to about 2~3mm so my stubble still has anough length to not be really hard and irritating...