It sounds like you're both dealing with insecurity. My husband also has diabetes and it sometimes affects his sexual abilities. During those times I think he avoids contact because it reminds him that he's not able to function.

The only thing I can suggest is that you speak with him honestly about these feelings of insecurity. Let him know how it made you feel when he said he'd rather finish his game first.