That would be frustrating for me as well. No one wants to see someone get frustrated with them. You really should let him know how bad it affects you.