Sounds like you are putting too much pressure on him, but your problem is a fairly common one.

According to a recent survey, men are more likely than women to reject their partners. It's usually for lack of self-esteem, performance anxiety and other issues.