I'm not sure his ex-girlfriend is the most reliable source. You must consider her motives for causing stress in your relationship.

If he does not have active HSV-1 (usually cold sores around the mouth) or HSV-2 (genital type) there is a small chance that you could still contract HSV. This would be through "viral shedding" in which the virus sheds itself at the skin surface. This is estimated to only occur for 5% of the year.

The best thing to do is get tested. there is treatment available.