Both of you are probably hurting. Talk with him, without any accusations. See how open he is about taking about the issue. Then take it from there.