First thing I would do is find a good time, when you are both calm, relaxed and not rushed and ask him if you can talk about this. Ask him if there is something you are doing, or something going on in his life that is making him feel insecure, or that you don't care about him as much as you did? This may all have something to do with things that are going on with him, not necessarily anything to do with you.

However, I do find it ... "off" that he gets upset over your contact with your ex but he does not feel there is any problem with him having contact with his ex. Now If you are fine with that. No problem. But I think that how you both deal with ex's should be fully talked about, explored and both people's feelings taken into consideration. From what you describe, he sounds like he is rationalizing about why it's okay for him to have contact with his ex, but he feels betrayed and angry that you also still have contact with yours. A little bit of a warning bell goes off there for me.

If this turns into a serious ongoing problem, I would suggest that you both go see a marriage counselor for a few visits, or more.