Lots of marriages come back from these kinds of things; couple's counseling would be a good idea. You should look into "Emotionally Focused Couple's Therapy" and see if this type of therapy might be a good fit for you. Feeling hopeless, insecure, unsure of what to do is different from feeling detached and ready to move on from a relationship. You also mentioned that you have both threatened to leave the relationship in one way or another over time - our brain's perceive this as a threat and it creates distance and influences behaviours in the relationship.

It's important to look at how your relationship was before the 'incident(s)' - likely you [both] have felt distant from each other in the relationship...maybe not getting what you need from the other and/or perhaps not asking for what you need (and there are lots of reasons why we don't ask for what we need). There are likely things you both brought into the relationship from your pasts that influence how you perceive the other person (e.g., seeing them as critical, cold, withdrawn, attacking etc.) and how you see yourself in the relationship (e.g., am I worthwhile? am I loveable?). Of course it'll also be important to address the incidents and how that has also effected the marriage.