I've been married 32 years and often think I have seen it all.
You obviously need totally out of this. This guy really sounds like he might get dangerous if this whole jealousy talk goes on.
Don't worry about losing three years with him. Be glad it wasn't 30. I've got a friend who left after 25 years. She said "I don't know why I married him. The first time he came to my house he was mean and kicked my dog." If that isn't dumb I don't know what is.