I have to confess that I had to read about polyamory after seeing your initial post. I guess that makes me a bit of an out of the loop middle-aged female coot.
I feel for you needing more affection from an unaffectionate spouse. Though my husband is affectionate, our sexual life together has long since disappeared due to his erectile dysfunction and unwillingness to do alternatives. Maybe if I was older it wouldn't be such an issue, but I'm 13 years his junior.
Are there valuable things you still get from your relationship with your husband? Beyond just financial and simple companionship? I do hope you can get what is missing elsewhere, if it is crucial to your happiness.
I don't have official "second relationships", but I do have a major emotional attachment to another man. It is a forbidden type, but it never crosses a line that makes it damaging to anyone. I don't think. There is no physical affection involved, but there is emotional affection, and likely on both ends. I get the latter from my husband, but extra helps make up for the deficit in other ways.
I actually do get plenty of platonic physical affection. A few men downtown hug me every time I see them. My husband knows all of this and accepts it without any feelings of being threatened.
Once my husband, in an emotional state, said I could go further in seeking sex outside of marriage. I don't really want to, though. Not that I don't miss it, but my love and connection with my hubby makes such a pursuit unappealing.
How do you really feel about your husband? I don't expect an answer.