If that is what you feel inside then you are absolutely correct in that you need to tell him. I think it is what you feel inside that is the best gauge of whether what you did was right or wrong for the relationship between you and your T.
NOTHING any of us think either way should over-ride what you feel is right for your relationship.