As everyone else said, definitely don't feel bad about keeping your cat indoors. Not only is it MUCH safer for the cat (Predators, cars, evil people, other cats and the deadly diseases they can carry and spread are very dangerous) But also much safer for other wildlife:
http://www.abcbirds.org/abcprograms/.../predation.pdf

To me you sound like a great owner, I think we all get that feeling sometimes, that we just need a break from our pets no matter how great they are. Maybe try getting something he could use to entertain himself when you feel like this? When I get this feeling I give my dog a bone which keeps her quiet and happy for a while which gives me that "Break"

You could try setting down some catnip, or using some catnip spray on something (My cat LOVED catnip spray on a paper bag or in a box) hiding some treats, putting down a fun toy he can play with by himself, ect.

Hopefully this helps!