I think it’s perfectly normal to get frustrated and annoyed and even angry at times. My pets drive my up the wall sometimes but I love them dearly. They know they are loved and I’m sure your cat knows it too.