I did not see anything in what you related that would have bothered me in my brothers' behavior (I have three). That he wanted you to go with him on a hike; you find that objectionable? My brother, at a high school football game, we were both in the same school, I went up to him to borrow some money for a drink and his date said, "Gee, Pat, I didn't know you had a sister!' and his reply was, "I don't publicize it." Your brother wants to be with you, obviously "likes" you.

He does not "get to" moan, rant or gloat," you sit around and listen? If you don't enjoy listening to him, use his "I don't care", "That doesn't affect me" or "How does this affect me?" lines and leave.

You cannot have a relationship with someone without investment. However, it does not sound like you enjoy this brother much so I'd not bother with him much, would find another friendly guy to start a relationship with like the brother you would like.