In my opinion it is not automatically rude to ignore a text, just as it is not rude to ignore a ringing phone or a ringing doorbell.

A text, a ringing phone, a ringing doorbell are all requests for interaction. One is free to decline.