I said "no" because, as others have said, it isn't always possible to accommodate everyone.

I had a wheelchair bound person apply for a job I had posted once. She submitted two pages of needs she had with her resume. There was no way I could remodel my whole office building to suit her needs, so I chose someone else. I got sued. I won in the end, but it was crushing to me, not to mention expensive.