You need to live in a place with mountains. I never had any sense of direction until living where I could see which way was what by remembering, "The big mountains are to the east and little mountains to the west." Of course that may change when you go someplace else, but it does seem like the big mountains usually are east of the city.

You also have to start attending to your surroundings. I'm bad at that. I don't notice things, and that is probably why I have always had sense of direction deficits. Learning to pay attention to landmarks comes easier to some people than others, but is a skill that can be developed with effort. If there is any particular local area that you need to get to know, then get out and go for walks, ride a bicycle, and just wander around a bit. You get to know it better than when driving or riding in a car.