Who tells you you need to work on your self-esteem and why/in what context?
I know I have low self-esteem, how it manifests itself is that I often feel I can't do things/tasks well enough as a consequence I've tended to avoid them which causes a spiral effect as it confirms my poor opinion of myself. I'm following CBT at the moment which in practice means I have to do things I fear without allowing doubts to creep in - it's actually working very well.
It's different to knowing strengths & weaknesses, with low self-esteem the tendency is to feel that it is *you* that you are deficient in some way, rather than having a balanced picture of strengths & weaknesses.
Hope this helps.