I think this is the case with a lot of people who have gender dysphoria. It gets worse with age as your secondary birth gender characteristics become more prominent (or pronounced).

There may be also be other reasons for you to feel like this (not related to gender dysphoria), but only a therapist can help you know for sure.