I was having similar issues with my last pdoc. He was in the office 4 hours a week, when he was even in the country.

Just last week I saw a new pdoc who seems awesome.

Ask if there is anyone else in pdoc's office (if it's a group practice) that you could see instead, and specifically tell them you need someone who is in the office more frequently.

If they can't refer you to someone, if you have insurance, check what providers they cover and try a new one