A few questions - how often is he eating, for how long, and does he seem to refuse to eat about the same time every day? Like in the afternoon or early evening? Some babies just tend to get fussy at a certain time each day - think of it as like the end of a really stressful day and you're exhausted - babies get the same way sometimes. He might not be hungry, might just need some comfort or relaxation or maybe just a distraction.

Does he get enough wet and poopy diapers? 6-8 wet diapers a day - that assures you that he's not dehydrated. (no, you don't have to count them...but are you filling up a diaper genie like once a week or so?)

Is he meeting milestones? Growing at a steady rate - not a huge drop-off on the curve?

If you know he's meeting milestones and not dehydrated, then he is getting enough to eat. Try not to let it bug you.....easier said than done, I know...but try.