Postscript - Following on some others who suggested Celiancs. I wouldn't change his diet without a specialist's care. I have read that if you go off gluten on your own, the doctors won't be able to find the enzymes produced in celiac's that need to be presnet for the diagnosis. They look for enzymes in the blood, but can also look for intestinal "damage" from the gluton with a biopsy. There is certainly the chance that the blood tests could have missed it or that he has a sensitivity to gluten that is not celiac's, but you want the doctors to actually be able to test for it, and if you take away gluten on your own, they won't find it.
Haven't you seen a gastrointerologist? Something like this is not something to be dealt with by a pediatrician or a family primary care doctor.
Here is my limited experience and thoughts.
I have an 18 month old whos is 23 pounds and has fallen form 90th to 25th percentile. She is maintaining height and head circumferance, but the doctor is quite concerned. She just ran a full blood and urine panel looking at the THYROID and looking for INFECTION affecting liver and kidney function. They had been thinking thar there is something going on to prevent absorbtion of nutrients. They were in particular looking for a urinary tract infection which she said could be underlying/lingering and using up excess calories.
The workup came up fine, so now they want to work on her diet for a few weeks. And if she doesn't gain they want her to see a gastro doctor. They mentioned in particular CELIAC'S DISEASE(gluton intolerance).
A friend of mine also has a similar problem with her 2 and a half year old - he weighs 25 pounds, but has always been small. They are currently testing him for MAPLE SYRUP DISEASE which is some sort of metabolic disorder.
I wish I had some more ideas for you, but I strongly strongly suggest you get to a different doctor and get a second opinion. Your doctor is not seeing the problem and a new set of eyes is needed. And if you haven't, you must get to a specialist.
Has your son been classified as "failure to thrive"? Has he always been small or has his groth pattern changed? Does he have other developmental symptoms? Does he have (or had as an infant) digestive/GI upset? Is there any GI problem or other problem in the family? Ask yourself all these questions and write down your thoughts.
I also suggest you keep a diary of his eating, activity levels, and urine and bowel movements. Also note how he feels after he eats - both discomfort but also behavioral changes. This can help identify what foods "work" and if any don't. Not only oculd this helps a doctor with a diagnosis but it will also help you deal with anyone who makes accuusations regarding parenting. I really feel for you as this must be very scary, and I hope you will post as to the outcome. Good luck.