My daughter had an allergy to egg whites. It was a contact allergy ... anywhere the egg white touched, she would get this horrible red rash. I only found out about it when I introduced the whole egg when she was 12 months old. Anyway, I had read that kids can outgrow these kinds of things. So, around age 16 months I began to slowly introduce egg whites. (she didn't have problem if they were IN something, like muffins or what not just with scrambled eggs) I just started adding a little bit of the egg white to her egg. Now, she can eat the whole thing.
I'm not saying this is the only way to do it... but it is possible that by just introducing it slowly, mildly, a little at a time that his body will adjust and he will be ok.
That being said... milk is not an absolutely necessity. Pastuerization and homegenation change the proteins in the milk to the point where they are very harsh on our stomachs. In addition, cows milk was made for baby cows. Not baby humans. The big reason pediatricians push milk is for the fat and the vitamin D that is added back into the milk.
There are other ways to get both. The fat content is important since their brains NEED fat (good fats) to grow properly. However, these good fats can be found in fish, olive oil, and avacado just to name a few. Vitamin D can be given as a liquid supplement during the winter months. In the summer... just a few minutes outside (unprotected because sunblock stops the skins ability to absorb the sun rays needed to make the vitamin D) every day will allow him to produce the vitamin D he needs.
I'm not surprised he had problems with dairy as an infant. As I said, the milk proteins are altered in the processing and are VERY harsh on little tummys. There is a lot of good information out there on providing good nutrition without milk.
www.nourishyourkids.com is a great site for nutritional information. It is run by a nutritionist and pediatrician.
I know my daughters situation wasn't exactly like yours, but I thought our experience might help.