I'm going to go against the tide, I give soy milk to my 3 year old son (dairy allergy), I also have 2 friends whose children have life threatening dairy/egg allergies who have used soy with their kids with no adverse effects.
We use either Silk or Costco brand regular or vanilla soy milk, not the kids or chocolate stuff, they are much higher in sugar. Rice milk is even higher in sugar than the kids soy milk and has even less fat. I was also concerned about getting enough fat in his diet as he was underweight at 1 year (he had fallen from the 95% at birth to less than 5% at 1 year, now he's a healthy 50% for weight). We use Smart Balance light for butter, and silk soy yougert (the yo-baby brand uses milk proteins). He also eats soy cheese, I have to buy it at Vitamin Cottage, the soy cheese that is sold at the grocery stores and wal mart again uses a milk protein - it has to be labeled vegan to be dairy and egg free.
Hope this helps!