My daughter (now 10 yo) is ODD. ODD is normally a component of ADD or ADHD. HOwever, some children can have the ODD without the others or the ODD is so bad that their minor ADD charecteristics are not noticible (my child).
Some of the characteristic you described are very much ODD. However it is best to have a competent psychologist/pshychiatrist diagnose him.
As for his behavior having "something to do within their family unit". You are wrong. If he is ODD the fact that his behavior is fine when he is with others is also part of it. And he will be. He understands these are places to visit and while he may have been there before, ha hasn't become comfortable enough with the surroundings or the people to begin acting out there.
I had a hard time convincing family members and friends that my daughter's acting out was more violent and out of the ordinary for a normal child as they never saw it for themselves. It made me feel isolated, alone and as though I were doing something wrong. The fact that my child has often been highly praised for her manners at other homes is proof that this is not the case.
ODD is caused by a hormonal imbalance in the brain. It can be controled with medications. I sought thereapy for my daughter in Kindergarted as that is when she began acting out to where it was noticed outside the home, and put off medication until she had progressed to other charecteristics that made me concerned for my much younger twins safety (violent, explosive tantrums) and shoplifting (3 years later).
The best thing you can do is offer as much emotional support to your nephew's family and convince his mother to seek a therapist/counselor/phsycologist whatever for her family and her son to help them cope with this very difficult disorder.
I recommend putting off meds as long as possible, to see if his ODD responds enough to simple counseling.
Good Luck!!! ;-)