That sounds like my son when he was 12, when he was finally diagnosed as highly functional autistic. He is now 15 and doing much better. We took him for social skills training through a counselor with other kids for quite awhile. I agree with others that you should seek a diagnosis and have a complete physical done as well. It could also be that something is making him not feel well, like a malfunctioning thyroid of something. You never know.
Have you tried rewarding him for being social at family functions? Practice with him ahead of time what you would like for him to try, like greet certain adults and then reward him with some Nintendo time afterwards. If he experienced some success at things like this, he might be willing to try it on more occasions.