The school can decide not to hold back, but in some school districts you can still sign a parent waiver that will force the school district to hold the child back. You can only use the waiver one time. If you later feel like your child is not able to move forward again, you will not be able to block it.
Before you decide to hold your child back, make sure it is really something you want to do.
It sounds like work wise, your Son is doing fine and he is in the appropriate grade level.
Your reasons for holding him back seem more to do with the social. By holding him back you can create bigger social problems than he already has with his peers now. You need to make sure you include your son, and let him voice how he feels. You really need to talk over and consider the different scenarios of social ramifications that can come from this decision. Once done it is not something that can be undone without consequences.
One suggestion, if it is a social,age and confidence reason you are considering holding your child back.
Look outside of school for activities your child can do. If it is an activity your child is good at, he can excel here and get the confidence he needs, and outside of school, in those activities find a good group of friends. If I am not mistaken, outside activities usually would go by actual age not grade as well.
You have a very hard decision on your plate. Best wishes for it all to get worked out for you.