You must take steps to fix this as quickly as possible. In 3rd grade my mom went through the same thing with me. The teachers set up an IEP with my mother to try and get me excelled a grade level for the next school year or to try and get me on an accelerated learning program. She told me no. I didn't need that because she didn't want my brother and sister to hate me for not having a learning disability like they have. By the end of the next school year I quit caring about school. I went from straight A's to straight F's. Imagine how miserable it is when you're able to complete all of your work before the class. Homework is done by the end of the day. You've read a book that was supposed to take a full semester in one night. Then you're always in trouble due to getting antsy just sitting with nothing to do or trying to talk to friends or for one thing or another. I finally just gave up. I wouldn't even do my work in class. I barely squeezed out a C average until 10th grade when I finally had enough and quit school and simply got my GED. I wound up missing only 5 questions total out of the entire thing.
I look back now and I'm discouraged at the fact I didn't get my diploma. I'm disappointed in my mother for not helping me to excel where I could have. Don't let your son feel that way to you. Get him the help he needs and deserves. Doing a search, I noticed there is a gifted and talented program class in your school district. Talk to your son about such a program. Then set up a conference with the teacher. The teachers cannot refuse a conference when a parent needs it so make sure to push and get it done for him if they are hesitant to set up one outside the normal times they have a conference. He deserves this kind of a program. Having cousins who were in a GT program in another town in TX, I know they will test him to see if he really does qualify for it.
Help him before he won't let anyone do it.