This is not a big deal, momma. You can handle it with a bit of creativity and common sense. Don't let your frustration get the best of you.
Step one, at the very top of the list, before anything else: You need to talk to his pediatrician. You need to rule out any possible physical conditions which could be causing this. If he can't FEEL that he needs to poo, it's not really his fault, right? And if he's constipated and can't push it all out, he needs some meds from the doctor to help. You have to make sure his body is working properly first and foremost.
Be sure he's eating a healthy diet and getting enough fiber (fruits, veggies, whole grains). Meats, dairy and processed grains can stop him up so he can't go comfortably. He might be having a hard time because it hurts to go. See what the doctor says.
If it isn't a physical issue, and all his pooping parts are working properly, it's a behavioral concern. Punishing him certainly is not going to help, so stop doing that now. Even if he were doing this out of spite, spankings will only worsen the behavior. I know it is frustrating, but you are not helping the situation if you show anger or lose your temper. You need to offer comfort to your child by showing that you are calm, collected, and in control. If you are calm, it's easier for him to be calm. Yelling or spanking does not model calmness to your son.
The best way to show your child that you are calm, collected, and in control is to make sure that his daily schedule is consistent. Do the same things at generally the same times in the same way....every day. Be where you say you are going to be, and don't go dragging him to a bunch of different people's houses or leaving him with different friends or family. Stick to the same sitter when you use one, and make sure they follow your rules. He needs to know that he has a bedtime that is the same each night, and on school nights he's in his own bed AT that time.
He needs to know what is expected of him. Without that knowledge he will be angry, lost, anxious, argumentative, and difficult to deal with on all levels. Schedules mean stability, and stability means comfort to a child (because they know you have it all under control)...which means fewer tantrums, outbursts, and bad habits. Don't be one of those stupid young mothers who are dragging their kids to Walmart at midnight when they should be sleeping in their own bed (and then they are slapping at the child when they cry or whine because they are tired). Be smarter than that.
You must also be sure that you are consistent in how you deal with him and the time you spend with him. Spend some extra time and focus on this issue a bit, even if you must remind him to use the toilet a few times a day. Figure out what time he usually needs to go (kids tend to be like little clocks when it comes to bowel movements) and remind him to go to the toilet 10-20 minutes prior to "the usual time." Don't take no for an answer, just make him go.
Sometimes kids will get so wrapped up in what they are doing that they don't want to stop to use the toilet, so be sure that there are no distractions at the usual time. Turn off the TV, video games, etc. and send him to the bathroom.
If he's having the same problem at school, consider the purchase of a small Timex watch. Set it so an alarm goes off each day about 10-20 minutes before the time he usually has a bowel movement so he knows that he needs to get up and go to the restroom. This might mean coordinating/discussing with his teacher, but I'm sure they'd rather deal with a little "beep beep beep" once a day than the alternative.
I wish you the very best of luck and hope some of my advice helped. As the mother of 3 boys, I've seen it all (or cleaned it off my floor).