Hello! My answer is going to come from my personal experience, not an experience with my son. (who is also 4!) I was enrolled early - and excelled in the gifted programs all the way until junior high when they pretty much dissipated in favor of college preparatory classes, which I also took. There were never any problems with my academic work, I had no trouble keeping up, or even excelling at a level higher than my current grade.
However, I did play varsity sports (cross country and track) beginning in 9th grade. When I started 9th grade, I was only 13. I turned 14 during the year, but spent the vast majority of my time with my team, which had people up to age 19. It's awkward to be in a situation where you are SO much younger than your peers. In every way, I was an equal, athletic ability, academic standing as well as socialization. The biggest problem was my age. My parents were very uncomfortable with my spending time with older teenage guys and girls. (mostly guys!)
Looking back in hindsight, they were right. I realize now, that I really had no business as a 13 y/o girl riding in cars with 16-19 y/o guys! My parents never really considered the effects of an early placement once I got past the middle school years.
Also, upon graduation, I moved into a dorm room at age 17. I was 9 hours away from home, and wasn't even a legal adult. Strange really....
Personally, I advise everyone I know whom are in a similar situation, to just wait. The rush we have to enroll our kids in school is usually unnecessary. If he is academically gifted, do significant research and find a pre-K program that has an actual curriculum.
My son is 4, and turns 5 in December. He is clearly a few months past the cut-off date, but I know for a fact would test very high on the placement tests. He can write all the uppercase and lowercase letters and numbers up to 10. He is very, very bright, with an understanding of life that I rarely see in young children. He is compassionate, diligent and eager to please. Socially he is more than ready - and physically is stronger and more agile that a lot of the other boys in his preschool... I know that he would succeed in kindergarten and then beyond, but have no desire to place him in the same awkward situation when he gets to high school/college that I was in.
I hope this helps, good luck with your little man - and what ever decision you make!!