Just enrich his enironment; he is 4. Potential is neurological, he is either gifted, or he is not, you cannot get him education so that he will absorb more education than he has the potential to. If you read, he sees you read, you read to him, you use an enriched vocabulary well in good gramatical structures, you challenge him to be courious and provide age appropriate expereinces, you will do everything you need to do.
Average children need good mothers just as much as gifted children and educationaly deficient, and good mothering does not produce giftedness, nor does poor mothering produce children whose intelectual status is on the left side of the bell curve. If your son is already bright, you don't really need to do all that much, and there is not much you can do to stop a child from being gifted if they already are.
Good gifted programs do not begin until 4th or 5th grade. Many children who learn to read and write very quickly and easliy will find reading and writing to learn more difficult, and will be more average in their academic persuits than many might have thought early on. This is why it is important for you to not just enrich your child's envioronment with vocabulary and instill how highly you value education, but to also be flexible enough to accept him as he is once you can make the measure that matters so much to you. By the time you get there, if he is, or is not gifted, it should not be a disapointement to either of you if he is not. Many more people are average than are gifted...and they do just fine. What you want is for him to be a happy, well educated person who lives up to his potential. Failing to live up to a mark that his mom finds too important will not help him in that regard, so I would let the gifted idea go, and just concentrate on helping him be all that he can be, instiling good values and giving him the gift of a large and rich vocabulary because you use it and you both read about it all the time.