Speaking as someone who works with families whose kids have all sorts of learning disabilities, let me tell you that the WORST thing you can do is push handedness in a 4yo. Some kids do not clearly establish hand dominance until as late as 9. Since handedness is a reflection of how the brain functions as it learns, if you push one side or the other, and the opposite turns out to be his dominant side, he will develop problems learning later.
You are doing ABSOLUTELY the right thing to present items perpendicular to his body, in the center. If he chooses one side one time and the other side the other time, let it go. He will eventually establish a preference for one or the other.
His dominance may show itself in other ways before he figures out his hands. You may notice that he turns his head a certain way to hear you better, or he kicks a ball better with one foot. Eyes, ears, hands, and feet are all part of dominance.
Don't let anyone tell you that he will be ruined for life if you don't get this established immediately. That's just not true. It's common for it to happen way beyond age 4. (I have not read the other posts yet, so I'm not pointing fingers at anyone. I just know some people think that.)
There are "programs" that will tell you that you've got to fix this or you'll ruin him. Not true. They are selling a product (the program). Let it go. Sounds like you have a normal 4yo to me!