Let's see....."How do you know...?" Well, they just STOP! You'll know it! LOL :)
My advice would be to wake him up from his nap so he is not sleeping too much/too close to bedtime. Probably by 3, he should be up. Either get him started earlier on the nap routine or wake him up by then.
Aren't the moods in the evening just lovely when they WON'T nap even though you know darn well they COULD nap and NEED a nap?! And now YOU need a nap because of all of the "NOT napping" going on! I hated it when the getting to the nap took longer than the actual nap itself. Sometimes it's not even worth trying the nap routine.
On the days he does not nap, there is nothing wrong with starting bath/bedtime an hour or half hour earlier. He will probably sleep better because he won't be over-tired when he misses a nap. What I did when my son started to not nap as much was put him in his room for "quiet time" to play quietly for an hour. Even though they aren't napping it still gives them that time to de-stress, relax and recharge a bit.
Is he still in a crib? I hope (for your sake) that he is, becasue it gets even trickier once they out and about! (Sob..sniff..sob...it so hard to say goodbye to those naps!)
Hope this helps! Good luck!