The rule here for Kindergartener's is that. If an approved adult is not at the bus stop to 'accept' the child, the child is not allowed off the bus and is driven back to the school after all of the other kids have been dropped off.
When my brother was in Kindergarten, my Grandmother was asked to get him from the bus stop. The bus driver did not know who she was. Gma thought that she could just stand at the end of the driveway and brother would come to her. The bus stopped. Waited. No one got off. My brother was in the back of the bus waiving at my Grandma. Gma was frantic. We had to walk up to the school to get my brother. My Gma cried for 2 days!
I think that he just needs to learn that taking a long bus ride back to the school is okay. Sometimes parents get stuck in traffic and we miss the bus. He won't be left at school by himself. He'll be with staff and possibly other kids until you come, which from the sounds of it, was right after. One fun thing you could do is follow the bus back to school one day and then wave from the back as you go.