My daughter started with her friends that we knew well around 4 or 5. To this day, she is now 18, there is someone at my house at least 1 weekend night.
It has been a special part of her growing up for us. Most sleepovers have been at my house because I have the larger house, they have the entire upstairs with 3 bedrooms library and game room for themselves, everyone has a private bed, most other children have siblings at home, and I know what is going on at my house.
A lot of the children who stay here tell me they like stepping away from their own homes, and families for a nice break. I always make a special breakfast on the mornings of sleepovers.
We never had any trouble. I have picked my daughter up a couple of times from other places when she has notified me that something was not right and she wanted to come home. No biggie.
The way I see it ..... you can't keep them under a rock or wrapped up in bubble wrap until they are of age to let go. Letting go is in baby steps which helps you as a parent and helps the children be more independent as they grow older.