I love my ergo and use it for my daughter who is almost 4 as well as my son who is a little over 2. It is fine to wear all day, but you will get a sweaty back :)
They can sleep in it just fine with no adjustment which I love. My daughter walks most of the time and my son rides in the ergo, but when I took them to DisneyLand it was really nice (once they fell asleep after the fireworks) to be able to carry my daughter in the ergo on my back and then my arms are free to carry my son. It is a lot of weight but it isn't so bad because the ergo supports the weight so well.