We too have gone (are going) through this. I built a cabinet in my house, it's pretty and decorative (you could use anything though) and it holds ANYTHING that can be used to write, draw, paint, be stick on (we have a sticker problem), glued or other wise permanently affixed to anything. Then I cleaned everything, and sorted out their rooms, and organized everything. After that its up to them. We stop and clean up everything 3 times a day, they have 10 minutes each time and what ever is on the floor after the timer I throw away. I know it sounds harsh but the first few times they actually watch their stuff get thrown away they care.
Now for the destroying part, well anythings they break gets thrown away and NEVER replaced if it is theirs, and if it is mine well then they have to "work it off" so to speak. Or I take away a favored something for a few days or a whole week if needed. My kids are in the same age rang as yours 3 and 5 and this seems to work.
Also I have learned that anything that is really important (like the game you wanted to play) needs to be taken and put in the cabinet till it's time, I forgot all the games are also in my lock cabinet.
I hope this helps give you some ideas, all families are different and function different, but this is what works for us. And don't worry if you know you are spending the time needed with your kids don't let other make you feel bad. LOL, all I wand was to get my 4 breakfast dishes done and in that 5 minute make my daughter climbed up and got down the scissors out of my purse (she had seen me put them in there to take with me that day) and cut off all her hair. Kids work fast, my bottom line theory is nothing in the rooms that can be broken or used for harm, and everything else is in closets or cabinets where it has to be asked for.
Sorry for all the long windedness, I'm not usually so bad.