Structure is helpful - and that should include some free play and some quiet time so you keep your sanity.
You've already discovered the library - they probably have lots of books on activities and crafts that use things you can find at little or no cost. And it's not too early to teach your kids that they can take out a book, read it, and return it. It also teaches them to take care of books that aren't their own.
There are a lot of things you can pull out of the recycling bin, for example. Kids also like to collect rocks, which can be washed and painted and then used as paperweights or porch/garden decorations & borders. Nice gifts for Grandma too, if they'll give them away!
Inexpensive wood frames from the craft store (or made from popsicle sticks) can be painted or decorated with macaroni shapes and then spray painted by you. Local newspapers will often give away the end rolls of their newsprint (rolls that don't have enough on them for another printing) or sell them for $2 - that's a ton of paper for crafts and artwork. My stepdaughter makes giant flowers and dinosaurs or seasonal decorations that go up over the windows. The local quick printer might give you the leftovers from jobs that didn't print the color right - use the back side for art. Ask the local teachers where they get their supplies - you'd be surprised what they get from local companies.
Libraries often have free or discount passes for the children's museum and the aquarium - short visits to museums are fun for kids and, if you use a pass, you don't feel badly if they get tired after 2 hours. They also often have play areas or puppet areas that are available when there is no organized activity.
Cooking together can be fun. Kids can sort socks and stack washcloths too, making laundry a family affair.
Check out yard sales for low-cost toys and crayons. And have a yard sale to get rid of stuff your kids don't need anymore.
Don't just rely on sets of toys - use kitchen things in the bathtub, use old blankets to make forts in the living room, etc. Make flower pots out of yogurt containers. Make bird feeders out of pine cones - make sure they aren't covered with sap. Use dryer lint in a mesh bag to provide nesting materials for birds. Collect sticks to make words that you put on the wall (kids' names, the word "family", etc.). Make front door decorations out of seasonal items. Make musical instruments out of margarine containers and dried beans. Make flower vases out of paper towel rolls covered with stickers or sticky paper (a factory near us that makes contact-type paper gives away its irregulars). Put crepe paper flowers on pipe cleaners and put in the vase. Go to the local senior center and donate them for their lunch tables. Make puppets out of single socks. You can buy pom poms and goggly eyes and felt at the craft store or fabric store, and a lot of books in the library give you instructions.
Have set times/days if it helps you - Mondays might be an at home day since places can be crowded on the Monday holidays when school is out, Tuesday could be art day and water play day, etc. You'll have to vary it based on weather, but you get the idea. Have a special activity, and then have some consistent activities every day (snack, lunch, nap/alone time, bath, story).
Go to the firehouse. Unless there's a fire, it's a quiet place. In our town, the firefighters let the kids sit in the truck and they also do a little fire safety with them. Not every day, but maybe every 6 weeks or so. Your kids can make something to take to the firefighters too - cookies, decorations, etc.
Find out if your town has nature areas for hikes, or if local agencies do - Audubon is one group that maintains them. There are tons of free resources if you investigate.
