It IS frozen water, and is healthier than sugary treats, but because there's no sugar and other ingredients will be harder than regular popsicles which have a somewhat lighter frozen consistency. Chewing ice is hard on the teeth, so I'd be concerned with children crunching on it. One of my sisters was even told not to eat ice in any form because of all the problems she had before and after wearing braces, too hard on her gums and teeth..
I've seen ice shavers to use for shaving ice for snow cones at Target, they may even be on clearance right now. Might be better for them to eat shaved ice rather than eating ice cubes, would melt faster in their mouths so less chance of them biting or chewing on it.
As a side note, my mom craved ice when she was pregnant with one of my brothers, and it was due to a mineral deficiency, like women who eat dirt during pregnancies. So make sure your girls are getting plenty of water to drink...and maybe hide the popsicle molds ; )