I haven't read the other responses...You sound very clear and reasonable about the dilemma and I am truly sorry that this situation isn't working out. You both sound like lovely people who have given this childcare scenario a fair shot.
What about a compromise? Is Grandma capable of getting the twins to the sitter say before or after lunch? She could have a little morning time, then pack them up and drive them to the sitters? But only if you feel she can handle the packing up and driving them over and dropping them off. If the sitter starts to report that they haven't arrived by an agreed upon time, that would be an indication that she's not organized enough.
I hear only true concern in your voice for all parties and am disappointed that your husband goes as far to call you names over this babysitting dilemma. He has got some serious denial going on, as your thoughts are clearly compassionate towards all.
Why do you think he cannot see what is happening? If he continues down this road of denial, you will have to have a heart to heart with Gram's without him.
I would also recommend that you layout meals and snacks before hand for Gram's so that she isn't watching kids and figuring out what to feed them. That will ease her load. Prepare anything ahead of time that will ease her time.
If you find that even prepping the home and food for her ahead of time, still leaves her overwhelmed and your babies are still in said condition, then you will have to step up and be the 'bad guy' in your relationship and change the daycare arrangements.