Set the boundaries now when she is 4. Your Mom & you are the adults. Adults make the decisions. Teaching boundaries is so very important. You do not want your daughter to grow up & only relate with people she can control. She will always lose friends. I am a mental health counselor(13+yrs) & I see the "lack of boundaries" problems at the adult end, adults that have a very hard time relating to others. Remember that empathy is very important for your daughter to learn & develop. It sounds like you have it. Empathy is key to any relationships, especially close ones.
Define to your daughter how visits at grandma's are to be, what you expect from her end. Stick to it even if she gets angry. If she gets angry take her aside 1. ask her what she wants & listen to her, 2. tells her you understand why she is angry & resay part of what she just told you, tell her the guidelines to the visiting again & stick to it ie."if you cannot let Mommy & Grandma talk now then we will have to go home now". Very important--make sure Grandma is on the same page & goes with the plan. It's OK for Grandma to repeat the things you said. This may take a few times but it will work.
Doing this teaching will help her prepare how to behave in school with teachers & other kids.
The answer to your question is that ALL significant adults in your daughter's life are responsible in teaching her healthy & social boundaries. Your Dad could help by just saying to her "I would like you to do what your Mom & Grandma have told you to do." Be on the same plan together every time. Firm & enforced boundaries along with always acknowledging your daughter's feelings.