My sister & BIL divorced and it was bitter too. I was very close with his family at one time, even dated his younger brother. I am friendly with the x-boyfriend/younger brother's girlfriend, that's how close I am (him I could do without).
By the time the divorce was going through, grandchildren were well on the way and now there are four of them. As well, there has been a wedding. The two had to learn to get along, as there are children involved. I watched my sister avoid a lot of parties and milestones, which is only too bad for her. He was there and she chose to stay away. She has finally gotten off her behind and started coming. So yes, without choosing sides, I was there and she wasn't.
I have gotten over my hard feelings for my XBIL, which is remarkable for me, but I could really do without him as well. However, I am still friends with his brothers and sisters and my younger daughter is friends with my XBIL's daughter. I really enjoy the gatherings and am glad I chose to get over it and just know that I don't have to live with him, so my moment of liking him is minimal.
Really, your brother brought her into your life and not under the condition that you love her for as long as he does. I don't know how he can rightfully remove her from your life.