My SS and SD are about 4 yrs apart. There were plenty of times when one was not invited to the party and that was just fine. We sometimes invited only the girl cousins to SD's party and vice versa. Sometimes invites are simply for the friend, and not the friend and all the sibs. In fact, it was years before we put the facts together and realized that SS and SD each had a friend from a particular family, but we never saw the sibs together. I recall one comment from Mrs. Dilley (she had sextuplets and there were a few tv specials) saying that she can't just substitute a Dilley. Not all invites were for all of them. So...I say go ahead and invite who you want.
Now, I usually invite the family if the kids are close in age, get along with my kid and the parents are good friends. However, this year I'm also pruning the list from all my friends' kids to the kids DD really hangs out with. One boy will bring Mom, Dad and baby bro, too, and I'm OK with that. But if one child or another is really your child's friend, then I would probably discuss it with the parents before sending out the invites. And, there comes a point where some kids are too old or too young for venues. So if it's OK for the 7 yr old, it might not be OK for a 4 yr old little sister. Or if it's for the 4 yr old sister, the soft play room may say that kids over 6 need to keep out. So the venue may decide you can't include older sibs (and honestly that may be one way to handle it - "Marie, DD wants her party at the Soft Play Room at the community center. However, the restriction is for children under 6. I hope you understand, but the invite is for Jenny only, not Julie." It may also be that the 7 and 8 yr olds will not be inviting any 4 and 5 yr olds to their next parties, either, so you don't have to reciprocate.
This is also a reason to address the invite to the child specifically, to ask for RSVPs and follow up on those that don't respond. "Oh, I'm sorry. We are keeping the party smaller this year and it's only for the little kids. We would love to see all of you at another time, but this party is just for Jenny. I hope you understand." Once kids get to a point where they have real friends and care, then I think it's important to consider it.
It's kind of like getting a wedding invite for Mr. and Mrs. and assuming (incorrectly) that it includes the kids.