My daughter and grandson are onlies, and while she wished for a couple of years that she had a sister, mostly she's been very happy being an only child. My grandson is now 5, and is quite clear that he prefers being an only. Also, I'm the oldest of four, and since reaching adulthood, we have very little contact with each other. I have many difficult childhood memories of interactions with my sisters, and few nice ones. I will probably end up with all the care of my aging mother, with little or no support from my sisters.
As many wonderful reasons as there are to have more children (and your impulse is certainly loving), there are equally compelling reasons not to. I chose to stop with one and make the most of that experience. It was/is wonderful, and I have no regrets. My daughter has stopped with one, and it is wonderful, and she doubts she'll have any regrets.
It's a strong human quirk to wish for what we don't have. I've known quite a few onlies who sometimes think they'd like siblings, and even more siblings who sometimes wished they were onlies, or wished they had different siblings than the ones they were saddled with. But it's also true that we don't really miss what we never had. We might desire it, but we don't miss it.
There's also the common concern about spoiled, entitled onlies. But onlies grow up to be thoughtful, appreciative, well-mannered adults if that's the way they're raised. Kids in big families grow up to be demanding, spoiled adults if that's the way they're raised.
The pluses of raising an only are big ones: the parents don't have to split whatever resources they have (time, energy, material, emotional) between kids. And, the earth's resources (space, climate, energy, clean water and air) aren't burdened more than they are already (the population curve is rising so sharply now, it's a line almost straight up). And social resources aren't crowded harder than they are already (schools, transportation/traffic, health and social services, etc.).
I'd say, if you don't feel eager to expand your family, don't try to maneuver yourself into taking that on. We get to choose like-minded sisters and brothers when we grow up, if that appeals to us. I have many.