Sorry your child has been taken advantage of. This is such a painful situation. I haven't seen this approach mentioned yet:
As you describe yourself, you are indeed so blessed. I wonder whether the neighbor boy with the shabby toys and aggressive behavior is blessed, too. It's possible he's not. This is an area where I see our moral principals most severely tried – with our own neighbors.
Of course it's good and healthy to set clear boundaries. And I hope your boundaries will be consistent with whatever spiritual truths you want your children to learn. That people are more important than things? Or, love your neighbor? In the actual, messy, give-and-take of life, what does that mean? That we love our neighbors only when we like how they treat us? The Golden Rule is actually pro-active, not re-active. Forgiving, too, is pro-active. When broken down, it means "giving before," and in that sense it acknowledges that we're all capable of fault and in need of mercy just for being human.
In this case, I see loving this neighborhood child as treating him with tender respect and careful listening. This is not easy, and I haven't always done it well myself, so I'm not casting stones. But consider sitting down with him and asking him what he hoped to achieve with his trade. Asking him how he would like to be treated by others. Asking him if the trade was fair. Telling him that if your son had traded for something of such value, you would have made him give it back. And finally, requesting that the boy return the Legos. Since your son lost the bartered piece, ask him to pitch in – a quarter or a dollar – in its stead for the return of his toys. (A reasonable consequence.)
Then talk to both children together about appropriate boundaries. Be sure they hear in each other's presence the what and why of your household rules. And I love the suggestion other moms have made about setting up neighborhood trading days. It doesn't have to be trades of equal value – all sorts of considerations can come into the valuation. Including simple generosity.
Bear in mind that this boy reflects a combination of upbringing and genes, and is not entirely to blame for the way he is. If he is still very young, he's probably simply acting the way he knows how to act, based on what he sees in his family. I would guess that his parents may not be too receptive if you contact them directly, though you can certainly try. They may be just as shocked as you were by this trade. But even without their cooperation, you can still interact with their child in a way that doesn't label him.
Labels have the unfortunate effect of becoming self-fulfilling prophecies. "Bully" now, "delinquent" in high school, "criminal" as an adult. I've seen it happen too often, and it's not loving to the child or to the society of which he's a part. Care and guidance are loving.
An afterthought: the neighbor boy may go home with a story about how YOU bullied HIM. It happens. I see two ways to minimize fallout. (1) Write down your intended conversation ahead of time (this will also help you stay clear during your interaction), and add afterward his answers and anything that didn't go according to plan. Make a video or tape recording of the chat, if you have the tools. That way you are less likely to be yelled at by parents that have never learned a better way. (2) After you talk to him, tell the boy you will talk to or write a note to his parents about everything the two of you said (or invite them to hear the tape/vid). He'll be less likely to confabulate that way.