I'm not Jewish, but we are actually going to a bar mitzvah this weeknend, so here is my two cents'. I was very happy to be invited, but it would never have OCCURRED to me to be offended if we had not been.
I definitely would not feel 'ripped off' and you absolutely do not have to take a 'Jewish' trip or something to off-set having a smaller party! If you would like to let people know about your son's charity project, that's fine, but I don't think you have to at all.
My understanding is that this is a special ceremony which is about religion and family more than anything else.
Of course wanting to include your wider community of friends, neighbors, co-workers, etc. is perfectly natural, whether for a bar mitzvah or a quincenara or graduation. But I am sure no one would want the occasion to be anything but joyous for you.
If having a big to-do is too much of a strain financially or otherwise, you will only be worried and that should not be what the day is about. What you talked about above sounds lovely, appropriate- and if it is what you want to do, then that is what you should do! End of story. Giant mega-weddings and bar mitzvah's and parties are all so early 90s now anyway! This is about your son and God first- don't worry about the party or the co-workers! Congratulations and good luck!