I wouldn't think staying at home with a nanny all day would be any "worse" for him developmentally than staying home alone with mommy all day, as long as you have a good and competent nanny who loves him! No different than the first or only child af any stay at home mom. My first daughter and mow both my son and daughter stay at home alone with me all day nearly every day. We have never really joined any mom/baby groups because I am surounded by moms who work in our little town and all their kids are in daycare, so there really anren't any options like that nearby - and aour town is only 1400 people, and the closest one of any decent size is 45 miles away, so we don't have lots of "kiddy class" options either. That said, my kids are perfectly happy, have never really been sick much because they aren't around all the other kids and germs, and they are not socially stunted or anything.
Really, at your baby's age, just learning about their body, their environment/home and the people around them, and accomplishing their physical and mental development milestones are all the "entertainment" they need - you don't need schedules special activities or other kids, or "classes" at all. Just human interaction, daily routines and experiences, and good sleep and nutrition, and some tummy time and lots of talking/reading and singing to them.
Take them outside for walks, even bundled up in the stroller in the winter! Read lots of board books, few words, big pictures, go to the park, take baths more for fun than for cleanliness, they don't get very dirty at that point, do tummy time, sing silly songs, put the baby in a high chair to watch you cook, talk to them about everything you do and see. They will be fine!
Visit family and friends and go shopping with them in tow, talk to people, they don't have to be around other babies to be social/learn those skills, they learn from watching you! Yes, they CAN get bored, but only if they are ignored for a long time, all they need is s new toy and some interaction from an adult to be not bored, not a classroom full of toher kids and teachers.
Good Luck, if the nanny is more affordable, stick with that, almost as good as having a stay at home mom, I would say, as long as it is a great nanny!