This really isn't advice, just something I learned when my daughter was 1 1/2 years old, one year after we gave up trying to breast feed. And I discovered this while watching Mr. Rogers one morning.
I had a terrible time breastfeeding, and I was getting so much conflicting information that ist was very upsetting. I pumped a lot, though I stayed home with my daughter. She was terribly frustrated with nursing. I got yelled at a lot by my husband and my know-it-all sister-in-law and mother-in-law because I was using the pump instead of nursing. But the day I came home with her from the hospital she refused to nurse for over twelve hours.. .
So we spent the next five months very frustrated. I pumped, nursed and supplemented. Lactation specialists didn't help, I was just told I had plenty of milk. I tried fenugreek, too, yuck!
Then I watched Mr. Roger's Neighborhood one day and he was discussing mammals and nursing. He showed lots of animals nursing their young, and ended with a human mother nursing her baby. She unlatched her baby and the milk kept squirting in the baby's mouth. The light bulb went on--I have no let-down reflex because that never, never happened for me. I never leaked, either. My daughter could be screaming with hunger and my breasts never reacted. My daughter had to work too hard to get the milk because it never squirted into her mouth. When I pumped, I turned the suction up has high as possible to get anything out. There are prescription drugs you can ask about to increase your milk supply, but nothing is available to initiate a let-down reflex.
I share this with you just in case you have a problem with let-down and that is why your son has trouble nursing. I commend you for all of your efforts, and you've given him a really good start with the breast milk. Kids turn out fine everyday who are on formula. My four year-old daughter is very bright and considerate; a very pleasant person and all of that comes from her upbringing, not from breastmilk versus formula. Please don't buy into any guilt trips should you decide to give up nursing. It's better for both you and your son that you be happy and relaxed and not frustrated. And women who found breastfeeding easy don't understand women who struggle with it (like my know-it-all sister-in-law).
Hope this helps.