You can still potty train with a speech delay, as long as he is able to indicate to you that he has to go. Every child gets to this point at a different time and through many different manners. My oldest son has Down syndrome and was not fully potty trained until 5 years old. Your little guy will get there before then I'm sure.
Here are a few suggestions you can try: If you aren't too shy, you or your husband can sit on the big toilet while he sits on the little toilet. During that time, you can talk about going poo and what you are doing. Kids love to emulate their parents.
You can also try getting a seat that you can put on the big potty so he can sit on the big toilet. If he sees you sit there maybe he'll want to do the same.
Reward him when he sits on the potty and does something in it. My girlfriend used M&Ms for each successful potty experience. Only one per pee/poo!
If he's not ready though. Don't feel compelled to force him. He will get it eventually. The main things he must be able to do for potty training are:
1. Be able to pull down/up his pants
2. Sit for a period of time
3. Have an awareness of when he has to pee or poo
4. Some level of control (be able to hold it until you get to the potty)
Those are the major ones. His awareness and control levels may go up and down at this age. Keep the faith. He will get it and you will look back fondly at this time on what you had to go through to get him there.
Regarding your mommy guilt, it's a balancing act. And sometimes the scales don't tip in our favor. Be consistent in your love and your quality time with the kids. There is never enough time and it gets worse as they get older. Through it all, they will know you love them and will always be there for them. Good luck!