K.G.
First of all, I would suggest this...think of how you would feel if he had custody of your son and he told you the things that you told him your son said.
He probably thinks you made it up to be nasty to him. This has to be a relationship between your son and his dad... not between you and his dad. If your son won't speak up for himself, then you have to just let him have time to be able to do that.
It could be that he feels he has to say these things to you about his dad because he knows how you feel about his dad. Deep down he may really want a relationship with his dad and he probably loves him very much.
I realize you may not like your ex, there is probably a lot of hurt involved however, he is still your son's father and he should be allowed to have a relationship with him. If later in life your son decides not to have anything to do with him, thats fine, but it has to be his decision. If you keep him from his dad, he will know it and could hold it against you when he gets older.
It's important for kids to figure this out on their own, don't try to force your opinion of his dad on him, he will know that you did that.
I've been in your situation and believe me, you don't want this coming back to bite ya in the butt.
Let your ex know that he can call or visit, but visits must be scheduled. Calls should be allowed whenever he wants to talk to his son... done so at reasonable times of course. If he has questions or concerns about your son, let him ask you and answer him cordially, but when you are done telling him the updates, hand the phone off to your son.
There is no reason, short of an abusive relationship, to be enemies with your ex. You don't have to be best of friends, but you don't have to hate each other either.... remember, you once loved each other at least some or you wouldn't have been together and had a child in the first place.
When I was a single mom, I was careful not to talk badly about my ex in front of or around my kids and I didn't let anyone else do that either. It about killed me because my younger son always thought my ex was a great dad... even though he never called or came around... but as he got older, he learned on his own, with no influence from me. My (now) husband adopted my boys and has been the only father they have really known. They do see my ex once in a while, as we are still close with his family and they do have a brother and sister from him.... but, when you ask who their dad is, they say my husband's name... and if anyone says my ex is their dad, they correct the person and say, he is the bio-dad, but that is all... their real dad is my husband.
I know it's hard to deal with, but you have to take a step back from the hurt and the hate and remember, if you were in your ex's place and he had your son, how would you want to be treated?