You presented a lot of issues here so I'll touch on a few. If you want to hear more of my point of view, send a private message.
Consistency is the key. When he starts to have his screaming fits, move him to a safe place away from you (where you can still see him or are confident he cannot hurt himself) and tell him when he is finished he can join you again. If he throws up because of it, make him help clean it up. After the fit, you can very calmly tell him you think he's losing his touch - that it wasn't nearly as good as the last fit. This tends to take the wind out of their sails as they try to figure out what that means and realize it didn't have the desired effect on mom. If he acts out in public, it's tougher but not impossible. As soon as the tantrum begins, escort him out to the car where he has to sit until the class (etc) would have been over - no movies if you have a DVD player, no radio, etc. If this happens EVERY time, you will see a change in behavior. Also, try to schedule activities at a time when he's not going to be tired, hungry, etc. Karate classes always seem to be right before dinner or right before bedtime around here!
As far as not stopping something when you tell him, I had a HUGE problem with this with my one daughter at that age. I read the book "Creative Correction" and got this useful tool: Play a game with them called Freeze/Go. When you are walking down the sidewalk or in the store, yell "Freeze!" and everybody stops. Then they proceed when you say "Go!" Practice everywhere safe - at home while dancing, brushing teeth, setting the table, playing at the park, etc. Then, when it really matters, such as when they are about to drink a bottle of bubbles (yes, my experience), when you say "Freeze!", they will! For whatever reason, it is much more effective than "stop" or "no" and then you have a moment to redirect the behavior.
As far as clinginess, I don't have much to offer. I can say it gets better over time and the more experienced they become with new situations and being left with new people. My 5 year old, although better, is still very clingy and shy. She has gone to ballet class once a week since September last year and she STILL clings to me and hides her head if any of the kids try to talk to her.
Best of luck,
Former foster mother with loads of discipline issues under my belt!