Do you understand what the 'arythmias' are? He's saying that your heart is beating somewhat irregularly, and therefore some blood pools in one chamber of the heart ...when that happens, it can start to clot. They are concerned that the clot will then get pumped out of your heart and travel somewhere in your body to create a blockage: in your brain , that would be a stoke, if it's in a leg, it is deep vein thrombosis, if it goes to the uterus, it could hurt the baby.
It seems to me that your doctors have done exactly as they ought: if you have a cardiologist, then you've had heart problems before, and a high risk group will have dealt with these problems before, rather than being with a 'regular' OB who hasn't had the experience. In medical matters, almost the most important variable is the doctor's experience with difficulties: if they have done thousands of procedures with very few complications, then they are a far better bet than the doctor who sees only one of such patients per month.
I'm not a doctor, I just read a lot, and had my own issues with high blood pressure during my pregnancies, but it seems to me the best thing you can do is to get the doctors at your 'high risk group' to give you an appointment just for talking, or to ask for extra time for your next appointment. Tell them that you aren't trying to run things, but you need to understand. Ask about the risks, ask about the side effects, ask about danger signs and instructions of what to do 'in case'.Get them to tell you the actual percentages of risk: I often thought early on that while the dangers I was facing were catastrophic, they couldn't be that likely....and that I'd let them convince me to have a c section unnecessarily. I later understood, that, no, it wasn't unnecessary: the risk in my case *was* too high. Knowing that has made me much more content with the decisions I made.