The hospital nurses are correct. No need to take a child to ER unless the fever is over 104. You can ask your pediatrician if that will help you feel more confident.
Fevers are a good thing. The body heat kills bacteria and viruses that are making the baby sick. Only give Tylenol for discomfort or if the fever is high. Ask your pediatrician when to give Tylenol.
Also antibiotics are only given when one is ill with a bacteria based infection. Antibiotics do not work on viruses and most of our illnesses are viral in nature. Also antibiotics do not directly affect a fever. The fever goes down when the bacteria are killed.