Yes, L., call her. Have your husband on one line and you on the other. Talk about your memories, things you've done together, what you enjoyed about and with her. If you want, make a list of them to refer to so you don't get tongue-tied.
Ask her how she feels about the hospice people. Has she met them? Let her talk. Tell her about the upcoming birth and the baby's upcoming name. Tell her how proud you are that she fought the fight so well.
Remember this - it might be awkward for you and your husband, but it's what she NEEDS. Loving words from family and friends to remind her that she mattered to people. That she made a difference in people's lives. That she truly isn't alone as she passes to the next life. I've been privileged to have had several conversations like this, though not as many as I would have liked. I just lost a friend (mentor, former business partner) who I talk to once a year, and didn't get around to calling this summer yet. The last time we talked was last summer. I'm so disappointed in myself that I missed that chance to have our yearly talk - his death was quick and all at once. But my aunt and I talked on the phone while she was in the hospital. I was the first person in the family who she opened up to saying that she was going to go ahead and accept hospice. Before that, she wouldn't admit that she had cancer. We were all worried about that. The next day, she died. We felt like her being able to really talk about things let her go in peace.
So don't feel like it might be the wrong thing to do. It isn't. Just don't end the phone call seeming like it's the last time you'll talk.
My best to you and your aunt.