J. - it matters what type of company you are starting and who your target customers are. If you are targeting government agencies, then yes, definite advantage to woman-owned, as you will fill a set criteria for contractors. But you can still own with your husband 51/49 (so you remain majority owner).
Be forewarned that government contracts are very hard to get and take a really, really long time to procure (I work for a public agency, so I am very familiar with the procurement process) Once you do get them, though, they're pretty lucrative.
Other than that, no, there is no distinct advantage to being woman-owned.
If you're not targeting public sector for customers, I suggest you allow your husband either 50/50 or majority ownership. The continued resentment and ego bruising of NOT doing this will damage your relationship and your company and isn't worth that risk.
Good luck with your venture!