It's tough because the law says that the exemption goes to the custodial parent and it can come down to just a matter of days. However, the custodial parent can sign the exemption over to the other parent with Form 8332. In your case, if you can agree that you're the custodial parent, then he can sign the form each year that it's suppose to be your deduction or there's a line for future years. I would recommend that you get him to sign it for "all future even-numbered years (or odd-numbered if that's what it is.) Then you keep that form and file a copy of it with your tax return in the appropriate years. If you e-file, then just hang on to the copy with your records for that year. If he claims the exemption when he's not suppose to, then you claim it as well. The IRS will send you both a letter, but you can produce the form as evidence for your side.
Everyone else seems to read it that you were married and have a divorce decree. I read it as the two of you were never married, and therefore, you don't have the benefit of a divorce decree. Either way, I would still have him sign the IRS Form 8332 signing the exemption over to you every other year. Filing first gets you the efiling and the refund first, but he can still file a paper return and claim the exemption, resulting in an IRS letter for both of you, with you having to pay back your refund with interest if you lose. Keep your calendar, meanwhile, marking off the days (and hours if necessary) that you had your child with you. The IRS puts a lot of emphasis on contemporary written records as proof. The form is the best way to reduce arguments. Call me at Gallatin Valley Tax, ###-###-####, if you have more questions.