Meh. Go or not go. Such a personal thing. I am with Pam on this one. Enough money is such a nebulous thing and with kids it's a sliding scale at best. If I decided to go and for whatever reason on the back end at a later date, money became tight I wouldn't have the room to complain.
My bigger issue was the idea of missing out on travel experiences because of kids (or fill in the blank reason). Life can always be constructed in terms of coulda shoulda woulda. Shrug. I did what I did before marriage and again before kids not because of those things but because that was where my life was at in those points in time. I could have done more or made different choices or... But I didn't. Now I do what I do with my family and it is satisfying because I choose to say it is such.
Good luck whatever you decide.