Between 1800 and 3000 babies are born with FASD each year. The disorder is not curable but early diagnosis and early intervention can help children thrive.
However in parts of the country it's hard to get diagnosed, and support is patchy, according to the Human Rights Commission's report.
And it found while a widely touted action plan - for 2016-2019 - was meant to have addressed many of those issues, too little progress had been made, the Human Rights Commission report said.
A Ministry of Health spokesperson said they "recognises that more needs to be done to support individuals and whānau affected by FASD".
"This requires a cross-agency approach including from Oranga Tamariki, Corrections, Education and other agencies."
The work had focused on preventing future cases of FASD - for example with campaigns warning women about drinking in pregnancy - with not enough help for people who already have FASD, and their families.
Work across different agencies was piecemeal, with "little accountability across government for ensuring the plan is delivered".
The report also said while everyone with FASD has a disability, only the 20 per cent who have an IQ under 70 can access Disability Support Services.
Disability Rights Commissioner Paula Tesoriero and Children's Commissioner Judge Andrew Becroft said that criteria was "utterly arbitrary" and illogical.
"You only have to talk to the families / whānau to understand how unfair this situation is. FASD is a lifelong and irreversible complex condition which is disabling. It requires extensive support across a range of mechanisms, including DSS."
Successive governments had "shamefully" fallen short of their obligations to te Tiriti o Waitangi and international and domestic laws, the pair said.
"This must now be viewed as an appalling abdication of government responsibility for a significant group of children," Becroft said.
"The cost of this failure will not only be paid by these children and their whānau/families. There are also massive ongoing costs to our health, education, social welfare and criminal justice systems."
Becroft said witnessing the effects of the disorder first-hand in the Youth Court was one of the reasons he became Children's Commissioner.
Tesoriero said she was "dismayed" that such an urgent issue had been so low priority for successive governments.
$35 million is being rolled out four years to expand and enhance the capacity of Ministry of Health-funded Disability Support Services' Child Development Services.
Alongside this, $7 million is being rolled out over four years for New Pregnancy and Parenting Services in eastern Bay of Plenty and Whanganui.
Recommendations from the Fetal Alcohol Spectrum Disorder: A Call to Action report
• A potential select committee or other type of inquiry into the issues raised
• Immediately asking Cabinet to change the eligibility criteria for disability support services so everyone with FASD is eligible
• Ensure the 2016-2019 FASD Action Plan is enacted, especially around support for people with FASD and their families
• Establish a ministerial roundtable to fix the issues and ensure a "cross-agency focus" on FASD
• Conduct an FASD Prevalence study in New Zealand