The accuracy and consistency of these judgments are central to the success of the national standards system.
But the report found evidence that strongly suggests they lack dependability: "It should be noted that there is no suggestion that all OTJs are inaccurate, but evidence indicates that a reasonable proportion may be".
OTJ data cannot be taken as evidence that student achievement is improving over time, the report concluded.
Reporting to parents was also flagged as needing improvement.
The clarity of how national standards are reported has been reasonably consistent, the report found, although "concerningly low" - only 40 to 50 per cent of reports were rated as clear from 2010 to 2014.
Education unions and opposition parties including Labour and the Green Party fiercely oppose national standards as unnecessary, unreliable and likely to narrow the curriculum focus.
However, the Government argues it provides valuable information to parents and shows where extra resources are needed.
Green Party education spokesperson Catherine Delahunty told the Herald the new report showed the National Government's flagship education policy had delivered minimal benefit.
"The results can't be depended on. Clearly national standards aren't working for students, parents or teachers.
"Students need a broad curriculum, rather than more testing and measuring."
Karl Le Quesne, the ministry's associate head of early learning and student achievement, said the report showed that 89 per cent of schools were using national standards to target extra help to children who needed it.
"The vast majority of teachers are doing a great job applying them. The report notes more than 80 per cent of principals were confident about the assessments their school was making. But a much lower proportion were confident about the judgments of other schools."
Le Quesne said the report contained data up to 2014, and there had been significant improvement since then.
Software called the Progress and Consistency Tool (PaCT) that helps teachers make more consistent national standards judgments was being used by 420 schools.
"Our focus is on getting more schools and teachers using PaCT to ensure their judgements are robust and reliable."