Editorial: Don't ditch the NCEA, fix it - fast

The Prime Minister stamped her foot this week over the school exams debacle, complaining that her minister was not alerted as soon as the Qualifications Authority knew of the wide variation in scholarship results for different subjects. She had "yet to get to the bottom" of that. The statement shows she has yet to begin to understand what has happened. It goes very much deeper than the authority's failure to trigger Government damage control. It goes deeper than a single exam, scholarship, and the way the Qualifications Authority has handled it, which seems to be the extent of the inquiry announced by the Minister of Education on Tuesday. The problem probably lies at the heart of the new examination system.

This system differs fundamentally from the old in the way it assesses students. The previous system used to unabashedly "scale" students' marks to ensure that the range of results was broadly the same in every subject and in every year. That was done to ensure that students taking easier subjects did not have an advantage, and to iron out the results of exam papers that might have been too hard or too easy, or marked too harshly or gently. The downside was that the scaled mark did not give the student a true measure of achievement but rather a ranking against the rest of the students taking the subject that year. And it gave no indication of how well the country's schools were performing. Every year the results were juggled to produce the same range.

It would be better to avoid such jiggery-pokery if possible, but the debacle of the new NCEA raises serious doubt that it is possible. In the three years of its phase-in, the NCEA has produced wide variations in results from one year to the next. The scholarship was offered for the first time last year and has produced results that varied greatly from expectations in many cases. But something similar has been noted in results of level 1 and 2 of the NCEA, which replaced the school certificate and university entrance exams, and in interim results from the first level 3 exams now available.

Whether the variation amounts to a problem might be a matter of degree. Having done away with scaling, a greater variation is to be expected, even welcomed. The Qualifications Authority admits to concerns about the scholarship but not about the standard NCEA results. Critics of the system tend to regard any variation in the success rates of different subjects as evidence of a fault in it, but that is not necessarily so. If most students find it easier to do well in, say, English than chemistry, it should not matter.

Students who want to pursue careers in science should have to meet necessary standards in that subject, not have their results inflated for the sake of comparability with entirely different subjects. And if the gatekeepers of tertiary education consider only aggregate results, the fault is theirs and not in the school-leaving exams that assess each subject in its own right.

But to do away with scaling and assess students against objective standards assumes that those who set exams and mark them can readily agree on what those standards are. Standards-based assessment is no problem when the subject involves clearly defined knowledge or skills. It is not hard for driving testers to agree on whether an applicant knows the road code or can park a car. But most academic subjects demand more than knowledge of a set of facts and some technical skills. They involve critical understanding, interpretation, comparisons, argument and clear expression. It is harder for different examiners to assess those qualities against objective standards, and the Qualifications Authority's definitions of the standards leaves much to be desired.

It is too early to ditch this whole system, which took the authority more than a decade to develop. But there may need to be much more rigorous definitions of the standards students are expected to meet. Idealists of "internal assessment" may be reluctant to impose more definite national criteria on the new system but without those it will not be palatable. The NCEA needs drastic surgery, urgently.