Universities' gradings stun adviser

By SIMON COLLINS

A key adviser to the body that rates tertiary research is dismayed at how poorly New Zealand universities compare with British ones under a new grading system.

Professor Jonathan Boston, a Victoria University political scientist who helped to shape the system - the new performance-based research fund - was reported in a High Court judgment yesterday as having told the universities' research committee that raw data on research quality in the two countries "would look very bad for New Zealand".

The Tertiary Education Commission (TEC) subsequently turned the crude raw comparisons into a more sophisticated form - one which saw New Zealand institutions compare "moderately well".

But even the moderate version was too much for two universities, which yesterday succeeded in having an appendix which contained that comparison in a major TEC report suppressed.

High Court Justice Hugh Williams found that the TEC breached natural justice and the universities' "legitimate expectations" by proposing to publish the more "sophisticated" comparison last month without adequately consulting them first.

The commission will now rewrite its report to take out the British comparison. It will notify tertiary institutions of their research gradings this month and release the results publicly soon after.

It will also consult the institutions about the international comparison. The judge ruled that the commission should give the tertiary institutes "full opportunity either to limit the use to which the [institutes'] data could be put, or to dissuade TEC from using the data in that way".

But he also warned: "The universities need to recognise that one possible consequence is that, after the [institutes] have had the opportunity of appropriate consultation and input, TEC may still publish an international comparison either in its present or a modified form."

He found that the commission did not act irrationally or in bad faith and that, although the comparison with Britain might be "difficult, even flawed", it was not impossible.

The ban on publication of the existing report, imposed on an application by Auckland and Victoria Universities on March 17, remains in place until further order of the court.

If the commission and the universities cannot reach agreement by consultation, Justice Williams will hold a telephone conference with lawyers for both sides on May 11.

The research fund will allocate $18 million in research funding to tertiary institutes based on their research quality. The fund will rise to $185 million by 2007.

The commission has graded the research of 5771 academics at 22 tertiary institutes as A, B, C or R (research-inactive), with A-grade research described as of "world-class standard".

The British system ranks academics on a seven-point scale. In 2001, it ranked 55 per cent of research-active academics as either 5 or 5* (five-star), meaning that at least some of their research was at levels of "international excellence".

But it is clear that far fewer than 55 per cent of New Zealand academics have been awarded A-grades.

Fund project manager Roger Staples said last October that only about 10 per cent of research-active academics in New Zealand were likely to get "A"s, although he later said that figure had not yet been established.

The High Court judgment said the TEC first compared its data with the British figures in mid-November when provisional results for New Zealand academics were "rather lower than expected".

"However, since crude comparisons of raw data would be misleading because the assessment criteria and methodologies differed, TEC undertook a more sophisticated analysis by making assumptions and adjustments to take account of the differences," Justice Williams said.

Dr Boston, who did the analysis, used "conversion rates" of 100, 75 and 50 per cent between the two sets of quality measures, assuming that either all, three-quarters or half of the top-graded British academics would have qualified as A-grade in New Zealand.

He argued that the conversion rate should be "in all probability towards the lower end of the 50 to 100 per cent range" because of the narrower definition of "world-class" research here. On that basis, he told the court, "New Zealand universities compare moderately well with their relevant British counterparts".

What's at stake?

* The new performance-based research fund will allocate $18 million in research funding to tertiary institutes based on their research quality.

* This will rise to $185 million by 2007.

* The commission has graded the research of 5771 academics at 22 tertiary institutions.

* Far fewer than 55 per cent of NZ academics have been awarded A grades - denoting world-class standard.