Students must ponder mixed bag under new research funds grading

By SIMON COLLINS

The biggest upheaval for years in New Zealand's tertiary education system gives students a lot more choice than virtually anyone had expected. Auckland University, by far the biggest and wealthiest institution, predictably leads the grading list for the new performance-based research fund (PBRF), with 10.8 per cent of its academics rated A ("world-class").

But any student looking for "the best" university in any particular field, such as health, physical sciences or business, faces a mixed picture. No one university dominates any category.

Auckland has the highest proportion of As in 11 of the 41 subjects, Otago in seven, Canterbury and Victoria in six each, Waikato in five, Massey in three and Auckland University of Technology in one (oddly, that one is anthropology and archaeology).

Two subjects, nursing and design, have no A-graded researchers anywhere, raising questions about whether three-year qualifications in those subjects should still be called "degrees". One university, Lincoln, failed to come first in any subject.

Surprisingly, Otago produced more doctorates (117) than Auckland (107) in 2002, the base year for the PBRF calculations.

Massey produced more masters graduates (280) than either Auckland (263) or Otago (190).

The grading exercise, which has cost $3 million plus an average of perhaps two days' work by each of the 5771 academics who filled in the assessment forms, is intended to encourage tertiary institutes to concentrate their research spending in areas where they do well.

"They are encouraged to aim for depth rather than breadth," says the acting head of the Tertiary Education Commission, Kaye Turner.

But for the institutes, as much as for students, the results make it difficult to know where to focus.

Should Auckland, for example, focus on medicine, where it beat off its only rival, Otago, in both the core medical subjects of clinical medicine and biomedical science?

Should it abandon nursing, sports science and other health sciences where it has no A-graded researchers, even though elements of those subjects might be useful for some of its core medical students? And should the country as a whole aim to keep pockets of excellence spread around the campuses, or focus on building up "critical mass" in just one or two that have a chance to compete with the best in the world, presumably Auckland and Otago?

The whole exercise has huge social implications. When the new system is fully phased in by 2007, it will transfer millions of dollars from the polytechnics, teachers' colleges, wananga and private training institutions to the universities.

The universities all support it, because as a group they have gained just over $1 million extra this year compared with what they would have got under the old system, and by 2007 they will be $46 million a year better off.

But by 2007 the shift will have taken $5.3 million a year out of the polytechnics, $1.1 million out of the teachers' colleges, $834,000 from the wananga and $440,000 from private training establishments.

Ironically, even one of the surprise winners in the new grading system, the Tuhoe charitable trust of Anamata, which teaches mainly Maori language and social work in Whakatane and Taneatua, is worried about the Government's policy of capping the growth of funded places in the private sector.

"For several years now we have been observing Government policy really shifting back to favouring the mainstream," says Anamata chairman Tamati Kruger.

"Yet Anamata struggles to make a profit, so you can see that the strongest motive for anyone working for Anamata is really the sense of social justice."

If the PBRF succeeds in creating "a more differentiated tertiary sector", as Tertiary Education Minister Steve Maharey predicts in his introduction to the funding report, it also raises the spectre of a hierarchy of institutions as in Europe, the US or Japan where degrees from Cambridge, Harvard or Tokyo University are worth more than the same degrees from poorer, less prestigious institutions.

New Zealand universities, which were once simply colleges of the united University of New Zealand, have traditionally prided themselves on offering a uniformly good education.

Under the new system, Auckland and Otago will both get more money from the public partly because they have the country's only medical schools, which attract the most external research income - which counts for 15 per cent of the overall weighting in the PBRF.

At the other extreme, Canterbury, Victoria and the Auckland University of Technology (AUT) will all get less proportionately when the new system is fully phased in because, despite the high gradings of Canterbury and Victoria academics, they attract little external research income.

In effect, the new system takes money from the "have-nots" and gives more to those who already have it.

The grading system itself, which accounts for 60 per cent of PBRF weightings, is based on a mixed assessment of the researchers' choice of their four best research outputs in the previous six years and up to 50 research outputs in that period (42 per cent), "peer esteem" (9 per cent) and "contribution to the research environment" such as supervising graduate students and organising academic conferences (9 per cent).

The Tertiary Education Commission's report notes that it was virtually impossible for young academics to achieve an A because, even if they produced some world-class research, it was usually too soon for them to have built a wide reputation or to have been chosen to supervise students or run conferences.

"In a significant number of cases, high-calibre researchers failed to provide sufficient detail in the peer esteem and/or contribution to research environment categories," it says.

Auckland University Association of University Staff president Peter Wills says he deliberately left much of this part of the assessment form blank because he did not want to "skite".

"From what I can estimate, I am held in very high esteem by my peers internationally," he says. "They came up with a very low number, so as far as I'm concerned it's just a lot of silly nonsense."

The final 25 per cent of the PBRF weighting, for numbers of research degrees completed, is less controversial because it is similar to the previous funding system based on graduate student numbers. It gives higher weighting in the funding formula to students studying costly subjects such as medicine than to those studying arts or business.

It also gives a weighting of 2 for every Maori and Pacific student completing a research degree, and 1 for all other students. Of the 1727 masters and doctoral degrees completed in 2002, 1232 were completed by Pakeha students, 255 by Asians, 47 by Maori, 24 by Pacific Islanders, 139 by "other" ethnic groups and 30 by students who did not state their ethnicity.

There is no doubt that Auckland University comes out best overall. The commission comments that "the overwhelming majority of Auckland's 58 nominated academic units achieved a quality score above the tertiary sector average".

"Twelve of Auckland's nominated academic units obtained a quality score of 5.0 or more - which means Auckland represents over half of the 22 units to achieve such a score," it says.

"Most of Auckland's high-scoring units have well in excess of 10 PBRF-eligible staff, thus indicating a considerable depth and breadth of research capability.

"Another sign of the relative research strength of Auckland University is the fact that, in 26 of the 41 subject areas, its quality score ranked first or second in the country."

It led the A scores in the two core medical subjects, two sciences (ecology/evolution and statistics), architecture and planning, education, sociology and social work, English, dance and drama, economics and marketing.

Canterbury University came second, both in terms of the proportion of A-graded researchers (9.3 per cent) and in its overall weighted quality score. Its strengths are in sciences (chemistry and earth sciences), humanities (foreign languages and religion), visual arts and law.

The country's oldest university, Otago, came first in traditional subjects such as history, philosophy and accounting, as well as in several health fields and in technology. It fell marginally behind Wellington's Victoria University after allowing for B and C grades.

Waikato University's overall grade was dragged down by its second-biggest unit, education, but the commission notes that the university would have matched Victoria if education was excluded. Education departments got low grades generally, with 73 per cent of their academics rated R (research-inactive), evidently because they concentrate on teaching.

"While research is vitally important (especially for universities), so too are teaching and service to the community," the report says.

"In many cases, PBRF-eligible staff members are employed primarily, if not solely, for their teaching expertise rather than as researchers. This, of course, is perfectly appropriate. High-quality teaching, after all, should not be regarded as an optional extra."

However, the report questions whether we should have no A-grade researchers at all in nursing and design, with fewer than five As in each of eight other fields.

The commission is already reviewing the results of this first round of funding and will repeat the entire exercise in 2006. After that, it will be repeated only at six-year intervals, more or less in line with the five-yearly reviews in Britain's research assessment exercise, on which the PBRF was modelled.

During the 20 years in which Britain's grading system has operated, there has been a gradual inflation in the number of researchers working in top-graded departments - up in the last round alone from 11 per cent to 19 per cent.

This may mean that the system has had the desired effect of strengthening resources in the best research teams.

Alternatively, it may mean that British universities have simply learnt to "play the game" of maximising the "outputs" which earn the highest credits in a performance-based system, such as writing articles for the leading international journals rather than doing research that might be just as valuable but only suitable for more obscure journals.

From the students' point of view, the risk is that those international journal articles may also have taken the academics' time away from the basic purpose of an educational institution - education.

Few other countries have followed the British example. When the Tertiary Education Commission tried to compare its results with other countries, Britain was the only country with a system that was similar enough to even attempt a comparison.

The commission says New Zealand has learnt from the British example and produced a system that is much more comprehensive and therefore harder to "game" than its British predecessor.

Once again, as in so many areas, New Zealand has staked out an extreme position which goes where no one else has quite dared to go before. New Zealand students will not be the only ones watching the results.