Tapu Misa: League tables are an unhelpful sideshow

Everyone has an opinion on education and why so many of our kids aren't making the grade. We all think we know whose fault it is: the hopeless parents, the useless teachers, the meddling politicians. It tends to depend on our experience and ideological bent.

We're good at laying blame; not so good with solutions.

Each time I write about education, particularly Maori and Pacific education, there is no shortage of theorising and blaming. Kids would succeed if their parents got off their butts and did more to help them. Teachers need to up their game and raise their expectations of students. It's peer pressure that keeps Maori and Pacific students from achieving. It's their family's economic circumstances. It's under-resourced schools. It's low expectations. It's culture. It's racism.

And the trouble is that there is some truth in all of them. They all matter. But how much is a little harder to unpick.

For those of us who like evidence, there are tens of thousands of studies on education. The difficulty is that what works and doesn't work in education isn't as easily measurable as what works in, say, medicine. We know how many Maori boys leave school without NCEA Level 1, but working out where things went wrong for each one is much trickier.

John Hattie, professor of education at Auckland University, illustrates just how difficult it is to get a clear picture in his book Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, which some have called the "holy grail" of education.

In a meta-analysis, all the findings from the research on a given subject are collected and blended together to produce an average overall "effect size". In medicine, meta-analyses have shown that a particular treatment thought to be ineffective was actually the opposite; and vice versa.

Hattie synthesised 800 meta-analyses based on 50,000 quantitative studies. But anyone expecting definitive answers to our most hotly contested education issues will be disappointed. How much does family background matter compared to the school? Hattie doesn't go there.

As he writes in one of several important caveats, his book isn't about "what cannot be influenced in schools - thus critical discussions about class, poverty, resources in families, health in families, and nutrition are not included - this is NOT because they are unimportant, indeed they may be more important than many of the issues discussed in this book. It is just that I have not included these topics in my orbit."

Unfortunately, politicians looking for silver bullets are inclined to miss that qualification.

A critique of the book by a group of Massey University academics concerned that Hattie's findings may be "appropriated by political and ideological interests and used in ways which the data do not substantiate" points out that education isn't like drug research where it's somewhat easier to measure "effect size" between drug A and drug B.

And averaging out a bunch of disparate studies can ignore the complexity of education - "variables such as age, ability, gender and subject studied are set aside".

For example, on the question of whether homework raises achievement, Hattie finds the overall effect size is a relatively low 0.29. But dig a little deeper and one finds that while homework is relatively unimportant for primary school students it's very important for high school students. It's more important in maths than in science and social studies, and for low-ability students than for high-ability students.

Same with class size; it can matter a lot or not much, depending on whether your child is in a private high school in Remuera or a primary school in Mangere.

It's easy to forget that the biggest challenge in New Zealand education is the long (and disproportionately brown and poor) tail of underachievement, where some 20 per cent of our students languish. Educationists blame the economic and social inequalities entrenched by the government policies of the last 25 years for the size of the gap and the length of the tail (and incidentally our fall in world literacy rankings from number 1 to 24), while many politicians blame failing schools.

Education Minister Anne Tolley has said poverty can't be used as an excuse for failure, and I agree with her. But that doesn't mean it's not important.

Lifting the achievement of those at the tail end seems to be a primary motive for the introduction of national standards in literacy and numeracy. Tolley has said she expects it to help raise Maori achievement. I'm sure everyone wishes it were that simple.

No one is arguing that parents don't have a right to know how their children are doing at school. The sticking point for many principals is that it will lead to schools being ranked in simplistic league tables, and overseas experience shows they have good reason to be leery of the potential negative effects on schooling and students.

Whatever its impact, the national standards and league tables are an unhelpful sideshow obscuring the more pressing needs in education.

If, as expected, they confirm what we already know by ranking poor schools at the bottom and rich schools at the top, what then? How does this help students who have nowhere else to go?

I've met very few teachers and principals who didn't care deeply about raising achievement for all children.

We might make more progress if the minister were to take some time to work with them on this.