Dr Fiona Ell: Real issue is the gap, not rankings

It is important that our education system helps all students. Photo / Thinkstock

The New Zealand experience: More important than standings is chasm between our high and low achievers

Ranked lists seem to fascinate us. At this time of the year top 10s proliferate: books, music, political gaffes, and sporting moments. And everyone loves it when New Zealanders come out on top: winning the Booker Prize, topping the US song charts, winning the Rugby World Cup. In the world of education the big ranked list is the OECD's Pisa assessments. The 2012 results were released on Tuesday and we are not top of the pops.

Pisa - Programme for International Student Assessment - is an OECD initiative that has been testing 15-year-olds around the world since 1997. In 2012, representative samples of students in 66 countries sat a 120-minute written test comprising a carefully constructed battery of items. The items were designed to assess the students' preparation for adult life and participation as engaged citizens by testing mathematical literacy, reading literacy and scientific literacy.

In 2012, for the first time since 2003, mathematical literacy was the focus of the assessment. And what happened? New Zealand's score dropped, statistically significantly. We remain just above the OECD average, but with a score that seems to be inexorably falling (we have a high "annualised change score" that shows that our score is deteriorating significantly faster than average). In fact, 25 countries went up between 2003 and 2012, and only 14 went down.

Does it matter? Should we be worried? What does it mean? These seem like simple questions, but in education, like many areas of social policy, the answers can be quite complex.

Should we worry about our ranking? No, not per se. Focusing on one number, one position in the race, who our neighbours on the list are is missing the point.

The OECD provides a great deal of information about our students and their performance in the Pisa assessment. The ranking is a red herring: easy to shock with, but distracting us from the real story.

The real story is about equity. It is in everyone's interests that our education system serves all our students well. You can make a moral justification for it or an economic justification for it, but in both ways, decreasing the achievement differences between groups of students is a win for New Zealand. The OECD Pisa report highlights that while 245 points (equivalent to six years of schooling) separates the highest and the lowest countries, there can be a 300-point difference (equivalent to seven years of schooling) between the highest and the lowest achievers in one country. Countries with big differences between their highest and lowest achievers are characterised as having "low equity". In 2003, we had one of the biggest gaps between low and high achievers. In 2012, this gap has worsened. We have above average performance, but below average equity, and both our performance and our equity have deteriorated since 2003. We are above the OECD average on two measures that we don't want to be above average on: there are bigger differences in mathematics performance between rich and poor children in New Zealand than many other countries, and there is a significantly stronger-than-average relationship between the mathematics test scores and socio-economic status than most other countries (so it is harder to do well in New Zealand if you are poor than in other places). This is what we need to be talking and thinking about, because the solutions to this problem are different to "raise the ranking" solutions - although a by-product of improving equity will probably be an improved ranking.

Focus on the Pisa "score" and the Pisa "rank" have led other countries to narrow their education agenda, effectively "teach to the test" and increase accountability and surveillance of teaching. There's plenty of evidence that this doesn't work.

Let's take the opportunity offered by the Pisa results to develop a robust policy for education that utilises what we know about how to improve schooling, and recognises the fundamental complexity of the problem highlighted by these results. One initiative, one policy, one political viewpoint will not eliminate the inequity of achievement in mathematics documented by the OECD. We need to think creatively and broadly, using the research evidence we have to make deliberate and informed choices about our response. Across all OECD countries a 53-point gain (equivalent to one year of schooling) is found for children who had pre-primary education. The effect of early childhood education is still observable, and still advantaging them, when they are 15. This is but one indicator that any policy response to Pisa results should be broad in its scope, not focused on rankings. Let's address the real issue and improve the life chances of all our students.

• Dr Fiona Ell is a lecturer in primary teacher education in the School of Learning, Development and Professional Practice at the Faculty of Education, University of Auckland.