1. Pisa provides only a narrow base on which to judge education systems.
2. There have been serious concerns raised about the statistical approaches underlying the rankings.
3. The technical papers that allow researchers to unpick the rankings are only released the year after the main results, by which time the media storm over the rankings has long passed.
4. The Pisa questions are intended to work for students in all participating countries but this is likely easier said than done. India is one country that refuses to take part because of the "socio-cultural disconnect" between its students and the Pisa questions.
5. The results are based on a sample of students and there is evidence that only a select group is involved in some settings. In the case of Shanghai, which topped the league tables last time, more than a quarter of students were not represented.
6. The OECD's attempts to use Pisa data for subsequent analyses (for instance its "Pisa in focus" monthly series) reveal simplistic assumptions.
7. Successes of countries that rank highly are hard to replicate elsewhere. Before Finland topped the 2003 Pisa rankings, the Finns believed their education system was only average. But Finland's advantages were based on decades of social history and politics, not something that can be easily copied.
8. Another example of a country ranking that is hard to replicate: Shanghai educators point to the coherence of their education system. But students in Shanghai also typically do many hours of homework each night in a way that would not be welcomed by most New Zealand families.
9. Governments that worry too much about falling in the rankings often go into "Pisa shock". This typically leads to knee-jerk bad policy decisions such as increasing testing where there is a lot of it already.10. Governments often cherry-pick Pisa findings to justify policy.
More generally, the OECD seems to be keen on opening up public education to the for-profit global education industry.
In the case of New Zealand, the OECD's chief education spokesman, Andreas Schleicher, allowed a video clip supporting the "Investing in Educational Success" policy to appear on Hekia Parata's National Party website in 2014.
Far from being non-partisan, this was endorsing the Government's announcement of new education spending in an election year.
The present Government and future ones would do better to focus on accounts that get closer to the day-to-day work of New Zealand schools.
At the same time New Zealand educational research needs to be allowed to ask hard questions rather than being framed up by policymakers, as has often become the case in New Zealand in recent years.
There are numerous problems caused by education policies of the present Government but we don't need Pisa rankings to tell us that.
