What is the secret of a successful school? For a while in the US, researchers studying the characteristics of the country's most successful schools were convinced that, all else being equal, the distinguishing factor was size.
Data showing that small schools were over-represented among the top performing schools encouraged the Gates Foundation to invest heavily (some US$1.7 billion by 2001) in the creation of small schools, and sometimes in splitting up large schools into smaller units. At least half a dozen prominent institutions climbed on board, including the Pew Charitable Trust and the US Department of Education's Smaller Learning Communities Programme.
It's easy to see why the idea appealed. It made intuitive sense, as the psychologist Daniel Kahneman explains. "It is easy to construct a causal story that explains how small schools are able to provide superior education and thus produce high-achieving scholars by giving them more personal attention and encouragement than they could get in larger schools."
Unfortunately, the facts were wrong. As the statisticians Howard Wainer and Harris Zwerling pointed out in a 2006 paper, if the researchers who reported to the Gates Foundation had looked at the characteristics of the worst schools, they would have found that "bad schools" also tend to be smaller than average.
In other words, small schools weren't better on average; they were simply more variable. If anything, large schools tended to produce better results, especially in higher grades where curriculum choice and specialised teachers matter.
Just to be clear - and I feel the need to labour the point given last week's column was widely misinterpreted as a personal endorsement of David Shearer's leadership bid (it wasn't) - this is not an argument against charter schools, which the Government intends to try out in Christchurch and South Auckland. It is an argument for caution, and the acknowledgment of complexity.
Frankly, I'd love charter schools to be the answer to the "tail of underachievement" into which some 20 per cent of our kids are consigned.
But there are good reasons to be cautious. As well as the much-cited Stanford University study which concluded that while 17 per cent of charter schools were superior to a similar public school, 46 per cent were no different, and 37 per cent were worse - there is a 2010 study by the US Department of Education, which found that charter middle schools were "neither more nor less successful than traditional public schools in improving student achievement, behaviour and social progress".
Closing the achievement gap has become the holy grail of the education field, but it has remained stubbornly resistant to well-intentioned experiments.
The education field is littered with failed and expensive reforms (witness George W. Bush's No Child Left Behind), and promising breakthroughs that turned out to be little more than statistical mirages. As emeritus professor of education at Massey University Ivan Snook has pointed out, there have been many instances where odds-defying, gap-closing shining successes turned out to be somewhat less impressive on closer examination.
In some cases, the high achievements were achieved through a selection process that "cream-skimmed" the best and most motivated students. In others, the achievements weren't all they were cracked up to be; to qualify as a "high flying" school in one US example, children had to excel only in one grade, only in one subject and only in one year.
This is not to deny the valiant efforts, and successes, of many public schools.
We can be rightly proud of our standing in international surveys, which rank our students' performance in literacy, numeracy and scientific literacy among the highest in the world.
But among 30 countries in the OECD's programme for international student assessment study, our achievement gap is the second widest.
Teachers are right to argue that the gap is the consequence of our widening income inequality, which an OECD report last week confirmed had grown faster than most other OECD nations, and that a government concerned to raise achievement in poor communities needs to start by alleviating poverty. In an ideal world we would have a well-resourced public school system providing quality education for all children from early childhood onwards, regardless of their parents' income or address.
Our public schools need to be strengthened and safeguarded, but kids in low-income communities can't afford to wait. They need extraordinary measures now.
Charter schools aren't a miracle cure, but amid the well-documented failures there is evidence that some have raised achievement levels in disadvantaged communities.
Profiling the successful Knowledge is Power Programme schools in a 2006 New York Times article, Paul Tough noted that "the message inherent in the success of [these charter schools] is that if poor students are going to catch up, they will require not the same education that middle-class children receive but one that is considerably better; they need more time in class than middle-class students, better-trained teachers and a curriculum that prepares them psychologically and emotionally, as well as intellectually for the challenges ahead of them".
The question that should concern us all is not what makes some charter schools successful, but what makes any school successful.