Michael Johnston: Publishing National Standards may do more harm than good

National Standards in reading, writing and mathematics for students in Years 1-8 are now in their third year of implementation. Photo / Thinkstock

Opinion

National Standards in reading, writing and mathematics for students in Years 1-8 are now in their third year of implementation, and Fairfax Media recently published data facilitating the comparison of schools on the performance of their students against the standards. The Ministry of Education has also published National Standards data.

There are two arguments often voiced by politicians and media commentators in favour of the publication of these kinds of data. The first is that the publication of National Standards data is a legitimate mechanism to hold schools accountable to the taxpayers who fund them and the parents whose children attend them. The second is that comparative data can assist parents to make informed choices about the schools in which to enrol their children.

The New Zealand Assessment Academy, a group of leading researchers in educational assessment and measurement, does not agree with either of these arguments. While we acknowledge that accountability is important, we do not believe that the data published by Fairfax will serve accountability, either by the way they presented the data or by the conclusions which are likely to be drawn from them.

Neither, at this stage, do we support the publication of National Standards data by the Ministry of Education. We believe that what might seem valuable information about the performance of schools in fact runs a serious risk of misinforming the public and, in some cases, of unfairly tarnishing schools' reputations.

If data are to be used for accountability purposes they must have a high degree of integrity and reliability. At present, there is no evidence that this is so.

Even if sufficient reliability can be achieved, a simple comparison of schools on the basis of proportions of students at, above, below, and well below the standards, without taking into account the characteristics of schools, is likely to misrepresent schools with substantial proportions of students from disadvantaged socio-economic backgrounds. Educational data from across the developed world show that socio-economic capital is strongly correlated with educational success. Although there are salutary examples of schools that overcome negative social and economic influences on students' achievement, there are factors beyond the control of teachers and schools that render any point-in-time achievement comparison almost meaningless.

Any comparison between schools ought to focus on measures of progress, and not on point-in-time performance. To some extent this can begin to address the difficulty of comparing schools catering to very different demographics; for example, a low-decile school that compares unfavourably with higher-decile schools on the proportions of their students at or above the standard, might nonetheless be able to show that its students make as much or more progress than students at higher-decile schools.

At present, however, there is no sound mechanism to measure progress in the National Standards domains.

Additionally, the regular publication of any table of assessment data is very likely to produce a bias in teaching towards the elements of learning that are captured in the table. Literacy and numeracy are important skills that ought to be (and are) emphasised in the primary curriculum; however, they are not the only things that are important.

Data like these could produce an incentive for schools to drill aspects of reading, writing and mathematics in order to support favourable judgments, potentially losing other important skills in the process. In other words, comparing schools on assessment outcomes is likely to lead to a narrowing of the curriculum to just those aspects that are most easily measured or assessed.

While the New Zealand Assessment Academy acknowledges the right of all New Zealanders to access educational data and supports the right of the press in a free society to publish those data, we do not support the production of tables of data that can be used to compare schools, like those produced by Fairfax. We urge media representatives to think carefully about, and take much greater responsibility for, the effect that their publication of these data is likely to have.

Neither do we support the publication of National Standards data by the Ministry of Education at present, because we remain unconvinced that the quality of the data are sufficient to justify publication. Instead, we urge the Ministry to devote its resources to providing support to schools to improve the quality and consistency of teachers' judgments, to developing a sound method of measuring and reporting individual students' progress over time in reading, writing, and mathematics, and to provide further guidance and practical assistance to enable teachers to meet the learning needs of all students.

Michael Johnston is a senior lecturer in education at Victoria University and writes on behalf of the New Zealand Assessment Academy.