John Roughan: Mechanistic marking dooms NCEA

A few weeks ago I visited a secondary school writing class and was given an insight to what is happening when subjects such as English have to be marked to the mechanistic standards of the NCEA.

The students told me that when they get their exam papers back they find practically no imprint from the markers except for the underlining of certain words. Dull words, judging by the students' tone. They have the impression, more or less confirmed by teachers, that their work is assessed largely on the use of expected words.

But educationists, of all people, should know that those who understand a subject very well are actually less likely to use the jargon of it. Students who have really mastered a subject will be able to discuss it in ordinary language.

That is particularly true of English, where good students will have an aversion to ugly conventional terms and cliche. But it is also true of history, social studies, economics and, I suspect, of mathematics and any of the sciences that require deductive thinking.

When you read the work of scholars you notice that first-class minds typically express themselves in everyday terms. They let nothing obscure the clarity of their ideas. It is the second-rate who need to use academic code to signal their credentials and to lend false respectability to propositions that become banal or ridiculous when translated into clear English.

Sadly, the study and administration of education seems to be dominated by the second-class. Even to understand what they have done to school examinations requires a translation of terrible terms such as "standards-based assessment" and "norm referencing".

Professor Warwick Elley is one first-rate educational theorist who has always said the NCEA's standards-based assessment could not be applied to all academic exercises and he looks to be vindicated by the backtrack on the scholarship exam this week.

Standards-based assessment, Professor Elley explains, is the kind of test you face for a driving licence. If you answer the questions correctly and demonstrate the required driving skills, you pass. If you can't, you don't.

It is a matter of setting a reasonable standard and it doesn't matter how many pass or fail - the standard is objective and fixed. "Norm referencing" is quite the opposite. If driving tests were norm-referenced, they would give us all a mark out of 100 and only the top 50 per cent would probably be given a licence.

If education theorists had used labels such as objective assessment and comparative assessment the problem now might not be so hard to grasp.

School exams used to be comparative ranking exercises.

There was nothing objective about the standard required in School Certificate, for example. They would look at the spread of marks and the mid-point would become the standard. If the average result was over 50 per cent they would conclude the standard set by the exam was too low and they would scale all the marks down. If not enough students reached 50 per cent it must have been too hard and they would scale them up.

That always sounds diabolical to me but practically everybody in education thinks it legitimate. It was not the reason they decided to change to an objective system.

The change came because comparative tests are unfair to those who know a subject but not as well as most of their contemporaries. It was as though you would be refused a driving licence because, while you had shown you could drive safely, most others taking the test drove better than you.

Professor Elley is one of those who see nothing wrong with comparative ranking, though he would not use it for driver licensing. He has long argued that objective tests are fine for some purposes but not for others. When I listened to those English students a few weeks ago I began to see what he means.

How do you set objective standards for a subject like English? Well, he would concede, some components of the subject can be objectively measured fairly easily. Spelling is either right or wrong and the professor cites the spelling level tests we had at primary school as an early, perfectly good use of "standards-based assessment".

But when you go beyond the fairly mechanical rules of spelling and grammar, how do you define objective standards for, say, literary appreciation or written expression?

On the evidence I heard, they have resorted to a mechanical standard for those as well. Use certain words and you will get credit. Use a few more and you achieve merit, I guess. Recite the whole damned academic lexicon and you are considered excellent. Unless the exam reformers can do much better than this, the NCEA is doomed.

* John Roughan is a Herald assistant editor.