A difficult Bursary calculus paper has teachers and students wondering how fair school exams are. Education reporter REBECCA WALSH explains the mysteries.
It left some students in tears and others wondering what had hit them.
Last year's Bursary maths with calculus exam was so hard it prompted complaints to the Qualifications
Authority.
Teachers labelled it a disgrace, and students questioned whether it was their fault or simply an unfair exam.
Last week the authority said it had to increase most students' totals by 10 to 15 per cent because of the lower-than-usual marks.
Of the 8700 students who sat in November, thousands would have gained less than 50 per cent had the results not been marked up.
An inquiry is being held into how the panel of examiners came to set such a difficult paper.
rdWho sets exams? The Qualifications Authority contracts a group of experienced teachers and examiners to write a prescription of what can or must be examined for each subject, based on Ministry of Education curriculum statements.
The examiner sets the paper and a moderator then makes sure it assesses the appropriate range of work and is pitched at the right level.
Examiners and moderators are usually secondary or university teachers expert in their fields.
Once the paper is formatted on computer, it is checked for grammar, spelling and ambiguities. Diagrams and artwork are designed.
The formatted paper goes back to the examiner and moderator for close scrutiny. Then an independent checker answers the paper as if he or she were a candidate, looking for any problems.
By the time it is ready for printing, an exam will have been checked and completed by at least three people and generally five or six.
The whole process takes about three months. There are about 1500 markers, examiners and moderators.
rdHow are exams marked? The chief examiner nominates a panel of markers halfway through the year. Like examiners and moderators, they are usually secondary or university teachers and under contract to the Qualifications Authority.
Each subject has a chief marker, responsible for supervising markers and the marking process.
A marking schedule is prepared when the exam is written. In the case of subjects involving large numbers of students, marking panels are set up around the country.
For some subjects, such as maths, each question generally has one correct answer.
But for others, such as history or English, the marking schedule will outline a list of points students are expected to cover and the way marks are allocated.
The marking schedule is not finalised until the markers have seen some student work.
Immediately after an exam, the chief marker will photocopy 20 to 30 exam booklets. A marking panel will meet to agree on the sort of answers that will be accepted and how the marks will be allocated.
"For a subject like mathematics, there is usually a right or wrong answer and there won't be much to discuss. But there could be a case where they say, 'If you get the working right down to here, you'll get half a mark,"' NZQA communications manager Bill Lennox said.
A proportion of each marker's work was checked "to make sure everyone is accepting the same sorts of answers."
For some subjects, such as workshop technology, assessment is different because the subject is internally assessed. Moderators visit schools during the year to ensure the marking is consistent.
School Certificate and Bursary art are different again.
For School Certificate art, a sample of folios - from the top work down to the bottom - is sent to Wellington for assessment. Remaining folios are ranked based on those benchmarks.
For Bursary art, all folios are sent to Wellington. Each is graded once by three individual markers and then by a group of assessors. Finally it is ranked by the marking panel.
Mr Lennox said Bursary art was more comprehensively marked than any other subject. In the end, nine people made a decision on the grade.
rdHow does scaling work? School Certificate exam results are not scaled, but the internally assessed component of the qualification is.
About two-thirds of School Certificate and Bursary subjects have some internal assessment - ranging from about 20 to 40 per cent.
Schools send their internally assessed marks to the NZQA and they are compared with the range of marks students gained in the exam.
"The internally assessed marks are adjusted to match the range of marks that group of students from that school got in the exam," Mr Lennox said.
If schools are within 5 per cent of getting their average right, internally assessed marks are not changed.
"There is often a bit of concern among teachers and students over that, because they know what they got for internal assessment and NZQA changes it. But they would be more concerned if we didn't check it, because people would say different schools have got different standards. So we have to have some way of moderating it."
In contrast to School Certificate, every Bursary subject is scaled.
This is to make sure the range of marks for every subject is similar so that when they are added together for University Entrance and A and B bursaries, the system is fair, regardless of which subjects students took.
That counteracts the effects, for example, of a particularly difficult geography exam.
The Qualifications Authority computer looks at the marks all students sitting the geography exam received in their other subjects.
"That gives us a measure of their general ability. If all the students sitting geography did really well in their other subjects, it means they are a high-performing group of kids. Then the marks for geography would be adjusted so they are a reasonably high range of marks," Mr Lennox said.
Median marks usually ranged from 52 to 56 per cent.
Two years ago the median for Latin was 67 per cent - indicating that the 63 students who took the exam did "very, very well" in their other subjects.
Last year's calculus exam was "very difficult," so most of the marks were scaled up.
rdWhy is the system changing? This year the Qualifications Authority will run a publicity campaign about the National Certificate of Educational Achievement - the qualification being phased in from next year to replace School Certificate and Bursary.
The campaign will include four main points outlining the reasons for change.
The first is the need for what the NZQA calls a "sensible approach" to combining internal and external assessment.
Mr Lennox said it was important people realised national exams would not disappear.
Skills that could be assessed in an exam would be, but other skills such as research, laboratory work, performances and speaking would be assessed internally. The National Certificate would therefore assess a fuller range of skills than before.
Second, to report more "useful" results.
Students now get a single percentage mark at School Certificate and Bursary level for their year's work. Mr Lennox said the new system would allow teachers to report in more detail about students' "separate skills and abilities" - for example that a student was good at speaking but not so good at reading and writing.
Results would be broken down into different areas. In English, for example, formal writing, creative writing and speaking would be assessed separately. Students would be marked against national standards and would gain a credit, merit or excellence grade.
Third, the NZQA says the standards have been designed so that students of all levels will be challenged.
Some people argue that the new system will not challenge the most able students.
Mr Lennox said the new system meant less able students would be able to get credit for their particular skills. For example, a student who received 30 per cent for English might be good at speaking or creative writing, but those good results would be buried in the 30 per cent."
At the same time, the excellence standard would be high enough to challenge all students.
Mr Lennox said it would be an "extraordinary" achievement for a student to achieve excellence in the eight English standards at level 1 (School Certificate level).
There would be no pre-determined number of credit, merit and excellence grades. Students who achieved the required standard would get the grade.
Finally, the NZQA says the new national certificate will allow schools and students more flexibility in their courses.
To sit School Certificate or Bursary today, students had to complete the whole one-year course.
In the future, schools could offer courses of different lengths, mix subjects and let students study at different levels.
A difficult Bursary calculus paper has teachers and students wondering how fair school exams are. Education reporter REBECCA WALSH explains the mysteries.
It left some students in tears and others wondering what had hit them.
Last year's Bursary maths with calculus exam was so hard it prompted complaints to the Qualifications
AdvertisementAdvertise with NZME.