Documents released under the Official Information Act by the New Zealand Qualifications Authority (NZQA) include feedback from one school that, "we were run off our feet solving problems over the whole exam. It had us thinking about pulling out altogether".
"When good people with proven track records and a well managed computer system have difficulties, you know the system is calibrated too tightly for national success," the school stated.
"Our supervisors froze when something went wrong and they then picked up the manual to figure out what to do".
NZQA says the issue arose for students who were assigned to multiple digital assessments. It was resolved within 10 minutes of the exam starting. Students couldn't access the other exams as they did not have the exam codes.
In November, NZQA told new Education Minister Chris Hipkins' office there was a possibility the issue could reach media, despite it being "a very minor issue". A quarter of students didn't log-on, but it wasn't yet clear why: "it could simply be that they got cold feet".
Later feedback from participating schools outlined other issues including a slow computer that meant words took about 30 seconds to appear, frozen screens, lost internet connection, and copy-and-paste issues.
Last month NZQA revealed another glitch related to the digital exams, after some students were mistakenly given fail grades. The Level 1 and 2 exams were offered in Classical Studies, English and Media Studies.
NZQA's deputy chief executive of digital assessment transformation, Andrea Gray, said the log-in issue was down to human error, and was part of the learning process around digital assessments. NZQA was still on track to meet the 2020 goal, and remained convinced of the benefits of online exams.
"These issues arise, and we are very disappointed when they arise, but they are not show stoppers. And students are telling us, 'Keep going NZQA.'"
Comparable education systems overseas were also moving assessment online, with Singapore and the Netherlands particularly advanced. Gray said digital assessment was needed because students were increasingly learning on digital devices.
"And once you get into digital assessment you have more opportunity to personalise it. For example, you could offer an examination at a different time of year from the paper-based examinations.
"With digital assessment you could potentially have adaptive assessment, where if a student has given something a shot and it doesn't quite work, they get served a different question that can set them back to dealing with the more difficult question. That is operating in Australia already."
More pilots would be run this year, with level 3 exams being offered for the first time, but only to students who had previously completed digital assessments. School registration hasn't opened yet, but Gray said the number participating grew by over 200 per cent last year.
Michael Williams, president of the Secondary Principals' Association and also principal of Pakuranga College, said the technical problems were part of the reason why trial and pilot exams were run. His association supported taking exams online because students were now using digital devices to learn and process information.
"We can't have schools being some archaic blast from a couple centuries ago with no relation to the real world. The challenges for schools are around equity – how do we provide equitable access to the technology, both for learning and assessment.
"The equity issues are becoming pointier and pointier."
Education Minister Chris Hipkins said he had been assured that, apart from some minor technical issues, the pilot went well.
"These pilots are opt in and support is on hand in the event of technical issues. NZQA has processes in place to ensure no student is disadvantaged.
"There is no reason to think the 2020 goal cannot be reached. The purpose of the digital trials and pilots is to learn about what works well, and where improvements need to be made. NZQA is working with schools to ensure this happens."
Digital assessment
• NZQA is working towards offering almost all exams online by 2020, and as part of that ran a handful of pilot exams online last year.
• Schools and students could opt-in to the pilots, and results counted. A technical problem meant some students were unable to log-in to a media studies exam, and were presented with log-ins for other assessments.
• Digital pilot exams will be extended to NCEA Level 3 this year, for certain students. NZQA is looking at providing digital exams off-line in some circumstances.
