A fresh and extraordinary privacy breach at Archives NZ has been uncovered - 33,000 coronial files on public display revealing child abuse, mental health details and suicide notes.
The files related to deaths which occurred between 1979 and 2000 and saw psychiatric reports, details of child sex abuse, suicide details - including a suicide note - and other highly personal information available to and accessed by the public.
The coronial files were available to the public between 2004 and 2012 until a Ministry of Justice staff member raised concerns after an upset family member made contact over details of her brother's suicide being publicly available.
• Govt departments say they were told of private information being open to public
• Whistleblower: Agencies knew about Archives NZ privacy breach for years
• Sensitive personal state ward details open to all in Archives NZ blunder
• Big Read: Privacy breach exposes life in children's homes
An investigation in 2014 found the information had been accessed 5845 times over that period, with an analysis of those requesting files showing family, media, researchers and even insurance companies had been given files.
Despite the scale of the privacy breach, the Ministry of Justice did not make it public and apparently did not inform family of those whose details were accessed because it could not identify them.
It's the latest disclosure of information intended to be private and comes after claims issues at Archives NZ were "historical" and came about as a result of practices before the Privacy Act and Public Records Act.
The Herald revealed the existence of personal information sitting in open Archives NZ files after accessing files relating to state wards at children's homes from the 1960s through to the 1980s.
The ongoing Herald investigation has revealed Archives NZ, the Ministry of Social Development and Ministry of Education were aware of vulnerabilities in the archiving system which had exposed personal information.
Minister of Internal Affairs Tracey Martin, who is responsible for Archives NZ, has now asked for a report on the issue.
The Ministry of Justice breach is the largest scale example so far. It was never made public at the time, even though thousands of people with no connection to those who died had accessed their personal records.
The privacy breach was discovered when the sister of a man who died in 1997 - believed to be by suicide - discovered the coronial file was available publicly at Archives NZ and asked for it to be restricted. A review of the file found it contain toxicology reports, the post mortem and the police report on the person who had died and the factors associated with their death.
The same month, the files on siblings who had died was requested by their father. The file contained witness statements from the mother and a psychiatrist, which included details of child sex abuse. A report from the psychiatrist - with further sex abuse details - was also openly available, as was the port mortem report and police report.
The ministry contracted Deloittes to carry out an investigation into the breach, which found Archives NZ had accepted the files on condition "all sensitive material was sealed".
It transpired the only information to be "sealed" was photographs of the deceased and suicide notes - and even that material found its way into files.
At one stage, the ministry hired contractors to check the files to ensure photographs and suicide notes had not been left in public view, but even this basic step wasn't completed.
The Deloitte report found the work stopped at 1991 files because the project ran out of money.
The Deloittes investigation selected coronial files at random - two suicides, a child homicide, a sudden infant death and a car accident - to review what sensitive information was publicly available.
It found post mortem reports, toxicology reports and police reports, along with scene investigation details from the sudden infant deaths and a suicide note, albeit in a sealed envelope.
The material was meant to be restricted from public view for 50 years, as was the case for deaths before 1977 and after 2000.
The Deloittes inquiry said there was a "high risk" to the ministry's reputation and function relating to people accessing and possibly copying files to which they had no relationship. It also meant there was a risk information about living people contained in the file had also been copied.
The inquiry found a "management oversight" at the ministry and Archives NZ was behind the breach, including a failure to vet the information before it was sent and a lack of understanding of the sensitivity of the information in the files.
There was a lack of proper systems, along with a "lack of management focus and investment in documentation".
Broad changes were recommended to systems, along with checks on access to files, setting the access restriction to 50 years and giving the Chief Coroner the right to approve access settings for coronial records.
Ministry of Justice chief operating officer Carl Crafar said the files were now restricted from access for 50 years and most other recommendations made in the Deloittes report had been carried out.
He said the work following the review included the management of Official Information Act requests at the coronial information unit to make sure enough funding and oversight was in place.
Coronial files could still be requested by the public but were scrutinised to ensure no sensitive information was released.
The ministry said anyone concerned a loved one's coronial file had been accessed in 2012 or before should contact Archives NZ.
Archives NZ operates on the basis the agencies which provide it with information are responsible for the contents. Herald inquiries have shown those agencies transferring records have not properly checked the contents, allowing sensitive material to appear in "open access".
Archives NZ director of government record keeping Antony Moss said the coronial files case was another example of the issue which had led to sensitive information being in public access.
He said Archives NZ did not consider it to be a systemic issue but could not guarantee other cases would not emerge.
"It's true record keeping practises in government need improvement in many areas and this is one of many."
A spokesman for Privacy Commissioner John Edwards said his office was briefed regularly at the time and "made it clear that the data breach was of concern".
He said the Ministry of Justice had found it could not identify family members who might have been affected by the breach and so was unable to tell them it had occurred.
"The commission has no jurisdiction to require an agency to notify individuals who might have been affected by a breach."