A whistleblower who researched historic abuse claims for the government in Archives NZ says she raised repeated concerns about sensitive personal information sitting in "open access" files but nothing was done.
The researcher, who worked for a major government department involved in the Royal Commission into state abuse, raised the concerns from 2013 to 2018.
She said the issue was known and should have prompted action by Archives NZ, the Ministry of Social Development, Ministry of Health and Ministry of Education.
The Herald has separately seen documentary evidence showing the Ministry of Social Development and Ministry of Education were aware of personal information being present in open public files.
• Sensitive personal state ward details open to all in Archives NZ blunder
• Big Read: Privacy breach exposes life in children's homes
• Mongrel Mob member Harry Tam's role in state abuse inquiry being investigated
• Bishop of Dunedin eyes full reveal of sexual abuse claims
Archives NZ took the unprecedented step of closing access to all material relating to children's homes after the Herald revealed the names and personal details of hundreds of former state wards was publicly available.
It has asked other state agencies responsible for personal and sensitive information to guard against further privacy breaches.
The details the Herald found included personal details of more than 500 children, including details of alleged criminal offending, medical treatment, perceived behavioural issues and family and fostering information.
They were scattered through files which were part of the Archives NZ collection of records from the Hokio Boys' School and Kohitere Training Centre.
The whistleblower, who asked not to be identified, said she felt obliged to speak out because she was "concerned about just how disingenuous Archives NZ and Oranga Tamariki have been about this breach".
She said responses gave the impression the privacy breach had "come out of nowhere".
In statements to the Herald, Archives NZ and Oranga Tamariki said the information had been provided to the Herald about 30 years ago, before modern privacy and record-keeping legislation.
But the whistleblower said she discovered the issue while compiling information for a major government agency as part of its response to historical claims of abuse by the state.
She said she worked as a researcher on Archives NZ files from 2013 to 2018.
"It did not take me long to discover just how many files are held at Archives NZ that were completely open access when they should have been restricted. And when I did find out, it took a long time to make anyone care."
The whistleblower said she alerted Archives NZ over private information in "hundreds" of files from the Ministry of Social Development, the Ministry of Health, Ministry of Education and State Services Commission.
She said a process was established with Archives NZ to manage information which had been wrongly classified but no attempt was made to deal with the system-wide problem.
"All these government departments have been aware of this issue for at least five years that I know of. For Archives NZ to say 'this has come out of nowhere' is rubbish.
"The problem is not that they didn't know, the problem is that they didn't care when they did know."
The whistleblower said the first files found in open access were from the Ministry of Social Development.
"I went back to work and said these files have to be restricted immediately."
The details were passed to MSD and months went by before the access status of the files was changed to protect personal information.
The whistleblower said there were files containing personal information throughout Archives NZ because "nobody is checking this stuff".
The nature of the disclosures on matters in recent history - with people still alive - were deeply personal, she said.
"If you had read what I've read, you'd be pretty be pretty pissed off with the way the government departments have behaved."
She said she found information throughout her time working on the Archives NZ material, from 2013 to 2018, and believed concerns were raised from 2015 at the latest.
"I was finding stuff right to the bitter end. It's that big an issue."
The whistleblower said the information held in Archives NZ about state-run care institutions needed to be available, if it did not breach privacy.
"We have a responsibility for the truth to be known."
Archives NZ was identified as a key part of the Royal Commission into state and faith-based abuse and developed a protocol for protecting records needed for the inquiry.
In doing so, it told those who had been in state care: "A record containing information about your care will be restricted. This means that even your name will not be listed publicly on [Archives NZ's search engine]. And your record can't be accessed by anyone without permission."
Former state wards whose details were in the files told the Herald they were upset the information was public.
Internal Affairs minister Tracey Martin said $36 million in new funding had been made available for Archives NZ storage and infrastructure issues.
She said the Herald's discovery of the private material showed there were gaps in how information had been managed "historically" when the files were sent to Archives NZ in 1989.
"I think it's really unfortunate that the personal details of those individuals were available. The rules and responsibilities around government are quite different now."
The briefing she received from Internal Affairs in 2017 warned Archives NZ in Wellington was running out of space and its systems were under pressure.
The briefing said: "Critical infrastructure issues are compromising our ability to effectively preserve our documentary heritage, and to ensure ongoing access."
Former Minister of Internal Affairs Peter Dunne said he considered Archives NZ to be a valuable national institution which struggled to compete for funding with other agencies focused on the dealing with issues in the present.
As a result, he said Archives NZ did not appear to aggressively pitch for funding because it seemed reconciled to not getting much.
He said he would be concerned if concerns over privacy - which needed to be protected - led to Archives NZ restricting access to records.
Dunne said the "sheer volume of material" the agency had to deal with made processing and managing the contents of files difficult.
Archives NZ confirmed this afternoon it had been alerted to files containing personal information.
It earlier said the files identified by the Herald had been accessed by this journalist, government researchers and two private individuals.
MSD deputy chief executive Stephen Crombie said the agency had identified three occasions on which it was alerted to private information being displayed in public files.
The three occasions were in February and May of 2017 and in each case action was taken on the same day to restrict the files.
On two of those occasions, concerns about the individual file led to a series of records being restricted.
He said MSD was currently reviewing all records transferred to Archives NZ to ensure none breached the Privacy Act.
The Ministry of Education and Oranga Tamariki were also unable to respond with comment by publication.