David Fisher: Archives NZ needs focus and support to preserve our history and protect out future

Personal and sensitive information has been classified incorrectly and is available to the public at Archives NZ. Photo / File

COMMENT: On any given day, it is possible to walk off the street and into Archives NZ to find private and sensitive information about living people which should not be on public view.

The Herald showed this when it accessed and read files of documents about children sent to Hokio Boys School and Kohitere Training Centre, two of the institutions to be investigated in the upcoming Royal Commission of inquiry into abuse in state and faith-based care.

The files contained hundreds of names of boys, with details about their medical and psychiatric care, alleged child and youth offending, behavioural descriptions and, extraordinarily, commentary on one boy's appearance.

It didn't stop there. Other files recorded hundreds of names and comments about girls and young women sent to other state-run institutions.

The privacy breach was nothing new to Archives NZ. It is familiar with files marked "open access" needing to be re-classified as "restricted" on the basis of personal and sensitive information being inadvertently, sometimes carelessly, placed in public display.

Take the latest revelation - that the Ministry of Justice dumped 33,000 coronial files of investigation in deaths as recent as 2000 into the public access section. More than 5000 people viewed those records - it was a breach on an almost industrial scale.

Archives NZ can't guarantee there will not be further privacy breaches, and rightly so. There are other examples of files still available containing details which would make people uncomfortable, if it were known they were there.

It is not an ideal situation, yet it is tolerable only when balanced against the alternative of restricting access to all files covering the lifetime of those still alive. Considering doing so is intolerable because it highlights the rich value of Archives NZ's holdings and the benefit it has to New Zealand.

Archives NZ describes itself this way: "The more than seven million archives that we do hold shed light on the historical relationship between the New Zealand public and the state. This relationship makes up the official memory of government."

Our modern democracy was not conceived in an instant. It was built over generations, through the successive lives of those who were born and came here, through the 52 Parliaments which have formed and the institutions it established to order our lives. Those seven million records are our foundation stones and building blocks, and contain the answers to the question of how we built our modern New Zealand.

Archives NZ holds seven million records across four repositories. Photo / File

Archives NZ is our ballast, a pilgrimage for those with mysteries to solve and, in doing so, providing knowledge and a chance at wonder. It is a place of answers to questions which are personal and to those which help us understand what it means to be a New Zealander today.

It is a true taonga, a public resource which is tremendously open to the public. It is a lodestone for accountability and democracy, unerringly pointing in the direction we are heading through the relationship it forges with the past.

For all that, the records recording how we lived, who lived here and the decisions we made which have led us to the country we have today exist as a roughly ordered, often haphazard blizzard of documentation.

There is work underway to "strengthen" the benefit Archives NZ can offer. Culture and Arts Minister Grant Robertson and Internal Affairs Minister Tracey Martin set up a ministerial group to investigate what needs to be done to put the institution on the best footing for the future.

Chief Archivist Richard Foy. Photo / Supplied

Submissions on the issues confronting Archives NZ highlight its vulnerability to political whim, and the variable standing to which it is subject.

It has had periods during which it operated under the Department of Internal Affairs, that catch-all agency for government functions which didn't fit elsewhere. It has also had periods (most recently 2000-2011) where it was a standalone agency.

Many of the submissions called for the Chief Archivist to be an Officer of Parliament, like the Chief Ombudsman or Inspector General of Intelligence and Security. It would enhance the role from where it currently languishes, in government speak as a "Tier three" organisation, meaning its a cul de sac on the highway of Internal Affairs.

It's hardly a guarantee of success. There are plenty of such agencies which struggle for funding, or to have their voice heard.

Chief Archivist Richard Foy rightly says there is no need for this because the Public Records Act gives him all the power he needs. However, he does advocate for Archives NZ to be broken free so as to be a "Tier one" organisation.

He said "the position of the Chief Archivist in the third tier of the Department of Internal Affairs' management hierarchy does have an effect on the status of the office, and with it the perception of the office's authority".

The power the Chief Archivist wields through the Public Records Act "allows the public to use documentary evidence as the basis of a check on the power of government".

"Public sector agencies are more likely to act to comply fully with the Public Records Act if there is respect for the regulatory authority and power of the Chief Archivist and Archives New Zealand."

Archives NZ needed to be a standalone agency with its own budget, he said.

Something needs to change because what we have isn't working. A review of Archives NZ's assessment of government compliance with the Public Records Act, and its record keeping, shows it to be in poor state. Foy said some agencies had developed systems but others "need substantial guidance in managing digital records for enduring access and preservation".

Government agencies were making technology choices, and structuring their systems, in ways which made it difficult to access and preserve information. And its clear from the Chief Archivist's reports it isn't just an emerging issue - our state sector has been slipping into a crisis of information morbidity for years.

The scale of the records it needs to accept has massively expanded as government moved from paper-based systems to digital, and this shift has created significant issues for storage and access - how does an archive preserve records on computer systems which are constantly antiquated through successive technology revolutions? And how to future-proof that preservation?

For records past, look to the briefing prepared for Internal Affairs Minister Tracey Martin when she took office in 2017. It stated: "Critical infrastructure issues are compromising our ability to effectively preserve our documentary heritage, and to ensure ongoing access."

Martin has secured more funding for Archives NZ, but money isn't the only answer. It actually needs government, and the government sector, to seriously give a damn.

Internal Affairs minister Tracey Martin has set up a ministerial working group to study how national institutions, such as Archives NZ and the National Library, operate. Photo / Supplied

As Archives NZ keeps pointing out, it holds the information but it is the various government agencies responsibility to provide it in a form which can be preserved, to manage the classification of the material and to support public access through doing so.

This is, quite simply, not happening the way it should.

If it was, it would not be possible to walk off the street and read personal and sensitive details about our neighbours.

The response to the Herald investigation into Archives NZ holdings has seen the affected agencies scurrying along to check for sensitive information in its holdings. Our reporting has shown that none of this is really a surprise to any of those agencies - they just didn't seem to care until a journalist wrote about it.

The immediate response was to shut access to all records. In doing so, the public instantly lost and Archives NZ's primary mission was thwarted.

What's almost worse is that the repository is filled with masses of records which should be public yet are set to "restricted". It is a tragedy that no one is proactively reviewing those files to provide the public with access to records to which they are entitled.

The Herald has heard stories describing the haphazard and sometimes utterly chaotic way files have been classified and delivered to Archives NZ. Classifications based on what is written on the front of a file, contents never checked.

It means it's impossible to have confidence that private information isn't publicly available.

It also means it is impossible to be confident necessary public information isn't locked out of sight on the basis of bad, or non-existent, decision-making.

We are fond of saying in this country we must walk forward while looking backward - ka mua, ka muri.

This taonga which is Archives NZ should give us the best vision possible. Instead, we are in danger of losing our sight, and as a result, we risk losing our way.