Library systems miss out on NZ technology

By Adam Gifford

New Zealand leads development of digital archives systems, but our own National Library cannot use the technology, says a database expert.

Ian Witten, computer science professor at Waikato University and world authority on databases and digital libraries, says the failed attempt to create a new computer system for the National Library stops New Zealand from using the latest high-tech cataloguing systems.

"A lot of money was wasted on a system which was never deployed," he said.

"People were burned badly trying to put new technology into libraries."

As a result they were falling back on older technologies, despite New Zealand being at the forefront of development in the field. The $8.5 million National Library project was scrapped early last year.

Professor Witten's department has developed a number of digital libraries which are accessible as CD-ROMS or through the Internet at www.nzdg.org These include the world's largest collection of computer science technical reports, providing access to more than 46,000 research documents.

Other resources include a Humanity Development Library set up for a United Nations agency, an archive of documents relating to indigenous peoples, an oral history archive for the Hamilton Public Library, an archive of minutes and memos for the North Shore City Council, and a music archive which can match a fragment of a tune search with an archive of 10,000 folk songs.

The department is also preparing a searchable archive of Maori newspapers for the Alexander Turnbull Library and is discussing projects with the World Bank and other international agencies.

Professor Witten opened the Australasian Computer Science conference at Auckland University yesterday with a speech on browsing digital libraries.

The technologies underlying the New Zealand Digital Library were freely available in the public domain, he said: PreScript, which converted the PostScript in which most computer science reports were written to plain text which could be searched; MG, a full-text retrieval system he had developed and described in his book Managing Gigabytes; and Sequitur, a method for inferring hierarchies of repetitions from sequences.

MG stores the index, and the text that is indexed, in compressed form, making efficient use of disc resources.

Professor Witten said working in digital libraries would be qualitatively different. One interface the team had developed, Phrasier, allowed the user to compose a document "and as you type, key phrases which appear in the library will be highlighted in your document, which in turn could bring up related information.

"They're like kitchens for knowledge preparation," he told the conference.
Digital libraries meant librarians were even more important.

"Already with the existence of the [World Wide] Web people realise how much information is out there, but it's poorly organised.

"Librarians are gatekeepers.

"We believe that in next few years there will be lot of job creation supporting digital libraries, and we want to be in the forefront of that work."

Professor Witten said there had been a lot of work done of digital libraries since the United States Government announced the Digital Library Initiative (an associated research funding) five years ago, but the Waikato team appeared to be the only one which took the librarian's role into account.