Storing electronic data for the long haul has government archivists perplexed
Did he or did he not pee against a tree? Was he in a fit state to drive after several hours at GPK? Andrew Williams: people's champion or pissed chump?
These questions about the North Shore Mayor's naughty night remain unanswered because the security system at the council's offices writes over the video tapes after 72 hours, rather than keeping them for weeks or months as at other councils.
Storage of electronic records is taxing the minds of our tax-spenders, with Archives New Zealand consulting on a new digital recordkeeping system standard.
John Roberts, the acting group manager for government recordkeeping, says digital records have been an issue for government for the past 20 years or so, but much of what the archives have been concerned with up to now has been digitisation.
But with most information now created and used digitally, it's important the systems keep up.
"There is a time lag for material coming through from agencies to Archives, so the majority is still on paper and traditional formats," Roberts says.
"We have not yet seen the crashing of the wave from digital formats, and part of the reason we are doing the work now is to ensure there are good systems in government for management. We want to make sure the stuff there survives and is usable." That means making good decisions about the lifespan of information.
"The most effective technique is to make sure you think about long-term preservation needs up front and design systems to take into account that key records need to be exported, and they need to be well supported in terms of metadata and data dictionaries.
"It's about saving the important stuff rather than trying to manage every email."
Those sort of questions could be asked about North Shore City's security camera footage. If it's good enough to create a digital record, it's good enough to make some decisions about how long you keep it, rather than letting budget constraints or change make the decision for you.
A minute? A day? Seven years? Forever?
"At what point is it unreasonable to say you should be able to produce a record? There is a level of judgment about whose interests are at stake," Roberts says.
"What is the life of information, what is the likelihood it will be accessed again after it is created, what are the implications of not having it there?"
Some of those obligations are specified in company and tax laws or the Public Records Act, which defines various classes of public records and in come cases requires express permission from the archivist before they are destroyed.
"Everything has some optimal use. The question is just when it tails off so the cost of continuing to maintain it outweighs its likely value," he says.
On a technical level, keeping up with the demands of digital storage is expensive. Archives NZ bypassed tape and went straight to disc, so it has a storage area network with upwards of 50 terabytes of data on board.
Roberts says while file compression may be appropriate in some circumstances, the archivists' preference is for a high-quality storage layer and a separate access layer so the database is not serving out huge files.
Roberts says the broader issues of digital continuity and digital information management are fundamental to any organisation, not just the public sector.
"There are basic questions like, are you capturing information in a form you can manage over time, that you know what it is when you get it back - is this the email that was sent or a draft?
"In the United States, questions of e-discovery in legal cases is one of the drivers of digital management."
Submissions on the draft standard close on June 8. Next month, Archives NZ is hosting the Future Perfect Digital Continuity Conference in Wellington, featuring three of the leaders in digital preservation: Canadian Seamus Ross, Andreas Rauber from Austria and Steve Bailey from the United Kingdom.
PUBLIC AND PRIVATE ACT
More than 100 people gave up a rare sunny Wellington Sunday to attend an internet New Zealand-backed meeting opposing secrecy surrounding the Anti-counterfeiting Trade Agreement.
Out of the meeting came the Wellington Declaration, which was to be presented to Acta negotiators who are meeting in the capital this week.
Michael Geist, a Canadian intellectual property law expert, addressed the meeting only days after telling a European Parliament committee in Brussels that claims by the European Commission and other negotiating parties that Acta would not lead to substantive law changes in Acta countries don't stack up.
Those changes could include cutting people's internet access if they are alleged to have downloaded copyrighted material.
Geist hopes New Zealand and other smaller countries will use the Wellington round to bring some transparency into the process by releasing an official draft of the agreement.
"The document is readily accessible to anyone with an internet connection, but governments refuse to comment on it," Geist says.
He says Acta seems to be about rewriting copyright and intellectual property rights law in a way that suits the interests of powerful groups in the US, groups which sidestep multilateral bodies like the World Intellectual Property Organisation.
ON THE WEB
publicacta.org.nz
