A report says NZSIS unlawfully accessed Customs' data for 17 years. Photo / Brett Phibbs
A report says NZSIS unlawfully accessed Customs' data for 17 years. Photo / Brett Phibbs
For years, the movements of New Zealand travellers were collected and scrutinised by intelligence officers working for the NZ Security Intelligence Service.
As you returned from holiday or business, or came home from backpacking, the entry details would be secured in a growing computer database operated by Customs NZ.
The information was a valuable collection of travellers' personal details showing who was travelling to, from or through New Zealand airports.
For citizens, it was another example of playing an unwitting part in the defence of our nation. Those travelling became part of the broad weave of data studied by intelligence staff for those who don't fit, or don't belong - those who are a risk to New Zealand.
For the NZSIS, the Customs NZ database was such an invaluable tool that it arranged for a terminal to be located at the NZSIS headquarters in Wellington where staff could search past travellers or put in place an alert for anyone of interest who might arrive.
"It is really critical to us," says NZSIS director-general Rebecca Kitteridge. "We were accessing it to make specific inquiries on individuals of concern.
"Especially when you have got ... the mobility of people of security concern - particularly on the terrorism front.
"You want to know who is entering your country and the legal advice I had was that it was lawful to carry on doing that."
SIS director Rebecca Kitteridge outside their headquarters in Wellington. Photo / Mark Mitchell
The discovery of the illegality and how it was resolved is another chapter in the emergence of our intelligence agencies into the modern world.
If anything, it is a signal of the extraordinary change our agencies have undergone in the past five years.
It was change that started with the discovery in 2012 of illegal spying on Kim Dotcom and Bram van der Kolk. As New Zealand residents they should have been protected from surveillance by the Government Communications Security Bureau (GCSB).
When the Dotcom fiasco emerged, then-Prime Minister John Key appointed Kitteridge to investigate the error and to find out what else was going wrong at the bureau.
Kitteridge was Cabinet secretary at the time. An accomplished lawyer, she was noted for her "old-fashioned constitutional lawyer" adherence to the letter of the law and the systems which stemmed from that.
As she reported in April 2013, the GCSB didn't understand its own law and was operating on a rule book that was an inaccurate distillation of its legislation.
There is much that was damning in Kitteridge's report.
There was its failed understanding of the law, meaning dozens of New Zealanders had been subject to illegal surveillance. It was found to have so many cones of silence that few knew what any others were doing. Inadequate staff were kept on because of concerns over security if they were let go.
One fact which those close to the bureau kept pointing to was Kitteridge's affirmation that none of this happened in bad faith. The bureau staff were dedicated servants of New Zealand and were devastated by the issues found.
Rather than attempting over-reach, the bureau had lost its way many years before and had little exposure since to other parts of the public service which may have indicated how awry it and its systems had become.
As it turns out, it's not just the GCSB which had an issue.
Kim Dotcom in the Auckland District Court in February 2016. Photo / Nick Reed
It was 2013. By then Edward Snowden had walked out of the United States' National Security Agency so the entire world was watching how the spies worked.
That was especially true for those watching intelligence agencies in the Five Eyes grouping of Australia, Canada, New Zealand, Britain and the United States.
Here, there was an open hearing into new legislation for the bureau. The GCSB was developing a "communications strategy". The Department of Prime Minister and Cabinet (DPMC) - the engine room of government - was building links between the intelligence agencies and other parts of the public service.
And Kitteridge had gone to work for the NZSIS as director.
There was a lot happening in the Government's intelligence and security sector and most of it was linked to updating old and overly isolated cultures which had been allowed to grow inside our intelligence agencies.
There was an urgency to doing so. The combination of Dotcom and Snowden, and public interest in the new law in 2013, meant the intelligence community had to emerge from the shadows.
That was especially the case as the war against Islamic State morphed from a battle in a foreign land to one which might be fought at home. The public needed to have confidence in the agencies tasked with keeping it safe.
And so there was a new oversight regime. Former Deputy Solicitor General Cheryl Gwyn was hired and provided with staff and a reasonable budget.
As the first of the new inquisitors, Gwyn was an inspired and unlikely choice by the Government - a top-flight lawyer with an unconventional background who showed no sign at all of ever being anyone's patsy.
But the problems with the intelligence community were deep-rooted.
That year, 2015, the State Services Commission report into the intelligence community said there were ""significant and necessary system improvements across all the agencies of the wider intelligence and security sector".
DPMC was still trying to get a handle on how out of kilter the intelligence services were, and was looking at hiring someone who could "dimension" the scale of the change project.
American NSA whistleblower Edward Snowden speaking at Kim Dotcom's The Moment of Truth event, held at the Auckland Town Hall before the 2014 election. Photo / Brett Phibbs
What hasn't really been understood is how it was both main intelligence agencies - and not just the GCSB - that were severely lacking.
The GCSB became an unfortunate byword for a lack of competence. Some wag set up a Twitter account dripping supposed intelligence titbits, usually mocking. Cartoonists portrayed them as hamfisted while Dotcom lampooned the bureau on social media.
In a speech about a month ago, Kitteridge talked of her time at the GCSB being one of "organisational distress" for the bureau.
"I saw people who were proud of their work, who were passionate New Zealanders, and who were motivated to do what Parliament had mandated them to do in the way that Parliament had intended.
"They knew that they were in difficulty and they were horrified about it. They were not at all defensive about working with me to identify the root causes."
The GCSB's problems were given a considerable public airing while the NZSIS quietly carried on fulfilling its role.
Kitteridge moved on from the GCSB and applied for the NZSIS job. Even though the GCSB and NZSIS share a building in Wellington, Kitteridge said she knew little of what the human intelligence agency did.
Aside from security clearance checks, few of us do knew much of anything about what the NZSIS did.
We know better now, with Gwyn having now issued at least five reports into the actions of the NZSIS.
Kitteridge, as director of the NZSIS, concedes this. "There were lots of similarities," she says of the GCSB and NZSIS.
"There wasn't a compliance team, there wasn't a compliance framework, very small legal team.
"And, actually, all of the agencies need more resources so they could strengthen those systems and processes … essential to provide confidence to the public that you're doing what the public expects you to do be doing."
She said it had been a similar process to that charted at the bureau. Funding has increased for the agencies, and each has compliance regimes on which the Inspector General has commented favourably.
"There's been a very significant effort to modernise, update, strengthen in the last three years."
Gwyn wrote about the "lack of precision and forthrightness" by the NZSIS when dealing with issues around Customs data. She "found the service reluctant in its engagement with me", had been "delayed and waylaid" while the NZSIS "obstructed" her ability to access "original and unredacted documents".
Kitteridge told the Herald that the NZSIS was flummoxed to be told it was operating a valuable database without the legal authority to do so.
"We had been operating on the basis of the law as it stood for a long time which was a very broadly stated exemption to the Privacy Act. Previous Inspector Generals had come and gone with our access being known.
"It came as a great surprise [when] this suggestion that it might not be lawful came up in 2014.
"Up until we received legal advice from Crown Law, we believed the activity was lawful."
But it's not just the Customs' database. Gwyn issued another report this week which saw a string of recommendations to the NZSIS around how it gave people warnings - an important issue for an agency with no powers to arrest or detain. The NZSIS has told Gwyn it doesn't intend taking up some of its recommendations.
Kitteridge says it's not a case of operational needs overtaking the law and claims there is no issue in apparent differences of opinion with the Inspector General.
Things have changed - and continue to change - in New Zealand's intelligence sector, says Andrew Little, the minister responsible for the NZSIS. Photo / Getty Images
There is the story told of a former Inspector General who, when walking to Parliament to deliver a report on one of the intelligence agencies, found himself at the wrong end of The Terrace. Confused about where the Beehive might be, the Inspector General was gently pointed towards the other end of the road.
It's an unkind parable intended to characterise the doddery, sleepy oversight to which the intelligence agencies were said to have had been subject.
The kind of inquiry report issued by Gwyn were unheard of a decade ago. Twenty years ago, there wasn't even legislation guiding what the GCSB could do.
Kitteridge's damning 2013 report into the GCSB is a master class of diplomacy. No single sentence would give a reader pause but the entirety of the report is shocking as an illustration of systemic breakdown.
Gwyn seems to have eschewed diplomatic niceties for the benefit of plain-speaking. It is not common to have oversight watchdogs speaking so bluntly.
And when Gwyn does speak so, the agencies' minister Andrew Little has reminded Kitteridge that she is obliged to listen.
He also mentions the " incomplete, inaccurate and misleading information" released by the NZSIS through the Official Information Act (OIA) which hit Labour during an election campaign.
"As a consequence of these things, there was big change."
That change has meant there have been issues identified with how the agencies work.
The issue with the Customs database is now resolved - a new intelligence law this year has made use of it legal.
There have also been clear guidance documents released spelling out rules for the way our spies work - even on issues such as surveillance in a public place.
It may once have been the Wild West, but now - as Kitteridge likes to say - it's not Homeland.
