New Zealand's electronic surveillance agency, the GCSB, has dramatically expanded its spying operations during the years of John Key's National Government and is automatically funnelling vast amounts of intelligence to the US National Security Agency, top-secret documents reveal.
Since 2009, the Government Communications Security Bureau intelligence base at Waihopai has moved to "full-take collection", indiscriminately intercepting Asia-Pacific communications and providing them en masse to the NSA through the controversial NSA intelligence system XKeyscore, which is used to monitor emails and internet browsing habits.
The documents, provided by US whistleblower whistleblower Edward Snowden, reveal that most of the targets are not security threats to New Zealand, as has been suggested by the Government.
Instead, the GCSB directs its spying against a surprising array of New Zealand's friends, trading partners and close Pacific neighbours. These countries' communications are supplied directly to the NSA and other Five Eyes agencies with little New Zealand oversight or decision-making, as a contribution to US worldwide surveillance.
The New Zealand revelations mirror what the Snowden documents showed in Europe, where the US and Britain were found to be spying on supposedly close and friendly neighbouring nations in the European Union.
The Herald has collaborated with US news site The Intercept to report on the New Zealand-oriented Snowden papers (read the Intercept article here). They reveal the secret activity called signals intelligence - the interception of private phone calls, emails and internet chats - globally.
The documents identify nearly two dozen countries that are intensively spied on by the GCSB. On the target list are most of New Zealand's Pacific neighbours, including small and vulnerable nations such as Tuvalu, Nauru, Kiribati and Samoa.
Other South Pacific GCSB targets are Vanuatu, the Solomon Islands, New Caledonia, Fiji, Tonga and French Polynesia. The spy agency intercepts the flows of communications between these countries and then breaks them down into individual emails, phone calls, social media messages and other types of communications. All this intelligence is immediately made available to the NSA, which is based in Maryland, near Washington, DC.
The South Pacific targeting was confirmed by a New Zealand intelligence source, who said the GCSB monitoring included Pacific government ministers and senior officials, government agencies, international organisations and non-government organisations.
Mr Key, who is also the Minister of National Security and Intelligence, has argued that the GCSB is needed to protect New Zealand from terrorism threats such as those emanating from Islamic State (Isis).
But the Snowden papers show that counter-terrorism is at most a minor part of the GCSB's operations. Most projects are assisting the US and allies to gather political and economic intelligence country-by-country around the world.
Monitoring the Pacific nations is part of New Zealand's role as a member of the Five Eyes intelligence alliance. Mr Key said in January that sending troops to Iraq this year was, for his Government, "the price of the club". He named the club as the US, Britain, Canada and Australia, the other Five Eyes members.
The same Anzus-era language is used to justify GCSB intelligence operations. An NSA report on the GCSB, found in the Snowden papers, said the GCSB provided "valuable access not otherwise available to satisfy US intelligence requirement".
In effect, the New Zealand spy agency gathers information on the country's nearest neighbours to help secure its place in the US-led alliance.
South Pacific spying has been greatly expanded in the past six years. A July 2009 GCSB document (scroll to end of article to see it) describes plans to move the Waihopai intelligence base to "full-take collection" - possibly the most important expansion since the station opened in 1989. Previously, according to 1990s GCSB staff, Waihopai intercepted millions of emails and phone calls from the Asia-Pacific region but retained and gave its allies only ones from specified targets. "Full-take collection" means the base now collects and retains everything it intercepts: both the content of all the messages and the "metadata" showing who is calling or emailing whom, at what times and (for mobile calls) location information showing from which cell tower the phone calls were made.
The 2009 report said the Waihopai engineers had been working to overcome problems in storage capacity and planned to have "full-take collection on Mission carriers running by October." Mission carriers refers to the large regional communications satellites that the Waihopai base is tasked with eavesdropping on.
In 2009, according to the report, two NSA trainers visited GCSB to help the Waihopai station upgrade its "Xkeyscore suite" in "anticipation of full-take collection and second-party sharing". Xkeyscore is the controversial US intelligence system that gathers and stores the billions of intercepted communications and "metadata" from all around the world, making them easily searchable by intelligence staff.
A 2008 NSA PowerPoint, obtained by Snowden and released publicly in 2013, included a slide headed "Where is Xkeyscore?" The accompanying map had red dots marking Xkeyscore sites around the world, including one on New Zealand's South Island. This was Waihopai.
Metadata is highly valued by the Five Eyes intelligence agencies as it allows a picture to be built of a person's activities and their network of friends and associates. It avoids the time-consuming effort of reading, listening to and/or translating each individual communication. The "second-party sharing" meant sharing the intelligence with the Five Eyes agencies.
These developments allow the Waihopai station, codenamed Ironsand, to collect, retain and share metadata and content for every communication it intercepts during its 24/7, year-by-year monitoring of Asia-Pacific communications satellites.
, which studied Snowden documents about Xkeyscore, wrote that "from the more than 500 million data communications to which the NSA has access every month, around 182 million of them are collected with the spying tool Xkeyscore". It noted that "the program also enables 'full-take' of all unfiltered data over a period of several days - meaning not just metadata but also the content of online communications."
All the Waihopai full-take intelligence was automatically shared with the "club": initially with the NSA and, the 2009 report said, "it is hoped that sharing with [the Australian and British sister agencies] DSD and GCHQ will be achieved soon after we can offer full-take collection data".
A British intelligence document a few years later (about 2011) said "GCSB have given us access to their XKS [Xkeyscore] deployments at Ironsand, a GCSB comsat [communications satellite] site which is rich in data for the South Pacific region". It said, "Specifically, we can access both strong selected data and full-take feed from this site." Strong selected data means communications contained targeted email addresses and key words. Full-take feed means everything intercepted from the region. Click here to read document excerpt
The GCSB intelligence collecting occurs completely within a US-run system. The documents show that, far from New Zealand retaining control over the intelligence it intercepts from its neighbours, the GCSB transfers it all directly into the US network. The intelligence is probably stored in computers at Waihopai, but Xkeyscore sites are part of an NSA-run distributed network of computer systems, the same as at any NSA-run listening posts.
The documents show that when GCSB staff want to access communications intercepted at Waihopai, they have to log into NSA computer databases. Minutes of a June 2009 meeting at the NSA headquarters, where a GCSB officer was present, show how integrated the GCSB is into the NSA systems. The GCSB officer, manager of an intelligence analysis unit, told the meeting that 20 per cent of GCSB's analytic workforce did not have accounts or access to key NSA databases.
"This is a particularly significant issue for GCSB," she said, "as they provide NSA with NZL [New Zealand] data which they have traditionally accessed via NSA tool/databases." That meant some GCSB analysts were "unable to query or access NZL data". Click here to read document excerpt
Spying on Samoa
An example of the routine South Pacific spying is GCSB monitoring of Samoa. The US-led Five Eyes alliance has allocated spying on Samoa to New Zealand, as part of what the July 2009 document calls the GCSB's "South West Pacific Area of Responsibility". The report, authored by the acting head of the GCSB's computer network exploitation unit, discusses changes to Pacific Island mobile phone systems that were creating problems for GCSB monitoring of "target telecommunications networks within GCSB's Area of Responsibility".
The report - headed Top Secret, Communications Intelligence, release to USA, AUS, CAN, GBR, NZL - expressed concern about a new undersea cable link connecting Samoa to the outside world. Previously Samoa channelled all its overseas communications via the Pacific Intelsat satellite, which was monitored at the GCSB's Waihopai facility. The undersea cable was taking over most Samoan international communications and so removing them from Waihopai's spying.
SNOWDEN AND NZ: FROM THE VAULT
• John Key 'comfortable' that NSA is not spying on NZ
• Intercept: NZ launched mass surveillance project while publicly denying it
• Edward Snowden: New Zealand's Prime Minister isn't telling the truth
The GCSB report said: "Unfortunately, SIGINT [signals intelligence] has already lost access to Samoan bearers due to the [recently installed] American Samoa-Hawaii cable. In all likelihood all but some backup carriers will be off the air by the end of the year."
The GCSB had got help from the New Zealand Defence Force to monitor a commercial cable-laying ship, the Ile De Re, that was installing the new undersea cable. Defence staff in the Joint Electronic Warfare Support Facility used Defence Force resources to track the ship day by day in March 2009 to provide information to GCSB on the progress of the Samoan cable.
When the Prime Minister of Samoa, Tuilaepa Lupesoliai Sailele Malielegaoi, visited New Zealand in October 2012, Mr Key said: "Samoa is the only country in the world with which New Zealand has a formal Treaty of Friendship." The treaty had "been at the heart of our relationship ever since". But the Five Eyes obligations trumped this and Samoa continues to be monitored as part of the GCSB's area of responsibility.
It is the same with all the other South Pacific countries. The same 2009 GCSB report on "target telecommunications networks" discussed mobile phone networks in Nauru and Kiribati. By 2015 Samoa, Vanuatu, Fiji and Tonga have undersea cable communications but all the other Pacific Island nations still use satellites that are monitored by GCSB.
Fiji phone taps
Australia and New Zealand collaborate closely on South Pacific spying operations. A GCSB report on "continued effort against the South Pacific region" at the June 2009 NSA meeting said: "GCSB's access development activities [researching new communications to spy on] will be focused on the South Pacific region and entail close partnering and engagement with DSD, NZSIS and ASIS." Click here to read document excerpt
DSD is the Australian Five Eyes agency (since renamed ASD, the Australian Signals Directorate) and ASIS is the Australian equivalent of the US Central Intelligence Agency (CIA). NZSIS is New Zealand's Security Intelligence Service, a domestic intelligence agency which has in recent years been expanding into some overseas operations. The same minutes discussed "pushing the priority up on GCSB [undersea] cable access effort and capabilities".
The July 2009 report said GCSB staff had provided all their information on Fijian communications to the Australian DSD's Military Support Unit that year. This was "to provide a Target Systems Analysis on the Command, Control and Communications of the Fiji Government ... Up until now, GCSB's major targets in the [Fiji] Government and [Fiji military] have kept a preference for Vodafone services", it said, but they were increasingly shifting to Digicel cellphones. This strongly suggests there was a listening post in the New Zealand or Australian high commission in Suva targeting local mobile calls.
In the same way that the Five Eyes alliance allocates the southwest Pacific to GCSB, the Australian ASD is allocated surveillance of Indonesia and Southeast Asia. The report discussed a GCSB officer seconded to Canberra to work in the Australian agency's Network Infrastructure Analysis section. His job was to assist spying operations by studying Indonesian cellphone firm Telkomsel.
Overall, the leaked documents suggest an astonishing lack of independence in New Zealand intelligence operations. The Government claims - most recently in its successful bid for a seat on the United Nations Security Council - that it runs an independent foreign policy. The GCSB and allied documents suggest the opposite.
Some of the Pacific spying - and other operations further afield - provide intelligence of use to the New Zealand Government. But GCSB operations are primarily contributions to the NSA and other allies: the price of the club.
2009 report from the GCSB's signal intelligence division discussing interception and training on NSA's XKeyscore system (app users tap here to view)
The Government Communications Security Bureau. This is New Zealand's electronic surveillance agency, which is tasked with collecting foreign intelligence. It is New Zealand's contribution to the Five Eyes network.
Five Eyes: The Five Eyes are made up of the intelligence agencies of Australia, Canada, New Zealand, the United Kingdom and the United States. Its roots go back to a post-WWII pact between the US and UK, which included three Commonwealth nations. The US is the lead partner - the others, "secondary" partners.
NSA: The United States' National Security Agency. This is the US electronic surveillance agency and the lead partner in the Five Eyes network.
XKeyscore: An NSA computer program which is able to search through the majority of communications around the globe which have been harvested largely by Five Eyes partners.
Ironsand: Codename for the Waihopai GCSB base, at the top of the South Island in New Zealand. It is a signal intelligence base.
Edward Snowden: The former NSA contractor who walked out in 2012 with a massive number of files, citing concerns about the extent and style of US-led surveillance. He is currently living in exile in Russia.
The Intercept: An online news site, largely led by journalist and lawyer Glenn Greenwald. It was Greenwald who Snowden approached with his trove of data.
• Nicky Hager is a New Zealand-based investigative journalist and an internationally recognised expert on surveillance since the publication of his ground-breaking book Secret Power in 1996. Ryan Gallagher (@rj_gallagher) is an award-winning Scottish journalist whose work at The Intercept is focused on government surveillance, technology and civil liberties.