NSA intent on using technology to have surveillance cover of the planet, scooping up data on friend and foe
The National Security Agency gathers intelligence to keep America safe. But leaked documents reveal the NSA's dark side - and show an agency intent on exploiting the digital revolution to the full.
US President Barack Obama hailed United Nations Secretary-General Ban Ki Moon as a "good friend" after the two had sat down in the White House in April to discuss the issues of the day: Syria and alleged chemical weapons attacks, North Korea, the Mideast peace process and climate change.
But long before Ban's limousine had passed through the White House gates for the meeting, the US Government knew what he was going to talk about, courtesy of the world's biggest eavesdropping organisation.
One NSA document - leaked to the Guardian by whistleblower Edward Snowden just a month after the meeting and reported in partnership with the New York Times - boasts how the spy agency had gained "access to UN Secretary-General talking points prior to meeting with Potus [President of the United States]".
The White House declined to comment on whether Obama had read the talking points in advance of the meeting.
Spying on Ban and others at the UN is in contravention of international law, and the US, forced on the defensive over the Snowden leaks, ordered an end to surveillance of the organisation, according to Reuters.
That the US spied on Ban is no great surprise. What is revealing is the disclosure is listed in the NSA's "top-secret" weekly report from around the world as an "operational highlight". It sits incongruously alongside other "operational highlights" from that week: details of an alleged Iranian chemical weapons programme; communications relating to an alleged chemical weapons attack in Syria and a report about the Mexican drug cartel, Los Zetas. Bracketing the benign, US-friendly Ban alongside drug traffickers and weapons in the Middle East and central Asia points to a spy agency that has lost its sense of proportion.
The incident is consistent with the portrait of the NSA that emerges from the tens of thousands of documents leaked by Snowden. Page after page shows the NSA engaged in the kind of intelligence gathering it would be expected to carry out: eavesdropping on Taleban insurgents planning attacks in remote Afghanistan valleys, or listening in on hostage-takers in Colombia.
But the documents reveal the NSA is indiscriminate in the information it is collecting. Nothing appears to be too small for it. Nothing too trivial. Rivals, enemies, allies and friends - US citizens and "non-Americans" - are all scooped up.
The documents show the NSA, intent on exploiting the communications revolution to the full, developing ever more intrusive programmes in pursuit of its ambition to have surveillance cover of the planet: total command of what the NSA refers to as the "digital battlefield".
When the NSA was founded in 1952, its task was primarily to target the Soviet Union. And so it did, until the fall of the Berlin Wall in 1989 and the end of the Cold War.
With the collapse of the Soviet Union, the NSA entered a decade of uncertainty. Morale slumped. The mood is caught in a document dated February 2001, only a few months before 9/11. In it, the agency admitted its capacity for intercepting electronic communications had been eroded during the 90s.
"NSA's workforce has been greying and shrinking. The operational tools have become antiquated and unable to handle the emerging signal structure," it says. "Ten years ago, we had a highly skilled workforce with intimate knowledge of the target and the tools to analyse the data. We have now reached the point of having a workforce where the majority of analysts have little-to-no experience."
Tellingly, in the light of the attacks on New York and Washington six months later, the document complained about a lack of linguists and analysts covering Afghanistan. The pool of experts covering Afghanistan as a whole were the same that "assist NSA's Office of Counter-terrorism in following the Taleban-Usama bin Laden relationship", it said.
The attacks on New York and Washington ended the NSA's decade of torpor. Suddenly, it found funding, and staff recruitment was no longer a problem. The NSA was one of the main beneficiaries of the doubling of the intelligence budget since 9/11.
Its proposed budget allocation for 2013 is US$10.8 billion ($13 billion), with 35,000 staff and bases in Georgia, Texas, Colorado, Hawaii and Utah adding to its headquarters at Fort Meade, Maryland. Its antennae can be found on the rooftops of 80 American embassies around the world.
It has large posts in the UK, Australia and Japan, but also operates elsewhere, sometimes covertly. In one country, Americans are secretly present at a base where exposure of their presence would provoke a major diplomatic incident, as it is in breach of an international treaty signed by the NSA's host nation. Agency staff visiting the base have to hide their identities, posing as contractors working on communications equipment and carrying fake business cards to back up their story.
The NSA refers to the people it serves as "external customers": the White House, the State Department, the CIA, the US mission to the UN, the Defence Intelligence Agency and others. During the Cold War, the NSA mainly targeted state institutions: the political, military and intelligence structures of Russia and eastern Europe. Today, the main targets - al-Qaeda and its related groups - are much more diffuse and elusive.
In the NSA's mission statement in its five-year plan, it insists Sigint (signals intelligence, or the interception of communications) will adhere to the highest standards. "Sigint professionals must hold the moral high ground, even as terrorists or dictators seek to exploit our freedoms. Some of our adversaries will say or do anything to advance their cause; we will not." Summing up the reason for its existence, it says: "Our mission is to answer questions about threatening activities that others mean to keep hidden."
But its scope goes well beyond that. It is hard to see where surveilling Ban or German Chancellor Angela Merkel fits into answering questions about "threatening activities". In August, Obama described the agency's remit purely as counter-terrorism. "We do not have an interest in doing anything other than that," he said.
Counter-terrorism has been the justification for huge budget increases, but the agency discloses in one leaked document that only 35 per cent of available resources are dedicated to the "global war on terrorism".
Obama later amended his statement. The NSA was not only engaged in counter-terrorism, he said, but also cyber-security and combating weapons of mass destruction.
The NSA's own list of strategic targets includes: support for US military in the field; gathering information about military technology; anticipating state instability; monitoring regional tensions; countering drug trafficking; gathering economic, political and diplomatic information; ensuring a steady and reliable energy supply for the US; and ensuring US economic advantage. It boasts that it can collect information from "virtually every country".
Hundreds of the documents show the NSA engaged in activities that would generally be applauded.
One credits the NSA's Texas base as intercepting 478 emails that helped to foil the Jihad Jane plot to kill Swedish artist Lars Vilks over his depiction of the Prophet Muhammad. Another shows the NSA, during a deadly takeover of the Intercontinental hotel in Kabul by the Haqqani group, able to listen in to what the gunmen were saying. There is an account, too, of the NSA's part in disrupting a human trafficking racket based in Fuzhou, China. It led to two arrests at New York's JFK airport. One of those allegedly carried details of the smuggling routes in his pocket.
Remote surroundings might fool some into thinking they are beyond snooping. An alleged cocaine smuggler might have thought he was relatively safe aboard a yacht in the Caribbean. But his partner, also on board, was chatting on Facebook, providing valuable information about the boat's location and planned landfall; information intercepted by one of the NSA's intelligence partners.
In 2009, the NSA was able to track almost every move made by Iran's Supreme Leader, Ayatollah Ali Khamenei, on a visit to Kurdistan province.
The most valuable service the NSA has provided for America and its allies since 9/11 is in support of the military in Iraq and Afghanistan. A 2007 NSA file, called "State of the Enterprise", is typical of many of the spy agency's documents which list wartime successes. "Specific results included the identification and location of a sniper targeting personnel inside the Baghdad Green Zone; the confirmation that a CIA asset was operating as a potential 'bad actor'." Other intelligence agencies such as the CIA complain privately about the degree of co-operation from the NSA in sharing intelligence, but in the end, like most other intelligence agencies, it is generally thankful for it. There are complaints, too, from soldiers in the field that live information is not always transferred to them fast enough, but they, too, express gratitude for snippets passed on about potential Taleban attacks.
The NSA, according to one document, overheard a Taleban figure, Mullah Rahimullah Akhund, known on the US military's kill-or-capture list by the codename Objective Squiz Incinerator, instructing an associate to buy and organise components for a roadside bomb, suicide vests and a Japanese motorbike.
The expansion in surveillance that accelerated under George W. Bush has continued under Obama. And this growth has not been matched by any corresponding reform of the legal framework or political oversight.
While there are frequent warnings in the documents reminding NSA staff of rules for protecting the privacy of Americans, other documents show repeated violations.
There is a gulf between what the NSA says in public and what it says in documents, in which technicians and analysts express their glee at finding novel ways of cracking into electronic communications and expanding their reach in ever more imaginative ways.
The question critics of the NSA ask is: just because it has the technical ability to do these things, should it?
One document shows the NSA engaged in a massive snooping operation targeting a UN climate change conference in Bali in 2007. The NSA's Australian base at Pine Gap wanted to collect the numbers of Indonesian security officials in case of a future emergency. "Highlights include the compromise of the mobile phone number" for one senior Balinese official, an NSA report boasted. "Site efforts revealed previously unknown Indonesian communications networks and postured us to increase collection in the event of a crisis."
This falls under the category of information that spies have always gathered. The rationale is: should there be an attack at the conference or some future outrage, such numbers could be valuable. The counter-argument is that Indonesia is a friend of the US and might be expected to share information in the event of an attack, so why does the NSA devote grand resources to harvesting such numbers?