The Obama administration had earlier conceded that its surveillance program scooped up more domestic phone calls and emails than Congress or a court authorised. But many details of the program's abuse were not known until today.
In a sweeping violation of court-imposed surveillance rules that went on daily between 2006 and 2009, the documents show the NSA tapped the bulk telephone records and compared them with thousands of others without "reasonable, articulable suspicion," the required legal standard.
The NSA told the Foreign Intelligence Surveillance Court it misunderstood restrictions on accessing data once it was archived, but Judge Reggie B. Walton wrote in a March 2009 order that such an interpretation of the court's orders "strains credulity". He was so fed up with the government's overreaching that he threatened to shutter the surveillance program.
After discovering government officials had been accessing domestic phone records for nearly three years without "reasonable, articulate suspicion" that they were connected to terrorism, the judge said in a blistering opinion that he had "lost confidence" in officials' ability to legally operate the program.
Walton noted, for instance, that just 1,935 phone numbers out of 17,835 on a list investigators were working with in early 2009 met the legal standard.
The judge ordered the NSA to conduct an "end-to-end" review of its processes and policies while also ordering closer monitoring of its activities.
Later in 2009, a Justice Department lawyer reported to the spy court a "likely violation" of NSA surveillance rules. The lawyer said that in some cases, it appeared the NSA was distributing the sensitive phone records by email to as many as 189 analysts, but only 53 were approved by the spy court to see them.
Walton wrote that he was "deeply troubled by the incidents," which he said occurred just weeks after the NSA had performed a major review of its internal practices because of the initial problems reported earlier in the year.
The judge said in November 2009 that on the same day the NSA counterterrorism office reminded employees they were not allowed to indiscriminately share phone records with co-workers - and one day after a similar reminder from the agency's lawyers - an NSA analyst improperly shared information with colleagues who were not approved by the intelligence court to see it.
Walton also noted that sometimes a US phone number would be reassigned by a phone company, and in such cases the NSA would scrutinise the records of an innocent customer. Walton called such cases "a source of concern by the court". He noted that, months earlier, the court ordered the NSA to explain more fully how it chooses which phone numbers to search and to delete any information that was improperly collected.
"This report was not sufficiently detailed to allay the court's concerns," Walton wrote. He ordered the NSA going forward to regularly tell the court the number of phone records searched, the time period when they could be searched, and details about how the NSA analysts were conducting searches suggested by results from other searches.
The hundreds of previously classified documents federal officials released today came in response to a lawsuit filed by the Electronic Frontier Foundation.
The Obama administration has been facing mounting pressure to reveal more details about the government's domestic surveillance program since a former intelligence contractor released documents showing massive trawling of domestic data by the NSA.
The data included domestic telephone numbers, calling patterns and the agency's collection of Americans' Internet user names, IP addresses and other metadata swept up in surveillance of foreign terror suspects.
The Obama administration's decision to release the documents comes just two weeks after it declassified three secret Foreign Intelligence Surveillance Court opinions - including one in response to a separate EFF lawsuit in the federal court in Washington. In that October 2011 opinion, Judge John D. Bates said he was troubled by at least three incidents over three years where government officials admitted to mistaken collection of domestic data.
The NSA's huge surveillance machine proved unwieldy even for the experts inside the agency. In a long report to the surveillance court in August 2009, the Obama administration blamed its mistakes on the complexity of the system and "a lack of shared understanding among the key stakeholders" about the scope of the surveillance.
Complexity has been a theme since the NSA leaks began this summer. Though Obama said Congress was briefed on the programs, members of Congress said they were surprised to learn how vast and intrusive the surveillance was. Even Rep. James Sensenbrenner, who sponsored the Patriot Act, said he never knew it would be used to sweep up phone records of every American.
- AP
