The National Security Agency ended the program that collected email logs and timing, but not content, in 2011 because it decided it didn't effectively stop terrorist plots, according to the NSA's director, Gen. Keith Alexander, who also heads the U.S. Cyber Command. He said all data was purged in 2011.
Britain's Guardian newspaper on Thursday released documents detailing the collection, though the program was also described earlier this month by The Washington Post.
The latest revelation follows previous leaks from ex-NSA contractor Edward Snowden, who is presumed hiding at a Moscow airport transit area, waiting to hear whether Ecuador, Iceland or another country might grant him asylum. He fled Hong Kong over the weekend and flew to Russia after being charged with violating American espionage laws.
The collection appears similar to the gathering of U.S. phone records, and seems to overlap with the Prism surveillance program of foreigners on U.S. Internet servers, both revealed by Snowden. U.S. officials have said the phone records can only be checked for numbers dialed by a terrorist suspect overseas. According to the documents published by The Guardian on Thursday, the Internet records show whom they exchanged emails with and the specific numeric address assigned to a computer connected to the Internet, known as the IP, or Internet Protocol, address.
The program, described in a top secret draft report from the NSA inspector general, described the efforts of then-NSA Director Gen. Mike Hayden to fill gaps in intelligence gathering after the Sept. 11, 2001, attacks. One NSA officer quoted in the report described "NSA standing at the U.S. border looking outward for foreign threats" and "the FBI looking within the United States for domestic threats. But no one was looking at the foreign threats coming into the United States. That was the huge gap that NSA wanted to cover."
The draft added that the sweeping phone and Internet data-gathering programs were meant to speed up the process of surveillance of a terrorist suspect overseas, because "the average wait time was between four and six weeks" to get a court order from the Foreign Intelligence Surveillance Court. "Terrorists could have changed their telephone numbers or Internet addresses" before the NSA received permission to spy on them on U.S.-based phone or Internet systems.
Alexander said at a Baltimore conference on cybersecurity that the NSA decided to kill the Internet data gathering program because "it wasn't meeting what we needed and we thought we could better protect civil liberties and privacy by doing away with it."
He said the program was conducted under provisions of the Patriot Act, and that NSA leaders went to the Obama administration and Congress with the recommendation to shut it down.
Shawn Turner, a spokesman for the director of national intelligence, said the program has not resumed.
The Washington Post had described the Internet surveillance in an earlier report, without publishing the documents or releasing as many details. The Post described it as part of four secret surveillance programs - two aimed at phone and Internet metadata, while two more target contents of phone and Internet communications.
Alexander, who has been up on Capitol Hill frequently for hearings and meetings since the NSA phone and email surveillance was made public, laid out a broad defense of the programs.
He said he worries that more leaks are coming, adding that "every time a capability is revealed we lose our ability to track those targets."
While never mentioning Snowden by name, Alexander said his irresponsible releases of classified information "will have a long term detrimental impact on the intelligence community's ability to detect future attacks."
He declined to provide more details on what the NSA is doing to prevent such leaks in the future. He has said that the agency is changing passwords and improving its ability to track what system administrators are doing.
On Thursday, he said he was looking at how the leak happened and the people involved. He said the NSA can't do its job without contractors because it doesn't have all the talent or access it needs to do the job.