You have to remember that the NSA was created after World War II to prevent another surprise attack. That was the whole raison d'etre for NSA: Pearl Harbor. We don't want another Pearl Harbor. — James Bamford

When al-Qaida terrorists attacked the World Trade Center and Pentagon on Sept. 11, 2001, the staff at the world's most technologically sophisticated intelligence agency learned about it from a news broadcast on a $300 TV set. Shortly afterward, a transformation began at the NSA that would irrevocably alter the way the U.S. government regarded its citizens' privacy, just as Americans faced choices about what freedoms to sacrifice to prevent more attacks on home soil.

"You have an agency that was created to prevent something like 9/11 — they are not allowed to turn their eyes and ears on America. But of course, the attack came from al-Qaida operatives who had been living inside America," said Michael Kirk, producer, director and writer of a new FRONTLINE documentary, "United States of Secrets," which premiered Tuesday on PBS.

The remedy, as we now know, was to turn an agency with a strict legal mandate to look outward at the activities of foreign governments inward, toward its own. "So the government was faced with a question: What do they have to do to walk right up to the edge of what's legal and right to protect the rest of the country?," Kirk, whose other film credits include the Peabody award-winning "League of Denial" and "Bush's War," told Yahoo News in a telephone interview.

Part 1 of "United States of Secrets," a two-hour film, examines the beginnings of "The Program," or "Stellar Wind" as it was known internally: The effort to collect unprecedented amounts of information from emails, phone and cell conversations from millions of Americans — through its continuation today, surviving a change in presidents, threats of resignations by high-level government officials and exposure by Edward Snowden, who took a job at intelligence contractor Booz Allen Hamilton to specifically reveal it.

Though the film begins with Snowden's efforts to interest former Guardian journalist Glenn Greenwald, documentary film producer Laura Poitras and journalist Barton Gellman with what he found, Part 1 is less about Snowden and more about how many members of the NSA, the U.S. Justice Department and other government officials acted on their various beliefs in what was right, and wrong.

The film depicts how "The Program" started in then-Vice President Dick Cheney's office, in the form of a document authored by David Addington, Cheney's lawyer, and kept in Addington's safe. "To most people who know how things work inside the government, that the vice president's lawyer, who has no statutory authority, [would author] what would become the deepest, darkest secret the American government has" is not only unusual, but unique in the history of U.S. intelligence, said Kirk. Its heritage caused intense and acute legal debate from the beginning.

Still, the NSA, asked after the 9/11 attacks to create a list of exactly what it would take to prevent anything like them from happening again, was receptive to the challenge. Michael Hayden, a retired four-star U.S. Air Force general and head of the agency from 1999 to 2005, took it on.

Starting up "The Program" required compartmentalizing it from a substantial part of NSA staff, and, of course, Congress. All kinds of equipment began appearing in the agency's Fort Meade, Md., headquarters, to the surprise of many who worked there. In fact, though, the agency already had a program called "Thinthread" that could be configured to do the job, but its creators included safeguards that would render the data more or less anonymous, protecting the privacy of the newly monitored.

But when the NSA decided to remove those protections, the longtime agency code writers behind it — one of whom breaks into tears on camera at the thought that 9/11 might have been prevented had Thinthread been properly used — decided to quickly retire. Their supervisor, Thomas Drake, who had joined the agency on the day of the 9/11 attacks, first unearthed the internal "skunkworks" that created Thinthread. But NSA executives wound up rejecting it and rebuffing Drake when he questioned the legality of "The Program" to his superiors. They assured him "The Program" was legal, and told him, "Don't ask any more questions, Mr. Drake." He eventually lost his job and pension at the NSA after talking to a reporter about waste and fraud at the agency, and as of summer 2013 was selling equipment in an Apple store.

Other stories of would-be whistle-blowers emerge in the film, based, says FRONTLINE, on interviews with more than 60 elected officials, journalists, intelligence insiders and Cabinet officials. What also emerges is that despite revelations in 2005 and 2006 in The New York Times, USA Today and other media — years before the world heard of Edward Snowden — the story never got legs, and "the program continues to operate all over the world and has not been stopped or dramatically altered," Kirk said.

Not only that, it continued to thrive despite a change in presidents. President Barack Obama, who repeatedly remarked during his campaign on the importance of governance under the rule of law, did little to rein the NSA in until the Snowden revelations. "More whistleblowers or leakers have been indicted under this president than all the other presidents put together, and that he would be that kind of president is surprising," Kirk said. Those who supported Obama hoping for change in how the government operates may now be wondering, "who is this guy in terms of what he promised once?" Kirk said.

The story, of course, doesn't yet have an ending. Snowden collected more than a million pages of documents, which likely contain many more revelations about NSA digital surveillance. "It's pretty clear that they have 'everything,'" Kirk said. "It's no longer six degrees of separation from Kevin Bacon, it's three steps, and then you have everybody."

Part 2 of "United States of Secrets" airs next Tuesday, May 20, on PBS.

[via FRONTLINE / Yahoo News]