The NSA, in its current form, owes less to 1984 than to 1991, the year the USSR dissolved and Congress passed the High-Performance Computing Act, the bill that funded the development of the US’s fiber optic network and the Mosaic web browser. The end of the cold war and the beginning of the internet, almost universally celebrated, were a disaster for the US intelligence establishment.

The NSA has been eavesdropping on long-distance communications since its inception in 1952, tapping underwater phone cables and picking up radio waves. But it didn’t really hit its stride until the introduction of international satellite communications in the ’60s made it possible to listen in on any long-distance signal you wanted, if you were in the right place and had a big enough antenna. In its baroque phase, during the defense buildup of the ’80s, the agency had dozens of listening stations—on top of the Teufelsberg in West Berlin, in the Arctic Circle, in the middle of the Australian Outback—filled with lonely men who listened to signals from the Russians. Some of them wrote poetry. Some of them committed suicide.

By the mid-’90s most long-distance communications had shifted from satellites to the cheaper and faster medium of fiber optic cables. The new network and the industry it fostered threatened almost all the NSA’s activities. Although the NSA had been responsible for much of the technology that led to the commercial internet, its own systems were ill-equipped to process the amount and the kind of information being generated by computer users. Its code-breaking preeminence was jeopardized by the development of cheap and virtually unbreakable encryption. Even Cray, its supplier of customized supercomputers, verged on bankruptcy, as university research labs, the only other customers for these giant machines, began to shift to parallel processing systems.

The NSA’s first response to this new set of circumstances was hostility. It reluctantly shut down many of its secret listening stations, and it pressed Congress to ban export licenses for encryption-protected computers. One official told Seymour Hersh that rogue groups within the organization had declared they would not monitor any communications signal newer than radio waves. Another said that his main contingency plan for field intelligence involved hiring local fixers to cut the hated fiber optic cables and force communications signals “up in the air,” like grouse.

As a massive, slow-moving bureaucracy designed to fight a giant, stationary enemy that no longer existed, the NSA was exactly the sort of government institution that Bill Clinton had promised to streamline once he came into office. Beyond shrinking costs, his administration wanted to simplify bureaucracy along the lines suggested by David Osbourne and Ted Gaebler’s 1992 book Reinventing Government: to make it smaller and more transparent (and “a catalyst for new ideas”) by reducing the number of government employees, cutting down on pension and benefit obligations, and moving federal money away from large-scale, long-term commitments. These ideas were a continuation of Reagan’s push for decentralization a decade earlier, but the Clinton Administration was the first to apply them to the defense and intelligence sectors. By 1999, the NSA had laid off or given early retirement to almost a third of its workforce.

At the same time, the NSA was suffering from what it referred to as its “Enemy of the State problem.” As an employer, it was unpopular among young people, who were likely to have formed their impressions of the agency from the 1998 movie in which the NSA hunts a fugitive Will Smith and steals his blender. At the peak of the dot-com bubble, when tech startups were hiring programmers straight out of school for three times a government salary and the chance to work in an open-plan office with a beanbag chair, the computer scientists and engineers who remained at the NSA began to abandon it for the private sector.

By the end of the decade, the NSA had developed a response. Its new strategy, outlined in a secret thirty-five-page transition memo presented to Bush on his first day in office and later declassified, was sweeping and practical, saturated in the New Management rhetoric of the ’90s. Terms like “stakeholders,” “strategic alliances,” “best practices,” and “return on investment” all appear alongside the byword of the dot-com era (“synergy”). The report identified “information” as the “product” that it delivered to its “customers” in other government agencies. The agency, the report promised, would continue to pension off its veteran analysts or get them hired by private contractors (“Operation Soft Landing”), bring in a chief financial officer and an information technology officer from outside the agency, and institute a new corporate information management system. Under the heading “Changes in Ethos,” the memo noted that the NSA director had recently acquired his own email address.

The report also contained a warning: “The Information Age will cause us to rethink and reapply the procedures, policies, and authorities born in an earlier electronic surveillance environment.” One of the first ways the change made itself felt was the decision to outsource its IT systems. Bidding consortiums made up of almost every private intelligence contractor and every telecommunications service provider in the country competed not only to supply the NSA with computers and telephones but to provide the service to keep them running. The Groundbreaker contract, as it was called, awarded in the summer of 2001 to a group of companies called the Eagle Alliance, was reported as a successful public-private partnership, one of the largest in the government’s history. Hundreds of NSA staff left work one night as salaried employees and returned the next day as contractors.

Several years later, Joseph Nacchio, the CEO of Qwest, accused the NSA of trying to use the Groundbreaker bidding process as a chance to gain backdoor access to his company’s fiber optic networks. He said the NSA had contacted him in February 2001—that is to say, before the September 11 attacks—looking for ways to access Americans’ phone records without a warrant. Nacchio had credibility problems, since his allegations were first made public when he appealed his conviction for insider trading. The SEC had charged him with dumping Qwest stock after he learned the company would miss its revenue targets by a wide margin. Nacchio said Qwest only missed its revenue targets because the NSA, which wanted to punish him for his refusal to play along, dropped a lucrative contract. Given what we now know about the NSA’s relations with private companies, Nacchio’s accusations seem closer to the truth than they once did. The NSA was interested in more than the companies’ expertise; it was interested in their information.

After September 11, contract hiring accelerated as the quickest way to build up agencies that had been scaled back in the previous decade. By 2006, 70 percent of the intelligence budget—about $42 billion a year—was spent on private contracts.

The rapid growth of the intelligence and defense industries has made the area around Washington DC one of the only places where someone can show up with a new BA in communications or international relations, get hired at a full-time entry-level job, and sign the lease on a $2,000-a-month loft apartment the next day. Between 2000 and 2010, as federal spending on contracts almost tripled, the area’s population increased by three-quarters of a million people, most of them under 35.

Like Washington DC, the NSA has been turned over to the very young. After hollowing out its mid-level workforce in the 1990s and then rushing to restaff in the early 2000s, the agency is divided between the inexperienced and the nearly retired. More than half its intelligence analysts—“the least experienced analytic cadre since the formation of the intelligence community in 1947,” according to an NSA-commissioned study on the agency’s internal culture—have been hired since September 11.

Contract hiring has helped the NSA to fill this demographic gap. One of the best places to find NSA code words is LinkedIn, where “Top Secret” clearance holders looking for work advertise their proficiency in the classified database software developed for the agency and its contractors: ANCHORY, NUCLEON, PINWALE, MAINWAY, MARINA. It’s not surprising that they spend so much time looking for work; under the contract system, most projects that can be are broken up into discrete parts and farmed out on a finite schedule, leaving government employees tasked mostly with managing contracts and providing information to other government departments using systems developed for them by contractors. In another manifestation of the revolving door between government and industry, many leave to work for contractors or to become independent consultants themselves as soon as they have enough experience to market their skills.

This constant fluctuation and lack of coordination contributes to the intelligence sector’s general disorder. As the Washington Post reported in its “Top Secret America” investigation in 2010, by that year 1,271 different government agencies and 1,931 private companies worked in the intelligence sector—a decentralized and unmonitored tangle of contractors, subcontractors, consultants, and tiny compartmentalized “Special Access Programs” (SAPs) that pursue their own secret ends. “There’s only one entity in the entire universe that has visibility on all SAPs—that’s God,” said James Clapper, the director of national intelligence, who has a knack for unreassuring public reassurances about the NSA’s surveillance.

In many ways, this disorganization and sprawl are now the NSA’s defining characteristics. Each revelation about the mass surveillance programs that has come to light over the past eleven years—and especially in the last six months—has unfolded along the lines of incompetence and often inadvertent illegality. In 2001, the Bush Administration instructed the Pentagon’s Defense Advanced Research Projects Agency (DARPA) to establish the Information Awareness Office. Bush thought of the office as “a new Manhattan Project”—its stated goal was to achieve Total Information Awareness. Playing a cruel joke on paranoids everywhere, the IAO’s logo depicted the Eastern Hemisphere surveilled by the Masonic eye from the dollar bill. Its research projects included HumanID, designed to identify individuals from up to five hundred feet away, and Communicator, which would allow soldiers to talk to computers, like Siri. But the Information Awareness Office also spent time trying to figure out how it was going to examine and analyze the enormous amount of data it was planning to collect.

Weeks after its launch, the secret office’s existence was leaked to the New York Times. Within a year, it had been defunded by Congress, although some of its data-mining programs were transferred in zombie form to the NSA. The NSA’s own massive data analysis program, Trailblazer—implemented by SAIC, Boeing, and Booz Allen—was abandoned in 2006, years behind schedule and running hundreds of millions of dollars over budget.

Since the quasi-legal structure built up around the PATRIOT Act has made it possible to collect everything, and the falling cost of digital storage has made it possible to stockpile everything, the NSA’s activities in effect have become indiscriminate electronic surveillance. The agency collects data from spliced fiber optic cables in the switching centers of all the major telecommunications companies; from PRISM requests to Apple, Google, and Facebook; from individual service providers; from the communications satellites still in operation; from mobile phones; from government records; from information that is free and publicly available to anyone online.

The NSA has a much easier time collecting information, and then collecting more information, than it does sorting and analyzing it, and it doesn’t really know how to stop. This imbalance can be partially attributed to the logic of the contract system, which directs more NSA funds to the pursuit of new technology from outsiders than to training programs for its own workforce. It also has to do with the tendency of automated data collection to turn itself into massive and indiscriminate data capture. Once you start collecting more information than you know what to do with, statistical analysis tools, which become more accurate with larger data sets, become a promise that the data will eventually sort itself for you, if you only collect enough of it.

This promise has driven the NSA’s collection of email and telephone metadata, now one of its main tools (and the subject of enormous criticism since Edward Snowden made his escape to Hong Kong). Although the NSA argues that collecting metadata—information about where a message originates and where it’s going—doesn’t violate the Fourth Amendment, the files that Snowden made public reveal that the NSA uses “contact chaining” to create maps of terrorist organizations on the basis of who is talking to whom, in the same way that Twitter and Facebook use network analysis to suggest new friends to their users.

Network analysis so far has not been a foolproof surveillance tool. Soon after September 11, the NSA created a data-mining program called Stellar Wind. By gaining access to bulk telephone metadata records, the program mapped all the phone numbers potential terrorists were calling as well as all the numbers these second-order phone numbers were calling. The resulting network maps were supposed to reveal complex social networks of potential terrorists. Invariably, the phone numbers called most frequently by terror suspects, which seemed to point to sinister hubs of activity, belonged not to radical mosques but to pizza delivery places.

The NSA is a not only a consumer of private companies’ information and information-sorting tools; it also acts as a packager and distributor of information to the rest of the executive branch. Its single most important information product is its contribution to the President’s Daily Brief, the compendium of possible national security threats ritualistically presented as the first item on the President’s schedule every morning. In addition to the White House, the NSA provides information to other government departments, some of which are listed on its website in answer to the frequently asked question, “Who are NSA’s customers?” The clients it mentions include the CIA, the State Department, and the chairman of the Joint Chiefs of Staff, as well as military departments in the field. Unacknowledged by this list are the NSA’s domestic customers: the FBI, the Department of Homeland Security, and, through them, the “fusion centers” that share information with local and regional police. Almost all these organizations are also looking for terrorists.

Digital technology and private contracts made the expansion of the NSA’s surveillance programs possible, but it is the political logic of the war on terror—threats are everywhere, so nobody can know about them, nor about the actions taken to address them—that has made that expansion desirable. Edward Snowden’s leaks uncovered the reach of NSA surveillance; they also revealed the extent to which the foreign policy work of the US and its allies has been consumed by the phantom threats of the war on terror. In London, the US worked with England to spy on delegates to the G20 summit, targeting Turks and South Africans in particular. In New York, the NSA tapped into the UN’s internal videoconferencing system.

These operations presumably will not help the government decode the global terrorist conspiracy, but maybe that is because there is no global terrorist conspiracy. A terrorist is a disgruntled aspiring boxer who bullies his little brother into helping out with a bombing. Or he is someone mouthing off on a radical discussion board who is slowly coaxed into buying the parts for a fertilizer bomb by undercover federal agents. These agents teach him how to put the bomb together, and they also help him select a target before making an arrest. If you are the NYPD, a terrorist might also be anyone who has attended prayers at one of a dozen city mosques that have been the object of “Terrorism Enterprise Investigations,” which stretch back for years and consider mosques themselves to be de facto terrorist organizations. In an editorial for the Wall Street Journal, Senator Dianne Feinstein, chair of the Senate Intelligence Committee, argued that the NSA’s metadata collection has to remain in place. “The US must remain vigilant against terrorist attacks against the homeland,” she wrote. She then quoted James Clapper, who at a Senate hearing testified that a number of phone numbers and emails collected overseas suggested an impending attack on the US. “Fortunately,” Feinstein wrote, “the NSA call-records program was used to check those leads and determined that there was no domestic aspect to the plotting.” So desperate is the NSA for terrorist plots to thwart that it must remain vigilant against the nonplotting of terrorists, who are determined not to strike on American soil. An intelligence apparatus ostensibly designed to prevent another September 11 is now also used to harass Occupy Wall Street protesters, keep a permanent eye on antifracking activists, and watch for illegal immigrants.

As it currently functions, the NSA is perfectly suited to fueling the prosecution of a war that has no real object and no boundaries. Appearing before the secret Foreign Intelligence Surveillance Court in 2009, the NSA made the following defense of its data-gathering programs: although the NSA had collected too much information, it didn’t mean to. The system was too complicated. None of the agency’s employees knew how to use it. The unnamed intelligence official who explained the NSA’s metadata collection to reporters this fall echoed this refrain, at once an admission of the agency’s guilt and a defense against it. “There was nobody at the NSA who really had a full understanding of how the program was operating at the time,” the official said. This is still true.