May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

History of the USA total surveillance efforts

Top Visited
Past week
Past month


Old News ;-)

How the US institutionalized surveillance

Al Jazeera America

Information is power. This is the logic - or at least the aspiration - behind the U.S. government's current approach to intelligence gathering: the more data (or metadata) in hand, the more control. The National Security Agency's surveillance leviathan, funded by a black budget and presided over by a star-chamber court, suctions up almost inconceivable amounts of material from around the world, including your phone and computer. How did this begin, and where will it end?

History shows us that this is a story about empire. For more than a century, major innovations in U.S. intelligence-collection capacity have accompanied major expansions of U.S. influence on the world stage. In some cases, U.S. government agencies used distant theaters to test approaches they would later deploy on the home front. Elsewhere, they helped foreign police build internal surveillance systems. The trainers then returned to work in domestic law enforcement, employing the same practices locally. Either way, U.S. residents should worry. The information-management strategies the U.S. has used in projecting its power abroad have usually come home to roost.

It was during the United States' bloody occupation of the Philippines after the Spanish-American War that U.S. policymakers first yoked intelligence collection to imperial expansion and then repatriated it. As the historian Alfred W. McCoy writes in "Policing America's Empire," U.S. colonial police, powered by a nascent information revolution and unfettered by constitutional restrictions, built an elaborate covert surveillance apparatus to help quell resistance. Their system maintained individual file cards on an astonishing 70 percent of the local population.

When the U.S. scaled down the occupation during World War I, veterans of the counterinsurgency effort, including the military intelligence pioneer Ralph Van Deman, returned to lead a large-scale ramp-up of domestic surveillance infrastructure, designed to provide the enforcement muscle for new legislation such as the 1917 Espionage Act and the 1918 Sedition Act. Among Van Deman's achievements: a collaboration between his military intelligence division and the American Protective League, a private network of 300,000 citizen spies that even after the war spent decades targeting German-Americans, repressing labor militancy, spying on civil rights activists and identifying Hollywood communists for blacklisting. (Van Deman also amassed a personal archive of file cards on a quarter-million suspected U.S. subversives.)

As fears about fascism and communism escalated during the 1930s, U.S. attentions turned outward again. Franklin Roosevelt's administration, eager to access the intelligence collected by foreign police forces, directed the Federal Bureau of Investigation to develop relationships with its counterparts abroad. The FBI helped countries such as Brazil and Colombia set up secret intelligence services from which the U.S. could profit, both by gaining access to foreign surveillance data and by honing strategies that could later be integrated into domestic police practices.

As the Cold War set in and the Central Intelligence Agency and the NSA were established (in 1947 and 1952, respectively), the U.S. not only stepped up its own intelligence collecting capabilities but also trained police forces in Japan, Greece and Uruguay, among others, in anti-communist counterinsurgency methods. Best dramatized by Costa-Gavras in the film "State of Siege," this proxy training aimed to build on-the-ground surveillance capacity, allowing local allies to share the work of Cold War containment and simultaneously guaranteeing the U.S. government access to the information their allies could now capture.

Take the case of Guatemala. There, soon after the CIA helped orchestrate the 1954 coup that ousted the democratically elected leftist President Jacobo Arbenz, U.S. trainers arrived to help the new military government consolidate power. Their first order of business, as one U.S. adviser reported back, was to help the Guatemalan police optimize its "almost neurotic hypersensitiveness to communist activity" by updating its "hopelessly inadequate" filing system. Simply put, to hunt down enemies of the state - to track their movements, record their political opinions, identify their associates, map their daily routes and, ultimately, eliminate them - you had to keep good files on them.

Arguably, the most lethal tools sent to Guatemala's police by the United States were not guns, munitions or helicopters but file cards and filing cabinets. U.S. technicians ran daily classes in records management for Guatemalan agents, teaching them the latest information management methods and supervising the creation of a new records bureau. But that wasn't all.

U.S. agencies, most famously the State Department's now defunct Office of Public Safety, provided filing cabinets for safe document storage, updated the Guatemalans' dated fingerprinting system, oversaw a transition to the use of three-by-five file cards, compiled blacklists of "subversives" and beefed up the police's special investigation squads. They also built a telecommunications center (connected to a Central America–wide system) that allowed the country's various security forces to share intelligence with one another, with neighboring countries and, most important, U.S. officials in the Panama Canal Zone.

Guatemalans soon came to know that center, tellingly, as "the archive." As the government's counterinsurgency campaign heated up, the police used its new archival capabilities to murder and disappear tens of thousands of students, trade unionists and opposition politicians - a surgical strategy enabled by modern information technology. Arguably, the most lethal tools sent to Guatemala's police by the United States were not guns, munitions or helicopters but file cards and filing cabinets.

At the same time, the U.S. trainers - back after stints in Guatemala, Vietnam, Colombia or the other countries where this template was used - adapted their expertise to domestic policing, either as private consultants or by integrating into metropolitan police forces in cities like Detroit and Chicago.

According to historians Jeremy Kuzmarov and Stuart Schrader, practices developed by the Office of Public Safety at the peripheries of the U.S. empire were repurposed locally to pacify the social unrest of the 1960s and 1970s.

The FBI's Cointelpro, a series of covert projects to spy on, infiltrate and discredit groups and individuals the FBI deemed suspect (including New Left and civil rights organizations), echoed the programs U.S. trainers had set up abroad. Strategies and tactics thus hopscotched across national borders and back, traveling anywhere the U.S. sought to increase its influence.

After 9/11 and the Afghanistan and Iraq invasions, the U.S.'s global intelligence-gathering system went into hyperdrive. In Afghanistan, U.S. forces have aimed to collect extensive biometric data on every single living Afghan for counterinsurgency purposes. Civil liberties watchdogs in the U.S., such as the Electronic Frontier Foundation and Public Intelligence, are concerned by how this emphasis on high-tech biometric collection has already taken root in domestic law enforcement. (Biometrics aren't inherently problematic, but their use can present very real privacy threats, and they can easily be repurposed for state or corporate surveillance.)

Enabled by new technologies, a new modus operandi has emerged as well. As one former U.S. intelligence official explained, "rather than look for a single needle in the haystack" - scanning for information on particular cases of interest - the new strategy is now to "collect the whole haystack." This began in earnest with the Real Time Regional Gateway program, implemented in Iraq and then in Afghanistan to vacuum up all possible information. The ethos of RTRG appeared in the U.S. in the form of the PRISM data-mining program. Americans were scandalized to learn from former National Security Agency contractor turned whistle-blower Edward Snowden that the whole haystack included their phone calls and emails. They should understand that this will remain the case for as long as the U.S. is permitted to maintain its amorphous campaign against "terror," the diffuse goals of which are now seen to require a blanket approach to information gathering.

Policing is, at its core, informational and archival in nature. High-octane data mining may have replaced the file card, but the underlying concept is the same. So long as the United States chooses to continue in its self-appointed role as global policeman, it will, necessarily, maintain what Snowden described before fleeing the country as a "massive surveillance machine" - nothing less than an archive of the world, the home front included.

Kirsten Weld is an assistant professor of history at Harvard University and the author of "Paper Cadavers: The Archives of Dictatorship in Guatemala."

Recommended Links

Google matched content

Softpanorama Recommended

Top articles




Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy


War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes


Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law


Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright © 1996-2021 by Softpanorama Society. was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site


The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019