Posted by: Patrick Allen Foster | August 5, 2013

Graphing Metadata

In The Origins of Totalitarianism (p. 433), Hannah Arendt writes:

The Okhrana, the Czarist predecessor of the GPU, is reported to have invented a filing system in which every suspect was noted on a large card in the center of which his name was surrounded by a red circle; his political friends were designated by smaller red circles and his nonpolitical acquaintances by green ones; brown circles indicated persons in contact with friends of the suspect but not known to him personally; cross-relationships between the suspect’s friends, political and nonpolitical, and the friends of his friends were indicated by lines between the respective circles. Obviously the limitations of this method are set only by the size of the filing cards, and, theoretically, a gigantic single sheet could show the relations and cross-relationships of the entire population. And this is the utopian goal of the totalitarian secret police.

This passage made me think of Kieran Healy’s post on using meta data to identify Paul Revere as a central member of the American Independence movement, based on his known acquaintanceship with men like Joseph Warren and Samuel Adams and on his membership in certain organizations. (I previously noted Healy’s post here.) I also thought of the Arendt passage when I read this post by Daniel Little:

One element of the NSA revelations of the past month is the apparent fact that the NSA’s PRISM program enables the agency to collect wholesale the transactions that occur on the Internet, including email header information. This follows the revelation that all metadata for phone calls made on the Verizon network (and presumably others) have been collected for a period of time — perhaps as long as seven years.

Here are the results for my own case. In the past three years I’ve transacted something like 60,000 email exchanges. The Immersion tool is able to analyze the metadata of this set of exchanges in a few minutes, and the resulting graphs are stunning. The vast and apparently disorderly library of messages turns out to have a very simple and revealing order. …

Graphs at the link. Some of Prof. Little’s conclusions (emphasis added):

What this implies to me is that the people who are most worried about the NSA’s wholesale data collection programs are probably right. The data that is being collected — phone and email metadata, perhaps auto license readers, credit card transactions, and electronic toll and transportation cards as well — this fund of data suffices to map out our personal lives in much, much greater detail than we would ever have imagined possible in a democratic society. We no longer have the luxury of “privacy through anonymity” or “haystack privacy”. Once there is full information recorded over time of our electronic transactions (including, remember, locational data from our phones), our lives can be played back in detail at any point. And the Immersion tool shows that the software exists to make sense of such vast databases, enabling agencies to produce customized and individually tailored “dossiers” for all of us.

It’s practically a utopian dream come true.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: