Misha's Blog

Marina Abramović sat there all day, every day for weeks and just...looked. In some ways she was performing the role of surveillance camera.

More and more of us find ourselves living in the panopticon where the few surveil the many. If we earn the better vision of the future according to David Brin [1], we will all have access to surveillance data. In this possible future, we will each be able to employ what David Lyon [2] and Thomas Mathiesen [3] called the ‘Synopticon’ where the many can watch the few. The first problem in surveillance is always the question of who has access to the data. Eventually, when everyone has the data, the issue of computational power becomes primary so that the data can be sorted/mined. Finally, as the manicured data is delivered, our own perception of the data becomes primary.

As I described previously (here and here), I’m using computer vision software called SoYummy to create synopses (summaries) of videos. The software dissects and categorizes visual data and creates a set of the most interesting still images. The word synopsis itself means ‘seeing together’ and the summaries provide us with the ability to see the whole field of data together in a smaller representation. The concept of ‘seeing together’ at the root of the Synopticon is also essential in balancing state and corporate power: the watchers are themselves watched.

But what are we looking for? Currently, surveillance is mostly used to discover some kind of deviancy. I am intrigued by the inverse: normalcy. Luckily, the distance matrix generating tool that is part of the SoYummy suite compares a set of images and can be used to find the most normal and the most anomalous one. Driving this work are such questions as:

• What rules define and how relative is normalcy?
• What can we learn if we look for the most normal and most unique person?

I decided to use a set of portraits that was already available. I remembered this summer’s performance in NYC by Marina Abramović and the resulting photo portrait archive of the audience. What attracted me to this set is the active gaze of the participants. They are not ambivalent passerbys being caught on camera but are actively looking, even staring, at the artist. Marina herself, sitting almost motionless in the same seat in the gallery all day, every day for weeks, became a surveillance camera. In a way, she was only ingesting views of the visitors, as if it was life-giving sustenance. I harvested the portraits (taking out images of Marina herself) and strung them into a video.

I generated a distance matrix of all the portraits. The result was is a text file with space separated floating point numbers. There are 1490 portraits in the archive. Each image got a distance score to every other image, resulting in 1490^2 = 2226064 values. My Macbook Pro (2.8 GHz Intel Core 2 Due, 4 GB RAM) took days to complete this task. While the computer worked, I asked humans to give me their best guess of who the machine would pick out as the most normal and the most unique looking visitor. What I got was these two people:

A human picked this image as most 'visually normal' because this man participated and was photographed daily.

A human picked this image as the most visually anomalous because no one else wore a burka in their portrait.

Finally, my computer churned out the distance matrix. I took the average of each portrait’s distance from all the others. Here is the algorithmically picked most normal and most anomalous portraits:

Software Selected 'Most Normal' Person

Software Selected 'Most Unique' Person

One person in my class who I showed the results too immediately said “Your software is racist”. However, it is important to realize that our starting data set skews the results. I had a studio visit with the satyrical art world institutional critique painter William Pohida soon after this, and I showed him this project. According to him, this summary exposed the demographics of the MoMA audience, telling us more about that institution itself than the individuals.

The next step was clearly to contact Marina and ask her who she remembered the most and compare it to these results. Until I hear from her, I can only imagine that this software gives us some sense about what she remembers from the performance, and perhaps what images permeated her dreams. I wondered if she dreams of one single representative audience member. I decided to generate this epitome of the audience by finding the most normal set of eyes, the most normal nose, and the most normal mouth across all the portraits. I roughly sliced up the images into regions and generated distance matrix comparisons on all the eyes, all the noses, and all the mouths. Here is my approximation of what Marina dreams about now:

Algorithmically determined 'most normal' gallery goer overlaid with most normal eyes, nose, and mouth harvested across all gallery goers.

What can this exercise tell us about our own dreams? We are constantly barraged by media designed to affect us consciously and subconsciously (such as advertising). It is harder to detect and reason through the subconscious influence because it’s meant to alter our feelings and penetrate to the deeper levels of our psyche. They might even control our dreams. In order to deprogram ourselves from this psychological influence, a deconstruction of our dreams is necessary.

Above is a music video set to Nina Simone’s classic song about summary identity. Summarization software was used to perform social sorting on the actual features of audience participants at MoMA’s The Artist Is Present installation. The most normal facial features were excised and used to create a composited “most normal” portrait.

References:
1. David Brin, The Transparent Society
2. David Lyon, Surveillance after 9/11 available in full here.
3. Thomas Mathiesen, The viewer society: Michel Foucault’s “panopticon” revisited