Subverting The Algorithm

Technology is neither good nor bad; nor is it neutral.” !

Melvin Kranzenberg

“

Gabi !Hi Everyone. I’m Gabi & this is Zach. We come from Boston where we both taught at the Dynamic Media Institute at the Massachusetts College of Art & Design. I’ll be continuing to do so this year, while Zach is deserting me for greener pastures up the street at MSU in the fall. !We also work in a design collective called Skeptic…so that might give you an idea of where things are going. !We’re going to go through some of our ideas and our work today and then give you a chance to share yours. We’ll start off by throwing a bunch of theory out there which we found extremely valuable to our development as educators and practitioners in this space. We’re going to add a list of resources to the etherpad, so please check that out. The link is on the AMC website, we’ll also put it up on the final slide—so you gotta stick around! !Also, during our talk, we’re going to touch upon issues of identity and self identification—and identities with which we may not feel comfortable being mapped on to us. We know that can be a sensitive subject to some, so please do not hesitate to step out if need be.

A L G O R I T H M

Zach !Why don’t we go around the room and have everyone say their name and tell us what you think about when you hear (or see, in this case) the word “algorithm.”

Zach !The definition that we’re using here for algorithm comes from a report done by Nicholas Diakopoulos from the Tow Center at the Columbia Journalism School called Algorithmic Accountability Reporting: On the Investigation of Black Boxes. !In the report, Nicholas (and we hope you don’t mind us using his first name here) notes that “An algorithm can be defined as a series of steps undertaken in order to solve a particular problem or accomplish a defined outcome.”

unspeakablevorn // Flickr

Zach !He goes on: !“The way you learned to do long division in grade school or the recipe you followed last night to cook dinner are examples of people executing algorithms.”

A L G O R I T H M

Gabi !Alright, so these algorithm things seem relatively benign. So the next question here is Why should we be trying to subvert them? !

A L G O R I T H M Sdetermine a lot of things about our lives.

Gabi !Well, firstly, Algorithms determine a lot of things about our lives. That is to say, stuff about us goes in, inferred stuff comes out. !In that Tow report, Nicholas writes that “algorithms adjudicate more and more consequential decisions in our lives.” !He goes on, “Algorithms, driven by vast troves of data, are the new power brokers in society.”

O F T E N ,they work without us ever knowing.

And that becomes even more problematic when we’re not aware of their use. !Matthew Fuller and Andrew Goffey write about something they call Evil Media, which is made up of grey zones of media. These are abstract infrastructures where something is being decided, calculated, suggested, or created.

The Suncoast News // http://suncoastnews.com/news/news/2010/oct/13/pn-er-delays-cause-patients-to-forgo-care-study-sa-ar-380221/

Take, for example, wait times at hospitals. Not just the estimated times you see on billboards, but also the ones mandated by hospital administration to the staff: “make sure patients are seen within X number of minutes.” !These aren’t just numbers that are decided based on personal experience. They’re personal experience, business requirements, logistical stipulations, and plenty of other parameters, all combined to come up with a number. In the case of this billboard, you’re being told what to expect based on that calculation, which will determine whether or not you go to emergency care. In the case of performance metrics, health professionals are being evaluated based on these thresholds. !The reason Fuller and Goffey call this a grey box instead of a black box is twofold: the first is that, visually, when you think about lines and lines and lines of code and database entries, it feels like a very large pile of grey. Secondly, though, this isn’t a black box where you are kept completely in the dark: you know that there are some kind of parameters affecting this decision. But it’s soft, it’s mushy. What exactly are the parameters? Where are the thresholds? Whose interests do those parameters and thresholds reflect? What is it REALLY? It’s hard to conceptualize. It’s grey.

A L G O R I T H M I C I D E N T I T I E S .

we are emerging out of these experiences with new, non-self-determined

Finally, when you consider our experiences with algorithms, it’s important to think about it in the way that John Chenney-Lippold writes about it in his paper, A New Algorithmic Identity. !This is something over which you have virtually no control.

We all probably know about Amazon.com’s recommendations engine. The site looks at what you’ve purchased, compares it to people that it has determined are like you, and people who have made similar purchases, and then makes recommendations on what else you might like. !This here is my current recommendations page. !It’s an easy punchline to call out the fact that, as a self-identifying male with no children, it is highly unlikely that I will ever buy a sports bra or a pair of Luvable Friends Girls Sandals—at least for now. But my wife made some purchases for herself and for friends, and now Amazon is throwing a whole slew of products at me. !But going beyond this relatively obvious user experience of supposedly “inaccurate” algorithmic inference, there’s a lot more going on here.

So here’s something that Amazon recommended to me. It’s a 6” basketball. I don’t play basketball, I don’t like it, I don’t have kids, and I don’t plan on having any in the very near future. !Of course, all I have to do is ignore this recommendation. And I do. !But that’s not what’s at stake here. What’s at stake is the realization that I am, whether I like it or not, someone who would buy a 6” basketball. Whatever that label is in the Amazon database, I have it. It’s on me.

Cheney-Lippold calls this “X=Male” and he uses the example of CNN. !CNN is, according to media surveys, primarily browsed by men. Everyone who goes to CNN, gets a cookie. When that cookie goes to another site, that site thinks you are probably a man, whether you self-identify as one or not. Whether you check a box elsewhere or not. And there’s nothing you can really do about it. You have no control over your algorithmic identity. !Your algorithmic identity is dynamic and responsive both to your actions and the actions of those whose entries are determined to be similar to yours. !And yet, your database entry, no matter how much its properties might change, is forever. !And that sucks.

S U B V E R Tthat shit.

Zach !So let’s subvert that shit. !When we talk about subverting the algorithm, we’re talking about doing so in a few ways.

O V E R F L O Wthat shit.

Maybe you overflow it with input.

E X P O S Ethat shit.

Maybe you expose it

R E V E R S Ethat shit.

or reverse engineer it.

S C R A M B L Ethat shit.

Maybe you scramble it.

I N T E R V E N Ethat shit.

Or intervene

B R E A Kthat shit.

Or break its expected patterns. !Here are some examples.

S Y M B O L I C

A R T S P A C Ein

The first is a project we did in a gallery installation earlier this year. It’s called Whisper. We use this project as a way to point towards future subversions of inference and recommendation algorithms. We installed the piece in a gallery, and as such, we consider it to be a symbolic subversion in an artspace. It probably falls into the “scramble” category.

So what is this Whisper thing? From a big picture standpoint, it is a fictional device that scrambles your data before other algorithms can use it to provide recommendations, facilitating surprise and serendipity in a world where your actions are increasingly algorithmically anticipated—in a world where, if all our objects are connected to the internet, they can all infer things about us and recommend things to us, curtailing the conditions of human possibility. Whisper subverts this possible future by intercepting data being transmitted from connected devices and objects, and alters it before it arrives at your home operating system or mobile device. In doing so, it aims to thwart the efforts of algorithmic inference and recommendation. !To demonstrate this idea in today’s world, we created a prototype to get a sense of how it might work and what the experience of having your data scrambled would be like. !So Gabi and I developed the Whisper “Stage 1 prototype” – a way for people to see how Whisper would work, demonstrating the manner in which Whisper treats the data it would intercept from your connected devices and objects. We framed it as a product demo that would eventually be brought to market— something that conveys the idea of what it would be like, without building out the entire thing. !The prototype consists of a hidden microcontroller, a USB microphone, and a small receipt printer. !Here’s a quick video of it in action: https://vimeo.com/85116051

First, a user says, “Whisper,” to initiate the interaction. Whisper then asks the user how she feels. The user the simply tells Whisper how she feels, beginning with, “I feel.” Whisper takes the last word in the string beginning with “I feel,” and applies an algorithm to that word, returning a scrambled, reinterpretation of it. At the same time, it orders a product on amazon.com, queried with the scrambled data, demonstrating the way in which it would interface and/or interfere with a product or service in the home. !This is Sofie. She couldn’t believe what Whisper did when she said, “I feel sexy.”

The receipt, intended here as a feedback mechanism illustrating how the scrambling software works, says the following: You said sexy. Whisper thought of aphrodisiac. Whisper thought of aphrodisiacal. Whisper thought of anaphrodisiac. Whisper thought of aphrodisiac. Whisper is searching for stimulant… Then, while it searches Amazon, it tells you, “I promise I am working… breaking algorithms takes time.” Then it reads: “Whisper ordered sunmark stool softener Plus stimulant laxative tablets $10.98.

Of course, we displayed the piece with a bunch of, what else? Amazon boxes.

S Y M B O L I C

R E A L S P A C Ein

This next project is symbolic, though it takes on a slightly tactical quality by taking place outside of the gallery in realspace.

Zach Blas // http://www.zachblas.info/projects/facial-weaponization-suite/

How many of you have seen Zach Blas’s Facial Weaponization Suite? !As Blas writes on his website (http://zachblas.info), the project “protests against biometric facial recognition–and the inequalities these technologies propagate–by making “collective masks” in community-based workshops that are modeled from the aggregated facial data of participants, resulting in amorphous masks that cannot be detected as human faces by biometric facial recognition technologies.” !So, to translate: you can sort of see in these masks generally the face—chin, lips, nose, eye sockets. What Zach did, was take data from 3D scans of faces and put it through his own multiplication algorithm. These are masks of faces, but certainly not normative ones—which is really what those facial recognition algorithms in security cameras and Facebook and all that are expecting.

Zach Blas // http://www.zachblas.info/projects/facial-weaponization-suite/

Here are the masks in realspace, undetected by facial recognition software. !In this photo, the pink masks are, as Zach calls them, “the Fag Face Masks, generated from the biometric facial data of many queer men’s faces, is a response to scientific studies that link determining sexual orientation through rapid facial recognition techniques.”

T A C T I C A L

R E A L S P A C Ein

Finally, since AMC is a place where theory and the symbolic intersect with the tactical, we wanted to show something done by Ricardo Dominguez and the Electronic Disturbance Theater.

The virtual sit-in uses software that hits a website over and over, purposefully asking a server for a page that does not exist. This, when done en-mass, will not only slow down a server, but—because of the way the project is programmed—if a human administrator actually goes in to check on what’s being “Not Found”, they will see hidden messages in the error logs. For instance, in this version, a protest demanding the release of prisoners in Tehran, the script looks for pages called things like “Justice” and “Equality”. That way, the algorithm that reports errors, notes that “Justice is not found” and “Equality is not found” when the system admin goes looking for the reason for the slow down.

Plot a current condition of the present into the future & design technologies that resist that condition.

Gabi !So now it’s your turn. We want to explore together other ways in which algorithms might wield influence over us in the future, and how we might work as individuals and as communities to subvert those forces. !We’ve put four prompts up on the walls. Each one highlights a quality of algorithms and tries to make the argument for its use—a ready acceptance of convenience at the cost of unforeseeable side effects. We think that inspiring criticality and affect in those who experience works of art is an act of resistance and protest. So we encourage you to come up with projects that challenge these assertions. !The way we do that is we plot our current condition and begin to project where that puts us in the future. Whisper, for instance, is a product that will eventually sit in your house, listening to the interactions of the connected products and services in your home, subtly and quietly interjecting its scrambling mechanism in order to reintroduce surprise and serendipity. !What you come up with in your work here today does not have to be a product or object or self-contained. Since all you have is markers and paper and about half an hour, it could basically be anything. !Also, the prompts are there to help. If, in your group, you decide that you have a problem with a specific algorithm and you want to try to break it, get at it. We’re not going to stop you. !Draw, write, build, cut, tear, even a performative solution would be awesome. We often use skits to illustrate a concept for a

R E F L E C T

G Ohttp://bit.ly/algoresist