Click here to load reader
Click here to load reader
May 14, 2020
Intermodulation: Improvisation and Collaborative Art Practice for HCI Laewoo (Leo) Kang
Information Science, Cornell University, Ithaca, NY
Steven J. Jackson Information Science, Cornell
University, Ithaca, NY [email protected]
Phoebe Sengers Information Science, Cornell
University, Ithaca, NY [email protected]
ABSTRACT This paper integrates theory, ethnography, and collaborative artwork to explore improvisational activity as both topic and tool of multidisciplinary HCI inquiry. Building on theories of improvisation drawn from art, music, HCI and social science, and two ethnographic studies based on interviews, participant observation and collaborative art practice, we seek to elucidate the improvisational nature of practice in both art and ordinary action, including human- computer interaction. We identify five key features of improvisational action – reflexivity, transgression, tension, listening, and interdependence – and show how these can deepen and extend both linear and open-ended methodologies in HCI and design. We conclude by highlighting collaborative engagement based on ‘intermodulation’ as a tool of multidisciplinary inquiry for HCI research and design. Author Keywords improvisation; art practice; creativity; collaboration.
“This conviction that direct deed is the most meaningful reflection, I believe, has prompted the evolution of the extremely severe and unique disciplines of the jazz or improvisational musician… Aside from the weighty technical problem of collective coherent thinking, there is the very human, even social need for sympathy from all members to bend for the common result.”
- Bill Evans, liner notes to Kind of Blue  INTRODUCTION The introduction of artistic methods and processes into academic research can enrich and enliven both research and art. In the HCI research community, art-based methods have begun to push and refigure core HCI notions of collaboration, design, and arguably computing itself (see, inter alia, [18,28,32,33]). Artists’ creative processes and
outcomes can help us to see and imagine opportunities and dimensions of technology and design that may elude more behavioral or engineering models [5,17,41]. In recent years, a range of artistic practices – from performance-based, to collective making, to adversarial engagement - have been highlighted as types of research methods that may draw on, widen, and extend interdisciplinary and experimental approaches [6,9,31,40,61].
This paper builds on this work to explore the forms of improvisational and open-ended learning commonly found in collaborative art and musical practices, and their potential as methods for HCI research and inquiry. Rather than casting improvisational art practices primarily as a step on the way to more systematic forms of thinking , as mysterious and personal matters of voice and expression , or as inscrutable sources of inspiration for more recognized forms of scholarly work, here we explore their potential as valid and generative research sites and activities in their own right. Our goal is to further understanding of the potential and nature of such artistic approaches to generating insight for HCI research. We do so by exploring existing theory and two sets of empirical studies based on the study team’s ethnography and collective art practice.
Derived from a Latin word meaning “unforeseen”, improvisation refers broadly to the practice of composing or inventing extemporaneously, through some kind of responsive and situationally-dependent departure from pre- formed plans or expectation [44,57]. More recently, the word has been used to indicate a process of “un- premeditated” composition or performance across a wide range of fields or endeavors. Outside the worlds of musical and artistic performance, the language of improvisation has sometimes been adopted to underscore the indeterminate and evolutionary qualities of everyday human behavior, and as such has begun to challenge and inspire work across a broad range of disciplines: from anthropology, economics and cognitive science; to architecture and urban planning; to mechanical and robot engineering [27,42].
HCI work to date has explored improvisation across several dimensions. This includes theoretical analysis around forms of complexity and uncertainty in human-computer interactions and the emergent and “circumstantial” character of human action [2,9,43,45,58]; and as a creative methodological approach wherein improvisational practices
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected] CHI 2018, April 21–26, 2018, Montreal, QC, Canada © 2018 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-5620-6/18/04…$15.00 https://doi.org/10.1145/3173574.3173734
CHI 2018 Honourable Mention CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 160 Page 1
are employed in the service of ideation, discovery, and the elicitation of user preferences and worldviews [24,25,52,59].
This paper seeks to extend these lines of analysis in several key directions. Drawing from an integrated program of theory, ethnography, and collaborative art production, we argue for improvisation as (a) a form of active learning that fosters emerging creativity through tension between structure and freedom, and (b) a holistic and complex socio- material practice that transcends and transforms the experience and capabilities of individual creative actors. We identify five key features of collaborative improvisational practice – reflexivity, tension, transgression, listening and interdependence – and explain their relevance to other linear and open-ended methods in HCI and design research. Finally, we explain how a process of intermodulation can be one way to explore multidisciplinary creativity and knowledge for HCI design.
The paper that follows proceeds in five main stages. First, we review approaches to improvisation in art and music to understand how improvisation functions as an ordinary feature of human learning and collaborative creativity. Next, we review related HCI work that explores the nature of improvisation both theoretically and methodologically. We then report on two ethnographic and improvisational projects of our own. The first involves an extended interview and observational study with fifteen media artists and musicians who employ improvisation as central methods of creative work. The second, ‘Intermodulation,’ involves a series of collaborative improvisations undertaken in conjunction with three groups of amateur and professional musicians, culminating in a series of public audio-visual performances. In discussion, we outline five key dimensions of improvisational action, and explain how this understanding of improvisation can challenge and extend existing HCI theory and methods. The paper concludes by exploring the potential of improvisational methods for multi-disciplinary HCI inquiry. IMPROVISATION IN ART AND MUSIC Improvisation as a process of emergent and un-scripted behavior is broadly accepted in the art and music fields, especially in contemporary art and jazz, as a way of producing more effective, open-ended, and sometimes participatory aesthetic and creative outputs. When individuals or groups are working in an improvisational mode, their actions do not follow a linear and predetermined plan, but are taken contingently in response to the emerging contours of real-time situations. Take for example, Picasso’s creative process, as captured in the film ‘The Mystery of Picasso ,’ a five hour time-lapse video shot in Picasso’s studio. As Sawyer  describes:
In his studio, Picasso is painting free-form, without preconceived image or composition; he is experimenting with colors, forms, and moods. He starts with a figure of a reclining nude, but then loses interest, and the curve of the
woman’s leg reminds him of a matador’s leg as he flies through the air after being gorged by bull - so he paints over the nude and creates an image of a bull and matador. But this leads him to yet another idea; he paints over the bullfight image and begins work on a Mediterranean harbor - with water-skiers, bathers in bikinis, and a picturesque hilltop village. The free-form inspiration continues. Five hours later, Picasso stops and declares that he will have to discard the canvas - it has not worked. But time was not wasted- he has discovered some new ideas, ideas that have emerged from his interaction with the canvas, ideas that he can use in his next painting. Picasso says, “Now that I begin to see where I’m going with it, I will take a new canvas and start again.”
Figure 1. Pablo Picasso’s process in ‘The Mystery of Picasso’
From a woman’s leg to a Mediterranean harbor to “I will take a new canvas and start again”, Picasso’s idea does not follow a linear progression or blueprint, but continuously changes through both purposeful and accidental interactions with the circumstances around it. Such u