NCorreia, AVOL 2011-09-14

AVOL: Towards an Integrated Audiovisual Expression

ISEA Istanbul, 14.9.2011

Nuno N. Correia Aalto University, Media Lab Helsinki

Introduction

•  AVOL (AudioVisual Online) is an interactive audiovisual project for the Web,

•  by Portuguese new media artists Video Jack (Nuno N. Correia and André Carrilho).

•  It aims to allow for an integrated musical and visual expression, in a way that is playful to use and engaging to experience.

•  It was released in December 2007, and further developed until 2010.

•  Link: http://www.videojackstudios.com/avol/

Introduction

•  AVOL is one of the 4 projects chosen by the Portuguese Ministry of Culture for their net art portal.

•  It follows upon previous work by Video Jack, such as Heat Seeker (2006).

•  In previous Video Jack projects, the sound element was manipulated separately from the visual element.

•  In AVOL, Video Jack aimed to integrate the two elements – audio and image – under the same application and the same interface.

Introduction – Presentations

•  In 2008, AVOL was presented as installation at the following festivals: –  Cartes Flux, Espoo;

–  Re-New, Copenhagen;

–  Create, London;

–  Live Herring, Jyväskylä.

•  And as performance (also in 2008) at these festivals: –  Abertura Festival, Lisbon;

–  Electro-Mechanica Festival, St. Petersburg.

Introduction – Recent Developments

•  In 2010, 3 of the 28 audio loops were updated.

•  Also in 2010, video captures of AVOL were uploaded to Video Jack video websites (YouTube and Vimeo), and embedded on the Video Jack website.

•  AVOL music was also composed and embedded.

•  These additional media elements are meant to provide complementary ways for users to experience the project, and also to understand its possibilities.

•  http://www.videojackstudios.com/projects/avol/

Introduction – Contextualization

•  AVOL can be contextualized with other works that have explored integrated audiovisual expression.

•  In the early 20th century, Oskar Fischinger and Walther Ruttman created visual music films in Germany – a combination of tinted animation with live music (Moritz 1997).

•  John Whitney is “widely considered ‘the father of computer graphics’” for his explorations of computer-generated manipulation of visuals through mathematical functions (Paul 2003, p. 15).

Introduction – Contextualization

•  Golan Levin is one of the artists that have explored audiovisual creative expression, in works such as Audiovisual Environment Suite (AVES) (1998-2000), –  “an interactive software that allows for the creation and

manipulation of simultaneous visuals and sound in real time” (Paul 2003, p. 133).

•  In 2006, Nintendo released Electroplankton, by artist Toshio Iwai, –  a series of aquatic worlds “filled with different species of

plankton that can produce sound and light when you interact with them” (Davis 2006).

Design

Design – IAVOs

•  Visuals and music, together with the graphical user interface (GUI) that controls them, are grouped in 7 IAVOs (Interactive AudioVisual Objects).

•  Each IAVO contains 4 audiovisual options (“sets”), allowing users to create diversified combinations of music and visuals.

•  The visuals consist of abstract concentric vector animations that are audio-reactive.

•  The reactivity to each sound is based on the scaling of the respective animation proportionally to the sound amplitude.

Design – Sound

•  The sound loops were grouped into coherent entities as much as possible, similarly to band members on a stage.

•  Four of the loops are rhythmical (bass drum, snare drum, hats and clicks) and the remaining three melodic (keyboard, guitar, strings).

•  All loops have same tempo (120 bpm) and the same duration (16 seconds).

•  They were composed with the purpose of being coherent and harmonious independently of the active combination of sounds.

Design – Graphics

•  Each of the 4 “sets” of animations has its color palette and type of shape.

•  There was an interpretative differentiation according to the nature of the corresponding sound.

•  Despite these nuances, the animations were designed to be harmoniously interchangeable.

•  The use of vector graphics ensures that the animations are scalable and react fluidly to sound.

•  The animations, although abstract, suggest natural shapes.

Design – Interaction

•  The four white petal-shaped buttons trigger each of the four loops associated with the IAVO.

•  When users activate one of the loops, new elements appear on the IAVO’s interface: –  three “traffic light” buttons (stop, auto and solo),

–  and a draggable “ring” encompassing the “petal” buttons,

–  Incorporating minus and plus volume buttons.

•  The “petal” aesthetics of IAVO buttons is meant to be harmonious with the animations, which also resemble flowers.

Design – Interaction

IAVO diagram

Design – Comparisons

•  The animations in AVOL resemble John Whitney’s floral compositions: –  “all colors move into the ring simultaneously from all sides,

forming circles within circles all scintillating smoothly in a floral configuration” (Youngblood 1970, p. 220).

•  There is also some resemblance between AVOL’s flower-like objects and the plankton in Electroplankton, even more apparent when collisions occur.

Evaluation

Evaluation – Initial Conclusions

•  The author considers that AVOL was successful in terms of playfulness and engagement, due to:

–  the IAVO approach – entities composed of UI elements controlling sound and audio-reactive animations;

–  the harmony, coherence and mutual agreement of the audiovisual content.

Evaluation – Initial Conclusions

•  However, the author detects several shortcomings in AVOL: –  a limited amount of sounds and animations to interact with;

–  few audio manipulation options.

•  These shortcomings limit the versatility and expressiveness of AVOL.

•  A new project was released in 2010 to address these issues: AV Clash (http://www.avclash.com).

Evaluation – AV Clash

•  AV Clash connects to Freesound.org, retrieving nearly 240 sounds from the online database.

•  It contains a larger number of animations (96).

•  Audio manipulation capabilities include audio effects ('echo' and 'filter') and sound trimming.

•  The sound character is substantially different.

•  But there are several similarities between both, mainly in terms of visuals and interaction design

•  The number of IAVOs in AV Clash was reduced to 4.

Evaluation – Questionnaire

•  In the first half of 2011, an online questionnaire was setup to evaluate AV Clash.

•  The questionnaire contained closed and open-ended questions, and was answered by 22 anonymous respondents.

•  It included questions comparing AV Clash with AVOL.

Evaluation

16

5

1

Did you spend more time interacting with AV Clash or with

AVOL?

AV Clash

AVOL

Approximately the same time

Evaluation

•  Why did you spend more time interacting with AV Clash than AVOL? [16 users; multiple choice question]

–  73% [11]: larger amount of manipulation options;

–  40% [6]: larger amount of content;

–  47% [7]: other.

Evaluation

0

1

2

3

4

5

6

7

8

9

1-Not appealing at

all

2 3 4 5-Very appealing

Did you find the possibility of accessing more sounds and visuals in AV Clash appealing,

compared with AVOL?

Evaluation

0

1

2

3

4

5

6

7

8

9

1-Not interesting at

all

2 3 4 5-Very interesting

Comparing to AVOL, are the additional audio manipulation options in AV Clash interesting?

Evaluation

0

1

2

3

4

5

6

7

1-AVOL, much more

2 3 4 5-AV Clash, much more

Comparing the different sound and music approaches of AV Clash and AVOL, which one contributes to a more enjoyable experience?

Evaluation

•  Why did you prefer the sound and music approach of AVOL? [7 users; open-ended question]

–  synchronization of loops;

–  curation of sounds;

–  inclusion of percussive elements.

Evaluation

8

8

6

Which is easier / more intuitive to use, AV Clash or AVOL?

AV Clash

AVOL

They are on the same level regarding intuitiveness

Evaluation

•  Why is AVOL easier to use? [8 users; open-ended question]

–  6 users mention the simpler interface and fewer options.

Evaluation

13

2

5

2

Which project gives you a higher feeling of creativity?

AV Clash

AVOL

They both gave me the same feeling of creativity I didn't get any feeling of creativity from either

Evaluation

•  Why does AV Clash give you a higher feeling of creativity? [13 users; open-ended question]

–  7 mention more manipulation options,

–  1 mentions more variety in sound.

Evaluation

•  Why does AVOL give you a higher feeling of creativity? [2 users; open-ended question]

–  One user mentions that the sounds "fit together nicely", and that switching between them created interesting results.

Evaluation

•  Why did you get the same / no feeling of creativity from either? [7 users; open-ended question]

–  One user considers that both projects shape the sound and visuals too much to allow for his/her own creativity,

–  and another mentions that the projects are "too structured". 

Conclusions

Conclusions

•  The results show that users value the additional content and functionalities of AV Clash.

•  These additions contribute to a greater engagement than its predecessor.

•  However, extra manipulation options and added content come at a cost in terms of usability.

•  Moreover, AVOL comes close to AV Clash in terms of sonic enjoyment.

Conclusions

•  Therefore, even though the majority of respondents favor AV Clash,

•  there seems to be a significant number of users who prefer a simpler project such as AVOL,

•  with fewer options and content,

•  but with curated and more harmonious sounds.

•  A possible future path could concentrate more on playfulness and intuitiveness, targeting these users.

Conclusions

•  Some of the users are dissatisfied with the creative and expressive potential of both projects, –  considering that they are "too structured" and shape the

end result too much.

•  Further developments in this line of projects could address these limitations, –  expanding the manipulation options, content, and

customization possibilities.

Conclusions

•  Personal views on the future of related projects:

•  In future developments, emphasis should be given to recording and sharing capabilities. –  Net art audiovisual projects should take advantage of their

online presence to facilitate the recording and distribution of user-generated content, namely by leveraging social media.

•  Multi-touch mobile devices are attractive for future interactive audiovisual projects, –  since they allow for a more direct and flexible manipulation

of visuals than a pointing device interface.

References

•  Davis, R., 2006. Electroplankton Review. GameSpot, January. http://www.gamespot.com/ds/puzzle/electroplankton/review.html, referenced 22/08/2011.

•  Moritz, W., 1997. The Dream of Color Music and the Machines that Made it Possible. In Animation World Magazine, Apr. 1997. http://www.awn.com/mag/issue2.1/articles/moritz2.1.html, referenced 22/08/2011.

•  Paul, C., 2003. Digital Art. Thames & Hudson, London.

•  Youngblood, G., 1970. Expanded Cinema. P. Dutton & Co., New York. http://www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html, referenced 22/08/2011.

Thank you!

Any questions?

www.nunocorreia.com [email protected]

www.videojackstudios.com