Experiential Media - Dok.fest 2015

Virtual Reality: From Storytelling to Story Experiencing

A brief history of media Chapter 1: One-Way Communication

•  1436: Gutenberg Press. Text.

•  1919: Radio broadcast. The spoken word.

•  1937: Television broadcast. Images, live video.

A brief history of media Chapter 2: Interactive Communication

•  1988: CD-ROM. Words, images, audio. Interact with content.

•  1994: Internet. Find what you want, interact with people!

•  1999: Internet capable phone. Anytime, anywhere access.

A brief history of media Chapter 3: Experiential Media

•  Now, 579 years after Gutenberg, the next chapter is being written: experiential media.

•  Jump inside a story and interact with the environment in full 360 degrees.

•  Transport your consciousness to another place, feel physically present in a story.

Like in a Star Trek Holodeck

What this means for journalism …

•  Don’t stop at writing or showing a story of what happened. Transport someone into the scene.

•  Move from taking 2D photos of 3D objects to scanning their structure to experience in a virtual reality headset.

•  Let people interact within a scene where something happened, and make their own choices.

Merging of technologies across industries

Film

Motion capture

Gaming

Interactive environments

Storytelling

Factual storytelling

Nonny de la Peña

Engadget: “The Godmother of Virtual reality.”

Oculus Rift

Palmer Luckey, USC lab assistant, launches Kickstarter campaign to mass produce the prototype he built. The name: Oculus Rift.

First version for sale now

Samsung GearVR, in partnership with Oculus. Samsung Galaxy Note 4 phone that works with a $250 headset.

How Oculus Works •  A screen overlaid with

two lenses focus a stereoscopic image to make it look 3D.

•  Tricks the brain into thinking it’s physically present elsewhere.

On Your Facebook? $2 Billion acquisition

“… we're going to make Oculus a platform for many other experiences. Imagine enjoying a court side seat at a game, studying in a classroom of students and teachers all over the world or consulting with a doctor face-to-face ...” – Mark Zuckerberg

In other words …

Virtual reality isn’t just a gaming platform. It’s the next communication platform, which makes it a new media platform.

In other words … “Just like we capture photos and videos today and then share them on the Internet to let others experience them too, we’ll be able to capture whole 3D scenes and create new environments and then share those with people as well. It will be pretty wild.”

4/14/2015, International Business Times: http://bit.ly/1zieWZU

More platforms on the way Valve / HTC Vive

Easy to set up, faceplant-proof walkaround VR.

On the horizon: Augmented Reality Microsoft HoloLens

Augmented Reality Magic Leap - backed by hundreds of millions from Google and Andreesen Horowitz. Possible future for Google Glass?

Depth cameras •  Depth cameras can see physical

structure and movement through physical space.

•  The photo or video of the future may contain 3D structure.

•  Down the line: potential for live, “transferred reality” of an environment.

What you look like to a depth

camera

Structure: the future of photos?

Structure Sensor

My summer at Gannett •  Hired as a consultant to

help with Google Glass and smartwatch projects.

•  After trying Oculus Rift they caught the VR bug.

Two approaches

1.  Virtually recreate the scene of a story to use as the primary navigational aid.

2. Transition into 360 videos of the same place. (Partnership with Total Cinema 360).

Two approaches

Notice how there’s no

“behind the camera” in 360!

Completely different way of

filming.

Impeccable attention to detail

“Transferred reality?”

Now, let’s look at Harvest of Change from the inside out.

What other stories could be experienced?

•  Environmental stories. Import topography of coastal areas, dial in sea level rise. Experience your future flooded city.

•  Science and medical stories. See how the new cancer drug works at the cellular level.

•  3D Data visualizations. Walk through the Polar Vortex on a 3D map.

•  Enter the crime scene. Examine the evidence.

OK, so how do you get started? •  Spent a year learning the ins and outs of game

design in Unity, with pointers from Nonny. •  Started new Virtual Reality Storytelling class in

January. 15 students made it through. •  Class materials are at http://vrstorytelling.org

OK, so how do you get started? •  Spent a year learning the ins and outs of game

design in Unity, with pointers from Nonny. •  Started new Virtual Reality Storytelling class in

January. 15 students made it through. •  Class materials are at http://vrstorytelling.org

ENOUGH TALK

LET’S MAKE A VR SCENE!

Part 1: Download the tools

http://www.vrstorytelling.org

(Click Downloads under Resources)

Part 2: Create a simple scene Find and place assets

Scan objects from your environment Add some interactivity

Part 3: Shoot 360 video

Stitching (broad overview) Placing multimedia in a VR scene

THANKS!

DAN PACHECO

Chair of Journalism Innovation,

S.I. Newhouse School, Syracuse University

Principal, FutureForecast Consulting, LLC

[email protected] | [email protected] | 303.465.5560 | @pachecod