Top Banner
STEM and Virtual Reality A/Professor Tomasz Bednarz QUT & CSIRO Data61 Prepared for QUT’s VC STEM Camp 2016
42

STEM Camp Virtual Reality

Apr 15, 2017

Download

Science

Tomasz Bednarz
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: STEM Camp Virtual Reality

STEM and Virtual RealityA/Professor Tomasz Bednarz

QUT & CSIRO Data61

Prepared for QUT’s VC STEM Camp 2016

Page 2: STEM Camp Virtual Reality

https://immersivelifeblog.files.wordpress.com/2015/04/vr_history.jpg

Page 3: STEM Camp Virtual Reality

Mobile Virtual Reality on track to hit $861M in revenues in 2016

Page 4: STEM Camp Virtual Reality

360-degree Panoramas

Page 5: STEM Camp Virtual Reality

Immersive 360-degree images and videos

• Viewer can look around, observe different views

Page 6: STEM Camp Virtual Reality

Amazon!

Approx. 35 hours

from Brisbane – Sydney –

Santiago, Chile –

Lima, Peru –

to Pucallpa, Peru

Approx. 12 hours

Pucallpa – Camito, Amazon, Peru

Page 7: STEM Camp Virtual Reality

Technology to capture panoramas

360-degree cameras: Freedom 360 Explorer+ with 6x GoPro Hero 4

installed, 2x Ricoh Theta M, and 2x Kodak SP360.

Stereo camera: Dual Hero System with 2x GoPro Hero 3+ installed.

IR camera: Reconyx XR6 Ultrafire (infra-red, motion sensing

trap camera).

Sound: Zoom H2n Handy Recorder, Olympus VN-733PC.

Virtual Reality: Samsung GearVR, Samsung Galaxy S6, Google

Cardboard Glasses.

USB Hub: ORICO 10 Port-Hub.

GPS: Garmin 64ST GPS System.

Page 8: STEM Camp Virtual Reality

Taking 360-degree photos

Page 9: STEM Camp Virtual Reality

Processing with Autopano Giga, Autopano Video Pro

• To stitch one 360-degree panoramic image:• Drag and drop six images into Autopano Giga

Page 10: STEM Camp Virtual Reality

Processing

Lightroom

Photoshop

After Effects

Premiere

Page 11: STEM Camp Virtual Reality

Processed

Page 12: STEM Camp Virtual Reality

Priceless!Beautiful & Beautiful Scenery

Warmth of village people, curious eyes and laugh of children

Daily canoeing in sunrise and sunset time

Time to truly engaging to wild life conservation issue

Page 13: STEM Camp Virtual Reality

True power of VR in Conservation

• Essential aspects of VR technology are:• Presence

• Immersion

• Interactivity

• Empathy

[Chris Milk: How virtual reality can create the ultimate empathy machine]

VR technology can be used to provide people experience of wild life creating better connection between human and natures

Page 14: STEM Camp Virtual Reality

Reaching Through the Looking Glass | Craig James15 |

Firing the Imagination

Page 15: STEM Camp Virtual Reality

Milgram’s Reality-Virtuality continuum

The area between the completely real and completely virtual, consists of both augmented reality, where the virtual augments the real, and augmented virtuality, where the real augmented the virtual.

P. Milgram and A.F. Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.

Page 16: STEM Camp Virtual Reality

Virtual Reality

Page 17: STEM Camp Virtual Reality

Hand gesturesto operate VR

• Enhance e-Learning

• Experience interactions = remember more

• Test in class-room situations

Use of iPhone’s touch screenImmersive e-book

Page 18: STEM Camp Virtual Reality

Introduction to Unity

Page 19: STEM Camp Virtual Reality

Unity as cross-platform game engine

Page 20: STEM Camp Virtual Reality

Unity setup

• Download from https://unity3d.com/get-unity/download

Page 21: STEM Camp Virtual Reality

Create new project

• Specify project name

• Specify project location

• Specify whether project is 3D or 2D

• Specify asset packages (pre-made content)

Page 22: STEM Camp Virtual Reality

User Interface

Project Window

Hierarchy Window

Inspector Window

Toolbar

Scene View

Page 23: STEM Camp Virtual Reality

Creating scenes

• GameObjects• Every object in your scene is a

GameObject• They are containers (for instance 3D

object, texture, animation)

• Object properties can be viewed and edited in Inspector

• Position of GameObjects, cameras, lights can be seen and adjusted in Scene view

• Interactions are added using Scripts attached to objects

Page 24: STEM Camp Virtual Reality

Asset Workflow

• Asset from outside of Unity: a 3D model, an audio file, an image, or any of the other types of file that Unity supports.

• Asset created within Unity: an Animator Controller, an Audio Mixer, a Render Texture.

• For instance to add 3D model:• Add file to Assets folder in Project• Drag mesh from Project View into

Hierarchy or Scene View

Page 25: STEM Camp Virtual Reality

GameObjects

• GameObject contain Components, for instance Transform, Camera, etc. (seen from the Inspector)

Page 26: STEM Camp Virtual Reality

Positioning GameObjects

Page 27: STEM Camp Virtual Reality

New Project

Page 28: STEM Camp Virtual Reality

Change layout (Windows Layouts Tall)

Page 29: STEM Camp Virtual Reality

Moving main camera

• Select Main Camera, select Translate Icon, Move camera

Page 30: STEM Camp Virtual Reality

Creating immersive 360-degree viewer

• Create new project

• Add the Cardboard plugin VR SDKhttps://developers.google.com/vr/unity/

• Add 360-degree image asset

• Create a Skybox

• Add to the Scene

• Publish on the Android devicehttps://developer.android.com

Page 31: STEM Camp Virtual Reality

Google Cardboard – see next slides

• Open Unity and create new 3D project

• Import the Unity package: Assets Import Packages Custom Package.• Select the GoogleVRForUnity package where you downloaded it and click on

Open.

• Check all the boxes and click Import.

• You can test whether it works by opening Assets GoogleVR DemoScenes

HeadsetDemo DemoScene.• Press Play (you should see a red cube).

• Press the Control key on your keyboard and move your mouse around to tilt the view. Pressing Alt will cause too pan around the view of the VR.

Page 32: STEM Camp Virtual Reality

Import Google VR

• Import the Unity package: Assets Import Packages Custom Package.• Select the GoogleVRForUnity package where downloaded, and click on Open.

• Check all the boxes (uncheck iOS) and click Import.

Page 33: STEM Camp Virtual Reality

Demo (Assets Google VR DemoScenes HeadsetDemo)

• Press Play (should see a red cube, or green when navigated to it)• Press the Control key and move mouse to tilt your view (inside SceneView)• Press the Alt key and move mouse to pan around (inside SceneView)

Page 34: STEM Camp Virtual Reality

Adding Cardboard to Empty Project

• Drag GvrViewerMain from Assets GoogleVR Prefab to Hierarchy• Click on Play, should render stereo VR view (navigate using mouse and Control or Alt keys)

Page 35: STEM Camp Virtual Reality

Import panoramic image (Disneyland, why not?)

Page 36: STEM Camp Virtual Reality

Import panoramic image (Disneyland, why not?)

• Create folder Textures in Assets• Drag and drop 360-disney.jpg to Assets Textures• Click on the imported image: Inspector will show you properties of the texture

Page 37: STEM Camp Virtual Reality

Change texture type

• Change Texture Type from Texture to Cubemap• Change Mapping o Latitude-Longitude Layout (Cylindrical)• Click on Apply

Page 38: STEM Camp Virtual Reality

Create Skybox material

• Create folder Materials in Assets, and click right mouse button in that folder and Create Material• Name material Sky01 and click in inspector on Select• In Select Cubemap choose 360-disney earlier created

Page 39: STEM Camp Virtual Reality

Create Skybox

• In Main Menu click on Window Lightening• Drag Sky01 material into Skybox field• Background of SceneView should now display panoramic image in proper projection

Page 40: STEM Camp Virtual Reality

Install on Android

Change Bundle Identifier to something like au.com.qut.helloworld

Page 41: STEM Camp Virtual Reality

Pushing to device

Page 42: STEM Camp Virtual Reality

Thank youNow try it yourself :-D