Top Banner
Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center, San Jose
24

Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

Apr 01, 2015

Download

Documents

Savanah Benskin
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

Virtual Worlds Technology Roadmap

December 2008

Seung Wook Kim, Francisco Imai, Stefan MartiHCI Research Team, Computer Science Lab

Samsung R&D Center, San Jose

Page 2: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

VW Technology Trends Analysis

1. State of The Art1.1 Virtual World Clients for Fixed Platform1.2 Virtual World Clients for Mobile Platform1.3 Virtual World Server Technologies

2. Catalyst Elements for VW2.1 3D content2.2 Natural interaction for 3D2.3 Artificial Intelligence in VW

3. Mixed Reality as a Killer Application

Backup Slides Mixed Reality Workgroup Mission and Goals Issues in 3D

- 2 -

Page 3: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing” - 3 -

Trends: 1. Variety of VW clients: - standalone, Web-based (ActiveX or Flash), CE-based (game console), etc 2. Multimodal in-world communication (e.g. textual + bubble + voice chat) 3. Support of user created 3D content via standard format (e.g. COLLADA) Challenges:•Insufficient client performance: 3D rendering quality, physics, animation•Voice chat is becoming standard, but it still doesn’t benefit from non-verbal communication (i.e., non-intuitive avatar gesture/action control)•Complicated 3D content creation pipeline: CAD/CAM skills are required

Trends: 1. Variety of VW clients: - standalone, Web-based (ActiveX or Flash), CE-based (game console), etc 2. Multimodal in-world communication (e.g. textual + bubble + voice chat) 3. Support of user created 3D content via standard format (e.g. COLLADA) Challenges:•Insufficient client performance: 3D rendering quality, physics, animation•Voice chat is becoming standard, but it still doesn’t benefit from non-verbal communication (i.e., non-intuitive avatar gesture/action control)•Complicated 3D content creation pipeline: CAD/CAM skills are required

1.1 State of The Art: VW Clients for Fixed Platforms

VW Web Plug-In

VW on CE Devices

Sony’s Playstation Home

VIVATY Scenefor AIM

Opportunities:•Actively apply multicore/parallel computing technologies (for both CPU and GPU) on VW client development •Sensor technologies that will accurately translate physical motions into virtual actions•Design intuitive and intelligent UI for easier creation and/or manipulation of virtual objects

Opportunities:•Actively apply multicore/parallel computing technologies (for both CPU and GPU) on VW client development •Sensor technologies that will accurately translate physical motions into virtual actions•Design intuitive and intelligent UI for easier creation and/or manipulation of virtual objects

Flash-based VW

WebFlock SmallWorlds

Standards for 3D content

Forterra’s Olive VW Platform supporting COLLADA*

Current status:• The Electric Sheep

Company: Providing Flash-based & private-labeled VW service, WebFlock

• SMALLWORLDS: Flash-based VW

• VIVATY: Combining Web plug-in Virtual Community (VIVATY Scene) with social networking services (AIM and Facebook)

• SONY: Providing VW community service (HOME) based on the game console Playstation3

• FORTERRA: Support for COLLADA* into its open and standards-based VW platform, OLIVE

Current status:• The Electric Sheep

Company: Providing Flash-based & private-labeled VW service, WebFlock

• SMALLWORLDS: Flash-based VW

• VIVATY: Combining Web plug-in Virtual Community (VIVATY Scene) with social networking services (AIM and Facebook)

• SONY: Providing VW community service (HOME) based on the game console Playstation3

• FORTERRA: Support for COLLADA* into its open and standards-based VW platform, OLIVE

Page 4: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. Mobile 3D clients start providing users with permanent access to VWs 2. Connecting VW users in real-time from instant messaging client 3. Emergence of motion sensors and haptic feedback on mobile devices

Challenges:• Limited performance of remote rendering & streaming technologies• No voice/text enabled thin client connecting VW on mobile platform• Lack of new use models for sensor-mounted mobile devices

Trends: 1. Mobile 3D clients start providing users with permanent access to VWs 2. Connecting VW users in real-time from instant messaging client 3. Emergence of motion sensors and haptic feedback on mobile devices

Challenges:• Limited performance of remote rendering & streaming technologies• No voice/text enabled thin client connecting VW on mobile platform• Lack of new use models for sensor-mounted mobile devices

1.2 State of The Art: VW Clients for Mobile Platforms

State-of-Art Mobile Device for 3D Graphics

Mobile VW client

Vollee’s SL client

Vivox’s SLim client

Opportunities:•Enhance the performance of 3D graphic accelerating processor for mobile devices

•Integrate VoIP/messaging system with mobile immersive 3D client

•Develop highly-efficient mobile VW client and its UI mechanism

Opportunities:•Enhance the performance of 3D graphic accelerating processor for mobile devices

•Integrate VoIP/messaging system with mobile immersive 3D client

•Develop highly-efficient mobile VW client and its UI mechanism

3D Motion Tracking

Apple’s iPhone with ARM1176JZF & Samsung S3C6400

Thin VW client

- 4 -

Current status:• VOLLEE: Leader in mobile VW

service with remote rendering & streaming technologies

• VIVOX (w/ Linden Lab): First commercial voice-driven IM client (SLim) for Second Life

• APPLE: iPhone is equipped with a top-level chip (ARM1176JZF) with 3D accelerating coprocessor (Samsung S3C6400)

Current status:• VOLLEE: Leader in mobile VW

service with remote rendering & streaming technologies

• VIVOX (w/ Linden Lab): First commercial voice-driven IM client (SLim) for Second Life

• APPLE: iPhone is equipped with a top-level chip (ARM1176JZF) with 3D accelerating coprocessor (Samsung S3C6400)

Page 5: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. Open-source approach to interoperable server platform 3. Efforts for building MMOG environments in VW 2. Both open and private-labeled virtual world (VW) services are increasing

Challenges:• Client-neutral, flexible and highly-trusted server platform is required• Difficult to attract and hold a large volume of concurrent users• Server maintenance cost is still expensive (per user)

Trends: 1. Open-source approach to interoperable server platform 3. Efforts for building MMOG environments in VW 2. Both open and private-labeled virtual world (VW) services are increasing

Challenges:• Client-neutral, flexible and highly-trusted server platform is required• Difficult to attract and hold a large volume of concurrent users• Server maintenance cost is still expensive (per user)

1.3 State of The Art: VW Server Technologies

MMOG* in VW

Multiverse

Opportunities:•Develop software standards for interoperable platforms for online 3D applications and virtual worlds

•Connect different hardware platforms (e.g. mobile devices and fixed systems) via VW applications

•Rethink VW platform architecture (e.g. cloud computing)

Opportunities:•Develop software standards for interoperable platforms for online 3D applications and virtual worlds

•Connect different hardware platforms (e.g. mobile devices and fixed systems) via VW applications

•Rethink VW platform architecture (e.g. cloud computing)

Interoperable VW

Open-Source Community

Open Grid by IBM+Linden Lab

OpenSimulator

- 5 -

Current status:• OpenSIM: Open-sourced and

flexible 3D application server supporting various VW clients

• IBM + Linden Lab: Releasing the first interoperable VW service platform (Open Grid Beta)

• Multiverse: Providing open development toolkit for MMOG-type VW, including authoring tools and source code

Current status:• OpenSIM: Open-sourced and

flexible 3D application server supporting various VW clients

• IBM + Linden Lab: Releasing the first interoperable VW service platform (Open Grid Beta)

• Multiverse: Providing open development toolkit for MMOG-type VW, including authoring tools and source code

Page 6: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. Renaissance of 3D movies - 11 movie titles released (from 2005 to 2008) and many more in 2009 2. Consistent popularity of games with 3D content 3. Explosive increase in medical imaging with 3D content ($1.4B by 2013) 4. Web-based VW and Mirror World (SL, Google Earth) 5. Stereo Synthesis - 2D to 3D conversion, depth capturing, multi-aperture imaging

Challenges:•Displays: auto-stereoscopic displays are expensive and cause user fatigue and most of 3D viewing solutions require wearing awkward glasses•UX: there is no good solution in the market for natural and intuitive user interface for 3D content

Trends: 1. Renaissance of 3D movies - 11 movie titles released (from 2005 to 2008) and many more in 2009 2. Consistent popularity of games with 3D content 3. Explosive increase in medical imaging with 3D content ($1.4B by 2013) 4. Web-based VW and Mirror World (SL, Google Earth) 5. Stereo Synthesis - 2D to 3D conversion, depth capturing, multi-aperture imaging

Challenges:•Displays: auto-stereoscopic displays are expensive and cause user fatigue and most of 3D viewing solutions require wearing awkward glasses•UX: there is no good solution in the market for natural and intuitive user interface for 3D content

2.1 Catalyst Elements: 3D Content

Opportunities:•Create innovative and intuitive user interaction model for immersive 3D VW client on consumer electronic devices

•Provide mobile solutions for 3D content navigation

•3D interactive movies for VW

•Remote medical service and training in VW

Opportunities:•Create innovative and intuitive user interaction model for immersive 3D VW client on consumer electronic devices

•Provide mobile solutions for 3D content navigation

•3D interactive movies for VW

•Remote medical service and training in VW

- 6 -

Current status:• Philips: Developed auto-

stereoscopic displays. Technical issues such as narrow angle of view and user fatigue. No compelling UX

• 3DV Systems: Detection of body gestures using depth cameras for avatar control. Issues of user fatigue and lack of resolution

• GestureTek: Control of digital content by tracking gestures. Could lead to user fatigue

Current status:• Philips: Developed auto-

stereoscopic displays. Technical issues such as narrow angle of view and user fatigue. No compelling UX

• 3DV Systems: Detection of body gestures using depth cameras for avatar control. Issues of user fatigue and lack of resolution

• GestureTek: Control of digital content by tracking gestures. Could lead to user fatigue

Source: Insight Media

Source: 3D SMPTE standard activities

Wendy Aylsworth (Warner Bros.)

Page 7: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. Beyond keyboard and mouse: evolution towards alternative HCI tools 2. Multi-touch: Evolution towards gesture interfaces on surface 3. Multitude of sensors: Progressive adoption of orientation, position and motion sensing (optical sensors, inclinometers, accelerometers, compass)

Challenges:•User fatigue: solutions that rely on mapping human body movements often lead to user fatigue•Lack of feedback: it is difficult to give appropriate feedback for 3D interaction

Trends: 1. Beyond keyboard and mouse: evolution towards alternative HCI tools 2. Multi-touch: Evolution towards gesture interfaces on surface 3. Multitude of sensors: Progressive adoption of orientation, position and motion sensing (optical sensors, inclinometers, accelerometers, compass)

Challenges:•User fatigue: solutions that rely on mapping human body movements often lead to user fatigue•Lack of feedback: it is difficult to give appropriate feedback for 3D interaction

2.2 Catalyst Elements: Natural UI for 3D

Opportunities:•Combine different technologies to provide a more immersive and engaging UX for 3D interaction

•Create closed-loop systems for user experience and 3D object rendering by dynamically displaying 3D object according to user position and interaction with virtual content

Opportunities:•Combine different technologies to provide a more immersive and engaging UX for 3D interaction

•Create closed-loop systems for user experience and 3D object rendering by dynamically displaying 3D object according to user position and interaction with virtual content

OrangeSeeingmachines Softkinetic

Mgestyk

- 7 -

Current status:• Seeingmachines: Developer

of face API that tracks human face with normal webcam and displays images based on view-dependent-rendering

• Softkinetic: Provides 3D real-time gesture recognition middleware

• Mgestyk: Combines 3DV systems depth camera with hand-gesture processing.

Current status:• Seeingmachines: Developer

of face API that tracks human face with normal webcam and displays images based on view-dependent-rendering

• Softkinetic: Provides 3D real-time gesture recognition middleware

• Mgestyk: Combines 3DV systems depth camera with hand-gesture processing.

Page 8: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. A.I. engines programmed to learn by interacting with humans in VW - VW gives advantage of embodiment to A.I. engine without hassle or costs that comes with physical robotics - VW allows exponential increase in interaction for AI engines to learn 2. There are efforts to make VW presence and interaction more engaging

Challenges:•Recreate self-mind in VW: there is need to create a self-awareness as present in human interaction by appropriately planning•Pattern mining: finding complex patterns requires prohibitively inefficient searches•Self-modeling: creating learning algorithms capable to embody intelligence•Language and gesture processing: interpretation of language, gestures

Trends: 1. A.I. engines programmed to learn by interacting with humans in VW - VW gives advantage of embodiment to A.I. engine without hassle or costs that comes with physical robotics - VW allows exponential increase in interaction for AI engines to learn 2. There are efforts to make VW presence and interaction more engaging

Challenges:•Recreate self-mind in VW: there is need to create a self-awareness as present in human interaction by appropriately planning•Pattern mining: finding complex patterns requires prohibitively inefficient searches•Self-modeling: creating learning algorithms capable to embody intelligence•Language and gesture processing: interpretation of language, gestures

2.3 Catalyst Elements: Artificial Intelligence in VW

Opportunities:•Virtual pets can be a killer application for A.I. in VWs

•Adding ability to have pets genuinely learn and respond to the environment will make them more real to the user and increase the user/virtual pet bond

•Autonomous avatars for creating rich virtual environments

Opportunities:•Virtual pets can be a killer application for A.I. in VWs

•Adding ability to have pets genuinely learn and respond to the environment will make them more real to the user and increase the user/virtual pet bond

•Autonomous avatars for creating rich virtual environments

- 8 -

Current status:• Novamente IRC: methodology

for teaching:

• Imitative learning: instructor shows by example

• Reinforcement learning:– student repeats movements with feedback

• Corrective learning:– instructor actively corrects student’s movements

Current status:• Novamente IRC: methodology

for teaching:

• Imitative learning: instructor shows by example

• Reinforcement learning:– student repeats movements with feedback

• Corrective learning:– instructor actively corrects student’s movements

Page 9: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Trends: 1. Increase in information overlay (Googlemaps, VWs)2. See-through devices as a platform for mixing VW and RW (iPhone App)

Challenges:•Design of wearable displays: current solutions are bulky and too geeky•Need for an intelligent agent: user should not be overwhelmed by profusion of virtual data but system should have an embedded agent that filters relevant or preferred information•Lack of precision of location sensors: current location sensors need to improve precision in order to identify location for data augmentation

Trends: 1. Increase in information overlay (Googlemaps, VWs)2. See-through devices as a platform for mixing VW and RW (iPhone App)

Challenges:•Design of wearable displays: current solutions are bulky and too geeky•Need for an intelligent agent: user should not be overwhelmed by profusion of virtual data but system should have an embedded agent that filters relevant or preferred information•Lack of precision of location sensors: current location sensors need to improve precision in order to identify location for data augmentation

3. Mixed Reality as a Killer Application

Opportunities:•Develop technologies for an unobtrusive system with natural UX for mixed reality

•Possible applications: - Travel planning with shared

experience and enhanced participation

- Training of first responders- Inter-reality physics: virtual reality

system coupled with real-world one

Opportunities:•Develop technologies for an unobtrusive system with natural UX for mixed reality

•Possible applications: - Travel planning with shared

experience and enhanced participation

- Training of first responders- Inter-reality physics: virtual reality

system coupled with real-world one

Virtual Continuum (by Paul Milgram, 1994)

- 9 -

Current status:• BBC R&D: Have explored the

impact of mixed reality in media: http://www.bbc.co.uk/rd/projects/virtual/mixtv/index.shtml

• Nokia Research: Research on mobile mixed reality system with augmentation of real world using mobile phones.

• GaTech: Augmented Reality in SL

Current status:• BBC R&D: Have explored the

impact of mixed reality in media: http://www.bbc.co.uk/rd/projects/virtual/mixtv/index.shtml

• Nokia Research: Research on mobile mixed reality system with augmentation of real world using mobile phones.

• GaTech: Augmented Reality in SL

Nokia Research BBC R&D

Page 10: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Backup slides

Page 11: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Mixed Reality Workgroup: Mission

We believe that Mixed Realities have the potential to change the way we interact with the world in a radical way.

Under Mixed Reality, we refer to the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time.

But radical change can only happen if the technology is available on a broad basis.

The more people who can interact with Mixed Realities, the clearer the advantages of such a technology will become.

Therefore, our mission is to improve human interaction with real and virtual worlds by encouraging the mass adoption of Mixed Reality technologies, and the creation of an industry with very low barriers to innovation and business success.

Page 12: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Mixed Reality Workgroup: Goals

We propose to provide information aimed at technology providers, investors, and business decision makers:

• Casebook that documents past efforts, lessons learned, and best practices

• Roadmap of key technical requirements for "pacing mass usages“

• “Grand Challenges” collection to inspire innovation in the industry

We are an Active Workgroup that develops documents which frame the usages, as well as key technical and business model barriers, to the success of the Mixed Reality vision.

Page 13: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Issues in 3D

3D content creation - Quality x Quantity - Mastering 3D content for home x cinema - 3D creation and workflow training and education

3D content storage, transmission and distribution - File formats, compression, transmission - Coordinate work in standard bodies (DCI, MPEG, IEEE 3D Consortium, 3D4YOU, etc…)

3D Promotion - Common naming conventions and vocabulary - Develop content for web distribution - End-user and retailer/reseller education and in-store demos

3D Displays - 3D display quality metrics – ghosting, brightness, etc… - 3D human factors – eye fatigue, viewing time, viewing angle dependency, etc

Page 14: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D Display Technologies

Page 15: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3DTV vs. 3D-ready TV

Page 16: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

Cost

Page 17: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

PDP display market forecast

Samsung manufactures 3D-ready PDP-TV. This is the expected PDP forecast in the USA with 3D TVs dominating the market in the next years.

Page 18: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D LCD display market forecast

It is expected that 3D LCD is going to emerge in the market and grow exponentially

Page 19: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D and 3D-ready display market

Page 20: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D content – movie as a driving force

Page 21: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

TV – How do you know it is 3D-ready?

Page 22: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D-ready TV – 3D glasses and PC requirements

Page 23: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D-ready TV – 3D glasses and PC requirements

Page 24: Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

SISA CSL “Innovate by doing”

3D-ready TV – PC requirements