Virtual Storytelling 2001, Avignon - E. Badique - Under construction in Europe - Virtual and Mixed Reality for a Rich Media Experience Eric Badique [email protected].

Virtual Storytelling 2001, Avignon - E. Badique’

- Under construction in Europe -Virtual and Mixed Reality

for a Rich Media Experience

Eric Badique’

[email protected]


On behalf of….

INRIA

UCL

U.LEEDS

IST

ADETTI

KULUCL

CSELT

UPCUPM

PHILIPSHHIGMD

FHG

U.Munchen

P.MilanoREALVIZ

U.GENEVAEPFL

INTRACOM

RSU

Tampere U.VTT

MATRA

Symah

BT

BBC

SONY

Siemens

NOKIA

Erricson

TU. Wien

~ 15 projects

~60 organisations

15 countries


Reality-Virtuality Continuum

Mixed RealityAugmented Reality Augmented Virtuality

Real environment Virtual environment


Why Mixed Reality ?

• Most attractive and modern media exploit a form of mixed reality

• Enabling technology for the future of media

• Important investment in AV representation and coding over past 10 years (MPEG…)

• Significant activities in CG, visualisation, VR

• Some activities in computer vision

• Efforts to promote synthetic/natural coding for 5 years + (MPEG-SNHC..)

• Combines realism of real world pictures with flexibility of CG

• Multi-disciplinary - good framework for collaborative projects

• P.Milgram, MR Labs and ISMR’99 did provide some inspiration and a framework...

• … but MR picture was refined over several years

MR as main theme for IST WP2000-2001


Projects portfolio

Analyzing and modeling the world

Merging Real & Virtual

ContentInteractivity

Audio-VisualInteractiveServices


RE

VE

Communication

TelepresenceVIRTUE

ARARTHUR

ARIS

EnhancedNarrative

Space ALIVE

EmotionalInterface

INTERFACESAFIRAMEGAHybrid IB/3D

renderingINVIEW

3D modeling(uncalibrated)

VISIREEVENTS

Medical Imaging

VR-SUR

2nd GeniTVPISTE MR

for trainingSTAR

Programmeproduction

ORIGAMICrowd

SimulationCROSSES

ExcavatorSimulation

SHE

Consumption

Current IST MR Portfolio

interactivity Next gen.

Portals

SONG


The full story...

ResearchStandards

Content, authoringtools & stories

Mixed Reality technologies

ApplicationsiTV

Culture &art

Industry Mobile services

IST/KAIII


Research

• Computer graphics and animation

• Emotional interfaces

• Components and systems• Human factors and usability -> Guidelines and fitness-for-purpose measures

CalibrationBackground segmentation, reference image managementHandling illumination changes and shadow detectionSegmentation and object extractionMotion estimation, motion segmentation , structure from motion Tracking (Head-tracking)Disparity and stereo, stereo vision3D pose estimation and 3D tracking

3D reconstruction, 3D geometry from video.High-level features extractionData fusionMosaickingView synthesis and renderingReal-time aspectsBenchmarking

• Image Processing & Computer vision -> Clustering and Bench-marking


Standards

• Bridging the gap between coding and CG world (MPEG-4-SNHC/VRML/Web3D)

• Rich media standards for next gen. iTV (DVB/MHP-MPEG-4)

• Agents (FIPA)

• Indexing, search and retrieval (MPEG-7, TV-Anytime..) From low level (shape, size, texture, color..) to semantic information

This area is at the forefront of technological developments and should play its ‘ standard-pulling ’ role


MR opportunities

• Completely new markets ?

• Provides bandwidth hungry applications: Distributed mixed reality environments are the essence of future communication services - provide requirements for future networks

• “Revaluation” of manual work through AR

• Next wave of innovation in industry following CAD and robotics

• Next generation of interactive TV

• AR as new telescope and microscope of the 21st century ?

• Mixed Reality also to be seen in a wider context: Integration of visualisation and interfaces with mechanics, robotics and appliances


Upcoming FP5 calls

• IST 7th call (deadline October 17, 2001)

– IV.4.2 Mixed realities and new imaging frontiers

– III.1.1 Publishing digital content

– III.5.1 X-content future

– FET Open

• IST 8th call (November 15, deadline February 21, 2002) – IV.5.1 Networked audio-visual systems and services

– CPA 15 Technology platforms for cultural and arts creative expressions


WP2002 - CPA15: Technology platforms for cultural and artistic creative expressions

• Generic platforms for artistic and cultural content creation..

.. in media production, cultural assets exploitation, artistic design, contemporary arts and performance.

• Exploit technologies for virtual, augmented and mixed realities, new displays, multisensory and multimodal interfaces.

• Involve independent artists, young talents, cultural designers and creators.

• Invent new forms of content and applications.


FP6: Media Futures - From information to experience?

Context and enhanced interactivity, storytelling Multi-sensor(y) data collection (including real-time) Visualisation (including 3D, VR and Mixed Reality

etc..) Delivery over heterogeneous networks (including

broadcasting) Databases, knowledge management & statistics Metadata Personalisation (including multi-linguality)


ImTV

CDTV

Interactive TV

VoD

VCR

Rock concertFootball

Theatre

Cinema

IMAX

Consumer electronics Real live events

Immersive TV: Closing the Gap to Live Sensation

Sensation

Costs

[Source: ITC]


We need

• High fidelity representation of the (MR) world

• Use our five senses

• Continuous acquisition of data about the world around us

• Assimilation of data with information and knowledge from the past

• Organising information in a way that people experience, by time and space and around concepts, not keywords.

Storytelling to play a central role ?


First experiences Portal for live sport events / tracking and alarm – integration with database for past and

future events (Praja.com) 24 hours boats tracking - race around the world - stats and AV coms Cars tracking and inclusion in a game in real-time – merging broadcasting and gaming

(Kalisto) Multi-camera recording - possibility to choose viewing angle (Superball 2000, Kanade) Augmentation of sport (other) TV broadcasting with interactive elements (3D,

animation, statistics etc…- eg. IST Piste project) Enhanced narrative spaces (IST Art.Live) ...

Emphasis on integration

Powerful concept for new media

FP6 Contributions welcome


FocusFocusAdded-value Added-value Clear objectivesClear objectivesMulti-disciplinarityMulti-disciplinarityAddress fundamental issuesAddress fundamental issuesStandards and interoperability Standards and interoperability Concentrate on generic aspects Concentrate on generic aspects Enabling technology as opposed to technological gadgetryEnabling technology as opposed to technological gadgetry

If you want to be part of IST...If you want to be part of IST...


Contacts Information Society Technologies

http://www.cordis.lu/ist

Mixed Reality http://www.cordis.lu/ist/ka4/vision/

- Project descriptions

- Reference documents

- Info on events

Virtual Storytelling 2001, Avignon - E. Badique - Under construction in Europe - Virtual and Mixed Reality for a Rich Media Experience Eric Badique [email protected].

Documents

virtual storytelling

badique standards

europe virtual

storytelling multi

badique fp6

badique mr opportunities

role slide

form of mixed reality