Page 1
An Assessment of 3DTV Technologies
An Assessment of 3DTV Technologies
NAB-2006 - Las Vegas - 26 April 2006
© Copyright 2006
Levent Onural Thomas Sikora Jörn Ostermann Bilkent Un. Tech. Univ. Of Berlin Univ. Of Hanover
Aljoscha Smolic M. Reha Civanlar John WatsonFraunhofer Inst.-HHI Koç Univ. Univ. Of Aberdeen
Page 2
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
2
Presentation Overview
•Introduction to EC funded 3DTV project
•Introduction and Historical Overview
•Technical Overview of Integrated 3DTV
•Capture
•Representation
•Transmission
•Signal Processing
•Display
•Applications
•Conclusion
Page 3
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
3
EC Funded 3DTV Project
•19 Partners from seven European countries
•Universities, Research Institutions, SMEs
•About 200 contributors
•All technical aspects of 3DTV are covered
•A four year project (2004-2008)
•“Network of Excellence”
Page 4
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
4
EC Funded 3DTV Project
•Partners:
•Bilkent Univ – Turkey (Coordinator)
•BIAS – Germany
•CLOSPI-BAS – Bulgaria
•De Montfort Univ – UK
•Fraunhofer Inst. HHI – Germany
•FogScreen – Finland
•Un. Of Ilmenau - Germany
Page 5
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
5
EC Funded 3DTV Project
•Partners (Cont’d):
•Tampere Un. Of Tech – Finland
•ITI-CERTH – Greece
•Koç Univ. – Turkey
•METU – Turkey
•Momentum – Turkey
•Max Planck Inst. – Germany
•U. West Bohemia Plzen – Czech Rep
Page 6
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
6
EC Funded 3DTV Project
•Partners (Cont’d):
•U. Of Hanover – Germany
•Tech. Univ. Berlin – Germany
•U. Of Tuebingen - Germany
•U. Of Aberdeen - UK
•Yogurt Tech. – Turkey
Page 7
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
7
Page 8
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
8
3D Scene
Capture Representation Coding
Signal Conversion
DisplayTransmission
Its Replica
3DTV Technical Overview
Page 9
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
9
3DTV Historical Overview
•“Realistic” ghost-like illusions have always been a goal
•2D (still or motion) picture technologies are well developed and well accepted
•3D counterparts are almost as old as 2D
•Stereoscopic 3D pictures – 1838 (Wheatsone)
•Popular by 1844 in USA and Europe
•2D Photography - 1839
Page 10
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
10
3DTV Historical Overview
•3D counterparts are almost as old as 2D (continued)
•2D Movies – 1867 (Lincoln)
•3D stereoscopic cinema – early 1900
•2D TV – 1920 (Belin and Baird)
•Stereoscopic 3DTV – 1920 (Baird)
•Stereoscopic 3D cinema was popular by 1950s
•Stereoscopic 3DTV broadcast – 1953
•First commercial 3DTV broadcast – 1980s
Page 11
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
11
Observations
• Stereoscopic 3DTV is quite an old concept
•Almost as old as its 2D counterpart
•2D has flourished
•3D “failed” to be popular
•Stereoscopic 3D is simpler
•Based on human visual system and perception
•Usually creates “mismatches” of various 3D cues
•Current research on stereoscopic 3DTV is mostly targeted to overcome such problems
Page 12
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
12
An Overview of Candidate 3DTV Techniques
•Stereoscopic 3DTV
•Simplest, based on “perception”
•Viewer discomfort
•Autostereoscopic
•Head-tracking stereoscopic
•Holographic 3DTV
•Basic principle -1948, first holograms – 1960
•Based on physics : duplication of light field
•Digital holographic techniques
•Holographic cameras
•Experimental holographic motion pictures – 1989
•Still at basic reseach phase
Page 13
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
13
An Overview of Candidate 3DTV Techniques
•Integral Imaging Based 3DTV
•A principle known since 1908
•Based on “capturing a scene from different angles”
•And then reverse projection from corresponding angles
•Lenslet arrays
•Variants can be classified under “holographic techniques”
Page 14
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
14
Past Decade
•NHK – Stereoscopic 3D-HDTV (Since 1991)
•2002 FIFA World Cup broadcast in Stereo 3D-HDTV
(ETRI-Korea)
•European ATTEST project (2002-2004)
•Backward compatible to DVB
•Uses “depth data” (efficiently compressed by H.264/AVC)
•Matches MPEG4 with minor modifications
•MERL projection-based system (2004)
Page 15
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
15
3DTV Project -Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 16
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
16
3DTV Project - Technical Overview - Capture
• Multi-camera• Single Camera• Holographic• Human Face and Body Specific• Pattern Projection• Other
Page 17
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
17
3DTV Project -Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 18
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
18
3DTV Project – Technical Overview - Scene Representation
• Point Representation Based• Surface Representation Based• Volume Representation Based
• Human Face and Body Specific Representations• Object Specific Representations• Cloth and Garment Representation
• Rendering Issues• Formats and Standardization Issues
•DXF, OBJ, VRML, LWO, 3DS, 3DIF, U3D, Web3D, X3D, MPEG-4-AFX
• Texture Representation and Rendering
Page 19
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
19
3DTV Project -Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 20
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
20
3DTV Project – Technical Overview-3D Coding
•3D Coding Techniques:•Holographic Signal Compression•Multiview Video Coding•Object-based coding•3D Mesh compression•Dynamic Texture Coding•Multiple Description Schemes•Interactivity Issues•Lightfield Compression•Point-cloud Compression•MPEG4-AFX, BIFS•Interactivity Issues•Compound Bit-stream Syntax•Security and Rights Management
Page 21
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
21
3DTV – Standardization Activities
•Current Standardization Activities in 3DTV•MPEG4-3DAV
•Multiview Coding
Page 22
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
22
3DTV Project -Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 23
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
23
3DTV Project–Technical Overview - Transport Techniques for 3DTV
– Media Transport over IP networks• QoS Controls
– Protocols• Media Synchronization• RTP• DCCP• H.264 over RTP
– Loss Concealment Techniques– Channel Coding Issues– Mobile 3DTV Applications
Page 24
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
24
3DTV Project Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 25
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
25
Review Mtg – Technical Overview - Signal Processing Issues
• Modeling of Diffraction from a Systems Point of View
• Forward and Inverse Problems in Holographic 3DTV
• Signal Processing Problems in Different 3DTV Techniques– Holographic 3DTV– Stereoscopic TV– Integral Imaging– Hybrid Techniques– SAR techniques
Page 26
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
26
3DTV Project -Technical Overview
3D Scene
Its Replica
Capture Representation Coding
Signal Conversion
DisplayTransmission
Page 27
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
27
3DTV Project – Technical Overview – Display Techniques
– Holographic Displays•SLMs•DMDs•Acousto-optic Modulators
– Stereoscopic Displays– Autostereoscopic Displays– Volumetric– Pseudo-volumetric– Head-mount displays
Page 28
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
28
3DTV Review Mtg – Technical Overview - Display Techniques
• Autostereoscopic Displays (cont’d):– Pseudo-volumetric FogScreen
Page 29
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
29
3DTV Review Mtg – Technical Overview - Display Techniques
Page 30
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
30
Page 31
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
31
3DTV Technologies – Other Applications
•Education, Training and Simulation•Scientific Simulation and Visualization•Medicine and Dentistry•Cultural Heritage•Technical Applications•Market Applications•Telepresence•Entertainment and Game Industry•Air Traffic Control•Military•Security•Communications •Computers
Page 32
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
32
Page 33
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
33
Page 34
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
34
Page 35
26 April 2006 Levent Onural --- www.3dtv-research.org
© Copyright 2006
35
Thank you.... Levent Onural, Thomas Sikora, Jörn Ostermann,
Aljoscha Smolic, M. Reha Civanlar and John Watson
www.3dtv-research.org