Overview of Multi- view Video Coding Yo-Sung Ho; Kwan-Jung Oh; Systems, Signals and Image Proces sing, 2007 and 6th EURASIP Confer ence focused on Speech and Image Processing, Multimedia Communicat ions and Services. 14th Internati onal Workshop on
Dec 21, 2015
Overview of Multi-view Video Coding
Yo-Sung Ho; Kwan-Jung Oh;Systems, Signals and Image Processing, 2007 and 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services. 14th International Workshop on
Outline
Introduction Applications of Multi-view Video Requirements for Multi-view Video Coding Test Data Sets and Test Conditions Joint Multi-view Video Model (JMVM) Conclusion
Introduction
Introduction
Multimedia Demands holography two-view stereoscopic system with special glasses multi-view video
Multi-view Video: FVV, FVT, 3DTV
What is multi-view video? Why we need multi-view video coding (MVC)? MVC has been studied in the past.
MVP, MCP, DCP, MPEG4 MAC, H.263/4
Application of MVC
Application of MVC
Free Viewpoint Television (FTV) Three-dimensional TV (3DTV) Immersive Teleconference
FTV
What is FTV (Free Viewpoint Television)? Application of FTV:
Entertainment Education Sightseeing Surveillance Archive
FTV
3DTV
What is 3DTV? Interaction may not be required
To broadcast on 3DTV
3DTV
3DTV
Capture by various types of multiple cameras 1D parallel 2D parallel 1D arc …etc.
Intermediate view reconstruction (IVR)
Immersive Teleconference
What is immersive teleconference? Interaction
Requirements for Multi-view Video Coding
Requirements for Multi-view Video Coding
Requirements for multi-view video coding: Compression related requirements System support related requirements
Compression Related Requirements
Compression efficiency View scalability Free viewpoint scalability Spatial/Temporal/SNR scalability Backward compatibility Resource consumption Low delay Robustness
Compression Related Requirements
Resolution, bit depth, chroma sampling format
Picture quality among views Temporal random access View random access Spatial random access Resource management Parallel processing
System Support Related Requirements
Synchronization View generation Non-planar imaging and display systems Camera parameters
Test Data Sets and Test Conditions
Test Data Sets and Test Conditions
Test Data Sets and Test Conditions
Joint Multi-view Video Model
Joint Multi-view Video Model
Random Access
GGOP contains
frames. For accessing any frame within a GGOP, we
have to decode maximum number of frames.
b4(S5/T7), following 18 referencing frames.
Time-first coding order
Encoder Complexity
Minimum decoded picture buffer (DPB)
EX: GOP_length=16, number_of_views=8, the DPB size = 42
MVC codec will have the same coding delay as single view video coding since time-first coding is mandated.
GOP structures and view prediction structure
GOP structures and view prediction structure
GOP structures and view prediction structure
Illumination compensation
ICA MC: illumination change-adaptive motion compensation
Macroblocks(MB) mode in h.264/MPEG-4 AVC: Inter 16*16 mode, Direct 16*16 mode (include B_Skip), and P_Skip mode
DVIC: difference value of illumination change ICA ME: illumination change-adaptive motion
estimation
SAD calculation for the motion estimation of S*T blocks:
In order to compensate the illumination change
1
1 Illumination compensated residual signal
Other Technical Issues View-temporal prediction structure
Single video v.s. Multi-view video Three main coding structures
Encode multiple video sequences separately Utilizes inter-view correlation only Utilizes both temporal and inter-view correlation
View interpolation prediction Decoder side disparity estimation Computing depth at encoder side and transmitting this to t
he decoder Motion/Disparity vector coding
Highly correlated each other
Conclusion
Conclusion
The multi-view video includes multi-viewpoint video sequence captured by several cameras at the same time.
Compress multi-view video efficiently MPEG and JVT are leading the
standardization of MVC.