C. -Y. Tsai 2006/4/20 1 Communication & Multimedia Multiview Video Compression Multiview Video Compression Student: Chia-Yang Tsai Student: Chia-Yang Tsai Advisor: Prof. Hsueh-Ming Hang Advisor: Prof. Hsueh-Ming Hang Institute of Electronics, NCTU Institute of Electronics, NCTU
Multiview Video Compression. Student: Chia-Yang Tsai Advisor: Prof. Hsueh-Ming Hang Institute of Electronics, NCTU. Outline. Introduction Coding methods Performance Conclusions. References. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
C. -Y. Tsai 2006/4/201Communication & Multimedia
Multiview Video CompressionMultiview Video Compression
Student: Chia-Yang TsaiStudent: Chia-Yang TsaiAdvisor: Prof. Hsueh-Ming HangAdvisor: Prof. Hsueh-Ming Hang
Institute of Electronics, NCTUInstitute of Electronics, NCTU
Reasons for multiview compressionReasons for multiview compression PC is powerful enoughPC is powerful enough Increasing network bandwidthIncreasing network bandwidth Future 3D videoFuture 3D video
C. -Y. Tsai 2006/4/207Communication & Multimedia
MPEG StandarizationMPEG Standarization
Call for proposal (N7567, Oct. 2005)Call for proposal (N7567, Oct. 2005) Proposal competition (M12969, Jan. 2006)Proposal competition (M12969, Jan. 2006)
NTT and Nagoya UniversityNTT and Nagoya University Thomson and University of Southern CaliforniaThomson and University of Southern California KDDI Corp.KDDI Corp. ETRI and Sejong University (=M12871)ETRI and Sejong University (=M12871) MERL (=M12828)MERL (=M12828) KBS and Yonsei University (=M12874)KBS and Yonsei University (=M12874) Fraunhofer-HHI (=M12945)Fraunhofer-HHI (=M12945) Technical University of BerlinTechnical University of Berlin
Different viewpoints have different Different viewpoints have different depthdepth Rotation, translation speedRotation, translation speed
SolutionSolution Synthesis virtual images before real Synthesis virtual images before real
predictionprediction
C. -Y. Tsai 2006/4/2015Communication & Multimedia
VSPVSPtime
view
View Synthesis ViaView Interpolation
View Synthesis ViaView Warping
)(],,,[]1,,[)()(],,[ 1 cTyxtcDyxcAcRwvu
)}'(],,{[)'()'(]',','[ 1 cTwvucRcAzyx
R: Rotation matrixR: Rotation matrix
D: Depth informationD: Depth information
T: Translation matrixT: Translation matrix
A: Intrinsic matrixA: Intrinsic matrix
C
C’
C. -Y. Tsai 2006/4/2016Communication & Multimedia
How to get depths?How to get depths?
From camera recordFrom camera record From well-known computer vision algorithmsFrom well-known computer vision algorithms Block-based depth searchBlock-based depth search
||]',',,'[],,,[||),,,( min}
max...2
min,
min,
min{
yxtcIyxtcIyxtcD
Dstep
DDstep
DDDd
where || I [c,t,x,y] – I [c’,t,x’,y’] || denotes the average error between the block at (x,y) in camera c at time t
C. -Y. Tsai 2006/4/2017Communication & Multimedia
How to get depths?How to get depths?
Depths map:Depths map: Left: computer vision algorithmLeft: computer vision algorithm Right: block based depth searchRight: block based depth search
Compression result:Compression result: Depth information: 5-10% total bitratesDepth information: 5-10% total bitrates Left and right have equal performanceLeft and right have equal performance
C. -Y. Tsai 2006/4/2018Communication & Multimedia
Prediction structurePrediction structure
C. -Y. Tsai 2006/4/2019Communication & Multimedia
Hierarchical B picturesHierarchical B pictures
Hierarchical B picturesHierarchical B pictures Fully compatible to AVC Main profileFully compatible to AVC Main profile Non-dyadic decomposition is availableNon-dyadic decomposition is available
A B3 B2B1 AB3 B3 B3B2
L3 H1 H2H3H2 L3H1 H1 H1
GOP boundaries
AVC Main Profilecompatible base layer
MCTF enhancementlayer
C. -Y. Tsai 2006/4/2020Communication & Multimedia
Hierarchical B picturesHierarchical B pictures
C. -Y. Tsai 2006/4/2021Communication & Multimedia
PerformancePerformance
C. -Y. Tsai 2006/4/2022Communication & Multimedia
ExperimentsExperiments
C. -Y. Tsai 2006/4/2023Communication & Multimedia
ExperimentsExperiments
C. -Y. Tsai 2006/4/2024Communication & Multimedia
ConclussionConclussion
C. -Y. Tsai 2006/4/2025Communication & Multimedia
ConclussionConclussion
DCVP & VSP can reduce the correlation DCVP & VSP can reduce the correlation between viewpointsbetween viewpoints