Marc CHAUMONT ICIP 2003 Fully scalable object based video coder based on analysis- synthesis scheme Marc Chaumont, Nathalie Cammas 1 and Stéphane Pateux.
Post on 03-Jan-2016
214 Views
Preview:
Transcript
Marc CHAUMONTICIP 2003
Fully scalable object based video coder based on analysis-
synthesis schemeMarc Chaumont, Nathalie Cammas1 and Stéphane Pateux
Temics, IRISA/INRIA, France, 1France Telecom, France
Marc CHAUMONTICIP 2003
What is scalability ? ordering the coded information by decreasing significant order. different scalability : object, SNR, temporal, spatial, bitstream.
Why looking for scalability ? bitstream can be decoded at different bitrate and different qualities
Why using an object coder instead of an non object coder ? object manipulation bitrate allocation better motion estimation (limit mesh degeneracy on occlusion boundaries) good tradeoff between pixel based and model based approach.
Context
Marc CHAUMONTICIP 2003
Objectives : scalability on the 3 informations : motions - textures - shapes independent coding of the 3 informations long term approach
Why those objectives : scalability : … independent coding : to allow a better rate distribution between motion -texture - shape. long term approach : to allow an efficient coding with wavelet
Our proposed solution : using wavelet (to allow scalability) decorrelation (to allow independent coding) long term motion estimation (to allow a long term approach)
Proposed approach
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
z-ordershapes
frames
z-ordershapes
frames1 - ANALYSIS2 - CODING3 - DECODING
z-order
NOSHAPE
frames
z-order
frames
shapes
1 - ANALYSIS2 - CODING3 - DECODING
sequence
SYNTHESIS
SEGMENTATION
obj1
obj2reconstructed
sequence
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
z-ordershapes
frames
z-ordershapes
frames1 - ANALYSIS2 - CODING3 - DECODING
z-order
NOSHAPE
frames
z-order
frames
shapes
1 - ANALYSIS2 - CODING3 - DECODING
sequence
SYNTHESIS
SEGMENTATION
obj1
obj2reconstructed
sequence
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
z-ordershapes
frames
z-ordershapes
frames1 - ANALYSIS2 - CODING3 - DECODING
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
z-ordershapes
frames
z-ordershapes
frames1 - ANALYSIS2 - CODING3 - DECODING
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
1 - ANALYSIS2 - CODING3 - DECODING
z-ordershapes
frames
z-ordershapes
frames
GOP GOP
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
long termmotion
estimation(active mesh) decorrelation
thanks tomotion
and paddingz-order
shapes
frames
z-ordershapes
frames
GOP
SYNTHESIS
motions
contours
textures
motions
textures
contours
codingdecoding
codingdecoding
codingdecoding
GOP
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
long termmotion
estimation(active mesh) decorrelation
thanks tomotion
and paddingz-order
shapes
frames
z-ordershapes
frames
GOP
SYNTHESIS
motions
contours
textures
motions
textures
contours
codingdecoding
codingdecoding
codingdecoding
GOP
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
long termmotion
estimation(active mesh) decorrelation
thanks tomotion
and paddingz-order
shapes
frames
z-ordershapes
frames
GOP
SYNTHESIS
motions
contours
textures
motions
textures
contours
codingdecoding
codingdecoding
codingdecoding
GOP
Marc CHAUMONTICIP 2003
Decorrelation:motion projection & padding
textures projected
contours projected
contours projected
MOTIONPROJECTION PADDING
MOTIONPROJECTION PADDING
contours padded
textures paddedinitial textures
initial contours
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
long termmotion
estimation(active mesh) decorrelation
thanks tomotion
and paddingz-order
shapes
frames
z-ordershapes
frames
GOP
SYNTHESIS
motions
contours
textures
motions
textures
contours
codingdecoding
codingdecoding
codingdecoding
GOP
Marc CHAUMONTICIP 2003
motion - texture - contour
motion (mesh) textures projected and padded contours projected and padded
3 independent informationGOP size = 8
Marc CHAUMONTICIP 2003
Decorrelation allows :
independent coding of the 3 informations : motion -texture - shape. allows independent lossy coding on each information
a better rate distribution between motion -texture - shape.
distribution example for Foreman foreground CIF 15Hz at 85 Kb/s :
texture : 76 %
motion : 17 %
shape : 7 %
long term approach efficient coding with wavelet
Example : our scheme on Erik sequence CIF 15Hz at 64Kb/s is better than H26L VM 8.4 (less than 1 dB)
scalability on the 3 information thanks to wavelet
Benefits ofthe decorrelation
Marc CHAUMONTICIP 2003
Proposed approachanalysis-synthesis scheme
long termmotion
estimation(active mesh) decorrelation
thanks tomotion
and paddingz-order
shapes
frames
z-ordershapes
frames
GOP
SYNTHESIS
motions
contours
textures
motions
textures
contours
codingdecoding
codingdecoding
codingdecoding
GOP
Marc CHAUMONTICIP 2003
Coding step
textures
contours
motions
Spatialdecomposition
(9/7 Daubechies filter)
Temporal decomposition
(5/3 lifting filter)
Temporaldecomposition
(9/7 filter)
Spatialdecomposition
(9/7 Daubechies filter)
Temporal prediction
IPB
Spatialpyramidal
decomposition
Bit planarithmetic
coder
EBCOT
Bit planarithmetic
coder
Spatio-temporal transformation
Marc CHAUMONTICIP 2003
Results
Background object :
Foreground object :
Shape : 3 Kb/s
Texture : 39,7 Kb/s Motion : 7,3 Kb/s
Texture + Motion : 11,5 Kb/s
Reconstructed sequence at 62 Kb/s
Marc CHAUMONTICIP 2003
With or without shape distortion
Results
Without shape distortion Reconstructed sequence at 62 Kb/s(with shape distortion)
Marc CHAUMONTICIP 2003
H264/AVC versus our object scalable scheme
Results
Our scheme62 Kb/s
PSNRtexture-foreground = 29.3
H264/AVC non object62 Kb/s
PSNRforeground = 27.9. 1 B frame,. RD optimization,. CABAC.
Marc CHAUMONTICIP 2003
Spatial scalability
Results
Sequence 62 Kb/s CIF 15 Hz
shape : 3 Kb/sbackground : text + mvt : 11,5 Kb/sforeground : text + mvt : 47 Kb/s
Sequence 49 Kb/s QCIF 15 Hz
shape : 1,7 Kb/sbackground : text + mvt : 9,6 Kb/sforeground : text + mvt : 37,5 Kb/s
Marc CHAUMONTICIP 2003
Spatio-temporal scalability
Results
Sequence 49 Kb/s QCIF 15 Hz Sequence 36 Kb/s QCIF 7.5Hz
shape : 1,2 Kb/sbackground : text + mvt : 6,7 Kb/sforeground : text + mvt : 27,9 Kb/s
shape : 1,7 Kb/sbackground : text + mvt : 9,6 Kb/sforeground : text + mvt : 37,5 Kb/s
Marc CHAUMONTICIP 2003
SNR scalability
Results
Sequence 49 Kb/s CIF 15 Hz Sequence 126 Kb/s CIF 15 Hz
shape : 6,8 Kb/sbackground : text + mvt : 24,4 Kb/sforeground : text + mvt : 94,9 Kb/s
shape : 1,7 Kb/sbackground : text + mvt : 9,6 Kb/sforeground : text + mvt : 37,5 Kb/s
Marc CHAUMONTICIP 2003
Based on : analysis-synthesis scheme decorrelation of the 3 informations
(active mesh - padding - z-order) fully scalable
Benefits : fully scalable (SNR, spatial, temporal & on each information) independent coding of the 3 informations
better bitrate distribution long term approach
allow the use of longer wavelet kernels
Future work : improving texture coding improving motion estimation in occlusion part
Conclusion :A novel object based video coder
top related