Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz
Dec 19, 2015
Image Sequence Codingby Split and Merge
Patrice Willemin, Todd R. Reed and Murat Kunt
Presented by: Idan Shatz
Outline
• Split and Merge Coding– For Still Images
• Split and Merge Coding– For Image Sequence
• Conclusions
Image Coding
• First Generation– Pels coding (prediction or transform)
• Second Generation– Object Based coding– Region based coding
• Split and Merge Method
Split & Merge
• Split…
• Recursive algorithm– Approximate a region by a 2D polynomial– If (approximation error > threshold)
• Split the region into 4 sub-regions• Approximate each sub-region
Split Example
Original Image Split, depth=1
Split Example
Original Image Split, depth=2
Split Example
Original Image Split, depth=3
Split Example
Original Image Split, depth=4
Split Example
Original Image Split, depth=5
Split Example
Original Image Split, depth=6
Split Example
Original Image Split, depth=7
Split Example
Original Image Split, Final resultsThreshold=100,000
The threshold100,000 50,000 20,000 10,000 5,000
2,000 1,000 500 200 100
5,000
100,000
100
102
104
106
108
0
500
1000
1500
2000
2500
3000
3500
4000
4500
# of
reg
ions
Threshold
The threshold
Split & Merge
• An improvement of the Split algorithm– At each step decide between x-split and y-split
• produces fewer regions
Split Example (2)
Original Image Split, depth=1
Split Example (2)
Original Image Split, depth=2
Split Example (2)
Original Image Split, depth=3
Split Example (2)
Original Image Split, depth=4
Split Example (2)
Original Image Split, depth=5
Split Example (2)
Original Image Split, depth=6
Split Example (2)
Original Image Split, depth=7
Split Example (2)
Original Image Split, depth=8
Split Example (2)
Original Image Split, depth=9
Split Example (2)
Original Image Split, depth=10
Split Example (2)
Original Image Split, depth=11
Split Example (2)
Original Image Split, depth=12
Split Example (2)
Original Image Split, depth=13
Split Example (2)
Original Image Split, Final resultsThreshold = 100,000
Split Example (Comparison)
Split – 4 sub regionsThreshold = 100,000# regions = 613
Split – 2 sub regionsThreshold = 100,000# regions = 387
Comparison
100
102
104
106
108
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Threshold
# of
reg
ions
4-split2-split
Image Coding
• Each region is codes by:– The Border– 2D polynomial coefficients
The borders of the regions
• BSP-Tree– Binary space partition tree– Tree Nodes:
• “X” – mark an X-split• “Y” – mark an Y-split• “0” – mark an Region
Regions Borders
• Example of BSP-Tree
Y
X 0
0 Y
X
0 0
0
Storing a BSP Tree
• N regions 2N-1 BSP-Nodes
• Storing the tree in preorder
Y
X 0
0 Y
X
0 0
0
Y X Y X 00 0 0 0
Image Coding
• For N regions– Border (BSP) – N*3[bit]– 2D polynomial coefficients – N*24[bit]
• 8[bit] x 3 – gray level.• 8[bit] x 3 x 3 - for RGB.
Image Coding
• Without Compression– 196,608[Byte]
• With Compression– # regions = 387– 3628[Byte]
• Compression Raito:– 1 to 54
Split – 2 sub regions
Split & Merge
• Merge…– For each region:
• 3[bit] border << 54[bit] Coefficients
• Reduce the number of regions– Merging neighbor regions with similar
coefficients.
Merge Example (Comparison)
Split – 4 sub regionsThreshold = 100,000# regions = 613After Merge# region = 411
Split – 2 sub regionsThreshold = 100,000# regions = 387After Merge# regions = 348
Comparison
100
101
102
103
104
105
106
107
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Threshold
# of
reg
ions
split4split2split4 + mergesplit2 + merge
The borders of the regions
• The borders are more complex– Simple BSP Tree is not enough
• Therefore– BSP Tree– Regions labels
Regions Labeling
• How many bits are needed for labeling?– ~1000 regions ~10[bit]
• Non-neighbor regions can have the same labeling– ~3[bit] for labeling– Does not depend on the
number of regions
Image Coding
• For N regions made from M BSP-regions– Border (BSP) – M*3[bit]– Labeling – M*3[bit]– 2D polynomial coefficients – N*24[bit]
• 8[bit] x 3 – gray level.• 8[bit] x 3 x 3 - for RGB.
Image Coding
• Without Compression– 196,608[Byte]
• With Compression– # BSP regions = 387– # regions = 348– 3446[Byte]
• Compression Raito:– 1 to 57 (>54)
Split – 2 sub regionsAnd merging
• Same Idea.– Still Image (x,y)– Video (x,y,t)
• Split Stage– Decide between x-split, y-split and t-split.
• Merge Stage– Same
Image Sequence Coding
y
x
t
Real Time Transmission
• Compress all movie at once– Achieving best compression.– However
• can’t be transmitted on-line• Very high complexity
• Deal with chunks of 5~25 frames– Compression ratio decreases– Can be transmitted on-line
Conclusions
• Still Images compression– Achieves good results with good compression
ratio.– 2 major defects due to region boundaries:
• Disappearance of important boundaries• Border artifacts between regions.
– Merge Regions• using the same threshold has little effect on performance• adaptive threshold for achieving a giving compression
ratio
Conclusions
• Image Sequence compression– Achieves good results
• At the article: 1 to 200 comp. ratio.
– But, compression ratio decreases on high movement scenes.
• compression ratio still images.• Merging regions can be used to balance the buffer size.
– Hard to decode at real-time.• Must be coded at frame chunks.• Decoding process is hard to implement in real time
environment.
Go To…• Still Images
– Split• to 4 sub-regions• to 2 sub-regions• Threshold (2)• Compression
– Merge• Threshold• Compression
• Image Sequence• Conclusions