This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Slides are compiled from Alexander Kolesnikov’s image compression lecture notes
Block Truncation Coding
• Divide the image into 4×4 blocks;• Quantize the block into two representative values a and b;Quantize the block into two representative values a and b;• Encode (1) the representative values a and b
and (2) the significance map in the block.
Original Bit-plane Reconstructed
2 11 11 9
2 9 12 15
0 1 1 1
0 1 1 1
2 12 12 12
2 12 12 12
3 3 4 14
2 3 12 15
0 0 0 1
0 0 1 1
2 2 2 12
2 2 12 12
x = 7.94 q = 9 a = 2.3a=[2.3] = 2 = 4.91
q
b = 12.3σa [2.3] 2b=[12.3]=12
How to construct quantizer?
• The first two moments preserving quantization:
∑=
>=<m
iim xx
1
1 ∑=
>=<m
iim xx
1
212
222 ><−>=< xxσ
• Threshold for quantization: T=<x>; n +n =m• Threshold for quantization: T=<x>; na+nb=mbnanxm ba +>=<
222 bnanxm +>=<
bnxa ⋅−>=< σ anxb ⋅+= σ
bnanxm ba +>=<
anxa >=< σ
bnxb ⋅+= σ
Example of BTC
Original Bit-plane Reconstructed
2 11 11 9
2 9 12 15
0 1 1 1
0 1 1 1
2 12 12 12
2 12 12 12
3 3 4 14
2 3 12 15
0 0 0 1
0 0 1 1
2 2 2 12
2 2 12 12
T 9x = 7.94
= 4.91
q = 9 a = 2.3
b = 12.3σT= 9na=7nb=9
5043722D
a=[2.3] = 2b=[12.3]=12
5043722 =+=+= baD σσ
2 3 4 5 6 7 8 9 10 11 12 13 14 15a bT
Representative levels compression
• Main idea of BTC:
Image → ”smooth part” + ”detailed part” (a and b) (bit-planes)
• We can treat set of a’s and b’s as an image:1. Predictive encoding of a and b2. Lossy compression: DCT (JPEG)
• Bigger block → smaller bitrate R, bigger distortion D
• Smaller block → bigger bitrate R, smaller distortion D
Trade-off between Rate and DistortionTrade off between Rate and Distortion
8. JPEG
• JPEG = Joint Photographic Experts Group
• Lossy coding of continuous tone still images (color andgrayscale)
• Based on Discrete Cosine Transform (DCT):
0) Image is divided into block N×N
) h bl k f d h 2 C1) The blocks are transformed with 2-D DCT
2) DCT coefficients are quantized
3) Th ti d ffi i t d d3) The quantized coefficients are encoded
JPEG: Encoding and Decoding
Source FDCT Quantizer Entropy CompresseImage Data
8x8 blocks
FDCT Quantizer pyEncoder
pImage Dat
8x8 blocks TableSpecifications
TableSpecifications
IDCTDequantizerEntropyDecoder
Reconstructed Image Data
CompressedImage Data
TableS ifi ti
TableS ifi tiSpecifications Specifications
Divide image into N×N blocks
8x8 blockInput image
2-D DCT basis functions: N=8
Low Higho
LowHigh
Low
High High
8x8 block
High High
HighLow
2-D Transform Coding
+y00y01 y10
y12y23y01 y10
y12
...
1-D DCT basis functions: N=8
1.0u=0
1.0u=1
1.0u=2
1.0u=3
0.5
0
-0.5
0.5
0
-0.5
0.5
0
-0.5
0.5
0
-0.5
-1.0 -1.0 -1.0 -1.0
1.0
0 5
u=41.0
0 5
u=51.0
0 5
u=61.0
0 5
u=7
0.5
0
-0.5
-1.0
0.5
0
-0.5
-1.0
0.5
0
-0.5
-1.0
0.5
0
-0.5
-1.0
( ) ( ) ( )∑−
⎥⎦⎤
⎢⎣⎡ +
⋅=1
212cos
N
j NkjkCkx πα ( )
⎪
⎪⎨
⎧ ==
121f2
0for 1
Nk
kNkα∑=
⎥⎦⎢⎣0 2kj N ⎪
⎩−= 1,...,2,1for 2 NkN
2-D DCT basis functions: N=M = 8where
Zig-zag ordering of DCT coefficients
Converting a 2-D matrix into a 1-D array, so that the g y,frequency (horizontal and vertical) increases in this orderand the coefficents variance are decreasing in this order.
• Color 24 bits images: - 0.25-0.50 bpp: moderate to good - 0.50-0.75 bpp: good to very good- 0 75-1 00 bpp: excellent sufficient for most applications0.75 1.00 bpp: excellent, sufficient for most applications - 1.00-2.00 bpp: indistiniguishable from original
JPEG ⇒JPEG2000
For illuminanceFor illuminance
JPEG: 0.25 bpp JPEG2000: 0.25 bpp
9. JPEG 2000
• JPEG 2000 is a new still image compression standard• ”One for all” image codec:• One-for-all image codec:
* Different image types: binary, grey-scale, color, multi-componentmulti-component
* Different applications: natural images, scientific, medical remote sensing text rendered graphicsmedical remote sensing text, rendered graphics
* Different imaging models: client/server, consumer electronics, image library archival, limited bufferelectronics, image library archival, limited buffer and resources.
History
• Call for Contributions in 1996Call for Contributions in 1996• The 1st Committee Draft (CD) Dec. 1999• Final Committee Draft (FCD) in March 2000( )• Accepted as Draft International Standard in Aug. 2000• Published as ISO Standard in Jan. 2002
Region of interest (ROI)Lossless color transformError resilience
2-D wavelet transform
Original128, 129, 125, 64, 65, …
Transform Coeff.4123, -12.4, -96.7, 4.5, …
Quantization of wavelet coefficients
Transform Coeff.4123, -12.4, -96.7, 4.5, …
Quantized Coeff.(Q=64)64, 0, -1, 0, …
Entropy coding
0 1 1 0 1 1 0 1 0 1 . . .
Coded BitstreamCoded Bitstream
Quantized Coeff.(Q=64)64, 0, -1, 0, …
Progressive encoding
ROI: Region of interest
Scale-down the coefficients outside the ROI so those are in lowerer bit-planes.Decoded or refined ROI bits before the rest of the image.
ROI: Region of interest
• Sequence based codeq– ROI coefficients are coded as independent sequences– Allows random access to ROI without fully decoding– Can specify exact quality/bitrate for ROI and the BG
• Scaling based mode:S l ROI k ffi i t (d d l d )– Scale ROI mask coefficients up (decoder scales down)
– During encoding the ROI mask coefficients are found significant at early stages of the codingsignificant at early stages of the coding
– ROI always coded with better quality than BG– Can't specify rate for BG and ROI
• Video images are three-dimensional generalizationVideo images are three dimensional generalization of still images, where the third dimension is time
E h f f id b d b• Each frame of a video sequence can be compressed by any image compression algorithm
• Motion JPEG (M-JPEG): the images are separately coded by JPEG
Correlation
• Let’s take an advantage of the temporal correlations;Let s take an advantage of the temporal correlations; i.e. the fact that subsequent images resemble each other very much:
• Still Images : spatial correlation • Video Images: spatial and temporal correlation
MPEG-1
The MPEG algorithm relies on two basic techniquesg q• Block based motion compensation• DCT based compression
MPEGs
• MPEG-1 (1992): VideoCD • MPEG-2 (1994): DVD, digital TV, SVCDMPEG 2 (1994): DVD, digital TV, SVCD
* about 50:1 compression, typically 3-10 Mbps • MPEG-3: was abandoned
G ( ) ( f )• MPEG-4 (1999+): DivX (starting from Version 5) * designed specially for low-bandwidthMPEG 7 (>1998):• MPEG-7 (>1998): * searching and indexing of a/v data, using DescriptionToolsTools
MPEG-1: Blocks
• The pictures are divided into 16x16 macroblocks, each consisting of four 8x8 elementary blockseach consisting of four 8x8 elementary blocks.
• The choice of the prediction method is chosen for eachmacroblock separately.
• The intra-coded blocks are quantized differently from the predicted blocks:the predicted blocks: * Intra-coded blocks contain information in all frequencies
and are quantized differently from the predicted blocks * Th di t d bl k t i tl hi h f i* The predicted blocks, contain mostly high frequencies
and can be quantized with more coarse quantization tables.
I: Intra pictures are coded as still images by DCT. p g yP: Predicted pictures are coded with reference to a past
picture. The difference between the prediction and the original picture is then compressed by DCT.
B: Bidirectional pictures, the prediction can be made both t t d f t f Bidi ti l i tto a past and a future frame. Bidirectional pictures are never used as reference.
Forward Forward
I B B B P B B B
Forwardprediction
P B B B I
Forwardprediction
Bidirectional prediction
Motion estimation and compensation
• The prediction block in the reference frame is not necessarily in the same coordinates than the block innecessarily in the same coordinates than the block inthe current frame.
• Because of motion in the image sequence, the most suitable predictor for the current block may exist anywhere in the reference frame.
• The motion estimation specifies where the best prediction (best match) is found. M ti ti i t f l l ti th diff• Motion compensation consists of calculating the difference between the reference and the current block.
Motion estimation: 1
• Exhaustive search block matchingg
Slow!
Motion estimation: 2
• Hierarchical block matchingg
Compression ratios
NTSC TV HDTV Film qualityChannel Bit rate 168 Mb/s 933 Mb/s 2300 Mb/s