computer graphics & visualization Image Synthesis Image Compression
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Motivation1920 x 1440 x 32 bit = 10,54 MB
High Quality JPEG 0,9 MB
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Classification• Lossless– TGA– GIF– PNG– …
• Lossy– JPEG– JPEG 2000– MPEG
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Lossless Compression schemes
• RLE• LZ– LZ77– LZ78 / LZW
• Huffmann• Deflate / “ZIP”
• GIF• PNG
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Run length EncodingExample:WWWWWWWWWWBWWWWWWWWWWBBB 10WB10W3B
What to do if numbers appear in the code?Uses control char (X in this case) and only encode sequences of more than two chars.121211111111122222212111 1212X91X6212111
What if X appears in the code?Use XX to encode a regular X (and hope that you will not see too many Xs in you data).
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
LZ - Lempel-ZivLZ77 (1977)Idea: Conversion of the data in (hopefully only few) 3-tupel Implementation: sliding window + look ahead approach
Example: ANANAS
01234567 0123 Output
ANAN (0,0,A)
A NANA (0,0,N)
AN ANAS (6,2,A)
ANANA S (0,0,S)
ANANAS Done
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DecodingLZ77 (1977)Idea: Conversion of the data in (hopefully only few) 3-tupel Implementation: sliding window + look ahead approach
Example: ANANAS
Input 012334567
(0,0,A) A
(0,0,N) AN
(6,2,A) ANANA
(0,0,S) ANANAS
Done
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Implementation DetailsEncoding of periodic entries
Example: 0101010101010101010101… (length n)
01234567 0123 Output
0101 (0,0,’0’)
0 1010 (0,0,’1’)
01 0101 (6,n-2,’1’)
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Implementation ChallengesFinding the right position
Example: 0120101601015
01234567 0123 Output
0120 (0,0,’0’)
0 1201 (0,0,’1’)
012 0101 (5,2,’0’)
012010 1601 (0,0,’1’)
0120101 6010 (0,0,’6’)
01201016 0101 (0,2,’1’) or (3,4,’5’)
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
ProblemsEncoding of non-periodic
Example: 0123456789ABCDE…
“Compression” actually increases data size by a factor of three Combination with Huffman encoding Deflate (later)
01234567 0123 Output
0123 (0,0,’0’)
0 1234 (0,0,’1’)
01 2345 (0,0,’2’)
012 3456 (0,0,’3’)
... .... .........
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
LZW - Lempel-Ziv-WelchDictionary based compression with adaptive dictionaryUsually 12 bit codebook, first 8 bits reserved for ASCII
LZWLZ7…string Entry Output New Entry
LZWLZ7… L L LZ <256>
ZWLZ7… Z Z ZW <257>
WLZ7… W W WL <258>
LZ7… LZ<256> <256> LZ7 <259>
7… 7 7 7. <260>
L Z W <256> 7…
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Decoding
Current Char New Entry Output
L - - L
Z Z LZ <256> Z
W W ZW <257> W
<256> L WL <258> LZ
7 7 LZ7 <259> 7
…
LZWLZ7…
L Z W <256> 7…
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Features and Issues
• „Codebook overflow/reset“• Simple deterministic encoding / decoding• Worst-Case 8 bit 12 bit (or 8 bit -> dict size)• was subject to software patent until 2002 (2006)
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Huffman Encoding 1951Given:A set of Symbols and their weights (usually probabilities).
Find:A prefix-free binary code (a set of codewords) with minimum expected codeword length (equivalently, a tree with minimum weighted path length).
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Variable Length Problem Example:
What is the meaning of 10010?
ACA (10 0 10) or ABC (10 01 0)?
Prefix – free codes
A B C
10 01 0
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Huffman EncodingStart with as many leaves as there are Symbols. Queue all leaf nodes into the first queue (in order). While there is more than one node in the queues:
– Remove two nodes with the lowest weight (frequency of appearance) from the queues.
– Create a new internal node, with the two just-removed nodes as children (either node can be either child) and the sum of their weights as the new weight.
– Update the parent links in the two just-removed nodes to point to the just-created parent node.
– Queue the new node into the second queue.
The remaining node is the root node; the tree has now been generated.
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Example
A
B
C
D
E
A B C D E
15 7 6 6 5
A B C D E
0 100 101 110 111
Codes:
Frequencies:
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Properties• Huffman coding is optimal
i.e. it generates an optimal tree for a given input• Requires knowledge of frequencies– can be extended to do adaptive tree updates
• Can be combined with LZ77 a post processing step, thus compressing the lengthy tuples Deflate, used primarily in ZIP, gzip, etc.
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Now back to imagesBMP, TGAstore images as an RLE array of n-bit (n usually 24/32) values
GIFUses 256 colors (8bit) and uses LZW compressiona few GIF versions exist, usually GIF89a is used today, allowing for a single transparent color and stacking multiple images together to generate an animation
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
PNG – Portable Network GraphicInitially developed to circumvent GIFs, patent pending LZW compression.
• uses prefilter + LZ77 + Huffman for compression• supports not only 8bit palettes but also 1,2,4,8
or 16bit per channel color information• Supports 8 or 16 bit alpha channel• Now finally supported by all major browsers• No support for animation
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
PNG Prefilter
ID Name Description
0 None Use original pixel values
1 Sub Use difference to left pixel
2 Up Use difference to upper pixel
3 Average Compute Average of upper and left and use difference
4 Paeth Use difference to paeth predictor
Used per line
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Paeth Predictorfunction PaethPredictor (a, b, c){ a = left, b = above, c = upper left p := a + b - c // initial estimate pa := abs(p - a) // distances to a, b, c pb := abs(p - b) pc := abs(p - c) // return nearest of a,b,c, // breaking ties in order a,b,c.
if pa <= pb AND pa <= pc then return a else if pb <= pc then return b else return c}
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Paeth Predictor: Idea
c b
a xGoal: Minimize Filter(x)
“Optimal way” to do this: Compute abs(a-x), abs(b-x), abs(c-x) and store minimal value
But this would require two bits to remember what difference was used.
Instead: compute a “good” estimate of x using only a,b,c (called p) and find the minimalabs(a-p), abs(b-p), abs(c-p) to select which of abs(a-x), abs(b-x), abs(c-x) is used
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Lossy CompressionJPEG (Joint Photographic Experts Group)• Lossless compression not working well for photos
• Useses lossy compression withadjustable compression ration
• Can do lossless • Fast deconding• Works for all types of static images,
no restrictions on color depth
Compression ratio
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Compression Overview• Color Space conversion to YUV• Block based color sub sampling• Discrete Cosine-transformation
(DCT) • Quantization of the DCT
Coefficients• Coefficient serialization
(dove tailing)• Coefficient encoding
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
YUVYUV
Y is LuminanceU Chrominance: Color change towards blueV Chrominance: Color change towards red
Red : Y= 77 U= -43 V= 127 Green: Y= 151 U= -84 V= –107 Blue : Y= 28 U= 127 V= -20
Example:
YUV ConversionY = 0.30 R + 0.59 G + 0.11 BU = -0.17 R - 0.33 G + 0.50 BV = 0.50 R - 0.42 G - 0.08 B
RGB ConversionR = Y + 1.40 VG = Y - 0.34 U - 0.71 VB = Y + 1.78 U
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
YUV Conversion Example
Y
U
V
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
YUVSub sampling Split YUV-ChannelsEye is more sensitive to brightness than to color changes:Color sub sampling
Default: Mode 4:2:2 Full Quality : Mode 4:4:4
• First digit: Luma horizontal sampling reference• Second and Third:
U and V (chroma) horizontal factor (relative to first digit) Except when third is zero. Zero indicates that V horizontalfactor is equal to second digit, and, in addition, both U and V are subsampled 2:1 vertically
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Sub sampling4:4:4 best quality 4:2:2 best quality
4:4:4 worst quality 4:2:2 worst quality
Without Sub sampling With Sub sampling
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
Preparation for DCT
From now on consider YUV planes separately
Index shiftTransform [0 - 255] to [-128 - 127]
BlockingAlways consider a 8x8 data block for the DCT
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT (1D)Related to FT in that it transforms a spacial signal to the frequency domain, however the DCT avoids the imaginary sine part by representing the signal as a sum of cosine waves
1t=0
( ) 2sonst
1
C t
DCT(u): Cosine amplitude for frequency uN: number of pixels 8x8=64 for JPEGf(x): Pixel value at position xC: correction factor
1
0
2 12( ) ( ) ( ) cos
2
N
x
x uDCT u C u f x
N N
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT (2D)
1t=0
( ) 2sonst
1
C t
Large, regular areas are stored in the upper right while higher frequencies are stored towards the lower left
1 1
0 0
2 1 2 11( , ) ( ) ( ) ( , ) cos cos
2 22
N N
i j
x i y jDCT i j C i C j Pixel x y
N NN
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCT
Pixel
(IDCT)DCT
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCT
Pixel
(IDCT)DCT
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCT700 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
87 87 87 87 87 87 87 87
87 87 87 87 87 87 87 87
87 87 87 87 87 87 87 87
87 87 87 87 87 87 87 87
87 87 87 87 87 87 87 87
87 87 87 87 87 87 87 8787 87 87 87 87 87 87 87
87 87 87 87 87 87 87 87
DCT IDCT
Result
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT IDCT
Result
700 100 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 00 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70105 102 97 91 84 78 73 70
105 102 97 91 84 78 73 70
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT IDCT
Result
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT IDCT
Result
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT IDCT
Result
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCTDCT IDCT
Result
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DCT PropertiesA real life photo has the following properties after DCT
the coefficient DC is by far the largestthe ACs become smaller with increasing frequencymost of the ACs are close to zero
Properties of the DCT?The DCT in general is a reversible transform thus applying the DCT and back transforming is lossless (in theory)
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
QuantizationFor JPEG the DCT gives us 64 Values, each of them is divided bya quantization matrix entry and rounded afterwards
Quantizermatrix
BacktransformationMultiply DCT-value with Quantizermatrix:
Quality SettingsAdjust Matrix Q
Data is lost here!
( , )( , )
( , )
DCT i jDCT i j Round
Q i j
( , ) ( , ) ( , )DCT i j DCT i j Q i j
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
QuantizationDCT
After quantization
Matrix Q
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DC EncodingStore first (upper left DC) as is, encode other values as differences from predecessor
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
DC EncodingCategorize values
Huffman encode category sequence and append index into category results in 2 tuple
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
AC Encoding
Symbol-1(Runlength, Size)
Symbol-2(Amplitude)
Runlength = Zero countSize = CategoryAmplitude = Index
First block is Huffman encoded (or table is used) and second block is appended
(2)(3), (1,2)( 1), (0,1)( 0), (0,1)( 0), (0,1)( 0), (2,1)(0), (0,0) ~ 51Bit
Runlength is 4 bit [0-15]
DC AC
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
JPEG 2000• does not necessarily split the image into blocks• uses wavelets instead of DCT• uses zero-tree encoder (EBCOT) instead of Huffman• embedded code allows for better preview
• better compression• better quality
JPG JPEG 2000
computer graphics & visualization
Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group
H.261• Divide image into 8x8 blocks• Motion compensation via cross correlation• Subtract from previous frame• Compress JPEG like 4:2:0• Uses one of 31 fixed quantizer matrices
+ one uniform matrix for DC