JPEG2000: An Introduction Part Ishirani/multi08/jpeg2000part1.pdf · • JPEG coding standard (1988): DCT (discrete cosine transform) based transform coding to compress bit-map images.

JPEG2000: An Introduction

Part I

Copyright S. Shirani

Introduction •  Joint Photographic Experts Group (JPEG) is an ISO standard

committee with a mission on “Coding and compression of still images”.

•  JPEG coding standard (1988): DCT (discrete cosine transform) based transform coding to compress bit-map images.

•  JPEG2000 efforts started in 1996 to use new methods such as fractals or wavelets. The target deliver date was year 2000 and hence the name.


JPEG2000: Objectives

•  Low bit-rate compression performance •  Lossless and lossy compression •  Large images

–  greater then 64K by 64K without tiling •  Single decompression architecture •  Transmission in noisy environments

–  Beyond restart intervals •  Computer generated imagery •  Compound document

–  text, natural image, and graphics mixed. •  Random code stream access and processing •  Progressive transmission by pixel accuracy and resolution


JPEG2000 features • High compression efficiency • Lossless color transformations • Lossy and lossless coding in one algorithm • Embedded lossy to lossless coding • Progressive by resolution and quality • Static and dynamic Region-of-Interest • Error resilience • Visual (fixed and progressive) coding • Multiple component images • Palletized Images


JPEG at 0.125 bpp (enlarged)

C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial)


JPEG2000 at 0.125 bpp

C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial)


Transformed Image Coding

(b) Decoder

Transform Quantization Lossless Coding

Lossless Decoding Dequantization Inverse

Transform

Bit Allocation

(a) Encoder

(b) Decoder

f F

Fq g

Fq


Linear Transform •  Energy compaction:

After transformation, it is desirable to have only few coefficients that are significant (large magnitude). Remaining coefficients are insignificant and can be discarded without significantly degrading perceptual quality of reconstructed image.

•  Optimal transformation –  K-L expansion (principal

component analysis) –  Data dependent basis

•  Data independent basis –  Discrete cosine transform –  Discrete Fourier transform –  Discrete wavelet transform –  Filter banks (sub-band

decomposition)


Quantization •  The purpose of quantization is to

approximate each transformed coefficient with few prototypical values so that fewer bits will be needed to encode the transformed coefficients.

•  Quantization is used only in lossy compression.

•  Progressive quantization –  Incrementally improve accuracy

by adding new quantization levels (prototypes).

–  When all bits are encoded, becomes lossless quantization.

•  Bit plane coding –  Progressive scalar quantizer –  Binary representation –  Start from MSB toward LSB


Bit Plane Coding

•  Coefficients are represented in sign-magnitude format •  Bit plane starts from the most significant bit (MSB) •  Sign bit is encoded after the MSB is encoded. •  Context (surrounding bit patterns) at each bit plane is examined. •  Key: explore patterns in binary bit-plane.

3 -1 7

4 -5 2

6 1 -2

0 0 1+

1+ 1- 0

1+ 0 0

1+ 0 1

0 0 1+

1 0 1-

1 1- 1

0 1 0

0 1+ 0

MSB LSB


JPEG Encoding and Decoding

www.jpeg.org

Encoding

Decoding


JPEG Coding Example

Original 64x64 image at 8 bits/pixel (8 bpp). Total 64x64x8 = 32768 bits. After encoding, 287 bits for DC coefficient, 1364 bits for AC coefficients. Compression ratio: 19.85:1 or 0.403bpp.

Block DCT coefficients Decoded image (19.85:1 ratio) 0.403 bpp. Note the blocking effect.

Histogram of AC coefficients


Wavelet Based Image Coding (generic)

Discrete Wavelet

Transform

Context-based Quantization

Entropy coding

2D discrete wavelet transform converts images into “sub-bands” Upper left is the DC coefficient Lower right are higher frequency sub-bands.


1D Discrete Wavelet Transform

π

y0 y1 y2 y3

π /2 π /4 π /8

HO: low pass digital filter, H1: high pass digital filter. Z-1: delay, ↓2: down-sample by 2

Recursive application of wavelet transform in spatial domain corresponds to dyadic partition of data in the frequency domain.


2D Separate DWT

•  1D DWT applied alternatively to vertical and horizontal direction line by line.

•  The LL band is recursively decomposed, first vertically, and then horizontally.

•  This is Mallat method. Other methods have also been proposed.

L H

LL

HL

LH

HH

Image in spatial domain

HL

LH

HH HL

LH

HH


Short summary of JPEG2000 •  If the image being compressed is in color it is decomposed

into three components •  Each component is partitioned into rectangular,

nonoverlapping regions called tiles. •  A tile is compressed in four main steps: 1.  Compute a wavelet transform (an integer and a floating point

are specified by the standard) 2.  Wavelet coefficients are quantized (if the user specifies a

target bit rate) 3.  The MQ coder is used to arithmetically encode the wavelet

coefficients


Short summary of JPEG2000 •  Each subband is divided into blocks (called code-blocks) that

are coded individually •  The bits resulting from coding several code-blocks become a

packet 4.  Packets as well as many markers construct the bitstream. •  Using markers decoder can decode certain code-blocks

before others, thereby displaying certain regions of the image before other regions.


Overall block diagram


Preprocessing •  Tile partition :

–  Each component of each tile encoded independently

•  DC level shifting : –  Unsigned sample values to Signed values

•  Colour transformation: –  To de-correlate the colour data


Image Size

XRsiz, YRsiz: x, y increment Image size in pixels: (width, height) = (x1-xo, y1-yo)


Tiles •  JPEG2000 allows the image to have a vertical offset at the top

and a horizontal offset on the left •  The origin of the tile grid can be located anywhere inside the

intersection area of the two offsets •  The main reason for having tiles is to enable the user to

decode parts of the image (region of interest) •  Different color components might have different sizes and

may use different tile sizes


Tiling

XTOsiz + XTsiz > XOsiz, YTOsiz + YTsiz > YOsiz


DC level shift and component transform

Purpose of component transform is to de-correlate among components. There are reversible and irreversible transforms.

Forward reversible component transform

Inverse reversible component transform


Reversible Color Transform

•  Make lossless color coding possible. •  All components must have identical sub-sampling

parameters and same depth


Wavelet Transform •  Filter-bank characteristics: symmetric, bi-orthogonal •  Two wavelet transforms are specified by the standard

–  9/7 floating point wavelet (irreversible) –  5/3 integer wavelet (reversible) (can give lossless compression)

•  2-D DWT = 1-D DWT on the columns followed by 1-D DWT on the rows

•  In JPEG 2000: 5 decompositions by default


Periodic Symmetric Signal Extension


Lossless 1D DWT

Reversible Integer DWT •  DWT coefficients are integers without any truncation error provided

image component pixel values are also integer-valued. •  Transform is exactly reversible. •  Non-causal filter.

Xext(), Yext(): symmetrically, cyclic extended signals.

Forward transform Reverse transform

I0-1 ≤ 2n < i1 -1; I0 ≤ 2n+1 < i1 ; I0-1 ≤ 2n+1 < i1 -1; I0 ≤ 2n < i1 ;


Lossy 1D DWT Daubechies’ (9,7) filter in the lifting format.

Step 1: i0-3 ≤ 2n < i1+3 Step 2: i0-2 ≤ 2n+1 < i1+2 Step 3: i0-3 ≤ 2n < i1+3 Step 4: i0-2 ≤ 2n+1 < i1+2 Step 5: i0-1 ≤ 2n < i1+1 Step 6: i0 ≤ 2n+1 < i1

α = -1.586 134 342, β = -0.052 980 118 γ = 0.882 911 075, δ = 0.443 506 852 K = 1.230 174 105

Step 1: i0-3 ≤ 2n+1 < i1+3 Step 2: i0-2 ≤ 2n < i1+2 Step 3: i0-1 ≤ 2n+1 < i1+1 Step 4: i0 ≤ 2n < i1 Step 5: i0 ≤ 2n+1 < i1 Step 6: i0 ≤ 2n < i1


Row-based Wavelet Transform

•  Problem with traditional wavelet transform: –  filtering to be performed in both vertical and horizontal directions.

While access in one direction is easy, access in the other will require whole image to be buffered

–  Difficult for implementation on PDA or other hand-held devices with limited amount of main memory.

•  Row-based wavelet transform –  consumes the minimum amount of resources, –  gives same results as traditional wavelet transform

•  Method –  Use a rolling window for each decomposition level to keep enough

number rows of image data in on-chip memory.


Quantization •  A uniform scalar quantization with dead-zone about the origin •  One quantization step size is allowed for each sub-band •  This operation is lossy unless for step size equal to 1

–  i.e. when reversible wavelet transform is used •  When step sizes are > 1:

–  The step sizes are chosen in conjunction with rate control –  They are transferred with the code stream

A zero output may be produced for larger values on the input, to avoid recording noise


Entropy coding •  Each subband is divided into rectangular blocks •  Precinct:

3 spatially consistent rectangle (from each subband at each resolution level)

•  Code Block: Each Precinct is partitioned into code blocks

•  Each Code Block is encoded independently


Entropy coding •  The quantizer indices corresponding to the quantized wavelet

coefficients in each subband are entropy encoded to create the compressed bit-stream.

•  The choice of the entropy coder in JPEG2000 is motivated by several factors. One is the requirement to create an embedded bit-stream, which is made possible by bitplane encoding of the quantizer indices

•  JPEG2000 uses a block coding paradigm in the wavelet domain as in the embedded block coding with optimized truncation (EBCOT) algorithm, where each subband is partitioned into small rectangular blocks, referred to as codeblocks, and each codeblock is independently encoded.


Entropy coding •  The independent encoding of the codeblocks has many advantages

including localized random access into the image, parallelization, improved cropping and rotation functionality, improved error resilience, efficient rate control, and maximum flexibility in arranging progression orders

•  The symbols that represent the quantized coefficients are encoded one bit at a time starting with the MSB and proceeding to the LSB.

•  During this progressive bitplane encoding, a quantized wavelet coefficient is called insignificant if the quantizer index is still zero.

•  Once the first nonzero bit is encoded, the coefficient becomes significant, and its sign is encoded.


Entropy coding •  Once a coefficient becomes significant, all subsequent bits are

referred to as refinement bits. •  Since the DWT packs most of the energy in the low frequency

subbands, the majority of the wavelet coefficients will have low amplitudes.

•  Many quantized indices will be insignificant in the earlier bitplanes, leading to a very low information content for those bitplanes.

•  JPEG2000 uses an efficient coding method for exploiting the redundancy of the bit planes known as context-based adaptive binary arithmetic coding.


Entropy Coding •  An adaptive binary arithmetic coder can be viewed as an

encoding device that accepts the binary symbols in a source sequence, along with their corresponding probability estimates, and produces a codestream with a length at most two bits greater than the combined ideal code lengths of the input symbols.

•  Adaptivity is provided by updating the probability estimate of a symbol based upon its present value and history.


Entropy Coding •  JPEG2000 coder divides the bitplane data into three groups and encodes

each group during a fractional bitplane pass. •  The passes are: significant propagation pass, refinement pass and clean up

pass •  Each coefficient in a block is assigned a binary state variable called its

significance state that is initialized to zero (insignificant) at the start of the encoding. The significance state changes from zero to one (significant) when the first nonzero magnitude bit is found


Entropy Coding •  In general, the probability distribution of each binary symbol

in a quantized wavelet coefficient is influenced by all the previously coded bits corresponding to that coefficient as well as the value of its immediate neighbours.

•  In JPEG2000, the probability of a binary symbol is estimated from a context formed from its current significance state as well as the significance states of its immediate eight neighbours as determined from the previous bitplane and the current bitplane, based on coded information up to that point.

•  In context-based arithmetic coding, separate probability estimates are maintained for each context, which is updated according to a finite-state machine every time a symbol is encoded in that context


Entropy Coding

•  The order in which the data in each pass are visited is a deterministic stripe-scan order with a height of four pixels. This stripe-based scan has been shown to facilitate software and hardware implementations.

•  The bit-stream can be truncated at the end of each coding pass


Entropy Coding •  Instead of encoding the entire bitplane in one coding pass, each bitplane is

encoded in three passes with the provision of truncating the bit-stream at the end of each coding pass.

•  A main advantage of this approach is near-optimal embedding, where the information that results in the largest reduction in distortion for the smallest increase in file size is encoded first.

•  A large number of potential truncation points facilitates an optimal rate control strategy where a target bit-rate is achieved by including those coding passes that minimize the total distortion.


Bit plane encoding orders •  During the first pass, referred to as the significance

propagation pass, the insignificant coefficients that have the highest probability of becoming significant, as determined by their immediate eight neighbours, are encoded.

•  In the second pass, known as the refinement pass, the significant coefficients are refined by their bit representation in the current bitplane.

•  Finally, during the cleanup pass, all the remaining coefficients in the bitplane are encoded as they have the lowest probability of becoming significant.


Neighboring states used to form context

•  Each coefficient in a code-block has an associated binary state variable called its significance state.

•  Significance states are initialized to 0 (coefficient is insignificant) and may become 1 (coefficient is significant) during the course of the coding of the code-block.

•  Four context are defined, one for each of the four coding operations:

–  significance propagation pass: •  significance coding, •  sign coding,

–  magnitude refinement pass •  magnitude refinement coding,

–  cleanup pass •  Cleanup coding.

•  The current context obtained during context coding is provided to the arithmetic MQ coder.


Significance propagation pass •  During this pass, the insignificant coefficients that have the highest

probability of becoming significant in the current bitplane are encoded. The data is scanned in the stripe order, and every sample that has at least one significant immediate neighbor, based on coded information up to that point, is encoded. All other coefficients are skipped.

•  The context is delivered to the arithmetic decoder (along with the significance bit) and the coded coefficient bit is returned.

•  If the value of this bit is 1 then the significance state is set to 1 and the immediate next bit to be coded is the sign bit for the coefficient. Otherwise, the significance state remains 0.


Significance propagation pass •  As soon as a coefficient is coded, its significance state is updated so that it

can affect the inclusion of subsequent coefficients in that coding pass. •  The significance state of the coefficient is arithmetic coded using contexts

that are based on the significance states of its immediate neighbours. In general, the significance states of the eight neighbours can create 256 different contexts, however, many of these contexts have similar probability estimates and can be merged together.

•  A context reduction mapping reduces the total number of contexts to only nine to improve the efficiency of the MQ-coder probability estimation for each context.


Significance propagation pass •  Since the codeblocks are encoded independently, if a sample

is located at the codeblock boundary, only its immediate neighbors that belong to the current codeblock are considered and the significance state of the missing neighbors are assumed to be zero.

•  If a coefficient is found to be significant, its sign needs to be encoded.

•  The sign value is also arithmetic encoded using five contexts that are determined from the significance and the sign of the coefficient’s four horizontal and vertical neighbors.


Context of Significance and Cleanup Passes


Sign Bit Coding •  Two phases:

–  Summarize contributions of vertical and horizontal neighbors

–  Reduces these contributions into 1 or 5 context labels

•  The context labels are sent to MQ arithmetic coder.

Signbit = AC(contextlabel) ⊕ XORbit

–  Signbit: sign bit of the current coefficient

–  AC(contextlabel) is the value returned from arithmetic coder given the context label and the bit stream.


Refinement pass •  During this pass, the magnitude bit of a coefficient that has

already become significant in a previous bitplane is arithmetic encoded using three contexts.

•  The refinement bits have an even distribution unless the coefficient has just become significant in the previous bitplane (i.e., the magnitude bit to be encoded is the first refinement bit).

•  This condition is first tested and if it is satisfied, the magnitude bit is encoded using two coding contexts based on the significance of the eight immediate neighbors. Otherwise, it is coded with a single context regardless of the neighboring values.


Magnitude Refinement

•  The magnitude refinement pass includes the bits from coefficients that are already significant (except those that have just become significant in the immediately proceeding significance propagation pass).

•  The context used is determined by the summation of the significance state of the horizontal, vertical, and diagonal neighbors. These are the states as currently known to the decoder, not the states used before the significance decoding pass.


Cleanup Pass •  All the remaining coefficients in the codeblock are encoded during the

cleanup pass. •  A special mode, referred to as the run mode, is used to aggregate the

coefficients that have the highest probability of remaining insignificant. •  A run mode is entered if all the four samples in a vertical column of the

stripe have insignificant neighbors. •  In the run mode, a binary symbol is arithmetic encoded in a single context

to specify whether all the four samples in the vertical column remain insignificant.

•  An encoded value of zero implies insignificance for all four samples, while an encoded value of one implies that at least one of the four samples becomes significant in the current bitplane.

•  An encoded value of one is followed by two additional arithmetic encoded bits that specify the location of the first nonzero coefficient in the vertical column.


Cleanup Pass •  After the position of the first nonzero coefficient in the run is

specified, the remaining samples in the vertical column are encoded in the same manner as in the significance propagation pass and use the same nine coding contexts.

•  Similarly, if at least one of the four coefficients in the vertical column has a significant neighbor, the run mode is disabled and all the coefficients in that column are coded according to the procedure employed for the significance propagation pass.

•  For each codeblock, the number of MSB planes that are entirely zero is signaled in the bit-stream.

•  Since the significance state of all the coefficients in the first non-zero MSB is zero, only the cleanup pass is applied to the first non-zero bitplane.


Cleanup Pass

•  The first pass and only coding pass for the first significant bit-plane. •  The third and the last pass of all the remaining bit-planes. •  Use both neighbor context as in significant propagation pass and run-length

coding.


Tier 1 and Tier 2 Coding •  The arithmetic coding of the bitplane data is referred to as

tier-1 (T1) coding. •  Since the codeblocks are independently encoded, the

compressed data corresponding to the various coding passes can be arranged in different configurations to create a rich set of progression orders to serve different applications.

•  The only restriction is that the sub-bitplane coding passes for a given codeblock must appear in a causal order starting from the most significant bitplane.

•  The compressed sub-bitplane coding passes can be aggregated into larger units named packets. This process of packetization along with its supporting syntax, is often referred to as tier-2 (T2) coding.

JPEG2000: An Introduction Part Ishirani/multi08/jpeg2000part1.pdf · • JPEG coding standard (1988): DCT (discrete cosine transform) based transform coding to compress bit-map images.

Documents