Data Compression - link.springer.com978-3-642-86092-8/1.pdf · video concepts. It continues with a general discussion of video compression, and it concludes with a description of

Data Compression

Second Edition

Springer New York Berlin Heidelberg Barcelona Hong Kong London Milan Paris Singapore Tokyo

David Salomon

Data Compression The Complete Reference

Second Edition

With 301 Figures, 5 in Full Color

, Springer

David Salomon Department of Computer Science California State University, Northridge Northridge, CA 91330-8281 USA [email protected]

Library of Congress Cataloging-in-Publication Data Salomon, D. (David), 1938-

Data compression: the complete reference/David Salomon.-2nd ed. p. cm.

Includes bibliographical references and index. ISBN 0-387-95045-1 (alk. paper)

1. Data compression (Computer science) 1. Title. QA76.9.D33 S25 2000 005. 74'6-dc21

Printed on acid-free paper.

© 2000,1998 Springer-Verlag New York, Inc. Sof'tcover reprint of the hardcover 2nd edition 2000

00-040048

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Production managed by Francine McNeill; manufacturing supervised by Jeffrey Taub. Photocomposed copy prepared from the author's TeX files. Printed and bound by Maple-Vail Book Manufacturing Group, York, PA.

9 8 7 6 5 4 321

ISBN-13: 978-3-540-78086-1 DOl: 10.1007/978-3-642-86092-8

e-ISBN-13: 978-3-642-86092-8

Springer-Verlag New York Berlin Heidelberg A member of BertelsmannSpringer Science+Business Media GmbH

To my family, without whose help, patience, and support this book would not have been written.

Preface to the Second Edition

This second edition has come about for three reasons. The first one is the many favorable readers' comments, of which the following is an example:

I ju t fini hed reading your book on data compr ion. Such joy. And as it contains many algorithms in a volume only orne 20 mm thick, the book itself erves as a fine example of data compr ion!

- Fred Veldmeijer 1998

The second reason is the errors found by the author and by readers in the first edition. They are listed in the book's web site (see below), and they have been corrected in the second edition.

The third reason is the title of the book (originally chosen by the publisher). This title had to be justified by making the book a complete reference. As a result, many compression methods and much background material have been added to the book in this edition. The most important additions and changes are the following:

• Three new chapters have been added. The first is Chapter 5, on the relatively young (and relatively unknown) topic of wavelets and their applications to image and audio compression. The chapter opens with an intuitive explanation of wavelets, using the continuous wavelet transform (CWT). It continues with a detailed example that shows how the Raar transform is used to compress images. This is followed by a general discussion of filter banks and the discrete wavelet transform (DWT), and a listing of the wavelet coefficients of many common wavelet filters. The chapter concludes with a description of important compression methods that either use wavelets or are based on wavelets. Included among them are the Laplacian pyramid, set partitioning in hierarchical trees (SPIRT), embedded coding using zerotrees (EZW), the WSQ method for the compression of fingerprints, and JPEG 2000, a new, promising method for the compression of still images (Section 5.19) .

viii Preface to the Second Edition

• The second new chapter, Chapter 6, discusses video compression. The chapter opens with a general description of CRT operation and basic analog and digital video concepts. It continues with a general discussion of video compression, and it concludes with a description of MPEG-1 and H.261.

• Audio compression is the topic of the third new chapter, Chapter 7. The first topic in this chapter is the properties of the human audible system and how they can be exploited to achieve lossy audio compression. A discussion of a few simple audio compression methods follows, and the chapter concludes with a description of the three audio layers of MPEG-1, including the very popular mp3 format.

Other new material consists of the following:

• Conditional image RLE (Section 1.4.2).

• Scalar quantization (Section 1.6).

• The QM coder used in JPEG, JPEG 2000, and JBIG is now included in Sec­tion 2.16.

• Context-tree weighting is discussed in Section 2.19. Its extension to lossless image compression is the topic of Section 4.24.

• Section 3.4 discusses a sliding buffer method called repetition times.

• The troublesome issue of patents is now also included (Section 3.25).

• The relatively unknown Gray codes are discussed in Section 4.2.1, in connection with image compression.

• Section 4.3 discusses intuitive methods for image compression, such as subsam-pling and vector quantization.

• The important concept of image transforms is discussed in Section 4.4. The discrete cosine transform (DCT) is described in detail. The Karhunen-Loeve trans­form, the Walsh-Hadamard transform, and the Haar transform are introduced. Sec­tion 4.4.5 is a short digression, discussing the discrete sine transform, a poor, un­known cousin of the DCT.

• JPEG-LS, a new international standard for lossless and near-Iossless image compression, is the topic of the new Section 4.7.

• JBIG2, another new international standard, this time for the compression of bi-Ievel images, is now found in Section 4.10.

• Section 4.11 discusses EIDAC, a method for compressing simple images. Its main innovation is the use of two-part contexts. The intra context of a pixel P consists of several of its near neighbors in its bitplane. The inter context of P is made up of pixels that tend to be correlated with P even though they are located in different bitplanes.

• There is a new Section 4.12 on vector quantization followed by sections on adaptive vector quantization and on block truncation coding (BTC).

Preface to the Second Edition ix

• Block matching is an adaptation of LZ77 (sliding window) for image compres-sion. It can be found in Section 4.14.

• Differential pulse code modulation (DPCM) is now included in the new Sec-tion 4.23.

• An interesting method for the compression of discrete-tone images is block decomposition (Section 4.25).

• Section 4.26 discusses binary tree predictive coding (BTPC).

• Prefix image compression is related to quadtrees. It is the topic of Section 4.27.

• Another image compression method related to quadtrees is quadrisection. It is discussed, together with its relatives bisection and octasection, in Section 4.28.

• The section on WFA (Section 4.31) was wrong in the first edition and has been completely rewritten with much help from Karel Culik and Raghavendra Udupa.

• Cell encoding is included in Section 4.33.

• DjVu is an unusual method, intended for the compression of scanned docu­ments. It was developed at Bell Labs (Lucent Technologies) and is described in Section 5.17.

• The new JPEG 2000 standard for still image compression is discussed in the new Section 5.19.

• Section 8.4 is a description of the sort-based context similarity method. This method uses the context of a symbol in a way reminiscent of ACB. It also assigns ranks to symbols, and this feature relates it to the Burrows-Wheeler method and also to symbol ranking.

• Prefix compression of sparse strings has been added to Section 8.5.

• FHM is an unconventional method for the compression of curves. It uses Fi­bonacci numbers, Huffman coding, and Markov chains, and it is the topic of Sec­tion 8.9.

• Sequitur, Section 8.10, is a method especially suited for the compression of semi-structured text. It is based on context-free grammars.

• Section 8.11 is a detailed description of edgebreaker, a highly original method for compressing the connectivity information of a triangle mesh. This method and its various extensions may become the standard for compressing polygonal surfaces, one of the most common surface types used in computer graphics. Edgebreaker is an example of a geometric compression method.

• All the appendices have been deleted because of space considerations. They are freely available, in PDF format, at the book's web site. The appendices are (1) the ASCII code (including control characters); (2) space-filling curves; (3) data struc­tures (including hashing); (4) error-correcting codes; (5) finite-state automata (this topic is needed for several compression methods, such as WFA, IFS, and dynamic Markov coding); (6) elements of probability; and (7) interpolating polynomials.

x Preface to the Second Edition

• The answers to the exercises have also been deleted and are available at the book's web site.

Currently, the book's web site is part of the author's web site, which is located at http://www.ecs.csun.edurdxs/.DomainnameBooksByDavidSalomon.com has been reserved and will always point to any future location of the web site. The author's email address is david. salomon@csun. edu, but it is planned that any email sent to (anyname)@BooksByDavidSalomon. com will be forwarded to the author.

Readers willing to put up with eight seconds of advertisement can be redirected to the book's web site from http://welcome . to/data. compression. Email sent to data. compression@welcome. to will also be redirected.

Those interested in data compression in general should consult the short sec­tion titled "Joining the Data Compression Community," at the end of the book, as well as the two URLs http://www.internz.com/compression-pointers . html and http://www.hn.is.uec.ac.jp/-arimura/compression_links.html.

Northridge, California David Salomon

Preface to the First Edition

Historically, data compression was not one of the first fields of computer science. It seems that workers in the field needed the first 20 to 25 years to develop enough data before they felt the need for compression. Today, when the computer field is about 50 years old, data compression is a large and active field, as well as big business. Perhaps the best proof of this is the popularity of the Data Compression Conference (DCC, see end of book).

Principles, techniques, and algorithms for compressing different types of data are being developed at a fast pace by many people and are based on concepts borrowed from disciplines as varied as statistics, finite-state automata, space-filling curves, and Fourier and other transforms. This trend has naturally led to the publication of many books on the topic, which poses the question, Why another book on data compression?

The obvious answer is, Because the field is big and getting bigger all the time, thereby "creating" more potential readers and rendering existing texts obsolete in just a few years.

The original reason for writing this book was to provide a clear presentation of both the principles of data compression and all the important methods currently in use, a presentation geared toward the nonspecialist. It is the author's intention to have descriptions and discussions that can be understood by anyone with some background in the use and operation of computers. As a result, the use of math­ematics is kept to a minimum and the material is presented with many examples, diagrams, and exercises. Instead of trying to be rigorous and prove every claim, the text many times says "it can be shown that ... " or "it can be proved that .... "

The exercises are an especially important feature of the book. They comple­ment the material and should be worked out by anyone who is interested in a full understanding of data compression and the methods described here. Almost all the answers are provided (at the book's web page), but the reader should obviously try to work out each exercise before peeking at the answer.

xii Preface to the First Edition

Acknowledgments

I would like especially to thank Nelson Beebe, who went meticulously over the entire text of the first edition and made numerous corrections and suggestions. Many thanks also go to Christopher M. Brislawn, who reviewed Section 5.18 and gave us permission to use Figure 5.64; to Karel Culik and Raghavendra Udupa, for their substantial help with weighted finite automata (WFA); to Jeffrey Gilbert, who went over Section 4.25 (block decomposition); to John A. Robinson, who reviewed Section 4.26 (binary tree predictive coding); to 0yvind Str0mme, who reviewed Section 5.10; to Frans Willems and Tjalling J. Tjalkins, who reviewed Section 2.19 (context-tree weighting); and to Hidetoshi Yokoo, for his help with Sections 3.15 and 8.4.

The author would also like to thank Paul Amer, Guy Blelloch, Mark Doyle, Hans Hagen, Emilio Millan, Haruhiko Okumura, and Vijayakumaran Saravanan, for their help with errors.

We seem to have a natural fascination with shrinking and expanding objects. Since our practical ability in this respect is very limited, we like to read stories where people and objects dramatically change their natural size. Examples are Gulliver's Travels by Jonathan Swift (1726), Alice in Wonderland by Lewis Carroll (1865), and Fantastic Voyage by Isaac Asimov (1966).

Fantastic Voyage started as a screenplay written by the famous writer Isaac Asimov. While the movie was being produced (it was released in 1966), Asimov rewrote it as a novel, correcting in the process some of the most glaring flaws in the screenplay. The plot concerns a group of medical scientists placed in a submarine and shrunk to microscopic dimensions. They are then injected into the body of a patient in an attempt to remove a blood clot from his brain by means of a laser beam. The point is that the patient, Dr. Benes, is the scientist who improved the miniaturization process and made it practical in the first place.

Because of the success of both the movie and the book, Asimov later wrote Fantastic Voyage II: Destination Brain, but the latter novel proved a flop.

Northridge, California

But before we continue here is a question that you might have already asked: "OK, but why should I be interested in data compression?" Very simple: "DATA COMPRESSION SAVES YOU MONEY!"

More interested now? We think you should be. Let us give you an example of data compression application that you see every day. Exchanging faxes every day ...

From http://wvw.rasip.etf.hr/research/compress/index.html

David Salomon

Contents

Preface to the Second Edition vii

Preface to the First Edition xi

Introduction 1

1. Basic Techniques 13 1.1 Intuitive Compression 13 1.2 Run Length Encoding 18 1.3 RLE Text Compression 18 1.4 RLE Image Compression 23 1.5 Move-to-Front Coding 32 1.6 Scalar Quantization 35

2. Statistical Methods 39 2.1 Information Theory Concepts 40 2.2 Variable-Size Codes 46 2.3 Prefix Codes 47 2.4 The Golomb Code 53 2.5 The Kraft-MacMillan Inequality 54 2.6 Shannon-Fano Coding 55 2.7 The Counting Argument 57 2.8 Huffman Coding 62 2.9 Adaptive Huffman Coding 76

2.10 MNP5 83 2.11 MNP7 88 2.12 Reliability 89 2.13 Facsimile Compression 91 2.14 Arithmetic Coding 101 2.15 Adaptive Arithmetic Coding 113 2.16 The QM Coder 115 2.17 Text Compression 126 2.18 PPM 127 2.19 Context-Tree Weighting 143

XIV Contents

3. Dictionary Methods 151

3.1 String Compression 153 3.2 LZ77 (Sliding Window) 154 3.3 LZSS 158 3.4 Repetition Times 161 3.5 QIC-122 163 3.6 LZ78 164 3.7 LZFG 168 3.8 LZRWI 171 3.9 LZRW4 175

3.10 LZW 175 3.11 LZMW 186 3.12 LZAP 189 3.13 LZY 189 3.14 LZP 192 3.15 Repetition Finder 199 3.16 UNIX Compression 202 3.17 GIF Images 203 3.18 The V.42bis Protocol 204 3.19 Zip and Gzip 205 3.20 ARC and PKZip 206 3.21 ARJ and LHArc 211 3.22 EXE Compressors 212 3.23 CRC 212 3.24 Summary 215 3.25 Data Compression Patents 215 3.26 A Unification 218

4. Image Compression 221

4.1 Introduction 223 4.2 Approaches to Image Compression 229 4.3 Intuitive Methods 242 4.4 Image Transforms 243 4.5 Test Images 269 4.6 JPEG 273 4.7 JPEG-LS 296 4.8 Progressive Image Compression 302 4.9 JBIG 311

4.10 JBIG2 320 4.11 Simple Images: EIDAC 331 4.12 Vector Quantization 333 4.13 Adaptive Vector Quantization 340 4.14 Block Matching 345 4.15 Block Truncation Coding 349 4.16 Context-Based Methods 355 4.17 FELICS 358 4.18 Progressive FELICS 361

Contents xv

4.19 MLP 365 4.20 PPPM 373 4.21 CALIC 375 4.22 Differential Lossless Compression 378 4.23 DPCM 380 4.24 Context-Tree Weighting 385 4.25 Block Decomposition 386 4.26 Binary Tree Predictive Coding 391 4.27 Quadtrees 397 4.28 Quadrisection 410 4.29 Space-Filling Curves 417 4.30 Hilbert Scan and VQ 418 4.31 Finite Automata Methods 421 4.32 Iterated Function Systems 438 4.33 Cell Encoding 455

5. Wavelet Methods 457

5.1 Fourier Transform 457 5.2 The Frequency Domain 458 5.3 The Uncertainty Principle 462 5.4 Fourier Image Compression 465 5.5 The CWT and Its Inverse 467 5.6 The Haar Transform 474 5.7 Filter Banks 492 5.8 The DWT 502 5.9 Multiresolution Decomposition 515

5.10 Various Image Decompositions 516 5.11 The Lifting Scheme 523 5.12 The IWT 534 5.13 The Laplacian Pyramid 537 5.14 SPIHT 540 5.15 CREW 553 5.16 EZW 553 5.17 DjVu 558 5.18 WSQ, Fingerprint Compression 561 5.19 JPEG 2000 567

6. Video Compression 581

6.1 Analog Video 581 6.2 Composite and Components Video 587 6.3 Digital Video 588 6.4 Video Compression 593 6.5 MPEG 605 6.6 H.261 627

xvi

7.

8.

Contents

Audio Compression 631

7.1 Sound 632 7.2 Digital Audio 636 7.3 The Human Auditory System 638 7.4 p,-Law and A-Law Companding 644 7.5 ADPCM Audio Compression 650 7.6 MPEG-1 Audio Layers 652

Other Methods 681

8.1 The Burrows-Wheeler Method 682 8.2 Symbol Ranking 687 8.3 ACB 691 8.4 Sort-Based Context Similarity 698 8.5 Sparse Strings 704 8.6 Word-Based Text Compression 715 8.7 Textual Image Compression 720 8.8 Dynamic Markov Coding 726 8.9 FHM Curve Compression 733

8.10 Sequitur 737 8.11 Triangle Mesh Compression: Edgebreaker 743

Bibliography 755

Glossary 773

Joining the Data Compression Community 801

Index 803

Colophon 823

Each memorable verse of a true poet has two or three times the written content

-Alfred de M lisset