Digital Image Processing - PKU...Digital Image Processing Ming Jiang Digital image processing: What, Why and How What Are the Difﬁculties Poor understanding of human vision system

Digital ImageProcessing

Ming Jiang

Digital Image Processing

Ming Jiang

School of Mathematical SciencesPeking University

[email protected]

May 18, 2010


Ming Jiang

Digital imageprocessing: What,Why and How

What Are theDifficultiesPoor understanding ofhuman vision system

Internal representation isnot directly understandable

Why is computer visiondifficult?

Imagerepresentation andimage analysistasks

Course Overview

References

Outline

Digital image processing: What, Why and How

What Are the DifficultiesPoor understanding of human vision systemInternal representation is not directly understandableWhy is computer vision difficult?

Image representation and image analysis tasks

Course Overview


Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Vision

I Image is better than any other information form forour human being to perceive. Vision allows humansto perceive and understand the world surrounding us.

I Human are primarily visual creatures. Not all animalsdepend on their eyes, as we do, for 99% or 90% ofthe information received about the world[Russ, 1995, Zhao and Zhong, 1982].


Ming Jiang






Course Overview

References

Vision

I Image is better than any other information form forour human being to perceive. Vision allows humansto perceive and understand the world surrounding us.

I Human are primarily visual creatures. Not all animalsdepend on their eyes, as we do, for 99% or 90% ofthe information received about the world[Russ, 1995, Zhao and Zhong, 1982].


Ming Jiang






Course Overview

References

Computer Vision

I Computer vision aims to duplicate the effect ofhuman vision by electronically perceiving andunderstanding an image.

I Books other than this one would dwell at length onthis sentence and the meaning of the word duplicate

I whether computer vision is simulating or mimickinghuman systems is a philosophical territory,

I and one very fertile territory, too.


Ming Jiang






Course Overview

References

Computer Vision






Ming Jiang






Course Overview

References

Computer Vision






Ming Jiang






Course Overview

References

Computer Vision






Ming Jiang






Course Overview

References

3D vs 2D

I Giving computers the ability to see is not an easytask — we live in a three-dimensional (3D) world.

I When computers try to analyze objects in 3D space,the visual sensors available (e.g., TV cameras)usually give two-dimensional (2D) images.

I This projection from 3D to a lower number ofdimensions incurs an enormous loss of information.

I Sometimes, equipment will deliver images that are3D but this may be of questionable value:

I analyzing such datasets is clearly more complicatedthan 2D;

I sometimes the ’three-dimensionality’ is less thanintuitive to us;

I terahertz scans are an example of this.


Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

3D vs 2D









Ming Jiang






Course Overview

References

Video AnalysisI Dynamic scenes such as those to which we are

accustomed, with moving objects or a movingcamera, are increasingly common and representanother way of making computer vision morecomplicated.

Figure: A frame from a video of a typical farmyard scene:the cow is one of a number walking naturally from right toleft.


Ming Jiang






Course Overview

References

Video Analysis: easy for human

I There are many reasons why we might wish to studyscenes such as this, which are attractively simple tous

I the beast is moving slowly;I it is clearly black and white;I its movement is rhythmic.

I However, automated analysis is very fraught.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References

Video Analysis: difficult for computer

I The animal’s boundary is often very difficult todistinguish clearly from the background;

I the motion of the legs is self occluding;I (subtly) the concept of cow-shaped is not something

easily encoded.


Ming Jiang






Course Overview

References




easily encoded.


Ming Jiang






Course Overview

References




easily encoded.


Ming Jiang






Course Overview

References

Video Analysis: proceduresI The application from

which this picture wastaken made use ofmany of the algorithmspresented in this book:

I starting at a low levelmoving features wereidentified andgrouped;

I a training phasetaught the systemwhat a cow mightlook like in variousposes (see the figureon the right), fromwhich a model of amoving cow could beestimated.

Various models for a cowsilhouette: a straight-lineboundary approximation hasbeen learned from training dataand is able to adapt to differentanimals and different forms ofocclusion.


Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References

Video Analysis: operations

I These models couldthen be fitted to new(unseen) videosequences.

I At this stage anomalousbehavior such aslameness could bedetected by the modelfailing to fit properly, orwell.

I Thus we see asequence of operations

I image capture,I early processing,I segmentation,I model fitting,I motion prediction,I qualitative and/or

quantitativeconclusion,

I that is characteristic ofimage understandingand computer visionproblems.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

Video Analysis: models and cow detection

Figure: Three frames from a cow sequence: notice the modelcan cope with partial occlusion as the animal enters the scene,and the different poses exhibited.


Ming Jiang






Course Overview

References

Video Analysis: disscusions

I Each of these phases (which may not occursequentially!) may be addressed by a number ofalgorithms which we shall cover in due course.

I The application was serious; there is a growing needin modern agriculture for automatic monitoring ofanimal health, for example to spot lameness.

I A limping cow is trivial for a human to identify, but it isvery challenging to do this automatically.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References


I This example is relatively simple to explain, butserves to illustrate that many computer visiontechniques use the results and methods of

I mathematics,I pattern recognition,I artificial intelligence (AI),I psycho-physiology,I computer science,I electronics,I and other scientific disciplines.


Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References





Ming Jiang






Course Overview

References

Difficuties???

I Consider a single gray-scale (monochromatic)image, write down a few reasons why you feelautomatic inspection and analysis of it may bedifficult.


Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Human VisionI How the human perceive process and store the

visual information?


Ming Jiang






Course Overview

References

How many legs does this elephant have?

I From the Home of Vision Illusion:I http://www.123opticalillusions.com/pages/opticalillusions15.php


Ming Jiang






Course Overview

References

How many legs does this elephant have?

I From the Home of Vision Illusion:I http://www.123opticalillusions.com/pages/opticalillusions15.php


Ming Jiang






Course Overview

References

What is it?

I Explanation and further information can be found atI http://www.psychologie.tu-

dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/perceptual_ambiguity.html


Ming Jiang






Course Overview

References

What is it?

I Explanation and further information can be found atI http://www.psychologie.tu-

dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/perceptual_ambiguity.html


Ming Jiang






Course Overview

References

Old woman and Young woman: discussions

I Do you see an old woman or a young woman in thisillustration?

I They are both present, but you will not be able to seeboth of them simultaneously.

I Once you perceive both figures, see if you can getthem to fluctuate back and forth between the twointerpretations.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References


I This type of reversible figure concerns themeaningful content of what is interpreted by yourbrain from the same static image.

I Your perception of each figure tends to remain stableuntil you attend to different regions or contours.

I Certain regions and contours tend to favor oneperception, others the alternative.

I Your visual system tends to group like or relatedregions together.

I It does not present you with some odd mixture of thetwo alternatives.

I Attending to different regions or contours does tendto initiate a change of perception.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

Human Vision

I We do not have a clear understanding how thehuman perceive, process and store the visualinformation.

I We do not even know how the human measuresinternally the image visual quality and discrimination.


Ming Jiang






Course Overview

References

Human Vision

I We do not have a clear understanding how thehuman perceive, process and store the visualinformation.

I We do not even know how the human measuresinternally the image visual quality and discrimination.


Ming Jiang






Course Overview

References

Perception ≡ DescriptionI If this image is looked at with a steady eye, it will still

change, though less often.I Researchers have stabilized the image directly onto

the retina to eliminate any effects that may arise fromeye movements.

I Even under these conditions, a perceptual reversalmay occur.

I This indicates that higher cortical processing occursthat strives to make meaning out of a stable imagepresented to the retina.

I This illustrates once more that vision is an activeprocess that attempts to make sense of incominginformation.

I As the late David Marr said, “Perception is theconstruction of a description.”


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

History of this illustrationI For many years the creator of

this famous figure wasthought to be Britishcartoonist W. E. Hill, whopublished it in 1915. Hillalmost certainly adapted thefigure from an originalconcept that was popularthroughout the world ontrading and puzzle cards.

I This anonymous datedGerman postcard (shown atthe top of the page) from1888 depicts the image in itsearliest known form.

The 1890 example on the left

shows quite clearly its

association as “My Wife and

Mother-in-Law.” Both of these

examples predate the Punch

cartoon that was previously

thought to serve as the figure’s

inspiration.


Ming Jiang






Course Overview

References











inspiration.


Ming Jiang






Course Overview

References











inspiration.


Ming Jiang






Course Overview

References











inspiration.


Ming Jiang






Course Overview

References











inspiration.


Ming Jiang






Course Overview

References

History of this illustration

I The figure was later altered and adapted by others,including the two psychologists, R. W. Leeper and E.G. Boring who described the figure and made itfamous within psychological circles in 1930. It hasoften been referred to as the “Boring figure.”

I Versions of the figure proved to be popular and theimage was frequently reprinted; however, perceptualbiases started to occur in the image, unbeknownst tothe plagiarizing artists and psychologists who werereprinting the images.

I Variations have appeared in the literature thatunintentionally are biased to favor one interpretationor another, which defeats its original purpose as atruly ambiguous figure.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References


I In the three versions shown above, can you tell whichone is biased toward the young girl, the old woman?


Ming Jiang






Course Overview

References


I In 1961, J, Botwinick redesigned this figure onceagain, and entitled it, "Husband and Father-in-Law."


Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Images as 2D functions

I Images are usually represented as a twodimensional function.

I Digitized images are usually represented by twodimensional array.

I However, those representations are not suitable formachine understanding, while the computer is ableto process those representations.

I General knowledge, domain-specific knowledge, andinformation extracted from the image will be essentialin attempting to understanding those arrays ofnumbers.


Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References

Experiment with images as 2D functions

I Read and display a image file as a two dimensionalfunction.

I The example matlab script file is here matlab displayexample.


Ming Jiang






Course Overview

References

Experiment with images as 2D functions

I Read and display a image file as a two dimensionalfunction.

I The example matlab script file is here matlab displayexample.


Ming Jiang






Course Overview

References

Images as 2D functions: discussions

I Both presentations contain exactly the sameinformation.

I But for a human observer it is very difficult to find acorrespondence between both.

I The point is that a lot of a priori knowledge is used byhumans to interpret the images;

I the machine only begins with an array of numbersand so will be attempting to us more likely the firstdisplay than the second display.


Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References

Outline




Course Overview


Ming Jiang






Course Overview

References

Why is computer vision difficult?

I This philosophical question provides some insightinto the rather complex landscape of computervision.

I It can be answered in many ways: we offer six.I Here, we mention the reasons only briefly — most of

them will be discussed in more detail later in thebook.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References

I. Loss of information

I Loss of information in projections from 3D to 2D is aphenomenon which occurs in typical image capturedevices such as a camera or an eye.

I Their geometric properties have been approximatedby a pinhole model for centuries (a box with a smallhole in it, called in Latin a camera obscura [darkroom]).


Ming Jiang






Course Overview

References

I. Loss of information

I Loss of information in projections from 3D to 2D is aphenomenon which occurs in typical image capturedevices such as a camera or an eye.

I Their geometric properties have been approximatedby a pinhole model for centuries (a box with a smallhole in it, called in Latin a camera obscura [darkroom]).


Ming Jiang






Course Overview

References

Pinhole camera

Figure: The pinhole model of imaging geometry does notdistinguish size of objects.

I This physical model corresponds to a mathematicalmodel of perspective projection.

I The projective transformation maps points along raysbut does not preserve angles and collinearity.


Ming Jiang






Course Overview

References

Pinhole camera

Figure: The pinhole model of imaging geometry does notdistinguish size of objects.

I This physical model corresponds to a mathematicalmodel of perspective projection.

I The projective transformation maps points along raysbut does not preserve angles and collinearity.


Ming Jiang






Course Overview

References

II. Interpretation of imagesI Interpretation of image(s) constitutes the principal

tool of computer vision to approach problems whichhumans solve unwittingly.

I When a human tries to understand an image thenprevious knowledge and experience is brought to thecurrent observation.

I Human ability to reason allows representation oflong-gathered knowledge, and its use to solve newproblems.

I Artificial intelligence has invested several decades inattempts to endow computers with the capability tounderstand observations;

I while progress has been tremendous, the practicalability of a machine to understand observationsremains very limited.

I Attempting to solve related multidisciplinary scientificproblems under the name cognitive systems is seenas a key to developing intelligent machines.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

Interpretation of images

I From the mathematical logic and/or linguistics pointof view, interpretation of images can be seen as amapping interpretation:

image data→ model (1)

I The (logical) model means some specific world inwhich the observed objects make sense.

I ExamplesI nuclei of cells in a biological sample,I rivers in a satellite image,I or parts in an industrial process being checked for

quality.I There may be several interpretations of the same

image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References







image(s).


Ming Jiang






Course Overview

References

Semantics of images

I Introducing interpretation to computer vision allowsus to use concepts from mathematical logic,linguistics as syntax (rules describing correctlyformed expression), and semantics (study ofmeaning).

I Considering observations (images) as an instance offormal expressions, semantics studies relationsbetween expressions and their meanings.

I The interpretation of image(s) in computer vision canbe understood as an instance of semantics.

I Practically, if the image understanding algorithmsknow into which particular domain (model in logicalterminology) the observed world is constrained, thenautomatic analysis can be used for complicatedproblems.


Ming Jiang






Course Overview

References

Semantics of images






Ming Jiang






Course Overview

References

Semantics of images






Ming Jiang






Course Overview

References

Semantics of images






Ming Jiang






Course Overview

References

III. Noise

I Noise is inherently present in each measurement inthe real world.

I Its existence calls for mathematical tools which areable to cope with uncertainty; an example isprobability theory.

I Of course, more complex tools make the imageanalysis much more complicated compared tostandard (deterministic) methods.


Ming Jiang






Course Overview

References

III. Noise





Ming Jiang






Course Overview

References

III. Noise





Ming Jiang






Course Overview

References

IV. Too much data

I Images and video sequences are huge.I An A4 sheet of paper scanned monochromatically at

300 dots per inch (dpi) at 8 bits per pixel correspondsto 8.5 MB.

I Non-interlaced RGB 24 bit color video 512× 768pixels, 25 frames per second, makes a data streamof 225 Mb per second.

I If the processing we devise is not very simple, then itis hard to achieve real-time performance; i.e., toprocess 25 or 30 images per second.


Ming Jiang






Course Overview

References

IV. Too much data






Ming Jiang






Course Overview

References

IV. Too much data






Ming Jiang






Course Overview

References

IV. Too much data






Ming Jiang






Course Overview

References

V. Complexity in mage formation

I Brightness measured in the image is given bycomplicated image formation physics.

I The radiance (brightness, image intensity)depends on the irradiance (light source type,intensity and position), the observer’s position, thesurface local geometry, and the surface reflectanceproperties.

I The inverse tasks are ill-posed — for example, toreconstruct local surface orientation from intensityvariations.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References

VI. Local window vs. for global view

I Commonly, image analysis algorithms analyze aparticular storage bin in an operational memory (e.g.,a pixel in the image) and its local neighborhood;

I the computer sees the image through a keyhole.

I Seeing the world through a keyhole makes it verydifficult to understand more global context.

I It is often very difficult to interpret an image if it isseen only locally or if only a few local keyholes areavailable.


Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References

Local parts of an image

Figure: Illustration of the world seen through several keyholesproviding only a very local context.


Ming Jiang






Course Overview

References

The global view of the image

Figure: How context is taken into account is an important facetof image analysis.


Ming Jiang






Course Overview

References

Two Approaches

I There are philosophically two approaches: bionicsand engineering (that is project attempt coordinated),approaches.

I The bionics approach has not been so successful,since we do have a through understanding about thebiological vision system.


Ming Jiang






Course Overview

References

Two Approaches

I There are philosophically two approaches: bionicsand engineering (that is project attempt coordinated),approaches.

I The bionics approach has not been so successful,since we do have a through understanding about thebiological vision system.


Ming Jiang






Course Overview

References

Image Understanding

I Image understanding by a machine can be seenas an attempt to find a relation between inputimage(s) and previously established models of theobserved world.

I Transition from the input image(s) to the modelreduces the information contained in the image torelevant information for the application domain.

I This process is usually divided into several steps andseveral levels representing the image are used.

I The bottom layer contains raw image data and thehigher levels interpret the data.

I Computer vision designs these intermediaterepresentations and algorithms serving to establishand maintain relations between entities within andbetween layers.


Ming Jiang






Course Overview

References

Image Understanding







Ming Jiang






Course Overview

References

Image Understanding







Ming Jiang






Course Overview

References

Image Understanding







Ming Jiang






Course Overview

References

Image Understanding







Ming Jiang






Course Overview

References

Image Representation

I Image representation can be roughly dividedaccording to data organization into four levels.

I The boundaries between individual levels arc inexact,and more detailed divisions are also proposed.


Ming Jiang






Course Overview

References

Image Representation

I Image representation can be roughly dividedaccording to data organization into four levels.

I The boundaries between individual levels arc inexact,and more detailed divisions are also proposed.


Ming Jiang






Course Overview

References

Image Representation & Hierarchy ofComputer Vision

I This suggests a bottom upway of informationprocessing, from signals withalmost no abstraction, to thehighly abstract descriptionneeded for imageunderstanding.

I The flow of information doesnot need to be unidirectional.

I Feedback loops are oftenintroduced to allow themodification of algorithmsaccording to intermediateresults.


Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References






Ming Jiang






Course Overview

References

Two Levels

I This hierarchy of image representation and relatedalgorithms is frequently categorized in an evensimpler way.

I Two levels are often distinguished:I low-level image processing;I high-level image understanding.


Ming Jiang






Course Overview

References

Two Levels




Ming Jiang






Course Overview

References

Two Levels




Ming Jiang






Course Overview

References

Two Levels




Ming Jiang






Course Overview

References

Low-level processing

I Low-level processing methods usually use very littleknowledge about the content of images.

I Low-level methods often include image compression,pre-processing methods for noise filtering, edgeextraction, and image sharpening.

I Ee shall discuss in this course.I Low-level image processing uses data which

resemble the input image.I Very often, such a data set will be part of a video

stream with an associated frame rate.I E.g., an input image captured by a TV camera is 2D

in nature, being described by an image functionf (x , y , t) whose value, at simplest, is usuallybrightness depending on parameters x , y and t .


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

High-level processing I

I High-level processing is based on knowledge, goals,and plans of how to achieve those goals.

I Artificial intelligence methods are widely applicable.I High-level computer vision tries to imitate human

cognition (although be mindful of the health warninggiven in the very first paragraph of this chapter) andthe ability to make decisions according to theinformation contained in the image.

I In the example described, high-level knowledgewould be related to the shape of a cow and thesubtle interrelationships between the different partsof that shape, and their (inter-)dynamics.


Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References







Ming Jiang






Course Overview

References

High-level processing III High-level vision begins with some form of formal

model of the world, and then the “reality” perceived inthe form of digitized images is compared to themodel.

I A match is attempted.I When differences emerge, partial matches (or

sub-goals) are sought that overcome themismatches.

I The omputer switches to low-level image processingto find information needed to update the model.

I This process is then repeated iteratively, and“understanding” an image thereby becomes aco-operation between top-down and bottom-upprocesses.

I A feedback loop is introduced in which high-levelpartial results create tasks for low-level imageprocessing.

I The iterative image understanding process shouldeventually converge to the global goal.


Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References

Low- vs. high-level representationsI Both representations contain exactly the same

information.I But for a human observer it is not difficult to find a

correspondence between them, and without thesecond, it is unlikely that one would recognize theface of a child.

I The point is that a lot of a priori knowledge is used byhumans to interpret the images.

I A machine only begins with an array of numbers andso will be attempting to make identifications anddraw conclusions from data that to us are moreuncomprehensible.

I Increasingly, data capture equipment is providingvery large data, sets that do not lend themselves tostraightforward interpretation by humans.

I We have already mentioned terahertz imaging as anexample.

I General knowledge, domain-specific knowledge, andinformation extracted from the image will be essentialin attempting to “understand” these arrays ofnumbers.


Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References


The following sequence of processing steps is commonlyrecognized:

I Image Acquisition: An image is captured by a sensor(such as a TV camera) and digitized. Image maycome in many formats and ways.

I Preprocessing: Image reconstruction or restoration,denoising and enhancement. E.g., computertomography.

I Image coding and compression: this is important fortransferring images.

I Image segmentation: computer tries to separateobjects from the image background.

I Object description and classification in a totallysegmented image is also understood as part oflow-level image processing.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

Image SegmentationI Image segmentation is to separate objects from the

image background and from each other.I Total and partial segmentation may be

distinguished.I Total segmentation is possible only for very

simple tasks, an example being the recognition ofdark non-touching objects from a light background.

I Example: optical character recognition, OCR.I Een this superficially simple problem is very hard to

solve without error.I In more complicated problems (the general case),

low-level image processing techniques handle thepartial segmentation tasks, in which only the cueswhich will aid further high-level processing areextracted.

I Often, finding parts of object boundaries is anexample of low-level partial segmentation.


Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References

Low-level Image Processing

I Low-level computer vision techniques overlap almostcompletely with digital image processing, which hasbeen practiced for decades.

I Object description and classification in a totallysegmented image are also understood as part oflow-level image processing.

I Other low-level operations are image compression,and techniques to extract information from (but notunderstand) moving scenes.

I .......


Ming Jiang






Course Overview

References





I .......


Ming Jiang






Course Overview

References





I .......


Ming Jiang






Course Overview

References





I .......


Ming Jiang






Course Overview

References

Low vs High

I Low-level image processing and high-level computervision differ in the data used.

I Low-level data are comprised of original imagesrepresented by matrices composed of brightness (orsimilar) values.

I High-level data originate in images as well, but onlythose data which are relevant to high-level goals areextracted, reducing the data quantity considerably.

I High-level data represent knowledge about theimage content. —

I E.g., object size, shape, and mutual relationsbetween objects in the image.

I High-level data are usually expressed in symbolicform.


Ming Jiang






Course Overview

References

Low vs High








Ming Jiang






Course Overview

References

Low vs High








Ming Jiang






Course Overview

References

Low vs High








Ming Jiang






Course Overview

References

Low vs High








Ming Jiang






Course Overview

References

Low vs High








Ming Jiang






Course Overview

References

Image ProcessingI Most current low-level image processing methods

were proposed in the 1970s or earlier.I Recent research is trying to find more efficient and

more general algorithms, implementations.I The requirement for better and faster algorithms is

fuelled by technology delivering larger images (betterspatial resolution), and color.

I A complicated and so far unsolved problem is how toorder low-level steps to solve a specific task, and theaim of automating this problem has not yet beenachieved.

I It is usually still a human operator who finds asequence of relevant operations.

I Domain- specific knowledge and uncertainty causemuch to depend on this operator’s intuition andprevious experience.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

High-level VisionI High-level vision tries to extract and order image

processing steps using all available knowledge.I Image understanding is the heart of the method, in

which feedback from high-level to low-level is used.I Unsurprisingly this task is very complicated and

computationally intensive.I David Marr’s book [Marr, 1982] influenced computer

vision considerably throughout the 1980s.I It described a new methodology and computational

theory inspired by biological vision systems.I Developments in the 1990s moved away from

dependence on this paradigm, but interest inproperly understanding and then modeling humanvisual systems.

I It remains the case that the only known solution tothe “vision problem” is our own brain!


Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References










Ming Jiang






Course Overview

References

3D Vision Problems

Figure: Several 3D vision tasks and algorithmic componentsexpressed on different abstraction levels. We adopt the user’sview, i.e., what tasks performed routinely by humans would begood to accomplish by machines.


Ming Jiang






Course Overview

References

3D Vision ProblemsI What is the relation of these 3D vision tasks to

low-level (image processing) and high- level (imageanalysis) algorithmic methods?

I There is no widely accepted view in the academiccommunity.

I Links between (algorithmic) components andrepresentation levels are tailored to the specificapplication solved, e.g., navigation of an autonomousvehicle.

I These applications have to employ specificknowledge about the problem solved to becompetitive with tasks which humans solve.

I More general theories are expected to emerge.I Many researchers in different fields work on related

problems.I There is a belief that research in ’cognitive systems’

could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References








could be the key.


Ming Jiang






Course Overview

References

Course Overview

I Digital image processing, image analysis, imageunderstanding are related branches of computervision.

I This course is about digital image processing.I The following topics are to be covered in this course.


Ming Jiang






Course Overview

References

Course Overview




Ming Jiang






Course Overview

References

Course Overview




Ming Jiang






Course Overview

References

Course SyllabusI Introduction and Course OverviewI Image representations and properties

I Images as a stochastic processes or linear systems,etc.

I Metric and topological properties of digital imagesI HistogramsI Noise in images

I Data Structures for Image AnalysisI Image Pre-processing

I Various pre-processing operatorsI Image Segmentation

I Thresholding, edge-based, region growing,segmentation method.

I Scale Space TheoryI Image processing and partial differential equations.


Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References









Ming Jiang






Course Overview

References

The textbook and web resource

I Milan Sonka, V. Hlavac, R. Boyle: Image Processing,Analysis and Machine Vision, 3rd edition. ThomsonLearning, 2008.

I Image Processing, Analysis, and Machine Vision: AMATLAB Companion, http://visionbook.felk.cvut.cz/


Ming Jiang






Course Overview

References

The textbook and web resource

I Milan Sonka, V. Hlavac, R. Boyle: Image Processing,Analysis and Machine Vision, 3rd edition. ThomsonLearning, 2008.

I Image Processing, Analysis, and Machine Vision: AMATLAB Companion, http://visionbook.felk.cvut.cz/


Ming Jiang






Course Overview

References

References

I Kenneth R. Castleman: Digital Image Processing.Prentice-Hall International, Inc. 1996. Or TsinghuaUniversity Press, 1998.

I Rongchun Zhao: Introduction to Digital ImageProcessing (in Chinese). Northwestern PolytechnicalUniversity Press, 2000.


Ming Jiang






Course Overview

References

References

I Kenneth R. Castleman: Digital Image Processing.Prentice-Hall International, Inc. 1996. Or TsinghuaUniversity Press, 1998.

I Rongchun Zhao: Introduction to Digital ImageProcessing (in Chinese). Northwestern PolytechnicalUniversity Press, 2000.


Ming Jiang






Course Overview

References

SummaryI Human vision is natural and seems easy; computer

mimicry of this is difficult.I We might hope to examine pictures, or sequences of

pictures, for quantitative and qualitative analysis.I Many standard and advanced AI techniques are

relevant.I “High” and “low” levels of computer vision can be

identified.I Processing moves from digital manipulation, through

pre-processing, segmentation, and recognition tounderstanding — but these processes may besimultaneous and co-operative.

I An understanding of the notions of heuristics, a prioriknowledge, syntax, and semantics is necessary.

I The vision literature is large and growing; books maybe specialized, elementary, or advanced.

I A knowledge of the research literature is necessaryto stay up to date with the topic.

I Developments in electronic publishing and theInternet are making access to vision simpler.


Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References












Ming Jiang






Course Overview

References

References I

Marr, D. (1982).Vision.Freeman.

Russ, J. C. (1995).The Image Processing Handbook, 2nd ed.CRC Press.

Zhao, K. and Zhong, X. (1982).Optics.Peking University Press, Beijing.2 volumes, in Chinese.

Digital Image Processing - PKU...Digital Image Processing Ming Jiang Digital image processing: What, Why and How What Are the Difﬁculties Poor understanding of human vision system

Documents