Intelligent image processing

Jake Larrimore, Andrew Stewart, Esteban Bernal 

Intelligent Image Processing

What will be discussed?

1. What is Image Processing?2. History3. Push- Consumer driven 4. Pull- Industry (internally driven)5. How does Image Processing work?6. Advantages and Disadvantages7. Current Applications8. State of the art9. Future direction

What is Intelligent Image Processing

•Well, can anyone define Intelligence?•A rat can process visual data and

interpret it in order to solve problems, but we would not consider a rat intelligent in comparison to a human. Yet this simple task is extraordinarily difficult to program a computer to do, and we have come nowhere close to enabling computers to match the image processing perceptibility of rats.

Image Processing Defined

•Computer vision broadly refers to the discipline where extraction of useful 2D and/or 3D information from one or more images is of interest [Chellepe et. al, 2005]

•Many computers and hand held devices have cameras embedded in them, but they do not process that information to perform a task, therefore we cannot say that such a device has vision

History

•Artificial Intelligence•We live in a three-dimensional and

dynamic world.  Therefore, in order for a robot or other A.I. artifact to interact with its surroundings, it must be able to obtain and process information through some sort of sensing ability.

Defining the field

•What information should be extracted from the outputs of visual sensors

•How is this information extracted•How should this information be

represented•How must this information be used to

allow a robot system to perform its task [Faugeras, 1949]

New Fields

•Neuromorphic Engineering - recreate the way the eye and other neurobiological sensing systems work and applying it to silicon chips.

•  Imaging device must contend with shadows and sunlight; conventional sensors, such as those in digital cameras, can't capture pictures well under these conditions

Push

•We now have regular access to computers with dual core processors, and some with multi-level processors that can manage multiple GHz.

•high speed networking

Pushing, continued• artificial neural networks

▫mathematical models derived from biological neural networks

• After the development of the “back-propagation learning algorithm for neural networks, [it was for the] first time...feasible to train a non-linear neural network equipped with layers of the so-called hidden nodes [Egmont-Petersena et. al, 2001].”▫preprocessing images, image reconstruction, image

restoration, image enhancement, data reduction and feature extraction, and image compression

Pull

•Robotic Vision▫Developed to give autonomous robots the

ability to interact freely with their environment

▫Some scientists say that “autonomous navigation has become a mandatory function of mobile intelligent robots [Kim et. al, 2008].”

Pull- Defense and Security

•The ability to detect threats to the public without human interaction would be vital to reducing the cost, time, and efficiency of such security▫TSA and Airports▫Crowd Control/Riot Control▫Tracking of Fugatives

Pull- Safety on the Roads• Traffic and automobiles

implemented in automobiles, traffic lights, and city streets

•National Highway Traffic and Safety Administration reports that there were a reported 6.4 million car accidents on the streets of the U.S. causing over 230 billion dollars in damage. These accidents killed almost 30 thousand people and injury about 2.9 million people

Pull-Medical Fields • (CT) scans are generally used to make the

diagnostic and to plan the surgery for liver cancer▫radiologist must trace the contour of the liver

manually as well as the tumor and the main vessels (which show up very similarly on scans)

• If we had intelligent processes to investigate these scans to more accurately determine the condition of patients and to enhance the scans to produce better and more vivid images

Still Pulling- Manufacturing Automation•computer–based machine vision system

applied in computer-aided inspection- Chips, coffee beans, etc

•Safer work areas to ensure that workers are not injured by automated devices

Pull- ENTERTAINMENT!• 20 percent of households with more than $77,000 a

year in pretax income, more money is spent on entertainment - $4,516 a year - than on health care, utilities, clothing or food eaten at home [Darlin, 2005]

• Billions of dollars driving the market toward more user friendly computer interaction

• We should be able to communicate in a more intuitive manner, directly with a context-aware environment, thus enabling them to achieve their goals more easily and freeing their minds to think even further ahead of their current tasks and problems [Meyer et al, 2003]

More Entertainment

•QB1-They are using multiple cameras to achieve depth perception in computers, which enables them to have an interface based the user directly touching and manipulating virtual components positioned around his body

•Applications in gaming

Basic Idea

•Intelligent / non-intelligent

•Humanistic Intelligence: Recognizing that the human brain is perhaps the greatest neural network of its kind ▫WearComp▫Eyetap

Basic Idea - WearComp

•“Always ready" device•Six informational paths of interaction

▫ Unmonopolizing of user’s attention▫ Attentive to the environment▫ Communicative to others▫ Unrestrictive▫ Observable▫ Controllable

Basic Idea - Eyetap• Lightspace analyzer• Lightspace modifier• Lightspace synthesizer

Computer and Machine Vision

•CV: "the science and technology of machines that see, where see in this case means that the machine is able to extract information from an image that is necessary to solve some task” - Wikipedia

•CV: focus on the complex real-world situations

•MV: focus on machines that can see

Image Processing Chain (IPC)

•Describe the steps and operations involved to successfully extract data from an image

•General operations utilized across different image processing systems:

IPC – Pre-processing

•Suppress unwilling distortions•Enhance Important features•Divided in three operations

▫Reconstruction▫Restoration▫Enhacement 

IPC - Segmentation•Partitioning into correlated and

not overlapped fragments• Statistical pattern recognition • Neural networks

IPC – Object Recognition

•Requires knowledge•Knowledge representation: 

▫grammars and languages▫predicate logic▫production rules▫fuzzy logic▫semantic nets▫frames and scripts

IPC – Object Recognition

IPC – Image understanding

•Find a relation between the input images and previously established models of the real world [Sonka et al, 2008]

•eTRIMS project (University of Bonn)

Performance: Advantages

•Technology improvements  storage  processing power bandwidth and wireless access image resolution Supercomputing processing

• Facilitate human's life (Google goggles)• Improve human's life (Medical usage, traffic

safety)• Improve economic (Manufacturing) 

Performance Disadvantages

•Some technologies are expensive To develop To maintain

• Reduce the need of human work?• Technical difficulties

Lost of information Interpretation Noise Too much data 

Applications

•Wherever you can image - Just a few examples:

Automotive industry pedestrian detection Potato chips image processing system to

control quality Medical applications - diseases detection Traffic control Autonomous driven cars 

•Limitations?  Human's ability of understanding the brain

The state of the art

•Image processing formerly the domain of large institutions

•Very specific applications

•Large projects

•Imaging technology is now widely available

•Consumer products

Google Goggles

•First came text-based text searching•Then came text-based image searching•Now image-based image searching

Microsoft Photosynth

•Stitches together a three-dimensional scene from several images of the same subject

•Creates a navigable scene

Microsoft Photosynth

Autonomous Driving

•An example of computer vision•Norman Bel Geddes’ Futurama (1939)

•Still a ways off…

Lane Departure Warning System

Lane Departure Warning System•Canny edge detection algorithm

•Line extraction by Hough transformation

Pedestrian tracking system

•Shape-based voting algorithm•Similar Gaussian and Hough methods•Other applications

▫Automatic doors▫Light usage (efficiency)▫Cash register (security)

Hand & gesture tracking

•Control of entertainment systems▫XBox Kinnect

•Sign language

Face detection

•Many applications▫Bankcard identification▫Access control▫Security monitoring▫Biometrics systems

•Advancement based on▫Large image databases▫Advances in algorithms▫Methods for evaluating performance

•More difficult than simple line detection

Face detection

•Traditional methods▫PCA▫Neural networks▫Sparse graph

matching▫HMMs▫Template matching

•Newer methods▫Improved template

matching w/ 3D models

▫Line Edge Map (LEM)

▫SVMs

Face recognition

Face recognition

Medical imaging

•Computerized tomography•Magnetic resonance imaging•Ultrasound•Nuclear medicine imaging•Computerized hematological cell analysis

Medical imaging

•Knowledge based systems▫Rule based expert systems▫Structural-functional correlation▫Artifact reduction

•Trending towards convergence of artificial intelligence and image analysis

Looking ahead…

Hyperspectral imaging

•Beyond the visible spectrum

Hyperspectral imaging

Hyperspectral imaging

Hyperspectral imaging

LIDAR

•Light Detection and Ranging•Remote optical sensing technology•Three-dimensional contoured imaging

LIDAR

LIDAR

Biologically motivated processing

Nonlinear methods

•Linear methods OUT▫Human visual system too complicated for

linear models▫Have difficulty removing unwanted noise

•Nonlinear methods IN▫Generally superior in edge smoothing,

enhancement, filtering, feature extraction, etc

▫Computationally expensive▫Reduced cost makes these practical and

effective