General introduction to computer vision

CS 591 E / CS 791 CS 591 E / CS 791 LL

(CRN: 18390 / (CRN: 18390 / 18490)18490)

Computer VisionComputer VisionInstructor: Guodong GuoInstructor: Guodong Guo

[email protected]@mail.wvu.edu

Welcome!Welcome!

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

Journals Journals ConferencesConferences

InstructorInstructor

Guodong GuoGuodong Guo Ph.D. in CS from UW-MadisonPh.D. in CS from UW-Madison http://www.cs.wisc.edu/~gdguo Major Research InterestMajor Research Interest

Computer Vision, Machine Learning, Computer Vision, Machine Learning, Pattern Recognition, Biometrics, Pattern Recognition, Biometrics, Multimedia, and HCIMultimedia, and HCI

About You …About You …

What do you know already?What do you know already? C/C++ (Visual C++)C/C++ (Visual C++) MatlabMatlab ImagesImages OpenCVOpenCV

http://sourceforge.net/projects/opencvlibrary/

Install OpenCV in your PC or laptop, Install OpenCV in your PC or laptop,

Read the manual introductionRead the manual introduction

Try to load and save images (homework #0)Try to load and save images (homework #0)

OutlineOutline

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

Meeting TimesMeeting Times

LecturesLectures M 17:00-19:30 pm M 17:00-19:30 pm Room ESB-E 449Room ESB-E 449

Office hoursOffice hours TR 1:00-2:00 pm (ESB 753)?TR 1:00-2:00 pm (ESB 753)? Or by appointmentOr by appointment

GradingGrading

The final grade depends on:The final grade depends on: Homework and programming Homework and programming

assignments: 40%assignments: 40% Exams (Midterm): 40%Exams (Midterm): 40% Final project (may include class Final project (may include class

presentation): 20%presentation): 20% Class participation: (-5%, if absent >= Class participation: (-5%, if absent >=

3times)3times) Extra: 1~10% (for creative ideas, paper Extra: 1~10% (for creative ideas, paper

submission, etc.)submission, etc.)

TextbookTextbook

Computer Vision: A Computer Vision: A Modern ApproachModern Approach, , 22thth Edition, by Edition, by David Forsyth and David Forsyth and Jean Ponce, Jean Ponce, Prentice Hall, 2003Prentice Hall, 2003

Look at the SyllabusLook at the Syllabus

Course ObjectivesCourse Objectives Expected learning outcomesExpected learning outcomes Detailed list of topics (maybe Detailed list of topics (maybe

updated)updated)

OutlineOutline

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

What is Computer What is Computer Vision?Vision?

Given an image or more, extract Given an image or more, extract properties of the 3D world properties of the 3D world

•Traffic sceneTraffic scene• Number of vehiclesNumber of vehicles• Type of vehiclesType of vehicles• Location of closest obstacleLocation of closest obstacle• Assessment of congestionAssessment of congestion

Computer Vision vs. Computer Vision vs. GraphicsGraphics

3D3D2D implies information loss2D implies information loss

sensitivitysensitivity to errors to errors need for need for modelsmodels

graphics

vision

Computer Vision vs. Computer Vision vs. BiometricsBiometrics

BiometricsBiometrics comprises methods for comprises methods for uniquely recognizing humans based upon uniquely recognizing humans based upon one or more intrinsic physical or one or more intrinsic physical or behavioral traitsbehavioral traits PhysiologicalPhysiological are related to the shape of the are related to the shape of the

body, e.g., fingerprint, face recognition, body, e.g., fingerprint, face recognition, DNA, hand and palm geometry, iris DNA, hand and palm geometry, iris recognition, which has largely replaced recognition, which has largely replaced retina, and odor/scentretina, and odor/scent

BehavioralBehavioral are related to the behavior of a are related to the behavior of a person, e.g., typing rhythm, gait, and voiceperson, e.g., typing rhythm, gait, and voice

Computer Vision vs. Computer Vision vs. BiometricsBiometrics

Biometrics is a branch of Computer Biometrics is a branch of Computer VisionVision

The development of Biometrics The development of Biometrics depends on Computer Vision depends on Computer Vision techniquestechniques

Computer Vision vs. Computer Vision vs. Machine LearningMachine Learning

Machine learningMachine learning is a scientific is a scientific discipline that is concerned with the discipline that is concerned with the design and development of algorithms design and development of algorithms that allow computers to change behavior that allow computers to change behavior based on data, such as from sensor data based on data, such as from sensor data or databases (from Wikipedia)or databases (from Wikipedia)

A major focus of machine learning A major focus of machine learning research is to automatically learn to research is to automatically learn to recognize complex patterns and make recognize complex patterns and make intelligent decisions based on data. intelligent decisions based on data.

Computer Vision vs. Computer Vision vs. Machine LearningMachine Learning

Machine Learning is very useful for Machine Learning is very useful for Computer Vision (e.g., learning for vision)Computer Vision (e.g., learning for vision)

Computer Vision is more than just learningComputer Vision is more than just learning ModelingModeling Example based learningExample based learning

In Machine Learning, it usually does not In Machine Learning, it usually does not care about how to obtain the data or sensorscare about how to obtain the data or sensors

In Computer Vision, we care how to obtain In Computer Vision, we care how to obtain the visual data (sensor design, active vision), the visual data (sensor design, active vision), how to represent the visual data, and othershow to represent the visual data, and others

VisionVision

Vision is the process of discovering Vision is the process of discovering what is present in the world and what is present in the world and where it is by looking.where it is by looking.

Computer VisionComputer Vision

Computer Vision is the study of Computer Vision is the study of analysis of pictures and videos in analysis of pictures and videos in order to achieve results similar to order to achieve results similar to those as by people.those as by people.

Why Computer VisionWhy Computer Vision

An image is worth 1000 wordsAn image is worth 1000 words Many biological systems rely on Many biological systems rely on

visionvision The world is 3D and dynamicThe world is 3D and dynamic Cameras and computers are cheapCameras and computers are cheap ……

Computer Vision Computer Vision ExamplesExamples

Finding People in imagesFinding People in images

Problem 1:Problem 1: Given an image I Given an image I

Question:Question: Does I contain an image of a Does I contain an image of a person?person?

““Yes” InstancesYes” Instances

““No” InstancesNo” Instances

Some Computer Vision Some Computer Vision TopicsTopics

Imaging GeometryImaging Geometry

Camera ModelingCamera Modeling

Pinhole CamerasPinhole Cameras LensesLenses Camera Camera

Parameters and Parameters and CalibrationCalibration

Image Filtering and Image Filtering and EnhancingEnhancing

Linear Filters Linear Filters and Convolutionand Convolution

Image Image SmoothingSmoothing

Edge DetectionEdge Detection PyramidsPyramids

Image Filtering and Image Filtering and Enhancing (cont.)Enhancing (cont.)

Region SegmentationRegion Segmentation

ColorColor

TextureTexture

Image RestorationImage Restoration

Original Synthetic

Perceptual OrganizationPerceptual Organization

Perceptual OrganizationPerceptual Organization

Shape AnalysisShape Analysis

StereoStereo

Motion and Optical FlowMotion and Optical Flow

High Level VisionHigh Level Vision

Image MosaicImage Mosaic

One Very Successful One Very Successful ExampleExample

Face detection in a digital cameraFace detection in a digital camera The camera detects faces in a scene The camera detects faces in a scene

and then automatically focuses (AF) and and then automatically focuses (AF) and optimizes exposure (AE) and, if needed, optimizes exposure (AE) and, if needed, flash output. flash output.

OutlineOutline

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

ApplicationsApplications autonomous cars, planes, missiles, robots, autonomous cars, planes, missiles, robots,

...... space explorationspace exploration aid to the blind, ASL recognitionaid to the blind, ASL recognition manufacturing, quality controlmanufacturing, quality control surveillance, security, biometricssurveillance, security, biometrics image retrievalimage retrieval medical imaging and analysismedical imaging and analysis ......

OutlineOutline

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

Computer Vision focuses Computer Vision focuses on:on:

What information should be What information should be extracted?extracted?

How can it be extracted?How can it be extracted? How should it be represented?How should it be represented? How can it be used to achieve the How can it be used to achieve the

goal?goal?

Related disciplinesRelated disciplines

Image processingImage processing Pattern recognitionPattern recognition PhotogrammetryPhotogrammetry Computer graphicsComputer graphics Artificial intelligenceArtificial intelligence Machine learningMachine learning Projective geometryProjective geometry Control theoryControl theory

Active Research TopicsActive Research Topics

Object recognitionObject recognition Human behavior analysisHuman behavior analysis Internet and computer visionInternet and computer vision Biometrics and soft biometricsBiometrics and soft biometrics Large scale 3D reconstruction (city level)Large scale 3D reconstruction (city level) Medical image processing Medical image processing Vision for roboticsVision for robotics ……

OutlineOutline

IntroductionsIntroductions Administrative MattersAdministrative Matters Course OutlineCourse Outline Applications of Computer VisionApplications of Computer Vision Computer Vision FocusComputer Vision Focus Computer Vision PublicationsComputer Vision Publications

Computer Vision Computer Vision PublicationsPublications

JournalsJournals IEEE Trans. on Pattern Analysis and Machine IEEE Trans. on Pattern Analysis and Machine

Intelligence (TPAMI)Intelligence (TPAMI) #1 IEEE, Thompson-ISI impact factor: 5.96#1 IEEE, Thompson-ISI impact factor: 5.96 #1 in both electrical engineering and artificial #1 in both electrical engineering and artificial

intelligenceintelligence #3 in all of computer science#3 in all of computer science

Internal Journal of Computer Vision (IJCV)Internal Journal of Computer Vision (IJCV) ISI impact factor: 5.358, Rank 2 of 94 in “CS, artificial ISI impact factor: 5.358, Rank 2 of 94 in “CS, artificial

intelligenceintelligence IEEE Trans. on Image ProcessingIEEE Trans. on Image Processing ……

Importance of CVImportance of CV

From these major journal rankings, From these major journal rankings, we can see the importance of we can see the importance of Computer Vision research in the Computer Vision research in the whole areas of whole areas of Computer ScienceComputer Science Electrical EngineeringElectrical Engineering

Computer Vision Computer Vision PublicationsPublications

ConferencesConferences International Conference on Computer International Conference on Computer

Vision (ICCV)Vision (ICCV) Conf. of Computer Vision and Pattern Conf. of Computer Vision and Pattern

Recognition (CVPR)Recognition (CVPR) Europe Conference on Computer Vision Europe Conference on Computer Vision

(ECCV)(ECCV) ……

Discussions and Discussions and QuestionsQuestions