1 Machine Vision lecture 1 Thomas Moeslund Computer Vision and Media Technology lab. Aalborg University [email protected]
Dec 23, 2015
1
Machine Visionlecture 1
Thomas Moeslund
Computer Vision and Media Technology lab.
Aalborg University
2
• Pose estimation– Pick and place applications
– Bin-picking
Machine vision applications
• Classification
• Quality control
THOR HOF
THOR
Conveyor belt
3
Machine Vision class
• Purpose: – Provide an overview of the different elements
in a machine vision system
4
Machine Vision class• Topics
– Acquisition of images: • Light, camera, lens
– Representation of digital images: • pixels, colors
– Processing of images • filtering, segmentation
– Analysis of images: • feature extraction, pattern recognition
– Your interests and needs?
5
Plan for today
• Where does an image come from?• Image definitions• Camera types• Image formation
– Lens • Creating light for machine vision• ( other applications )
6
Where does an image come from?
7
Where does an image come from?
8
Where does an image come from?
Charged coupled deviceCCD-chip
9
Where does an image come from?
• Integration over time– Exposure time/shutter
time• DK: lukketid
– Maximum charge• Saturation• Blooming
Under exposed
Over exposed
Correct exposed
10
Where does an image come from?
• Image elements, picture elements, pels, pixels
11
Image definitions
12
Digital Image Representation
• Image is seen as a discrete function f(x,y) as opposed to a continuous function
• x and y cannot take on any value!
y
x
f(x,y)
Origin
13
Discrete image coordinate system
y
x
f(x,y)
Origin
y
xf(0,0)
f(2,6)
14
Digital Image Representation
• Pixel representation (bits)– One bit: {0,1}
– One byte = eight bits
– One pixel: one byte = eight bits = one number: [0,255]
– Grey-scale, intensity, black/white: 8 bits = [0,255]
– Binary image: 1 bit {0,1}. Black and white: visualized as: 8 bit {0,255} ( colors: Another lecture )
• Video: 25 – 60 images per second. – Framerate: 25Hz – 60Hz– For example: 500 x 500 pixel at 50Hz => 1.25x107 bytes per second!!!!
15
Digital Image Representation
• An image f(x,y) is represented
as an Array• Width =
number of pixels in x-direction• Height = number of pixels in y-direction• Size (width x height, width > height)• ROI = region of interest
– To reduce the amount of data
Width
Hei
ght ROI
16
Spatial Image Resolution:
• Resolution– The size of an area in a scene that is represented
by one pixel in the image• Different Resolutions are possible (256x256….16x16)
• Lower resolution leads to data reduction!
17
Gray-level Resolution: Quantization
• Different gray-level resolutions: 256, 128, …, 2• Less gray-levels leads to data reduction.• For 256, 128, 64 gray-levels: Difference hardly visible
18
Camera types
19
Interlaced vs. Progressive
20
Sensor Chip Formats
Number of Pixelsfrom 500x500to 5000x5000
Pixel sizefrom 4m x 4 mto 16 m x 16 m
21
Line scan camera
22
Image formation- the lens
23
The lens
• A lens focuses a bundle of rays to one point• Parallel rays pass through a focal point at a distance F beyond the plane of the lens. F is the focal length• O is the optical center• F and O span the optical axis
24
Thin Lens & Gaussian Formula
fbg
111
25
Focus and depth-of-field • Depth-of-field (DK: dybteskarphed)• Distance range in which the blur does not exceed a certain value
fbg
111
26
Aperture (DK: blænde)
• More aperture => better depth-of-field• Downside: less light enters => decrease exposure time =>
risk of blur due to motion
Depth-of-field
27
Field-of-view
• Field-of-view depends on size of chip and focal length• ”Fisheye” lens => small focal length and large field-of-view
28
Lighting in machine vision
29
Levels of Natural Light
[Burke]
30
31
32
33
34
Lighting• ”In machine vision lighting is more than 50%”
• Use controlled lighting!
• Avoid direct/indirect sunlight– Build a housing covering the
field-of-view of the camera
• Avoid highlights
35
Illumination Setups
Directedillumination
Rear illumination
Light field illumination
Diffuse illumination
36
Backlighting
37
38
39
40
41
42
• Viewpoint invariant
• High reflectance in illumination direction
Spherical Marker
43
Infrared Illumination
Visually Opaque IR pass filter
• Fast segmentation by adaptive threshold
• Robust
44
What to remember
45
Exercises (1/3)
• Given a 512 x 512 x 8bit image. How many different images can be made?
• Given a 512 x 512 x 8bit image. How is the memory size reduced when you: – Decrease the grayscale resolution repeatedly by 2– Decrease the x-size and y-size of the image
repeatedly by 2• What do you want to learn in this class?
46
Exercises (2/3)
• Describe the following concepts and provide examples of their usages:– Classification, Quality control, Pose estimation, Bin-
picking
• What is a CCD-chip and how does it operate?• What is Depth-of-field (DK:dybteskarphed)?• Pros and cons of back-lighting?
47
Exercises (3/3)
• Show that the following is true for a thin lens:• Mick is 2m tall and standing 5m from a camera. The
camera’s focal length is 5mm.– At which distance from the lens is Mick in focus?
• How tall (in mm) will Mick be on the CCD-chip?• How tall (in pixels) will Mick be on the CCD-chip?
– The camera has a 1/2” CCD chip
– The camera image has a size of: 480x640 pixels
• What is field-of-view of the camera?
fbg
111
48
Working with images….
• Image manipulation– Simple operations, e.g., scale image
• Image processing– Improve the image, e.g., remove noise
• Image analysis– Analyze the image, e.g., find the person in the image
• Machine vision– Industry, e.g., Quality control, Robot control
• Computer vision– Everything: multiple cameras, video-processing, etc.
49
Image file types
• image.jpg, image.tif, image.gif, image.png, image.ppm, …. • Raw:
– No data is lost – Header + data (234 235 32 21…)– For example: image.pgm – The file can be viewed
• Lossless compression:– No data is lost, but the file cannot be viewed– For example: image.gif
• Lossy compression:– Better compression– Some data is lost (optimized from the HVS’ point of view)– The file cannot be viewed– For example: image.jpg
50
Image file types
• Normally you don’t care about the file type– The application will take care of it for you:– For example: rotate
• Application– image.x => raw– Rotate the raw image– Rotated raw => rotated_image.x
• But to write your own programs from scratch the images need to be in the raw format (without a header).
51
Video Standards
• RS-170– NTSC
• CCIR– PAL, SECAM
• USB, Firewire• Composite• S-video• RGB
52
Applications
53
Digital Image
• Why digital ?• Before 1920: Image transmission from USA to
Europe: more than a week: by ship!• Early 1920s: Bartline cable picture transmission
system: Transmission in three hours!
• Transmission via telegraph/wire, radio signals for newspapers
54
Small Progress
• Small progress in digital imaging until 1964
• Jet Propulsion Lab (JPL) in Pasadena, CA– Transmission and correction of lunar images
from Ranger 7.
• Not so good quality so the images had to be processed before they could be viewed
• Since then many applications…
55
Examples: Image Correction
• Needed when image data is erroneous: – Bad transmission
– Bits are missing: Salt & Pepper Noise
56
Image Deblurring: Motion Blur
• Can be used when a camera or object is moved during exposure
57
Deblurring
• Can be used when the camera was not focused properly!!
58
Image manipulation
• Image improvement, e.g. too dark image
• Rotate + scale
59
Medical Image Processing
• Image Processing is widely used
• E.g. Analysis of microscopic images
60
Medical Image Processing
• MR/CT Imaging of a human body
• Use for Brain Surgery
61
Biometrics
• Recognizing/verifying the identity of a person by analyzing one or more characteristics of the human body
• Characteristics:– Fingerprint, eye (retina, iris), ear, face, heat profile,
shape (3D face, hand), motion (gait, writing), …
• Applications: – Verifying: Access control (bio-passports)
– Recognizing: Surveillance: 9/11
62
Chroma keying
63
Analysis of Sport Motions
• Here: Analysis of motion of Sarah Hughes• 3D Tracking of body parts• Motion interpretation• Action recognition
64
Motion Capture
Andy Serkis
• Special effects– Advertising
– Movies
65
Motion Capture
66
Fundamental Steps in Computer Vision
Knowledge baseProblemdomain Image
acquisition
Preprocessing
SegmentationRepresentationand description
Recognitionand Interpretation
Result
Point 1: 22,33Point 2: 24, 39…..
Actor sitting
67
Shrinking the pinhole
[from Hecht 1987]
68
Shrinking the pinhole even more
[from Hecht 1987]Diffraction causes blur
69
Basic Optics
ReflectionRefraction & DispersionDiffraction
70
Minimum Object Distance
fb
bfMOD
fMODb
max
max
max
111