1 NIWeek Vision Summit August 2-3, 2011 Austin, Texas .

1

NIWeek Vision SummitAugust 2-3, 2011 Austin, Texas

www.ni.com/niweek/summit_vision

2

Breaking New Ground with Vision Inspection Systems

Dr. Dan MilkieSenior Developer

Coleman Technologies, Inc.

NIWeek Vision SummitAugust 2–3, 2011 Austin, Texas

What if there’s no existing solution?• Prototyping tips

• Planning guide

• Implementation

• Imaging algorithms– Finding small defects– 3D laser triangulation

• Final lessons

3

About us

• 16 year National Instruments Alliance Partner

• Basler Vision System Integrator

• NI Certified Developers & NI Professional Instructors

• Advanced engineering & science degrees

4

Vision Applications

5

Industrial

Research

– Dinnerware defect detection system – Dinnerware color pattern inspection system– Robotic seed germination classifier– High-speed seed counting system– Laser drilling inspection– Thin film defect identification– Mirror defect detection– PCB contact pad inspection system– Color tablet tracking– Particle size analysis (powders)– Particle size analysis (liquid suspension)– Glass rod inspection– Well plate inspection system– Bio-sample thermal imaging– Wellplate imaging system– Crystal finder/classifier– Biaxial tissue tester– Two-photon microscopy

Plate Inspection System

The ChallengeDinnerware Inspection

•Many different dishes

•1 dish per second

•Most difficult defect:– White bumps on

white plates

6

Defects

Can we image the defects?• Prototype with what you have

(or loaners)– Area-scan camera (GigE)– Lighting

• Directional : Desk lamp• Broad sources : Room lights, diffusers

– NI Vision Development Module

• Optimize lighting & camera placement– Tip : Replace camera with your eye

7

Okay, I see the defects, but can the computer?

8

• Start with NI Vision Assistant– Fastest way to test

processing functions– Estimate time using the

“Performance Meter”– Quickly turn scripts into

LabVIEW VIs

Okay, I see the defects, but can the computer?

• Start with NI Vision Assistant– Fastest way to test

processing functions– Estimate time using the

“Performance Meter”– Quickly turn scripts into

LabVIEW VIs

9

Proof of Principle• Are you confident in your plan yet?• If not, prototype in LabVIEW!

– Combine acquisition and analysis– Tip : Use tester with fresh data or saved image sets

• Practice good coding style– Prototypes Final version– Good code encourages trying new ideas– Documentation should be automatic

10

Going for it

Ready to build your system?•Know your test set, criteria

•Balance goals with sliding scales instead of all-or-nothing

•Solve most pressing issues first– Example : Our 1st generation machine tested 1 plate type

(their most popular) and only 4 defect types

11

Modular, Modular, ModularIndependent Stations1.Bottom view2.3D imaging3.Top view

Lesson learned : •Needed to add baffles•Found interference between stations•Changed how plates cross gaps•Vibration issues

12

NI PCIe-1430Dual CameraLink

NI PCIe-1430Dual CameraLink

NI PCIe-82354x GigE

NI PCIe-82354x GigE

PCPC

1 2 3

Small defect imaging• First version :

• Next version:

• Defects show up as dark spots:

13

LEDlightLEDlight

Line

scan

Ca

mer

a

Line

scan

Ca

mer

a

Glancing Angle

Transmission

Small Defects are Hard to Find!• Problems :

– Defects are small, low contrast– Large gradients in image– Plate-to-plate variations– Image size (8MP!)– Must process in < 0.5 second

• Solution:– A custom pixel-by-pixel threshold

for each image.

14

Defect!Defect!

Step 1 : Unwrap ImageProblem : We just need rim pixels•Use image masks?

– Longer process times– Still have large images

Solution : IMAQ Unwrap• 8M pixel square -> 1.5M pixel strip

•Aligns gradients, rim transitions in one direction

15

Step 2 : Create Custom Golden MasterWhat should this image look like if it were perfect?

•Remove defects using median smoothing– Tip : Use X Size >> Y Size to preserve gradients & edges– Performance Boost : Reduce image resolution , Smooth, then

Resample back to original size.

16

Unwrapped OriginalMedian Smoothed“Golden Master”

DefectDefect Defect Removed!Defect Removed!

Step 3 : Pixel-by-pixel Threshold

17

“Golden Master”

Results

Subtract aconstant to seta “lower”threshold.

Original

Defect Found!

Threshold original image using IMAQ Compare

Benefits of Pixel-by-pixel Thresholds

• Works with:– Gradients, edges, speckle, large dynamic contrasts

• Every image checks against itself – Robust against image-image variations, changing

lighting conditions

• Fast (<100ms for 1.5MP)– Limited by smoothing performance

18

3D Imaging

• Simple inspection for geometric errors– Gouges, bulges, warp

19

Warp examples

Laser Triangulation

20

LaserLine

LaserLine Area

CameraArea

Camera

Original

Thresholdapplied

Convert pixels to height

21

With a little more geometry, we can also correct for perspective :

WD

LaserLine

LaserLine Area

CameraArea

Camera

H

dyHeight =

3D Laser Height Measurement

22

• Each camera frame = 1 cross section (per laser line)– Tip : Add multiple laser lines for more cross sections

• 250 um cross section resolution (100 FPS @ 1” per sec)– Tip : Reduce ROI for fastest frame rates

• 100 um height resolution

Completed system

• Fast development with NI tools: – Completed in < 3 months

• Reliable– Inspected over 1 million dishes

• Multiple follow-up systems

23

Conclusions• Invest in good prototyping practices

• Prepare for unknown hurdles– Use modular, flexible architectures in hardware, layout,

software

• Defect finding algorithm– Pixel-by-pixel thresholds

• Simple 3D laser scanner– Inexpensive, easy to setup

24

Questions?

25

26

NIWeek Vision SummitAugust 2–3, 2011 Austin, TexasKeynotes

Industry Keynote: How NI Technology Powers the Space Elevator LaserMotive Tuesday, August 21:00-2:00

Academic Keynote: Industry Trends and Intelligent Production Systems of the Future Interdisciplinary Imaging and Vision Institute, RWTH Aachen University

Wednesday, August 31:00-2:00

Tuesday, August 2

3D Vision and the Kinect National Instruments 10:30-11:30

Wacky Optical Tricks for Machine Vision GE Global Research Center 2:15-3:15

Panel Discussion: Latest in Camera Technologies

Cyth SystemsBasler Vision TechnologiesPleora TechnologiesToshiba TELI

3:30-4:30

Machine Vision and Industrial Robotics: From Design Concepts to Factory Floor Deployment ImagingLab 4:45-5:45

Wednesday, August 3

Precision Metrology National Instruments 10:30-11:30

Is LabVIEW FPGA Right for My Vision Application? National Instruments 2:15-2:45

Autofocus System for an Ellipsometer Nanometrics Inc. 2:45-3:15

Developing a Quality Inspection Method for Selective Laser Melting of Metals Using a High-Speed NIR Camera Katholieke Universiteit Leuven 3:30-4:00

Web Inspection of Optical and Medical Fibers Adsys Controls Inc. 4:00-4:30

Breaking New Ground with Vision Inspection Systems Coleman Technologies Inc. 4:45-5:15

Development of a Digitally Multiplexed Bioassay Reader with Magnetic Bead Technology MoviMED 4:15-4:45

27