Vision for Automation Ranga Rodrigo Department of Electronic and Telecommunication Engineering University of Moratuwa Sri Lanka ICIAfS 2008 Vision for Automation Workshop December 10, 2008 ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 1/43
54
Embed
Vision for Automationranga/files/iciafs_workshop08.pdf · ICIAfS 2008 Vision for Automation Workshop December 10, 2008 ICIAfS 2008 Robotics and Automation Workshop Vision for Automation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Vision for Automation
Ranga RodrigoDepartment of Electronic and Telecommunication Engineering
University of MoratuwaSri Lanka
ICIAfS 2008 Vision for Automation Workshop
December 10, 2008
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 1/43
Outline
1 Introduction
2 Applications
3 Vision in Automation
4 Software Tools
5 Examples of State-of-the-Art
6 Summary
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 2/43
Introduction
Outline
1 Introduction
2 Applications
3 Vision in Automation
4 Software Tools
5 Examples of State-of-the-Art
6 Summary
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 3/43
Introduction
What Is Computer Vision?
The goal is the emulation of the visual capabilityof human beings using computers.
In other words, computer vision is making themachine see as we do!It is challenging.Steps:
1 Image acquisition2 Image manipulation3 Image understanding4 Decision making
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 4/43
Introduction
What Is Computer Vision?
The goal is the emulation of the visual capabilityof human beings using computers.In other words, computer vision is making themachine see as we do!
It is challenging.Steps:
1 Image acquisition2 Image manipulation3 Image understanding4 Decision making
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 4/43
Introduction
What Is Computer Vision?
The goal is the emulation of the visual capabilityof human beings using computers.In other words, computer vision is making themachine see as we do!It is challenging.
Steps:1 Image acquisition2 Image manipulation3 Image understanding4 Decision making
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 4/43
Introduction
What Is Computer Vision?
The goal is the emulation of the visual capabilityof human beings using computers.In other words, computer vision is making themachine see as we do!It is challenging.Steps:
1 Image acquisition2 Image manipulation3 Image understanding4 Decision making
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 4/43
Introduction
Main Driving Technologies
Signal processing.Multiple view geometry [2].Optimization.Machine learning.Hardware and algorithms.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 5/43
Applications
Outline
1 Introduction
2 Applications
3 Vision in Automation
4 Software Tools
5 Examples of State-of-the-Art
6 Summary
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 6/43
Applications
Applications 1
Automotive:Lane departure warning systems.Head tracking systems for drowsiness detection.Driver assistance systems.Reading automobile license plates, and trafficmanagement.
Photography:In camera face detection [6], red eye removal, andother functions.Automatic panorama stitching [1].
1(From http://www.cs.ubc.ca/spider/lowe/vision.html)ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 7/43
Applications
ApplicationsMovie and video (a very big industry):
Augmented reality.Tracking objects in video or film and solving for 3-Dmotion to allow for precise augmentation with 3-Dcomputer graphics.Multiple cameras to precisely track tennis and cricketballs.Human expression recognition.Software for 3-D visualization for sportsbroadcasting and analysis.Tracking consistent regions in video and insertvirtual advertising.Tracking for character animation.Motion capture, camera tracking, panoramastitching, and building 3D models for movies.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 8/43
Applications
Camera Tracking
Source: http://www.2d3.com/capability
Show 2d3 video.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 9/43
Tracking human gestures for playing games orinteracting with computers.Tracking the hand and body motions of players (tocontrol the Sony Playstation).Image-based rendering, vision for graphics.
General purpose:Inspection and localization tasks, people counting,biomedical, and security. etc.Object recognition and navigation for mobilerobotics, grocery retail, and recognition from cellphone cameras.Laser-based 3D vision systems for use on the spaceshuttles and other applications.Image retrieval based on content.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 10/43
Applications
ApplicationsIndustrial automation (a very big industry):
Vision-guided robotics in the automotive industry.Electronics inspection systems for componentassembly.
Medical and biomedical (maturing):Vision to detect and track the pose of markers forsurgical applications, needle insertion, and seedplanting.Teleoperations.Quantitative analysis of medical imaging, includingdiagnosis such as cancer.
Security and biometrics (thriving):Intelligent video surveillance.Biometric face, fingerprint, and iris recognition.Behavior detection.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 11/43
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 19/43
Vision in Automation
Inspection: Examples
Defects in parts, measurement of size.Robotic bin picking.If each slot is filled in a carton of pills.Character recognition.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 20/43
Vision in Automation
Visual Servoing
Uses vision in the servo loop [3].Dynamic look and move needs the accuracy of thevision sensor and robot end-effector.Having visual feedback in the control loop increasesthe overall accuracy of the control loop.
Visual ServoingMachine vision can provide closed-loop positioncontrol for a robot end-effector—this is referred to asvisual servoing.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 21/43
Vision in Automation
Visual Servoing
Uses vision in the servo loop [3].Dynamic look and move needs the accuracy of thevision sensor and robot end-effector.Having visual feedback in the control loop increasesthe overall accuracy of the control loop.
Visual ServoingMachine vision can provide closed-loop positioncontrol for a robot end-effector—this is referred to asvisual servoing.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 21/43
Vision in Automation
Visual Servoing—Camera Configuration
End-effector mounted Fixed configuration
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 22/43
Vision in Automation
Servoing Architectures
Is the control structure hierarchical, with thevision system providing set-points as input to therobot’s joint level controller, or does the visualcontroller directly compute the joint-level inputs?Is the error signal defined in 3-D (task space)coordinates, or directly in terms of imagefeatures?
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 23/43
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 34/43
Examples of State-of-the-Art
Outline
1 Introduction
2 Applications
3 Vision in Automation
4 Software Tools
5 Examples of State-of-the-Art
6 Summary
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 35/43
Examples of State-of-the-Art
Segmentation Using Graph Cuts [5]
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 36/43
Examples of State-of-the-Art
Segmentation Using Graph Cuts [5]
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 36/43
Examples of State-of-the-Art
Segmentation Using Graph Cuts [5]
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 36/43
Examples of State-of-the-Art
Segmentation Using Graph Cuts [5]
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 36/43
Examples of State-of-the-Art
3-D Reconstruction
Can we obtain a 3-D view of a scene, given onlya set of (2-D) images?
Yes. Using multiple view geometry, we canreconstruct a scene.Show Leibe et al. video [4].
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 37/43
Examples of State-of-the-Art
3-D Reconstruction
Can we obtain a 3-D view of a scene, given onlya set of (2-D) images?Yes. Using multiple view geometry, we canreconstruct a scene.Show Leibe et al. video [4].
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 37/43
Examples of State-of-the-Art
Object Detection: Face Detection
Show OpenCV sample.ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 38/43
Examples of State-of-the-Art
Navigation: Sanford’s Robot Stanley
Show video.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 39/43
Summary
Outline
1 Introduction
2 Applications
3 Vision in Automation
4 Software Tools
5 Examples of State-of-the-Art
6 Summary
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 40/43
Summary
Conclusion
Vision-based automation is promising.
Solutions are simple in a controlledenvironment.State-of-the-art is very interesting.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 41/43
Summary
Conclusion
Vision-based automation is promising.Solutions are simple in a controlledenvironment.
State-of-the-art is very interesting.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 41/43
Summary
Conclusion
Vision-based automation is promising.Solutions are simple in a controlledenvironment.State-of-the-art is very interesting.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 41/43
Summary
Thank you.
OpenCV examples, and Octave examples are here:http://www.ent.mrt.ac.lk/ ranga/publications.html
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 42/43
Matthew Brown and David Lowe.Recognising panoramas.In Proceedings of the 9th International Conference on Computer Vision, pages1218–1225, Nice, France, October 2003.
Richard Hartley and Andrew Zisserman.Multiple View Geometry in Computer Vision.Cambridge University Press, 2nd edition, 2003.
Seth Hutchinson, Gregory D. Hager, and Peter I. Corke.A tutorial on visual servo control.IEEE Transactions Robotics and Automation, 12 No. 5:651–670, 1996.
Bastian Leibe, Nico Cornelis, Kurt Cornelis, and Luc Van Gool.Dynamic 3D scene analysis from a moving vehicle.In Proceedings of the IEEE Computer Society Conference on Computer Vision andPattern Recognition, pages 1–8, Minneapolis, MN, June 2007.
Carsten Rother, Vladimir Kolmogorov, and Andrew Blake.“GrabCut”: Interactive foreground extraction using iterated graph cuts.ACM Transactions on Graphics: Proceedings of the 2004 SIGGRAPH Conference,23(3):309–314, August 2004.
Paul Viola and Michael Jones.Rapid object selection using a boosted cascade of simple features.In Proceedings of the IEEE Computer Society Conference on Computer Vision andPattern Recognition, pages 511–518, Hawaii, December 2001.
ICIAfS 2008 Robotics and Automation Workshop Vision for Automation 43/43