Vision-Guided Robot Position Control

SKYNETTony Baumgartner – Computer Science

Jeff Clements – Mechanical Engineer

Norman Pond – Electrical Engineer

Brock Shepard – Mechanical Engineer

Nicholas Vidovich – Computer Engineer

Advisors: Dr. Juliet Hurtig & Dr. J.D. Yoder

February 19, 2004

Vision-Guided RobotPosition Control

Problem Identification

Equipment Provided

• Desktop computer

• Two CCD (Charge-Coupled Device ) cameras

• ABB Robot

• Controller

RobotABB – Articulated IRB 140

Controller

Design Deliverables

Image Processing Completion Date– Difference Algorithm 12/19/03– Blob Detection 02/20/04– Object Recognition 03/17/04– User Interface 03/17/04

Overall– CPU/Robot Communication 03/17/04– Gripper Implementation 03/17/04– Testing 04/15/04

Vision Systems

Eye-On-Hand Configuration

• Limited view of work area

• Partial view of object

• Continually changes coordinate system mapping

Vision Systems

External Stereo Camera Configuration

• Distance fixed between cameras

• Coordinate system relation between camera space and real space fixed

• Full view of object

Software Block Diagram CaptureInitial

Capture Current

Compare Using

Difference

Ready?

Identify Object

/Find

Coordinates

Send Instructions

Controller Commands

Robot

Check Actions

Done?

Difference Algorithm Visuals

Object Recognition Algorithms

• Shape-Based Matching– Thresholding– Blob Detection– Pattern Recognition– Similarity Measure

Object Recognition VisualsNo Object in Scene

Object Recognition VisualsObject Enters Scene

Object Recognition VisualsBlob Show

Object Recognition VisualsBlob Find

Other Object Recognition Visuals

Other Object Recognition Visuals

Current Status

• Object recognition is currently being developed

• Serial communications with robot controller and computer are being established to send and receive coordinate values

Future Implementation

After Completion of Object Recognition

• Develop a coordinate system that can be converted from pixels to millimeters

• Develop calculations of where the object actually is in the camera view compared with the actual space

Future Implementation

After Completion of Controller/Computer Connection

• Develop algorithm that loops receiving values and transmitting values to move the robot

QUESTIONS

?

References<1> ABB Product Specification Sheet (2003)<2> Lin, C.T., Tsai D.M. (2002) Fast normalized cross correlation for

defect detection. Machine Vision. Yuan-Ze University,1-5.<3> Phil Baratti “Robot Precision” (personal communication, November 4

2003)<4> Stegar, C.., 2001. Similarity measures for occlusion, clutter, and

illumination invariant object recognition. In: B. Radig and S. Florczyk(eds), Mustererkennung 2001, Springer Munchen, pp. 145-154.

<5> Stegar, C., Ulrich, M. 2002 Performance Evaluation of 2D Object Recognition Techniques. Technical Report: Technische Universitat Munchen, 1-15.

<6> Robots and Manufacturing Automation, pg. 220-222.<7> http://www.prip.tuwien.ac.at/Research/RobotVision/vs.html