Columbia GraspIt!: A Versatile Simulator for Robotic Grasping Andrew T. Miller Columbia University
Jan 05, 2016
Columbia
GraspIt!: A Versatile Simulator for Robotic Grasping
Andrew T. Miller
Columbia University
Columbia
Outline
• Motivation• Simulator Overview• Simulator Front-end• Grasp Analysis• Dynamic Simulation• Performing a Grasping Task• Future Work
Columbia
Simulation for Hand Design
• Mechanical prototypes are costly– Spend time and money
• Evaluate design in simulation– Examine range of motion, finger placement,
kinematics, link geometry– Ensure it can grasp desired objects– Change design easily
Columbia
Simulation for Grasp Planning
• Integrated grasp analysis– Grasp quality, weak point, force optimization
• Visualize grasp from all angles
• Perform many grasps quickly– Faster than using a real arm and hand
• Build a library of saved grasps– Recall grasp when object is encountered again
Columbia
• Library of hands and objects
• Intuitive interface allows many grasps to be tested quickly
• Visualize grasp wrench space
• Quality measures evaluate grasp
GraspIt: A Tool for Grasping Research
Columbia
GraspIt Components
Hand/ObjectConstruction
• read object model• read link models• read kinematics• assemble hand
UserInterface
• view 3D scene• change hand pose• auto-grip• manually move joints
ContactDetermination
• detect collisions• adjust contact to object surface• find contact area• add friction cones
GraspAnalysis
• compute grasp wrench space• use metrics on space• grasp force optimization
Wrench SpaceVisualization
• create 3D projections of GWS
Front end
Back end
ObjectDynamics
• compute object motion
Columbia
Hand Construction
• The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints.
• The 3D link geometries are accurately described in CAD model files.
• The format is flexible and can easily model many of the available complex articulated hands.
Columbia
Hand Configuration File… Header info and palm description ...
#-----------------------------------f0---------------------------------------#number of joints3
#number of links3
#Transforms from palm origin to base of the finger#EXAMPLE:#r 90 x (rotation by 90 degrees about x axis)#rr 3.0368729 y (rotation by 3.0368729 radians about y axis)#t 1.92188 134.5 39.0717 (translation in mm)t 25.0 0 -1.0r 180 y
#Joint Descriptions (1 joint per line)#DOFNum theta d a alpha DOFminval DOFmaxval#(joints are ordered from closest to palm outward)#(linear equations are of the form: q#*k+c [no spaces!])#(coupled joints must come after joints they are coupled to)d0+90 0 50 -90 0 180d1+5 0 70 0 0 144d1*0.333333333+40 0 55 0 0 144
#Link Descriptions (1 link per line)#filename material lastJoint#(links are ordered from closest to palm outward)#(lastJoint is the last joint in the chain which can affect this link)link1 plastic 0link2 plastic 1link3 plastic 2
#-----------------------------------f1---------------------------------------etc...
Columbia
Hand Library
Barrett Rutgers Robonaut
Parallel Jaw DLR
Columbia
User Interaction• Inventor 3D viewer• Rotate hand• Translate hand• Auto-grasp• Manipulate individual
DOF’s
• Simple and intuitive• Direct interaction gives
sense of hand’s capabilities
Columbia
Collision Detection
• PQP system (Larsen et al.)• Each body is a polygonal soup• Creates bounding volume hierarchies• Fast recursive algorithm tests for overlap, or
provides minimum distance
Columbia
Finding Contacts
• Binary search until link is < 0.1mm away• Record all triangle pairs within this distance• Compute points of overlap between each pair• Only keep contact points on region boundary• Mark points with red friction cones
Collision detected Search for instant of contact
Determine contact region
Columbia
Grasp Analysis
• Occurs when a contact is formed or broken• Computes space of forces and torques that can be
applied by the grasp• Quality measures numerically evaluate grasp• Provides a means to evaluate grasps
– Compare grasps of one hand, one object
– Compare grasps of many hands, one object
– Compare grasps of many hands, across a task specified object set.
Columbia
Wrench Spaces
• In 3-space, a wrench is a 6D vector composed of a force and a torque:
• The space of wrenches that may need to be applied during a task is the task wrench space.
• The space of wrenches that can be applied by a grasp is the grasp wrench space.
• A possible quality measure:
τ
fw
GWS
TWS
Columbia
Special Types of Grasps
• A force-closure grasp completely restrains the object.– Origin is contained within grasp wrench space.
• A manipulable grasp can impart arbitrary velocities on the object without breaking contact.
Columbia
Friction Cones
• Friction at a contact point allows forces in directions other than the contact normal
• COF, is determined by the contacting materials• Estimate friction cone as convex sum of a force
vectors on the boundary assuming a unit normal force, .
m
j
jj
1
ff 1f
Columbia
Contact Wrenches
• Each force acts at a position on object.
• Compute the corresponding wrench with respect to object’s center of gravity:
• - torque scale factor,
jii
jiji
,
,, fd
fw
max1 d
i: contact numberj: (1-8) around cone boundary
Columbia
• Objective: find total space of wrenches that can be applied by a grasp of unit magnitude.
• Grasp vector:
• Define with norm: – Sum magnitude of contact normal forces is 1.
• Compute grasp wrench space using qhull:
n
imiiL ConvexHullW
1,1, ,,
1ww
Grasp Wrench Space
1,21 gfffgT
n
1Lg
Columbia
Two Measures of Quality
• Assume task wrench space is unknown.
• Estimate with wrench space ball - good grasps will resist all wrenches equally well.
• Previously proposed measures of quality:– Radius, , of the largest wrench space ball that
can fit within the unit grasp wrench space.– Volume, v, of unit grasp wrench space.
Columbia
Grasp Wrench Space Projections
• To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates.
• A useful choice:– See the forces that can be applied without
applying a net torque (or vice versa).
Columbia
Visualizing the Results
Numeric quality &Worst case indicator
Wrench SpaceProjection
Columbia
Comparing Grasps
Columbia
Comparing Grasps
Columbia
Comparing Grasps
Columbia
Comparing Grasps
Columbia
Grasp Force Optimization
• Given a specific wrench to resist, find the optimal force to apply at contact points
• Forces should obey torque limit constraints and avoid friction cone boundaries
• Previously done by using linearized friction cones and linear programming [Kerr & Roth ’86]– overly conservative
0 extc wGf λNwGf extc
Columbia
Grasp Force Optimization
• GraspIt implements LMI technique by Han et al. ’00, • Does not require friction cone approximation• Feasible Set: • Minimize:
• Determinant Maximization with LMI constraints• Using hand Jacobian, it can also compute optimal
joint torque values
)(detlog)( 1 xxwx PT
),0)(,0)(|{ FGxxxx CPx m
Contact forces Inverse friction cone LMIWeighting factor
Columbia
Grasp Force Optimization
Columbia
Simulating Object Dynamics
• Added realism• Can examine grasp formation process• Formulate non-penetration and friction cone constraints as a
Linear Complementarity Problem [Stewart & Trinkle ’00]
• Lemke’s algorithm solves LCP to find contact forces that satisfy constraints.
0 bAfa
0f0af T force OR acceleration must be 0
Columbia
Simulating Object Dynamics
Columbia
Simulating Object Dynamics
Columbia
Connecting GraspIt to theReal World
Object PoseEstimation
FrameGrabber
GraspExecution
TaskMonitoring
Robot/HandControl
Presented atICRA ‘01
Columbia
Environment Modeling• Easy to import
environmental objects like workbench.
• Puma arm with inverse kinematics module also supplied.
• Prevents user from planning infeasible grasps.
Columbia
Real-Time Vision System
• Supply GraspIt with object pose.
• Needs to track object that is partially occluded by the hand.
• Since we have an object model, why not use model-based pose estimation and tracking?
• Single camera located 2m from workspace, calibrated wrt robot coordinate system.
Columbia
Real-Time Pose Estimationand Tracking
• Position estimation based on Dementhon and Davis.• Tracking based on Araujo and Brown’s extension of Lowe.
3D ProjectionMatrix
Fitting 3DModel
POSE
AffineMotion
NormalFlow
LocateVisible Edges
RenderModel
CameraParameters
ManualInitialization
Columbia
Tracking System Example
Wireframe modelSelected points are
used to determine pose
Pose estimated,and model overlaid
Normal flow estimation
Columbia
Grasp Execution Experiments
Pose estimated User plans grasp Grasp performed
Columbia
Grasp Execution Experiments
Pose estimated User plans grasp Grasp performed
Columbia
Grasp Execution Experiments
Pose estimated User plans grasp Grasp performed
Columbia
Grasping and Visual Servoing
Once pose has been estimated,grasp may be planned,
then executed.
Columbia
Grasping and Visual Servoing
Destination plannedin GraspIt.
Goal position isprojected into image.
Image based servoing drives robot to goal.
Columbia
Planning a Grasping Task in GraspIt
Columbia
Planning a Grasping Task in GraspIt
Columbia
Planning a Grasping Task in GraspIt
Columbia
Contributions• Grasping simulator
– Flexible method to specify hand construction– Interactive interface – Contact determination system– Inclusion of material properties
• Integrated grasp analysis theory into simulation system– GWS, quality measures, GFO
• Visualization techniques• Object Dynamics: First implementation of S-T algorithm
with large scale and interactive bodies• Integration of simulation system into grasping task• A useful tool for research in grasping and hand design:
– Currently being used by a number of research groups for design and synthesis: NASA, Sandia, Rutgers, U of MD
Columbia
Future Work
• Evaluate manipulable grasps• Provide method of specifying task wrench space• Human hand model• Full dynamic simulation• Automatic grasp selection
– Combine high-level and low-level synthesis techniques
– Learning techniques