hap·tic adjective Haptics · hap·tic adjective relating to the sense of touch or to the perception and manipulation of objects using the senses of touch and proprioception 2 Teleoperation

HapticsCS327A - 2017

1

hap·tic adjective

relating to the sense of touch

or to the perception and manipulation of objects using the senses of touch and proprioception

2

Teleoperation - Mechanical

Courtesy of Walischmiller Engineering3

Slave Manipulator

Master Manipulator

Teleoperation - Mechanical

4

Slave Manipulator

Master Manipulator

Teleoperation - Mechanical

5

Slave Manipulator

Master Manipulator

Teleoperation - Mechanical

Passive gravity

compensation6

Courtesy of Walischmiller Engineering7

Teleoperation - MechanicalLimitations?

8

Teleoperation - MechanicalLimitations?

• Physical distance between operator and device is limited

• Passive compensation is not perfect

• Not easy to handle different reference frames for operator and workstation

• Not easy to scale workspace up or down

9

Teleoperation - Electro-mechanical

10

Teleoperation - Electro-mechanical

What feedback does the operator need?

11

Teleoperation - Electro-mechanical

What feedback does the operator need?

• Visual, stereo

• Tactile

• Audio?, smell?

• Other task specific information12

Teleoperation -ElectromechanicalHow do we sense each modality of information on remote manipulator?

• Visual:

• Tactile:

• Audio:

• Smell:

13

Teleoperation -ElectromechanicalHow do we sense each modality of information on remote manipulator?

• Visual: stereo cameras, 3D image sensors

• Tactile: force/torque sensors

• Audio: microphones

• Smell: sensors for specific gases

14

Teleoperation -ElectromechanicalHow do we display each modality of information on master manipulator?

• Visual:

• Tactile:

• Audio:

• Smell:

15

Teleoperation -ElectromechanicalHow do we display each modality of information on master manipulator?

• Visual: 3D monitors, head mounted displays

• Tactile: force control

• Audio: speakers

• Smell: ???

16

Haptic devices• Generic master manipulators with force feedback

• Many shapes, sizes and degrees of freedom

17

Haptic teleoperation

Remote Operator

Force measured Force displayed Force perceivedContact force

Position change applied

Position change measured

Position change enacted

18

Haptic teleoperation

Courtesy of Intuitive Surgical19

Haptic teleoperation

Courtesy of Intuitive Surgical20

Haptic teleoperation

21

Haptic teleoperation

Remote Operator

Force measured Force displayed Force perceivedContact force

Position change applied

Position change measured

Position change enacted

22

Haptic rendering

Remote Operator

Force measured Force displayed Force perceivedContact force

Position change applied

Position change measured

Position change enacted

23

Haptic rendering

Components?

Remote

Force measuredContact force

Position change enacted

24

Haptic rendering

Components?

• Simulated remote manipulator dynamics and controller

• Simulated environment dynamics

• Interaction model between manipulator and environment

Remote

Force measuredContact force

Position change enacted

25

Haptic rendering

Components?

• Simulated remote manipulator dynamics and controller

• Simulated environment dynamics

• Interaction model between manipulator and environment

Remote

Force measuredContact force

Position change enacted

26

Interaction model representation

Some possibilities:

• Polygonal meshes

• Point clouds

• Voxels

27

Interaction model representation

Some possibilities:

• Polygonal meshes

• Point clouds

• Voxels

Easy to extract geometrical features such

as surface normals

28

Determining sensed contact force

Collision detection

Contact force resolution

Contact force aggregation

At each simulation time step:

Extract geometric

information

29

Determining sensed contact force

Collision detection

Extract geometric

informationContact force

resolutionContact force aggregation

Simplest case: contact forces are resolved at a discrete number of surface points

At each simulation time step:

30

Collision detection

Bounding sphere hierarchy

31

Single point contact force resolution: virtual proxy model

massless proxy object to represent manipulator end effector

co-located with haptic tool point in free space motion

32

Single point contact force resolution: virtual proxy model

massless proxy object to represent manipulator end effector

constrained such that no penetration occurs

Haptic contact force is computed as impedance from proxy object

33

Single point contact force resolution: virtual proxy model

34

Single point contact force resolution: haptic shading

35

Multi-point contact force resolution

36

Contact space

S

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

7

6

5

4

3

2

1

6

4

1

010000000010000000001

xxxxxxx

xxx

• Augmented Contact Space (X⊕)

• Full set of contact points

• Non Independent

• Active Contact Space (X)

• Subset of contact points

• impulse (p) or contact force (f) > 0

• Independent but unknown

• Active Contact Matrix : X = SX⊕

37

Contact mass properties• Augmented space inertial

coupling

• Active Contact Space inertial coupling

• where is the augmented space Jacobian

TJAJ ⊕−

⊕−⊕ =Λ 11

TSS 11 −⊕

− Λ=ΛTJAJ ⊕

−⊕

−⊕ =Λ 11

38

Contact mass properties• Active Contact Space

inertial coupling

• where is the augmented space Jacobian

• Contact forces are determined by enforcing non-penetration constraint

TSS 11 −⊕

− Λ=Λ

TJAJ ⊕−

⊕−⊕ =Λ 11

3940

41

Multi-point contact force resolution: virtual proxy model

Haptic contact force is computed as impedance from proxy object

42

Demo

43