A fingertip haptic display for improving local perception of shape cues

A fingertip haptic display for improving local perception of shape cues

Massimiliano Solazzi Antonio Frisoli Fabio Salsedo Massimo BergamascoPERCRO, Scuola Superiore Sant’Anna

Via R.Piaggio 56025 Pontedera (PI) , Italy{m.solazzi,a.frisoli, f.salsedo, bergamasco}@sssup.it

Abstract

In this paper we present a novel haptic device capableof providing both kinesthetic and local haptic cues at thelevel of the fingerpad. The system is composed of a sup-porting haptic interface and a fingertip haptic display. Theaugmentation of locally displayed haptic information mayimprove the performance in tasks such as shape recognitionby haptic exploration. In this paper the overall device ispresented, and some preliminary experiments are reportedinvestigating the role of haptic local cues in haptic percep-tion of curvature.

1. Introduction

Haptic interaction allows to naturally perform both hap-tic exploration of shapes/surfaces and active manipulationof objects. Haptic perception by free exploration with handscan be accurate and a significant detriment in performanceis observed when only one finger is available for explorationof common objects [1]. The number of fingers has in fact agreat effect on the efficiency in identification of real objects,the largest difference appearing between the One finger andthe Two fingers conditions [2]. An experiment conductedby Jansson et al.[3] in shape identification of real objectsshowed that, attaching a hard sheath to the fingers in con-tact with the objects, in the sheath condition the explorationtime was the same in the One finger and in the Two fin-gers conditions. In [4], we found an experimental confir-mation of this hypothesis, since the haptic exploration ofhaptic shapes with a kinesthetic haptic device did not im-prove with the increase of contact points, from one to twofingers. This suggests that the restriction imposed on thefingerpad contact region can blunt the haptic perception ofshape and so indicates that local haptic cues play an im-portant role in haptic perception of shape. Factors that canaccount for the observed performance in these experimentsare lack of physical location of the contact on the finger-pad and lack of geometrical information on the orientation

of the contact area, that constitute interesting insights andsuggestions for the design of haptic displays.

Recently several new conceptual schemes have been pro-posed to solve some of the problems arising during the di-rect haptic exploration of virtual shapes, and in particularwe can identify two main approaches. The first approachhypothesizes [5] that the shape recognition is due eitherto the perception of slipping of the fingerpad over the ob-ject surface or to the displacement of the contact area overthe fingerpad. In [6] preliminary tests reveal that relativemotion can be used to render haptic sensation. In [5], anew haptic device is presented which integrates groundedpoint-force display with the presentation of contact locationover the fingerpad area. The second approach considers thatrecognition of shape is linked to the perception of the orien-tation of the object surface at the contact points. RecentlyHayward et al. [7] demonstrated how curvature discrimi-nation can be carried out through a device providing onlydirectional cues at the level of the fingerpad, without anykinesthetic information and moreover with a planar motionof the finger. This concept is also exploited to build ro-botic systems that can orient mobile surfaces on the tangentplanes to the virtual object that is simulated, only at the con-tact points with the finger [8].

Following this research direction, in this paper wepresent a novel device capable of providing simultaneouslyboth kinesthetic and local haptic cues at the level of the fin-gerpad, composed of a supporting haptic interface and a fin-gertip haptic display. We suppose that the augmentation oflocally displayed haptic information can improve the per-formance in tasks such as shape recognition by haptic ex-ploration. In the rest of this paper the overall device is pre-sented, and some preliminary experiments are reported in-vestigating the role of haptic local cues in haptic perceptionof curvature.

2. Structure of the system

The system is composed of a fingertip haptic interface[9] mounted on a kinesthetic haptic interface, according to

the overall configuration shown in figure 10. The latter hasthe aim to sustain the weight of the fingertip device and totrack its position.

Figure 1. The conceptual scheme of the de-vice

In figure 1 a conceptual scheme of the system is shown.Suppose the user is interacting with a virtual object: whenthe finger is out of the surface of the object, the plate of thefingertip haptic interface is kept far apart from the finger-pad. When the finger touches the virtual surface, the platecomes into contact with the fingerpad with an orientationdetermined by the geometric normal of the explored sur-face. Simultaneously a reaction force, proportional to thepenetration, is exerted by the supporting kinesthetic hapticinterface.

3. Supporting haptic interface

The supporting haptic interface is a pure translationalparallel manipulator with three degrees of freedom (DoF).The system was developed through a re-design process ofan existing haptic interface [10], which has led to a signif-icant improvement of performances in terms of exertableforces, gravity auto-compensation capability, backlash andconstructive simplification. The end-effector is connectedto the fixed base via three serial kinematic chains (legs),consisting of two links connected by an actuated revolutejoint and two universal joints at the leg end. Figure 2 showsa scheme of the device kinematics. The design of the de-vice was optimized in order to minimize the friction forcesand the inertia of the moving parts, obtaining the requiredtransparency of the mechanism during the haptic explo-ration. The actuation is realized by three brushed DC mo-tor through a steel cable transmission, characterized by lowfriction and zero backlash (figure 3). The transmission sys-tem realizes a speed reduction between the motor pulley andthe actuated joint and allows to mount the motors close tothe base, at the center of the first universal joint. In this waythe reflected inertia at the end-effector is reduced and theweight of the motors is supported by the base. The reach-able workspace which is formed by the intersection of threeanuluses centered in the middle of the base universal-joints(figure 2).

The motor positioning over the limb structure was cho-

Figure 2. Workspace of the 3dof haptic device

Figure 3. The configuration of the leg

sen among different mechanical solutions. In the final ver-sion the center of gravity of the actuation group is locatedon the first axle of the leg, which is fixed with respect tothe device base; so the weight of the actuation group iscompletely auto-compensated and the motor torques sup-port just the weight of the upper platform. This solutionallowed to improve the exertable forces on the end-effectorand increase the payload of the device.

4. Fingertip haptic interface

The fingertip haptic interface was devised to bring thefinal plate into contact with the fingertip with different ori-entations, according to the direction of the perpendicular tothe virtual surface in the point of contact [9]. Moreover, thecontact can occur at different points of the fingertip surface,depending on its orientation in respect of the virtual surface.These requirements can be satisfied by a kinematics withfive DoF, three translational and two rotational ones. The

most suitable solution resulted a hybrid kinematics, con-sisting of a first parallel translational stage (4) and a secondparallel rotational stage (5).

The translational stage has the same kinematics of thesupporting haptic interface, with 3-UPU legs. In each legthe cable connected to the motor and a compression springare mounted aligned to the centers of the universal joints.The spring works in opposition with the motor, in order togenerate the required actuation force and to guarantee a pre-load on the cable. The kinematics of the rotational stage isshown in figure 5; the axes of rotation X and Z ′ are fixedto the translational stage and allow a rotation of 90 and 180degrees respectively.

Figure 4. The kinematics of the translationalstage

Z’

Y

ZX

Y

X

Y

Z’

90°

Figure 5. The rotational stage of the device

The device is actuated by five DC motors, placed on afixed external support in order to reduce the mass and thebulk of the moving structure. This is allowed by a trans-mission system realized by steel cables guided with flexiblesheaths, that start from the actuation group and reach thedriven joints (figure 6). The overall device is shown in fig-ure 7.

Figure 6. The implementation of the leg forthe translational stage

Figure 7. Detail of the the fingertip haptic in-terface

5 Control architecture

The scheme in figure 8 represents the actuation system ofone leg of the fingerpad interface, composed of of one mo-tor, the sheathed tendon, the actuation cable and the returnspring for each leg.

The dynamic equations of the motor for each leg wereassumed as follows:{

τm − Tinr = Jmθ̈ + bmθ̇

Tout = r[Km(θ − θ0) + mθ̈] + Tg(1)

where Jm and bm are respectively the motor inertia anddamping constant, τm is the motor torque, Tin and Tout

represent the cable tension, τm the motor torque, Km thespring axial stiffness, while θ0 is the equivalent length atrest of the spring Km and Tg the requested term for thegravity compensation of the device.

The friction of the sheath was modeled according to thetheory of belts (β and f representing respectively the wind-ing angle of the tendon on the pulley and the friction co-efficient between the groove and the cable) plus a viscous

coefficient b. The relation between Tin and Tout was de-termined by the direction of motion, according to the driveeither by the motor or by the spring.{

Tin = Toutefβ + bθ̇ = Tout(1 + µ) + bθ̇ if θ̇ > 0

Tin = Tout

efβ + bθ̇ = Tout

(1+µ) + bθ̇ if θ̇ < 0(2)

Tin

shea th

r

Motor Jm bm

X

Tout

lo

Km

m

Figure 8. Scheme of the actuation system ofone leg

In order to compensate the friction generated by thesheath, a simple experimental apparatus was set-up to mea-sure the values of the friction coefficients between the steelcable and the sheath in different geometric configurations(for different curvature radii). The cable was fixed on oneside to a position-controlled DC motor and on the other oneto a weight of known mass. The motor was then moved ona trajectory with constant speed, in order to lift the weightat constant velocity.

The control was implemented with local position con-trollers at the joint level. An inverse kinematic modulewas used to convert the desired position expressed in carte-sian coordinates to the corresponding joint coordinates. Thenon-linear term due to the spring pre-load Kmθ0 and to thegravity term Tg was pre-compensated, by adding it in feed-forward to the motor torque τm in the control loop. Theplatform was controlled in order to maintain it at a give dis-tance from the finger, when there was no contact with thevirtual surface, and moving it in contact with the finger, dur-ing the contact phase. The orientation of the platform wascontrolled in order to be disposed tangentially to the virtualsurface.

For the supporting haptic interface a classic impedancehaptic control scheme was adopted, generating a force pro-portional to the penetration in the virtual surface.

Figure 9 reports the position and the interaction forces,evaluated by the currents supplied to the actuators, duringthe contact with a virtual cylinder of radius 70 mm. Thedashed blue line represents the position of the platform,while the red continuous line is the finger position, corre-sponding to the position of the supporting device. When thefinger is out of the cylinder (values greater than 70 mm), theplatform is moved far apart of a given offset from the fin-ger. When the finger comes in contact with cylinder (values

equal to 70 mm), the two positions coincide, meaning thatthe plate is in contact with the finger. The black continu-ous line represents a scaled representation of force (with anoffset only for the purpose of superimposing it to the plot)generated by the supporting haptic interface: the force isnull when the finger is out of the cylinder.

Figure 9. Experimental plot showing the re-sponse of the two devices

6. Can local information improve perception ofshape?

In order to assess the validity of the interface, a prelimi-nary evaluation of the new device was carried out accordingto classical psychophysics comparing with a kinesthetic de-vice. On the basis of the evidences already found by Hay-ward et al. [7] in the perception of curvatures with a simi-lar device, and due to the wide availability of experimentaldata on this topic, we decided to carry out an experimentrelated to discrimination of curvatures by means of hapticcues. Four participants were recruited for this experiment.All the participants voluntarily took part to the experimentand were students of the laboratory, all of male sex. Eachparticipant was informed about the procedure and did notpresent any dysfunction of the finger. They were experts onhaptic interfaces but novices on this device.

6.1 Methods and procedures

The method of constant stimuli was employed for thediscrimination of curvature. The participant was standingblindfolded in front of the device with a support for the el-bow, as shown in figure 10.

The test consisted in exploring in the virtual environmenta surface with curvature varying from 0 m−1 (plane surface)

Figure 10. Experimental set-up with superim-posed haptic cues displayed to the user.

to 6.67 m−1 (a sphere of radius equal to 150 mm). In fig-ure 11 it is reported a planar scheme of the displayed hapticcues, while their position in the space, relatively to the de-vice, is visible in figure 10.

Figure 11. Representation of the cues hapti-cally displayed to the user.

The adopted scheme was based on the constant stimuliprocedure for estimation of absolute threshold: eight dif-ferent stimuli were presented in random order several timesto the observers, that had to answer to the question ”is it acurved or a plane surface?”. The curvature values were 0,1.82, 2.22, 2.86, 3.33, 4, 5, 6.67 m−1, and 100 stimuli werepresented to each participant. The stimulus with a percent-age of 50% of affermative responses was considered as thethreshold for detection of curvature.

The test was carried out in two different modalities, Aand B: in condition A both the kinesthetic and the localgeometry haptic cues were provided to the observers, while

in condition B only the kinesthetic feedback was provided.In modality A, the mobile platform of the fingertip de-

vice was kept in contact with the fingerpad when the userwas in contact with the surface, with an orientation tan-gent to the displayed virtual surface. As a result the ob-server was perceiving in the contact point the indentationof the platform, oriented along the direction of the normalforce applied by kinesthetic device. In the second modal-ity the mobile platform was substituted by a fixed thimble,into which the user was required to insert its finger. In thiscase the only haptic cue applied to the fingerpad was theforce perpendicular to the virtual surface generated by thesupporting haptic interface; no local geometry informationwas provided. In both modalities the haptic exploration wascarried out in a restricted workspace, limited by a verticalcylinder with diameter of 25 mm.

6.2 Results and discussion

The data points were fitted with psychometric curves foreach of the three subjects. The average values of the pointwith the proportion of answers equal to 50% for the twoconditions A and B resulted respectively in 2.35± 0.35 and3.02 ± 0.58 m−1, with an improvement of 22.2% in theperformance, as reported in the box plot in figure 12. It is tobe expected that this value depends also on the size of theexploration workspace.

A B1.8

2

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

Curv

atu

re V

alu

es

(m −

1)

Figure 12. Box plot of performance in the twoconditions

The sigmoid curves for the two modalities are repre-sented in figure 13 for all the subjects, where the thresholdsare pointed out, resulted using the fingertip haptic device(red line) and with the only kinesthetic cues (blue line).

Although the ANOVA between the two conditions do notreach a significant level (p=0.069), we can however pre-sume that further investigations employing more accurateschemes and conditions, e.g. method of limits and TSD,and carried out on a larger sample of subjects/points, may

0 1 2 3 4 5 6 70

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Curvature (m−1

)

Pro

po

rtio

n o

f an

swer

s

local+kinesthetic cues

only kinesthetic cues

Figure 13. Sigmoid curves obtained in thetwo modalities

statistically point out this difference. In fact the main com-ments from subjects participating to the experiment was thatthe additional local cues make easier the identification ofcurvature.

The data are consistent with what has been already foundin literature for discrimination of curvature, either usinghaptic interfaces or real objects, such as [7, 11]. Furtherevaluation is needed to point out the ability of discriminat-ing curvature by means of this new device.

7. Conclusions

In this paper we have presented a novel concept of hap-tic interface, based on the combination of a fingerpad hapticinterface and a kinesthetic interface, capable of providinghaptic cues both at kinesthetic level and at the level of thefingerpad. We hypothesize that this configuration can en-hance the ability of humans of perceiving shapes in simu-lations of contact, and have provided some preliminary evi-dences in experiments for the detection of curvature. Futurework foresees an evolution of the presented device in orderto improve the position repeatability and reducing its size.

8. Acknowledgements

This work was carried out as part of the PRESENCCIAproject, an EU funded Integrated Project under the IST pro-gramme (Project Number 27731).

References

[1] G. Jansson. Effects of number of fingers involved in explo-ration on haptic identification of objects. excerpt from pure-form: The museum of pure form; haptic exploration for per-ception of the shape of virtual objects. Technical report, Up-psala University, 2000.

[2] Bergamasco M. Jansson, G. and Frisoli A. A new optionfor the visually impaired to experience 3d art at museums:Manual exploration of virtual copies. Visual Impairment Re-search, 5:1–12, 2003.

[3] G. Jansson and L. Monaci. Identification of real objects un-der conditions similar to those in haptic displays: providingspatially distributed information at the contact areas is moreimportant than increasing the number of areas. Virtual Real-ity, 9(4):243–249, 2006.

[4] A. Frisoli, S.L. Wu, E. Ruffaldi, and M. Bergamasco. Eval-uation of multipoint contact interfaces in haptic perceptionof shapes. In K. Salisbury F. Barbagli, D. Prattichizzo, ed-itor, Multi-point Interaction with Real and Virtual Objects,volume 18. Series: Springer Tracts in Advanced Robotics,2005.

[5] G. Niemeyer M. R. Cutkosky K. J. Kuchenbecker, W.R. Provancher. Haptic display of contact location. In Pro-ceedings of the Symposium on haptic interfaces for virtualenvironment and teleoperator systems, 2004.

[6] M. V. Lee P. M. Vishton M. A. Salada, J. E. Colgate. Vali-dating a novel approach to rendering fingertip contact sensa-tions. In Proceedings of the 10th IEEE Virtual Reality Hap-tics Symposium, pages 217–224, 2002.

[7] H. Dostmohamed and V. Hayward. Trajectory of contact re-gion on the fingerpad gives the illusion of haptic shape. Ex-perimental Brain Research, 164(3):387–394, 2005.

[8] Y. Sato T. Yoshikawa Y. Yokokohji, N. Muramori. Design-ing an encountered-type haptic display for multiple fingertipcontacts based on the observation of human grasping behav-ior. In Proceedings of the Symposium on haptic interfacesfor virtual environment and teleoperator systems, 2004.

[9] Marcheschi S. Salsedo F. Bergamasco M. Cini G., Frisoli A.A novel fingertip haptic device for display of local contactgeometry. In Proceedings of WorldHaptics, First Joint Eu-roHaptics Conference and IEEE Symposium on Haptic In-terfaces for Virtual Environments and Teleoperator Systems,pages 602–605, Pisa, 2005.

[10] A. Frisoli, E. Sotgiu, CA Avizzano, D. Checcacci, andM. Bergamasco. Force-based impedance control of a hapticmaster system for teleoperation . Sensor Review, 24(1):42–50, 2004.

[11] BJ van der Horst and AM Kappers. Curvature discriminationin various finger conditions. Exp Brain Res, 2006.