Page 1
Do You See What I See? The Effect of Gaze Tracking
on Task Space Remote Collaboration
KunalGupta1,GunA.Lee2,andMarkBillinghurst21HITLabNZ,UniversityofCanterbury
2EmpathicCompuIngLab,UniversityofSouthAustralia
ISMAR2016,September20th,2016
Page 2
Mo#va#on
• Improvingremoteassistanceofexpertuser(e.g.maintenance)• SupporIngrichcommunicaIoncues
Page 3
Task Space Teleconferencing
• Focusonsharingviewofremotetaskspace• Methods
• Handheldtabletswithcameras+ARcues,Fixedcamerasinworkspace• LimitaIons
• Fixedviewpoint,Difficulttoknowwhereuserlooking
Page 4
Head Worn Collabora#ve Systems
• Placecameraonhead+useheadmounteddisplay• HWC+HMD+remotepoinIngimprovescollaboraIon
• LimitaIons• Remoteviewfixed,Expertdoesn’tknowexactlywhereworkerlooking
Page 5
Gaze Tracking in Teleconferencing
• Monitorbased(Brennan2008),(Carle[a2010)• Gazeprovidesa[enIoncue,significantlyimprovedperformance
• Headmounted(Fussell2003),(Ou2005)–noHMD• Noperformanceimprovement,focusofa[enIoncanbepredicted
Page 6
Comparison to Previous Work
Rem=remotecollaboraIon,FtF=facetofacecollaboraIonGaze=eyetracking,HWC=headworncamera,HMD=headmounteddisplay
Page 7
Key Research Ques#ons
• Q1:Willsharingofgazeandpointercuesaffectthefeelingofco-presencebetweenusers?
• Q2:WillsharingofgazeandpointercuesimproveperformanceinaremotecollaboraIvetask?
Page 8
Prototype Design
• Combiningthefollowing(1)aheadmountedeye-tracker(2)headmountedcamera(3)headmounteddisplay(4)remoteviewingsohware
SystemDiagram
Page 9
Local Worker
BrotherAirScouter
MicrosohLifecamHD5000
LogitechWebcamc920
Page 10
Pupil Labs Eye Tracking
• Opensourceeye-tracking• UseIRreflecIonintoeye• ImageprocessingonPC• Trackseyeat30fps• Providesrawdata• www.pupil-labs.com
Page 11
Remote Expert Desktop
• Livecameraview• Gazeshownasreddot• Canaddpointercues
• Mouseinput• Greendot
• ShowninHMD
Page 12
User Experiment
• In remote expert collabora#on ...
• Does Pointer / Eye tracker cues have significant effect on co-presence?
• Does Pointer / Eye tracker cues have significant effect on task performance?
Page 13
Experimental Design
Eye tracker cue
Pointer cue No Yes
No NONE E
Yes P Both
Page 14
Experimental Design – Setup
Page 15
Experimental Design – Task
• Blockassembly• Fourdifferentstructures• 17piecesineach• Pilottestedtobalancedifficultylevel
• AssignedtocondiIonswithcounterbalancing• AcIveheadmovementencouragedthroughsecondarytask(Imer)andL-shapedesksetup
Page 16
Experimental Design – Procedure
• PracIcetrialinface-to-facecollaboraIon• ParIcipantsseparatedfortheexperimentaltrials• ForeachcondiIon:
• RemotehelpercreatesstructurebasedoninstrucIon• Performexperimentaltask• Answerper-condiIonquesIonnaire
• Post-experimentalquesIonnaire&debriefing
Page 17
Experimental Design – Par#cipants
• Within-subject• BalancedLaInsquaredesign• 30parIcipants(15pairs)recruited,26retained• 21-33yearsold,73%male• FluentEnglishspeaking• Noonehaddoneblockassemblyovervideoconferencingbefore
Page 18
Results - Summary • BoththePOINTERandEYETRACKINGvisualcueshelpedparIcipantstoperformthetasksignificantlyfaster.
• ThePOINTERcuesignificantlyimprovedbothlocalandremoteusers’perceivedqualityofcommunicaIon,collaboraIon,andco-presence.
• TheEYETRACKINGsignificantlyimprovedthecommunicaIonandcollaboraIonquality,andsenseofbeingfocusedforlocalworkers,andenjoymentforremotehelpers.
• TheBOTHcondiIonrankedasthebestinmostoftheaspectsofuserexperience,whiletheNONEcondiIonwasrankedastheworst.
• VisualcuesmadetheconversaIonmoreefficient,changedthechoiceofwordingindeicIcexpressions,andhelpedparIcipants’feelmoreconnected.
Page 19
Results – Task comple#on #me • Repeatedmeasuretwo-wayANOVA(α=0.05)
• POINTERcue• F(1,12)=4.908,p=.047*• 15%lessIme
• EYETRACKERcue• F(1,12)=5.811,p=.033*• 10%lessIme
• InteracIon• F(1,12)=0.566,p=.466
sec.
Page 20
Results – Per-condi#on ra#ng ques#onnaire • Q1 Ifeltconnectedwithmypartner.• Q2 IfeltIwaspresentwithmypartner.• Q3 Mypartnerwasabletosensemypresence.• Q4 Mypartner(orforRemoteHelper:I)couldtell
whenI(orforRemoteHelper:mypartner)neededassistance.• Q5 Ienjoyedtheexperience.• Q6 IwasabletofocusonthetaskacIvity.• Q7 Iamconfidentthatwecompletedthetaskcorrectly.• Q8 MypartnerandIworkedtogetherwell.• Q9 Iwasabletoexpressmyselfclearly.• Q10Iwasabletounderstandpartner’smessage.• Q11Informa9onfrompartnerwashelpful.
Adopted from [Kim et al. 2014]
Page 21
Results – Per-condi#on ra#ng ques#onnaire
• 7-pointLikertScale• 1:totallydisagree~7:totallyagree
• Internalconsistency:Cronbach’sα=.937• Aggregatedinto0~100scale
Page 22
Results – Per-condi#on ra#ng ques#onnaire
• AlignedRankTransform(ART)+RepeatedmeasureANOVA(α=0.05)
• POINTERcue• F(1,12)=7.414,p=.019*
• EYETRACKERcue• F(1,12)=26.822,p<.001*
• InteracIon• F(1,12)=2.023,p=.180
Page 23
Results – Per-condi#on ra#ng ques#onnaire
• AlignedRankTransform(ART)+RepeatedmeasureANOVA(α=0.05)
• POINTERcue• F(1,12)=11.914,p=.005*
• EYETRACKERcue• F(1,12)=15.929,p=.002*
• InteracIon• F(1,12)=5.157,p=.042*
Page 24
Results – Per-condi#on ra#ng ques#onnaire
• LocalWorkers
Page 25
Results – Per-condi#on ra#ng ques#onnaire
• RemoteHelpers
Page 26
Results – Ranking
Page 27
Results – Ranking
• Friedmantest(α=0.05)
• Localworker• E>NoneonC5
• Remotehelper• E>NoneonC2,4,5
1: the best ~ 4: the worst
Page 28
Results – Preference and qualita#ve feedback
• Understandingpartner• Localworkers:85%preferredcondiIonsincludingPOINTERcue“WithPointer,Icanrelatetowhatheistalkingabout,becauseIcouldunderstandhimmore.”
• Remotehelpers:70%preferredtheBOTHcondiIon“Theeyetrackerhelpsmetolookinthesameviewofmypartner.”
• Performingtaskefficiently• 77%ofLocal&85%ofRemoteuserspreferredtheBOTHcondiIon“TheeyetrackerwasgivingmypartnermoreinformaFonaboutwhereIlookedat,whilethepointerwasforgivingmetheinstrucFonfrommypartner.”
Page 29
Results – Behaviour observa#on
• Pointercuereducednumberofphrasessaid• LocalworkerF(1,11)=6.532,p=.027*• RemotehelperF(1,11)=8.479,p=.014*
• Referringtoobjects&direcIng
WithoutPointer WithPointer describefeatures
(colour,size,shape,...) “thisone”
“moveleG/right”,“infrontof” “putithere”,“nexttothis”
“that”object “this”object
Page 30
Discussion • Performanceimprovedbyusingeye-trackerandpointer
• Pointer–givingdirectguidance• Eye-tracker–showedworkersfocusofa[enIon
• Eyetrackerprovidedbenefitevenwithoutpointer• SameviewcondiIonforlocalworker,butimprovedcommunicaIonquality
• BenefitofgazeinformaIon• ImprovedcommunicaIon,moreenjoyable,focus,reduceinterrupIon
• Benefitofvirtualpointercue• EaseofdirecIon,increasedsenseofpresence,increasedpartnerawareness
• Differentuserneeds• Localworker–understanding/empathy–benefitsfromeye-tracking• Remoteexpert–givinginstrucIon–benefitsfromusingpointercue
Page 31
Implica#ons
1. Eye-trackingcanbeusedtochangethenatureofremotecollaboraIonwithheadwornsystems• MakeremoteuserawareofimplicitintenIons
2. ProvidinggazecuesalonecansignificantlyimprovetheremotecollaboraIonevenwithoutremotepoinIng• eye-trackingjustasbeneficialasusingremotepoinIngbyitself
3. CommunicaIoncueslikegazeandpoinIngplayaveryimportantroleincreaIngasenseofco-presenceanddeeperunderstanding• Mostoftheuserspreferredgaze+pointerduetoconnecIoncreated
Page 32
Limita#ons
1. PrototypetoobulkyandtetheredtoPC,notwearable• NewHMDswithintegratedcamerascouldovercomethis
2. TasklimitedcomparedtorealisIcremotecollaboraIon• DidhavekeyelementssuchasobjectidenIficaIon
3. Experimentalmeasuresnotsodetailed• Detailedbehavioralanalysis,conversaIonalanalysis
Page 33
Conclusions
• DoessharingeyetrackinginformaIonbetweenlocaluserandremoteexperthelpintermsofco-presenceandperformance?
• Usinggazeandpointera[enIoncuesimprovedperformanceIme.• Gazeandpointercuesimprovedthefeelingofco-presence
• Manyareasforfuturework• Exploreparalleltask–bothpeoplewithsameroles• ProvidesymmetriccommunicaIoncues–gazebothways• Useotherphysiologicalcues,GSR,heartrate,EEG
Page 34
Empathy Glasses (CHI 2016)
• Combinetogethereye-tracking,display,faceexpression• Implicitcues–eyegaze,faceexpression
++
PupilLabs EpsonBT-200 AffecIveWear
Masai,K.,Sugimoto,M.,Kunze,K.,&Billinghurst,M.(2016,May).EmpathyGlasses.InProceedingsofthe34thAnnualACMConferenceExtendedAbstractsonHumanFactorsinCompuFngSystems.ACM.
Page 35
Affec#veWear – Emo#on Glasses
• Photosensorstorecognizeexpression• UsercalibraIon• Machinelearning• Recognizing8faceexpressions
Page 36
Remote Collabora#on
• EyegazepointerandremotepoinIng• Faceexpressiondisplay• ImplicitcuesforremotecollaboraIon
Page 37
Contact Us
www.empathiccomputing.org
@marknb00
[email protected]
[email protected]