4/24/2009 1 Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study with E ye Movement Analysis Arzu Çöltekin, Simone Garlandini, Benedikt Heil, Sara Irina Fabrikant D t t fG h Department ofGeography University of Zurich Switzerland This study is about.. Tag cloud produced based on a journal article (http://www.geo.unizh.ch/~arzu/publications/ColtekinA_et_al_CaGIS2009.pdf ) on this subject, using http://www.wordle.net/ 2009/04/22 Çöltekin, Garlandini, Heil, Fabrikant ‐ EyeTrackUX 2009
15
Embed
Evaluating the Effectiveness of Interactive Map Interface Designs
Tobii eye tracker used to Evaluate the Effectiveness of Interactive Map Interface Designs: A Case Study with Eye Movement Analysis Arzu Çöltekin, Simone Garlandini, Benedikt Heil, Sara Irina Fabrikant Department of Geography University of Zurich Switzerland
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
4/24/2009
1
Evaluating the Effectiveness of Interactive Map Interface Designs:A Case Study with Eye Movement y y
Analysis
Arzu Çöltekin, Simone Garlandini, Benedikt Heil, Sara Irina Fabrikant
D t t f G hDepartment of GeographyUniversity of Zurich
Switzerland
This study is about..Tag cloud produced based on a journal article (http://www.geo.unizh.ch/~arzu/publications/ColtekinA_et_al_CaGIS2009.pdf) on this subject,
using http://www.wordle.net/
2009/04/22Çöltekin, Garlandini, Heil, Fabrikant ‐
EyeTrackUX 2009
4/24/2009
2
Overview
Question: How to evaluate interactive map interfaces designs?interfaces designs?
‐ A range of issues can be identified based on classical usability engineering methods‐Maps, however, changed a lot
‐More interactive‐Many more ‘interface’ elements in/on/around the cartographic display
‐ Could eye movement recording and analysis give us more information?
Especially what involves self reporting cannot beEspecially what involves self‐reporting cannot be trusted: Cognitive load, performance anxiety, social pressure
• When eyes move; ‐ Fixations ‐ eyes fixed to a point 50 to 500ms.
‐ Linked to (overt) attention
Saccades rapid movements between fixations
Figure: Yarbus, 1967
‐ Saccades– rapid movements between fixations‐ Sequences analyzed for understanding visual search process
Path (1) corresponds to free exploration. Path (2) was obtained when subjects were asked to judge the material status of the family, and path (3) when they were asked to guess the age of different individuals. Partially reproduced from Yarbus, A. L. (1967)
. N=30, between subject. 15 participants per stimuli
. 3 tasks, independent variables, within subjectAll subjects solve all tasks. All subjects solve all tasks
. Systematic rotation to avoid learning effects
. Cognitive walkthrough (AOIs)
2009/04/22Çöltekin, Garlandini, Heil, Fabrikant ‐
EyeTrackUX 2009
4/24/2009
6
TasksTwo close‐ended, one open‐ended
1) What is the number of assaults in Washington County (Maine) in the year 2000?
2) Which county in the state of Oregon has the highest murder rate in the year 2000?
3) Looking at the map of the U S A overall do you see a3) Looking at the map of the U.S.A, overall, do you see a relationship (if any) between poverty rates and burglaries in the year 2000?
• Welcome, sign consent form• Background questionnaire• Background questionnaire• Training for locations of relevant counties/states• Participant comfortable and in the field of view of the tracker
• Calibrate• Record as tasks are delivered verbally and solvedRecord as tasks are delivered verbally and solved
– 5 minute limit for successful task completion• SUS delivery• Thank participant
ResultsOverall, confirming our hypothesis, participants are significantly more efficient (faster) using Carto.net’s interface, F=7.359, p=.011<.05, but significantly more effective (accurate) using Natlas, F=5.095 and p=.032<.05.
• One design is faster, other more accurate – why?
• These results indicate both designs have advantages – where?have advantages where?
Eye movements can tell us a few things..
. more fixations may indicate a less efficient search strategy,
. longer fixations may indicate difficulty with the display plotting scan paths and fixations will allow documenting what. plotting scan paths and fixations will allow documenting what people look at, how often and how long (Goldberg and Kotval, 1999; Dix et al., 2004; Bojko, 2006)
Identify and Redraw Map buttons on Natlasmouse roll‐over behavior that reflects on the legend‐bar in Carto.net.
• Natlas (slower)
– Discovery (time to first fixation): • IdentifyM=116.2 s, SD=102.1 (67% of task time), Redraw mapM=50.6 s, SD=30.6 (27% of task time)
– First use (time to first mouse click): IdentifyM=186 s, Redraw Map M=54 s• Labeling of Redraw Map was more quickly understood by the users in comparison with the Identify
– 423 fixations before Identify and 302 fixations before Redraw Map • 75% of the participants spent time on the Menu area
– Redraw map button salient, yet 30% of the users have not used this button• Both redraw and identify buttons are probably too small: Identify 0.08%, Redraw Map 0.13%
– 100% of the participants who needed assistance (5 of 15) had trouble with the Identify button
– Mouse roll‐over• 80% attempted clicking with the mouse
• 40% used the right mouse click to explore what other options may be ‘hidden’
– The legend• 80% needed assistance needed help with legend
• Double column misleads? 48% looks at the first column longer (observation length M=24.5 s, SD=23.5, fixation length M=20.7 s SD=19.9) than the second column where the information is more relevant (observation length M=15.7 s, SD=19.6, fixation length M=13.3 SD=17.0).
– Only 26.6% have discovered that the mini‐windows can be moved on Carto.net.
Comparatively:
• Isolating the query‐returns reveals itself as a favorable design choice leading to more accurate results.
• Map size: the map is 43% of the screen for Natlas where it is 72% of the screen for Carto.net. – Lesser use of zoom buttons: fixation counts on Zoom‐in 0.7 and Zoom‐out 1.6 (Natlas fixated more
Observations & Interview• Participants prefer digital over paper maps when possible
– Interactivity and responsiveness listed• 33% of Natlas viewers and 13% Carto net viewers tried to leave the site within the33% of Natlas viewers and 13% Carto.net viewers tried to leave the site within the
first 5 minutes.
“Although it should not necessarily be the case always, I have a feeling that a poorly designed GUI (graphical user interface) is also an indication of a poorly implemented system”.
“They [legend information] did NOT always mean what IThey [legend information] did NOT always mean what I assumed. I think people don’t like reading too much, and it would be best if the legends would match what most people assume by default.”
• Only 7% of our observed map users user the Help button