element54 - using Eye Tracking for Survey Usability

Copyright 2010 element54 Inc. All rights reserved. Permission to copy and distribute this document is granted subject to the following: The materials provided herein are for internal, non-commercial use only, and any copies or portions thereof must include the copyright notice, and reference the author, element54.

bernie.malinoff@element‐54.com

Using Eye‐Tracking to Advance Survey Research 

Quick History of 'Eye Tracking' 

In1879inParis,LouisEmileJavalobservedthatreadingdoesnotinvolveasmoothsweepingoftheeyesalongthetext(as

previouslyassumed),butaseriesofshortstopsandquicksimultaneousmovements.

Measuring Attention & Engagement 

• Visualgaze;singlelocaPon.FixaPons

• simultaneousmovementsofbotheyesinthesamedirecPon.Saccades

Eye Tracking 

Eye‐Tracking  Applications 

Major R&D Initiatives in 2009 

Jan ‘09 April/May ’09

Data Consistency Respondent Usability

Origin of online research: ‘Radio Button’ question types 

Typical respondent experience 

ResponsePaTerns/Flatlining"Engagement

&Enjoyment"

OnscreenLayoutRespondent

ExpectaPons

Divergence/Variety of New Question Types 

Sources: Spyglass Solutions/2x4 Research & Consulting 

A Brief History of how we got here 

Door‐to‐Door

Mail CATI/Phone

Online

MethodologyEvoluPon….?

"1.0 vs. 2.0" Using Eye Tracking for Survey Usability 

... to better understand the destination, by examining the journey.

Drivers of Cooperation 

RespondentCooperaPon

SurveyInvite

QnreLength/IncenPve

TopicInterest

Panel/Client

ReputaPon

VisualAppeal

Eyes Don’t Lie ‐ Research Protocol 

EyeTracking

Recording

Interviewing

Eye Tracking ‐ 'Areas of Interest' 

Eyes Don’t Lie ‐ Initial Findings 

Inference vs. Instruction

Don’t interrupt me when I’m thinking!

Death by Ranking

EyesDon'tLie‐UnderstandingFixaPons

Error Prevention & Recovery

EngagementMetrics

Inference vs. Instruction 

Inference vs. Instruction 42%didnotreadthequesPon

33%didnotreadthe"tradiPonal"

quesPon

Don’t Interrupt Me When I’m Thinking 

Interpreting 'Fixations' 

Fixation #1 – “Latent” 

Fixation #2 – “Orientation” 

Fixation #3 ‐ “Cognition” 

37/49 Respondents Click Inside the Cognition Zone  

Fixations … & Social Desirability? 

Error Prevention & Recovery  

Error Prevention & Recovery  

Equal Ratings for ‘Engagement’ Metrics 

1.08.20

8.86

7.02

2.08.25

8.85

6.56

SurveyEnjoyment

EaseofUse

DesiretoRepeat

Eyes Don’t Lie ‐ Implications 

1.  Wemustreturntotheusabilityprincipleswhichguidedpapersurveydesign,notCATIdesign,tomaximizeconfidenceinourdata,andbusinessdecisions.

2.  TherearewaystomakeallquesPons/surveysmoreappealing,intuiPveandfuture‐relevant,buttheseimprovementsmustnotintroducenewformsofbiasandshouldbetestedbeforebeingfieldedforthefirstPme.

3.  Asanindustry,wemustdeviseandreinforcebestpracPcesforonlineresearchinterfaces,likewehaveforsomanyelementsoftheresearchprocess.

Eyes Don’t Lie ‐ Implications 

4.  Responsible Innova0oniskeytosuccessfulevoluPonofweb‐basedinterfaces.

•  Decidewhohastheresponsibilityforrespondentusability,andprovidethemwiththeappropriateskillset.

•  Involvesurveytakersinthedesignprocess.

5.  Simplythinkingabouttheseissues,andmakingsmallchangeswillensurehealthier,longer‐termOnlineresearch.

“The eye sees only what the mind is 

prepared to comprehend” 

~ Henri Bergson 

Thank you 

Copyright 2010 distribute this document is granted subject to the following: The materials provided herein are for internal, non-commercial use only, and any copies or portions thereof must include the copyright notice, and reference the author, element54.

bernie.malinoff@element‐54.com