Introduction: Eye- Movements and Eye- Tracking Irina A. Sekerina (Higher School of Economics and CUNY) Workshop on Reading in Cyrillic ● 21 September,

Introduction: Eye-Movements and Eye-TrackingIrina A. Sekerina (Higher School of Economics and CUNY)

Workshop on Reading in Cyrillic ● 21 September, 2015

1

Roadmap

A. Introduction: History, Terminology, Background

B. Eye-Tracking ApplicationsC. Techniques and EquipmentD. Eye-Tracking in

Psycholinguistics: ReadingE. Models of Reading

2

A. Introduction: History, Terminology,Background

A1. Some recent historyA2. Scanpaths and visual attention

A3. The eyeA4. Types of eye movements

3

A1: Some [Recent] History

Alfred Luk’yanovich Yarbus (1965)*:

demonstrated sequential, but variable, viewing patterns over particular image regions

Noton and Stark (1971)**: showed that participants tend to fixate identifiable regions of interest, containing “informative details”;

coined term “scanpath” describing eye movement patterns

4

1914-1986

* Ярбус, A. Л. (1965). Роль движений глаз в процессе зрения. Наука. Yarbus, A. L. (1967). Eye Movements and Vision. New York: Plenum Press.

** Noton, D., & Stark, L. (1971). Scanpaths in eye movements during pattern perception. Science, 171(3968), 308-311.

“Не ждали [Unexpected]” (by Ilya Repin, 1884)

5

A2: Scanpaths and Visual Attention

Yarbus’ early scanpath recording:

1: examine at will 2: estimate wealth 3: estimate ages 4: guess previous activity

5: remember clothing

6: remember position

7: time since last visit

6

A2: Scanpaths of a portrait

7

A3: The Eye

The eye—“the world’s worst camera”

suffers from numerous optical imperfections...

...endowed with several compensatory mechanisms

8

А3: Human visual field

The retina is a light sensitive structure inside of the eye responsible for transforming light into signals, which are later converted into an image by the visual cortex in the brain. 

The fovea is a section of the retina that contains a high density of both kinds of light receptor cells found in the eye, i.e. Cone and Rod cells. 9

A4: Types of eye movements

Eye movements are mainly used to reposition the fovea

Five main classes of eye movements:

Saccadic (saccades and fixations) Smooth pursuit Vergence Vestibular Physiological nystagmus

Other types of movements are non-positional (adaptation, accommodation)

10

A4: Saccades

Rapid eye movements between fixations used to reposition fovea

Voluntary and reflexiveRange in duration from 10 ms – 100 ms

Effectively blind during transitionDeemed ballistic (pre-programmed) and stereotyped (reproducible)

11

A4: Fixations

Possibly the most important type of eye movement for attentional applications

90% viewing time is devoted to fixations

duration: 150 ms – 600 ms

12

Duchowski, A. T. (2007).Eye-Tracking Methodology.2nd Ed. 360 p. ISBN: 978-1-84628-808-7

B. Applications

B1. Ergonomics and Human Factors

B2. Marketing and AdvertisingB3. WebsitesB4. Psychology, Psychophysics, Neuroscience

13

B. Applications

Wide variety of eye tracking applications exist, each class increasingly relying on advanced graphical techniques:

AdvertisingHuman FactorsDisplaysHCI & Collaborative SystemsVirtual RealityPsychology, Psychophysics, Neuroscience

14

B2: Ergonomics and Human Factors

Applications range from usability studies to testing effectiveness of cockpit displays

Examples: evaluation of tool icon groupings comparison of gaze-based and mouse interaction

organization of click-down menus testing electronic layout of pilots’ visual flight rules

testing simulators for training effectiveness

15

B3: Marketing and Advertising

16

Applications range from assessing ad effectiveness (copy testing) in various media (print, images, video, etc.) to disclosure research (visibility of fine print)

Examples: eye movements over print media (e.g., magazines)

eye movements over TV ads, web pages, etc.

B3: Marketing and Advertising

17

Scanpaths over printed magazine ads

18

B3: Websites

Most people view websites in a “F” shaped flow.

First they scan the page at the top, from left to right.

Then the eyes go back to the left and down the page.

They again scan to the right and back along the same pattern.

B4: Psychology, Psycho-physics, and Neuroscience

19

Applications range from basic research in vision science to investigation of visual exploration in aesthetics (e.g., perception of art).

Examples: psychophysics: spatial acuity, contrast, sensitivity

perception: reading, natural scenery, ...

neuroscience: cognitive load, with fMRI an ERP

psycholinguistics

B4: Perception of Art (Rembrandt’s Anatomy Lesson) (Duchowski, 2007)

20

small but visible differences in scanpaths similar sets of fixated image features

(a) Aesthetic group (b) Semantic group

C. Techniques and Equipment

C1. Electro-oculographyC2. Scleral contact lens/search coil

C3. Video-based combined pupil and corneal reflection

21

C. Techniques and Equipment

Two broad applications of eye movement monitoring/recording techniques:

measuring position of eye relative to the head

measuring orientation of eye in space, or the “point of regard” (POR)—used to identify fixated elements in a visual scene

The most widely used apparatus for measuring the POR is the video-based corneal reflection eye-tracker

22

C1: Techniques

23

First method for objective eye movement measurements using corneal reflection reported in 1901

Techniques using contact lenses to improve accuracy developed in 1950s (invasive)

Remote (non-invasive) trackers rely on visible features of the eye (e.g., pupil)

Fast image processing techniques have facilitated real-time video-based systems

C1: Electro-оculography Relies on

measurement of skin’s potential differences, using electrodes placed around the eye

• most widely used method some 30 years ago (still used today)

• similar to electro-mechanical motion-capture

• measures eye movements relative to head position

• not generally suitable for POR measurement (unless head is also tracked)

24

C2: Scleral Contact Lens/ Search Coil

search coil embedded in contact lens and electromagnetic field frames

• possibly most precise

• similar to electro-magnetic position/ orientation trackers used in motion-capture

25

C2: Scleral Contact Lens/Search Coil

26

Example of scleral suction ring insertion:

most intrusive method

insertion of lens requires care

wearing of lens causes discomfort

• highly accurate, but limited measurement range (~5°)

• measures eye movements relative to head position

• not generally suitable for POR measurement (unless head is also tracked)

C3: Video-Based Combined Pupil/ Corneal Reflection

H

Head-mounted video-based eye tracker

• most suitable for (graphical) interactive systems, e.g., VR

• Monocular and binocular systems

27

C3: Existing Eye-Trackers

SMI

TOBII

ISCAN

Remote system

ISCAN child head-mounted

ASL head-mounted

28

C3: Existing Eye-Trackers

1997 “Kindergarten-Path” article

29

2013 next to a student’s poster

C3: POR Method

A light source is used to cause reflection patterns on the cornea and pupil of the test person.

A camera will then be used to capture an image of the eye.

The direction of the gaze is then calculated using the angles and distances.

30

C3: How it Works

31

D. Eye-Tracking in Psycho-linguistics: Reading

D1. Visual World Eye-Tracking Paradigm

D2. Dual-Purkinjie Eye-Tracking in Reading

D3. Examples of Eye Movements in Reading (Tobii)

D4. Eye-Movement Parameters in Reading

32

D1: The Visual World Eye-Tracking Paradigm

33

Cooper (1974)* and Tanenhaus et al. (1995)**

The Mind-Eye hypothesis Relationship between eye fixations and the meaning of concurrently spoken sentence

Using this relationship as a research tool in cognitive psychology and psycholinguistics

Applications: Speech perception and memory Language processing

*Cooper, R. (1974). The control of eye fixation by the meaning of spoken language. Cognitive Psychology, 6, 84-107.

**Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in

spoken language comprehension. Science, 268(5217), 1632-34.Федорова, О. В. (2008). Методика регистрации движений глаз

Визуальный мир: шанс для сближения психолингвистических традиций. Вопросы языкознания, 6, 98–120.

34

D2: Dual-Purkinjie Eye-Trackers for Reading

Dual-Purkinje image (DPI) eye tracker

so-called generation-V trackers measure the 1st and 4th Purkinje images

DPI trackers measure rotational and translational eye movements

1st and 4th reflections move together through same distance upon eye translation, but separate upon eye rotation

highly precise used to be expensive and difficult to set up

35

D2: Pupil and Purkinje Images

Purkinje images appear as small white dots in close proximity to the (dark) pupil

tracker calibration is achieved by measuring user gazing at properly positioned grid points (usually 5 or 9)

tracker interpolates POR on perpendicular screen in front of user

Pupil and Purkinje images as seen by eye tracker’s camera

D3: Tobii Examples of Eye Movements in Reading

A gaze replay, recorded at 300Hz using the Tobii TX300 eye tracker, of a participant in a reading study:

https://www.youtube.com/watch?v=VBTZNydUh0w

Eye-tracking with Tobii (4-year-old child)

http://www.youtube.com/watch?v=VFIZDZwdf-0

36

D4. Eye- Movement Parameters

1. Studying eye movements per se to learn about reading

2. Using eye movements in reading as a means to infer cognitive processes (e.g., language processing)

37

38

D4. Eye-Movement Parameters

1. Saccade latency: 150-175 ms;2. Perceptual span: ~4 symbols to

the left, ~15 symbols to the right;

3. Skipping words: 2-3-letter words are skipped 75%, 8-letter words are never skipped;

4. Regressions: 10-letter spaces happen because of problems in understanding text

39

D4. Eye Movements and Word Recognition in Reading

1. Word frequency2. Word familiarity3. Age of acquisition4. Number of meanings5. Morphology6. Predictability7. Plausibility

40

E. Models of Reading

E1. Types of Models of ReadingE2. E-Z ModelE3. SWIFT

41

E. Models of Reading

Models of reading behavior try to explain how the eye movement control system makes two fundamental decisions involved in reading:

When the eyes should move Where the gaze should land

42

E1. Types of Models of Reading

1. Oculomotor control models2. Cognitive control models

1. E-Z Reader2. SWIFT

43

E1: E-Z Reader Model

E-Z Reader model

http://raynerlab.ucsd.edu/Keith_Rayner.html

Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85, 618-660.

Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372-422

Keith Rayner (UCSD)1943-2015

44

E2. E-Z Reader

1. 50 ms: visual uptake phase2. 75-100 ms: L1 phase of lexical

access3. Saccade planning4. L2 phase of lexical accessAccounts for frequency, predictability, spillover effects, covert attention, importance of parafoveal preview

45

E3: SWIFT

Reinhold Kliegl (University of Potsdam, Germany): SWIFT Model

http://www.psych.uni-potsdam.de/people/kliegl/

46

Reinhold Kliegl’s Lecture 1

11:30am-12:30pm“Distributed Processing During Fixation Durations in Reading”