A Novel Methodology for High Accuracy Fixational Eye ...ipcbee.com/vol29/24-ICBBT2012-H041.pdf · A Novel Methodology for High Accuracy Fixational Eye Movements Detection Xiaolin

A Novel Methodology for High Accuracy Fixational Eye Movements Detection

Xiaolin Zhang1+ and Jiamao Li 2 1Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuta, Midori-ku,

Yokohama 226-8503 JAPAN 2 Department of Information Processing, Tokyo Institute of Technology, 4259 Nagatsuta, Midori-ku,

Yokohama 226-8503 JAPAN

Abstract. Our eyes perform micro-movements when they gaze at a fixed target. Scientists have examined fixational eye movements for more than fifty years, and the importance of fixational eye movements has been experimentally demonstrated. However, fixational eye movements have not been explained clearly because of the technical problems associated with experimentation. To measure these eye movements, we developed a new eye movement detection device that offers sufficient accuracy. This device can detect rotation in three degrees of freedom and is also non-invasive. We discovered several new types of fixational eye movement by using the device. These findings require a review of several conventional opinions about fixational eye movements. The first new finding of our study is the occurrence of fixational eye movements associated with rotation around the optical axis. In our experiments, the microsaccades of these fixational eye movements around the optical axis of one eyeball are always in the same direction. The second new type of fixational eye movements that we observed is an impulse-like movement behaviour that occurs generally in the horizontal direction. Our other new findings include the following: most microsaccade tracks are not straight lines, and microsaccades always occur simultaneously in both eyes.

Keywords: fixational eye movements, microsaccade, drift, tremor, sclera images, POC

1. Introduction Our eyes need to move dynamically to project an image onto our retinas during fixation to prevent neural

adaptation caused by an unchanging environment. Without this involuntary eye movement, our visual perception would fade completely. In other words, we would be prevented from seeing a stationary object for an extended period. Three different types of eye motion can be detected when we gaze at an object: tremor, drift, and microsaccades (Fig. 1.a). The occurrence of several coincident types of movement demonstrates the complexity of the visual system and indicates the need for intensive research. To analyse these small movements, a measurement technique needs to be able to detect them in three rotational degrees of freedom simultaneously with an accuracy of 0.0001deg. Moreover, the methodology should be minimally invasive to the experimental subject. We have developed a new device in our laboratory in response to these demanding requirements. Here, we use our high-accuracy device to demonstrate several new types of fixational eye movement. These results indicate the need for a review of the conventional thinking about fixational eye movements.

The first new movement behaviour identified has the form of an impulse and is associated primarily with horizontal movements. The eye rotates forwards and backwards through 0.2 deg in approximately 0.03 sec. We have named this type of motion a ‘pulse lap’. Its features differ from those of a fast-moving microsaccade. Partial pulse laps occur at the beginning and the end of a microsaccade.

+ Corresponding author. Tel.: + 81-45-924-5083; fax: +81-45-924-5083. E-mail address: [email protected].

2012 4th International Conference on Bioinformatics and Biomedical Technology IPCBEE vol.29 (2012) © (2012) IACSIT Press, Singapore

133

In the second new movement behaviour, rotation around the optical axis (roll) has its own fixational movements. In our experiments, the microsaccades of rotation around the optical axis of one eyeball of the subject are always in the same direction.

In addition, several conventional perspectives on fixational eye movements need to be modified. Previously, microsaccades were assumed to occur only in straight lines. Our experiments show that microsaccades are fast and jerky and that their movements are not always linear, as previously assumed (Fig. 1). Moreover, all microsaccades were found to occur simultaneously in both eyes.

Fig. 1: A comparison of previous representations of fixational eye movements with our experimental results.

a. Sketch of three types of fixational eye movement (Pritchard, R.M. Stabilised images on the retina. Sci. Am. 204, 72–78 (1961)). The crooked line is the track line of a gazing point projected on the retina. The circular area is located on the

fovea of the retina and has a radius of approximately 25 µm. b. Actual experimental result using our method. The colour shows the velocity of the eye movement.

Because fixational eye movements occur as part of our visual process, they occur unconsciously. The

results of this paper may offer benefits for image recognition, medical studies, and human psychology. We have recently used the device to detect the relationship between fixational eye movements and mental states, such as indignation, psychological stress, and deep thought, and we have already obtained some interesting results. Scientists have been aware of fixational eye movements for many years. These movements occur as continuous movements of the eye when we fix our gaze on an object. Today, scientists agree that three main types of eye movement occur during visual fixation in humans: tremor, drift, and microsaccades. According to previous experimental results [1] (Fig.1.a), tremor is an aperiodic, wave-like motion of the eyes with a frequency of ~ 90 Hz, and its amplitude is approximately equal to the diameter of a cone in the fovea[1]. Drift, which is normally superimposed on tremor, is a slow movement with a mean speed of 6 min/sec and a maximum speed of 30 min/sec. The drift frequency is 95%–97% of the fixation time. Its duration is 0.3–0.8 sec [6], and it follows a curved path. The third type of eye movement is the microsaccade. These movements have an amplitude of 5–120 min (monocular) or 6–120 min (binocular) and are rapid, with speeds of approximately 6–120 deg/sec (monocular) or 10–120 deg/sec (binocular) [7]. Their motion is jerky.

Although many methods of detecting these features are available, the classical methodologies are unable to directly capture the smallest eye motions because of their significant technological limitations. The accuracy of devices such as an EOG (Electro-oculogram) [2] or a head-mounted camera [3] is not sufficient to detect the quality of fixational eye movements. Accordingly, Fig.1.a is a schematic diagram rather than an experimental result.

To clarify the properties of fixational eye movements, we invented a new device that enhances our ability to observe these small motions. The effectiveness of our new measurement system can be evaluated in the light of the results shown in Fig.1.b. This system can accurately define the characteristics of fixational eye movements. As shown in Fig.1.b, the drift on which tremor is superimposed does not move exclusively in curves. Instead, such movement can take the form of a curled line that is produced by the high-frequency tremor superimposed on the drift. The figure also explicitly shows that microsaccades do not always follow

134

straight lines. These results differ substantially from past views of fixational movement, shown in Fig.1.a, which suggest straight-line movements for microsaccades and curved-line motions for drift. Moreover, some new aspects of fixational eye movements have been discovered using our system.

2. Method Our system was established to measure the three degrees of freedom of eye rotation, as shown in Fig. 2.

Fig. 2: Definitions of the planes of eye movement.

The three-dimensional plane is defined relative to the eyeball, with its origin at the eyeball’s centre. The X-Y plane is on a horizontal surface with the X-axis directed outwards from the eye, and the Y-axis and Z-axis are determined by the rule of right angles. Roll-α, Pitch-β, and Yaw-γ represent rotation around the X-, Y-, and Z-axes, respectively. Two cameras (a left-side camera and a right-side camera) were positioned at an angle ϕ from the X-axis, at the left and right side of the eyeball, and set to capture images of the surface of the sclera. The XL-YL plane was defined as the image plane of the left-side camera. The right-side camera defined the XR–YR plane. The image plane was parallel and in contact with the eyeball’s surface such that its normal vector extended perpendicularly from the surface of the eyeball. The X- and Y-axes of the image plane followed the right-angle rule relative to their corresponding plane normal vectors.

a. The device developed in our laboratory b. Single-eye detection

c. The whole system (from above)

135

Fig. 3: Device for detecting fixational eye movements

The system provides accurate results through an image-processing algorithm applied to high-resolution photos of the surface of the sclera. Two high-speed cameras with telecentric lenses were stationed at an identical specified angle from the optical axis of the eye on opposite sides, as illustrated in Figure 3.

The displacement of the captured eye images was then recorded using an image-processing algorithm, POC (Phase-Only Correlation) [4], which can perform these measurements in sub-pixels. These high-accuracy image displacement values were then related to the movement of the eyeball in three rotational degrees of freedom. These relationships can be described by mathematical equations.

In our method, two eye images were obtained to define the displacement values at both sides of an eyeball on the scleral surface. Because the images are in two-dimensional spaces, we can define them in the horizontal and vertical directions. These motions from both images are then summed and subtracted in terms of each direction, so that three equations are obtained for the three-DOF eyeball rotations.

[ ] [ ] 180[ ]sin 2

y l rk y t y ttr ϕ π

−= ⋅ ⋅α (1)

[ ] [ ] 180[ ]cos 2

y l rk y t y ttr ϕ π

+= ⋅ ⋅β (2)

[ ] [ ] 180[ ]2

x l rk x t x ttr π

+= ⋅ ⋅γ (3)

Where, kx and ky are the parameters for translating a distant point (pixel) on the camera image to a distant imaging area (mm). Because telecentric lenses are used, kx and ky are constant. Parameter r is the radius of the eyeball. Because the value of this parameter is difficult to measure for individual subjects, we have used standard data for the human eye.

mmmmxx

p

ix

31064.1512

842.0 −×===κ

mmmmyy

p

iy

31065.1480

793.0 −×===κ

r = 12 mm Thus, using two cameras gives our method the advantage that the system can track fixational eye

movements with three degrees of freedom.

3. Experimental results

3.1. Experimental procedure The experimental procedure is as follow:

The subject’s head was placed in the station so that the head was securely anchored when the subject was gazing at an object placed in front of the subject at a distance of 1 m. (Fig.3.a)

Two high-speed cameras with telecentric power lenses were installed on the station table to the left and right side of the eyeball at identical angles of 45˚ from the optical axis to capture the scleral surface of the eyeball. (Fig.3.c) An image-space telecentric lens produces images of the same size, regardless of the distance between the lens and the image sensor. A telecentric lens suffers no parallax distortion and no magnification error; thus, the eye image can be clearly captured regardless of its position in the field of view. The system is therefore less complicated to set up and calibrate.

Phase-Only Correlation (POC), an image-processing algorithm, was then applied to find the displacement of the captured movement of the eye image. This algorithm enhanced our ability to detect image displacement with 1/100-pixel accuracy. The movement of the eyeball was then calculated using the relative equations for three degrees of freedom.

136

3.2. Three degrees of freedom of fixational eye movements

Fig. 4: the experimental results of three degrees of freedom of fixational eye movements. The figure shows horizontal

movement (deg), vertical movement (deg), and rotation around optical axis (deg) versus time (sec), respectively.

As shown in Fig. 4, the experimental results reveal these tiny motions. On the horizontal and vertical axes, drift can be interpreted as slow movement, and the rapid, stepped movement values can be identified as microsaccades. Moreover, the rotation of the eyeball around the optical axis is now detectable.

3.3. Microsaccade Closer study of microsaccades is feasible. By assuming that they are fast-moving, similar to a brief jerk,

we can distinguish the microsaccades from drift and tremor based on their velocity characteristics. As shown in Fig. 1b, their movement explicitly appears as a non-straight line, which is a contradiction of previous assumptions. Microsaccades may change direction during high-speed movement and stop in any undefined position. Thus, their direction is unpredictable. They can be distinguished from drift and tremor by their high velocity. We can use an algorithm for detection of microsaccades defined by R. Engbert [5] to extract the microsaccades.

137

In our studies, a significant type of motion illustrates a form of movement similar to an impulse. This movement departs from and returns to almost the same position, as shown in Figure 5. This feature is different from microsaccade motion, which moves across a wide range from one position to a new position. This feature has not been previously observed. Further study should be conducted to gain more understanding of this phenomenon. We named this feature the “pulse lap” type of motion. However, it occurs only in horizontal movements and is sometimes superimposed on microsaccade motion. As a result, it does not return to the same position. In summary, these results are markedly different from the previously assumed straight-line motion of a microsaccade.

Fig. 5: Microsaccade displacements are similar to an impulse.

a, Horizontal displacement (deg) of the eyeball versus time (sec). b, Close view of image a for a microsaccade that was similar to an impulse: displacement (deg) versus time (sec).

In addition to movement behaviour, the experiment also showed that microsaccades always occur in both

eyes at the same time, especially in the horizontal direction, as shown in Figure 6.

Fig. 6: Microsaccade displacements of the left and right eye show that microsaccades always occur together. Eye

movement velocity versus time: the blue line represents the horizontal movement velocity, whereas the red line shows 138

the vertical movement velocity. High velocity and amplitude, similar to a pulse, can be interpreted as microsaccades occurring on the eyeball. The dashed lines are coincident times to allow comparison of the occurrence of a

microsaccade at the same time in the left eye and right eye.

Our experiments also revealed microsaccades that rotated around the optical axis. We studied the directions of these microsaccades. Figure 7 shows rotation around the left eye of subject JL, whose data were sampled on different days. We found that JL’s microsaccades of rotation around the optical axis of the left eye were in the same direction. We confirmed this characteristic in all of the subjects’ data. Although the relationship between the directions of both eyes varies considerably between subjects, the microsaccades of rotation around the optical axis of one eyeball of a subject are always in the same direction.

Fig. 7: Data for rotation around the optical axis of one subject on different days.

a.) Jan. 15, 2010; b.) Sept. 24, 2009; c.) Sept. 23, 2009

4. Conclusion To measure these eye movements, we developed a new eye movement detection device that offers

sufficient accuracy. This device can detect rotation in three degrees of freedom and is also non-invasive.

139

Based on our experiment results, we can confidence that not all microsaccades occur in straight lines, as previously believed. In contrast, if a microsaccade does demonstrate straight movement, the line is always horizontal. Significantly, this phenomenon always occurs simultaneously in both eyes. This new system makes it possible to describe fixational eye movements in three degrees of freedom instead of two. The newly detected motion represents rotation around the optical axis. The direction of the microsaccades that show rotation around the optical axis of one eyeball is always the same. Further results also show the presence of new motion types that move similarly to an impulse and differently from a microsaccade. We have classified this type of movement as “pulse lap” motion.

5. Acknowledgments This work was partly supported by Panasonic. We also thank Ms. Kiyomi SAKAMOTO, Mr. Shoichi

AOYAMA, and Mr. Shigeo ASAHARA for their support and discussion.

6. Appendix The following table shown the specifications of the fixational eye movement detecting device.

Camera Manufacturer Photoron Type FASTCAM-PCI R2 Scanning area 7:4 µm (H) × 7:4 µm (V) Resolution 512 × 480 pix Size of Sensor 842 µm × 793 µm Speed 30 ~ 10000 FPS

Lens

Manufacturer VS Technology Type VSZ-M07545 Magnification 0.75 ~ 4.5 x Distortion 0.03% (Telecentric) W.D. 95.25 mm Focus content ± 3 mm

Illumination Manufacturer Sumita Optical Glass Type LS-M2105

Laser Manufacturer Sakaki Corporation Type Z3A-635-lg90

7. References [1] Susana Martinez-Conde, Stephen L. Macknik and David H. Hubel, “The Role of Fixational Eye Movements in

Visual Perception,” Nature Reviews. Neuroscience 2004, vol. 5, no3, pp. 229-240, Nature Publishing Group, London

[2] Tursky, B., “Recording of Human Eye Movements,” In Bioelectric Recording Techniques Part C, Academic Press, New York, 1974

[3] Sakashita Yusuke, Fujiyoshi Hironobu, Hirata Yutaka. Measurement of 3D Eye Movements Using Image Processing, Vol.6, No.3, Pages 236-243 (2006)

[4] Takita, K., Aoki, T., Sasaki, Y., Higuchi, T., Kobayashi, K. High-accuracy subpixel image registration based on phase-only correlation. IEICE Trans. Fundamentals E86-A (2003) pp.1925–1934

[5] Engbert R, Kliegl R. ” Microsaccades uncover the orientation of covert attention. ” Vision Research, Volume 43, Number 9, April 2003, pp. 1035-1045(11) Elsevier

[6] Yarbus, A. L. Eye Movements and Vision (Plenum, New York, 1967).

[7] Engbert, R. & Kliegl, R. in The Mind’s Eyes: Cognitive and Applied Aspects of Eye Movements (eds Hyona, J.,Radach, R. & Deubel, H.) 103–117 (Elsevier, Oxford, 2003)

140