From a certain point of view
Post on 02-Jan-2016
66 Views
Preview:
DESCRIPTION
Transcript
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
From a certain point of viewEye tracking with Java
Gerrit GrunwaldJava Technology EvangelistOracle
Simon RitterHead of Java Technology EvangelismOracle
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 2
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye tracking
TheEyeTribe
Java and Eye tracking
Demos
1
2
3
4
3
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye Tracking
TheEyeTribe
Java and Eye tracking
Demos
1
2
3
4
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
How It All Started
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Progress Was Made…
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Multi-Touch Has Become Popular
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Gaming Has Driven Several Interfaces
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Gestures Rather Than Touch…
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Interactions Without Hands
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye Tracking
TheEyeTribe
Java and Eye tracking
Demos
2
1
3
4
11
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
“Eye tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head.”
– Wikipedia
12
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Of the stimuli that reach the brain are visual
80%
13
The eyes are the windows to the soul
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Eyes Give Clues When Answering Questions
Visually Constructed Visually Remembered
Auditory Constructed Auditory Remembered
Feeling/Kinesthetic Internal Dialog
This is for a right-handed person. Positions reversed if left-handed
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Where to use it ?
15
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 16
Eye gaze correction for video conferencing
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 17
Minimize dangers in air traffic displays
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 18
Developing Video Games and Graphics
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 19
Marketing and E-Commerce website development
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20
Improved computer-human interaction
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
What can it do ?
21
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Determine whether a user is looking at the screen– Paying attention to course, video, etc
• If a user is reading or scanning– Speed of movement of the eyes over text
• Intensity of user’s attention– How long they look ata aprticular thing
• Whether the user is searching for specific information
Capabilities
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Eye attached tracking• Electric potential measurement• Optical tracking
23
Types of Eye Trackers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Special contact lense with embedded mirror or sensor (magnetic search coils)• Very precise• Allows measurement in horizontal,
vertical and torsion directions• Expensive
24
Eye attached tracking
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Measures electric potential field of eyes• Can measure eye movement even
if eyes are closed• Wearable method – also used in sleep research
• Relatively poor accuracy in gaze direction detection• Expensive
25
Electric potential measurement
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Measures reflection of IR light from the front of the cornea• Contactless method• Good for fast and slow eye
movements• Inexpensive
26
Optical tracking
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Optical Eye Tracking Principle
27
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 28
Optical Eye Tracking Principle
IR-Light emission
Eyeball
Cornea
Reflected IR-Light
Eye Tracker
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye Tracking
TheEyeTribe
Java and Eye tracking
Demos
2
1
3
4
29
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
TheEyeTribe
30
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Sampling rate
Accuracy
Latency
Operating range
Tracking area
Screen sizes
Support for
Connection31
TheEyeTribe
30 Hz and 60 Hz
0.5° - 1°
< 20ms at 60 Hz
45 cm - 75 cm
40 cm x 30 cm at 65 cm
up to 24"
C++, C# and Java
USB 3.0
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 32
TheEyeTribe
APPLICATION
SDK
EyeTribe SERVER
NETWORK
HW TRACKER
Open API
External applications (e.g. Java program)
Language spec. wrappers (C++, Java etc.)
The eye tracking process.
Initializes device(s), frameworks,servers etc.No GUI - output goes through API
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye Tracking
TheEyeTribe
Java and Eye tracking
Demos
1
4
33
5
2
3
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 34
import com.theeyetribe.client.*;
public enum EyeTracker implements IGazeListener { INSTANCE;
private GazeData gazeData;…
private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start();}
private void init() {…};private void initBuffers() {…};
}
Java and Eye Tracking
Java package
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 35
import com.theeyetribe.client.*;
public enum EyeTracker implements IGazeListener { INSTANCE;
private GazeData gazeData;…
private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start();}
private void init() {…};private void initBuffers() {…};
Java and Eye Tracking
implement Interface
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 36
import com.theeyetribe.client.*;
public enum EyeTracker implements IGazeListener { INSTANCE;
private GazeData gazeData;…
private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start();}
private void init() {…};private void initBuffers() {…};
}
Java and Eye Tracking
initialize EyeTracker
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 37
private void initEyeTracker() { GazeManager.getInstance().activate(GazeManager.ApiVersion.VERSION_1_0,
GazeManager.ClientMode.PUSH);}
Java and Eye Tracking
initialize EyeTracker
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 38
public void start() { if (running.get()) return; running.set(true); GazeManager.getInstance().addGazeListener(this); timer.start();}
Java and Eye Tracking
start EyeTracker
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 39
@Override public void onGazeUpdate(final GazeData GAZE_DATA) { gazeData = GAZE_DATA; xBuffer.remove(); yBuffer.remove(); xBuffer.addLast(gazeData.smoothedCoordinates.x); yBuffer.addLast(gazeData.smoothedCoordinates.y); xAvg = xBuffer.stream().mapToDouble((x) -> x).average(); yAvg = yBuffer.stream().mapToDouble((y) -> y).average();}
Java and Eye Tracking
get Gaze data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 40
public void stop() { GazeManager.getInstance().removeGazeListener(EyeTracker.this); GazeManager.getInstance().deactivate();}
Java and Eye Tracking
stop EyeTracker
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
What data do you get ?
41
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Gaze Data delivered in frames…
42
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Tracking stateState of the eye and gaze tracking process. (e.g. Person in field of view)
• FixationIf the tracked gaze is currently fixated, the frame will contain that information
• Gaze CoordinatesThe point on screen (pixels) that the user is currently looking at. (Needs calibration)
• Pupil coordinatesThe position of a tracked persons pupil relative to the Tracker sensor
43
Gaze data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Problem…
44
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Problem…SPEED
45
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 46
timer = new AnimationTimer() { @Override public void handle(final long now) { if (now > lastTimerCall + UPDATE_INTERVAL) { if (null == gazeData) return; focusPoint.set(new Point2D(xAvg.getAsDouble(), yAvg.getAsDouble())); eventCounter++; leftEyeOpen.set(gazeData.leftEye.pupilSize > 0); rightEyeOpen.set(gazeData.rightEye.pupilSize > 0); state.set(gazeData.state); lastTimerCall = NOW; } } };}
Problem…SPEED
use AnimationTimer to update data in intervals
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Eye tracking provides a useful and different way to interact with a UI– Control is hard, tracking a user’s interaction is easier
• Many potential applications– Helping the physically challenged– Reducing potential problems in critical systems
• www.eyetribe.com• dev.eyetribe.com/java
Conclusions And Resources
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Human Machine Interaction
Eye Tracking
TheEyeTribe
Java and Eye tracking
Demos
1
4
48
5
2
3
top related