Eye tracker accuracy and precision
一个研究者的眼动跟踪方案：Yuta Itoh (伊藤 勇太)
(http://campar.in.tum.de/Main/YutaItoh Fellow at the Chair for Computer Aided Medical Procedures & Augmented Reality, Munich, Germany
http://www.ar.c.titech.ac.jp/cpt_people/yuta-itoh/ principal investigator of Augmented Vision Lab 東京工業大学 情報理工学院)
Conference on Human Factors in Computing Systems 2017 paper:
Toward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for Design （2017）
tobii：Eye tracker accuracy and precision
Accuracy and precision are important concepts for understanding how an eye tracker performs, and how can one evaluate the quality of the recorded eye tracking data.
During data collection, accuracy and precision are used as indicators of the eye tracker data validity. A system with good accuracy and precision will provide more valid data as it is able to truthfully describe the location of a person’s gaze on a screen. Accuracy is deﬁned as the average difference between the real stimuli position and the measured gaze position. Precision is deﬁned as the ability of the eye tracker to reliably reproduce the same gaze point measurement, i.e. it measures the variation of the recorded data via the Root Mean Square (RMS) of successive samples.The required level of accuracy and precision depends on the nature of the eye tracking study. Small uncertainties, for instance, can be critical when analyzing gaze data in reading studies or studies with a small stimulus.
The accuracy error varies considerably across participants and experimental conditions. Accuracy is dependent on participant properties, illumination in the test environment, stimuli properties, calibration quality, data collection procedure and the eyes’ position in the track box.
Impact of large accuracy & precision errors
Aga Bojko's Blog：The Most Precise (or Most Accurate?) Eye Tracker
(Re-post from the “Eye Tracking the User Experience” Blog. Aga is currently writing Eye Tracking the User Experience, A Practical Guide, to be published by Rosenfeld Media in 2013.)
To keep up with the developments in research and technology, I have a Google Alert set up for “eye tracking” OR “eyetracking” OR “eye-tracking.” The daily email comes to my Inbox at 11:30am, just in time for my browsing lunch (more fun than a working lunch, less fun than a non-working lunch). Today, nine out of the twenty results in the alert email mentioned Tobii Technology introducing the “most precise eye tracking solution” for mobile device testing:
Most precise! Who could resist that?
The solution (Tobii Mobile Device Stand) described in the articles is actually quite clever. I’m not sure why it made the news today because it’s been available for a while now. Maybe it was just this morning when they found it was “most precise.” I continued reading in suspense.
To my disappointment, no explanation was offered for how this conclusion was reached. What’s more, I don’t even know what was meant by “precise.” I think the author was referring to the accuracy of the eye tracking solution but I can’t be sure. And that’s precisely where the problem lies – in the confusion between precision and accuracy (and people not realizing that there is confusion). Let me explain…
The accuracy of an eye tracker is the average difference between what the eye tracker recorded as the gaze position and what the gaze position actually was. We want this offset to be as small as possible but it is obviously unrealistic to expect it to be equal to zero.
Accuracy is measured in degrees of visual angle. Typical accuracy values fall in a range between 0.5 and 1 degree. To give you an idea of what that means, one degree corresponds to half an inch (1.2 cm) on a computer monitor viewed at a distance of 27 inches (68.6 cm). In other words, the actual gaze location could be anywhere within a radius of 0.5 inch (the blue circle below) from the gaze location recorded with an eye tracker with one degree of accuracy (the “X”):
Accuracy values reported in eye tracker manuals are measured under ideal conditions, which typically include, for example, testing participants with no corrective eyewear and taking the measurement immediately after calibration. During “real research,” the difference between the reported and actual gaze locations can be larger for participants wearing glasses or contact lenses or those who moved at some point following the calibration procedure.
Precision (aka “spatial resolution”), on the other hand, is a measure of how well the eye tracker is able to reliably reproduce a measurement. Ideally, if the eye is in the same exact location in two successive measurements, the eye tracker should report the two locations as identical. That would be perfect precision.
In reality, precision values of currently available eye trackers range from 0.01 to 1 degree. These values are calculated as the root mean square of the distance (in degrees of visual angle) between successive samples. Because the precision values reported by manufacturers are measured using a motionless artificial eye (pretty cool, huh?), tracking real eyes will exhibit less precision.
The table below summarizes the relationship between eye tracking accuracy and precision. The cross indicates the actual gaze location, while the dots are gaze locations reported by the eye tracker.
All in all, the “most precise eye tracking solution” was probably just a poor choice of words but it gave me an excuse to talk about precision vs. accuracy and sound like I’m up to date on current events. I do what I can.
Tristan Hume's report: Eye Tracker Reviews: Pupil Labs, Tobii, Eye Tribe, XLabs