This document describes the communication protocol of The Eye Tribe Tracker known as the Tracker API.
Using the Tracker API requires a connected and correctly positioned Tracker Device and a running Tracker Server as explained in the Getting Started guide.
A connection must be established between a client implementation and the Tracker Server before messages can be exchanged using the Tracker API. Establishing such a connection involves opening a TCP Socket in your programming language of choice and connecting to localhost
on port 6555
. For a full source code example see Tutorials section.
The Tracker API protocol consists in the exchange of JSON formatted messages over TCP sockets. The Tracker API protocol is explained in the following.
Client Message
Request Message
The request message is a generic structure, which has three mandatory attributes:
category
: The query category. The different types of requests are grouped into different categoriesrequest
: The actual request of the message. What constitutes a valid request depends on the categoryvalues
: an JSON array of parameters or a JSON object depending on the request type
Conceptually, the category
, request
and values
can be thought of as class.method.value. From this, the the general JSON structure for the client request message is defined as:
{ "category": string, "request" : string, "values" : [ … ] or { … } }
Category Type | Values |
tracker | for SDK Tracker state and information related requests |
calibration | for calibration related requests |
heartbeat | for signalling heartbeats to the server |
Response Message
The SDK Tracker reply message serves a dual purpose; firstly, for each request sent by the client, the tracker will provide the result of a request with reply message. Secondly, it serves as message publishing container for continuously published data, i.e. gaze data.
category
: the category of the incoming message. This allows the client to differentiate between replies for a request or published datarequest
: type of the requeststatuscode
: HTTP status code compliant integral value denoting the overall result of the requestvalues
: a JSON object of actual return values, if any
The valid categories for replies are:
Category Type | Values |
tracker | for SDK Tracker state and information related requests |
calibration | for calibration related requests |
heartbeat | for signalling heartbeats to the server |
On error, the reply will hold suited statuscode
and values
object may contain:
statusmessage
: string, an error description of why the request failed[get/set value]
: string, an error description of why get/set request failed
All requests related to the Tracker Server are specified with the category tracker
. This includes initial configuration requests to the tracker, once a client has connected to it, which consists of specifying desired frame rate, whether to push or pull and protocol version.
The Tracker category support two generic requests:
get
: read specified tracker valuesset
: write specified tracker values (if writable).
{ "push": bool, "heartbeatinterval": int, "version": int, "trackerstate": integer, "framerate": integer, "iscalibrated": bool, "iscalibrating": bool, "calibresult": object "frame": string, "screenindex": int, "screenresw" : int, "screenresh" : int, "screenpsyw" : float, "screenpsyh" : float }
Below is the meaning of each state item value dependent on whether it is get
or set
.
Attributes | Get | Set |
push: boolean | Tracker server is running in push, or pull mode. | Start tracker server in push mode |
heartbeatinterval: integer | The expected interval in millisecond heartbeats are sent from client | N/A - immutable, defined by Tracker Server |
version: integer | The protocol version currently used by the client Defaults to latest version | integer: The protocol version requested by the client |
trackerstate: integer | The state of the physical tracker device, see Tracker State | N/A - immutable, defined by SDK Tracker |
framerate: integer | The frame rate that the tracker is running at | N/A - immutable, configured by user (config file) |
iscalibrated: boolean | Indicates if we have a calibrated tracker. | N/A - immutable, state maintained by SDK |
iscalibrating: boolean | Indicates if we are in calibration state. | N/A - immutable, state maintained by SDK |
calibresult: object | Latest valid calibration result object | N/A - immutable, use for pull mode |
frame: string | Latest frame data available, see Frame Object | N/A - immutable, use for pull mode |
screenindex: integer | Index of screen in multi screen setup | Index of current screen |
screenresw: integer | Screen resolution width in pixels | Screen resolution width in pixels |
screenresh: integer | Screen resolution height in pixels | Screen resolution height in pixels |
screenpsyw: float | Screen physical width in meters | Screen physical width in meters |
screenpsyh: float | Screen physical height in meters | Screen physical height in meters |
Request get
Client needs to know if the server is running and the system is calibrated. This is done by submitting the following request:
{ "category": "tracker", "request" : "get", "values": [ "push", "iscalibrated" ] }
On success, the server replies:
{ "category": "tracker", "request" : "get", "statuscode" : 200, "values": { "push" : true, "iscalibrated": true } }
Note that a user calibration must be performed before any on-screen gaze coordinates are transmitted.
Request set
Client wants to start tracker. This is done submitting the following request:
{ "category": "tracker", "request": "set", "values": { "push": true, "version": 1 } }
On successful request the server replies:
{ "category": "tracker", "request": "set", "statuscode" : 200 }
If a request fails the server reply with a status code 500 and a message providing further information.
{ "category": "tracker", "request": "set", "statuscode" : 500, "values": { "statusmessage" : "set failed, internal error" } }
If a request fails the server reply with a status code 500 and a message providing further information.
{ "category": "tracker", "request": "set", "values": { "push" : false, "version" : 1 } }
Request fails as push
is misspelled and provided version
value is provided as string:
{ "category": "tracker", "request": "set", "statuscode": 400, "values": { "statusmessage": "puss, not such value exists", "version": "only integers are supported" } }
Tracker State
When requesting the tracker device connectivity state value trackerstate
it returns an integer/enum with a value as described below:
State | Description | Value |
TRACKER_CONNECTED | Tracker device is detected and working | 0 |
TRACKER_NOT_CONNECTED | Tracker device is not detected | 1 |
TRACKER_CONNECTED_BADFW | Tracker device is detected but not working due to wrong/unsupported firmware | 2 |
TRACKER_CONNECTED_NOUSB3 | Tracker device is detected but not working due to unsupported USB host | 3 |
TRACKER_CONNECTED_NOSTREAM | Tracker device is detected but not working due to no stream could be received | 4 |
Frame Object
When client connection to the SDK tracker has been established, the client will start receiving gaze data. The frame data is specified as:
{ "category": "tracker", "statuscode": 200, "values": { "frame" : { "timestamp": string, //string time representation e.g. '2014-04-15 15:28:46.628' "time": int, //timestamp in milliseconds "fix": bool, //is fixated? "state": int, //32bit masked state integer "raw": { //raw gaze coordinates in pixels "x": int, "y": int }, "avg": { //smoothed gaze coordinates in pix "x": int, "y": int }, "lefteye": { "raw": { //raw coordinates in pixels "x": int, "y": int }, "avg": { //smoothed coordinates in pix "x": int, "y": int }, "psize": float, //pupil size "pcenter": { //pupil coordinates normalized "x": float, "y": float } }, "righteye": { "raw": { //raw coordinates in pixels "x": int, "y": int }, "avg": { //smoothed coordinates in pix "x": int, "y": int }, "psize": float, //pupil size "pcenter": { //pupil coordinates normalized "x": float, "y": float } } } } }
The state of the SDK Tracker is embedded in the "frame" object as the "state" value. The value is a masked integer and can hold several states at the same time. The state is extracted according to below table:
State | Description | Values |
STATE_TRACKING_GAZE | Tracker is calibrated and producing on-screen gaze coordinates. Eye control is enabled. | Mask: (0x1) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_EYES | Tracker possibly calibrated and is tracking both eyes, including pupil and glint. | Mask: (0x2) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_PRESENCE | Tracker possibly calibrated and is tracking presence of user. Presence defined as face or single eye. | Mask: (0x4) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_FAIL | Tracker failed to track anything in this frame. | Mask: (0x8) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_LOST | Tracker has failed to detect anything and tracking is now lost | Mask: (0x10) true: ((state & mask) != 0) false: ((state & mask) == 0) |
Pull mode
If tracker value "push" has been set to false, the client is expected to pull frame data. This is done by issuing the tracker-category request get
with a value of frame
:
{ "category": "tracker", "request" : "get", "values": [ "frame" ] }
Note that requesting a frame can be done at any time, even when running in push mode.
To calibrate the tracker a client perform a calibration process. The API requests involved in a calibration process are encapsulated in the calibration
category and are described below.
Request start
The start
request prepares the tracker for a new calibration.
{ "category": "calibration", "request" : "start", "values": { "pointcount": integer } }
Expected request value
pointcount
: The number of calibration points the tracker needs to sample at to produce a valid calibration. The minimum number of calibration points allowed is 7, but it is recommended to use 9 or more.
Successful Tracker Server Response
{ "category": "calibration", "request" : "start", "statuscode" : 200 }
Request pointstart
Mark the beginning of a new calibration point for the tracker to process.
Request Definition
{ "category": "calibration", "request" : "pointstart", "values": { "x": integer, "y": integer } }
Expected request values
x
: x-coordinate of point y
: y-coordinate of point
Successful Tracker Server Response
{ "category": "calibration", "request" : "pointstart", "statuscode" : 200 }
Request pointend
Mark the end of processing a calibration point.
Request Definition
{ "category": "calibration", "request" : "pointend" }
Successful Tracker Server Response
After each endpoint
-request the tracker will reply with:
{ "category": "calibration", "request" : "pointend", "statuscode" : 200 }
For the final successful endpoint
request the tracker will send back the calibration result. The data format of the calibresult
is described further below.
{ "category": "calibration", "request" : "pointend", "statuscode" : 200, "values": { "calibresult" : [ { … } ], … } }
Request abort
At any point during the calibration an abort
request may be sent to cancel an ongoing sequence. If the tracker have had a previous working calibration it will be reinstated.
Request Definition
{ "category": "calibration", "request" : "abort" }
Successful Tracker Server Response
{ "category": "calibration", "request" : "abort", "statuscode": 200 }
Request clear
Removes the current calibration from the tracker.
Request Definition
{ "category": "calibration", "request" : "clear" }
Successful Tracker Server Response
{ "category": "calibration", "request" : "clear", "statuscode" : 200 }
When the calibration process is completed, the last pointend
request returns a calibresult
object. This object is defined as:
{ "result": bool, // was the calibration successful? "deg": float, // average error in degrees "degl": float, // average error in degs, left eye "degr": float, // average error in degs, right eye "calibpoints": [ { "state": int, //state of calibration point "cp": { //coordinates in pixels "x": float, "y": float }, "mecp": { //mean estimated coords in pixels "x": float, "y": float }, "acd": { //accuracy in degrees "ad": float, "adl": float, //left eye "adr": float //right eye }, "mepix": { //mean error in pixels "mep": float, "mepl": float, //left eye "mepr": float //right eye }, "asdp": { //average std deviation in pixels "asd": float, "asdl": float, //left eye "asdr": float //right eye } }, ... ] }
The calibpoints
array contains a complete list of the results from each calibration point. The state
describes the status of each point. The values are defined as:
Value | Description | State |
0 | No useful tracking data was gathered for the calibration point. The calibration point should be resampled. | RS_NODATA |
1 | The data gathered for the calibration point is of questionable quality. A resample can be beneficial. | RS_RESAMPLE |
2 | The calibration point is okay. No resample is required. | RS_VALID |
Once a client is connected to the tracker, it must send heartbeat requests to make itself known and prove that it is still alive. These heartbeats must be sent at a rate matching the heartbeatinterval
value of the tracker
category. The tracker will reply to each heartbeat allowing the client to recognize if connection have been lost.
Request heartbeat
Since no operation is performed within the heartbeat category no special requests are currently available.
Request Definition
{ "category": "heartbeat" }
Successful Tracker Server Response
{ "category": "heartbeat", "statuscode": 200 }
Each reply from the tracker holds a status code. The status codes are compliant to standard HTTP status codes but The Eye Tribe SDK also define a few status codes of their own.
Tracker State ChangeThe state of the connected tracker device has changed. Connected clients should update themselves.802
Title | Description | Value |
Calibration Change | Calibration state has changed. Connected clients should update themselves. | 800 |
Display Change | Active display index has changed in multi screen setup. Connected clients should update themselves. | 801 |
Tracker State Change | The state of the connected tracker device has changed. Connected clients should update themselves. | 802 |