Skip to content

Streaming Gaze Data

Use receive_gaze_data to subscribe to gaze data.

This function can return different types of gaze data (GazeDataType) depending on the device and its configuration:

See below samples for each type of gaze data.

GazeData(
    x=784.0623779296875,
    y=537.4524536132812,
    worn=False,
    timestamp_unix_seconds=1744294828.3579288
)

Neon +2.9.31 +1.8.0

BinoAndDualMonoGazeData(
    x=848.9487915039062,
    y=474.46356201171875,
    worn=True,
    mono_left_x=895.11083984375,
    mono_left_y=477.2065734863281,
    mono_right_x=800.8253173828125,
    mono_right_y=469.348388671875,
    timestamp_unix_seconds=1772109976.1031907
)

This method exposes gaze data for Neon in Dual Mono mode, which includes binocular and monocular gaze data.

BinoAndDualMonoGazeData

BinoAndDualMonoGazeData

Bases: NamedTuple

Binocular and dual monocular gaze data

Represents the binocular 2D gaze point, the left and right dual monocular gaze points, on the scene camera coordinates with a timestamp in seconds unix epoch and an indicator of whether the glasses are being worn.

Methods:

  • from_raw –

    Create a BinoAndDualMonoGazeData instance from raw data.

Attributes:

  • datetime (datetime) –

    Get the timestamp as a datetime object.

  • mono_left_x (float) –

    X coordinate of the left monocular gaze point.

  • mono_left_y (float) –

    Y coordinate of the left monocular gaze point.

  • mono_right_x (float) –

    X coordinate of the right monocular gaze point.

  • mono_right_y (float) –

    Y coordinate of the right monocular gaze point.

  • timestamp_unix_ns (int) –

    Get the timestamp in nanoseconds since Unix epoch.

  • worn (bool) –

    Whether the glasses are being worn.

  • x (float) –

    X coordinate of the gaze point.

  • y (float) –

    Y coordinate of the gaze point.

datetime property 

datetime: datetime

Get the timestamp as a datetime object.

mono_left_x instance-attribute 

mono_left_x: float

X coordinate of the left monocular gaze point.

mono_left_y instance-attribute 

mono_left_y: float

Y coordinate of the left monocular gaze point.

mono_right_x instance-attribute 

mono_right_x: float

X coordinate of the right monocular gaze point.

mono_right_y instance-attribute 

mono_right_y: float

Y coordinate of the right monocular gaze point.

timestamp_unix_ns property 

timestamp_unix_ns: int

Get the timestamp in nanoseconds since Unix epoch.

worn instance-attribute 

worn: bool

Whether the glasses are being worn.

x instance-attribute 

x: float

X coordinate of the gaze point.

y instance-attribute 

y: float

Y coordinate of the gaze point.

from_raw classmethod 

from_raw(data: RTSPData) -> BinoAndDualMonoGazeData

Create a BinoAndDualMonoGazeData instance from raw data.

Parameters:

  • data (RTSPData) –

    The raw data received from the RTSP stream.

Returns:

Source code in src/pupil_labs/realtime_api/streaming/gaze.py 
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
@classmethod
def from_raw(cls, data: RTSPData) -> "BinoAndDualMonoGazeData":
    """Create a BinoAndDualMonoGazeData instance from raw data.

    Args:
        data (RTSPData): The raw data received from the RTSP stream.

    Returns:
        BinoAndDualMonoGazeData: An instance of BinoAndDualMonoGazeData with the
        parsed values.

    """
    x, y, worn, mono_left_x, mono_left_y, mono_right_x, mono_right_y = (
        struct.unpack("!ffBffff", data.raw)
    )
    return cls(
        x,
        y,
        worn == 255,
        mono_left_x,
        mono_left_y,
        mono_right_x,
        mono_right_y,
        data.timestamp_unix_seconds,
    )

Neon +2.8.8 +1.2.0

EyestateGazeData(
    x=784.0623779296875,
    y=537.4524536132812,
    worn=False,
    pupil_diameter_left=4.306737899780273,
    eyeball_center_left_x=-29.3125,
    eyeball_center_left_y=11.6875,
    eyeball_center_left_z=-42.15625,
    optical_axis_left_x=0.09871648252010345,
    optical_axis_left_y=0.15512824058532715,
    optical_axis_left_z=0.9829498529434204,
    pupil_diameter_right=3.2171919345855713,
    eyeball_center_right_x=33.21875,
    eyeball_center_right_y=12.84375,
    eyeball_center_right_z=-45.34375,
    optical_axis_right_x=-0.20461124181747437,
    optical_axis_right_y=0.1512681096792221,
    optical_axis_right_z=0.9670844078063965,
    timestamp_unix_seconds=1744294828.3579288
)
This method also provides pupil diameter and eye poses.

Neon +2.9.0 +1.3.6

EyestateEyelidGazeData(
    x=784.0623779296875,
    y=537.4524536132812,
    worn=False,
    pupil_diameter_left=4.306737899780273,
    eyeball_center_left_x=-29.3125,
    eyeball_center_left_y=11.6875,
    eyeball_center_left_z=-42.15625,
    optical_axis_left_x=0.09871648252010345,
    optical_axis_left_y=0.15512824058532715,
    optical_axis_left_z=0.9829498529434204,
    pupil_diameter_right=3.2171919345855713,
    eyeball_center_right_x=33.21875,
    eyeball_center_right_y=12.84375,
    eyeball_center_right_z=-45.34375,
    optical_axis_right_x=-0.20461124181747437,
    optical_axis_right_y=0.1512681096792221,
    optical_axis_right_z=0.9670844078063965,
    eyelid_angle_top_left=-1.1484375,
    eyelid_angle_bottom_left=-1.2763671875,
    eyelid_aperture_left=1.6408717632293701,
    eyelid_angle_top_right=-0.6259765625,
    eyelid_angle_bottom_right=-1.2216796875,
    eyelid_aperture_right=7.2039408683776855,
    timestamp_unix_seconds=1744294828.3579288
)
This method also provides eye openness data.

Neon +2.9.31 +1.8.0

EyestateEyelidDualMonoGazeData(
    x=852.3287353515625,
    y=517.3056030273438,
    worn=False,
    pupil_diameter_left=4.567640781402588,
    eyeball_center_left_x=-32.25,
    eyeball_center_left_y=18.6875,
    eyeball_center_left_z=-44.9375,
    optical_axis_left_x=0.20407231152057648,
    optical_axis_left_y=-0.020999005064368248,
    optical_axis_left_z=0.9787306189537048,
    pupil_diameter_right=3.035294771194458,
    eyeball_center_right_x=30.15625,
    eyeball_center_right_y=14.375,
    eyeball_center_right_z=-43.3125,
    optical_axis_right_x=-0.08533422648906708,
    optical_axis_right_y=-0.019046271219849586,
    optical_axis_right_z=0.9961703419685364,
    eyelid_angle_top_left=-0.63037109375,
    eyelid_angle_bottom_left=-1.138671875,
    eyelid_aperture_left=6.09653902053833,
    eyelid_angle_top_right=-0.06329345703125,
    eyelid_angle_bottom_right=0.1409912109375,
    eyelid_aperture_right=2.228158473968506,
    mono_left_x=864.9857177734375,
    mono_left_y=526.5974731445312,
    mono_right_x=834.6492309570312,
    mono_right_y=481.4454040527344,
    timestamp_unix_seconds=1772108289.6118817
)
This method exposes binocular and monocular gaze data, pupil diameter, and eye poses.

EyestateEyelidDualMonoGazeData

EyestateEyelidDualMonoGazeData

Bases: NamedTuple

Gaze data with additional eyelid state information.

Contains binocular gaze point, left and right monocular gaze points, pupil diameter, eyeball center coordinates, optical axis coordinates, as well as eyelid angles and aperture for both left and right eyes.

Methods:

  • from_raw –

    Create an EyestateEyelidDualMonoGazeData instance from raw data.

Attributes:

eyeball_center_left_x instance-attribute 

eyeball_center_left_x: float

X coordinate of the eyeball center for the left eye.

eyeball_center_left_y instance-attribute 

eyeball_center_left_y: float

Y coordinate of the eyeball center for the left eye.

eyeball_center_left_z instance-attribute 

eyeball_center_left_z: float

Z coordinate of the eyeball center for the left eye.

eyeball_center_right_x instance-attribute 

eyeball_center_right_x: float

X coordinate of the eyeball center for the right eye.

eyeball_center_right_y instance-attribute 

eyeball_center_right_y: float

Y coordinate of the eyeball center for the right eye.

eyeball_center_right_z instance-attribute 

eyeball_center_right_z: float

Z coordinate of the eyeball center for the right eye.

eyelid_angle_bottom_left instance-attribute 

eyelid_angle_bottom_left: float

Angle of the bottom eyelid for the left eye(rad).

eyelid_angle_bottom_right instance-attribute 

eyelid_angle_bottom_right: float

Angle of the bottom eyelid for the right eye (rad).

eyelid_angle_top_left instance-attribute 

eyelid_angle_top_left: float

Angle of the top eyelid for the left eye(rad).

eyelid_angle_top_right instance-attribute 

eyelid_angle_top_right: float

Angle of the top eyelid for the right eye (rad).

eyelid_aperture_left instance-attribute 

eyelid_aperture_left: float

Aperture of the eyelid for the left eye (mm).

eyelid_aperture_right instance-attribute 

eyelid_aperture_right: float

Aperture of the eyelid for the right eye (mm).

mono_left_x instance-attribute 

mono_left_x: float

X coordinate of the left monocular gaze point.

mono_left_y instance-attribute 

mono_left_y: float

Y coordinate of the left monocular gaze point.

mono_right_x instance-attribute 

mono_right_x: float

X coordinate of the right monocular gaze point.

mono_right_y instance-attribute 

mono_right_y: float

Y coordinate of the right monocular gaze point.

optical_axis_left_x instance-attribute 

optical_axis_left_x: float

X coordinate of the optical axis for the left eye.

optical_axis_left_y instance-attribute 

optical_axis_left_y: float

Y coordinate of the optical axis for the left eye.

optical_axis_left_z instance-attribute 

optical_axis_left_z: float

Z coordinate of the optical axis for the left eye.

optical_axis_right_x instance-attribute 

optical_axis_right_x: float

X coordinate of the optical axis for the right eye.

optical_axis_right_y instance-attribute 

optical_axis_right_y: float

Y coordinate of the optical axis for the right eye.

optical_axis_right_z instance-attribute 

optical_axis_right_z: float

Z coordinate of the optical axis for the right eye.

pupil_diameter_left instance-attribute 

pupil_diameter_left: float

Pupil diameter for the left eye.

pupil_diameter_right instance-attribute 

pupil_diameter_right: float

Pupil diameter for the right eye.

timestamp_unix_seconds instance-attribute 

timestamp_unix_seconds: float

Timestamp in seconds since Unix epoch.

worn instance-attribute 

worn: bool

Whether the glasses are being worn.

x instance-attribute 

x: float

X coordinate of the gaze point.

y instance-attribute 

y: float

Y coordinate of the gaze point.

from_raw classmethod 

from_raw(data: RTSPData) -> EyestateEyelidDualMonoGazeData

Create an EyestateEyelidDualMonoGazeData instance from raw data.

Parameters:

  • data (RTSPData) –

    The raw data received from the RTSP stream.

Returns:

Source code in src/pupil_labs/realtime_api/streaming/gaze.py 
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
@classmethod
def from_raw(cls, data: RTSPData) -> "EyestateEyelidDualMonoGazeData":
    """Create an EyestateEyelidDualMonoGazeData instance from raw data.

    Args:
        data (RTSPData): The raw data received from the RTSP stream.

    Returns:
        EyestateEyelidDualMonoGazeData:An instance of EyestateEyelidDualMonoGazeData
        with the parsed values.

    """
    (
        x,
        y,
        worn,
        pupil_diameter_left,
        eyeball_center_left_x,
        eyeball_center_left_y,
        eyeball_center_left_z,
        optical_axis_left_x,
        optical_axis_left_y,
        optical_axis_left_z,
        pupil_diam_right,
        eyeball_center_right_x,
        eyeball_center_right_y,
        eyeball_center_right_z,
        optical_axis_right_x,
        optical_axis_right_y,
        optical_axis_right_z,
        eyelid_angle_top_left,
        eyelid_angle_bottom_left,
        eyelid_aperture_left,
        eyelid_angle_top_right,
        eyelid_angle_bottom_right,
        eyelid_aperture_right,
        mono_left_x,
        mono_left_y,
        mono_right_x,
        mono_right_y,
    ) = struct.unpack("!ffBffffffffffffffffffffffff", data.raw)
    return cls(
        x,
        y,
        worn == 255,
        pupil_diameter_left,
        eyeball_center_left_x,
        eyeball_center_left_y,
        eyeball_center_left_z,
        optical_axis_left_x,
        optical_axis_left_y,
        optical_axis_left_z,
        pupil_diam_right,
        eyeball_center_right_x,
        eyeball_center_right_y,
        eyeball_center_right_z,
        optical_axis_right_x,
        optical_axis_right_y,
        optical_axis_right_z,
        eyelid_angle_top_left,
        eyelid_angle_bottom_left,
        eyelid_aperture_left,
        eyelid_angle_top_right,
        eyelid_angle_bottom_right,
        eyelid_aperture_right,
        mono_left_x,
        mono_left_y,
        mono_right_x,
        mono_right_y,
        data.timestamp_unix_seconds,
    )

You can learn more about the payload in the Under the Hood guide.

Full Code Examples

Check the whole example code here
stream_gaze.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import asyncio
import contextlib

from pupil_labs.realtime_api import Device, Network, receive_gaze_data


async def main():
    async with Network() as network:
        dev_info = await network.wait_for_new_device(timeout_seconds=5)
    if dev_info is None:
        print("No device could be found! Abort")
        return

    async with Device.from_discovered_device(dev_info) as device:
        status = await device.get_status()
        sensor_gaze = status.direct_gaze_sensor()
        if not sensor_gaze.connected:
            print(f"Gaze sensor is not connected to {device}")
            return

        restart_on_disconnect = True
        async for gaze in receive_gaze_data(
            sensor_gaze.url, run_loop=restart_on_disconnect
        ):
            print(gaze)


if __name__ == "__main__":
    with contextlib.suppress(KeyboardInterrupt):
        asyncio.run(main())