camera ¶

Top-level entry-point for the pupil_labs.camera package

Modules:

radial –
utils –

Classes:

Camera –

A camera model with radial distortion.

Functions:

get_perspective_transform –

Computes a perspective transformation matrix from four point correspondences.
perspective_transform –

Applies a perspective transformation to 2D points.

Camera ¶

Camera(pixel_width: int, pixel_height: int, camera_matrix: CameraMatrixLike, distortion_coefficients: DistortionCoefficientsLike | None = None, extrinsics_affine_matrix: AffineMatrixLike | None = None, use_optimal_camera_matrix: bool = False)

A camera model with radial distortion.

Methods:

distort_image –

Distorts the provided image.
distort_points –

Distorts 2D image points using the camera's intrinsics.
project_points –

Projects 3D points onto the 2D image plane using the camera's intrinsics.
undistort_image –

Undistorts the provided image.
undistort_points –

Undistorts 2D image points using the camera's intrinsics.
unproject_points –

Unprojects 2D image points to 3D space using the camera's intrinsics.

Attributes:

optimal_camera_matrix (CameraMatrix) –

The "optimal" camera matrix for undistorting images.

Source code in src/pupil_labs/camera/radial.py

def __init__(
    self,
    pixel_width: int,
    pixel_height: int,
    camera_matrix: CT.CameraMatrixLike,
    distortion_coefficients: CT.DistortionCoefficientsLike | None = None,
    extrinsics_affine_matrix: CT.AffineMatrixLike | None = None,
    use_optimal_camera_matrix: bool = False,
):
    self.pixel_width = pixel_width
    self.pixel_height = pixel_height
    self.camera_matrix = camera_matrix
    self.distortion_coefficients = distortion_coefficients
    self.extrinsics_affine_matrix = extrinsics_affine_matrix
    self.use_optimal_camera_matrix = use_optimal_camera_matrix

optimal_camera_matrix `cached` `property` ¶

optimal_camera_matrix: CameraMatrix

The "optimal" camera matrix for undistorting images.

This method uses OpenCV's getOptimalNewCameraMatrix to calculate a new camera matrix that maximizes the retirval of sensible pixels in the undistortion process, while avoiding "virtual" black pixels stemming from outside the captured distorted image.

distort_image ¶

distort_image(image: Image, use_optimal_camera_matrix: bool | None = None) -> Image

Distorts the provided image.

This implementation uses cv2.remap with a precomputed map, instead of cv2.undistort. This is significantly faster when undistorting multiple images because the undistortion maps are computed only once.

Parameters:

image (Image) –

Image array
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix

Source code in src/pupil_labs/camera/radial.py

def distort_image(
    self,
    image: CT.Image,
    use_optimal_camera_matrix: bool | None = None,
) -> CT.Image:
    """Distorts the provided image.

    This implementation uses cv2.remap with a precomputed map, instead of
    cv2.undistort. This is significantly faster when undistorting multiple images
    because the undistortion maps are computed only once.

    Args:
        image: Image array
        use_optimal_camera_matrix: If True applies optimal camera matrix

    """
    map1, map2 = (
        self._optimal_distort_rectify_map
        if self._parse_use_optimal_camera_matrix(use_optimal_camera_matrix)
        else self._distort_rectify_map
    )

    remapped: CT.Image = cv2.remap(
        image,
        map1,
        map2,
        interpolation=cv2.INTER_LINEAR,
        borderValue=0,
    )
    return remapped

distort_points ¶

distort_points(points_2d: Points2DLike, use_optimal_camera_matrix: bool | None = None) -> Points2D

Distorts 2D image points using the camera's intrinsics.

Parameters:

points_2d (Points2DLike) –

Array-like of 2D point(s) to be distorted.
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix

Source code in src/pupil_labs/camera/radial.py

def distort_points(
    self,
    points_2d: CT.Points2DLike,
    use_optimal_camera_matrix: bool | None = None,
) -> CT.Points2D:
    """Distorts 2D image points using the camera's intrinsics.

    Args:
        points_2d: Array-like of 2D point(s) to be distorted.
        use_optimal_camera_matrix: If True applies optimal camera matrix

    """
    points_3d = self.unproject_points(
        points_2d,
        use_distortion=False,
        use_optimal_camera_matrix=use_optimal_camera_matrix,
    )
    distorted_points = self.project_points(
        points_3d,
        use_distortion=True,
        use_optimal_camera_matrix=use_optimal_camera_matrix,
    )
    return distorted_points

project_points ¶

project_points(points_3d: Points3DLike, use_distortion: bool = True, use_optimal_camera_matrix: bool | None = None, use_extrinsics: bool = False) -> Points2D

Projects 3D points onto the 2D image plane using the camera's intrinsics.

Parameters:

points_3d (Points3DLike) –

Array of 3D point(s) to be projected.
use_distortion (bool, default: True ) –

If True, applies distortion using the camera's distortion coefficients. If False, ignores distortion.
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix
use_extrinsics (bool, default: False ) –

If True, transforms the 3D points from world to camera coordinates using the camera's extrinsic parameters before projection. If False (default), assumes that the 3D points are already in camera coordinates.

Source code in src/pupil_labs/camera/radial.py

def project_points(
    self,
    points_3d: CT.Points3DLike,
    use_distortion: bool = True,
    use_optimal_camera_matrix: bool | None = None,
    use_extrinsics: bool = False,
) -> CT.Points2D:
    """Projects 3D points onto the 2D image plane using the camera's intrinsics.

    Args:
        points_3d: Array of 3D point(s) to be projected.
        use_distortion: If True, applies distortion using the camera's distortion
            coefficients. If False, ignores distortion.
        use_optimal_camera_matrix: If True applies optimal camera matrix
        use_extrinsics: If True, transforms the 3D points from world to camera
            coordinates using the camera's extrinsic parameters before projection.
            If False (default), assumes that the 3D points are already in camera
            coordinates.

    """
    np_points_3d = to_np_point_array(points_3d, 3)
    distortion_coefficients = self._get_distortion_coefficients(use_distortion)
    camera_matrix = self._get_unprojection_camera_matrix(use_optimal_camera_matrix)
    rvec, tvec = self._get_affine_transform_vectors(use_extrinsics)

    projected_2d, _ = cast(
        tuple[np.ndarray, np.ndarray],
        cv2.projectPoints(
            objectPoints=np_points_3d,
            rvec=rvec,
            tvec=tvec,
            cameraMatrix=camera_matrix,
            distCoeffs=distortion_coefficients,
        ),
    )
    projected_2d = projected_2d[:, 0]
    projected_2d = projected_2d.astype(np.float64)
    if np_points_3d.ndim == 1:
        return cast(CT.Points2D, projected_2d[0])

    return projected_2d

undistort_image ¶

undistort_image(image: Image, use_optimal_camera_matrix: bool | None = None) -> Image

Undistorts the provided image.

This implementation uses cv2.remap with a precomputed map, instead of cv2.undistort. This is significantly faster when undistorting multiple images because the undistortion maps are computed only once.

Parameters:

image (Image) –

Image array
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix

Source code in src/pupil_labs/camera/radial.py

def undistort_image(
    self,
    image: CT.Image,
    use_optimal_camera_matrix: bool | None = None,
) -> CT.Image:
    """Undistorts the provided image.

    This implementation uses cv2.remap with a precomputed map, instead of
    cv2.undistort. This is significantly faster when undistorting multiple images
    because the undistortion maps are computed only once.

    Args:
        image: Image array
        use_optimal_camera_matrix: If True applies optimal camera matrix

    """
    map1, map2 = (
        self._optimal_undistort_rectify_map
        if self._parse_use_optimal_camera_matrix(use_optimal_camera_matrix)
        else self._undistort_rectify_map
    )
    remapped: CT.Image = cv2.remap(
        image,
        map1,
        map2,
        interpolation=cv2.INTER_LINEAR,
        borderValue=0,
    )
    return remapped

undistort_points ¶

undistort_points(points_2d: Points2DLike, use_optimal_camera_matrix: bool | None = None) -> Points2D

Undistorts 2D image points using the camera's intrinsics.

Parameters:

points_2d (Points2DLike) –

Array-like of 2D point(s) to be undistorted.
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix

Source code in src/pupil_labs/camera/radial.py

def undistort_points(
    self,
    points_2d: CT.Points2DLike,
    use_optimal_camera_matrix: bool | None = None,
) -> CT.Points2D:
    """Undistorts 2D image points using the camera's intrinsics.

    Args:
        points_2d: Array-like of 2D point(s) to be undistorted.
        use_optimal_camera_matrix: If True applies optimal camera matrix

    """
    np_points_2d = to_np_point_array(points_2d, 2)
    camera_matrix = self._get_unprojection_camera_matrix(use_optimal_camera_matrix)
    undistorted_2d = cv2.undistortPoints(
        src=np_points_2d,
        cameraMatrix=self.camera_matrix,
        distCoeffs=self.distortion_coefficients,
        R=None,
        P=camera_matrix,
    )[:, 0]
    if np_points_2d.ndim == 1:
        return cast(CT.Points2D, undistorted_2d[0])

    return undistorted_2d

unproject_points ¶

unproject_points(points_2d: Points2DLike, use_distortion: bool = True, use_optimal_camera_matrix: bool | None = None) -> Points3D

Unprojects 2D image points to 3D space using the camera's intrinsics.

Parameters:

points_2d (Points2DLike) –

Array-like of 2D point(s) to be unprojected.
use_distortion (bool, default: True ) –

If True, applies distortion correction using the camera's distortion coefficients. If False, ignores distortion correction.
use_optimal_camera_matrix (bool | None, default: None ) –

If True applies optimal camera matrix

Source code in src/pupil_labs/camera/radial.py

def unproject_points(
    self,
    points_2d: CT.Points2DLike,
    use_distortion: bool = True,
    use_optimal_camera_matrix: bool | None = None,
) -> CT.Points3D:
    """Unprojects 2D image points to 3D space using the camera's intrinsics.

    Args:
        points_2d: Array-like of 2D point(s) to be unprojected.
        use_distortion: If True, applies distortion correction using the camera's
            distortion coefficients. If False, ignores distortion correction.
        use_optimal_camera_matrix: If True applies optimal camera matrix

    """
    np_points_2d = to_np_point_array(points_2d, 2)
    distortion_coefficients = self._get_distortion_coefficients(use_distortion)
    camera_matrix = self._get_unprojection_camera_matrix(use_optimal_camera_matrix)

    projected_3d = cv2.undistortPoints(
        src=np_points_2d,
        cameraMatrix=camera_matrix,
        distCoeffs=distortion_coefficients,
    )[:, 0]
    projected_3d = cv2.convertPointsToHomogeneous(projected_3d)[:, 0]
    projected_3d = projected_3d.astype(np.float64)
    if np_points_2d.ndim == 1:
        return cast(CT.Points3D, projected_3d[0])
    return projected_3d

get_perspective_transform ¶

get_perspective_transform(points1: Points2DLike, points2: Points2DLike) -> NDArray[float64]

Computes a perspective transformation matrix from four point correspondences.

Parameters:

points1 (Points2DLike) –

Array-like of 4 source 2D points.
points2 (Points2DLike) –

Array-like of 4 destination 2D points.

Returns:

NDArray[float64] –

3x3 perspective transformation matrix.

Source code in src/pupil_labs/camera/utils.py

def get_perspective_transform(
    points1: CT.Points2DLike, points2: CT.Points2DLike
) -> npt.NDArray[CT.float64]:
    """Computes a perspective transformation matrix from four point correspondences.

    Args:
        points1: Array-like of 4 source 2D points.
        points2: Array-like of 4 destination 2D points.

    Returns:
        3x3 perspective transformation matrix.

    """
    np_points_2d_1 = to_np_point_array(points1, 2)
    np_points_2d_2 = to_np_point_array(points2, 2)

    return cv2.getPerspectiveTransform(
        np_points_2d_1.astype(np.float32), np_points_2d_2.astype(np.float32)
    )

perspective_transform ¶

perspective_transform(points: Points2DLike, transform: NDArray[floating]) -> Points2D

Applies a perspective transformation to 2D points.

Parameters:

points (Points2DLike) –

Array-like of 2D point(s) to be transformed.
transform (NDArray[floating]) –

3x3 perspective transformation matrix.

Returns:

Points2D –

Transformed 2D points with the same shape as input.

Source code in src/pupil_labs/camera/utils.py

def perspective_transform(
    points: CT.Points2DLike, transform: npt.NDArray[CT.floating]
) -> CT.Points2D:
    """Applies a perspective transformation to 2D points.

    Args:
        points: Array-like of 2D point(s) to be transformed.
        transform: 3x3 perspective transformation matrix.

    Returns:
        Transformed 2D points with the same shape as input.

    """
    np_points_2d = to_np_point_array(points, 2)
    points_trans = cv2.perspectiveTransform(np_points_2d.reshape(-1, 1, 2), transform)
    return points_trans.reshape(-1, 2)

camera ¶

Camera ¶

optimal_camera_matrix cached property ¶

distort_image ¶

distort_points ¶

project_points ¶

undistort_image ¶

undistort_points ¶

unproject_points ¶

get_perspective_transform ¶

perspective_transform ¶

optimal_camera_matrix `cached` `property` ¶