almond_axol.lerobot

LeRobot-compatible Robot, Teleoperator, and Camera wrappers for the Axol hardware.

LeRobot-compatible wrappers for the Axol hardware. Requires the lerobot extra. These classes implement the LeRobot Robot, Teleoperator, and Camera interfaces so the Axol works with any LeRobot training or data-collection pipeline without modification. The collect-data and run-policy CLI commands are built on top of this layer.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig
from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

Use with the stock LeRobot CLIs

almond-axol is published to PyPI, and the companion lerobot_robot_axol plugin registers these wrappers through LeRobot’s plugin auto-discovery — so the stock LeRobot CLIs (lerobot-teleoperate, lerobot-record, lerobot-replay, …) drive the Axol with no changes to LeRobot itself. Installing the plugin pulls in almond-axol[lerobot]; then select the devices by type:

pip install lerobot_robot_axol

lerobot-teleoperate --robot.type=axol --teleop.type=axol_vr

lerobot-record \
    --robot.type=axol \
    --robot.cameras "{overhead: {type: zed, serial: 41234567, stereo: true}}" \
    --teleop.type=axol_vr \
    --dataset.repo_id=${HF_USER}/my-dataset \
    --dataset.num_episodes=5

Kind	type	Class
Robot	axol	AxolRobot
Teleoperator	axol_vr	AxolVRTeleop
Camera	zed	ZedCamera

Teleoperation still uses the VR headset — open the VR app and connect to the robot machine. For the highest-rate data collection (out-of-process ZED capture, on-Jetson NVENC dataset encoding, headset video streaming), prefer the SDK’s own collect-data / run-policy, which are built on these same classes.

Requires LeRobot ≥ 0.6.1, Linux, and Python 3.13 on the robot machine; the zed camera additionally needs pyzed (zed.install). See the plugin README for adding it to an existing one-command-install axol environment.

AxolRobot

LeRobot Robot wrapping the async Axol hardware driver. A background thread runs a dedicated asyncio event loop so motor telemetry keeps streaming while the synchronous get_observation() and send_action() calls block on the calling thread.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.camera import ZedCameraConfig

config = AxolRobotConfig(
    cameras={
        "overhead":  ZedCameraConfig(serial=41234567),
        "left_arm":  ZedCameraConfig(serial=41234568),
        "right_arm": ZedCameraConfig(serial=41234569),
    },
)
with AxolRobot(config) as robot:
    obs = robot.get_observation()           # joints + camera frames
    joint_obs = robot.get_joint_observation()  # joints only — use in tight control loops
    robot.send_action(obs)                  # hold current position

AxolRobotConfig fields

Field	Default	Description
cameras	{}	ZedCameraConfig instances keyed by name, each opening a locally attached ZED camera by serial number. A camera with stereo=True expands into two observation keys, <name>_left / <name>_right, backed by a single grab (see ZedStereoCamera below).
axol_config	AxolConfig()	Per-joint gains and safety parameters forwarded to the hardware driver
telemetry_hz	120.0	Background joint telemetry polling rate in Hz
observe_torques	False	Include joint torques in observation.state
observe_cartesian	False	Work in Cartesian end-effector space instead of 16 joint positions: observations report per-arm 6-axis EE poses (+ gripper) via FK, and send_action() accepts the same and resolves it to joint targets via IK
left_channel	"can_alm_axol_l"	SocketCAN interface for the left arm
right_channel	"can_alm_axol_r"	SocketCAN interface for the right arm
video_backend	"auto"	Camera capture backend: "gst" (GPU-resident zed-gstreamer pipeline, low latency), "sdk" (ZED Python SDK), or "auto" (prefer gst when its stack is installed, else fall back to the SDK). run-policy overrides this to "sdk" since it streams no headset video.

Key methods

Method	Description
get_observation()	Returns joint positions (+ torques if enabled) and latest camera frames
get_joint_observation()	Returns joint positions only — no camera reads; use in the high-frequency teleop path
send_action(action)	Sends joint position targets — or, with observe_cartesian, per-arm Cartesian EE poses resolved to joint targets via IK — via impedance control (arm) and position-force control (gripper)
positions	(left, right) cached arm positions from telemetry, each shape (8,)

AxolVRTeleop

LeRobot Teleoperator wrapping VRTeleop. Runs the VR WebSocket server and IK subprocess on a background thread so get_action() is non-blocking and safe to call from any thread.

from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig

teleop = AxolVRTeleop(AxolVRTeleopConfig())
pos_l, pos_r = robot.positions
teleop.connect(q_start_left=pos_l, q_start_right=pos_r)

while True:
    action = teleop.get_action()
    events = teleop.get_teleop_events()

AxolVRTeleopConfig fields

Field	Default	Description
vr_teleop_config	VRTeleopConfig()	Rest poses, IK frequency, filter parameters — see almond_axol.teleop
kinematics_config	KinematicsConfig()	IK solver weights — see almond_axol.kinematics
vr_server_config	VRServerConfig()	WSS port and TLS certificate paths — see almond_axol.vr

Key methods

Method	Description
get_action()	Returns the latest smoothed joint positions as a LeRobot RobotAction dict
get_teleop_events()	Returns and clears latched episode-control events (start_recording, TERMINATE_EPISODE, RERECORD_EPISODE)
request_reset()	Triggers a collision-aware trajectory back to the rest pose
is_resetting	True while the reset move is pending or in progress
send_feedback_state(state)	Broadcasts a VRState override (e.g. SAVING) to all connected VR headsets

ZedCamera

LeRobot Camera for a locally attached ZED camera. Opens the camera by serial number via the ZED SDK and grabs frames in a background thread. Requires pyzed (zed.install).

from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

cam = ZedCamera(ZedCameraConfig(serial=41234567))
cam.connect()
frame = cam.read_latest()   # shape (H, W, 3), non-blocking, returns most recent frame
cam.disconnect()

Each grabbed frame carries two timestamps on the local perf_counter clock: capture_perf_ts (when the sensor exposed the frame, derived from TIME_REFERENCE.IMAGE) and receive_perf_ts (when this process retrieved it). Since capture is local, both live on the same clock — no cross-machine sync needed. The retrieve-vs-capture skew is sampled on connect() and a warning is logged if the mean falls outside the expected range.

Method	Description
read_latest(max_age_ms=500)	Most recent frame, non-blocking; raises TimeoutError if it is older than max_age_ms.
read_latest_with_ts()	(frame, capture_perf_ts, receive_perf_ts) for the most recent frame.
read_at_or_after(target_capture_perf_ts, timeout_ms=500)	Block until a frame with capture_perf_ts >= target is available. Used by collect-data to align every camera and the joint sample on the same capture timeline.

ZedStereoCamera

Wrapper for a stereo ZED X. Opens one SDK connection and, on every grab, retrieves both eyes — so the camera is grabbed once. The eyes are exposed as left_view / right_view, which present the same read API as ZedCamera so the rest of the pipeline can treat them as ordinary cameras.

from almond_axol.lerobot.camera import ZedStereoCamera, ZedCameraConfig

stereo = ZedStereoCamera(ZedCameraConfig(serial=41234569, stereo=True))
stereo.connect()
left = stereo.left_view.read_latest()    # (H, W, 3) — left eye
right = stereo.right_view.read_latest()  # (H, W, 3) — right eye
stereo.disconnect()

When an AxolRobot camera config has stereo=True, the observation keys depend on eyes (see AxolRobotConfig.observation_cameras()): "both" expands camera X into X_left / X_right (the head camera records both eyes in the dataset), while "left" / "right" exposes a single eye under the plain name X — indistinguishable downstream from a mono camera (the wrist convention). Stereo is auto-promoted from the serial by AxolRobotConfig.apply_detected_stereo() — collect-data / run-policy call it so a stereo ZED X opens correctly without setting stereo by hand.

ZedCameraConfig fields

Field	Default	Description
serial	0	Serial number of the camera to open. 0 is the “unassigned” sentinel: collect-data / run-policy seed all three slots and drop the ones left at 0 (AxolRobotConfig.select_assigned_cameras()), so you only assign the cameras you have (at least one). A real ZED serial is positive. List connected cameras with list_zed_devices().
fps	60	Capture frame rate. None adopts the camera default on connect.
width	960	Frame width (SVGA). With height, must name a supported ZED resolution (SVGA 960×600, HD1080 1920×1080, HD1200 1920×1200). None auto-detects on connect.
height	600	Frame height (SVGA).
color_mode	RGB	Output color channel order.
warmup_s	1	Seconds to read frames during connect() before returning
stereo	False	Open the camera as a stereo ZED X carrying both eyes on one grab. Auto-detected from the serial at startup (apply_detected_stereo()), so it’s rarely set by hand.
eyes	"both"	Which eye(s) of a stereo camera to expose as recorded observations: "both" → X_left and X_right (the head convention); "left" / "right" → that single eye under the plain name X, so it looks like a mono camera downstream (the wrist convention). apply_detected_stereo() sets this to "both" for the overhead and "left" for any other slot. Ignored when stereo=False.
stream_eyes	None	Which eye(s) of a stereo camera to stream to the headset, independently of eyes. None follows eyes (the coupled default); set it to decouple the headset feed from the recording — e.g. stream "both" for depth in teleop while recording only "left". Honored by the collect-data / teleop relay only (run-policy streams no video). Ignored when stereo=False.
record	True	Whether this camera is recorded to the dataset at all. False drops it from observation_cameras() while still allowing it to stream.
stream	True	Whether this camera is streamed to the headset at all. False drops it from the relay’s encoded feed while still allowing it to be recorded.