Skip to main content
Records teleoperation episodes using VR controller inputs and locally attached ZED cameras. The Rust core owns arm CAN at 240 Hz. Saves to a LeRobot-format dataset. Loops until Ctrl+C.
For the end-to-end workflow, see the Data Collection guide.
This command is configured via draccus. The robot and teleop subsystems are exposed as the nested robot_config / teleop_config configs (cameras, per-joint gains, IK, VR server) — nest into them with dots or pass a whole-config file. See Command configuration for the syntax. The table below covers the commonly used flags; every nested field — robot_config.axol_config.* gains, robot_config.cameras.*, and the teleop_config.vr_teleop_config.* / kinematics_config.* / vr_server_config.* / cart.* subsystems — is listed with its default in Data Collection Config Parameters.

Headset camera views

During collection the camera feeds are also relayed to the headset over WebRTC (hardware-encoded on the Jetson’s NVENC, installed by gst.install), so the operator sees the overhead and both wrist views at once while recording — freely moved and resized, same as teleop. Just like teleop, the cameras run in a dedicated subprocess (almond_axol.video.video_proc.VideoRelayProcess) so the grab/encode/WebRTC pump never contends with the teleop/IK loops. A single grab per camera feeds two independent GPU NVENC branches: the full-resolution headset branch is always 30 fps, while the dataset branch runs at --fps, is downscaled to --dataset_resolution on the VIC, and is encoded to capped-VBR H.264. The relay wraps each dataset access unit and its sensor PTS with GStreamer Data Protocol before native shared-memory transport; the recorder only muxes it. A strict raw shared-memory path is the fallback when GDP support is absent. Collection uses the GStreamer source only when the resolved plugin binary matches the sensor-timestamp build stamp; otherwise it safely reopens the cameras through the ZED SDK. Either way it is still one grab—no second capture. See the VR Interface guide.

VR controller controls

Axol and Quest-source Mantis episodes are driven from Quest controllers. Managed Lighthouse/Ultimate Mantis uses rig trigger gestures or the control panel instead. The session is locked to data collection mode (no in-headset mode toggle). For the full controller layout and step-by-step flow, see the Data Collection guide. With a managed Lighthouse/Ultimate Mantis source, the rig triggers control episode boundaries: squeeze both together and release both to engage at the rigs’ current pose and start recording (the take opens once the teleop core acknowledges the engage); while recording, rapidly fully squeeze and release either trigger (or both together) three times to stop and save, or four times to stop and discard. Triple press resolves after a short pause so a fourth press can still select discard. A saved take is cut at the instant the first of those three squeezes began, so the clicks themselves (and the gripper commands they are) never end up in the dataset. A triple press while idle is ignored. Quest Mantis instead uses A, and a control-panel run can always use Start recording (engaged rig required) / Save episode / Discard episode. Launched from the control panel instead, the same decisions are also available as on-screen buttons — start a take, save it, discard it, and end a contact hold — alongside the mirrored camera feeds, so a session can be driven with the headset off. The controller stays live regardless; whichever input arrives first ends the take. On Axol, after each episode the arms return to the rest pose automatically, but guarded: a torque watchdog compares each joint’s measured torque against the gravity model while the move plays. If the residual stays above the reset torque threshold — a gripper still hooked on the scene, or an operator grabbing an arm — the move stops where it is and the arms drop into a limp gravity-compensation hold, free to be hand-guided clear (the episode keeps saving in the background). Press X when they’re clear and the collision-aware return replans from wherever the arms actually are. Recording is blocked while limp — a start press is answered with a reminder to reset first. The same watchdog can optionally cover the tracking phase too (--teleop_config.vr_teleop_config.teleop_torque_threshold, off by default): a trip mid-take discards the in-flight episode and drops the arms limp, then an X reset returns to rest to re-record. Mantis has no physical arm return: it force-disables both gripper motors between takes and requires fresh source readiness/alignment before the next recording. If an existing dataset is found at --root, collection resumes from where it left off; conversely, if no episodes were saved before exit the empty dataset directory is removed on shutdown so an aborted session does not leave a half-initialized dataset on disk. Stopping parks the arms. Ending a session — Ctrl+C, or Stop in the control panel — returns the arms to the rest pose and only then torques the motors off, so raised arms no longer drop under gravity on the way out. It is the same guarded return as the post-episode one, capped at 6 s; if it runs out of time or hits something, the arms are torqued off where they are. It is skipped when there is nothing to park or no safe way to park it: on Mantis (handheld grippers, no arms), when the arms are already at rest, when a contact hold has left them limp in the operator’s hands, when the IK worker that plans the path has died, or when the CAN bus has stopped reporting positions. A second Ctrl+C cannot interrupt it. Every episode is durable the moment it saves. LeRobot’s writers normally keep their parquet files open (footerless — unreadable) for the whole session and only make them readable at shutdown, so a killed session used to lose every episode it recorded, and shutdown had a session’s worth of flushing and video verification to do. Collect-data instead flushes the parquet footers right after each save_episode and rotates to fresh per-episode files (a legal LeRobot packing), so at any point between saves the dataset on disk is complete, readable, and resumable. This also retires the per-save video concat that rewrote the whole accumulating chunk file (saves no longer slow down as the session grows), and the decode-verification of each episode’s video runs in the background right after its save instead of over the whole session at exit — so Ctrl+C stops quickly, and killing the process outright costs at most the episode currently being written. A resume still runs a crash-consistency check for that one remaining window (and for datasets recorded before durability landed): if the dataset’s saved episodes aren’t a contiguous 0…N-1 — the signature of a recorder killed mid-save (SIGKILL, OOM, power loss), losing episodes whose writers never flushed while the episode counter had already advanced — the dataset is repaired in place rather than recorded past the gap (a gap silently pairs every later episode with a different episode’s video). The lost episodes’ frames are unrecoverable (they died in footerless parquet files), so the repair truncates back to the longest verified contiguous prefix — metadata row present, data rows readable and complete, video files present — deletes the crashed session’s orphaned files, rewrites info.json/stats.json to match, and resumes recording right after the last intact episode (logged loudly with exactly what was discarded). Only when not a single complete episode survives does collect-data refuse and point you to delete the dataset.
Dataset capture runs in a dedicated recorder subprocess (almond_axol.recording.record_proc), not just a thread — a thread would still share the control loop’s GIL, and the per-frame numpy / dataset.add_frame() / stats work was enough to stall the 120 Hz loop even with spare CPU cores. The control loop only writes timestamped joint/action snapshots to a short shared-memory history. On the Jetson, the recorder consumes one PTS-preserving H.264 access unit per camera, chooses the robot snapshot nearest the median camera exposure, and muxes the AU into the MP4 without re-encoding. Missing PTS, capture gaps, camera phase errors, or excessive state skew discard the episode instead of saving silently misaligned training data. On fallback paths the recorder samples and encodes raw frames at --fps.The hot control loop runs on the robot’s own asyncio event loop (AxolRobot.event_loop), so each motion_control is awaited inline — cooperatively interleaved on a single thread, exactly like axol teleop. This removes the per-step cross-thread send_action round trip (run_coroutine_threadsafe(...).result()) that otherwise capped — and jittered — the control rate once teleop was engaged.Joint state always comes from the Rust core’s native 240 Hz feedback stream. There is no separate Python telemetry poll loop competing with control traffic on CAN.