Teleoperation

Drive Axol from a VR headset, or run the Mantis rigs’ grippers straight from their triggers.

Teleoperation drives the arms live from a Meta Quest headset. It runs on either device: Axol (the robot, or the browser sim) or Mantis (the handheld rigs — teleop drives just the grippers from the rig triggers over CAN; tracking belongs to data collection). You can run either from the web control panel or the CLI (axol teleop).

Before you start

• Axol installed — the one-command install on the robot machine, or a development install for sim.
• The device selected and connected. The control panel’s Axol / Mantis tile carries the system-wide device choice (Select) and brings up CAN and checks the motors on connect; on the CLI run axol can.setup once. Mantis needs its hardware setup only — teleop never uses a pose source (that’s data collection’s job, set up under Mantis Tracking).
• A Meta Quest with the TLS certificate authorized — for Axol. Mantis teleop needs no headset.

No CAN setup or motor check is needed in sim — there’s no hardware to talk to.

Authorize the TLS certificate

The VR app connects over a self-signed HTTPS/WSS link, so its certificate needs a one-time approval in the headset browser — otherwise the WebSocket handshake is silently rejected.

The easiest way is from the app itself: when you press Connect and the certificate hasn’t been approved yet, the connection fails and an Authorize certificate button appears. Tap it, proceed past the warning in the popup, and the app reconnects automatically. You only need to do this once per headset (the cert is cached in ~/.almond/vr/certs/ on the host). To approve it manually instead, open https://<hostname>.local:8000 or https://<local-ip>:8000 in the headset browser and proceed past the warning.

Run it

Axol

1. Connect the host and robot

   Open the control panel and connect the Axol Host. With Axol selected on its tile, the panel brings up the robot automatically (running can.setup the first time); if the tile isn’t green, press Connect on it. For sim, you can skip the robot connection. See the Web Control Panel guide for the connect flow.

2. (Optional) assign cameras

   To see the robot’s camera feeds in the headset, open Axol settings → Cameras and assign the ZED cameras to their overhead / left arm / right arm slots. Skip this for camera-free teleop. See Cameras.

3. Select Teleoperation and Start

   Pick Teleoperation from the operation selector. Toggle sim to drive the browser visualizer with no hardware. Press Start. The panel begins streaming and is ready for the headset to connect.

4. Connect the headset

   Put on the Quest, open the VR app at axol.almond.bot, enter the host’s address, and press Connect — then Start to enter the session. From rest, press both grips together to engage; the controller layout below has the rest.

From the CLI, on the robot machine:

axol teleop

For sim, add --sim — this opens a browser visualizer at http://localhost:8002 and runs the same VR server on port 8000, so you can drive the simulated robot from the headset exactly like the real one (great for verifying network and tracking before powering the robot):

axol teleop --sim

The terminal prints the hostname and IP. Open the VR app at axol.almond.bot on the headset, enter either address, and press Connect, then Start.

Have ZED cameras attached? Pass --cameras "{overhead: 41234567, left_arm: 41234568, right_arm: 41234569}" to relay the overhead and wrist feeds into the headset — all shown at once, each movable and resizable with the controllers. See Camera views.

See the teleop reference for the full flag list (stiffness, gripper limits, per-joint gains, config files).

Mantis

Mantis teleop is grippers only, by design: the rig triggers drive the two physical grippers over CAN, and no tracking, headset, or cameras are used. Use it to prove the hub wiring, the left/right side mapping, and gripper health. Tracked Mantis runs — pose source, engage gesture, mount transforms — are data collection’s job, set up under Mantis Tracking.

1. Select and connect Mantis

   Press Select on the Mantis tile; the panel connects the rigs automatically (or press Connect). The tile shows both grippers’ CAN health. If you have not yet confirmed the side mapping, do that first — see Mantis Hardware.

2. Start and check the grippers

   With Mantis selected only Teleoperation and Collect data are listed. Pick Teleoperation (there is no sim or Jelly option on Mantis) and press Start. Once both triggers read live and released, the grippers open to their hard stops (keep them clear), then each trigger closes its gripper proportionally. Squeeze the physical left trigger: if the right gripper moves, swap the CAN side mapping. Only a faulted gripper or a stale trigger heartbeat blocks this run; Stop disables both grippers.

From the CLI (the saved Mantis settings channel map is used automatically):

axol teleop --mantis

Ready to track the rigs? Finish the pose-source setup in Mantis Tracking, then run Collect data — that operation owns the tracker bridge, the engage gesture, and the recording flow.

Quest controller layout

The bindings below are for Axol. (Mantis teleop uses no headset — the rig triggers drive the grippers directly. For the tracked Mantis gestures used by data collection, see Start gesture and recovery.)

#	Button	Action
1	Left grip	Press both 1 + 2 together to engage both arms; once engaged, each grip toggles its own arm between tracking and frozen (a frozen arm holds its pose and grasp while the other keeps going)
2	Right grip	Same as above
3	Left trigger	Actuate left gripper
4	Right trigger	Actuate right gripper
5	X	Reset — returns both arms to the rest pose; also retries the return out of the contact hold (below)
7	Y	Exit AR — closes the VR session (the arms return to rest on the way out)

Per-arm engage. From rest, press both grips together to start moving the arms. After that each grip is independent: a click toggles just that arm between tracking and frozen — a frozen arm holds its pose and keeps its gripper where it is (handy for steadying a grasped object) while the other keeps tracking. Prefer a hold instead of a toggle? Turn on Hold grips to engage (--teleop.hold_to_engage): hold both grips to start, release one to freeze that arm, keep holding to keep it going. Either way a forced disengage (reset, contact stop, or dropped link) drops both arms and needs a deliberate both-grips re-engage.

On hardware, every return to rest — an X reset and the startup move — is guarded: a torque watchdog compares each joint against the gravity model while the move plays. If the move meets unexpected resistance (something still grasped, or you grabbing an arm), it stops where it is and the arms drop into a limp gravity-compensation hold — hand-guide them clear, then press X to replan home from wherever you left them. The hold has no timeout: if you leave VR or the link drops while the arms are limp, they stay in gravity comp (free to move by hand, never stiffening on their own) until you put the headset back on and press X. Tune with the Reset contact threshold in the control panel’s Axol settings → Robot (--teleop.reset_torque_threshold on the CLI; 0 disables the watchdog). Not available in --sim (no torque feedback — and nothing to protect).

Stopping parks the arms. Ending the session — Stop in the control panel, Ctrl+C on the CLI — returns the arms to the rest pose and only then torques the motors off, so raised arms no longer drop under gravity on the way out. The move is the same guarded return described above, capped at 6 s; if it runs out of time or hits something, the arms are torqued off where they are. It is skipped when there is nothing to park or no safe way to park it: in --sim, when the arms are already at rest, when a contact hold has left them limp in your hands, when the IK worker that plans the path has died, or when the CAN bus has stopped reporting positions.

Contact stop while driving (opt-in). The return-to-rest guard above is always on; you can also arm a watchdog for the tracking phase itself — while you’re actively driving the arms. It’s off by default, since teleop pushes on the scene on purpose. Set the Teleop contact stop in the control panel’s Axol settings → Robot (--teleop.teleop_torque_threshold on the CLI; the panel suggests 16 Nm) and a sustained torque past the threshold disengages tracking and drops the arms into the same limp gravity-comp hold — hand-guide them clear, then press X to return to rest and carry on. 0 (the default) leaves it off. Hardware only.

Losing the link never moves the robot. If pose updates stop while tracking is engaged — you doff the headset, open the system menu, quit the VR app, or the link drops — the arms auto-disengage (freeze in place) after about half a second and then hold position wherever they are, for as long as it takes you to come back. Re-entering VR is inert until you deliberately re-engage (both grips), which takes a fresh engage snapshot — tracking resumes relative to where the arms already are, so they never snap toward wherever the controllers drifted during the gap. Press X when you want them back at the rest pose.

On a robot with Jelly, the headset thumbsticks drive Jelly’s base and lift during teleop — see Jelly. Nothing to enable: Jelly is inferred from the CAN devices attached, and the Robot tab’s Jelly wheels / Jelly lift switches (or --jelly.wheels false / --jelly.lift false) opt attached hardware out. Drive just Jelly with the arms left untouched by switching Axol arms off (or axol teleop --arms false).

Network tip

If VR tracking feels jittery or arrives in bursts, configure the following on your router or access point:

Setting	Value
DTIM interval	1
Beacon interval	100 ms
WMM APSD (U-APSD)	disabled

These prevent the AP from batching packets between beacon intervals, which causes intermittent latency spikes that are especially noticeable in VR.

Latency tuning

Teleop deliberately trades a little latency for smoothness: the pose stream is buffered and filtered before it reaches IK so that controller tracking glitches and network jitter don’t turn into arm jitter. The defaults add roughly 100–120 ms between your hand and the commanded arm on a clean link — the table below shows where it goes and which knob controls each stage. All of them are CLI flags (or config-file fields); none require code changes.

Knob	Default	What it buys	Latency cost
--vr_server.interp_smooth_window_s	0.12	Zero-phase pose smoothing + tracking-glitch rejection (glitches up to ~half the window are erased outright)	+60 ms, fixed — half the window. The largest single contributor
--vr_server.interp_max_delay_s	0.15	Cap on the adaptive jitter buffer that replays bursty network delivery smoothly	0 ms on a clean link (USB, quiet Wi-Fi); grows only while the network is actually jittery, up to the cap, and decays within seconds
--teleop.pose_cutoff	2.5	Tremor rejection on the raw VR poses — pole (Hz) of a linear lag-compensated low-pass (velocity feedforward cancels most of the filter’s own lag)	~40 ms effective; worst-case ~50 mm of tracking error during the fastest hand motions
--teleop.ik_alpha	0.3	Smoothing of the IK output before the velocity filter	~19 ms at the default; ~8 ms at 0.5

If teleop feels too laggy, adjust in this order — one knob at a time, on the real robot, since the same smoothing is what makes contact-heavy work (deburring, insertion) feel good:

1. --vr_server.interp_smooth_window_s 0.08 — buys back 20 ms; glitch rejection stays effective for the common short tracking glitches.
2. --teleop.ik_alpha 0.4 — a few ms, minor smoothness cost.
3. --teleop.pose_cutoff 3.5 — less pose-filter lag, but re-admits hand tremor in the band the arm’s structure responds to; watch for joint jitter before keeping it.
4. --vr_server.interp_smooth_window_s 0 — disables the fixed-lag smoother entirely (pure jitter-buffer playout). Only worth it on a wired USB connection with a well-tracked controller.

The jitter buffer (interp_max_delay_s) self-regulates — it only adds delay while packet delivery is actually bursty, so lowering its cap mostly changes how much of a bad Wi-Fi episode gets smoothed versus played as a catch-up. If you’re chasing network-induced lag, the USB connection and the router settings above are worth more than any knob here.

A stall is handled separately from jitter: a pose frame arriving more than --vr_server.interp_max_lag_s (default 0.5 s) later than the stream’s normal transit is not played. When the headset’s Wi-Fi uplink stalls for a few seconds and then drains its backlog, the arms hold instead of performing your motion seconds after you made it, then move to your current pose once on-time frames resume. The teleop log shows a warning when a stall starts (VR pose stream arriving … late) and when it ends (VR pose stream on time again after …). 0 disables the gate.

Next steps

Config Parameters

Every setting for teleop, with its default.

Remote Teleop

Drive the robot over the internet from anywhere with Tailscale.

Data Collection

Record teleoperation episodes to a LeRobot dataset.