almond_axol.teleop

Connect a VR headset to the robot (or simulator) for teleoperation.

Connects a VR headset to the robot (or simulator) for teleoperation. IK runs in a dedicated subprocess to keep JAX off the asyncio event loop. The pipeline is: VR frames → One Euro filtering → IK solve → EMA smoothing → trapezoidal velocity profiling → motion_control().

from almond_axol.teleop import VRTeleop, VRTeleopConfig

import asyncio
from almond_axol.robot import Axol
from almond_axol.teleop import VRTeleop

async def main():
    async with VRTeleop(Axol()) as teleop:
        await teleop.run()  # blocking; Ctrl+C to exit

asyncio.run(main())

Use step() instead of run() to integrate teleoperation into your own control loop:

async with VRTeleop(axol) as teleop:
    while True:
        left_q, right_q = teleop.step()  # returns latest smoothed (8,) arrays
        # ... custom logic ...
        await asyncio.sleep(1 / 120)

To show camera feeds in the headset, register frame sources after the session is enabled — VRTeleop.set_video_sources() forwards them to the VR server’s WebRTC relay (requires the GStreamer NVENC stack, installed by axol gst.install; see almond_axol.vr). Each value is a connected ZedCamera (registered directly) or any object exposing width / height / fps + wait_next:

async with VRTeleop(axol) as teleop:
    teleop.set_video_sources({"overhead": cam})
    await teleop.run()

See the teleop CLI command for the equivalent command-line entry point (--cameras wires the ZED cameras up automatically).

set_video_sources() runs the WebRTC relay in-process. The CLI instead prefers VRTeleop.set_video_manager(), which hands the camera grab/encode and WebRTC off to a dedicated subprocess (almond_axol.video.video_proc.VideoRelayProcess) so it can’t perturb the control loops. Both deliver the same headset video — see almond_axol.vr.

VRTeleopConfig fields

Field	Default	Description
frequency	120 Hz	Control loop rate used by run() and reset trajectory density
teleop_max_vel	1.0 rev/s	Trapezoidal filter velocity cap during normal teleoperation
teleop_max_accel	3.5 rev/s²	Trapezoidal filter acceleration cap
engage_max_vel	0.1 rev/s	Starting velocity cap when teleop is first engaged after a reset; smoothsteps up to teleop_max_vel over engage_duration
engage_duration	1.0 s	Seconds over which the velocity cap ramps (smoothstep) from engage_max_vel up to teleop_max_vel after engaging
ik_alpha	0.3	EMA blend factor on IK output; 1.0 disables smoothing
pose_min_cutoff	0.8 Hz	One Euro Filter tremor cutoff for raw VR poses
pose_beta	2.0	One Euro Filter speed coefficient (raises cutoff during fast moves)
position_multiplier	1.0	Scale on hand position (not orientation): the EE target moves this many times as far as the hand. >1 extends reach when the arm is longer than the operator’s
rotation_multiplier	1.0	Scale on hand orientation (not position): the EE target rotates this many times as far as the hand. >1 rotates the arm further than the wrist
reset_speed	0.1 rev/s	Average joint velocity of the worst-case joint during return-to-rest (peak is 1.5× this)
reset_min_duration	1.5 s	Floor on return-to-rest duration so near-rest starts don’t snap home
rest_pose_left / rest_pose_right	near-zero	Reset target for each arm, shape (7,) in ARM_JOINTS order

Engage behaviour. From rest, both grip buttons together engage both arms. Once engaged, each grip acts per arm — a click toggles that arm between tracking and frozen (a frozen arm holds its pose and grasp while the other keeps tracking). Set hold_to_engage for a dead-man scheme instead: hold both grips to start, release one to freeze that arm, hold on to keep it going. A rising edge on the reset button triggers a collision-aware trajectory back to the rest pose.