/diagnostics — open axol.almond.bot/diagnostics (or https://<axol-host>:8001/diagnostics on the LAN) while axol serve is running. Like the control panel, its layout is responsive and works from a phone or tablet.
Motor status
A card per motor shows its status, temperature, bus voltage, and live position, refreshed by the server’s 1 Hz health ping. Axol displays 8 motors per arm; Mantis displays the gripper at ID 8 on each side. A motor reads OK whether enabled or disabled — only genuine error codes (over-temperature, lost comm, …) or an unreachable motor light up red. Clicking a card opens the fullmotor.info readout — model, firmware, control mode, gains, and live readings — without launching a CLI session.
For a MyActuator motor the dialog also has a Parameters tab next to Details: the motor’s whole 0xC0 configuration table laid out like the vendor setup software’s panels — Protect parameters, Plan parameters, Motor parameters, and Motor information — with dropdowns and toggles for the selector settings. Type a new value and press Set (or Enter) to write that one parameter; it is persisted to flash and the table is re-read. If the motor stores a different value than you asked for, the row says so (Motor stored …). Parameters the motor’s firmware doesn’t implement show Not on this firmware. Read-only rows can’t be edited, and protected ones (marked with a lock — factory, calibration and bus-identity settings) stay locked until you switch on Edit protected parameters and confirm the write. The tab reads the motor over the idle diagnostics link, so it isn’t available while an operation owns the robot.
The buttons in this section run the motor plumbing ad hoc: CAN setup for automatic Axol/Mantis hub provisioning (can.setup), CAN enable for the adapter picker described below, and the applicable motor tools. Axol also offers zeroing via motor.set-zero-pos; it is hidden for Mantis because its Damiao grippers self-calibrate.
The same section also carries motor-servicing tools that used to require the vendor’s Windows software: Dump config (motor.dump-config) reads every configuration parameter from one motor — or all of them when the CAN ID is left blank — into the run log, and Set config (motor.set-config) tabs between reading a single parameter and writing it. File-based operations stay on the direct CLI: motor.flash flashes a local firmware image, and motor.restore-config restores a local snapshot. The web API deliberately accepts neither path, so a remote request cannot make the root service read an arbitrary host file.
Choosing CAN adapters
CAN enable opens the adapter picker. Choose Axol or Mantis, which sides are attached, and each side’s SocketCAN interface from the detected list or by name. The defaults arecan_alm_axol_l/r for Axol and can_mantis_l/r for Mantis; a generic adapter such as can0 also works. Save & connect brings the interfaces up and reconnects the diagnostics link. If the selected named hub has not been provisioned yet, the server probes and configures it automatically; silent or ambiguous hardware still asks you to run interactive axol can.setup. Axol mappings are persisted in the shared robot-operation settings; Mantis mappings are diagnostics-only, so inspecting the rig cannot overwrite the robot’s saved channels.
The selected profile and saved mapping drive the whole dashboard: diagnostics and calibration runs do not ask for arm interface names, and with a single side configured the other side is skipped automatically. Mantis mode streams only the two grippers, scopes health and fault checks to them, injects --target mantis into supported helpers, and limits ROM and direct-CAN diagnostics to the gripper. Axol-only lift commands are hidden in Mantis mode; their dialogs can still expose the separate lift/base bus. On the CLI, choose Mantis with --target mantis; can.setup and normal can.enable bring up every configured hub automatically. Arm mappings remain explicit where supported (--left-channel / --right-channel and --no-left / --no-right on two-arm commands, --channel on single-motor tools, and can.enable --channels … for bring-up).
Control loop and CAN timing
The Control loop & CAN timing panel measures what actually reached each arm’s CAN wire. Keep the Diagnostics page open, then startaxol teleop from the panel or a terminal. A new graph begins when PyRoKi has produced its first solution and the shared teleop run loop starts; CAN traffic from Rust bring-up and the PyRoKi compile wait is deliberately excluded. Logging stops with that run. Per-arm cards report command and feedback Hz, p95 command-period jitter, worst interval, p95 per-motor command-to-feedback latency, p95 command-send batch, p95 feedback batch, the full first-command-to-last-feedback CAN cycle, bus occupancy and remaining cycle headroom, missed 240 Hz deadlines, and lost feedback frames. Views cover loop rate, jitter, motor latency, full-arm CAN cycle, worst gaps, and misses. The rate and max-gap views include the loop-rate reference — 240 Hz / 4.17 ms on the impedance controller, 400 Hz / 2.5 ms on the position controller; the observer infers which is running from the measured command rate.
These are passive kernel timestamps, not timing printed by the controller being tested. The observer follows the first commanded joint per arm (every teleop tick commands every joint once), pairs each command with its feedback, and adds no CAN frames of its own. Send batch is first-to-last arm command; full cycle continues through the last arm feedback. Missed 240 Hz counts the nominal 4.17 ms cycles represented by longer command gaps: a clean Rust run stays near 240 Hz and zero misses, while a 120 Hz loop visibly records about one missed 240 Hz cycle per command. A card only reads 240 Hz clean when command and feedback are both within 2% of target, both p95 jitter values are at most 0.5 ms, motor response p95 is at most 3 ms, the CAN cycle retains at least 0.25 ms of p05 headroom, and neither counter reports a miss.
Live telemetry
One chart for the selected arm streams continuously while the robot link is connected, toggled between Position, Velocity, Torque (10 Hz), and Temp (from the 1 Hz health sweep). The server keeps a 10-minute rolling buffer for all of them, so the chart backfills on page load. The filter row scopes it: metric, time window presets (30s–10m), arm, and per-joint toggles; scroll to zoom and drag to pan, which pins the view until you press Go live. The chart’s header button expands it to full screen (Esc to exit) for a closer look. Bus access is split into command and observation. Only one process may command the motors at a time, but the server also keeps a passive, never-transmitting observer socket on each CAN channel: while a diagnostic, tuning run, or teleop session owns command, the observer decodes that task’s own motor traffic and the charts stay live (labeled “live from passive bus tap”). The chart only goes quiet when the running task isn’t actually commanding — e.g. while it sits at a prompt. The observer adds zero bus load (it cannot transmit), and the server’s own idle polling automatically goes silent on any joint another process is driving — including control loops started outside the dashboard, likeaxol teleop run from a terminal. Motion-command traffic on a joint’s CAN IDs is the signal: those joints are dropped from the poll sweeps and charted from the passive tap instead, so the full bus budget stays available to the control rate.
Tests
Each test — thediag.rom-enable soak, the mounted-robot diag.lift-cycle test, and the diag.zed-cable camera-cable check — is a card: click it, set parameters in the dialog, and hit Run. The Axol ROM soak exposes a joint checkbox picker so you can sweep a subset (e.g. just the wrist joints) while the rest of the arm holds home; the Mantis ROM soak automatically uses the gripper-only path. Lift cycle is Axol-only. To check the Mantis triggers against their grippers, run Teleoperation with Mantis selected — it needs only CAN and drives the grippers from the triggers whether or not tracking is set up. All motor-driving diagnostics run the arms through the Rust realtime core — the same controller as teleop. The card shows the running state, elapsed time, and the latest output line; completion lands as a notification, and the full log is kept with the run in the history below. The ROM and lift-cycle tests hand the CAN bus over for the duration of the run; the camera-only ZED cable check leaves motor telemetry streaming.
Lift cycle is the mounted-robot test: enter a positive cycle count, then it ramps the selected arm S1 joints to their mirrored 90° clearance pose, continuously verifies the held arm pose, and runs that many full down/up cycles. A successful run verifies the lift is up and stopped, smoothly returns only S1 to 0° rest while holding every other joint at its measured starting position, then disables the selected arm motors. By default it uses whichever lift bus can.setup pinned (can_alm_axol_c, or the shared wheel bus can_alm_axol_b when there is no chest adapter); its dialog accepts another interface for custom mappings.
Motor-driving runs are fault-gated, but only for the motors they actually touch: a one-arm run, guided zeroing of selected joints, or a single-motor tool isn’t blocked by unreachable or faulted motors outside its scope — e.g. a bench arm with only some motors on the bus. The ROM soak’s joint picker works the same way: an unreachable motor you have deselected is treated as absent and doesn’t block the launch (the run brings up only the motors that answer on the bus), while a fault on a selected joint — or an error state on a reachable motor you left unselected, which the run would still hold — does.
Hands-on steps — like clamping an item in the ROM grasp test — pause the run and surface a Continue button on the card with the instruction (e.g. “Position the item in the RIGHT gripper, then close it.”). The test waits for you and proceeds when you click, so nothing moves until you’re ready. Under the hood the diagnostic prints a [prompt] marker and blocks on stdin; the button writes a line back to it. (The motor.info / motor.health read set needs no action here — the motor cards above show it live.)
Helpers
Utility moves live in their own section, and each command has exactly one place on the page:diag.rom-disable opens the grippers a ROM soak left clamped and powers the arms down, and on robots fitted with Jelly’s telescoping lift, the lift’s lift.home calibration (legs together by default; its dialog’s Independent toggle homes each loose leg against its own stops to level them, off the robot only) and lift.goto install-height move appear as cards here, along with lift.update: pick the lift’s firmware.bin in its dialog, the file uploads to the robot host and is checked, then Run installs it over CAN (run it without a file to see the lift’s current firmware). The tuning suites are the workbench’s tabs below rather than cards.
Powering the robot host off or rebooting it is done from the control panel’s Axol Host tile, which carries the Restart and Shut down buttons whenever a host is connected.
Tuning
The tuning workbench runs the deterministic tuning suites inline — no dialogs, and every parameter is always visible. Pick what to run across the tabs, set the numbers, and hit Run. Sine and step probes chart live while they run — commanded vs actual (with the error lane) streams in at ~25 Hz as the probe executes — and the full-resolution charts and scorecards land below when the run finishes. Gain fields (kp, kd, kd_host, kd_host_hz, kd_host_q) show the selected joint’s current config value — the shipping defaults with this robot’s calibration overlaid, i.e. exactly what runs when the field is left empty — and a slider seeded there, so tuning is “nudge from where production is” rather than typing blind. Angles are degrees everywhere in the workbench (fields, charts, scores); gains keep their per-radian units (Nm/rad, Nm·s/rad) because that’s what the config and firmware speak:- Sine / Step —
tune.pidprobes on one joint. Leave the gain boxes empty to test the configured gains; space-separated kp/kd values sweep the grid, one saved run per candidate. kd_host, its band-pass centre kd_host_hz (aim it at the ring frequency the step scorecard reports), and the band’s width kd_host_q (bandwidth = centre/q — raise to 2–3 when the centre is pinned on a measured ring, so the damping stops dragging the slow final approach) are all overridable per run. The scorecard’s holder wobble column reports how far the other joints — held on impedance during the probe (--a4-holdersholds them on the firmware position servo instead) — moved from their baseline; past ~0.5° the structure was flexing and part of the measured ring came from a compliant neighbour, not the joint under test. - Friction —
tune.frictionidentifies one joint’s friction model; check save to calibration to write the fit to~/.almond/calibration.json. - Gravity —
tune.gravityfits one link’s real centre of mass from a friction-cancelled torque sweep, correcting the gravity feedforward — this removes the static droop a joint shows when parked under load (droop = unmodeled torque / kp), which no kp/kd tuning fixes cleanly. The chart is torque vs. joint angle: measured, model at CAD, and model after the fit, with the residuals in the lane. Run distal→proximal (wrist_3 first, shoulder_1 last), after friction. - Factory —
tune.factoryruns friction + gravity for all 14 joints in one go (both arms, distal→proximal), saving every fit to this robot’s calibration and uploading the document to the cloud keyed by the hub adapter serial when Supabase credentials are configured. This is the one-button factory calibration; expect a long run.
- Recorded motion —
tune.motionreplays a committed reference motion through the production Rust-core control path and scores tracking per joint. The gain table shows every joint’s current config value (defaults + this robot’s calibration) for each gain field; edit a cell to override it for this run only, on both arms — edited cells highlight, everything else runs with config. Clearing a cell drops the override and snaps it back to the config value (reset overrides clears them all). The controller select picks the control law for the whole run: impedance (the production MIT frame with the host feed-forward, 240 Hz) or position (--controller position: every joint on its motor’s own firmware position loop — MyActuator 0xA4, Damiao position-velocity, the gains on the Firmware-loop tab — streamed at 400 Hz). The controller per joint grid below it picks, inside the impedance controller, per arm and MyActuator joint, whether that joint runs on impedance or on its firmware loop (--a4 side.joint) for this run — e.g. shoulder_1 on the firmware loop with everything else on impedance, scored against the identical motion; a joint the config already pins towire_mode a4shows as firmware and cannot be switched back. Tick run as IK to drive the run through the IK solver instead of raw joint replay: the motion’s Cartesian end-effector path (FK of the reference, with elbow hints) is re-solved to joints like teleop’s pose→joints loop and the arms execute the solver’s output — still scored against the clean reference, so IK reconstruction error and tracking show up together in the same charts. The tab also carries the filter-stack test on real hardware: pick a noise source (network = seeded jitter/outliers/stalls injected before the pose low-pass, IK = solver churn/jumps injected after it, or combined — each at its real entry point in the pipeline, deterministic per seed) and toggle the filter stack (pose low-pass → EMA → trapezoid). The run is scored against the clean reference either way, and the charts overlay commanded (clean), sent (what was actually streamed), and actual per joint — run once with the filters off and once on to see, on the physical arm, what the stack removes and what it costs in lag. - Build motion —
motion.buildturns a recorded session — teleoperated (axol teleop --teleop.record) or hand-guided in gravity comp (axol gravity-comp --record, also settable per run on the gravity-comp operation panel) — into a reference motion, which then shows up in the Recorded motion picker. Every build saves a before/after run: the raw recorded command stream vs the built motion per joint (with a built−recorded lane), so the smoothing pass and the collision projection are visually checkable — with the same zoom/fullscreen as every other chart — before the motion is replayed. - IK —
diag.offline kinematicsanalyzes a teleop recording offline (no hardware, defaults to the newest recording): per-axis charts of the EE pose set vs actual (the world target given to the solver vs FK of what it solved) with an error lane in mm, the per-tick solve time trace (spikes here surface as teleop lurches), and a per-joint churn/jitter scorecard that catches a restless null space.
diag.offline).
Every run is persisted under ~/.almond/diagnostics/tuning/ on the serve host — the CLI (--save-run) and the dashboard share the same store — and the Past runs list reselects or deletes them at any time.
Run history
Every diagnostic test (the cards above — not the ad-hoc CAN/calibration buttons) is recorded as a run: command, arguments, duration, exit status, log, and the telemetry observed while it executed (persisted under~/.almond/diagnostics/runs on the serve host; the Clear button deletes the history). The list is paginated newest-first (10 runs per page). The ROM test additionally writes its own 5 Hz position/torque capture from its cached motor state, which is preferred over the passive-observer frames when charting — opening a run charts its capture with the same joint filter and zoom/pan as the live charts.