Files
chenlin ceca51b0d8
Checks / Layout, translation and Colab checks (push) Has been cancelled
Checks / Ruff (push) Has been cancelled
Checks / Framework tests (native backend, CPU) (push) Has been cancelled
Checks / Portable Rust controller (push) Has been cancelled
Improve generic robot onboarding, walking controls and end-effector interaction
Default humanoid intake to command-following gait and support bounded Cartesian target dragging for arms. Carry model and actuator semantics through calibration, native verification and packaging. Document reusable intake decisions and cover them with regression tests.
2026-11-06 10:54:58 +08:00

3.4 KiB

Operator controls

controller/vocabulary.json defines the names used by the gamepad and keyboard readers. Print them with python -m controller --vocabulary. controller.controls.load_controls validates the shared schema_version: 2 format.

Training template controls

Generate a body-frame locomotion binding beside the robot asset:

python .claude/skills/controls/scripts/write_controls.py --task my_robot \
  --out assets/my_robot/controls.yaml

The generated command is twist feeding velocity_commands. Its axes are lin_vel_x, lin_vel_y, and ang_vel_z, in that order. Pad deflections scale to the command range, with separate positive and negative ends. Keyboard keys bind directly to an axis direction and ramp while held. For a registered task the generator selects a layout by family. Bipeds and as-yet unregistered robots default to walking; other legged families use omni, except wheeled bipeds which use differential. An explicit --profile overrides that choice.

The walking layout uses W/S for forward/reverse and A/D for left/right yaw. The arrow keys mirror WASD. J/L request left/right lateral motion on policies that support it. The pad uses Ly for forward speed, Lx for lateral speed and Rx for yaw. Holding W+A combines forward and left turn. The explicit omni layout keeps A/D for lateral motion and J/L for yaw. Use --profile differential for a wheeled biped; its lateral axis is unbound on the keyboard and has a zero command range in the template.

Use --bind AXIS=SOURCE+ or --bind AXIS=SOURCE- for explicit signs and --release BUTTON for the pad release. Review the direction of motion and release behavior in simulation. Point the robot specification's controls_path to the generated file.

Deterministic controller controls

onboard.py setup-tier2 generates controls for the controller's named axes. The config's optional operator_axes selects up to four axes. Remaining joint sliders or targets can be addressed through the programmatic controller. Run native verification with the exact controller and controls paths. After changing bindings, generate a fresh report before packaging.

For end-effector IK, double-click the tool target marker, then hold Ctrl and drag with the right mouse button. Add Shift to change the translation plane. Release the button to hold the selected target.

Shared schema

The schema describes a list of command terms and the bindings for both devices. Each command declares its name, observation or controller feed, frame, ordered axes, and the full-deflection scaling rule. Each axis names its units and positive direction. The gamepad declares a signed source, and the keyboard declares key lists for both directions or an explicit reason that the axis is unbound.

The parser also supports split travel, shifted bindings, key chords, integrated axes with reset, and auxiliary controls for compatible consumers. A consumer can restrict that schema to the features it implements. The locked locomotion templates take one body-frame velocity command with zero rests and no auxiliary controls. Native mode selection uses --app-mode. Legacy FSM mode switches are separate from a policy's own device bindings and are declared in the legacy deployment manifest.

python -c "from pathlib import Path; from controller.controls import load_controls; load_controls(Path('assets/my_robot/controls.yaml'))"
python -m pytest tests/test_controls.py -q