release: v0.1 generic robot training and application platform
Checks / Layout, translation and Colab checks (push) Has been cancelled
Checks / Ruff (push) Has been cancelled
Checks / Framework tests (native backend, CPU) (push) Has been cancelled
Checks / Portable Rust controller (push) Has been cancelled

This commit is contained in:
2026-11-05 11:05:01 +08:00
commit c32df70326
662 changed files with 134260 additions and 0 deletions
+100
View File
@@ -0,0 +1,100 @@
---
name: bundle-manual
description: Translate a built bundle's manual.en.json into Chinese as manual.zh.json and pack it into the .app. Use right after `python scripts/deploy.py` writes a bundle -- every bundle an agent builds ships with the Chinese manual as well as the English one -- and whenever a bundle is found with manual.en.json and no manual.zh.json, or when asked for "操作说明", "中文说明", "按键说明" or "manual" of a bundle/app.
---
# The bundle's manual, in Chinese
`scripts/deploy.py` writes `manual.en.json` (`kk-bundle-manual/1`) into every
bundle: each pad control and each key the bundle reads, what it does in each
mode, how each mode is reached and left, and every mode switch. It is generated
from the controller's own account of its controls (`padGuide`), so it cannot
disagree with the bindings. The pad and the keyboard are two paths since
2026-09-29 and the manual lists them apart: in each mode `controls` is the pad
and `keys` the keyboard, and each switch says which `device` it is on. The
Chinese version is a translation of it, and this skill is how it is made: the
build cannot write natural Chinese, and a person or an agent can.
## Steps
1. Find the bundle directory. `deploy.py` prints it
(`[deploy] bundled <commit> -> …/out/bundle_<ts>/<name>`) and then prints the
`--translate` command with that path already filled in. It holds
`manual.en.json` and `bundle.json`.
2. Read `manual.en.json` in full.
3. Write `manual.zh.json` somewhere outside the bundle (a scratch file):
- the same JSON, field for field, in the same order;
- `"language": "zh"`;
- translate **only** the text fields: `source`, `summary`, `enter`, `leave`,
`keyboard`, `pad`, `does` -- and `does` wherever it is: in each mode's
`controls` (the pad), in each mode's `keys` (the keyboard), and in
`switches`. A `pad` that is `null` (a keyboard switch) stays `null`;
- leave every other value exactly as it is -- `schema`, `bundle`, `mode`,
`task`, `default`, `control`, `with`, `device`, `from`, the `key` of each
entry in a mode's `keys` (`W`, `;`, `Space`, `↑`), and the `keys`
lists of `switches` (`Space`, `Ctrl + 1`, `Num 1`). They are identifiers,
and the check below refuses a translation that changes one.
4. Add it to the bundle, which checks it and packs the `.app` again:
```bash
python scripts/deploy.py --translate out/bundle_<ts>/<name> --language zh --manual <scratch file>
```
It refuses a file whose non-text fields differ from the English one, and
holds the result to `deploy/app.schema`. Fix what it names and run it again.
5. If the `.app` is going to be committed (bundles are committed under
`out/bundle_<ts>/<name>.app`), commit it after this step, not before: the
translation changes the archive.
## How to write it
For someone holding a gamepad or sitting at a keyboard, in plain Chinese. Keep
mode names as identifiers where they name a mode (`进入 jump`), and say what the
mode is in the sentence when the English does. The manual lists each mode's
keys on their own; say what a key does, never which pad control it is -- the
English never says that either.
| English | 中文 |
|---|---|
| left stick / right stick | 左摇杆 / 右摇杆 |
| up and down / left and right | 上下 / 左右 |
| up: …; down: … | 上:……;下:…… |
| left trigger (LT) / right trigger (RT) | 左扳机(LT)/ 右扳机(RT) |
| left bumper (LB) / right bumper (RB) | 左肩键(LB)/ 右肩键(RB) |
| right stick click (R3) | 右摇杆按下(R3) |
| D-pad / D-pad up | 十字键 / 十字键上 |
| hold Menu, then D-pad up | 按住 Menu 再按十字键上 |
| Menu, pressed and let go on its own | 单独按下再松开 Menu |
| walk forward / walk back | 前进 / 后退 |
| step left / step right | 左移 / 右移 |
| turn left / turn right | 左转 / 右转 |
| twist left / twist right | 左扭 / 右扭 |
| nose down / nose up | 低头 / 抬头 |
| roll, right side down / left side down | 横滚,右侧低 / 横滚,左侧低 |
| stand taller / crouch lower | 升高 / 降低 |
| (moves it; let go, it stays) | (持续移动,松开即停留) |
| with right stick click (R3) held -- … | 按住右摇杆(R3)时——…… |
| hold for the other stick layer | 按住切换到摇杆的另一层 |
| tapped -- pressed and let go without the stick moving -- puts the moved axes back at rest | 轻点(按下后摇杆不动就松开):被移动的量回到静止 |
| let go of everything: every control back to rest | 全部松开:所有操作回到静止 |
| switch to X; press again to switch back | 进入 X;再按一次退出 |
| (from X, Y only) | (仅在 X、Y 中有效) |
| leave X, Y -- whichever is on -- back to locomotion | 退出 X、Y 中正在进行的那一个,回到 locomotion |
| click A once / double-click A / triple-click A | 单击 A / 双击 A / 三连击 A |
| click A four times / click A five times | 四连击 A / 五连击 A |
| switch to X; double-click again to switch back | 进入 X;再双击一次退出 |
A task's own words -- `closes the left claw as far as it is held`, from its
`controls.yaml` -- are translated like any other text: `按多深,左爪就夹多紧`;
`holds the carried arm straight out, thumb up, while held` is
`按住时手臂向前伸直,拇指朝上`, and thumb-web up is `虎口朝上`. A claw mode lists
only its own side's claw (the left claw's lists `LT` and Space, not `RT`), so
translate what is there and add nothing.
## Why a separate step
The English is the build's and the Chinese is a translation of it, never the
other way round: a manual written in Chinese first would have no generator to
hold it to the bindings. `--translate` is what keeps the two the same manual in
two languages -- it compares every identifier -- and it is also why a bundle
built without an agent still ships the English one.
+34
View File
@@ -0,0 +1,34 @@
---
name: controls
description: Configure operator keys, pad bindings, releases, or legacy FSM mode switches using the shared control vocabulary and schema.
---
# Bind operator input
Read [Controls](../../../docs/CONTROLS.md). Use
`python -m controller --vocabulary` as the source of device names.
For the locked locomotion templates, generate the ordered body-frame
velocity binding in the robot's asset directory:
```bash
python .claude/skills/controls/scripts/write_controls.py --task <robot_id>
```
Use `--profile differential` for `wheeled_biped`. W/S drive forward and
backward and A/D turn left and right. The keyboard leaves lateral motion
unbound. The required Lx pad channel stays in the layout and has zero range.
The default `omni` profile keeps A/D for lateral motion and J/L for turning.
Use `--out` to specify another destination file. Keep all three velocity
channels in order. Use `--bind AXIS=SOURCE+` or `--bind AXIS=SOURCE-` for
named changes and `--release BUTTON` when specified. Point `robot_spec.py`
at this file.
For deterministic modes, generate the binding with `onboard.py setup-tier2`.
Validate the final YAML with `controller.controls.load_controls`. Exercise
both ends of the active axes and release on the selected model. Repeat native
verification after changing bindings that a portable app will package.
A legacy FSM mode switch belongs in the deployment manifest. Check it against
the mode's operator bindings and validate the legacy bundle. Native app mode
selection uses `--app-mode`.
@@ -0,0 +1,293 @@
#!/usr/bin/env python3
"""Write a locomotion binding using the shared keyboard and gamepad vocabulary."""
from __future__ import annotations
import argparse
import re
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parents[4]
sys.path.insert(0, str(REPO))
# After the path insert, necessarily: this runs from a skill directory, not from
# an installed package, and `controller` is at the repository root.
from controller import vocabulary # noqa: E402
#: Command axis -> (unit, which way is positive).
#:
#: The velocity-tracking vocabulary the observation term uses; the directions
#: are the body frame's, ROS REP-103.
AXIS_MEANING = {
"lin_vel_x": ("m/s", "forward"),
"lin_vel_y": ("m/s", "left"),
"ang_vel_z": ("rad/s", "counter-clockwise"),
}
#: Which stick drives which axis, by default, and with what sign.
#:
#: evdev reports positive stick motion to the right and downward.
#: Confirm these signs on the intended input device during verification.
DEFAULT_BINDING = {
"lin_vel_x": ("Ly", -1),
"lin_vel_y": ("Lx", -1),
"ang_vel_z": ("Rx", -1),
}
#: The keyboard: command axis -> (the keys towards its positive direction, the
#: keys away from it). W S and the arrows walk forward and back, A D and the
#: arrows step left and right (the arrows asked for on 2026-09-28), J L turn --
#: the layout as the
#: user decided it on 2026-09-29. Two keys on one direction are summed while
#: both are held. Letting go of the keys is taking your hands off; there is no
#: key for it (`centre`, retired 2026-09-27).
#:
#: Every letter is also one of MuJoCo's viewer shortcuts -- W flips the scene to
#: wireframe -- and that was accepted for a keyboard laid out for the hands; the
#: dictionary's `_keys_note` has the table.
KEYS_FOR = {
"lin_vel_x": (("key_w", "key_up"), ("key_s", "key_down")),
"lin_vel_y": (("key_a", "key_left"), ("key_d", "key_right")),
"ang_vel_z": (("key_j",), ("key_l",)),
}
DIFFERENTIAL_KEYS = {
"lin_vel_x": KEYS_FOR["lin_vel_x"],
"lin_vel_y": None,
"ang_vel_z": (("key_a", "key_left"), ("key_d", "key_right")),
}
KeyDirections = tuple[tuple[str, ...], tuple[str, ...]]
#: The keys that let go of everything, as the pad's release button does: Esc.
#: The keyboard's own, whatever button `--release` gives the pad. It was B until
#: Control-agent 3.1 put the right claw there (2026-09-29).
RELEASE_KEYS = ("key_escape",)
#: Seconds a key is held for its direction to reach full deflection; it is back
#: at rest when the key comes up. Asked for on 2026-09-26, measured against nothing.
FULL_AFTER_S = 2.0
def parse_bind(entry: str) -> tuple[str, str, int]:
"""`lin_vel_x=Ly-` -> ("lin_vel_x", "Ly", -1)."""
if "=" not in entry:
raise SystemExit(f"[controls] --bind wants AXIS=SOURCE[+|-], got {entry!r}")
axis, source = entry.split("=", 1)
if not source.endswith(("+", "-")):
raise SystemExit(
f"[controls] --bind {entry!r} has no sign. Write `{axis}={source}-` or "
f"`{axis}={source}+`.\n"
f" Not defaulted: a sign is the one value here whose wrong "
f"answer looks exactly like the right one."
)
return axis, source[:-1], (-1 if source.endswith("-") else 1)
def check(name: str, kind: str, allowed) -> None:
if name in allowed:
return
absent = vocabulary.absent()
extra = f"\n {absent[name]}" if name in absent else ""
raise SystemExit(
f"[controls] {name!r} is not {kind} the robot's pad service publishes.\n"
f" It publishes: {' '.join(allowed)}{extra}\n"
f" python -m controller --vocabulary"
)
def keyboard_for(axes: list[str], profile: str) -> dict[str, KeyDirections | None]:
"""Check each selected keystroke against the dictionary."""
mapping = DIFFERENTIAL_KEYS if profile == "differential" else KEYS_FOR
out = {}
for axis in axes:
directions = mapping[axis]
if directions is not None:
for stroke in (*directions[0], *directions[1]):
try:
vocabulary.keystroke(stroke)
except ValueError as exc:
raise SystemExit(f"[controls] {exc}\n python -m controller "
"--vocabulary") from None
out[axis] = directions
return out
def render(task: str, axes: list[str], bind: dict[str, tuple[str, int]],
release: str, keys: dict[str, KeyDirections | None]) -> str:
out = [
f"# {task} operator controls -- how a person's input becomes the command the",
"# policy observes.",
"#",
"# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes",
"# every control name from `controller/vocabulary.json`. Run",
"# `python -m controller --vocabulary` to see the list; it is the same one the",
"# deployed controller refuses a binding against.",
"#",
"# The pad and the keyboard are two paths, each bound straight to the command",
"# axes -- a stick with a sign, a key with the direction it pushes -- and",
"# neither names a control of the other. Either device drives alone; the one",
"# touched last drives.",
"#",
"# Two rules this file inherits from the specification it replaced:",
"# * Every value carries its source. A number without one cannot be told",
"# apart from a guess.",
"# * A consumer fails on a missing or malformed value rather than",
"# substituting a default. A sign that defaults to +1 is a robot that walks",
"# backwards while the screen looks perfectly normal.",
"",
"schema_version: 2",
"",
"command:",
" - term: twist",
" # The observation term these numbers are written into.",
" feeds: velocity_commands",
" # The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).",
" frame: body",
" # Order is the order the command term writes them, and therefore the order",
" # they appear in the observation.",
" axes:",
]
for axis in axes:
unit, positive = AXIS_MEANING[axis]
out += [f" - name: {axis}", f" unit: {unit}", f" positive: {positive}"]
out += [
" scaling:",
" # Full deflection is the edge of the range the policy was **trained** on,",
" # not an absolute speed -- so a stick means \"as fast as this policy was",
" # ever asked to go\" and cannot ask for something off-distribution.",
" rule: full_deflection_is_range_edge",
" formula: axis * (range_max if axis >= 0 else -range_min)",
" # A range like (-0.5, 1.0) gives forward twice what it gives back, which is",
" # what the policy trained on; taking the larger end for both quietly",
" # commands something it never saw.",
" ends_scaled_separately: true",
"",
"devices:",
" gamepad:",
" # The stick position *is* the command.",
" scheme: absolute",
" layout: xbox",
" axes:",
]
for axis in axes:
source, sign = bind[axis]
if keys[axis] is None:
out.append(" # The wheeled template scales this required pad channel to zero.")
out.append(f" {axis}: {{ source: {source}, sign: {sign} }}")
out += [
" # MEASURED: the signs are not measured against the robot's pad service.",
" # `controller/xbox.py` states the same evdev convention -- ABS_Y positive",
" # is **down**, ABS_X positive is **right** -- and the service passes both",
" # through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on",
" # the robot is what would settle it.",
" #",
" # The service applies a 0.15 deadband and **rescales**, so full deflection",
" # still reaches ±1. A consumer that applies its own on top narrows the",
" # range a person can ask for and cannot tell that it has.",
" deadzone: device_reported_rescaled",
" # Hands the command back to the random sampler in `play --task`; on a robot, which",
" # has no sampler, it lets go of everything.",
f" release_button: {release}",
"",
" keyboard:",
" # Keystrokes bound to what they do: each pushes one direction of one axis,",
" # \"+\" being the axis's own positive direction as `positive:` above words",
" # it, so the keyboard has no signs of its own -- there is no stick for a",
" # sign to be relative to. A virtual pad read through the pad's mapping",
" # until 2026-09-29.",
" scheme: keys",
" # Seconds a key is held for full deflection: the axis climbs linearly",
" # until then and is back at rest the moment the key comes up, as a stick",
" # let go of. Asked for on 2026-09-26; measured against nothing.",
f" full_after_s: {FULL_AFTER_S}",
" axes:",
]
for axis, directions in keys.items():
if directions is None:
out.append(f" {axis}: {{unbound: lateral command range is zero}}")
else:
plus, minus = directions
out.append(f' {axis}: {{ "+": [{", ".join(plus)}], "-": [{", ".join(minus)}] }}')
out += [
" # Lets go of everything, as the pad's release button does.",
f" release: [{', '.join(RELEASE_KEYS)}]",
"",
]
return "\n".join(out)
def main() -> int:
ap = argparse.ArgumentParser(prog="write_controls.py", description=__doc__.splitlines()[0])
ap.add_argument("--task", required=True, help="robot id, e.g. sample_biped")
ap.add_argument(
"--profile", choices=("omni", "differential"), default="omni",
help="keyboard profile. Use differential for the wheeled-biped template",
)
ap.add_argument("--axes", default=",".join(AXIS_MEANING),
help="command axes in order (default: all three)")
ap.add_argument("--bind", action="append", default=[], metavar="AXIS=SOURCE[+|-]",
help="which stick drives an axis and with what sign. Repeat. "
"Omitted axes take the default binding.")
ap.add_argument("--release", default="B",
help="the button that hands control back to the sampler")
ap.add_argument("--out", type=Path, help="output (default: assets/<robot_id>/controls.yaml)")
ap.add_argument("--force", action="store_true", help="overwrite an existing file")
ap.add_argument("--print", action="store_true", help="write nothing; print it")
args = ap.parse_args()
if re.fullmatch(r"[a-z][a-z0-9_]*", args.task) is None:
ap.error("--task must be a lowercase robot id")
axes = [a.strip() for a in args.axes.split(",") if a.strip()]
if args.profile == "differential" and axes != list(AXIS_MEANING):
ap.error("the differential profile requires all three command axes in template order")
for axis in axes:
if axis not in AXIS_MEANING:
raise SystemExit(
f"[controls] {axis!r} is not a command axis this environment writes.\n"
f" It writes: {' '.join(AXIS_MEANING)}"
)
known_axes = vocabulary.axes()
bind = {a: DEFAULT_BINDING[a] for a in axes}
for entry in args.bind:
axis, source, sign = parse_bind(entry)
if axis not in axes:
raise SystemExit(f"[controls] --bind names {axis!r}, which is not in --axes")
if args.profile == "differential" and axis == "lin_vel_y":
ap.error("the differential profile has a zero lateral command range")
check(source, "an axis", known_axes)
bind[axis] = (source, sign)
driven = [bind[a][0] for a in axes]
if len(set(driven)) != len(driven):
raise SystemExit(
f"[controls] two command axes share a stick: {driven}. Splitting a stick's "
f"travel between two axes is an explicit schema edit."
)
check(args.release, "a button", vocabulary.buttons())
text = render(args.task, axes, bind, args.release, keyboard_for(axes, args.profile))
if args.print:
print(text, end="")
return 0
out = args.out or REPO / "assets" / args.task / "controls.yaml"
if out.exists() and not args.force:
raise SystemExit(f"[controls] {out} exists; pass --force to replace it, or --out")
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(text, "utf-8")
shown = out.relative_to(REPO) if out.is_relative_to(REPO) else out
print(f"[controls] wrote {shown}")
for axis in axes:
source, sign = bind[axis]
disabled = " (zero range)" if args.profile == "differential" and axis == "lin_vel_y" else ""
print(f" {axis:<12}{'-' if sign < 0 else '+'}{source}{disabled}")
print(f" release on {args.release}, and on the keyboard "
f"{' '.join(RELEASE_KEYS)}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+66
View File
@@ -0,0 +1,66 @@
---
name: deploy
description: Export, verify and package a robot controller as a native MuJoCo .app or a supported legacy board/web bundle. Use for deployment contracts, ONNX/RKNN conversion, runtime parity, bundle creation, or an explicitly requested hardware deployment.
---
# Verified application packaging
Read [deploy/README.md](../../../deploy/README.md) and
[deploy/BUNDLE.md](../../../deploy/BUNDLE.md). A directory containing an ONNX file
is an export, not an accepted application.
## Native application
1. Identify the robot ID, normalized MJCF and control mode. Use the onboarding
skill for source assets. Do not infer a home pose or motor mapping from order.
2. For a trained or adapted policy, inspect its `layout.json`, numerical parity
report and source provenance. Preserve wire/observation/action named orders,
term histories, transforms, rates and gain semantics.
3. Write explicit native physical-response probes and run `onboard.py verify`.
Require both physics and control acceptance for a controlled capability.
Changing tolerances solely to turn a failure green is not acceptance.
4. Run `deploy.py --robot <id> --model <normalized.xml> --control
<controller.json> --controls <controls.yaml> --verification <report.json>
--bundle <output-parent>`. Add `--policy <export>` for an ONNX mode.
Geometry-only display omits controller/verification and claims `model_viewer`.
5. Replay the emitted app and run `deploy.py --check-reference <app>`. Record
actual measured results and supported runtime, including missing capabilities.
6. Follow [bundle-manual](../bundle-manual/SKILL.md) to add the Chinese manual
through `deploy.py --translate`; report the resulting archive path.
The portable native runtime needs host Python and declared dependencies. It is
not a browser or board executable. Do not mark an unsupported host as available.
## Legacy Rust board/web application
1. Export the checkpoint through `scripts/export.py`. Refuse contracts the target
reader does not implement; native hybrid position/velocity contracts are not
legacy scalar-action contracts.
2. Choose explicit export paths, cascade and switches in `deploy/manifests.json`
or repeated `--mode` arguments. The generic repository defines no default robot.
Use the [controls skill](../controls/SKILL.md) for binding changes.
3. For a real board, run the read-only preflight script with the explicitly supplied
`--host` and machine configuration. Check software ABI, DDS/QoS and named motor
order before a build or transfer. No address is inferred.
4. Build through `scripts/deploy.py`. Its default legacy path performs RKNN
conversion, cross-builds, contract checks and reference generation. Follow
`deploy/convert/README.md` and `deploy/fsm/README.md` for toolchains.
5. `--allow-incomplete` is a development build with recorded omissions. Do not
describe one as board-ready. Add its Chinese manual before handoff too.
6. Hardware operation is separate from building. When explicitly authorized,
run the board controller's `--dry-run` then `--check-reference` before the
operational command with the machine's configuration. Stop on mismatch.
## Input conflicts
A mode's controls and the FSM's switches share a vocabulary and must not consume
one input twice. The legacy reader reports:
```
switches on the pad's ... controls use too: one press would do both.
Pick another control, or say with `from` that the switch is not pressed in
```
Change the binding or explicitly restrict the switch's source modes. Do not remove
validation or widen a contract to hide the conflict. Unknown observation terms,
joint names, unsupported versions and missing trajectory companions fail closed.
@@ -0,0 +1,250 @@
#!/usr/bin/env python3
"""Read-only preflight: does this board match what we are about to build for it?
Every mismatch this looks for is one that **does not raise on the robot**. A
CycloneDDS soname that differs makes the binary fail to load; an IDL that differs
makes the DDS reader never pair, leaving the topic empty forever; a joint order
that differs drives the wrong joints. None of them announce themselves, and the
last one moves a real machine.
Standard library plus `ssh` only, so it runs without this skill and without the
repository's virtualenv:
python3 .claude/skills/deploy/scripts/check_board.py --host <user>@<board>
python3 .claude/skills/deploy/scripts/check_board.py --host <user>@<board> \\
--bundle out/my_robot/<name>
`--host` has no default: a board's address belongs to the network it is on, not
to this repository.
**Read-only. It runs no binary on the board and publishes nothing.** The robot's
own controller usually runs while this does, and two controllers on one motor bus
is not a thing to find out about experimentally.
Exit status: 0 all checks passed, 1 a check failed, 2 could not reach the board.
"""
from __future__ import annotations
import argparse
import json
import re
import shutil
import subprocess
import tempfile
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parents[4]
#: What this repository builds against. Each is compared, not assumed.
EXPECT_RKNN = "2.3.2" # deploy/fsm/vendor/rknpu2 and convert/requirements.txt
EXPECT_DDS_SONAME = "libddsc.so.11"
class Result:
def __init__(self) -> None:
self.rows: list[tuple[str, bool | None, str]] = []
def add(self, name: str, ok: bool | None, detail: str) -> None:
self.rows.append((name, ok, detail))
def report(self) -> int:
width = max(len(n) for n, _, _ in self.rows)
for name, ok, detail in self.rows:
mark = {True: "ok ", False: "FAIL", None: "-- "}[ok]
print(f" [{mark}] {name:<{width}} {detail}")
failed = [n for n, ok, _ in self.rows if ok is False]
skipped = [n for n, ok, _ in self.rows if ok is None]
print()
if failed:
print(f"FAILED: {', '.join(failed)}")
return 1
print("all checks passed" + (f" ({len(skipped)} skipped)" if skipped else ""))
return 0
def ssh(host: str, command: str, timeout: int = 30) -> tuple[int, str]:
"""One read-only command on the board."""
proc = subprocess.run(
["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8", host, command],
capture_output=True,
text=True,
timeout=timeout,
)
return proc.returncode, proc.stdout.strip()
def check_reachable(host: str) -> str | None:
code, out = ssh(host, "uname -srm; . /etc/os-release 2>/dev/null && echo $VERSION")
if code != 0:
return None
return out
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
ap.add_argument("--host", required=True, help="ssh target, <user>@<board>")
ap.add_argument(
"--bundle",
type=Path,
help="an exported policy to check the board's joint order against "
"(the directory holding layout.json)",
)
ap.add_argument("--config", default=None,
help="a file on the board holding [robot] joint_names, to check the "
"bundle's wire order against. There is no default: the path this "
"had belonged to a controller that is not part of the product, "
"and a default pointing at it made the check look like a step "
"rather than a thing you can do if you happen to have such a file")
args = ap.parse_args()
if shutil.which("ssh") is None:
print("ssh not found", file=sys.stderr)
return 2
print(f"board: {args.host}\n")
banner = check_reachable(args.host)
if banner is None:
print(f"cannot reach {args.host} over ssh (key auth, BatchMode).", file=sys.stderr)
print(" Check the address, the route, and that your key is authorised.", file=sys.stderr)
return 2
r = Result()
r.add("reachable", True, banner.replace("\n", " "))
# ── glibc: a floor, not a ceiling ──────────────────────────────────
# A binary built against a newer glibc than the board's dies at startup
# with "version GLIBC_2.xx not found", nowhere near the build that caused it.
_, out = ssh(args.host, "ldd --version | head -1")
m = re.search(r"(\d+\.\d+)\s*$", out)
r.add("glibc", m is not None, f"{m.group(1) if m else out} (fsm/Dockerfile builds on 22.04 = 2.35)")
# ── The NPU runtime must be at least as new as the toolkit that built
# the .rknn. Older refuses to load the model.
_, out = ssh(args.host, "grep -a -o 'librknnrt version: [0-9.]*' /usr/lib/librknnrt.so | head -1")
got = out.split(":")[-1].strip() if ":" in out else ""
r.add("librknnrt", bool(got) and got >= EXPECT_RKNN,
f"{got or '<not found>'} (need >= {EXPECT_RKNN}, what convert/ builds with)")
# ── The soname the binary was linked against has to exist on the board.
_, out = ssh(args.host, "ls /usr/lib/libddsc.so.* 2>/dev/null | head -3")
r.add("cyclonedds", EXPECT_DDS_SONAME in out, f"{out.replace(chr(10), ' ') or '<none>'}")
_, out = ssh(args.host, "dpkg -l 2>/dev/null | awk '/libmbus/{print $3}'")
r.add("libmbus", bool(out), out or "<not installed>")
# ── The IDL is the wire contract. Compared by substance: idlc stamps the
# source path into the header comment and the include guard, so those
# differ between any two builds and mean nothing.
ours = REPO / "deploy" / "dds" / "idl"
r.add(*check_idl(args.host, ours))
# ── Joint order. The one that moves the robot if it is wrong.
if args.bundle and args.config:
r.add(*check_joints(args.host, args.config, args.bundle))
elif args.bundle:
r.add("joint order", None, "pass --config <file on the board> to check it")
else:
r.add("joint order", None, "pass --bundle <dir> and --config <file> to check it")
print()
return r.report()
def check_idl(host: str, ours: Path) -> tuple[str, bool | None, str]:
"""Compare the board's generated headers with ours, ignoring path noise.
The repository ships `.idl`, the board ships the `.h` its middleware was
built from, so ours are generated here with `idlc` and the two compared.
`idlc` stamps the source path into both the header comment and the include
guard, and those differ between any two builds -- filtering them is what
makes this comparison mean "same types" rather than "same file".
"""
def strip(text: str) -> list[str]:
skip = re.compile(r"File name:|Source:|DDSC_|Cyclone DDS IDL compiler")
return [ln for ln in text.splitlines() if ln.strip() and not skip.search(ln)]
names = ["idl_common", "idl_imu", "idl_motor_control", "idl_robot_control"]
if not ours.is_dir():
return "IDL types", None, f"{ours} not found"
if shutil.which("idlc") is None:
return ("IDL types", None,
"idlc not on PATH (it ships with CycloneDDS); cannot generate ours to compare")
with tempfile.TemporaryDirectory(prefix="mjrl-idl-") as tmp:
tmpdir = Path(tmp)
for f in ours.glob("*.idl"):
shutil.copy(f, tmpdir / f.name)
differing, absent = [], []
for n in names:
gen = subprocess.run(["idlc", f"{n}.idl"], cwd=tmpdir,
capture_output=True, text=True)
if gen.returncode != 0 or not (tmpdir / f"{n}.h").is_file():
return "IDL types", None, f"idlc failed on {n}.idl"
code, board = ssh(host, f"cat /usr/include/{n}.h 2>/dev/null")
if code != 0 or not board:
absent.append(n)
continue
if strip(board) != strip((tmpdir / f"{n}.h").read_text()):
differing.append(n)
if absent:
return "IDL types", None, f"not on the board: {', '.join(absent)}"
if differing:
return ("IDL types", False,
f"differ: {', '.join(differing)} -- the DDS reader will never pair, "
f"and the topic stays empty rather than erroring")
return "IDL types", True, f"{len(names)} headers identical in substance"
def check_joints(host: str, config: str, bundle: Path) -> tuple[str, bool, str]:
"""The board's wire order against the bundle's, by name."""
layout_path = next(
(bundle / n for n in ("layout.json", "isaac_layout.json", "contract.json")
if (bundle / n).is_file()),
None,
)
if layout_path is None:
return "joint order", None, f"no layout.json in {bundle}"
code, out = ssh(host, f"sed -n '/^joint_names = \\[/,/^\\]/p' {config}")
board = re.findall(r'"([^"]+)"', out)
if code != 0 or not board:
return "joint order", None, f"no [robot] joint_names in {config}"
layout = json.loads(layout_path.read_text())
wire = layout.get("wire_joint_order")
if wire is None:
# Bundles from before wire_joint_order existed key the home pose by it.
wire = list(layout.get("default_joint_pos", {}))
if board != wire:
for i, (a, b) in enumerate(zip(board, wire)):
if a != b:
return ("joint order", False,
f"differ at index {i}: board={a} contract={b}")
return ("joint order", False,
f"different lengths: board {len(board)}, contract {len(wire)}")
index = {n: i for i, n in enumerate(board)}
missing = [n for n in layout["obs_joint_order"] + layout["action_joint_order"]
if n not in index]
if missing:
return "joint order", False, f"not on the board: {missing}"
uncovered = [n for n in board if n not in layout.get("default_joint_pos", {})]
if uncovered:
return ("joint order", False,
f"default_joint_pos misses {uncovered}; every wire joint needs a home")
obs_idx = [index[n] for n in layout["obs_joint_order"]]
gaps = [i for i in range(len(board)) if i not in obs_idx]
return ("joint order", True,
f"{len(board)} joints, same order; observation covers {len(obs_idx)}, "
f"skipping wire {gaps} -- resolved by name")
if __name__ == "__main__":
try:
sys.exit(main())
except subprocess.TimeoutExpired:
print("timed out talking to the board", file=sys.stderr)
sys.exit(2)
+33
View File
@@ -0,0 +1,33 @@
---
name: new-task
description: Register a new legged robot for training by generating a data-only robot_spec.py against a locked morphology template.
---
# Register a robot
Read [Usage](../../../docs/USAGE.md#robot-specification) for the schema.
Select `biped`, `quadruped`, `wheeled_biped`, or `multipod` using inspected
model facts and control intent. Reuse the exact model that passed the
stance checks. Keep robot assets and optional controls outside `tasks/`.
```bash
python scripts/onboard.py scaffold-tier3 --config robot.json --calibration calibration.json
python scripts/train.py --list
python scripts/train.py --task <robot_id> --dry-run
python -m pytest tests/test_registry.py tests/test_template_lock.py -q
```
The scaffold writes `tasks/<robot_id>/robot_spec.py`. It carries the model and
controls paths, home joint values and base height, gains and effort limits,
action scales, and contact names. With `--calibration`, the JSON supplies the
robot ID, family, model path, action scales, and any optional contacts,
controls, or wheel joint specifications not supplied by the report.
For operator input, apply the [controls skill](../controls/SKILL.md) before
scaffolding and set `controls_path` in the scaffold JSON relative to that
file. Use `--profile differential` when generating bindings for
`wheeled_biped`.
Onboarding must not modify the template files, the registry source, the
entry points, or the lock. The only generated Python file is the literal
robot specification. If the asset requires a new family or behavior, treat
it as an explicit template revision.
+30
View File
@@ -0,0 +1,30 @@
---
name: onboard-asset
description: Inspect a new robot URDF, MJCF, mesh, or project directory, normalize it for MuJoCo, and select a viewer, existing policy, deterministic control, or legged training route.
---
# Onboard a robot asset
Read [Workflow](../../../docs/WORKFLOWS.md) and [Hardware](../../../docs/HARDWARE.md).
1. Run `python scripts/onboard.py inspect <source> --report <new-report.json>`.
Pass `--profile`, `--mesh-root`, `--policy`, or `--metadata` when supplied.
Review the selected model, missing dependencies, model checks, declared
control intent, and recommended tier. Resolve ambiguous inputs explicitly.
Keep candidate training configs associated with the actor that used them.
A Tier 1 recommendation with `needs_metadata` goes to
[tier1-adapt](../tier1-adapt/SKILL.md) for extraction and review.
2. Preserve the source and record the exact model selected. For URDF, MJCF,
or a standalone mesh, run `build-mjcf` with a new `--output` path. A
directory works when it contains one model, or one mesh if there is no model. Add
`--floating-base` only when the robot requires a free root. If generating
visual fallback collisions, record the approximation and inspect the result.
3. Check compilation and inspect joint names, units, axes, limits, inertias,
collision geometry, and frames. For a viewer use `scripts/deploy.py --robot
<id> --model <xml>` and run a finite headless replay.
4. Route an existing policy to [tier1-adapt](../tier1-adapt/SKILL.md),
deterministic control to [tier2-control](../tier2-control/SKILL.md), and
explicit legged training to [tier3-train](../tier3-train/SKILL.md).
A viewer pass establishes that the model loads. Carry separate measured
physics and command evidence into every controlled mode.
+160
View File
@@ -0,0 +1,160 @@
---
name: setup-env
description: Set up, verify or repair the mjrl-lab development environment on Linux, Windows or macOS, with or without an NVIDIA GPU -- the Python environment and the Rust toolchain the controller (deploy/fsm) builds with. Detects the machine, picks the right torch build, finds which of cargo, the wasm target, wasm-bindgen and CycloneDDS are missing, installs, and verifies Python and optional legacy targets. Use this whenever someone is getting this repository running on a new machine, onboarding, or hitting environment trouble: "ModuleNotFoundError: No module named 'mjlab'", torch not seeing the GPU, warp reporting only ['cpu'], a registered robot missing from `--list`, mujoco version conflicts, `cargo` not found, `idlc not found`, a wasm-bindgen version mismatch, the controller's tests skipping, or a suspicion that the wrong Python or the wrong checkout is being used. Reach for it before hand-rolling pip or cargo commands, because the failures here are quiet -- an environment that imports everything and trains happily can still be running another checkout's code or ignoring the GPU entirely, and a test suite on a machine without cargo is green because it skipped the controller.
---
# Setting up mjrl-lab
The reference procedure is [`docs/AGENT_SETUP.md`](../../../docs/AGENT_SETUP.md) — about 600 lines,
written to be read by an agent, with the full platform tables and a catalogue of known failures.
**This skill does not restate it.** It sequences it, automates the two parts worth automating,
and knows when to stop and read.
Two scripts carry the weight. Both are plain standard-library Python and run without this skill,
so an agent that cannot load skills can still use them:
| Script | Interpreter | Answers |
|---|---|---|
| `scripts/detect.py` | whatever the machine already has | what this machine is, and the exact commands that follow from it |
| `scripts/gates.py` | the environment's own | whether the environment actually works, and what to fix |
## The shape of the problem
Almost everything here fails **quietly**. Installing a cu126 torch on a Blackwell card succeeds
and dies later inside CUDA. An editable install pointing at a stale checkout imports cleanly and
runs somebody else's code. A `pip` into the wrong interpreter reports success and changes
nothing. A machine without cargo runs the test suite green, because the tests that build the
controller skip. None of these announce themselves, and all of them surface much later as
something that looks unrelated.
That is why this measures before deciding and verifies after acting, rather than running a
sequence of commands and trusting that silence means success.
## Procedure
### 1. Measure, before deciding anything
```bash
python3 .claude/skills/setup-env/scripts/detect.py
```
Run it with the machine's existing interpreter — establishing whether that interpreter is usable
at all is part of what it answers. It reports OS, architecture, Python version, GPU and compute
capability, any environment that already exists, which parts of the controller's toolchain are
already here, and it ends with the exact command sequence for this machine.
Read its output before continuing. In particular:
- **Python out of range** (needs 3.10–3.13): install a compliant interpreter *first*. Installing
dependencies on an old one produces failures much later that read as unrelated.
- **An environment already exists**: go to step 3 and verify it. The common case after the first
run is not a bare machine but a broken environment, and building a second one hides the fault
rather than fixing it.
### 2. Install
Follow the commands `detect.py` printed. They are section 1 of the reference doc with the
branches already resolved for this machine; consult the doc when something does not match — the
Debian `ensurepip` split, the Windows execution policy, and the MSVC prerequisite each have a
paragraph there.
Only one step genuinely branches: **which torch wheel**. `detect.py` chose it from the measured
compute capability, and its reasoning is on the `torch_plan` function if you need to check it.
Before any `pip`, confirm the interpreter is the intended one:
```bash
python -c "import sys; print(sys.prefix)"
```
It must be the environment you just made. Pip reports success regardless of where it installs, so
this is the cheapest place to catch a mistake that otherwise surfaces three steps later.
`pip install -e .` is not optional and is not a convenience: the vendored `mjlab` and `rsl_rl`
live under `rl/`, which is not on `sys.path`. That editable install *is* the mechanism that makes
them importable.
**The legacy controller toolchain is required for its browser and board targets** — sections 2.1 and 2.2
of the reference doc. It is additive rather than per-checkout, so `detect.py` prints only the
steps this machine is missing, and "nothing missing" is a normal answer on a second checkout.
Two of its parts are easy to get subtly wrong:
- **wasm-bindgen's version comes from `deploy/fsm/Cargo.lock`**, exactly — not from
`Cargo.toml`'s `"0.2"`, a range every 0.2.x satisfies and `cargo install --version` refuses.
Take the command `detect.py` printed; `scripts/deploy.py` refuses any other version.
- **CycloneDDS is a source build** (11.0.1, the soname the board carries) and needs
`sudo ldconfig` after installing into `/usr/local`. It is not needed for `deploy.py`, but once
cargo exists `tests/test_deploy_sources.py` stops skipping and fails without it.
- **Rockchip's NPU header is per checkout, not per machine.** `bash
deploy/fsm/vendor/rknpu2/fetch.sh` puts it in the repository, git-ignored because its licence
is Rockchip's. `detect.py` prints that step whenever it is missing, a second checkout included.
### 3. Verify
```bash
.venv/bin/python .claude/skills/setup-env/scripts/gates.py --python-only
```
Use `--python-only` for Python onboarding, training, and native apps. Omit it to verify the legacy controller toolchain as well. The exit code counts failures. Each prints what it measured and, on failure, the
remedy and where the long form lives:
- **A — warp device enumeration.** On a GPU machine `cuda:0` must appear in the device list.
`wp.init()` also prints a banner naming the card; that banner appears even when CUDA is
unusable, so the list is the criterion and the banner is not.
- **B — native MuJoCo's batch interface.** `native:cpu` is built on `mujoco.rollout`, so
importing `mujoco` is not sufficient evidence.
- **C — the vendored copies are this checkout's.** Stricter than the doc's version, which only
rules out `site-packages`. An editable install left pointing at an older copy of the repository
satisfies that and still executes another checkout's code — measured on a training machine on
2026-09-04, where `tasks` came from the current repository and `mjlab` from a snapshot three
days stale. Everything imported; the seam being edited simply was not the seam being run.
- **D — the task registry.** Loads no simulation dependencies, so it passes even where the GPU is
broken. A failure here is a packaging problem, not a simulation one — which is worth knowing
before looking in the wrong place.
- **E — the Rust toolchain builds the controller's host targets.** `cargo check` of the browser's
wasm and of `play --app`'s extension, and the wasm-bindgen CLI against `Cargo.lock`. It builds
rather than reading versions, because a missing target or linker passes `cargo --version`.
- **F — the device build links and loads.** `cargo test --lib -- --list` with the default
features: CycloneDDS, libclang, Rockchip's NPU header and a `libddsc` the loader finds. `--list` runs the binary
without running a test, so a failing crate test is not reported as an environment fault.
E and F take about 30 s together, cold, on the i9-14900KF with the crates already downloaded; a
first run also downloads them.
Do not proceed past a failure. Each remedy is printed; the long forms are section 6 of the
reference doc.
### 4. Smoke test
With the relevant gates green, smoke-test the backend you intend to use.
Choose a registered robot from `scripts/train.py --list`. If the registry is
empty, use the model viewer example in the root README.
```bash
python scripts/train.py --task <robot_id> --backend warp --device cuda --num_envs 4 --max-iterations 1 --headless
python scripts/train.py --task <robot_id> --backend native --device cpu --num_envs 4 --max-iterations 1 --headless
```
Check the backend in the run banner and confirm the PPO update completes.
## When something fails
Work from the measurement, not from the symptom. `detect.py` and `gates.py` both print what they
observed, and the reference doc's section 6 is indexed by the exact error text — search it for the
message rather than reasoning from the traceback, because several of these have causes that the
traceback does not mention (the `njmax` overflow, the wandb `start_method` rejection, and
`select_gpus` raising `IndexError` are all in that class).
Two rules survive every platform, and both are in section 7 of the doc:
- **Never `pip install mjlab` or `rsl-rl-lib`.** They exist as modifiable copies under `rl/`.
A PyPI copy in the same environment shadows them ambiguously and is very hard to notice.
- **Never install into an existing Isaac Lab environment.** mjlab needs `mujoco~=3.11`, Isaac Lab
ships 3.10, and they break each other.
## Reporting back
Say which branch the machine took, which gates passed, and what the smoke test measured. If a
gate was skipped or a step was worked around, say so plainly — an environment reported as ready
when one gate was quietly skipped is worse than one reported as broken, because the next failure
will be attributed to the code instead of the setup.
+445
View File
@@ -0,0 +1,445 @@
#!/usr/bin/env python3
"""Report what this machine is, and which install path follows from it.
Run with **whatever interpreter the machine already has** -- that is the point.
It answers "is this Python usable at all", so it cannot assume a modern one, and
it cannot import anything that is not in the standard library of Python 3.6.
Every later step depends on these four facts, and each of them changes what the
install has to do:
OS + arch -> whether a C++ compiler is a prerequisite (Windows)
Python version -> whether an interpreter has to be installed first
GPU present -> which torch wheel index, and whether a driver is required
compute cap -> which CUDA build; a cu126 torch dies on Blackwell
and, for the controller (`deploy/fsm`, Rust), what of its toolchain is already
here -- that part is additive, so only the missing steps are printed.
Prints a plain report and, at the end, the exact commands for this machine.
"""
from __future__ import print_function
import glob
import os
import platform
import re
import shutil
import subprocess
import sys
MIN_PY = (3, 10)
MAX_PY = (3, 13) # inclusive; 3.14 is excluded by pyproject
# The highest `rust-version` among the packages `deploy/fsm/Cargo.lock` pins,
# read from their manifests on 2026-09-29: hashbrown 0.17.1 and indexmap 2.14.2,
# both through `toml`. A floor for the report only -- gate E builds the crate,
# which is the criterion, and says so when the lock moves past this.
MIN_RUST = (1, 85)
# The CycloneDDS the device build is known to work with: what the development
# machine (i9-14900KF, Ubuntu 24.04) runs, `idlc` 11.0.1, and a release whose
# soname, libddsc.so.11, is the board's.
CYCLONEDDS_TAG = "11.0.1"
def sh(cmd):
"""Run a command, returning (ok, output). Never raises: a missing tool is an
answer, not an error."""
try:
out = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
return True, out.decode("utf-8", "replace").strip()
except Exception as e: # noqa: BLE001 - any failure means "not available"
return False, str(e)
def detect_gpu():
"""(has_gpu, name, compute_cap). No nvidia-smi means no usable NVIDIA GPU.
macOS is decided without asking: Apple Silicon has no CUDA and there is no
workaround, so a stray nvidia-smi there would be misleading.
"""
if sys.platform == "darwin":
return False, None, None
ok, out = sh(["nvidia-smi", "--query-gpu=name,compute_cap",
"--format=csv,noheader"])
if not ok or not out:
return False, None, None
first = out.splitlines()[0]
parts = [p.strip() for p in first.split(",")]
name = parts[0] if parts else None
cap = parts[1] if len(parts) > 1 else None
return True, name, cap
def torch_plan(has_gpu, cap):
"""Which torch wheel to install, and why.
cu128 is the recommendation whenever the capability is unknown or >= 8.0: it
is required on Blackwell (12.0) and backward compatible with Ada and Ampere,
so choosing it cannot be wrong in that range -- whereas guessing cu126 on a
Blackwell card installs cleanly and then dies when CUDA initialises, which
is a far more expensive mistake than an unnecessarily new wheel.
"""
if not has_gpu:
return ("pip install torch torchvision",
"CPU build -- no --index-url. native:cpu is the training backend here.")
try:
capf = float(cap)
except (TypeError, ValueError):
capf = None
if capf is not None and capf < 8.0:
return ("# consult https://pytorch.org for a build matching compute capability %s" % cap,
"compute capability %s is below 8.0; the right wheel needs looking up." % cap)
why = "cu128: required on Blackwell (12.0), backward compatible with Ada/Ampere."
if capf is not None:
why = "compute capability %s -> %s" % (cap, why)
return ("pip install torch torchvision --index-url "
"https://download.pytorch.org/whl/cu128", why)
def python_verdict():
v = sys.version_info[:3]
ok = MIN_PY <= v[:2] <= MAX_PY
return ok, ".".join(str(x) for x in v)
def interpreter_advice(system):
"""What to install when the interpreter is out of range."""
return {
"Linux": ("Ubuntu 22.04/24.04 already ship 3.10/3.12. On older releases:\n"
" sudo add-apt-repository ppa:deadsnakes/ppa\n"
" sudo apt install -y python3.11 python3.11-venv\n"
" then create the venv with python3.11"),
"Darwin": "brew install python@3.12, then create the venv with python3.12",
"Windows": ("Installer from python.org with \"Add python.exe to PATH\" ticked, "
"then create the venv with py -3.11"),
}.get(system, "Install a CPython between 3.10 and 3.13.")
def find_repo():
"""The repository root, if we are standing in it.
Identified by pyproject naming this project rather than by the directory
name, which is routinely different (kk-rl-mjlab, mjrl-lab, a worktree path).
"""
here = os.path.abspath(os.getcwd())
while True:
candidate = os.path.join(here, "pyproject.toml")
if os.path.isfile(candidate):
try:
with open(candidate, "rb") as fh:
text = fh.read().decode("utf-8", "replace")
if 'name = "mjrl-lab"' in text:
return here
except OSError:
pass
parent = os.path.dirname(here)
if parent == here:
return None
here = parent
def which(name):
"""A tool on PATH, then in `$CARGO_HOME/bin`, where rustup and `cargo
install` put things -- on the PATH of a login shell, and often not on that
of whatever launched this."""
found = shutil.which(name)
if found:
return found
home = os.environ.get("CARGO_HOME") or os.path.join(os.path.expanduser("~"), ".cargo")
for exe in (name, name + ".exe"):
candidate = os.path.join(home, "bin", exe)
if os.path.isfile(candidate) and os.access(candidate, os.X_OK):
return candidate
return None
def version_of(path, flag="--version"):
"""(text, (major, minor, patch)) from `<tool> --version`, or (None, None).
`flag` because idlc has no `--version`; it answers `-v`.
"""
if not path:
return None, None
ok, out = sh([path, flag])
m = re.search(r"(\d+)\.(\d+)\.(\d+)", out) if ok else None
if not m:
return None, None
return m.group(0), tuple(int(x) for x in m.groups())
def locked_version(repo, package):
"""The version of `package` that `deploy/fsm/Cargo.lock` pins."""
if not repo:
return None
try:
with open(os.path.join(repo, "deploy", "fsm", "Cargo.lock"), "rb") as fh:
text = fh.read().decode("utf-8", "replace")
except OSError:
return None
m = re.search(r'^name = "%s"\nversion = "([^"]+)"' % re.escape(package), text, re.MULTILINE)
return m.group(1) if m else None
def detect_rust(repo):
"""What of the controller's toolchain is already on this machine.
`rustup` is asked for separately from `cargo` because a distribution's
cargo (`apt install cargo`) answers `cargo --version` and cannot add the
wasm target -- it has no rustup to ask -- and is usually older than this
lock needs. The rest of the install goes through rustup.
"""
cargo = which("cargo")
rustup = which("rustup")
rustc_text, rustc = version_of(which("rustc"))
targets = []
if rustup:
ok, out = sh([rustup, "target", "list", "--installed"])
targets = out.split() if ok else []
bindgen_text, _ = version_of(which("wasm-bindgen"))
home = os.environ.get("CYCLONEDDS_HOME", "/usr/local")
return {
"cargo": cargo,
"on_path": shutil.which("cargo") is not None,
"rustup": rustup,
"rustc_text": rustc_text,
"rustc": rustc,
"wasm32": "wasm32-unknown-unknown" in targets,
"wasm_bindgen": bindgen_text,
"wasm_bindgen_wanted": locked_version(repo, "wasm-bindgen"),
"cc": which("cc"),
"idlc": which("idlc"),
"idlc_text": version_of(which("idlc"), "-v")[0],
"dds_home": home,
"dds_headers": os.path.isfile(os.path.join(home, "include", "dds", "dds.h")),
"libclang": find_libclang(),
# Per checkout, unlike everything above: Rockchip's header is fetched into
# the repository (deploy/fsm/vendor/rknpu2/README.md). None when there is
# no repository to look in.
"npu_header": (os.path.isfile(os.path.join(
repo, "deploy", "fsm", "vendor", "rknpu2", "include", "rknn_api.h"))
if repo else None),
}
def find_libclang():
"""Where bindgen will find libclang, on Linux; None where it will not.
The directories clang-sys searches after `LIBCLANG_PATH`, not all of them:
enough to tell a machine that has it from one that needs
`libclang-dev`. Elsewhere it ships with the compiler (Xcode's command-line
tools, an LLVM install on Windows), and gate F is what finds out.
"""
if not sys.platform.startswith("linux"):
return "not checked on this OS"
dirs = [os.environ.get("LIBCLANG_PATH", "")] + glob.glob("/usr/lib/llvm-*/lib") + [
"/usr/lib", "/usr/lib64", "/usr/lib/x86_64-linux-gnu", "/usr/lib/aarch64-linux-gnu",
"/usr/local/lib"]
for d in dirs:
hits = sorted(glob.glob(os.path.join(d, "libclang*.so*"))) if d else []
hits = [h for h in hits if "libclang-cpp" not in h]
if hits:
return hits[0]
return None
def rust_commands(system, rust):
"""The steps still missing for the controller, in order, for this OS.
Only what is missing: unlike the Python environment this is not built
fresh per checkout, and re-running an installer that is already satisfied
is noise at best.
"""
steps = []
fresh = rust["rustup"] is None
if system == "Windows":
if fresh:
steps.append("winget install --id Rustlang.Rustup -e # then open a new terminal;"
" it links with MSVC (section 1.5)")
else:
if rust["cc"] is None:
steps.append("sudo apt install -y build-essential curl" if system == "Linux"
else "xcode-select --install")
if fresh:
if rust["cargo"]:
steps.append("# cargo here is not rustup's (a distribution package?): it cannot "
"add the wasm target")
steps.append("curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs"
" | sh -s -- -y --profile minimal")
steps.append('. "$HOME/.cargo/env"')
if not fresh and rust["rustc"] and rust["rustc"][:2] < MIN_RUST:
steps.append("rustup update stable")
if fresh or not rust["wasm32"]:
steps.append("rustup target add wasm32-unknown-unknown")
wanted = rust["wasm_bindgen_wanted"]
if wanted and rust["wasm_bindgen"] != wanted:
steps.append("cargo install wasm-bindgen-cli --version %s --locked" % wanted)
return steps
def device_commands(system, rust):
"""The device build's prerequisites: CycloneDDS (idlc, headers, libddsc) and
libclang for bindgen. The source build is the Dockerfile's host half, at the
release this machine family runs. Rockchip's NPU header is the one step that
belongs to the checkout rather than the machine, so a second checkout can be
missing it on a machine that has everything else."""
steps = []
if rust["npu_header"] is False:
steps.append("bash deploy/fsm/vendor/rknpu2/fetch.sh # Rockchip's NPU header, once per checkout")
if system == "Linux" and rust["libclang"] is None:
steps.append("sudo apt install -y libclang-dev")
if rust["idlc"] and not rust["idlc_text"] and system == "Linux":
# Installed and not running: idlc links libddsc, so this is the loader,
# and rebuilding CycloneDDS would not be the fix.
steps.append("sudo ldconfig # idlc is installed and does not run: libddsc is not "
"in the loader's cache")
return steps
if rust["idlc"] and rust["dds_headers"]:
return steps
src = "/tmp/cyclonedds"
if system == "Darwin":
# The same source build into a prefix of one's own: no sudo, nothing under
# /usr/local, and no ldconfig, which macOS does not have -- the crate writes
# the prefix's lib/ into the test binary's rpath from CYCLONEDDS_HOME
# instead, so the exports belong in the shell that runs cargo. Measured on
# an M3 Max on 2026-10-08 (section 2.2 of docs/AGENT_SETUP.md).
prefix = "$HOME/.local/opt/cyclonedds-%s" % CYCLONEDDS_TAG
return steps + [
"brew install cmake",
"git clone --depth 1 --branch %s https://github.com/eclipse-cyclonedds/cyclonedds.git %s"
% (CYCLONEDDS_TAG, src),
"cmake -S %s -B %s/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%s"
" -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON" % (src, src, prefix),
"cmake --build %s/build -j" % src,
"cmake --install %s/build" % src,
"export CYCLONEDDS_HOME=%s PATH=%s/bin:$PATH"
" # in the shell that builds: the crate takes its rpath from it" % (prefix, prefix),
]
if system != "Linux":
return steps + ["# CycloneDDS %s from source with cmake, CYCLONEDDS_HOME at its prefix,"
" idlc on PATH" % CYCLONEDDS_TAG,
"# -- section 2.2 of docs/AGENT_SETUP.md; measured on Linux and macOS only"]
return steps + [
"sudo apt install -y cmake git",
"git clone --depth 1 --branch %s https://github.com/eclipse-cyclonedds/cyclonedds.git %s"
% (CYCLONEDDS_TAG, src),
"cmake -S %s -B %s/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local"
" -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON" % (src, src),
"cmake --build %s/build -j" % src,
"sudo cmake --install %s/build && sudo ldconfig" % src,
]
def existing_env(repo):
"""An environment that already exists, and whether it is this repo's .venv.
Reported rather than assumed away, because the common case after the first
run is not a bare machine -- it is an environment that exists and has gone
wrong, and blindly creating a second one hides the problem instead of
fixing it.
"""
rows = []
if repo:
for rel in (".venv/bin/python", ".venv/Scripts/python.exe"):
p = os.path.join(repo, rel)
if os.path.isfile(p):
rows.append(("repo .venv", p))
in_venv = sys.prefix != getattr(sys, "base_prefix", sys.prefix)
if in_venv or os.environ.get("CONDA_PREFIX"):
kind = "conda env" if os.environ.get("CONDA_PREFIX") else "active venv"
rows.append((kind + " (currently active)", sys.executable))
return rows
def main():
system = platform.system()
py_ok, py_ver = python_verdict()
has_gpu, gpu_name, cap = detect_gpu()
repo = find_repo()
cmd, why = torch_plan(has_gpu, cap)
print("== machine ==")
print(" os : %s (%s)" % (system, platform.machine()))
print(" python : %s %s" % (py_ver, "OK" if py_ok else "OUT OF RANGE (need 3.10-3.13)"))
print(" interpreter : %s" % sys.executable)
if has_gpu:
print(" nvidia gpu : %s (compute capability %s)" % (gpu_name, cap))
elif system == "Darwin":
print(" nvidia gpu : none -- macOS has no CUDA, no exceptions")
else:
print(" nvidia gpu : none detected (no nvidia-smi, or it failed)")
print(" repository : %s" % (repo or "NOT FOUND -- run this from inside the repo"))
envs = existing_env(repo)
print("\n== environments already present ==")
if envs:
for label, path in envs:
print(" %-28s %s" % (label, path))
print(" -> One exists. Verify it with gates.py before building another;")
print(" a second environment hides the fault instead of fixing it.")
else:
print(" none")
print("\n== what follows ==")
if not py_ok:
print(" ! Install a 3.10-3.13 interpreter FIRST. Do not install dependencies")
print(" on this one -- the failures come much later and read as unrelated.")
print(" %s" % interpreter_advice(system))
print(" backends available here : %s" % ("warp:cuda (primary), native:cpu"
if has_gpu else "native:cpu"))
if has_gpu:
print(" prerequisite : NVIDIA driver (nvidia-smi already answers, so it is present)")
if system == "Windows":
print(" prerequisite : a C++ compiler (MSVC Build Tools) -- warp compiles kernels")
print(" torch : %s" % why)
rust = detect_rust(repo)
rustc_ok = rust["rustc"] is not None and rust["rustc"][:2] >= MIN_RUST
wanted = rust["wasm_bindgen_wanted"]
print("\n== the controller's toolchain (deploy/fsm, Rust) ==")
print(" cargo : %s" % (rust["cargo"] or "not found"))
if rust["cargo"] and not rust["on_path"]:
print(" not on PATH -- a new shell, or . \"$HOME/.cargo/env\"")
print(" rustup : %s" % (rust["rustup"] or "not found"))
print(" rustc : %s %s" % (rust["rustc_text"] or "-", "OK" if rustc_ok else
"NEED >= %d.%d" % MIN_RUST))
print(" wasm32 target : %s" % ("installed" if rust["wasm32"] else "missing"))
print(" wasm-bindgen : %s %s" % (rust["wasm_bindgen"] or "not found",
"OK" if rust["wasm_bindgen"] == wanted
else "Cargo.lock pins %s" % wanted))
if system != "Windows":
print(" C linker (cc) : %s" % (rust["cc"] or "not found"))
idlc = rust["idlc_text"] or ("at %s, and it does not run" % rust["idlc"] if rust["idlc"]
else "not found")
print(" CycloneDDS : idlc %s, headers %s under %s" % (
idlc, "present" if rust["dds_headers"] else "missing", rust["dds_home"]))
print(" libclang : %s" % (rust["libclang"] or "not found"))
print(" NPU header : %s" % {True: "fetched", False: "missing (once per checkout)",
None: "-"}[rust["npu_header"]])
print("\n== commands ==")
venv_py = "python3" if system != "Windows" else "py -3.11"
activate = (".venv\\Scripts\\Activate.ps1" if system == "Windows"
else "source .venv/bin/activate")
print(" %s -m venv .venv" % venv_py)
print(" %s" % activate)
print(" %s" % cmd)
print(" pip install -e .")
steps = rust_commands(system, rust)
print("\n # the controller's toolchain (section 2.1)%s" % ("" if steps else
": nothing missing"))
for step in steps:
print(" %s" % step)
steps = device_commands(system, rust)
print("\n # the device build: CycloneDDS, libclang, the NPU header (section 2.2)%s" % (
"" if steps else ": nothing missing"))
for step in steps:
print(" %s" % step)
print("\n Then verify: python <skill>/scripts/gates.py")
return 0
if __name__ == "__main__":
sys.exit(main())
+426
View File
@@ -0,0 +1,426 @@
#!/usr/bin/env python3
"""Check that an mjrl-lab environment actually works, and say what to fix.
Run this **with the environment's own interpreter** (`.venv/bin/python gates.py`,
or whatever interpreter the environment uses) from anywhere; pass `--repo` if the
working directory is not inside the repository.
Each gate exists because its failure is quiet. An environment can import
everything, start training, and produce numbers, while the GPU is unused, or the
vendored code being executed belongs to a different checkout. So each gate prints
what it measured, not merely OK, and names the remedy -- an agent reading only
"FAIL" would have to go and rediscover it.
Exit code is the number of failed gates, so this drops into a script:
python gates.py || echo "environment is not ready"
"""
from __future__ import annotations
import argparse
import os
import re
import shutil
import subprocess
import sys
from pathlib import Path
DOC = "docs/AGENT_SETUP.md"
CRATE = Path("deploy") / "fsm"
#: What cargo says when the toolchain, not the source, is the problem, and what
#: to do about it. Matched against the tail of stderr; the tail is printed too,
#: so a message nobody listed here still reaches the reader.
RUST_HINTS = (
("may not be installed", "rustup target add wasm32-unknown-unknown"),
("linker `", ("no C linker: build-essential (Linux), xcode-select --install (macOS),\n"
f"MSVC Build Tools (Windows, section 1.5 of {DOC})")),
("is not supported by the following package", "rustup update stable"),
("does not understand this lock file", "rustup update stable"),
("feature `edition2024` is required", "rustup update stable"),
("no Python 3.x interpreter found",
"pyo3's build script needs a python3 on PATH: activate the environment, or set PYO3_PYTHON"),
("lock file needs to be updated",
("Cargo.toml and Cargo.lock disagree in this checkout -- a source problem, "
"not the environment")),
("idlc not found",
f"CycloneDDS is not installed, or idlc is not on PATH: section 2.2 of {DOC}"),
("dds/dds.h", ("CycloneDDS headers are not under $CYCLONEDDS_HOME/include "
f"(default /usr/local): section 2.2 of {DOC}")),
("Unable to find libclang", "bindgen needs libclang: sudo apt install -y libclang-dev"),
("rknn_api.h is missing",
("Rockchip's NPU header is fetched, not committed, once per checkout: "
"bash deploy/fsm/vendor/rknpu2/fetch.sh")),
("-lddsc", "libddsc is not under $CYCLONEDDS_HOME/lib (default /usr/local)"),
("error while loading shared libraries",
("built, and the loader cannot find libddsc: sudo ldconfig, or add its lib/ to "
"LD_LIBRARY_PATH")),
("Library not loaded: @rpath/libddsc",
("built, and the macOS loader cannot find libddsc: the crate writes its rpath from "
"$CYCLONEDDS_HOME at build time, so export it and rebuild (cargo clean -p mjrl-fsm), "
"or add its lib/ to DYLD_LIBRARY_PATH for one run")),
)
def which(name: str) -> str | None:
"""A tool on PATH, then where `cargo install` and rustup put it.
The same rule `scripts/deploy.py::which` follows, so this gate and the
script agree on whether a tool is installed. `$CARGO_HOME/bin` is on the
PATH of a login shell and often not on that of an editor or a service, and
a tool found only there is installed, not missing.
"""
found = shutil.which(name)
if found:
return found
home = Path(os.environ.get("CARGO_HOME") or (Path.home() / ".cargo"))
for candidate in (home / "bin" / name, home / "bin" / f"{name}.exe"):
if candidate.is_file() and os.access(candidate, os.X_OK):
return str(candidate)
return None
def locked_version(repo: Path, package: str) -> str | None:
"""The version of `package` that `deploy/fsm/Cargo.lock` pins."""
lock = repo / CRATE / "Cargo.lock"
if not lock.is_file():
return None
m = re.search(rf'^name = "{re.escape(package)}"\nversion = "([^"]+)"',
lock.read_text(encoding="utf-8"), re.MULTILINE)
return m.group(1) if m else None
def find_repo(start: Path | None = None) -> Path | None:
"""The repository root, identified by pyproject rather than directory name.
The directory is routinely named something else -- kk-rl-mjlab, a git
worktree path -- so matching on the name would fail on the machines this is
most needed on.
"""
here = (start or Path.cwd()).resolve()
for candidate in [here, *here.parents]:
pyproject = candidate / "pyproject.toml"
if pyproject.is_file() and 'name = "mjrl-lab"' in pyproject.read_text(
encoding="utf-8", errors="replace"
):
return candidate
return None
class Gates:
def __init__(self, repo: Path) -> None:
self.repo = repo
self.failed: list[str] = []
def report(self, name: str, ok: bool, detail: str, remedy: str = "") -> None:
print(f"[{'PASS' if ok else 'FAIL'}] {name}")
for line in detail.splitlines():
print(f" {line}")
if not ok:
self.failed.append(name)
if remedy:
for line in remedy.splitlines():
print(f" -> {line}")
print()
# ── A ───────────────────────────────────────────────────────────────
def gate_a_warp(self) -> None:
"""Whether warp can enumerate devices.
The criterion is `cuda:0` in the device list on a GPU machine. wp.init()
also prints a banner to stderr naming the card -- that banner appears
even when CUDA is unusable, so reading it instead of the list is how a
broken GPU gets mistaken for a working one.
"""
try:
import warp as wp
wp.init()
devices = [str(d) for d in wp.get_devices()]
except Exception as e: # noqa: BLE001
self.report("A warp device enumeration", False, f"{type(e).__name__}: {e}",
f"warp is missing or broken; see section 2 of {DOC}")
return
has_gpu = self._nvidia_present()
detail = f"devices: {devices}"
if has_gpu:
self.report("A warp device enumeration", "cuda:0" in devices,
detail + " (GPU machine: cuda:0 required)",
"the driver or CUDA toolchain is wrong -- warp:cuda is unavailable.\n"
f"see 'torch.cuda.is_available() returns False' in section 6 of {DOC}")
else:
self.report("A warp device enumeration", True,
detail + " (no GPU: ['cpu'] alone is the expected result)")
@staticmethod
def _nvidia_present() -> bool:
if sys.platform == "darwin":
return False
try:
subprocess.check_output(["nvidia-smi"], stderr=subprocess.STDOUT)
return True
except Exception: # noqa: BLE001
return False
# ── B ───────────────────────────────────────────────────────────────
def gate_b_mujoco(self) -> None:
"""native:cpu is built on `mujoco.rollout`, so importing mujoco is not enough."""
try:
import mujoco
from mujoco import rollout # noqa: F401
v = mujoco.__version__
ok = v.startswith(("3.11", "3.12"))
self.report("B native MuJoCo batch interface", ok, f"mujoco {v}",
"expected 3.11.x or 3.12.x; pyproject pins mujoco~=3.11.0.\n"
"an Isaac Lab environment carries 3.10 and the two break each other")
except Exception as e: # noqa: BLE001
self.report("B native MuJoCo batch interface", False, f"{type(e).__name__}: {e}",
"pip install -e . was not run, or ran into the wrong environment")
# ── C ───────────────────────────────────────────────────────────────
def gate_c_vendored(self) -> None:
"""The vendored copies must come from **this** checkout's `rl/`.
The published gate only rules out site-packages, which is not enough. An
editable install left pointing at an older copy of the repository
satisfies "not site-packages" while executing code from a different
checkout -- measured on a training machine on 2026-09-04, where `tasks`
resolved from the current repository and `mjlab` from a snapshot three
days stale. Everything imported; the seam simply was not the one being
edited. So compare against the repository root, not against a substring.
"""
try:
import mjlab
import rsl_rl
except Exception as e: # noqa: BLE001
self.report("C vendored copies are this checkout's", False,
f"{type(e).__name__}: {e}",
"pip install -e . creates the mapping that makes rl/ importable;\n"
f"without it nothing under rl/ is on sys.path. See section 2 of {DOC}")
return
want = (self.repo / "rl").resolve()
rows, ok = [], True
for mod in (mjlab, rsl_rl):
path = Path(mod.__file__).resolve().parent
inside = path == want / path.name
ok = ok and inside
rows.append(f"{mod.__name__:7s} {path}{'' if inside else ' <-- not this repo'}")
self.report("C vendored copies are this checkout's", ok,
"\n".join(rows) + f"\nexpected under: {want}",
"pip uninstall -y mjlab rsl-rl-lib # if they came from PyPI\n"
f"pip install -e . # from {self.repo}\n"
"then re-run this gate: an editable install can point at another checkout")
# ── D ───────────────────────────────────────────────────────────────
def gate_d_registry(self) -> None:
"""The registry, exercised the way a user meets it.
This loads no simulation dependencies -- tasks register through lazy
factories -- so it passes even where the GPU is broken. A failure here is
a packaging problem, not a simulation one, and that distinction saves
looking in the wrong place.
"""
train = self.repo / "scripts" / "train.py"
try:
out = subprocess.run([sys.executable, str(train), "--list"],
cwd=self.repo, capture_output=True, text=True, timeout=180)
except Exception as e: # noqa: BLE001
self.report("D task registry", False, f"{type(e).__name__}: {e}")
return
expected = {path.parent.name for path in (self.repo / "tasks").glob("*/robot_spec.py")}
first_section = out.stdout.split("\n\n", 1)[0]
lines = first_section.splitlines()[2:]
listed = {line.split()[0] for line in lines if line and not line[0].isspace()}
ok = out.returncode == 0 and (
listed == expected if expected else out.stdout.strip() == "The registry is empty."
)
detail = f"{len(listed)} robots: {', '.join(sorted(listed))}" if listed else "no robots registered"
if not ok:
detail += f"\nExpected robot IDs: {', '.join(sorted(expected)) or '(none)'}"
detail += f"\n{(out.stderr or out.stdout).strip()[:400]}"
self.report("D task registry", ok, detail,
"verify the editable install and the literal tasks/<robot_id>/robot_spec.py files")
# ── E ───────────────────────────────────────────────────────────────
def cargo(self, cargo: str, *args: str,
passthrough: tuple[str, ...] = ()) -> subprocess.CompletedProcess:
"""Run cargo on the controller's crate.
`--locked` always: a gate must not rewrite `Cargo.lock`, and the
toolchain floor this setup states was measured against the locked
versions, not against whatever resolves today. `passthrough` goes
after `--`, to the test binary, so it has to follow the options that
are cargo's.
"""
return subprocess.run(
[cargo, *args, "--locked", "--manifest-path", str(self.repo / CRATE / "Cargo.toml"),
*(("--", *passthrough) if passthrough else ())],
cwd=self.repo, capture_output=True, text=True, timeout=900, check=False)
@staticmethod
def errors(stderr: str) -> list[str]:
"""The lines of cargo's stderr worth reading: its errors, the last few.
Not simply the tail. A link failure ends in the linker's command line,
one line of several kilobytes that buries the one naming the library.
"""
lines = stderr.strip().splitlines()
shown = [ln for ln in lines if "error" in ln.lower()][-6:] or lines[-6:]
return [" " + (ln if len(ln) <= 160 else ln[:157] + "...") for ln in shown]
@staticmethod
def hints(stderr: str) -> list[str]:
"""The remedy for every message in `RUST_HINTS` that stderr contains."""
return list(dict.fromkeys(fix for needle, fix in RUST_HINTS if needle in stderr))
def gate_e_rust(self) -> None:
"""The controller's host builds: cargo, the wasm target, pyo3, wasm-bindgen.
`deploy/fsm` is the controller on all three hosts, and `scripts/deploy.py`
builds two of them here with cargo -- the browser's wasm and `play
--app`'s extension (the board's binary and the Windows extension are
cross-built in Docker, not by this toolchain). The failure this gate is
for is quiet at the level of the test suite: without cargo,
`tests/test_fsm_extension.py` and `tests/test_deploy_sources.py` skip,
and `pytest tests/` is green on a machine that has never built the
controller.
So it builds rather than reading versions. `cargo check` of each host's
features proves the target is installed and that build scripts and
proc macros link on this machine, which `cargo --version` does not.
The wasm-bindgen CLI is held to `Cargo.lock`, as `scripts/deploy.py`
holds it: the CLI and the crate share an ABI and version separately,
and `Cargo.toml`'s `"0.2"` is a range every 0.2.x satisfies. Checked
here so a mismatch is found while setting up, not at the first bundle.
"""
name = "E Rust toolchain builds the controller's host targets"
cargo = which("cargo")
if cargo is None:
self.report(name, False, "cargo: not on PATH and not in $CARGO_HOME/bin",
f"install rustup: section 2.1 of {DOC}")
return
rows, ok, remedies = [], True, []
version = subprocess.run([cargo, "--version"], capture_output=True, text=True,
check=False).stdout.strip()
on_path = shutil.which("cargo") is not None
rows.append(f"{version or 'cargo --version failed'} ({cargo})")
if not on_path:
rows.append(" not on PATH: deploy.py and the tests look in $CARGO_HOME/bin too; "
"a shell will not until it sources ~/.cargo/env")
for label, args in (
("web (wasm32)", ["check", "--lib", "--target", "wasm32-unknown-unknown",
"--no-default-features", "--features", "web"]),
("py (play --app)", ["check", "--lib", "--no-default-features",
"--features", "py"]),
):
try:
out = self.cargo(cargo, *args)
except Exception as e: # noqa: BLE001
ok = False
rows.append(f"{label}: {type(e).__name__}: {e}")
continue
ok = ok and out.returncode == 0
rows.append(f"{label}: {'builds' if out.returncode == 0 else 'FAILS'}")
if out.returncode != 0:
rows += self.errors(out.stderr)
remedies += self.hints(out.stderr)
wanted = locked_version(self.repo, "wasm-bindgen")
cli = which("wasm-bindgen")
have = None
if cli:
have = subprocess.run([cli, "--version"], capture_output=True, text=True,
check=False).stdout.split()[-1:]
have = have[0] if have else None
match = cli is not None and have == wanted
ok = ok and match
rows.append(f"wasm-bindgen CLI: {have or 'not found'}, Cargo.lock pins {wanted}")
if not match:
remedies.append(f"cargo install wasm-bindgen-cli --version {wanted} --locked")
self.report(name, ok, "\n".join(rows), "\n".join(dict.fromkeys(remedies)))
# ── F ───────────────────────────────────────────────────────────────
def gate_f_device(self) -> None:
"""The device build: CycloneDDS, libclang, and a libddsc the loader finds.
The crate's default feature is `device`, the robot's DDS bus, and it is
what a plain `cargo test` and `tests/test_deploy_sources.py` build.
`build.rs` runs `idlc`, compiles its output against `$CYCLONEDDS_HOME`
(default /usr/local), generates bindings through libclang, links libddsc
and compiles a probe against Rockchip's NPU header, which each checkout
fetches (deploy/fsm/vendor/rknpu2/fetch.sh). deploy.py's host builds need
none of it, and the board's build
has its own inside the Docker image -- but once cargo is installed,
test_deploy_sources stops skipping and fails without it.
Built and **run**: `--list` executes the test binary without running a
test, so a libddsc that links and then cannot be loaded fails here, as
`cargo check` never would. A failing crate test is not an environment
fault, and listing keeps this gate from reporting one.
"""
name = "F device build (CycloneDDS, libclang)"
cargo = which("cargo")
if cargo is None:
self.report(name, False, "cargo: not found (gate E)", f"section 2.1 of {DOC} first")
return
idlc = which("idlc")
home = os.environ.get("CYCLONEDDS_HOME", "/usr/local")
detail = [f"idlc: {idlc or 'not on PATH'}", f"CYCLONEDDS_HOME: {home}"]
try:
out = self.cargo(cargo, "test", "--lib", passthrough=("--list",))
except Exception as e: # noqa: BLE001
self.report(name, False, "\n".join(detail + [f"{type(e).__name__}: {e}"]))
return
listed = sum(1 for ln in out.stdout.splitlines() if ln.endswith(": test"))
ok = out.returncode == 0 and listed > 0
if ok:
detail.append(f"built, linked and loaded; {listed} tests listed")
else:
detail += ["cargo test --lib: FAILS", *self.errors(out.stderr)]
self.report(name, ok, "\n".join(detail),
"\n".join([*self.hints(out.stderr), f"see section 2.2 of {DOC}"]))
def run(self, *, python_only: bool = False) -> int:
print(f"repository: {self.repo}")
print(f"interpreter: {sys.executable}\n")
self.gate_a_warp()
self.gate_b_mujoco()
self.gate_c_vendored()
self.gate_d_registry()
if not python_only:
self.gate_e_rust()
self.gate_f_device()
if self.failed:
print(f"{len(self.failed)} gate(s) failed: {', '.join(self.failed)}")
print(f"Remedies are above; the long forms are in section 6 of {DOC}.")
else:
scope = "four Python" if python_only else "six"
print(f"All {scope} gates passed. Next: the smoke test in section 4 of {DOC}.")
return len(self.failed)
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--repo", type=Path, default=None,
help="repository root; found from the working directory if omitted")
ap.add_argument("--python-only", action="store_true",
help="check onboarding, training and native apps without legacy Rust targets")
args = ap.parse_args()
repo = args.repo.resolve() if args.repo else find_repo()
if repo is None or not (repo / "pyproject.toml").is_file():
print("error: not inside an mjrl-lab checkout; pass --repo", file=sys.stderr)
return 2
return Gates(repo).run(python_only=args.python_only)
if __name__ == "__main__":
sys.exit(main())
+47
View File
@@ -0,0 +1,47 @@
---
name: tier1-adapt
description: Recover explicit policy semantics from upstream training configs or unified metadata, adapt an ONNX or TorchScript actor, and verify parity and native response.
---
# Adapt an existing policy
1. Use [onboard-asset](../onboard-asset/SKILL.md) to identify the matching
model, actor and training configs. Treat a Tier 1 recommendation with
`needs_metadata` as a request to recover policy semantics.
2. If the source supplies a complete `mjrl-policy-adapter/1` document, use it
directly. Otherwise extract evidence from the matched config files.
```bash
python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
--output metadata-draft
```
Review `metadata.json` and the provenance, conflicts, `missing_fields` and
`unresolved` entries in `manifest.json`. An incomplete extraction writes
the draft and exits 1. Python inputs are read as static syntax only.
Resolve unknown imports, inheritance or expressions from source evidence
and record the confirmed values in a JSON or YAML overrides file. The
actor's observation and action definitions establish channel order.
A joint-name inventory, a dictionary of default angles, MuJoCo joint order
and graph widths each establish different facts.
3. Rerun extraction with `--overrides reviewed.json --output metadata-ready`.
Require `ready: true` in the manifest. Review ordered observation terms,
history, joint and action order, home pose, scaling, gains, limits and rate.
Run the adapter with the exact model.
```bash
python scripts/onboard.py adapt-tier1 actor.onnx --metadata metadata-ready/metadata.json \
--model robot.xml --output policy --report adapter-report.json
```
After review, `--config env.yaml --config agent.yaml --overrides reviewed.json`
can replace `--metadata` and run the same extraction during adaptation.
4. Review the adapter parity result and layout against the supplied semantics.
Build a verification config for the ONNX controller with `kind: rl_onnx`,
its relative `policy` and `layout` paths, and the robot root when needed.
Run native command probes with meaningful expected motion and tolerances.
5. Package a compatible controlled mode only with passing verification for
those files. Use [deploy](../deploy/SKILL.md) and replay the resulting app.
A graph comparison verifies the transformation. Native command response
establishes how the adapted controller moves that model.
+31
View File
@@ -0,0 +1,31 @@
---
name: tier2-control
description: Set up a deterministic native MuJoCo controller for a complete robot model, including direct actuation, joint sliders, IK, differential or omnidirectional wheels, trajectories, and CPG.
---
# Set up deterministic control
1. Inspect the model with [onboard-asset](../onboard-asset/SKILL.md).
Choose a controller from the actual kinematics and intended motion.
Confirm joint axes and limits, actuator semantics, effort, and rates.
2. Declare a JSON controller config with its `kind`. Use
`joint_sliders` for named position joints, `ik` for a
named chain and end-effector site or body, `differential` or `omni` for
measured wheel geometry, and `trajectory` or `cpg` for explicit motions.
The `direct` API can inspect authored actuator inputs, but a packaged
`model_viewer` must have `actuators: []` and carries no control claim. Choose
one of the verified control kinds above for a controlled application.
3. Run setup to a fresh path.
```bash
python scripts/onboard.py setup-tier2 --model robot.xml --config setup.json --output control
```
Use the resulting `robot.xml` and `controller.json`. Inspect the optional
`controls.yaml` with the [controls skill](../controls/SKILL.md).
4. Run `onboard.py verify` with a probe JSON that references the generated
controller and controls and declares expected measured responses. Use
[Tutorial](../../../docs/TUTORIAL.md) for a runnable joint-slider example.
Read both the physics and control stages and repeat after any relevant edit.
5. Package with the exact evidence and replay the app with finite steps.
Report observed responses, limits, and any failing probe.
+28
View File
@@ -0,0 +1,28 @@
---
name: tier3-train
description: Calibrate a floating-base biped, quadruped, wheeled biped, or multipod for locked-template locomotion training, then scaffold and run measured replay.
---
# Calibrate and train a legged robot
1. Run [onboard-asset](../onboard-asset/SKILL.md). Establish the family, named
supports, single floating root, explicit locomotion intent, and joint
control semantics. Wheeled bipeds require named unlimited wheel hinges.
2. Prepare stance JSON with the seed pose, named supports, optional height
adjustments, and measured effort limits. Run
`python scripts/onboard.py calibrate --model robot.xml --config stance.json
--report calibration.json`. Inspect the measured contact loads, drift,
limits, effort headroom, and complete report before using it.
3. Follow [new-task](../new-task/SKILL.md) to generate the literal robot spec.
Supply explicit action scales and reuse the passing report for the same
XML and compiled dependencies. If operator input is requested, generate
bindings before scaffolding and set `controls_path` in the scaffold JSON.
Use `--profile differential` for a wheeled biped so A/D command yaw.
Keep robot-specific changes in data.
4. Follow [train](../train/SKILL.md). Verify discovery and a backend smoke
before allocating a learning run. Measure the selected checkpoint on
trained ranges with the required command and observation-noise setting.
5. If packaging is requested, export with `--native-layout` and verify the
exported model, controller and bindings. The native layout supports the
wheeled-biped template's hybrid position and velocity targets. Follow
[deploy](../deploy/SKILL.md) for the selected runtime.
+38
View File
@@ -0,0 +1,38 @@
---
name: train
description: Train or resume a locomotion policy through a locked template, validate backend and run size, replay the checkpoint, and report measured results.
---
# Train and measure
1. Confirm the robot appears in `scripts/train.py --list`. Use
[new-task](../new-task/SKILL.md) when the robot still needs registration.
Run `.claude/skills/setup-env/scripts/detect.py` and verify the environment
with the [setup skill](../setup-env/SKILL.md) when needed.
2. Resolve with `--dry-run`. Use `native:cpu` for the CPU path or `warp:cuda`
for a supported CUDA installation. A small environment count and one
iteration exercise the path. Choose the learning run size from the
backend's available memory and measured throughput. If Warp reports an
overflow, raise `--warp-max-constraints N` for per-world constraints or
`--warp-max-contacts N` for the shared contact pool and rerun the smoke.
Use positive integers. Training saves the selected and effective capacities
in `warp-allocation.json` beside the checkpoints. Warp resume, replay, and
export use that record with the current task or scene capacity floor.
Explicit flags override the saved setting and floor.
3. Run `scripts/train.py --task <robot_id>` with the chosen backend, device
and environment count. Use `--headless` for unattended runs. Watch
tracking rewards, episode lengths, termination counts, action spread,
and curriculum level. Use `--resume` for the latest run or
`--checkpoint <path>` for a specific one. On resume, an iteration override
counts additional iterations.
4. Run `scripts/play.py --task <robot_id> --checkpoint <path> --headless
--steps <count> --speed 0 --measure`. Use `--checkpoint-command-ranges`
for the saved curriculum range. For a fixed command inside that range,
add `--command <vx> <vy> <yaw_rate> --evaluation-out <json>`. Match replay
physics to training and report the observation noise setting.
5. Report the checkpoint, backend, environment count, iteration count,
evaluation command and duration, measured tracking, and resets. Describe
a one-iteration smoke as an execution check. Record when replay was not
run or results are unavailable.
See the [tutorial](../../../docs/TUTORIAL.md) for the end-to-end sequence.
+15
View File
@@ -0,0 +1,15 @@
# Optional local defaults: copy to .env and customize.
# Blank values use CLI built-in defaults. Shell variables and CLI flags win.
MJRL_TASK=
MJRL_MODEL=
MJRL_SCENE=
MJRL_TERRAIN_ROW=
MJRL_TERRAIN_COL=
MJRL_BACKEND=
MJRL_DEVICE=
MJRL_NUM_ENVS=
MJRL_PLAY_NUM_ENVS=
MJRL_CPU_THREADS=
MJRL_STRIP_VISUAL=
MJRL_TENSORBOARD=
MJRL_TB_PORT=
+108
View File
@@ -0,0 +1,108 @@
name: Checks
on:
push:
pull_request:
workflow_dispatch:
permissions:
contents: read
jobs:
layout-and-translations:
name: Layout, translation and Colab checks
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.13"
# These checks parse files, validate translations and exercise Colab backup
# contracts with short-lived subprocesses. They need no MuJoCo or GPU stack.
- name: Install test runner
run: python -m pip install --disable-pip-version-check pytest
- name: Check layout, translations and Colab helpers
run: python -m pytest tests/test_layout.py tests/test_translations.py tests/test_colab.py tests/test_colab_metrics.py tests/test_colab_backup_inspection.py tests/test_colab_notebook_workflow.py -q
lint:
name: Ruff
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.13"
# The rule set is written into pyproject.toml, so the ruff version is not
# what decides what passes here; CONTRIBUTING.md asks for this command
# before handing a change over, and this is the same command.
- name: Install ruff
run: python -m pip install --disable-pip-version-check ruff
- name: Lint
run: python -m ruff check .
framework-tests:
name: Framework tests (native backend, CPU)
runs-on: ubuntu-latest
timeout-minutes: 40
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: pyproject.toml
# The whole suite on the native CPU backend: the seam, the vendored
# copies, the registry, the task contracts, export and the bundle checks.
# No GPU, so CUDA-specific checks skip. Current Rust/wasm runtimes are
# built from temporary deterministic test exports, not old applications
# under out/. The CPU torch wheel limits installation disk usage.
#
# mjlab sets MUJOCO_GL=egl on Linux before mujoco is imported, so every
# test that imports a task needs an EGL the loader can find, GPU or not:
# Mesa's. Without it collection fails on 29 files with
# `'NoneType' object has no attribute 'eglQueryString'` (measured in a
# python:3.12-slim container on 2026-10-08; the packages below fixed it).
- name: Install Mesa EGL for headless rendering
run: sudo apt-get update -qq && sudo apt-get install -y -qq libegl1 libegl-mesa0 libgl1-mesa-dri
- name: Install PyTorch (CPU build)
run: python -m pip install --disable-pip-version-check torch torchvision --index-url https://download.pytorch.org/whl/cpu
- name: Install this repository
run: python -m pip install --disable-pip-version-check -e ".[dev]"
- name: Install portable controller build tools
run: |
rustup target add wasm32-unknown-unknown
cargo install wasm-bindgen-cli --version 0.2.128 --locked
- name: Run framework and onboarding checks
run: python -m pytest tests/ --ignore=tests/test_replay.py -q
- name: Run all locked-template replay checks
run: python -m pytest tests/test_replay.py -q
controller-core:
name: Portable Rust controller
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
- name: Test the hardware-independent controller
run: cargo test --manifest-path deploy/fsm/Cargo.toml --no-default-features
+57
View File
@@ -0,0 +1,57 @@
# Python
__pycache__/
*.py[cod]
*.egg-info/
.pytest_cache/
.venv/
venv/
build/
dist/
# Training artefacts
logs/
runs/
outputs/
# scripts/export.py's default output directory
out/
*.pt
*.onnx
# deploy/onnx2rknn.py's output, and the record of what built it
*.rknn
*.rknn.json
wandb/
# Rust build output. No leading slash: it matches at every depth, so
# deploy/fsm/target/ is covered without naming it.
target/
# cargo-zigbuild probes zig by compiling an empty file into the working
# directory -- the crate's, in docker-build.sh -- on every macOS build.
# `docker-build.sh` removes it; ignored as well, because `deploy.py` reads
# `git status` to decide whether the source is clean, and one left by a failed
# build would mark every later one `-dirty`.
.intentionally-empty-file.o
# Warp kernel cache
.warp/
# Rockchip's NPU header and runtime, fetched by deploy/fsm/vendor/rknpu2/fetch.sh.
# They are under Rockchip's own licence, which this repository cannot re-grant.
deploy/fsm/vendor/rknpu2/include/
deploy/fsm/vendor/rknpu2/lib/
# Derived caches for user assets.
assets/*/.cache/
# The log MuJoCo leaves in the working directory at runtime
MUJOCO_LOG.TXT
# Editors / OS
.vscode/
.idea/
.DS_Store
# Local configuration and secrets
# Copy .env.example to .env for local defaults; never commit local secrets.
.env
.env.local
.claude/launch.json
+41
View File
@@ -0,0 +1,41 @@
# Repository instructions
Read [CLAUDE.md](CLAUDE.md) for working rules and [Workflow](docs/WORKFLOWS.md)
to choose the robot intake route. The repository contains onboarding, native
control, simulation training, and packaging. Use the local sources and the
asset's supplied documentation to establish its model and control assumptions.
Use lightweight models for bounded reading and documentation, standard models
for routine changes, and advanced models for architecture or difficult
debugging. Delegate independent tasks when appropriate and preserve validation.
| Work | Reference |
|---|---|
| Commands and configuration | [Usage](docs/USAGE.md) |
| Environment setup or repair | [Agent setup](docs/AGENT_SETUP.md) |
| Model units, frames, collision, and calibration | [Hardware](docs/HARDWARE.md) |
| Controller and input bindings | [Controls](docs/CONTROLS.md) |
| Application build or policy deployment | [Deploy guide](deploy/README.md) and [bundle format](deploy/BUNDLE.md) |
| Repository conventions and checks | [Contributing](CONTRIBUTING.md) |
## Skills
Read an applicable `SKILL.md` in full before starting the work.
| Skill | Use for |
|---|---|
| [setup-env](.claude/skills/setup-env/SKILL.md) | Establish or repair the Python and controller build environment. |
| [onboard-asset](.claude/skills/onboard-asset/SKILL.md) | Inspect assets, normalize geometry, choose an intake tier, and verify physics. |
| [tier1-adapt](.claude/skills/tier1-adapt/SKILL.md) | Adapt an existing policy from declarative metadata and check numerical parity. |
| [tier2-control](.claude/skills/tier2-control/SKILL.md) | Configure and verify deterministic robot controls. |
| [tier3-train](.claude/skills/tier3-train/SKILL.md) | Calibrate and train a robot using a locked locomotion template. |
| [new-task](.claude/skills/new-task/SKILL.md) | Create the data-only robot specification and verify discovery. |
| [train](.claude/skills/train/SKILL.md) | Run or resume training and report measured replay results. |
| [controls](.claude/skills/controls/SKILL.md) | Bind keys and pad controls using the shared vocabulary. |
| [deploy](.claude/skills/deploy/SKILL.md) | Export and package policies for the supported host. |
| [bundle-manual](.claude/skills/bundle-manual/SKILL.md) | Translate the built application's operator manual. |
Every new skill belongs in this table. The template directories carry their own
lock instructions. Onboarding writes one literal `robot_spec.py` per robot and
keeps the calibrated asset, controller, reports, and controls outside the locked
templates.
+58
View File
@@ -0,0 +1,58 @@
# Repository working rules
The project is a generic robotics onboarding, simulation, and training platform.
Choose an intake route in [Workflow](docs/WORKFLOWS.md). Use
[Usage](docs/USAGE.md) for commands and [Contributing](CONTRIBUTING.md) for layout.
## Checks
Run from the repository root with the editable install active.
```bash
python -m pip install -e ".[dev]"
python -m pytest tests/ -q
python -m ruff check .
python scripts/train.py --list
```
For a host with unrelated pytest plugins on its Python path, run pytest in the
project environment with `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1`. Use
[agent setup](docs/AGENT_SETUP.md) if imports resolve to a different checkout.
## Architecture
- The root and `rl/` are separate package roots. `rl/` has no `__init__.py`.
Keep imports from `mjrl`, `mjlab`, and `rsl_rl` in their existing form and
use the editable install to resolve them.
- `mjrl.backend.select.use_backend` must run before creating an environment.
The manager resolves mjlab's simulation class when it builds the environment.
Native MuJoCo implements the batched simulator interface that Warp provides.
- Native rollout scratch data is per worker. Restore each environment's state,
controls, and warm starts from its own buffers. Constant recomputation uses
dedicated scratch data and preserves visual statistics.
- `tasks/registry.py` discovers literal `tasks/<robot_id>/robot_spec.py`
files. Listing tasks must stay free of simulation imports. The selected
family resolves to a locked template in `tasks/templates/`.
- Keep physical facts in the robot specification and asset. Keep reward and
observation logic, randomization, curriculum, and PPO in the templates.
Adding a robot requires no changes to a template or a command entry point.
- Onboarding and verification are in `rl/mjrl/onboarding/`. Deterministic
controls live in `rl/mjrl/controllers/`, and the portable native runtime
lives in `rl/mjrl/platform/`. The legacy Rust controller and target build
tools remain in `deploy/`.
## Changes and verification
- Preserve the lock in `tests/test_template_lock.py`. Review an authorized
template revision before updating the recorded hashes, including shared
dependencies that change template behavior.
- Verify physics and control response separately. Report measured outcomes with
the model, backend, and run size. A shape check, a visual preview, or a
completed iteration alone cannot establish behavior or hardware readiness.
- Keep changes to vendored libraries identifiable with an `[mjrl]` marker,
reason, and upstream version. See [Vendoring](docs/VENDOR.md).
- Write repository material in English. A Chinese `*.zh.md` translation
carries the digest of its English source. Update both before refreshing the
marker with `tools/checks/stamp_translations.py`.
- Exercise tests against the quiet failure they claim to prevent. When an
assertion could pass vacuously, include a control that demonstrates detection.
+52
View File
@@ -0,0 +1,52 @@
# Contributing
Run checks from the repository root with an editable install.
```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
python -m pytest tests/ -q
python -m ruff check .
```
Use the [agent setup guide](docs/AGENT_SETUP.md) for platform-specific
installation and [usage](docs/USAGE.md) for the workflow commands.
## Directory layout
```
mjrl-lab/
├── assets/ # robot models and source assets
├── controller/ # input vocabulary, parsing, and device access
├── deploy/ # native app contract and legacy target builders
│ ├── convert/ # ONNX to RKNN conversion
│ ├── dds/ # legacy wire interface
│ └── fsm/ # legacy Rust controller
├── docs/ # manuals and design notes
├── rl/
│ ├── mjlab/ # vendored simulator and manager library
│ ├── mjrl/
│ │ ├── backend/ # native and Warp integration
│ │ ├── controllers/ # deterministic controller models
│ │ ├── onboarding/ # inspect, build, calibrate, verify
│ │ ├── platform/ # native app assembly and runtime
│ │ └── tasks/ # reusable MDP and curriculum code
│ └── rsl_rl/ # vendored learning algorithms
├── scenes/ # built-in world configurations
├── scripts/ # onboard, train, play, export, deploy
├── tasks/
│ └── templates/ # locked family recipes
├── tests/ # regression checks and synthetic fixtures
└── tools/ # development and validation utilities
```
Add robot-specific inputs under `assets/<robot_id>/` or reference an existing
asset checkout. Use onboarding to generate `tasks/<robot_id>/robot_spec.py`.
Model-specific tools belong beside their asset. Reusable tools belong in
`tools/` or the relevant `mjrl` package.
Tests should pin shared contracts using neutral fixtures. Exercise a native
build and command response when changing model or controller logic. Run the
lock test and verify both backends when changing template behavior. Follow
[CLAUDE.md](CLAUDE.md) for authorized lock revisions and translations.
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+53
View File
@@ -0,0 +1,53 @@
Robot asset and control platform
Copyright 2026 KingKong Robotics.
Licensed under the Apache License, Version 2.0 (see LICENSE).
This repository includes the third-party material below, each part under its
own licence. The full texts are in the files named.
rl/mjlab/
mjlab 1.6.0 (https://github.com/mujocolab/mjlab)
Copyright 2025, The mjlab Developers
Apache License 2.0 -- rl/mjlab/LICENSE
Modified: five files, each marked "[mjrl]" with the reason for the change
(listed in docs/VENDOR.md).
rl/mjlab/utils/lab_api/
Derived from Isaac Lab (https://github.com/isaac-sim/IsaacLab)
Copyright (c) 2022-2025, The Isaac Lab Project Developers
BSD 3-Clause License -- licenses/IsaacLab.txt
rl/mjlab/asset_zoo/robots/unitree_go1/
rl/mjlab/asset_zoo/robots/unitree_g1/
assets/go1_example/go1.xml
assets/g1_example/g1.xml
From MuJoCo Menagerie (https://github.com/google-deepmind/mujoco_menagerie)
Copyright (c) 2016-2022 (Go1), 2016-2023 (G1)
HangZhou YuShu TECHNOLOGY CO.,LTD. ("Unitree Robotics")
BSD 3-Clause License -- licenses/mujoco_menagerie-unitree_go1.txt,
licenses/mujoco_menagerie-unitree_g1.txt
rl/mjlab/asset_zoo/robots/i2rt_yam/
From MuJoCo Menagerie, apart from xmls/assets/d405.stl, which is not in
Menagerie and comes with mjlab
Copyright (c) 2025 i2rt robotics
MIT License -- licenses/mujoco_menagerie-i2rt_yam.txt
rl/rsl_rl/
rsl_rl 5.4.2 (https://github.com/leggedrobotics/rsl_rl)
Copyright (c) 2026, ETH Zurich
Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES
BSD 3-Clause License -- rl/rsl_rl/LICENSE
rl/rsl_rl/modules/normalization.py
Contains code from pfrl (https://github.com/pfnet/pfrl)
Copyright (c) 2020 Preferred Networks, Inc.
MIT License -- licenses/pfrl.txt
Not included in this repository
deploy/fsm/vendor/rknpu2/
Filled by fetch.sh with Rockchip's rknn_api.h and librknnrt.so, which are
under Rockchip's RKNN SDK License and are not covered by this repository's
licence. See deploy/fsm/vendor/rknpu2/README.md.
+80
View File
@@ -0,0 +1,80 @@
# MJRL Robotics Platform
[简体中文](README.zh.md) · [Workflow](docs/WORKFLOWS.md) · [Command reference](docs/USAGE.md)
Bring a robot model into MuJoCo, verify its dynamics and controls, and package it
as a native simulation application. The platform also adapts compatible
external policies and trains locomotion policies with the same environment
on native MuJoCo CPU or MuJoCo Warp GPU.
| Tier | Starting point | Output |
|---|---|---|
| 0 | Mesh, URDF, or an MJCF without verified controls | Compiled model and viewer application |
| 1 | Compatible policy and explicit observation, action, and joint metadata | Adapted ONNX policy and layout with numerical parity results |
| 2 | Complete robot model and known kinematics | Deterministic controller and native verification report |
| 3 | Legged robot with a calibrated stance and explicit control intent | Robot specification and training through a locked morphology template |
## Run the model viewer
From the checkout, with Python 3.10–3.13:
```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
python scripts/onboard.py inspect assets/smoke/smoke_biped.xml
python scripts/deploy.py --robot viewer_smoke --model assets/smoke/smoke_biped.xml --bundle out/viewer_demo
python scripts/play.py --app out/viewer_demo/viewer_smoke.app --headless --steps 20
```
Choose an unused output directory. The smoke model exercises loading and
packaging. For a new robot, replace the input with your own model and follow
the report's recommended route. On Windows, activate with
`.venv\Scripts\Activate.ps1`.
## Bring your own robot
1. Run `scripts/onboard.py inspect` with the source model and any available
profile or policy metadata. Use `build-mjcf` for URDF and standalone meshes.
2. Choose the route in [Workflow](docs/WORKFLOWS.md). Simulate both physics and
command response before building a controlled application.
3. For legged locomotion, calibrate the stance and run `scaffold-tier3`. It
generates `tasks/<robot_id>/robot_spec.py`. The biped, quadruped, wheeled
biped, and multipod templates supply observations, rewards, domain
randomization, curriculum, and PPO.
4. Run `python scripts/train.py --list` to discover registered robots and
`python scripts/train.py --task <robot_id> --dry-run` to check resolution.
A native `.app` contains the model, its dependencies, controller or policy, and
a Python runtime entry point. The host supplies Python and the listed runtime
libraries. The existing Rust browser and board workflow uses the legacy bundle
format. The native exporter and runtime support wheeled-biped hybrid
actions. The legacy Rust runtime accepts its position-action layout.
The four templates completed one PPO update each on native CPU and Warp GPU
using synthetic robot fixtures. These runs validate the execution paths.
The [Go1 example](assets/go1_example/README.md) also completed 1,000 PPO updates
on an RTX 5080. Its selected checkpoint averaged 0.401 m/s at a 0.400 m/s
forward command over 18 seconds, with 0.022 m/s mean absolute error and no
resets. Native checkpoint and ONNX replay passed forward, reverse, lateral,
turning and idle probes. These are flat-floor simulation results, not hardware
qualification or evidence that every robot will learn with the same run size.
The [wheeled-biped example](assets/wheeled_biped_example/README.md) verifies
hybrid leg-position and wheel-velocity control. Its selected policy passed
native forward, reverse, yaw and idle probes, with keyboard and gamepad tests
at its declared ±0.91 m/s and ±0.77 rad/s ranges. The
[G1 biped example](assets/g1_example/README.md) passed seven isolated native
probes and full-range keyboard tests after 7,800 Warp updates. The
[development record](docs/REFACTOR_STATUS.md) includes accepted applications
and unsuccessful trials, with their measured reasons.
## Reference
- [Project guide](docs/PROJECT_GUIDE.md) for navigation and supported boundaries.
- [Tutorial](docs/TUTORIAL.md) for verification, training, and measured replay.
- [Colab](docs/COLAB.md) for GPU training with your own registered robot.
- [Hardware and model facts](docs/HARDWARE.md) for calibration and actuator inputs.
- [Deployment format](deploy/BUNDLE.md) for application contents and runtime contracts.
- [Contributing](CONTRIBUTING.md) for repository layout and checks.
+73
View File
@@ -0,0 +1,73 @@
# MJRL 机器人平台
<!-- tracks: README.md @ sha256:5e58614ff2c467bd -->
[English](README.md) · [接入流程](docs/WORKFLOWS.md) · [命令手册](docs/USAGE.zh.md)
把机器人模型导入 MuJoCo,验证动力学和控制响应,再打包为原生仿真应用。
平台可以适配兼容的外部策略,也可以通过同一训练环境在原生 MuJoCo CPU 或
MuJoCo Warp GPU 上训练运动策略。
| 层级 | 起点 | 产物 |
|---|---|---|
| 0 | 网格、URDF 或缺少已验证控制的 MJCF | 可编译模型和查看应用 |
| 1 | 兼容策略与明确的观测、动作和关节元数据 | 适配后的 ONNX、布局及数值一致性结果 |
| 2 | 完整模型与已知运动学 | 确定性控制器及原生验证报告 |
| 3 | 站姿已标定且控制意图明确的足式机器人 | 机器人规格及锁定形态模板训练 |
## 运行模型查看应用
在代码库根目录中使用 Python 3.10–3.13:
```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
python scripts/onboard.py inspect assets/smoke/smoke_biped.xml
python scripts/deploy.py --robot viewer_smoke --model assets/smoke/smoke_biped.xml --bundle out/viewer_demo
python scripts/play.py --app out/viewer_demo/viewer_smoke.app --headless --steps 20
```
选择尚未使用的输出目录。Windows 上使用 `.venv\Scripts\Activate.ps1` 激活。
该模型用于检验加载与打包。接入自己的机器人时,替换为实际模型并按检查报告的建议继续。
## 接入自己的机器人
1. 用 `scripts/onboard.py inspect` 检查模型和可用的配置或策略元数据。URDF
和独立网格可用 `build-mjcf` 转换。
2. 根据[接入流程](docs/WORKFLOWS.md)选择层级。打包可控制应用前,分别验证
动力学和命令响应。
3. 足式机器人先标定站姿,再用 `scaffold-tier3` 生成
`tasks/<robot_id>/robot_spec.py`。双足、四足、轮足双足和多足模板负责
观测、奖励、域随机化、课程和 PPO。
4. 用 `python scripts/train.py --list` 查看注册机器人,用
`python scripts/train.py --task <robot_id> --dry-run` 检查参数解析。
原生 `.app` 包含模型、资产依赖、控制器或策略以及 Python 入口。宿主提供
Python 和所列依赖。现有 Rust 浏览器与板端走独立的旧版包格式。轮足双足的混合
动作可导出并在原生运行时回放,旧版 Rust 使用其位置动作布局。
四个模板分别在原生 CPU 和 Warp GPU 上用合成模型完成过一次 PPO 更新,
用于验证执行路径。
[Go1 示例](assets/go1_example/README.md)还在 RTX 5080 上完成了 1,000 次 PPO
更新。选定检查点在 0.400 m/s 前进命令下,18 秒回放平均速度为 0.401 m/s,
平均绝对误差为 0.022 m/s,无重置。原生检查点与 ONNX 回放通过了前进、后退、
横移、转向及静止探测。这些是平地仿真结果,不是硬件资格认证,也不代表每个
机器人都能用相同训练规模学会控制。
[轮足双足示例](assets/wheeled_biped_example/README.md)验证了腿部位置与车轮速度
混合控制。所选策略通过了原生前进、后退、转向和静止探测,并在其声明的
±0.91 m/s 和 ±0.77 rad/s 范围内完成键盘及手柄测试。
[G1 双足示例](assets/g1_example/README.md)在完成 7,800 次 Warp 更新后,通过了
七项独立原生探测和满量程键盘测试。
[研发记录](docs/REFACTOR_STATUS.md)汇总已验收应用、未通过的试验及实测原因。
## 参考
- [项目指南](docs/PROJECT_GUIDE.zh.md)
- [教程](docs/TUTORIAL.zh.md)
- [Colab](docs/COLAB.zh.md) 用于在 GPU 上训练自己注册的机器人
- [硬件与模型参数](docs/HARDWARE.zh.md)
- [部署包格式](deploy/BUNDLE.md)
- [贡献指南](CONTRIBUTING.md)
+11
View File
@@ -0,0 +1,11 @@
{
"schema": "mjrl-controller/1",
"kind": "cpg",
"joints": ["fl", "fr", "rl", "rr"],
"frequency_hz": 0.6,
"amplitude": [0.08, 0.08, 0.08, 0.08],
"phase": [0, 3.141592653589793, 3.141592653589793, 0],
"kp": 40,
"kd": 4,
"effort_limit": 4
}
+46
View File
@@ -0,0 +1,46 @@
<mujoco model="cpg_stance">
<option timestep="0.002" integrator="implicitfast"/>
<default>
<joint axis="0 1 0" range="-30 30" armature="0.08" damping="0.15"/>
<default class="leg">
<geom type="capsule" size="0.013" fromto="0 0 0 0 0 -0.13" mass="0.07"
rgba="0.21 0.57 0.74 1"/>
</default>
<default class="foot">
<geom type="sphere" pos="0 0 -0.13" size="0.03" mass="0.05"
friction="0.7 0.01 0.001" rgba="0.8 0.44 0.13 1"/>
</default>
</default>
<worldbody>
<light pos="0 -1 2" dir="0 0 -1"/>
<geom name="floor" type="plane" size="5 5 0.1" rgba="0.8 0.82 0.86 1"/>
<body name="base" pos="0 0 0.183">
<freejoint name="root"/>
<geom type="box" size="0.13 0.1 0.025" mass="1.2" rgba="0.16 0.25 0.33 1"/>
<body name="front_left" pos="0.12 0.09 -0.025">
<joint name="fl"/>
<geom class="leg" mass="0.08"/>
<geom class="foot" name="fl_foot"/>
</body>
<body name="front_right" pos="0.12 -0.09 -0.025">
<joint name="fr"/>
<geom class="leg"/>
<geom class="foot" name="fr_foot"/>
</body>
<body name="rear_left" pos="-0.12 0.09 -0.025">
<joint name="rl"/>
<geom class="leg"/>
<geom class="foot" name="rl_foot"/>
</body>
<body name="rear_right" pos="-0.12 -0.09 -0.025">
<joint name="rr"/>
<geom class="leg"/>
<geom class="foot" name="rr_foot"/>
</body>
</body>
</worldbody>
<keyframe>
<!-- A two-degree roll perturbation makes the startup measurement nontrivial. -->
<key name="start" qpos="0 0 0.183 0.999847695156 0.017452406437 0 0 0 0 0 0"/>
</keyframe>
</mujoco>
+25
View File
@@ -0,0 +1,25 @@
{
"root_body": "base",
"keyframe": "start",
"ground_geom": "floor",
"foot_geoms": ["fl_foot", "fr_foot", "rl_foot", "rr_foot"],
"controller": "controller.json",
"controls": "controls.yaml",
"steps": 5200,
"thresholds": {
"max_root_tilt_rad": 0.08,
"max_root_drop_m": 0.015,
"max_root_translation_m": 0.03,
"position_atol": 0.035
},
"probes": [{
"command": {"amplitude_scale": 1},
"actuator_controls": {
"mjrl_fl": 1.2063715789784806,
"mjrl_fr": -1.2063715789784806,
"mjrl_rl": -1.2063715789784806,
"mjrl_rr": 1.2063715789784806
},
"joint_positions": {"fl": 0.079842138, "fr": -0.079842138, "rl": -0.079842138, "rr": 0.079842138}
}]
}
+10
View File
@@ -0,0 +1,10 @@
{
"kind": "differential",
"left_joint": "left_wheel",
"right_joint": "right_wheel",
"wheel_radius": 0.1,
"track_width": 0.26,
"max_wheel_speed": 8,
"kv": 5,
"effort_limit": 5
}
+27
View File
@@ -0,0 +1,27 @@
<mujoco model="differential_drive">
<option timestep="0.002" integrator="implicitfast"/>
<default>
<joint damping="0.02" armature="0.01"/>
<geom friction="1 0.01 0.001"/>
</default>
<worldbody>
<light pos="0 -1 2" dir="0 0 -1"/>
<geom name="floor" type="plane" size="2 2 0.1" rgba="0.8 0.82 0.86 1"/>
<body name="chassis" pos="0 0 0.12">
<freejoint name="root"/>
<geom type="box" size="0.14 0.1 0.025" mass="1.5" rgba="0.24 0.53 0.68 1"/>
<geom type="sphere" pos="-0.13 0 -0.095" size="0.025" mass="0.05"
friction="0.03 0.01 0.001" rgba="0.28 0.30 0.34 1"/>
<body name="right" pos="0 -0.13 -0.02">
<joint name="right_wheel" axis="0 1 0"/>
<geom name="right_tire" type="cylinder" euler="90 0 0" size="0.1 0.02"
mass="0.25" rgba="0.19 0.22 0.25 1"/>
</body>
<body name="left" pos="0 0.13 -0.02">
<joint name="left_wheel" axis="0 1 0"/>
<geom name="left_tire" type="cylinder" euler="90 0 0" size="0.1 0.02"
mass="0.25" rgba="0.19 0.22 0.25 1"/>
</body>
</body>
</worldbody>
</mujoco>
+25
View File
@@ -0,0 +1,25 @@
{
"root_body": "chassis",
"controller": "controller.json",
"controls": "controls.yaml",
"ground_geom": "floor",
"foot_geoms": ["left_tire", "right_tire"],
"steps": 750,
"thresholds": {
"base_displacement_atol_m": 0.055,
"base_angular_velocity_atol_rad_s": 0.18
},
"probes": [
{
"pad": {"Ly": -0.7},
"actuator_controls": {"mjrl_left_wheel": 3.5, "mjrl_right_wheel": 3.5},
"joint_velocities": {"left_wheel": 3.5, "right_wheel": 3.5},
"base_displacement_m": [0.525, 0, 0]
},
{
"pad": {"Rx": -1},
"actuator_controls": {"mjrl_left_wheel": -1.3, "mjrl_right_wheel": 1.3},
"base_angular_velocity_rad_s": [0, 0, 1]
}
]
}
+196
View File
@@ -0,0 +1,196 @@
# Unitree G1 biped training example
This 29-joint humanoid uses the Unitree G1 model and STL meshes from the
local [mjlab asset zoo](../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/g1.xml),
introduced in commit `004afda1ac1dafc3aedd4e9ca8c6d2242b399f57`. The files
come from MuJoCo Menagerie and are copyright 2016–2023 HangZhou YuShu
TECHNOLOGY CO., LTD. (Unitree Robotics), under the
[BSD 3-Clause License](../../licenses/mujoco_menagerie-unitree_g1.txt).
The repository [NOTICE](../../NOTICE) and [provenance](model-provenance.json)
identify the files and their hashes.
`g1.xml` rebases mesh paths and applies the reflected motor armatures from
the vendored [G1 constants](../../rl/mjlab/asset_zoo/robots/unitree_g1/g1_constants.py).
The compiled masses, joint limits, mesh vertices and reference pose match
the source exactly. The MJCF round trip changes collision geom sizes by
at most 4.4e−8 m and quaternion components by at most 4.2e−7. Several
articulated links carry visual meshes and inertia only. Each named ankle
roll body carries seven collision capsules that represent its sole.
## Stance calibration
`stance.json` specifies a symmetric, slightly bent stance with a 0.24 rad
knee angle and −0.12 rad hip and ankle pitch. It uses the vendored effort
limits. The calibration estimates each joint's apparent inertia as
`1 / diagonal(inverse(M))`, using the full mass matrix `M` of the floating
robot at that pose. This captures the inertia when other coordinates can
move in response to the applied joint effort. Gains use 14 Hz, 28 Hz at the
ankles and a damping ratio of 0.85. In the reference hold, the floating-base
model with mass-matrix diagonal gains and the vendored home pose fell.
The two-second native hold completed 1,000 steps. The peak base drift was
2.30 mm and the peak base rotation was 0.00410 rad. The soles supported
99.9998% of the robot's 327 N weight on average. The maximum joint tracking
error was 0.0126 rad and peak effort was 12.6% of its configured limit. There
were zero self contacts and zero MuJoCo warnings.
`robot.json` specifies an action scale of 60% of effort limit divided by
proportional gain per joint. Limb contacts carry a penalty. Pelvis, torso
and head contacts terminate the episode. The generated
[robot specification](../../tasks/g1_example/robot_spec.py) is the only
training task file and uses the locked biped template.
From the checkout root with the Python environment activated, use fresh
output paths for the reports. The scaffold command writes a new robot
specification when its destination is empty.
```bash
python scripts/onboard.py inspect assets/g1_example/g1.xml \
--profile assets/g1_example/profile.json --report out/g1_example/intake.json
python scripts/onboard.py calibrate --model assets/g1_example/g1.xml \
--config assets/g1_example/stance.json --report out/g1_example/calibration.json
python scripts/onboard.py scaffold-tier3 --config assets/g1_example/robot.json \
--calibration out/g1_example/calibration.json --tasks-root out/g1_example/scaffold
python scripts/train.py --task g1_example --backend native --device cpu \
--num-envs 4 --headless --no-tensorboard --max-iterations 1
```
## Warp constraint allocation
A 1,024-environment run with the default Warp allocation aborted before
completing its first update. It reported that the 64-row constraint buffer
was too small during contacts. A 64-environment smoke also logged overflows.
The model needs room for the constraints from its soles and contacts made
during falls. The following run uses the vendored G1 flat configuration's
300-row allocation while leaving model contacts and template behavior as
configured. Training saves the allocation for matching Warp replay.
```bash
python scripts/train.py --task g1_example --backend warp --device cuda \
--num-envs 1024 --warp-max-constraints 300 --headless --no-tensorboard \
--max-iterations 1200
```
## Measured training and replay
The first run completed 1,200 updates and 29,491,200 environment steps on
an RTX 5080. Physics ran at 400 Hz and policy control at 50 Hz. The Warp
allocation record reported 300 constraints per environment and 49,152
contacts shared across 1,024 environments. The replay used the saved
command ranges and disabled observation noise.
In a 12-second Warp replay with a +0.25 m/s held command, the first
checkpoint averaged +0.0002 m/s with 0.250 m/s mean absolute error.
Checkpoint 1199 averaged +0.218 m/s with 0.070 m/s mean absolute error.
The same checkpoint averaged −0.185 m/s under a −0.25 m/s command with
0.066 m/s error. All seven held-command and idle replays had zero episode
resets on both Warp and native MuJoCo. The lateral and yaw tests missed
their targets. For example, Warp measured +0.007 m/s against +0.12 m/s
lateral and +0.047 rad/s against +0.30 rad/s yaw. The exported native ONNX
actor passed the physics stage and failed the tracking stage on the same
seven commands.
The held-command checks use forward and reverse at 0.25 m/s, left and
right at 0.12 m/s, yaw in both directions at 0.30 rad/s, and idle. Each
replay lasts 12 seconds. The ONNX acceptance requires a mean error and
mean absolute error within 0.12 m/s forward, 0.06 m/s lateral and
0.15 rad/s yaw, along with native physics checks. For the Warp forward
measurement, substitute the actual checkpoint path from the run.
```bash
python scripts/play.py --task g1_example --checkpoint path/to/model_1199.pt \
--backend warp --device cuda --warp-max-constraints 300 \
--headless --steps 600 --speed 0 --measure --checkpoint-command-ranges \
--no-obs-noise --command 0.25 0 0 --evaluation-out out/g1_example/forward.json
```
A continuation from checkpoint 1199 initially widened the command range
before collecting a complete fresh episode. The runner had randomized
starting episode ages, so the restored curriculum clock admitted biased
tracking evidence. That run was stopped and excluded. After the resume
fix, training restarted from checkpoint 1199 and kept the saved curriculum
level through the fresh 1,000-step window.
The checkpoint 1800 forward preview shows a short shuffle. Native ONNX
contact measurements between 2 and 12 seconds found both soles loaded in
65.8% of samples and one sole loaded in the remainder. Peak sole clearance
was 5.7 mm. Loaded ground contact points slid at mean speeds of 0.111 and
0.127 m/s on the two soles. Both soles stayed loaded throughout lateral
and yaw replay. These are simulator gait diagnostics.
The corrected continuation added 1,600 updates for a total of 68,812,800
environment steps across both valid runs. Checkpoint 2798 stayed at
curriculum level 0. In Warp it averaged +0.257 m/s forward and −0.210 m/s
reverse, with 0.036 and 0.046 m/s mean absolute errors. All seven native
and Warp replays completed with zero resets. The standalone lateral and
yaw commands still missed their targets. Native ONNX replay passed physics
and failed tracking acceptance.
Mixed commands showed that the policy could control all three axes when
the forward command was present. Under `[0.25, 0.12, 0.30]`, native replay
averaged `[0.244, 0.107, 0.289]` in body-frame m/s, m/s and rad/s. Under
`[0.25, -0.12, -0.30]`, it averaged `[0.290, -0.112, -0.303]`. Switching
from forward motion to a side-only or yaw-only command brought the robot
back to a planted stance. The example currently provides the calibrated
model and training task. The ONNX export is retained as a diagnostic.
A second continuation added 3,000 updates, bringing the total to 5,800
updates and 142,540,800 environment steps. The run stayed at curriculum
level 0 and logged no Warp capacity overflows. Checkpoint 5797 returned
these 12-second results on the training backend with observation noise
turned off.
| Command | Target | Mean | Mean absolute error |
| --- | ---: | ---: | ---: |
| Forward, m/s | +0.25 | +0.240 | 0.017 |
| Reverse, m/s | −0.25 | −0.240 | 0.013 |
| Left, m/s | +0.12 | +0.003 | 0.117 |
| Right, m/s | −0.12 | −0.115 | 0.017 |
| Yaw left, rad/s | +0.30 | +0.225 | 0.089 |
| Yaw right, rad/s | −0.30 | −0.264 | 0.059 |
All seven replays, including idle, completed without resets on both Warp
and native MuJoCo. The native ONNX policy passed physics verification. It
still failed the leftward command and exceeded the 0.15 rad/s yaw-error
limit by 0.00025 rad/s during the forward probe. In the ONNX forward gait,
peak sole clearance remained 5.5 mm. Loaded contact slip decreased to
0.057 and 0.089 m/s. Both soles remained loaded during the left command.
This checkpoint is retained for diagnosis without a release app. The
compliant-gain trial trained a fresh policy with the same mechanical model
and softer hip and knee gains, but failed acceptance. Its task and source assets
were removed; the failure record remains in
[the refactor report](../../docs/REFACTOR_STATUS.md#g1-learning-and-selection).
## Verified checkpoint 7796
The last continuation added 2,000 updates. Across the four valid runs, the
original-gain model completed 7,800 updates and 191,692,800 environment
steps. The saved curriculum level was 0, which provides command ranges
of ±0.28 m/s forward, ±0.14 m/s lateral and ±0.32 rad/s yaw.
Checkpoint 7796 passed the original seven isolated-command probes in
native ONNX replay. The table gives 12-second body-frame measurements for
the commanded axis. The corresponding Warp and native checkpoint replays
had zero resets, and the ONNX verifier recorded zero physics warnings.
| Command | Target | Warp mean | Native ONNX mean | ONNX mean absolute error |
| --- | ---: | ---: | ---: | ---: |
| Forward, m/s | +0.25 | +0.259 | +0.258 | 0.020 |
| Reverse, m/s | −0.25 | −0.241 | −0.242 | 0.015 |
| Left, m/s | +0.12 | +0.105 | +0.105 | 0.028 |
| Right, m/s | −0.12 | −0.152 | −0.152 | 0.034 |
| Yaw left, rad/s | +0.30 | +0.217 | +0.215 | 0.108 |
| Yaw right, rad/s | −0.30 | −0.296 | −0.297 | 0.068 |
The idle ONNX replay held both soles loaded, with mean body-frame speeds
of +0.00002 m/s forward, −0.0022 m/s lateral and +0.0046 rad/s yaw. In a
separate test, each of W, S, A, D, J and L was held for 12 seconds, including
the configured two-second ramp to its saved command limit. The highest
per-axis mean absolute error across the six tests was 0.023 m/s forward,
0.037 m/s lateral and 0.102 rad/s yaw. Escape brought the robot back to idle
in each test, with zero physics warnings.
The final forward gait still used short, low steps. Both soles carried load
in 51% of post-transient samples, and peak sole clearance was 5.7 mm. Left
motion unloaded one sole in 49% of samples, with peak sole clearance 9.0 mm.
These results describe simulated operation on the training plane.
+97
View File
@@ -0,0 +1,97 @@
# g1_example operator controls -- how a person's input becomes the command the
# policy observes.
#
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
# every control name from `controller/vocabulary.json`. Run
# `python -m controller --vocabulary` to see the list; it is the same one the
# deployed controller refuses a binding against.
#
# The pad and the keyboard are two paths, each bound straight to the command
# axes -- a stick with a sign, a key with the direction it pushes -- and
# neither names a control of the other. Either device drives alone; the one
# touched last drives.
#
# Two rules this file inherits from the specification it replaced:
# * Every value carries its source. A number without one cannot be told
# apart from a guess.
# * A consumer fails on a missing or malformed value rather than
# substituting a default. A sign that defaults to +1 is a robot that walks
# backwards while the screen looks perfectly normal.
schema_version: 2
command:
- term: twist
# The observation term these numbers are written into.
feeds: velocity_commands
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
frame: body
# Order is the order the command term writes them, and therefore the order
# they appear in the observation.
axes:
- name: lin_vel_x
unit: m/s
positive: forward
- name: lin_vel_y
unit: m/s
positive: left
- name: ang_vel_z
unit: rad/s
positive: counter-clockwise
scaling:
# Full deflection is the edge of the range the policy was **trained** on,
# not an absolute speed -- so a stick means "as fast as this policy was
# ever asked to go" and cannot ask for something off-distribution.
rule: full_deflection_is_range_edge
formula: axis * (range_max if axis >= 0 else -range_min)
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
# what the policy trained on; taking the larger end for both quietly
# commands something it never saw.
ends_scaled_separately: true
devices:
gamepad:
# The stick position *is* the command.
scheme: absolute
layout: xbox
axes:
lin_vel_x: { source: Ly, sign: -1 }
lin_vel_y: { source: Lx, sign: -1 }
ang_vel_z: { source: Rx, sign: -1 }
# MEASURED: the signs are not measured against the robot's pad service.
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
# is **down**, ABS_X positive is **right** -- and the service passes both
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
# the robot is what would settle it.
#
# The service applies a 0.15 deadband and **rescales**, so full deflection
# still reaches ±1. A consumer that applies its own on top narrows the
# range a person can ask for and cannot tell that it has.
deadzone: device_reported_rescaled
# Hands the command back to the random sampler in `play --task`; on a robot, which
# has no sampler, it lets go of everything.
release_button: B
keyboard:
# Keystrokes bound to what they do: each pushes one direction of one axis,
# "+" being the axis's own positive direction as `positive:` above words
# it, so the keyboard has no signs of its own -- there is no stick for a
# sign to be relative to. A virtual pad read through the pad's mapping
# until 2026-09-29.
scheme: keys
# Seconds a key is held for full deflection: the axis climbs linearly
# until then and is back at rest the moment the key comes up, as a stick
# let go of. Asked for on 2026-09-26; measured against nothing.
full_after_s: 2.0
axes:
# W S and the arrows walk, A D and the arrows step sideways, J L turn:
# the layout as the user
# decided it on 2026-09-29. Every letter is also a MuJoCo viewer
# shortcut -- W flips the scene to wireframe -- which was accepted for a
# keyboard laid out for the hands; left and right also step a paused
# scene.
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
ang_vel_z: { "+": [key_j], "-": [key_l] }
# Lets go of everything, as the pad's release button does.
release: [key_escape]
+267
View File
@@ -0,0 +1,267 @@
<mujoco model="g1_29dof_rev_1_0">
<compiler angle="radian" />
<default>
<default class="g1">
<site group="5" rgba="1 0 0 1" />
<default class="visual">
<geom type="mesh" contype="0" conaffinity="0" group="2" density="0" material="silver" />
</default>
<default class="collision">
<geom type="capsule" group="3" rgba="0.2 0.6 0.2 0.3" />
<default class="foot_capsule">
<geom size="0.01 0 0" />
</default>
</default>
</default>
</default>
<asset>
<material name="silver" rgba="0.7 0.7 0.7 1" />
<material name="black" rgba="0.2 0.2 0.2 1" />
<mesh name="pelvis" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/pelvis.STL" />
<mesh name="pelvis_contour_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/pelvis_contour_link.STL" />
<mesh name="left_hip_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_pitch_link.STL" />
<mesh name="left_hip_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_roll_link.STL" />
<mesh name="left_hip_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_yaw_link.STL" />
<mesh name="left_knee_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_knee_link.STL" />
<mesh name="left_ankle_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_ankle_pitch_link.STL" />
<mesh name="left_ankle_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_ankle_roll_link.STL" />
<mesh name="right_hip_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_pitch_link.STL" />
<mesh name="right_hip_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_roll_link.STL" />
<mesh name="right_hip_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_yaw_link.STL" />
<mesh name="right_knee_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_knee_link.STL" />
<mesh name="right_ankle_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_ankle_pitch_link.STL" />
<mesh name="right_ankle_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_ankle_roll_link.STL" />
<mesh name="waist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/waist_yaw_link_rev_1_0.STL" />
<mesh name="waist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/waist_roll_link_rev_1_0.STL" />
<mesh name="torso_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/torso_link_rev_1_0.STL" />
<mesh name="logo_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/logo_link.STL" />
<mesh name="head_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/head_link.STL" />
<mesh name="left_shoulder_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_pitch_link.STL" />
<mesh name="left_shoulder_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_roll_link.STL" />
<mesh name="left_shoulder_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_yaw_link.STL" />
<mesh name="left_elbow_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_elbow_link.STL" />
<mesh name="left_wrist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_roll_link.STL" />
<mesh name="left_wrist_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_pitch_link.STL" />
<mesh name="left_wrist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_yaw_link.STL" />
<mesh name="left_rubber_hand" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_rubber_hand.STL" />
<mesh name="right_shoulder_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_pitch_link.STL" />
<mesh name="right_shoulder_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_roll_link.STL" />
<mesh name="right_shoulder_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_yaw_link.STL" />
<mesh name="right_elbow_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_elbow_link.STL" />
<mesh name="right_wrist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_roll_link.STL" />
<mesh name="right_wrist_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_pitch_link.STL" />
<mesh name="right_wrist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_yaw_link.STL" />
<mesh name="right_rubber_hand" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_rubber_hand.STL" />
</asset>
<worldbody>
<body name="pelvis" childclass="g1" pos="0 0 0.793">
<inertial pos="0 0 -0.07605" quat="1 0 -0.000399148 0" mass="3.813" diaginertia="0.010549 0.0093089 0.0079184" />
<joint name="floating_base_joint" type="free" />
<geom class="visual" material="black" mesh="pelvis" />
<geom class="visual" mesh="pelvis_contour_link" />
<geom name="pelvis_collision" class="collision" size="0.07" pos="0 0 -0.08" type="sphere" />
<site name="imu_in_pelvis" pos="0.04525 0 -0.08339" size="0.01" />
<camera name="tracking" pos="1.734 -1.135 0.35" quat="0.683353 0.555871 0.298684 0.367183" mode="trackcom" />
<light pos="0 0 2" dir="0 0 -1" castshadow="false" mode="trackcom" />
<body name="left_hip_pitch_link" pos="0 0.064452 -0.1027">
<inertial pos="0.002741 0.047791 -0.02606" quat="0.954862 0.293964 0.0302556 0.030122" mass="1.35" diaginertia="0.00181517 0.00153422 0.00116212" />
<joint name="left_hip_pitch_joint" pos="0 0 0" axis="0 1 0" range="-2.5307 2.8798" armature="0.0101775" />
<geom class="visual" material="black" mesh="left_hip_pitch_link" />
<body name="left_hip_roll_link" pos="0 0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
<inertial pos="0.029812 -0.001045 -0.087934" quat="0.977808 -1.97119e-05 0.205576 -0.0403793" mass="1.52" diaginertia="0.00254986 0.00241169 0.00148755" />
<joint name="left_hip_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.5236 2.9671" armature="0.0251019" />
<geom class="visual" mesh="left_hip_roll_link" />
<geom name="left_hip_collision" class="collision" size="0.06 0.04" pos="0.02 0 -0.04" />
<body name="left_hip_yaw_link" pos="0.025001 0 -0.12412">
<inertial pos="-0.057709 -0.010981 -0.15078" quat="0.600598 0.15832 0.223482 0.751181" mass="1.702" diaginertia="0.00776166 0.00717575 0.00160139" />
<joint name="left_hip_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.7576 2.7576" armature="0.0101775" />
<geom class="visual" mesh="left_hip_yaw_link" />
<geom name="left_thigh_collision" class="collision" size="0.055 0.0761577" pos="-0.03 0 -0.1" quat="0.979578 0 0.201066 0" />
<body name="left_knee_link" pos="-0.078273 0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
<inertial pos="0.005457 0.003964 -0.12074" quat="0.923418 -0.0327699 0.0158246 0.382067" mass="1.932" diaginertia="0.0113804 0.0112778 0.00146458" />
<joint name="left_knee_joint" pos="0 0 0" axis="0 1 0" range="-0.087267 2.8798" armature="0.0251019" />
<geom class="visual" mesh="left_knee_link" />
<geom name="left_shin_collision" class="collision" size="0.045 0.075" pos="0.01 0 -0.075" />
<geom name="left_linkage_brace_collision" class="collision" size="0.03 0.04" pos="0.01 0 -0.24" />
<body name="left_ankle_pitch_link" pos="0 -9.4445e-05 -0.30001">
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074" diaginertia="1.89e-05 1.40805e-05 6.9195e-06" />
<joint name="left_ankle_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.87267 0.5236" armature="0.00721945" />
<geom class="visual" mesh="left_ankle_pitch_link" />
<body name="left_ankle_roll_link" pos="0 0 -0.017558">
<inertial pos="0.026505 0 -0.016425" quat="-0.000481092 0.728482 -0.000618967 0.685065" mass="0.608" diaginertia="0.00167218 0.0016161 0.000217621" />
<joint name="left_ankle_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.2618 0.2618" armature="0.00721945" />
<geom class="visual" material="black" mesh="left_ankle_roll_link" />
<geom name="left_foot1_collision" class="foot_capsule" size="0.01 0.025005" pos="0.075 -0.0265 -0.025" quat="0.707107 -0.0141393 0.706965 0" />
<geom name="left_foot2_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 -0.018 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="left_foot3_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 -0.01 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="left_foot4_collision" class="foot_capsule" size="0.01 0.093" pos="0.039 0 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="left_foot5_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 0.01 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="left_foot6_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 0.018 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="left_foot7_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 0.026 -0.025" quat="0.707107 0 0.707107 0" />
<site name="left_foot" pos="0.04 0 -0.037" />
</body>
</body>
</body>
</body>
</body>
</body>
<body name="right_hip_pitch_link" pos="0 -0.064452 -0.1027">
<inertial pos="0.002741 -0.047791 -0.02606" quat="0.954862 -0.293964 0.0302556 -0.030122" mass="1.35" diaginertia="0.00181517 0.00153422 0.00116212" />
<joint name="right_hip_pitch_joint" pos="0 0 0" axis="0 1 0" range="-2.5307 2.8798" armature="0.0101775" />
<geom class="visual" material="black" mesh="right_hip_pitch_link" />
<body name="right_hip_roll_link" pos="0 -0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
<inertial pos="0.029812 0.001045 -0.087934" quat="0.977808 1.97119e-05 0.205576 0.0403793" mass="1.52" diaginertia="0.00254986 0.00241169 0.00148755" />
<joint name="right_hip_roll_joint" pos="0 0 0" axis="1 0 0" range="-2.9671 0.5236" armature="0.0251019" />
<geom class="visual" mesh="right_hip_roll_link" />
<geom name="right_hip_collision" class="collision" size="0.06 0.04" pos="0.02 0 -0.04" />
<body name="right_hip_yaw_link" pos="0.025001 0 -0.12412">
<inertial pos="-0.057709 0.010981 -0.15078" quat="0.751181 0.223482 0.15832 0.600598" mass="1.702" diaginertia="0.00776166 0.00717575 0.00160139" />
<joint name="right_hip_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.7576 2.7576" armature="0.0101775" />
<geom class="visual" mesh="right_hip_yaw_link" />
<geom name="right_thigh_collision" class="collision" size="0.055 0.0761577" pos="-0.03 0 -0.1" quat="0.979578 0 0.201066 0" />
<body name="right_knee_link" pos="-0.078273 -0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
<inertial pos="0.005457 -0.003964 -0.12074" quat="0.923439 0.0345276 0.0116333 -0.382012" mass="1.932" diaginertia="0.011374 0.0112843 0.00146452" />
<joint name="right_knee_joint" pos="0 0 0" axis="0 1 0" range="-0.087267 2.8798" armature="0.0251019" />
<geom class="visual" mesh="right_knee_link" />
<geom name="right_shin_collision" class="collision" size="0.045 0.075" pos="0.01 0 -0.075" />
<geom name="right_linkage_brace_collision" class="collision" size="0.03 0.04" pos="0.01 0 -0.24" />
<body name="right_ankle_pitch_link" pos="0 9.4445e-05 -0.30001">
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074" diaginertia="1.89e-05 1.40805e-05 6.9195e-06" />
<joint name="right_ankle_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.87267 0.5236" armature="0.00721945" />
<geom class="visual" mesh="right_ankle_pitch_link" />
<body name="right_ankle_roll_link" pos="0 0 -0.017558">
<inertial pos="0.026505 0 -0.016425" quat="0.000481092 0.728482 0.000618967 0.685065" mass="0.608" diaginertia="0.00167218 0.0016161 0.000217621" />
<joint name="right_ankle_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.2618 0.2618" armature="0.00721945" />
<geom class="visual" material="black" mesh="right_ankle_roll_link" />
<geom name="right_foot1_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 -0.026 -0.025" quat="0.707107 0 0.707107 0" />
<geom name="right_foot2_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 -0.018 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="right_foot3_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 -0.01 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="right_foot4_collision" class="foot_capsule" size="0.01 0.093" pos="0.039 0 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="right_foot5_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 0.01 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="right_foot6_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 0.018 -0.025" quat="0.707107 0 -0.707107 0" />
<geom name="right_foot7_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 0.026 -0.025" quat="0.707107 0 0.707107 0" />
<site name="right_foot" pos="0.04 0 -0.037" />
</body>
</body>
</body>
</body>
</body>
</body>
<body name="waist_yaw_link">
<inertial pos="0.003494 0.000233 0.018034" quat="0.289697 0.591001 -0.337795 0.672821" mass="0.214" diaginertia="0.000163531 0.000107714 0.000102205" />
<joint name="waist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.0101775" />
<geom class="visual" mesh="waist_yaw_link" />
<body name="waist_roll_link" pos="-0.0039635 0 0.044">
<inertial pos="0 2.3e-05 0" quat="0.5 0.5 -0.5 0.5" mass="0.086" diaginertia="8.245e-06 7.079e-06 6.339e-06" />
<joint name="waist_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.52 0.52" armature="0.00721945" />
<geom class="visual" mesh="waist_roll_link" />
<body name="torso_link">
<inertial pos="0.00203158 0.000339683 0.184568" quat="0.999803 -6.03319e-05 0.0198256 0.00131986" mass="7.818" diaginertia="0.121847 0.109825 0.0273735" />
<joint name="waist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.52 0.52" armature="0.00721945" />
<geom class="visual" mesh="torso_link" />
<geom class="visual" pos="0.0039635 0 -0.044" quat="1 0 0 0" material="black" mesh="logo_link" />
<geom class="visual" pos="0.0039635 0 -0.044" material="black" mesh="head_link" />
<geom name="torso_collision" class="collision" size="0.09 0.06" pos="0.01 0 0.14" quat="0 1 0 0" />
<geom name="head_collision" class="collision" size="0.06" pos="0 0 0.43" type="sphere" />
<site name="imu_in_torso" pos="-0.03959 -0.00224 0.14792" size="0.01" />
<body name="left_shoulder_pitch_link" pos="0.0039563 0.10022 0.24778" quat="0.990264 0.139201 1.38722e-05 -9.86868e-05">
<inertial pos="0 0.035892 -0.011628" quat="0.654152 0.0130458 -0.326267 0.68225" mass="0.718" diaginertia="0.000465864 0.000432842 0.000406394" />
<joint name="left_shoulder_pitch_joint" pos="0 0 0" axis="0 1 0" range="-3.0892 2.6704" armature="0.00360972" />
<geom class="visual" mesh="left_shoulder_pitch_link" />
<body name="left_shoulder_roll_link" pos="0 0.038 -0.013831" quat="0.990268 -0.139172 0 0">
<inertial pos="-0.000227 0.00727 -0.063243" quat="0.701256 -0.0196223 -0.00710317 0.712604" mass="0.643" diaginertia="0.000691311 0.000618011 0.000388977" />
<joint name="left_shoulder_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.5882 2.2515" armature="0.00360972" />
<geom class="visual" mesh="left_shoulder_roll_link" />
<body name="left_shoulder_yaw_link" pos="0 0.00624 -0.1032">
<inertial pos="0.010773 -0.002949 -0.072009" quat="0.716879 -0.0964829 -0.0679942 0.687134" mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661" />
<joint name="left_shoulder_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.00360972" />
<geom class="visual" mesh="left_shoulder_yaw_link" />
<geom name="left_shoulder_yaw_collision" class="collision" size="0.035 0.065" pos="0 0 -0.015" quat="0 1 0 0" />
<body name="left_elbow_link" pos="0.015783 0 -0.080518">
<inertial pos="0.064956 0.004454 -0.010062" quat="0.541765 0.636132 0.388821 0.388129" mass="0.6" diaginertia="0.000443035 0.000421612 0.000259353" />
<joint name="left_elbow_joint" pos="0 0 0" axis="0 1 0" range="-1.0472 2.0944" armature="0.00360972" />
<geom class="visual" mesh="left_elbow_link" />
<geom name="left_elbow_yaw_collision" class="collision" size="0.035 0.045" pos="0.035 0 -0.01" quat="0.707107 0 -0.707107 0" />
<body name="left_wrist_roll_link" pos="0.1 0.00188791 -0.01">
<inertial pos="0.0171394 0.000537591 4.8864e-07" quat="0.575338 0.411667 -0.574906 0.411094" mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05" />
<joint name="left_wrist_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.97222 1.97222" armature="0.00360972" />
<geom class="visual" mesh="left_wrist_roll_link" />
<body name="left_wrist_pitch_link" pos="0.038 0 0">
<inertial pos="0.0229999 -0.00111685 -0.00111658" quat="0.249998 0.661363 0.293036 0.643608" mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648" />
<joint name="left_wrist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-1.61443 1.61443" armature="0.00425" />
<geom class="visual" mesh="left_wrist_pitch_link" />
<geom name="left_wrist_collision" class="collision" size="0.035 0.035" pos="0.025 0 0" quat="0.707107 0 -0.707107 0" />
<body name="left_wrist_yaw_link" pos="0.046 0 0">
<inertial pos="0.0708244 0.000191745 0.00161742" quat="0.510571 0.526295 0.468078 0.493188" mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566" />
<joint name="left_wrist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-1.61443 1.61443" armature="0.00425" />
<geom class="visual" mesh="left_wrist_yaw_link" />
<geom class="visual" pos="0.0415 0.003 0" quat="1 0 0 0" material="black" mesh="left_rubber_hand" />
<geom name="left_hand_collision" class="collision" size="0.035 0.0412311" pos="0.11 -0.01 0" quat="0.707107 -0.171499 -0.685994 0" />
<site name="left_palm" pos="0.08 0 0" size="0.01" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<body name="right_shoulder_pitch_link" pos="0.0039563 -0.10021 0.24778" quat="0.990264 -0.139201 1.38722e-05 9.86868e-05">
<inertial pos="0 -0.035892 -0.011628" quat="0.68225 -0.326267 0.0130458 0.654152" mass="0.718" diaginertia="0.000465864 0.000432842 0.000406394" />
<joint name="right_shoulder_pitch_joint" pos="0 0 0" axis="0 1 0" range="-3.0892 2.6704" armature="0.00360972" />
<geom class="visual" mesh="right_shoulder_pitch_link" />
<body name="right_shoulder_roll_link" pos="0 -0.038 -0.013831" quat="0.990268 0.139172 0 0">
<inertial pos="-0.000227 -0.00727 -0.063243" quat="0.712604 -0.00710317 -0.0196223 0.701256" mass="0.643" diaginertia="0.000691311 0.000618011 0.000388977" />
<joint name="right_shoulder_roll_joint" pos="0 0 0" axis="1 0 0" range="-2.2515 1.5882" armature="0.00360972" />
<geom class="visual" mesh="right_shoulder_roll_link" />
<body name="right_shoulder_yaw_link" pos="0 -0.00624 -0.1032">
<inertial pos="0.010773 0.002949 -0.072009" quat="0.687134 -0.0679942 -0.0964829 0.716879" mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661" />
<joint name="right_shoulder_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.00360972" />
<geom class="visual" mesh="right_shoulder_yaw_link" />
<geom name="right_shoulder_yaw_collision" class="collision" size="0.035 0.065" pos="0 0 -0.015" quat="0 1 0 0" />
<body name="right_elbow_link" pos="0.015783 0 -0.080518">
<inertial pos="0.064956 -0.004454 -0.010062" quat="0.388129 0.388821 0.636132 0.541765" mass="0.6" diaginertia="0.000443035 0.000421612 0.000259353" />
<joint name="right_elbow_joint" pos="0 0 0" axis="0 1 0" range="-1.0472 2.0944" armature="0.00360972" />
<geom class="visual" mesh="right_elbow_link" />
<geom name="right_elbow_yaw_collision" class="collision" size="0.035 0.045" pos="0.035 0 -0.01" quat="0.707107 0 -0.707107 0" />
<body name="right_wrist_roll_link" pos="0.1 -0.00188791 -0.01">
<inertial pos="0.0171394 -0.000537591 4.8864e-07" quat="0.411667 0.575338 -0.411094 0.574906" mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05" />
<joint name="right_wrist_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.97222 1.97222" armature="0.00360972" />
<geom class="visual" mesh="right_wrist_roll_link" />
<body name="right_wrist_pitch_link" pos="0.038 0 0">
<inertial pos="0.0229999 0.00111685 -0.00111658" quat="0.643608 0.293036 0.661363 0.249998" mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648" />
<joint name="right_wrist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-1.61443 1.61443" armature="0.00425" />
<geom class="visual" mesh="right_wrist_pitch_link" />
<geom name="right_wrist_collision" class="collision" size="0.035 0.035" pos="0.025 0 0" quat="0.707107 0 -0.707107 0" />
<body name="right_wrist_yaw_link" pos="0.046 0 0">
<inertial pos="0.0708244 -0.000191745 0.00161742" quat="0.493188 0.468078 0.526295 0.510571" mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566" />
<joint name="right_wrist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-1.61443 1.61443" armature="0.00425" />
<geom class="visual" mesh="right_wrist_yaw_link" />
<geom class="visual" pos="0.0415 -0.003 0" quat="1 0 0 0" material="black" mesh="right_rubber_hand" />
<geom name="right_hand_collision" class="collision" size="0.035 0.0412311" pos="0.11 0.01 0" quat="0.707107 0.171499 -0.685994 0" />
<site name="right_palm" pos="0.08 0 0" size="0.01" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</worldbody>
<contact>
<exclude body1="pelvis" body2="left_hip_roll_link" />
<exclude body1="pelvis" body2="right_hip_roll_link" />
<exclude body1="left_elbow_link" body2="left_wrist_pitch_link" />
<exclude body1="right_elbow_link" body2="right_wrist_pitch_link" />
</contact>
<sensor>
<gyro site="imu_in_pelvis" name="imu_ang_vel" />
<velocimeter site="imu_in_pelvis" name="imu_lin_vel" />
<accelerometer site="imu_in_pelvis" name="imu_lin_acc" />
<framezaxis objtype="body" objname="world" reftype="site" refname="imu_in_pelvis" name="imu_upvector" />
<subtreeangmom body="pelvis" name="root_angmom" />
</sensor>
</mujoco>
+138
View File
@@ -0,0 +1,138 @@
{
"schema": "g1_example_asset_provenance/1",
"local_source_commit": "004afda1ac1dafc3aedd4e9ca8c6d2242b399f57",
"source_xml": "rl/mjlab/asset_zoo/robots/unitree_g1/xmls/g1.xml",
"source_xml_sha256": "febdcbeffbbf84051556ae41a5ac1b43fb479a5d76bdb3f54824dbc2721c20aa",
"normalized_xml": "assets/g1_example/g1.xml",
"normalized_xml_sha256": "a29a91395e727135341377c71ec3c27c55e9849a07c4b5df3daf7fc7ddf77cf7",
"mesh_sha256": {
"head_link.STL": "005fb67fbd3eff94aa8bf4a6e83238174e9f91b6721f7111594322f223724411",
"left_ankle_pitch_link.STL": "d49e3abc6f5b12e532062cd575b87b5ef40cd2a3fc18f54a1ca5bba4f773d51d",
"left_ankle_roll_link.STL": "c4092af943141d4d9f74232f3cfa345afc6565f46a077793b8ae0e68b39dc33f",
"left_elbow_link.STL": "fa752198accd104d5c4c3a01382e45165b944fbbc5acce085059223324e5bed3",
"left_hip_pitch_link.STL": "4725168105ee768ee31638ef22b53f6be2d7641bfd7cfefe803488d884776fa4",
"left_hip_roll_link.STL": "91f25922ee8a7c3152790051bebad17b4d9cd243569c38fe340285ff93a97acf",
"left_hip_yaw_link.STL": "a16d88aa6ddac8083aa7ad55ed317bea44b1fa003d314fba88965b7ed0f3b55b",
"left_knee_link.STL": "8d92b9e3d3a636761150bb8025e32514c4602b91c7028d523ee42b3e632de477",
"left_rubber_hand.STL": "cff2221a690fa69303f61fce68f2d155c1517b52efb6ca9262dd56e0bc6e70fe",
"left_shoulder_pitch_link.STL": "f0d1cfd02fcf0d42f95e678eeca33da3afbcc366ffba5c052847773ec4f31d52",
"left_shoulder_roll_link.STL": "fb9df21687773522598dc384f1a2945c7519f11cbc8bd372a49170316d6eee88",
"left_shoulder_yaw_link.STL": "1aa97e9748e924336567992181f78c7cd0652fd52a4afcca3df6b2ef6f9e712e",
"left_wrist_pitch_link.STL": "b251d8e05047f695d0f536cd78c11973cfa4e78d08cfe82759336cc3471de3a9",
"left_wrist_roll_link.STL": "edc387c9a0ba8c2237e9b296d32531426fabeb6f53e58df45c76106bca74148c",
"left_wrist_yaw_link.STL": "83f8fb3a726bf9613d65dd14f0f447cb918c3c95b3938042a0c9c09749267d3b",
"logo_link.STL": "8571a0a19bc4916fa55f91449f51d5fdefd751000054865a842449429d5f155b",
"pelvis.STL": "5ba6bbc888e630550140d3c26763f10206da8c8bd30ed886b8ede41c61f57a31",
"pelvis_contour_link.STL": "5cc5c2c7a312329e3feeb2b03d3fc09fc29705bd01864f6767e51be959662420",
"right_ankle_pitch_link.STL": "15be426539ec1be70246d4d82a168806db64a41301af8b35c197a33348c787a9",
"right_ankle_roll_link.STL": "4b66222ea56653e627711b56d0a8949b4920da5df091da0ceb343f54e884e3a5",
"right_elbow_link.STL": "1be925d7aa268bb8fddf5362b9173066890c7d32092c05638608126e59d1e2ab",
"right_hip_pitch_link.STL": "e4f3c99d7f4a7d34eadbef9461fc66e3486cb5442db1ec50c86317d459f1a9c6",
"right_hip_roll_link.STL": "4c254ef66a356f492947f360dd931965477b631e3fcc841f91ccc46d945d54f6",
"right_hip_yaw_link.STL": "e479c2936ca2057e9eb2f7dff6c189b7419d7b8484dea0b298cbb36a2a6aa668",
"right_knee_link.STL": "63c4008449c9bbe701a6e2b557b7a252e90cf3a5abcf54cee46862b9a69f8ec8",
"right_rubber_hand.STL": "99533b778bca6246144fa511bb9d4e555e075c641f2a0251e04372869cd99d67",
"right_shoulder_pitch_link.STL": "24cdb387e0128dfe602770a81c56cdce3a0181d34d039a11d1aaf8819b7b8c02",
"right_shoulder_roll_link.STL": "962b97c48f9ce9e8399f45dd9522e0865d19aa9fd299406b2d475a8fc4a53e81",
"right_shoulder_yaw_link.STL": "a0b76489271da0c72461a344c9ffb0f0c6e64f019ea5014c1624886c442a2fe5",
"right_wrist_pitch_link.STL": "d22f8f3b3127f15a63e5be1ee273cd5075786c3142f1c3d9f76cbf43d2a26477",
"right_wrist_roll_link.STL": "a7ee9212ff5b94d6cb7f52bb1bbf3f352194d5b598acff74f4c77d340c5b344f",
"right_wrist_yaw_link.STL": "bc9dece2d12509707e0057ba2e48df8f3d56db0c79410212963a25e8a50f61a6",
"torso_link_rev_1_0.STL": "11ddb46f2098efbbd8816b1d65893632d8e78be936376c7cdcd6771899ccc723",
"waist_roll_link_rev_1_0.STL": "b67c347a05abc3e8ddae600d98a082bee273bb39f8e651647708b0a7140a8a97",
"waist_yaw_link_rev_1_0.STL": "ec6db442b11f25eed898b5add07940c85d804f300de24dcbd264ccd8be7d554c"
},
"license": "licenses/mujoco_menagerie-unitree_g1.txt",
"nq": 36,
"nv": 35,
"nu": 0,
"mass_kg": 33.341142000000005,
"roundtrip_max_absolute_differences": {
"body_mass": 0.0,
"body_inertia": 0.0,
"body_pos": 0.0,
"body_quat": 0.0,
"jnt_range": 0.0,
"jnt_axis": 0.0,
"geom_pos": 2.7755575615628914e-17,
"geom_quat": 4.161878089103954e-07,
"geom_size": 4.3743823403830095e-08,
"geom_contype": 0.0,
"geom_conaffinity": 0.0,
"qpos0": 0.0
},
"armature_before": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"armature_after": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0101775,
0.0251019,
0.0101775,
0.0251019,
0.00721945,
0.00721945,
0.0101775,
0.0251019,
0.0101775,
0.0251019,
0.00721945,
0.00721945,
0.0101775,
0.00721945,
0.00721945,
0.00360972,
0.00360972,
0.00360972,
0.00360972,
0.00360972,
0.00425,
0.00425,
0.00360972,
0.00360972,
0.00360972,
0.00360972,
0.00360972,
0.00425,
0.00425
]
}
+40
View File
@@ -0,0 +1,40 @@
{
"base_body": "pelvis",
"control_intent": "learned",
"family": "biped",
"foot_bodies": [
"left_ankle_roll_link",
"right_ankle_roll_link"
],
"position_joints": [
"left_hip_pitch_joint",
"left_hip_roll_joint",
"left_hip_yaw_joint",
"left_knee_joint",
"left_ankle_pitch_joint",
"left_ankle_roll_joint",
"right_hip_pitch_joint",
"right_hip_roll_joint",
"right_hip_yaw_joint",
"right_knee_joint",
"right_ankle_pitch_joint",
"right_ankle_roll_joint",
"waist_yaw_joint",
"waist_roll_joint",
"waist_pitch_joint",
"left_shoulder_pitch_joint",
"left_shoulder_roll_joint",
"left_shoulder_yaw_joint",
"left_elbow_joint",
"left_wrist_roll_joint",
"left_wrist_pitch_joint",
"left_wrist_yaw_joint",
"right_shoulder_pitch_joint",
"right_shoulder_roll_joint",
"right_shoulder_yaw_joint",
"right_elbow_joint",
"right_wrist_roll_joint",
"right_wrist_pitch_joint",
"right_wrist_yaw_joint"
]
}
+60
View File
@@ -0,0 +1,60 @@
{
"action_scales": {
"left_ankle_pitch_joint": 0.102076,
"left_ankle_roll_joint": 0.12836,
"left_elbow_joint": 0.170088,
"left_hip_pitch_joint": 0.126324,
"left_hip_roll_joint": 0.092448,
"left_hip_yaw_joint": 0.355179,
"left_knee_joint": 0.191022,
"left_shoulder_pitch_joint": 0.053176,
"left_shoulder_roll_joint": 0.039043,
"left_shoulder_yaw_joint": 0.239303,
"left_wrist_pitch_joint": 0.063044,
"left_wrist_roll_joint": 0.489431,
"left_wrist_yaw_joint": 0.070853,
"right_ankle_pitch_joint": 0.102076,
"right_ankle_roll_joint": 0.12836,
"right_elbow_joint": 0.170127,
"right_hip_pitch_joint": 0.126313,
"right_hip_roll_joint": 0.092451,
"right_hip_yaw_joint": 0.355601,
"right_knee_joint": 0.191022,
"right_shoulder_pitch_joint": 0.053254,
"right_shoulder_roll_joint": 0.039045,
"right_shoulder_yaw_joint": 0.239308,
"right_wrist_pitch_joint": 0.063045,
"right_wrist_roll_joint": 0.489431,
"right_wrist_yaw_joint": 0.070853,
"waist_pitch_joint": 0.099043,
"waist_roll_joint": 0.028443,
"waist_yaw_joint": 0.133551
},
"controls_path": "controls.yaml",
"family": "biped",
"model_path": "g1.xml",
"penalized_contact_geoms": [
"left_hip_collision",
"left_thigh_collision",
"left_shin_collision",
"left_linkage_brace_collision",
"right_hip_collision",
"right_thigh_collision",
"right_shin_collision",
"right_linkage_brace_collision",
"left_shoulder_yaw_collision",
"left_elbow_yaw_collision",
"left_wrist_collision",
"left_hand_collision",
"right_shoulder_yaw_collision",
"right_elbow_yaw_collision",
"right_wrist_collision",
"right_hand_collision"
],
"robot_id": "g1_example",
"termination_contact_geoms": [
"pelvis_collision",
"torso_collision",
"head_collision"
]
}
+79
View File
@@ -0,0 +1,79 @@
{
"damping_ratio": 0.85,
"effort_limits": {
"left_ankle_pitch_joint": 50.0,
"left_ankle_roll_joint": 50.0,
"left_elbow_joint": 25.0,
"left_hip_pitch_joint": 88.0,
"left_hip_roll_joint": 139.0,
"left_hip_yaw_joint": 88.0,
"left_knee_joint": 139.0,
"left_shoulder_pitch_joint": 25.0,
"left_shoulder_roll_joint": 25.0,
"left_shoulder_yaw_joint": 25.0,
"left_wrist_pitch_joint": 5.0,
"left_wrist_roll_joint": 25.0,
"left_wrist_yaw_joint": 5.0,
"right_ankle_pitch_joint": 50.0,
"right_ankle_roll_joint": 50.0,
"right_elbow_joint": 25.0,
"right_hip_pitch_joint": 88.0,
"right_hip_roll_joint": 139.0,
"right_hip_yaw_joint": 88.0,
"right_knee_joint": 139.0,
"right_shoulder_pitch_joint": 25.0,
"right_shoulder_roll_joint": 25.0,
"right_shoulder_yaw_joint": 25.0,
"right_wrist_pitch_joint": 5.0,
"right_wrist_roll_joint": 25.0,
"right_wrist_yaw_joint": 5.0,
"waist_pitch_joint": 50.0,
"waist_roll_joint": 50.0,
"waist_yaw_joint": 88.0
},
"foot_bodies": [
"left_ankle_roll_link",
"right_ankle_roll_link"
],
"frequency_hz": 14.0,
"inertia_mode": "coupled",
"joint_frequency_hz": {
"left_ankle_pitch_joint": 28.0,
"left_ankle_roll_joint": 28.0,
"right_ankle_pitch_joint": 28.0,
"right_ankle_roll_joint": 28.0
},
"pose": {
"left_ankle_pitch_joint": -0.12,
"left_ankle_roll_joint": 0.0,
"left_elbow_joint": 0.8,
"left_hip_pitch_joint": -0.12,
"left_hip_roll_joint": 0.0,
"left_hip_yaw_joint": 0.0,
"left_knee_joint": 0.24,
"left_shoulder_pitch_joint": 0.2,
"left_shoulder_roll_joint": 0.35,
"left_shoulder_yaw_joint": 0.0,
"left_wrist_pitch_joint": 0.0,
"left_wrist_roll_joint": 0.0,
"left_wrist_yaw_joint": 0.0,
"right_ankle_pitch_joint": -0.12,
"right_ankle_roll_joint": 0.0,
"right_elbow_joint": 0.8,
"right_hip_pitch_joint": -0.12,
"right_hip_roll_joint": 0.0,
"right_hip_yaw_joint": 0.0,
"right_knee_joint": 0.24,
"right_shoulder_pitch_joint": 0.2,
"right_shoulder_roll_joint": -0.35,
"right_shoulder_yaw_joint": 0.0,
"right_wrist_pitch_joint": 0.0,
"right_wrist_roll_joint": 0.0,
"right_wrist_yaw_joint": 0.0,
"waist_pitch_joint": 0.0,
"waist_roll_joint": 0.0,
"waist_yaw_joint": 0.0
},
"root_body": "pelvis",
"seconds": 2
}
+93
View File
@@ -0,0 +1,93 @@
# Unitree Go1 quadruped training example
The model comes from the local mjlab asset zoo at
[unitree_go1/xmls/go1.xml](../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/go1.xml).
The vendored source entered this checkout in commit
`004afda1ac1dafc3aedd4e9ca8c6d2242b399f57`. The source XML SHA-256 is
`2f1856b0c83994c009cafbad1f5eac38d09efa1a64db73681ff72c9237822db3`.
The five STL assets are referenced in place. The model and meshes are from
MuJoCo Menagerie, copyright 2016–2022 HangZhou YuShu TECHNOLOGY CO., LTD.
(Unitree Robotics), under the
[BSD 3-Clause License](../../licenses/mujoco_menagerie-unitree_go1.txt).
See the repository [NOTICE](../../NOTICE).
`go1.xml` reuses the source geometry, masses, joint stops, collision geoms and
reference frames. It adds reflected rotor inertia from the vendored
[go1_constants.py](../../rl/mjlab/asset_zoo/robots/unitree_go1/go1_constants.py)
and rebases mesh paths. `stance.json` uses the same source for the seed joint
positions and effort limits. Its gains come from the model inertia at the
calibrated pose, using 14 Hz and a damping ratio of 1.25. At that setting a
2-second native hold supported the robot on all four named feet. Peak base
displacement was 16.1 mm, peak joint error was 0.0562 rad, and peak effort
reached 20.0% of its joint limit. The original vendored gains let the stance
settle onto a calf in the same test.
`robot.json` sets each policy action scale to 60% of its effort limit divided
by its calibrated proportional gain. This is an exploration scale for the
simulator. Thigh and calf ground contacts carry a penalty. Trunk and head
ground contacts terminate the episode. The only task file is the calibrated
[robot specification](../../tasks/go1_example/robot_spec.py), which uses the
locked quadruped template.
## Reproduce the intake
Run from the checkout root with its installed Python environment and fresh
report paths.
```bash
python scripts/onboard.py inspect assets/go1_example/go1.xml \
--profile assets/go1_example/profile.json --report out/go1_example/intake.json
python scripts/onboard.py calibrate --model assets/go1_example/go1.xml \
--config assets/go1_example/stance.json --report out/go1_example/calibration.json
python scripts/train.py --task go1_example --backend warp --device cuda \
--num-envs 2048 --headless --no-tensorboard --max-iterations 1000
```
The measured stance and any learned policy results are simulation evidence.
## Measured training run
The RTX 5080 completed 1,000 Warp PPO updates with 2,048 environments and 24
steps per environment per update, totaling 49,152,000 environment steps. The
run reached all three curriculum levels. Checkpoint 900 was selected from
that run after its four feet remained loaded for the final three seconds of
a 12-second idle test. Checkpoint 999 tracked motion but unloaded its
front-right foot during the same idle check.
With a +0.40 m/s command in an 18-second Warp replay, checkpoint 900
averaged +0.401 m/s with 0.022 m/s mean absolute forward error and no resets.
The first saved checkpoint was nearly stationary with 0.400 m/s mean
absolute forward error under the same command. Both used the saved curriculum
ranges, 400 Hz physics, 50 Hz control and observation noise disabled.
The following 12-second tests used held body-frame commands, native MuJoCo
physics at 400 Hz and 50 Hz control. Both columns use checkpoint 900 with
observation noise disabled. The checkpoint replay had zero resets in each
test. The standalone ONNX replay had zero MuJoCo warnings.
| Commanded axis and target | Checkpoint mean | ONNX mean | ONNX mean absolute error |
| --- | ---: | ---: | ---: |
| Forward +0.40 m/s | +0.400 | +0.400 | 0.022 |
| Reverse −0.35 m/s | −0.337 | −0.337 | 0.018 |
| Left +0.25 m/s | +0.232 | +0.232 | 0.019 |
| Right −0.25 m/s | −0.235 | −0.235 | 0.018 |
| Yaw left +0.60 rad/s | +0.587 | +0.587 | 0.039 |
| Yaw right −0.60 rad/s | −0.593 | −0.591 | 0.040 |
With an idle command, ONNX replay averaged +0.0002 m/s forward, −0.0014 m/s
lateral and −0.0014 rad/s yaw. A separate keyboard test held W, S, A, D, J and
L for 12 seconds each, including the configured two-second ramp to the saved
command limit. Peak per-axis mean absolute tracking error across these tests
was 0.038 m/s for translation and 0.082 rad/s for yaw. Each test stayed upright
and settled after Escape.
The packaged runtime reproduced seven 12-second probes, including sampled
joint state and per-axis body-frame tracking metrics. The app includes the
asset license and provenance along with English and Chinese operator manuals.
After building the app, use the repository's native reference checker.
```bash
python scripts/deploy.py --check-reference out/platform_release_20261104/go1_example.app
python scripts/play.py --app out/platform_release_20261104/go1_example.app --headless \
--steps 200 --command 0.4 0 0
```
+97
View File
@@ -0,0 +1,97 @@
# go1_example operator controls -- how a person's input becomes the command the
# policy observes.
#
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
# every control name from `controller/vocabulary.json`. Run
# `python -m controller --vocabulary` to see the list; it is the same one the
# deployed controller refuses a binding against.
#
# The pad and the keyboard are two paths, each bound straight to the command
# axes -- a stick with a sign, a key with the direction it pushes -- and
# neither names a control of the other. Either device drives alone; the one
# touched last drives.
#
# Two rules this file inherits from the specification it replaced:
# * Every value carries its source. A number without one cannot be told
# apart from a guess.
# * A consumer fails on a missing or malformed value rather than
# substituting a default. A sign that defaults to +1 is a robot that walks
# backwards while the screen looks perfectly normal.
schema_version: 2
command:
- term: twist
# The observation term these numbers are written into.
feeds: velocity_commands
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
frame: body
# Order is the order the command term writes them, and therefore the order
# they appear in the observation.
axes:
- name: lin_vel_x
unit: m/s
positive: forward
- name: lin_vel_y
unit: m/s
positive: left
- name: ang_vel_z
unit: rad/s
positive: counter-clockwise
scaling:
# Full deflection is the edge of the range the policy was **trained** on,
# not an absolute speed -- so a stick means "as fast as this policy was
# ever asked to go" and cannot ask for something off-distribution.
rule: full_deflection_is_range_edge
formula: axis * (range_max if axis >= 0 else -range_min)
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
# what the policy trained on; taking the larger end for both quietly
# commands something it never saw.
ends_scaled_separately: true
devices:
gamepad:
# The stick position *is* the command.
scheme: absolute
layout: xbox
axes:
lin_vel_x: { source: Ly, sign: -1 }
lin_vel_y: { source: Lx, sign: -1 }
ang_vel_z: { source: Rx, sign: -1 }
# MEASURED: the signs are not measured against the robot's pad service.
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
# is **down**, ABS_X positive is **right** -- and the service passes both
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
# the robot is what would settle it.
#
# The service applies a 0.15 deadband and **rescales**, so full deflection
# still reaches ±1. A consumer that applies its own on top narrows the
# range a person can ask for and cannot tell that it has.
deadzone: device_reported_rescaled
# Hands the command back to the random sampler in `play --task`; on a robot, which
# has no sampler, it lets go of everything.
release_button: B
keyboard:
# Keystrokes bound to what they do: each pushes one direction of one axis,
# "+" being the axis's own positive direction as `positive:` above words
# it, so the keyboard has no signs of its own -- there is no stick for a
# sign to be relative to. A virtual pad read through the pad's mapping
# until 2026-09-29.
scheme: keys
# Seconds a key is held for full deflection: the axis climbs linearly
# until then and is back at rest the moment the key comes up, as a stick
# let go of. Asked for on 2026-09-26; measured against nothing.
full_after_s: 2.0
axes:
# W S and the arrows walk, A D and the arrows step sideways, J L turn:
# the layout as the user
# decided it on 2026-09-29. Every letter is also a MuJoCo viewer
# shortcut -- W flips the scene to wireframe -- which was accepted for a
# keyboard laid out for the hands; left and right also step a paused
# scene.
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
ang_vel_z: { "+": [key_j], "-": [key_l] }
# Lets go of everything, as the pad's release button does.
release: [key_escape]
+154
View File
@@ -0,0 +1,154 @@
<mujoco model="go1">
<compiler angle="radian" />
<default>
<default class="go1">
<default class="visual">
<geom type="mesh" contype="0" conaffinity="0" group="2" material="dark" />
</default>
<default class="collision">
<geom type="capsule" group="3" />
<default class="thigh1">
<geom size="0.015 0 0" />
</default>
<default class="thigh2">
<geom size="0.015 0 0" />
</default>
<default class="thigh3">
<geom size="0.015 0 0" />
</default>
<default class="calf1">
<geom size="0.01 0 0" />
</default>
<default class="calf2">
<geom size="0.01 0 0" />
</default>
<default class="foot">
<geom size="0.023 0 0" type="sphere" />
</default>
</default>
</default>
</default>
<asset>
<material name="dark" rgba="0.2 0.2 0.2 1" />
<mesh name="trunk" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/trunk.stl" />
<mesh name="hip" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/hip.stl" />
<mesh name="thigh_mirror" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/thigh_mirror.stl" />
<mesh name="calf" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/calf.stl" />
<mesh name="thigh" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/thigh.stl" />
</asset>
<worldbody>
<body name="trunk" childclass="go1" pos="0 0 0.445">
<inertial pos="0.0223 0.002 -0.0005" quat="-0.00342088 0.705204 0.000106698 0.708996" mass="5.204" diaginertia="0.0716565 0.0630105 0.0168101" />
<joint name="floating_base_joint" type="free" />
<geom class="visual" mesh="trunk" />
<geom name="trunk_collision" class="collision" size="0.13 0.09 0.05" type="box" />
<geom name="head_collision" class="collision" size="0.06" pos="0.24 0 0" type="sphere" />
<site name="head" pos="0.3 0 0" size="0.02" group="5" rgba="1 0 0 1" />
<site name="imu" pos="-0.01592 -0.06659 -0.00617" group="5" />
<camera name="tracking" pos="0.846 -1.3 0.316" quat="0.791164 0.554139 0.148485 0.211997" mode="trackcom" />
<camera name="top" pos="-1 0 1" quat="0.653281 0.270598 -0.270598 -0.653281" mode="trackcom" />
<camera name="side" pos="0 -1 0.3" quat="0.850651 0.525731 0 0" mode="trackcom" />
<camera name="back" pos="-1 0 0.3" quat="0.601501 0.371748 -0.371748 -0.601501" mode="trackcom" />
<light pos="0 0 2" dir="0 0 -1" mode="trackcom" />
<body name="FR_hip" pos="0.1881 -0.04675 0">
<inertial pos="-0.0049166 0.00762615 -8.865e-05" quat="0.507341 0.514169 0.495027 0.482891" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
<joint name="FR_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
<geom class="visual" mesh="hip" />
<geom name="FR_hip_collision" class="collision" size="0.05 0.01" pos="0 -0.03 0" quat="0.707107 -0.707107 0 0" />
<body name="FR_thigh" pos="0 -0.08 0">
<inertial pos="-0.00304722 0.019315 -0.0305004" quat="0.65243 -0.0272313 0.0775126 0.753383" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
<joint name="FR_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
<geom class="visual" mesh="thigh_mirror" />
<geom name="FR_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
<geom name="FR_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
<geom name="FR_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
<body name="FR_calf" pos="0 0 -0.213">
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
<joint name="FR_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
<geom class="visual" mesh="calf" />
<geom name="FR_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
<geom name="FR_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
<geom name="FR_foot_collision" class="foot" pos="0 0 -0.213" />
<site name="FR" pos="0 0 -0.213" size="0.023" group="5" />
</body>
</body>
</body>
<body name="FL_hip" pos="0.1881 0.04675 0">
<inertial pos="-0.0049166 -0.00762615 -8.865e-05" quat="0.482891 0.495027 0.514169 0.507341" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
<joint name="FL_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
<geom class="visual" mesh="hip" />
<geom name="FL_hip_collision" class="collision" size="0.05 0.01" pos="0 0.03 0" quat="0.707107 0.707107 0 0" />
<body name="FL_thigh" pos="0 0.08 0">
<inertial pos="-0.00304722 -0.019315 -0.0305004" quat="0.753383 0.0775126 -0.0272313 0.65243" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
<joint name="FL_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
<geom class="visual" mesh="thigh" />
<geom name="FL_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
<geom name="FL_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
<geom name="FL_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
<body name="FL_calf" pos="0 0 -0.213">
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
<joint name="FL_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
<geom class="visual" mesh="calf" />
<geom name="FL_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
<geom name="FL_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
<geom name="FL_foot_collision" class="foot" pos="0 0 -0.213" />
<site name="FL" pos="0 0 -0.213" size="0.023" group="5" />
</body>
</body>
</body>
<body name="RR_hip" pos="-0.1881 -0.04675 0">
<inertial pos="0.0049166 0.00762615 -8.865e-05" quat="0.495027 0.482891 0.507341 0.514169" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
<joint name="RR_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
<geom class="visual" quat="0 0 0 -1" mesh="hip" />
<geom name="RR_hip_collision" class="collision" size="0.05 0.01" pos="0 -0.03 0" quat="0.707107 -0.707107 0 0" />
<body name="RR_thigh" pos="0 -0.08 0">
<inertial pos="-0.00304722 0.019315 -0.0305004" quat="0.65243 -0.0272313 0.0775126 0.753383" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
<joint name="RR_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
<geom class="visual" mesh="thigh_mirror" />
<geom name="RR_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
<geom name="RR_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
<geom name="RR_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
<body name="RR_calf" pos="0 0 -0.213">
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
<joint name="RR_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
<geom class="visual" mesh="calf" />
<geom name="RR_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
<geom name="RR_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
<geom name="RR_foot_collision" class="foot" pos="0 0 -0.213" />
<site name="RR" pos="0 0 -0.213" size="0.023" group="5" />
</body>
</body>
</body>
<body name="RL_hip" pos="-0.1881 0.04675 0">
<inertial pos="0.0049166 -0.00762615 -8.865e-05" quat="0.514169 0.507341 0.482891 0.495027" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
<joint name="RL_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
<geom class="visual" quat="0 0 1 0" mesh="hip" />
<geom name="RL_hip_collision" class="collision" size="0.05 0.01" pos="0 0.03 0" quat="0.707107 0.707107 0 0" />
<body name="RL_thigh" pos="0 0.08 0">
<inertial pos="-0.00304722 -0.019315 -0.0305004" quat="0.753383 0.0775126 -0.0272313 0.65243" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
<joint name="RL_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
<geom class="visual" mesh="thigh" />
<geom name="RL_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
<geom name="RL_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
<geom name="RL_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
<body name="RL_calf" pos="0 0 -0.213">
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
<joint name="RL_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
<geom class="visual" mesh="calf" />
<geom name="RL_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
<geom name="RL_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
<geom name="RL_foot_collision" class="foot" pos="0 0 -0.213" />
<site name="RL" pos="0 0 -0.213" size="0.023" group="5" />
</body>
</body>
</body>
</body>
</worldbody>
<sensor>
<gyro site="imu" name="imu_ang_vel" />
<velocimeter site="imu" name="imu_lin_vel" />
<accelerometer site="imu" name="imu_lin_acc" />
<framezaxis objtype="body" objname="world" reftype="site" refname="imu" name="imu_upvector" />
<subtreeangmom body="trunk" name="root_angmom" />
</sensor>
</mujoco>
+11
View File
@@ -0,0 +1,11 @@
{
"base_body": "trunk",
"control_intent": "learned",
"family": "quadruped",
"foot_geoms": [
"FR_foot_collision",
"FL_foot_collision",
"RR_foot_collision",
"RL_foot_collision"
]
}
+46
View File
@@ -0,0 +1,46 @@
{
"action_scales": {
"FL_calf_joint": 0.18693,
"FL_hip_joint": 0.083974,
"FL_thigh_joint": 0.087029,
"FR_calf_joint": 0.18693,
"FR_hip_joint": 0.084209,
"FR_thigh_joint": 0.087029,
"RL_calf_joint": 0.18693,
"RL_hip_joint": 0.083974,
"RL_thigh_joint": 0.087029,
"RR_calf_joint": 0.18693,
"RR_hip_joint": 0.084209,
"RR_thigh_joint": 0.087029
},
"controls_path": "controls.yaml",
"family": "quadruped",
"model_path": "go1.xml",
"penalized_contact_geoms": [
"FR_thigh_collision1",
"FR_thigh_collision2",
"FR_thigh_collision3",
"FR_calf_collision1",
"FR_calf_collision2",
"FL_thigh_collision1",
"FL_thigh_collision2",
"FL_thigh_collision3",
"FL_calf_collision1",
"FL_calf_collision2",
"RR_thigh_collision1",
"RR_thigh_collision2",
"RR_thigh_collision3",
"RR_calf_collision1",
"RR_calf_collision2",
"RL_thigh_collision1",
"RL_thigh_collision2",
"RL_thigh_collision3",
"RL_calf_collision1",
"RL_calf_collision2"
],
"robot_id": "go1_example",
"termination_contact_geoms": [
"trunk_collision",
"head_collision"
]
}
+40
View File
@@ -0,0 +1,40 @@
{
"damping_ratio": 1.25,
"effort_limits": {
"FL_calf_joint": 35.55,
"FL_hip_joint": 23.7,
"FL_thigh_joint": 23.7,
"FR_calf_joint": 35.55,
"FR_hip_joint": 23.7,
"FR_thigh_joint": 23.7,
"RL_calf_joint": 35.55,
"RL_hip_joint": 23.7,
"RL_thigh_joint": 23.7,
"RR_calf_joint": 35.55,
"RR_hip_joint": 23.7,
"RR_thigh_joint": 23.7
},
"foot_geoms": [
"FR_foot_collision",
"FL_foot_collision",
"RR_foot_collision",
"RL_foot_collision"
],
"frequency_hz": 14,
"pose": {
"FL_calf_joint": -1.8,
"FL_hip_joint": -0.1,
"FL_thigh_joint": 0.9,
"FR_calf_joint": -1.8,
"FR_hip_joint": 0.1,
"FR_thigh_joint": 0.9,
"RL_calf_joint": -1.8,
"RL_hip_joint": -0.1,
"RL_thigh_joint": 0.9,
"RR_calf_joint": -1.8,
"RR_hip_joint": 0.1,
"RR_thigh_joint": 0.9
},
"root_body": "trunk",
"seconds": 2
}
+92
View File
@@ -0,0 +1,92 @@
# hybrid_wheeled_smoke operator controls -- how a person's input becomes the command the
# policy observes.
#
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
# every control name from `controller/vocabulary.json`. Run
# `python -m controller --vocabulary` to see the list; it is the same one the
# deployed controller refuses a binding against.
#
# The pad and the keyboard are two paths, each bound straight to the command
# axes -- a stick with a sign, a key with the direction it pushes -- and
# neither names a control of the other. Either device drives alone; the one
# touched last drives.
#
# Two rules this file inherits from the specification it replaced:
# * Every value carries its source. A number without one cannot be told
# apart from a guess.
# * A consumer fails on a missing or malformed value rather than
# substituting a default. A sign that defaults to +1 is a robot that walks
# backwards while the screen looks perfectly normal.
schema_version: 2
command:
- term: twist
# The observation term these numbers are written into.
feeds: velocity_commands
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
frame: body
# Order is the order the command term writes them, and therefore the order
# they appear in the observation.
axes:
- name: lin_vel_x
unit: m/s
positive: forward
- name: lin_vel_y
unit: m/s
positive: left
- name: ang_vel_z
unit: rad/s
positive: counter-clockwise
scaling:
# Full deflection is the edge of the range the policy was **trained** on,
# not an absolute speed -- so a stick means "as fast as this policy was
# ever asked to go" and cannot ask for something off-distribution.
rule: full_deflection_is_range_edge
formula: axis * (range_max if axis >= 0 else -range_min)
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
# what the policy trained on; taking the larger end for both quietly
# commands something it never saw.
ends_scaled_separately: true
devices:
gamepad:
# The stick position *is* the command.
scheme: absolute
layout: xbox
axes:
lin_vel_x: { source: Ly, sign: -1 }
# The wheeled template scales this required pad channel to zero.
lin_vel_y: { source: Lx, sign: -1 }
ang_vel_z: { source: Rx, sign: -1 }
# MEASURED: the signs are not measured against the robot's pad service.
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
# is **down**, ABS_X positive is **right** -- and the service passes both
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
# the robot is what would settle it.
#
# The service applies a 0.15 deadband and **rescales**, so full deflection
# still reaches ±1. A consumer that applies its own on top narrows the
# range a person can ask for and cannot tell that it has.
deadzone: device_reported_rescaled
# Hands the command back to the random sampler in `play --task`; on a robot, which
# has no sampler, it lets go of everything.
release_button: B
keyboard:
# Keystrokes bound to what they do: each pushes one direction of one axis,
# "+" being the axis's own positive direction as `positive:` above words
# it, so the keyboard has no signs of its own -- there is no stick for a
# sign to be relative to. A virtual pad read through the pad's mapping
# until 2026-09-29.
scheme: keys
# Seconds a key is held for full deflection: the axis climbs linearly
# until then and is back at rest the moment the key comes up, as a stick
# let go of. Asked for on 2026-09-26; measured against nothing.
full_after_s: 2.0
axes:
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
lin_vel_y: {unbound: lateral command range is zero}
ang_vel_z: { "+": [key_a, key_left], "-": [key_d, key_right] }
# Lets go of everything, as the pad's release button does.
release: [key_escape]
+27
View File
@@ -0,0 +1,27 @@
<mujoco model="hybrid_wheeled_smoke">
<compiler angle="radian"/>
<worldbody>
<body name="base" pos="0 0 .27">
<freejoint name="root"/>
<geom name="trunk" type="box" size=".13 .09 .025" mass="1.5"/>
<geom name="back_caster" type="sphere" pos="-.11 0 -.245" size=".025" mass=".04" friction=".03 .01 .001"/>
<geom name="front_caster" type="sphere" pos=".11 0 -.245" size=".025" mass=".04" friction=".03 .01 .001"/>
<body name="left_leg" pos="0 .13 0">
<joint name="left_hip" axis="0 1 0" range="-.5 .5"/>
<geom type="capsule" fromto="0 0 -.025 0 0 -.13" size=".018" mass=".12"/>
<body name="left_wheel" pos="0 0 -.17">
<joint name="left_wheel_joint" axis="0 1 0"/>
<geom name="left_tire" type="cylinder" euler="1.570796326795 0 0" size=".1 .02" mass=".25"/>
</body>
</body>
<body name="right_leg" pos="0 -.13 0">
<joint name="right_hip" axis="0 1 0" range="-.5 .5"/>
<geom type="capsule" fromto="0 0 -.025 0 0 -.13" size=".018" mass=".12"/>
<body name="right_wheel" pos="0 0 -.17">
<joint name="right_wheel_joint" axis="0 1 0"/>
<geom name="right_tire" type="cylinder" euler="1.570796326795 0 0" size=".1 .02" mass=".25"/>
</body>
</body>
</body>
</worldbody>
</mujoco>
+21
View File
@@ -0,0 +1,21 @@
# Authored example cube, metres; repository Apache-2.0 license.
v -0.1 -0.1 -0.1
v 0.1 -0.1 -0.1
v 0.1 0.1 -0.1
v -0.1 0.1 -0.1
v -0.1 -0.1 0.1
v 0.1 -0.1 0.1
v 0.1 0.1 0.1
v -0.1 0.1 0.1
f 1 3 2
f 1 4 3
f 5 6 7
f 5 7 8
f 1 2 6
f 1 6 5
f 2 3 7
f 2 7 6
f 3 4 8
f 3 8 7
f 4 1 5
f 4 5 8
+28
View File
@@ -0,0 +1,28 @@
# Six-legged locomotion example
This synthetic MuJoCo robot has six radial legs and 18 actuated hinges.
The coxa yaw joint sweeps each leg horizontally. The femur and tibia pitch
joints lift and extend its foot. The body and links use box, capsule and
sphere collision geometry with explicitly specified masses and inertias.
The model's front is +X and its total mass is 5.14 kg.
`profile.json` identifies the six named supports and the locomotion intent.
`stance.json` supplies the nominal joint angles and trial PD parameters
used for native stance calibration. `robot.json` supplies joint action
scales and contact selections for the scaffold. The assets use the
repository's Apache 2.0 license.
```bash
python scripts/onboard.py inspect assets/multipod_example \
--profile assets/multipod_example/profile.json
python scripts/onboard.py calibrate --model assets/multipod_example/multipod.xml \
--config assets/multipod_example/stance.json --report out/multipod-calibration.json
python scripts/onboard.py scaffold-tier3 --config assets/multipod_example/robot.json \
--calibration out/multipod-calibration.json --tasks-root out/multipod-scaffold
python scripts/train.py --task multipod_example --backend native --device cpu \
--num_envs 4 --max-iterations 1 --headless --no-tensorboard
```
The checked-in robot specification is the calibrated task used by the
training command. The scaffold example writes a separate copy for inspection.
W/S command forward and reverse, A/D command left and right, and J/L command yaw.
+91
View File
@@ -0,0 +1,91 @@
# multipod_example operator controls -- how a person's input becomes the command the
# policy observes.
#
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
# every control name from `controller/vocabulary.json`. Run
# `python -m controller --vocabulary` to see the list; it is the same one the
# deployed controller refuses a binding against.
#
# The pad and the keyboard are two paths, each bound straight to the command
# axes -- a stick with a sign, a key with the direction it pushes -- and
# neither names a control of the other. Either device drives alone; the one
# touched last drives.
#
# Two rules this file inherits from the specification it replaced:
# * Every value carries its source. A number without one cannot be told
# apart from a guess.
# * A consumer fails on a missing or malformed value rather than
# substituting a default. A sign that defaults to +1 is a robot that walks
# backwards while the screen looks perfectly normal.
schema_version: 2
command:
- term: twist
# The observation term these numbers are written into.
feeds: velocity_commands
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
frame: body
# Order is the order the command term writes them, and therefore the order
# they appear in the observation.
axes:
- name: lin_vel_x
unit: m/s
positive: forward
- name: lin_vel_y
unit: m/s
positive: left
- name: ang_vel_z
unit: rad/s
positive: counter-clockwise
scaling:
# Full deflection is the edge of the range the policy was **trained** on,
# not an absolute speed -- so a stick means "as fast as this policy was
# ever asked to go" and cannot ask for something off-distribution.
rule: full_deflection_is_range_edge
formula: axis * (range_max if axis >= 0 else -range_min)
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
# what the policy trained on; taking the larger end for both quietly
# commands something it never saw.
ends_scaled_separately: true
devices:
gamepad:
# The stick position *is* the command.
scheme: absolute
layout: xbox
axes:
lin_vel_x: { source: Ly, sign: -1 }
lin_vel_y: { source: Lx, sign: -1 }
ang_vel_z: { source: Rx, sign: -1 }
# MEASURED: the signs are not measured against the robot's pad service.
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
# is **down**, ABS_X positive is **right** -- and the service passes both
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
# the robot is what would settle it.
#
# The service applies a 0.15 deadband and **rescales**, so full deflection
# still reaches ±1. A consumer that applies its own on top narrows the
# range a person can ask for and cannot tell that it has.
deadzone: device_reported_rescaled
# Hands the command back to the random sampler in `play --task`; on a robot, which
# has no sampler, it lets go of everything.
release_button: B
keyboard:
# Keystrokes bound to what they do: each pushes one direction of one axis,
# "+" being the axis's own positive direction as `positive:` above words
# it, so the keyboard has no signs of its own -- there is no stick for a
# sign to be relative to. A virtual pad read through the pad's mapping
# until 2026-09-29.
scheme: keys
# Seconds a key is held for full deflection: the axis climbs linearly
# until then and is back at rest the moment the key comes up, as a stick
# let go of. Asked for on 2026-09-26; measured against nothing.
full_after_s: 2.0
axes:
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
ang_vel_z: { "+": [key_j], "-": [key_l] }
# Lets go of everything, as the pad's release button does.
release: [key_escape]
+108
View File
@@ -0,0 +1,108 @@
<mujoco model="multipod_example">
<!-- Six radial legs with an outward coxa, pitch femur, and pitch tibia. -->
<compiler angle="radian" inertiafromgeom="false" />
<option timestep="0.0025" integrator="implicitfast" />
<worldbody>
<body name="base" pos="0 0 0.25">
<freejoint name="base_free" />
<inertial pos="0 0 0" mass="2.8" diaginertia="0.0114 0.0385 0.0476" />
<geom name="base_collision" type="box" size="0.20 0.105 0.035" rgba="0.16 0.28 0.40 1" />
<body name="left_front_coxa" pos="0.15 0.105 0" euler="0 0 0.7853981633974483">
<joint name="left_front_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="left_front_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="left_front_femur" pos="0.085 0 0">
<joint name="left_front_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="left_front_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="left_front_tibia" pos="0.12 0 0">
<joint name="left_front_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="left_front_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="left_front_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
<body name="left_middle_coxa" pos="0 0.105 0" euler="0 0 1.5707963267948966">
<joint name="left_middle_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="left_middle_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="left_middle_femur" pos="0.085 0 0">
<joint name="left_middle_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="left_middle_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="left_middle_tibia" pos="0.12 0 0">
<joint name="left_middle_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="left_middle_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="left_middle_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
<body name="left_rear_coxa" pos="-0.15 0.105 0" euler="0 0 2.356194490192345">
<joint name="left_rear_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="left_rear_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="left_rear_femur" pos="0.085 0 0">
<joint name="left_rear_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="left_rear_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="left_rear_tibia" pos="0.12 0 0">
<joint name="left_rear_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="left_rear_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="left_rear_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
<body name="right_front_coxa" pos="0.15 -0.105 0" euler="0 0 -0.7853981633974483">
<joint name="right_front_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="right_front_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="right_front_femur" pos="0.085 0 0">
<joint name="right_front_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="right_front_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="right_front_tibia" pos="0.12 0 0">
<joint name="right_front_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="right_front_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="right_front_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
<body name="right_middle_coxa" pos="0 -0.105 0" euler="0 0 -1.5707963267948966">
<joint name="right_middle_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="right_middle_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="right_middle_femur" pos="0.085 0 0">
<joint name="right_middle_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="right_middle_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="right_middle_tibia" pos="0.12 0 0">
<joint name="right_middle_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="right_middle_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="right_middle_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
<body name="right_rear_coxa" pos="-0.15 -0.105 0" euler="0 0 -2.356194490192345">
<joint name="right_rear_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
<geom name="right_rear_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
<body name="right_rear_femur" pos="0.085 0 0">
<joint name="right_rear_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
<geom name="right_rear_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
<body name="right_rear_tibia" pos="0.12 0 0">
<joint name="right_rear_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
<geom name="right_rear_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
<geom name="right_rear_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
</body>
</body>
</body>
</body>
</worldbody>
</mujoco>
+33
View File
@@ -0,0 +1,33 @@
{
"base_body": "base",
"control_intent": "wasd",
"family": "multipod",
"foot_geoms": [
"left_front_foot",
"left_middle_foot",
"left_rear_foot",
"right_front_foot",
"right_middle_foot",
"right_rear_foot"
],
"position_joints": [
"left_front_coxa_joint",
"left_front_femur_joint",
"left_front_tibia_joint",
"left_middle_coxa_joint",
"left_middle_femur_joint",
"left_middle_tibia_joint",
"left_rear_coxa_joint",
"left_rear_femur_joint",
"left_rear_tibia_joint",
"right_front_coxa_joint",
"right_front_femur_joint",
"right_front_tibia_joint",
"right_middle_coxa_joint",
"right_middle_femur_joint",
"right_middle_tibia_joint",
"right_rear_coxa_joint",
"right_rear_femur_joint",
"right_rear_tibia_joint"
]
}
+49
View File
@@ -0,0 +1,49 @@
{
"action_scales": {
"left_front_coxa_joint": 0.26,
"left_front_femur_joint": 0.32,
"left_front_tibia_joint": 0.44,
"left_middle_coxa_joint": 0.26,
"left_middle_femur_joint": 0.32,
"left_middle_tibia_joint": 0.44,
"left_rear_coxa_joint": 0.26,
"left_rear_femur_joint": 0.32,
"left_rear_tibia_joint": 0.44,
"right_front_coxa_joint": 0.26,
"right_front_femur_joint": 0.32,
"right_front_tibia_joint": 0.44,
"right_middle_coxa_joint": 0.26,
"right_middle_femur_joint": 0.32,
"right_middle_tibia_joint": 0.44,
"right_rear_coxa_joint": 0.26,
"right_rear_femur_joint": 0.32,
"right_rear_tibia_joint": 0.44
},
"controls_path": "controls.yaml",
"family": "multipod",
"model_path": "multipod.xml",
"penalized_contact_geoms": [
"left_front_coxa_collision",
"left_front_femur_collision",
"left_front_tibia_collision",
"left_middle_coxa_collision",
"left_middle_femur_collision",
"left_middle_tibia_collision",
"left_rear_coxa_collision",
"left_rear_femur_collision",
"left_rear_tibia_collision",
"right_front_coxa_collision",
"right_front_femur_collision",
"right_front_tibia_collision",
"right_middle_coxa_collision",
"right_middle_femur_collision",
"right_middle_tibia_collision",
"right_rear_coxa_collision",
"right_rear_femur_collision",
"right_rear_tibia_collision"
],
"robot_id": "multipod_example",
"termination_contact_geoms": [
"base_collision"
]
}
+101
View File
@@ -0,0 +1,101 @@
{
"effort_limits": {
"left_front_coxa_joint": 6,
"left_front_femur_joint": 9,
"left_front_tibia_joint": 9,
"left_middle_coxa_joint": 6,
"left_middle_femur_joint": 9,
"left_middle_tibia_joint": 9,
"left_rear_coxa_joint": 6,
"left_rear_femur_joint": 9,
"left_rear_tibia_joint": 9,
"right_front_coxa_joint": 6,
"right_front_femur_joint": 9,
"right_front_tibia_joint": 9,
"right_middle_coxa_joint": 6,
"right_middle_femur_joint": 9,
"right_middle_tibia_joint": 9,
"right_rear_coxa_joint": 6,
"right_rear_femur_joint": 9,
"right_rear_tibia_joint": 9
},
"foot_geoms": [
"left_front_foot",
"left_middle_foot",
"left_rear_foot",
"right_front_foot",
"right_middle_foot",
"right_rear_foot"
],
"height_joints": {
"left_front_foot": "left_front_tibia_joint",
"left_middle_foot": "left_middle_tibia_joint",
"left_rear_foot": "left_rear_tibia_joint",
"right_front_foot": "right_front_tibia_joint",
"right_middle_foot": "right_middle_tibia_joint",
"right_rear_foot": "right_rear_tibia_joint"
},
"kd": {
"left_front_coxa_joint": 0.42,
"left_front_femur_joint": 0.55,
"left_front_tibia_joint": 0.65,
"left_middle_coxa_joint": 0.42,
"left_middle_femur_joint": 0.55,
"left_middle_tibia_joint": 0.65,
"left_rear_coxa_joint": 0.42,
"left_rear_femur_joint": 0.55,
"left_rear_tibia_joint": 0.65,
"right_front_coxa_joint": 0.42,
"right_front_femur_joint": 0.55,
"right_front_tibia_joint": 0.65,
"right_middle_coxa_joint": 0.42,
"right_middle_femur_joint": 0.55,
"right_middle_tibia_joint": 0.65,
"right_rear_coxa_joint": 0.42,
"right_rear_femur_joint": 0.55,
"right_rear_tibia_joint": 0.65
},
"kp": {
"left_front_coxa_joint": 11,
"left_front_femur_joint": 17,
"left_front_tibia_joint": 19,
"left_middle_coxa_joint": 11,
"left_middle_femur_joint": 17,
"left_middle_tibia_joint": 19,
"left_rear_coxa_joint": 11,
"left_rear_femur_joint": 17,
"left_rear_tibia_joint": 19,
"right_front_coxa_joint": 11,
"right_front_femur_joint": 17,
"right_front_tibia_joint": 19,
"right_middle_coxa_joint": 11,
"right_middle_femur_joint": 17,
"right_middle_tibia_joint": 19,
"right_rear_coxa_joint": 11,
"right_rear_femur_joint": 17,
"right_rear_tibia_joint": 19
},
"max_adjust": 0.1,
"pose": {
"left_front_coxa_joint": 0.0,
"left_front_femur_joint": 0.42,
"left_front_tibia_joint": 0.95,
"left_middle_coxa_joint": 0.0,
"left_middle_femur_joint": 0.42,
"left_middle_tibia_joint": 0.95,
"left_rear_coxa_joint": 0.0,
"left_rear_femur_joint": 0.42,
"left_rear_tibia_joint": 0.95,
"right_front_coxa_joint": 0.0,
"right_front_femur_joint": 0.42,
"right_front_tibia_joint": 0.95,
"right_middle_coxa_joint": 0.0,
"right_middle_femur_joint": 0.42,
"right_middle_tibia_joint": 0.95,
"right_rear_coxa_joint": 0.0,
"right_rear_femur_joint": 0.42,
"right_rear_tibia_joint": 0.95
},
"root_body": "base",
"seconds": 2
}
+8
View File
@@ -0,0 +1,8 @@
{
"kind": "ik",
"joints": ["shoulder", "elbow"],
"end_effector_body": "tool_tip",
"kp": 20,
"kd": 0.75,
"effort_limit": 10
}
+7
View File
@@ -0,0 +1,7 @@
{
"kind": "joint_sliders",
"joints": ["shoulder", "elbow"],
"kp": 20,
"kd": 0.75,
"effort_limit": 10
}
+57
View File
@@ -0,0 +1,57 @@
<?xml version="1.0"?>
<robot name="planar_arm">
<material name="base_color"><color rgba="0.20 0.27 0.34 1"/></material>
<material name="link_color"><color rgba="0.25 0.63 0.78 1"/></material>
<material name="tip_color"><color rgba="0.91 0.68 0.32 1"/></material>
<link name="base">
<inertial>
<origin xyz="0 0 0.04"/>
<mass value="1.8"/>
<inertia ixx="0.004" iyy="0.004" izz="0.006" ixy="0" ixz="0" iyz="0"/>
</inertial>
<visual><origin xyz="0 0 0.04"/><geometry><box size="0.14 0.14 0.08"/></geometry>
<material name="base_color"/></visual>
<collision><origin xyz="0 0 0.04"/><geometry><box size="0.14 0.14 0.08"/></geometry></collision>
</link>
<link name="upper_link">
<inertial>
<origin xyz="0.11 0 0"/>
<mass value="0.4"/>
<inertia ixx="0.0005" iyy="0.002" izz="0.002" ixy="0" ixz="0" iyz="0"/>
</inertial>
<visual><origin xyz="0.11 0 0"/><geometry><box size="0.20 0.038 0.04"/></geometry>
<material name="link_color"/></visual>
<collision><origin xyz="0.11 0 0"/><geometry><box size="0.20 0.038 0.04"/></geometry></collision>
</link>
<link name="lower_link">
<inertial>
<origin xyz="0.09 0 0"/>
<mass value="0.25"/>
<inertia ixx="0.0003" iyy="0.001" izz="0.001" ixy="0" ixz="0" iyz="0"/>
</inertial>
<visual><origin xyz="0.09 0 0"/><geometry><box size="0.16 0.03 0.035"/></geometry>
<material name="link_color"/></visual>
<collision><origin xyz="0.09 0 0"/><geometry><box size="0.16 0.03 0.035"/></geometry></collision>
</link>
<link name="tool_tip">
<inertial>
<mass value="0.025"/>
<inertia ixx="0.00001" iyy="0.00001" izz="0.00001" ixy="0" ixz="0" iyz="0"/>
</inertial>
<visual><geometry><sphere radius="0.017"/></geometry><material name="tip_color"/></visual>
<collision><geometry><sphere radius="0.017"/></geometry></collision>
</link>
<joint name="shoulder" type="revolute">
<parent link="base"/><child link="upper_link"/><origin xyz="0 0 0.16"/>
<axis xyz="0 0 1"/><limit lower="-1.5" upper="1.5" effort="10" velocity="4"/>
<dynamics damping="0.2"/>
</joint>
<joint name="elbow" type="revolute">
<parent link="upper_link"/><child link="lower_link"/><origin xyz="0.22 0 0"/>
<axis xyz="0 0 1"/><limit lower="-2" upper="2" effort="10" velocity="4"/>
<dynamics damping="0.2"/>
</joint>
<joint name="tip_mount" type="fixed">
<parent link="lower_link"/><child link="tool_tip"/><origin xyz="0.18 0 0"/>
</joint>
</robot>
+13
View File
@@ -0,0 +1,13 @@
{
"kind": "trajectory",
"joints": ["shoulder", "elbow"],
"kp": 20,
"kd": 0.75,
"effort_limit": 10,
"trajectory": {
"schema": "mjrl-trajectory/1",
"joint_order": ["elbow", "shoulder"],
"time_s": [0, 0.4, 0.8, 1.2],
"q": [[0, 0], [-0.45, 0.25], [0.35, -0.2], [-0.4, 0.2]]
}
}
+11
View File
@@ -0,0 +1,11 @@
{
"root_body": "base",
"controller": "controller.json",
"controls": "controls.yaml",
"steps": 750,
"thresholds": {"site_atol_m": 0.003},
"probes": [{
"command": {"target_x": -0.08, "target_y": 0.08},
"site_positions": {"mjrl_ee": [0.32, 0.08, 0.16]}
}]
}
@@ -0,0 +1,12 @@
{
"root_body": "base",
"controller": "controller.json",
"controls": "controls.yaml",
"steps": 750,
"thresholds": {"position_atol": 0.008},
"probes": [{
"command": {"shoulder": 0.25, "elbow": -0.4},
"actuator_controls": {"mjrl_shoulder": 0.25, "mjrl_elbow": -0.4},
"joint_positions": {"shoulder": 0.25, "elbow": -0.4}
}]
}
+10
View File
@@ -0,0 +1,10 @@
{
"root_body": "base",
"controller": "controller.json",
"controls": "controls.yaml",
"steps": 1000,
"thresholds": {"position_atol": 0.008},
"probes": [{
"joint_positions": {"shoulder": 0.2, "elbow": -0.4}
}]
}
+104
View File
@@ -0,0 +1,104 @@
<!--
A minimal model for the framework's own smoke tests, unrelated to any real
robot.
The design goal is not to resemble anything, but to cover every path the
framework has to validate at the smallest possible scale:
· freejoint -- reading and writing a floating base's root state
· contact -- feet against the ground, exercising the contact solver
· position actuators -- the same PD control form as a real legged robot
· sensor -- framequat / gyro / velocimeter, through the batched
sensordata path
· left-right symmetry -- convenient for validating indexing and mirroring
4 actuated degrees of freedom plus 6 for the base, with primitive geometry
throughout (no meshes), so it simulates extremely fast -- suitable as the
control group for backend-consistency regressions and throughput benchmarks.
-->
<mujoco model="smoke_biped">
<compiler angle="radian" autolimits="true"/>
<option timestep="0.002" integrator="implicitfast"/>
<default>
<default class="robot">
<joint damping="0.05" armature="0.01"/>
<default class="motor">
<position kp="60" kv="2.0" forcerange="-20 20"/>
</default>
<default class="link">
<geom type="capsule" size="0.02" density="800" rgba="0.55 0.6 0.68 1"/>
</default>
<default class="foot">
<geom type="sphere" size="0.028" density="800" rgba="0.85 0.6 0.25 1"
friction="1.0 0.01 0.01" priority="1" condim="3"/>
</default>
</default>
</default>
<worldbody>
<light pos="0 0 2.5" dir="0 0 -1" directional="true"/>
<geom name="floor" type="plane" size="0 0 0.05" rgba="0.3 0.33 0.36 1"
friction="1.0 0.01 0.01" condim="3"/>
<body name="base" pos="0 0 0.42" childclass="robot">
<freejoint name="root"/>
<inertial pos="0 0 0" mass="2.0" diaginertia="0.012 0.016 0.010"/>
<geom name="torso" type="box" size="0.06 0.045 0.035" rgba="0.42 0.47 0.55 1"/>
<site name="imu" pos="0 0 0" size="0.008"/>
<body name="L_thigh" pos="0 0.05 -0.035">
<joint name="L_hip" axis="0 1 0" range="-1.4 1.4"/>
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
<body name="L_calf" pos="0 0 -0.16">
<joint name="L_knee" axis="0 1 0" range="-2.2 0"/>
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
<body name="L_foot" pos="0 0 -0.16">
<geom name="L_foot_geom" class="foot"/>
<site name="L_foot_site" size="0.008"/>
</body>
</body>
</body>
<body name="R_thigh" pos="0 -0.05 -0.035">
<joint name="R_hip" axis="0 1 0" range="-1.4 1.4"/>
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
<body name="R_calf" pos="0 0 -0.16">
<joint name="R_knee" axis="0 1 0" range="-2.2 0"/>
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
<body name="R_foot" pos="0 0 -0.16">
<geom name="R_foot_geom" class="foot"/>
<site name="R_foot_site" size="0.008"/>
</body>
</body>
</body>
</body>
</worldbody>
<actuator>
<position class="motor" name="L_hip_ctrl" joint="L_hip" ctrlrange="-1.4 1.4"/>
<position class="motor" name="L_knee_ctrl" joint="L_knee" ctrlrange="-2.2 0"/>
<position class="motor" name="R_hip_ctrl" joint="R_hip" ctrlrange="-1.4 1.4"/>
<position class="motor" name="R_knee_ctrl" joint="R_knee" ctrlrange="-2.2 0"/>
</actuator>
<sensor>
<framequat name="base_quat" objtype="site" objname="imu"/>
<gyro name="base_gyro" site="imu"/>
<velocimeter name="base_vel" site="imu"/>
<framepos name="L_foot_pos" objtype="site" objname="L_foot_site"/>
<framepos name="R_foot_pos" objtype="site" objname="R_foot_site"/>
</sensor>
<keyframe>
<!--
Standing pose: hip +0.3 / knee -0.6 (the knee's limit is [-2.2, 0], so it
only bends negative).
Vertically: 0.035 (half the base height) + 0.16*cos(0.3)*2 + 0.028 (foot
radius) ~= 0.369. The hip's forward swing 0.16*sin(0.3) and the knee's
return 0.16*sin(-0.3) cancel, so the feet land directly under the base.
-->
<key name="home"
qpos="0 0 0.369 1 0 0 0 0.3 -0.6 0.3 -0.6"
ctrl="0.3 -0.6 0.3 -0.6"/>
</keyframe>
</mujoco>
+67
View File
@@ -0,0 +1,67 @@
# Wheeled biped example
This synthetic MuJoCo robot has two articulated legs, each with a pitch
hip, pitch knee and continuous wheel hinge. The two tires provide the
ground support. The battery case hangs below the upper frame and lowers the
center of mass. At the calibrated neutral pose the 7.62 kg robot has a
0.421 m root height, a 0.19 m wheel radius and a 0.35 m track.
The locked wheeled-biped template applies position PD to the four leg joints
and velocity feedback to the two wheels. The model declares 12 N·m hip,
16 N·m knee and 5 N·m wheel effort limits. Forward, reverse and yaw controls
use the shared keyboard and gamepad vocabulary. The template's lateral
command range is zero.
These commands reproduce the intake and stance checks. The last command
writes a review copy of the generated task specification.
```bash
python scripts/onboard.py inspect assets/wheeled_biped_example \
--profile assets/wheeled_biped_example/profile.json
python scripts/onboard.py calibrate --model assets/wheeled_biped_example/robot.xml \
--config assets/wheeled_biped_example/stance.json --report out/wheeled_calibration.json
python scripts/onboard.py scaffold-tier3 --config assets/wheeled_biped_example/robot.json \
--calibration out/wheeled_calibration.json --tasks-root out/wheeled_specs
```
In the two-second native stance check, both tires maintained support over
the final measurement window, carrying 74.75 N of total normal load. The
base settled 2.12 mm with no measured rotation, self-contact or engine warning.
Calibration uses a bounded position hold for the wheels. Training and
replay drive wheel velocity. In a separate hold using zero wheel velocity
targets, an initial pitch of ±0.03 rad settled to within 0.00066 rad in two
seconds. The peak effort per hip, knee and wheel was 0.291, 0.171 and
0.076 N·m respectively. Only the tires touched the ground.
Copy `verify_policy.json` beside the native export's `controller.json` and
`controls.yaml`. Its forward, reverse, left yaw, right yaw and idle probes
evaluate tracking after a one-second warmup. A policy export needs to pass those
physical response checks before it can be packaged as a controlled app.
```bash
python scripts/train.py --task wheeled_biped_example --backend native --device cpu \
--num-envs 4 --max-iterations 1 --headless --no-tensorboard
python scripts/export.py --task wheeled_biped_example --backend native --device cpu \
--checkpoint <checkpoint.pt> --native-layout --out out/wheeled_policy
cp assets/wheeled_biped_example/verify_policy.json out/wheeled_policy/verify-input.json
python scripts/onboard.py verify --model out/wheeled_policy/robot.xml \
--config out/wheeled_policy/verify-input.json --report out/wheeled_policy/verification.json
```
One training iteration exercises the pipeline. Use a trained checkpoint
for the physical tracking probes.
A Warp run on an RTX 5080 used 1,024 environments and saved the selected
checkpoint after 201 updates. With observation noise off, six-second Warp
replays averaged +0.146 and −0.184 m/s for forward and reverse commands of
±0.18 m/s, and +0.308 and −0.347 rad/s for yaw commands of ±0.35 rad/s. All
five replays, including idle, ran without resets. The exported actor passed
the matching native verification probes.
The selected checkpoint exposes ±0.91 m/s forward and reverse and
±0.77 rad/s yaw. At full keyboard deflection, the native app averaged
+0.866 and −0.927 m/s and +0.769 and −0.773 rad/s. The 0.19 m tires and
5 rad/s wheel target limit give a nominal rolling speed of 0.95 m/s.
The final checkpoint after 1,000 updates averaged +0.899 m/s on a +1.30 m/s
command. The published app uses the earlier checkpoint and its measured
command range.
@@ -0,0 +1,92 @@
# wheeled_biped_example operator controls -- how a person's input becomes the command the
# policy observes.
#
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
# every control name from `controller/vocabulary.json`. Run
# `python -m controller --vocabulary` to see the list; it is the same one the
# deployed controller refuses a binding against.
#
# The pad and the keyboard are two paths, each bound straight to the command
# axes -- a stick with a sign, a key with the direction it pushes -- and
# neither names a control of the other. Either device drives alone; the one
# touched last drives.
#
# Two rules this file inherits from the specification it replaced:
# * Every value carries its source. A number without one cannot be told
# apart from a guess.
# * A consumer fails on a missing or malformed value rather than
# substituting a default. A sign that defaults to +1 is a robot that walks
# backwards while the screen looks perfectly normal.
schema_version: 2
command:
- term: twist
# The observation term these numbers are written into.
feeds: velocity_commands
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
frame: body
# Order is the order the command term writes them, and therefore the order
# they appear in the observation.
axes:
- name: lin_vel_x
unit: m/s
positive: forward
- name: lin_vel_y
unit: m/s
positive: left
- name: ang_vel_z
unit: rad/s
positive: counter-clockwise
scaling:
# Full deflection is the edge of the range the policy was **trained** on,
# not an absolute speed -- so a stick means "as fast as this policy was
# ever asked to go" and cannot ask for something off-distribution.
rule: full_deflection_is_range_edge
formula: axis * (range_max if axis >= 0 else -range_min)
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
# what the policy trained on; taking the larger end for both quietly
# commands something it never saw.
ends_scaled_separately: true
devices:
gamepad:
# The stick position *is* the command.
scheme: absolute
layout: xbox
axes:
lin_vel_x: { source: Ly, sign: -1 }
# The wheeled template scales this required pad channel to zero.
lin_vel_y: { source: Lx, sign: -1 }
ang_vel_z: { source: Rx, sign: -1 }
# MEASURED: the signs are not measured against the robot's pad service.
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
# is **down**, ABS_X positive is **right** -- and the service passes both
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
# the robot is what would settle it.
#
# The service applies a 0.15 deadband and **rescales**, so full deflection
# still reaches ±1. A consumer that applies its own on top narrows the
# range a person can ask for and cannot tell that it has.
deadzone: device_reported_rescaled
# Hands the command back to the random sampler in `play --task`; on a robot, which
# has no sampler, it lets go of everything.
release_button: B
keyboard:
# Keystrokes bound to what they do: each pushes one direction of one axis,
# "+" being the axis's own positive direction as `positive:` above words
# it, so the keyboard has no signs of its own -- there is no stick for a
# sign to be relative to. A virtual pad read through the pad's mapping
# until 2026-09-29.
scheme: keys
# Seconds a key is held for full deflection: the axis climbs linearly
# until then and is back at rest the moment the key comes up, as a stick
# let go of. Asked for on 2026-09-26; measured against nothing.
full_after_s: 2.0
axes:
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
lin_vel_y: {unbound: lateral command range is zero}
ang_vel_z: { "+": [key_a, key_left], "-": [key_d, key_right] }
# Lets go of everything, as the pad's release button does.
release: [key_escape]
@@ -0,0 +1,8 @@
{
"family": "wheeled_biped",
"control_intent": "wasd",
"base_body": "base",
"foot_geoms": ["left_tire", "right_tire"],
"position_joints": ["left_hip", "left_knee", "right_hip", "right_knee"],
"velocity_joints": ["left_wheel_joint", "right_wheel_joint"]
}
+21
View File
@@ -0,0 +1,21 @@
{
"robot_id": "wheeled_biped_example",
"family": "wheeled_biped",
"model_path": "robot.xml",
"controls_path": "controls.yaml",
"action_scales": {
"left_hip": 0.09,
"left_knee": 0.10,
"right_hip": 0.09,
"right_knee": 0.10
},
"velocity_joints": [
{"name": "left_wheel_joint", "kp": 0, "kd": 1.1, "effort_limit": 5, "action_scale": 5, "armature": 0.01},
{"name": "right_wheel_joint", "kp": 0, "kd": 1.1, "effort_limit": 5, "action_scale": 5, "armature": 0.01}
],
"penalized_contact_geoms": [
"left_thigh_collision", "left_shank_collision",
"right_thigh_collision", "right_shank_collision"
],
"termination_contact_geoms": ["upper_housing", "spine", "battery_case"]
}
+55
View File
@@ -0,0 +1,55 @@
<mujoco model="wheeled_biped_example">
<compiler angle="radian"/>
<option timestep="0.0025" integrator="implicitfast"/>
<default>
<joint damping="0.04" armature="0.012"/>
<geom friction="0.9 0.01 0.0004"/>
</default>
<worldbody>
<body name="base" pos="0 0 0.421">
<freejoint name="root"/>
<geom name="upper_housing" type="box" size="0.085 0.125 0.026"
mass="0.65" rgba="0.19 0.36 0.48 1"/>
<geom name="spine" type="box" pos="0 0 -0.145" size="0.027 0.040 0.125"
mass="0.35" rgba="0.25 0.49 0.63 1"/>
<geom name="battery_case" type="box" pos="0 0 -0.349"
size="0.140 0.090 0.033" mass="5.2" rgba="0.24 0.29 0.35 1"/>
<body name="left_thigh" pos="0 0.125 0">
<joint name="left_hip" axis="0 1 0" range="-0.7 0.7" actuatorfrcrange="-12 12"/>
<geom name="left_thigh_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
size="0.016" mass="0.24" rgba="0.89 0.47 0.19 1"/>
<body name="left_shank" pos="0 0 -0.115">
<joint name="left_knee" axis="0 1 0" range="-0.55 0.8" actuatorfrcrange="-16 16"/>
<geom name="left_shank_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
size="0.015" mass="0.21" rgba="0.89 0.62 0.25 1"/>
<body name="left_wheel" pos="0 0.050 -0.115">
<joint name="left_wheel_joint" axis="0 1 0" limited="false"
damping="0.015" armature="0.010" actuatorfrcrange="-5 5"/>
<geom name="left_tire" type="cylinder" euler="1.5707963267948966 0 0"
size="0.190 0.024" mass="0.24" rgba="0.12 0.13 0.16 1"/>
<geom name="left_hub" type="box" pos="0 0.024 0" size="0.071 0.004 0.012"
mass="0.02" contype="0" conaffinity="0" rgba="0.68 0.70 0.75 1"/>
</body>
</body>
</body>
<body name="right_thigh" pos="0 -0.125 0">
<joint name="right_hip" axis="0 1 0" range="-0.7 0.7" actuatorfrcrange="-12 12"/>
<geom name="right_thigh_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
size="0.016" mass="0.24" rgba="0.89 0.47 0.19 1"/>
<body name="right_shank" pos="0 0 -0.115">
<joint name="right_knee" axis="0 1 0" range="-0.55 0.8" actuatorfrcrange="-16 16"/>
<geom name="right_shank_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
size="0.015" mass="0.21" rgba="0.89 0.62 0.25 1"/>
<body name="right_wheel" pos="0 -0.050 -0.115">
<joint name="right_wheel_joint" axis="0 1 0" limited="false"
damping="0.015" armature="0.010" actuatorfrcrange="-5 5"/>
<geom name="right_tire" type="cylinder" euler="1.5707963267948966 0 0"
size="0.190 0.024" mass="0.24" rgba="0.12 0.13 0.16 1"/>
<geom name="right_hub" type="box" pos="0 -0.024 0" size="0.071 0.004 0.012"
mass="0.02" contype="0" conaffinity="0" rgba="0.68 0.70 0.75 1"/>
</body>
</body>
</body>
</body>
</worldbody>
</mujoco>
+37
View File
@@ -0,0 +1,37 @@
{
"root_body": "base",
"foot_geoms": ["left_tire", "right_tire"],
"pose": {
"left_hip": 0,
"left_knee": 0,
"left_wheel_joint": 0,
"right_hip": 0,
"right_knee": 0,
"right_wheel_joint": 0
},
"kp": {
"left_hip": 130,
"left_knee": 150,
"left_wheel_joint": 1,
"right_hip": 130,
"right_knee": 150,
"right_wheel_joint": 1
},
"kd": {
"left_hip": 3.4,
"left_knee": 3.7,
"left_wheel_joint": 1.1,
"right_hip": 3.4,
"right_knee": 3.7,
"right_wheel_joint": 1.1
},
"effort_limits": {
"left_hip": 12,
"left_knee": 16,
"left_wheel_joint": 5,
"right_hip": 12,
"right_knee": 16,
"right_wheel_joint": 5
},
"seconds": 2
}
@@ -0,0 +1,143 @@
{
"root_body": "robot/base",
"controller": "controller.json",
"controls": "controls.yaml",
"ground_geom": "terrain",
"foot_geoms": [
"robot/left_tire",
"robot/right_tire"
],
"steps": 2400,
"probes": [
{
"command": {
"lin_vel_x": 0.18,
"lin_vel_y": 0,
"ang_vel_z": 0
},
"evaluation_start_s": 1,
"expect": {
"base_velocity_mean_m_s": {
"target": [0.18, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_velocity_tracking_mae_m_s": {
"target": [0.18, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_angular_velocity_mean_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
},
"base_angular_velocity_tracking_mae_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
}
}
},
{
"command": {
"lin_vel_x": -0.18,
"lin_vel_y": 0,
"ang_vel_z": 0
},
"evaluation_start_s": 1,
"expect": {
"base_velocity_mean_m_s": {
"target": [-0.18, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_velocity_tracking_mae_m_s": {
"target": [-0.18, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_angular_velocity_mean_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
},
"base_angular_velocity_tracking_mae_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
}
}
},
{
"command": {
"lin_vel_x": 0,
"lin_vel_y": 0,
"ang_vel_z": 0.35
},
"evaluation_start_s": 1,
"expect": {
"base_velocity_mean_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_velocity_tracking_mae_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_angular_velocity_mean_rad_s": {
"target": [0, 0, 0.35],
"tolerance": [null, null, 0.12]
},
"base_angular_velocity_tracking_mae_rad_s": {
"target": [0, 0, 0.35],
"tolerance": [null, null, 0.12]
}
}
},
{
"command": {
"lin_vel_x": 0,
"lin_vel_y": 0,
"ang_vel_z": -0.35
},
"evaluation_start_s": 1,
"expect": {
"base_velocity_mean_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_velocity_tracking_mae_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_angular_velocity_mean_rad_s": {
"target": [0, 0, -0.35],
"tolerance": [null, null, 0.12]
},
"base_angular_velocity_tracking_mae_rad_s": {
"target": [0, 0, -0.35],
"tolerance": [null, null, 0.12]
}
}
},
{
"command": {
"lin_vel_x": 0,
"lin_vel_y": 0,
"ang_vel_z": 0
},
"evaluation_start_s": 1,
"expect": {
"base_velocity_mean_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_velocity_tracking_mae_m_s": {
"target": [0, 0, 0],
"tolerance": [0.08, 0.04, null]
},
"base_angular_velocity_mean_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
},
"base_angular_velocity_tracking_mae_rad_s": {
"target": [0, 0, 0],
"tolerance": [null, null, 0.12]
}
}
}
]
}
+143
View File
@@ -0,0 +1,143 @@
"""Game controllers, as their own axis.
A fourth thing alongside the framework, the tasks and the scenes, and separate for
the same reason a scene is: **it knows nothing about robots.** Nothing here has an
opinion about what a stick means. It reports that the left stick is at (0.4, -0.9)
and that A is down; deciding that this is 0.2 m/s forward belongs to the task, in
`mjrl.tasks.operator` or the native controller, the same way a key's meaning does.
controller/ this package: find a pad, read it, name it
vocabulary.json what a pad can say -- the dictionary, and the only
copy of that list anywhere
mjrl/tasks/operator.py training command semantics from controls.yaml
deploy/fsm/ the same dictionary, for the robot and the browser
scripts/ replay forwards input to the selected controller
`python -m controller --vocabulary` prints the dictionary. Three programs read
it and none of them writes it: this package, a task validating its
`controls.yaml`, and the deployed controller refusing a `[[fsm.button]]`.
## What it is for
`play.py` samples random velocity commands. That is right for seeing a policy
across its training distribution and useless for "walk it into the corner and see
what happens". The keyboard already covers that (`mjrl.viewer.keys` plus a task's
teleop term); a stick covers it better, because walking is a continuous thing and
a key press is not.
## Using it
import controller
pad = controller.open() # None if nothing is plugged in
if pad is not None:
s = pad.state() # a snapshot; never blocks
print(s.lx, s.ly, s.a)
`open()` returns None rather than raising when there is no pad, because "no
controller today" is the normal case and not an error. It *does* print why when
there is a device it cannot use -- a pad that is plugged in but unreadable is a
problem to be told about, and the cause is almost always permissions rather than
anything to do with this code.
## Permissions, which is where this actually fails on Linux
`/dev/input/event*` is `crw-rw---- root input`, so reading it needs membership of
the `input` group:
sudo usermod -aG input "$USER" # then log out and back in
Until that is done a paired, working, blinking controller reads as absent. This
package says so explicitly rather than looking like it simply found nothing --
`describe()` reports what it saw and why each candidate was rejected.
## A new top-level package needs the editable install refreshed
`pip install -e .` writes a **strict name-to-path table**, not a path entry, so a
package added after the install is invisible to anything that does not happen to
be run from the repository root. The symptom is confusing: `python -c "import
controller"` works (the current directory is on `sys.path`) while
`python scripts/play.py` raises `ModuleNotFoundError`, because a script's
`sys.path[0]` is `scripts/`. `pip install -e .` again, once, fixes it -- the same
thing was true of `scenes` when it arrived.
## Platform
Linux through evdev (`device.py`), Windows through XInput (`xinput.py`), both with
no dependency beyond the standard library. The Windows reader presents each
reading as the evdev table Linux's `xpad` driver would have produced, so both
end in the same `xbox.normalise` and a stick means the same number on either.
macOS exposes nothing equivalent without a package, and adding one for a
convenience feature is the wrong trade; `open()` returns None there, `describe()`
says it is the platform, and everything that uses it carries on without a pad.
"""
from __future__ import annotations
import sys
from . import device, xinput
from .device import Gamepad, GamepadState, PadInfo
from .xinput import XInputPad
__all__ = [
"Gamepad", "GamepadState", "XInputPad", "available", "describe", "find_pads", "open",
]
def _backend():
"""The module that reads pads on this platform, or None where nothing does."""
if sys.platform == "linux":
return device
if sys.platform == "win32":
return xinput
return None
def find_pads(readable_only: bool = False) -> list[PadInfo]:
"""Every pad this platform's reader can see, in a stable order."""
backend = _backend()
return backend.find_pads(readable_only) if backend is not None else []
def describe() -> str:
"""What was found and, for anything rejected, why. For diagnosing "the pad is
connected but nothing happens", which is otherwise indistinguishable from
"no pad"."""
backend = _backend()
if backend is None:
return (
f"pads are read on Linux (evdev) and Windows (XInput), not on "
f"{sys.platform}; the keyboard drives."
)
return backend.describe()
def available() -> bool:
"""Is there a controller this process can actually read?
Cheap: it stats the device nodes and checks readability (on Windows, asks each
XInput slot once), and never opens a reader thread. Safe to call while building
a config.
"""
return bool(find_pads(readable_only=True))
def open(name: str | None = None) -> Gamepad | XInputPad | None:
"""Open the first usable controller, or None.
Args:
name: substring of the device name to require, for a machine with more
than one pad. None takes the first.
"""
backend = _backend()
if backend is None:
return None
pads = backend.find_pads(readable_only=True)
if name is not None:
pads = [p for p in pads if name.lower() in p.name.lower()]
if not pads:
return None
pad = (XInputPad if backend is xinput else Gamepad)(pads[0])
pad.start()
return pad
+72
View File
@@ -0,0 +1,72 @@
"""Live readout of the connected pad: `python -m controller`.
The one thing no test on this side can settle is whether pushing a stick forward
reads as forward. Everything else about a controller can be checked from synthetic
values, but the mapping from a physical direction to a number needs a hand on the
pad -- so this exists to make that a five-second check rather than a guess, and to
be the first thing to run when a pad "does nothing".
python -m controller # live values
python -m controller --raw # every (type, code) the pad emits
python -m controller --vocabulary # the words a binding may use
`--raw` is for a pad whose layout `xbox.py` does not know: press the control, see
which code moves, and add it there.
"""
from __future__ import annotations
import sys
import time
import controller
from controller.xbox import layout_of
def main() -> int:
if "--vocabulary" in sys.argv:
# No pad needed: this is the dictionary, not a reading of one.
from . import vocabulary
print(vocabulary.describe(), end="")
return 0
raw_mode = "--raw" in sys.argv
print(controller.describe())
pad = controller.open()
if pad is None:
print("\nnothing to read. If a pad is listed above as unusable, the reason "
"is on that line.")
return 1
layout = layout_of(pad._ranges)
print(f"\n{pad.name}: {layout.name} layout, "
f"axes {sorted(pad._ranges)}")
print("\nPush each control and watch the numbers. Ctrl-C to stop.")
print("Expected: left stick up -> ly negative (evdev's Y is positive down),")
print(" left stick left -> lx negative, right stick right -> rx positive\n")
try:
while True:
if raw_mode:
items = sorted(pad.raw().items())
line = " ".join(f"{t:x}:{c:x}={v}" for (t, c), v in items if v)
else:
s = pad.state()
pressed = " ".join(sorted(s.buttons)) or "-"
line = (
f"L({s.lx:+.2f},{s.ly:+.2f}) R({s.rx:+.2f},{s.ry:+.2f}) "
f"LT {s.lt:.2f} RT {s.rt:.2f} "
f"hat({s.hat_x:+d},{s.hat_y:+d}) [{pressed}]"
)
print(f"\r{line:<100}", end="", flush=True)
time.sleep(0.05)
except KeyboardInterrupt:
print()
finally:
pad.close()
return 0
if __name__ == "__main__":
raise SystemExit(main())
+991
View File
@@ -0,0 +1,991 @@
"""Read a task's `controls.yaml`, the operator-control specification.
**The specification is the source of truth and this is the only reader.** The
signs, the splits and the bindings live in the file; `mjrl.tasks.operator` drives
`replay from what this returns, `scripts/export.py` embeds it in the deployment
contract, and `deploy/fsm/src/operator.rs` drives the robot and the browser from
that. Programs that must agree about which way is forward agree by reading one
file rather than by people reading each other's docstrings.
Everything here fails loudly, for the reason `load_servo_curve` gives about the
motor spec: every quiet alternative is worse than a stopped import. A missing
axis, a sign left out, a `sign: 0`, a binding naming a key that does not exist --
each would otherwise become a default, and a default sign is a robot that walks
the wrong way while the screen looks entirely normal.
## What the file says
`command` is a list, one entry per command term an operator drives -- one for a
velocity-tracking task and one more if the policy takes posture input. **The pad and the keyboard are two
paths**, each bound straight to what it does, and neither names a control of
the other (2026-09-29):
* **The pad** binds a stick or a trigger to a command axis, with a sign. A stick
can carry more than one axis: a binding may take part of its travel
(`travel: [0.0, 0.5]`), hand it on to the next one (`[0.0, 0.5, 0.5, 0.75]`
climbs over the first half and falls back to zero by three quarters), sit on
a layer a button brings in (`shifted: true`), or be one of several controls
summed into one axis (a list). The loader refuses two bindings that would
climb over the same part of the same stick, which is the one way this could go
quietly wrong.
* **The keyboard** binds keystrokes to one direction of an axis -- `"+"` is the
axis's own positive direction, as its `positive:` words it, so the keyboard
has no signs: there is no stick for a sign to be relative to. A keystroke is a
key (`key_j`), a modifier on its own (`shift`), or a modifier held with a key
(`shift+key_j`), which is a different keystroke from the key alone: J and
Shift + J are two things and never both (`controller/vocabulary.py`). A key
held pushes its direction further the longer it is held, full after
`full_after_s`, and let go it is back at rest. An axis a task deliberately
leaves off the keyboard says why (`unbound:`).
Until 2026-09-29 the keyboard was a *virtual pad*: a key held moved one of
the pad's controls, read through the pad's own mapping. That made the two
devices agree by construction, and it made a keyboard that could not lay
itself out for hands: a key could only be a stick, so J could not turn while
Shift + J twisted, and Shift, Alt and Ctrl could not hold an arm out. The
layout Control-agent 3.1 asks for needs both, so the keyboard became a path
of its own, and agreement is the file's: both paths name the same axes, and
`load_controls` refuses a command axis either one leaves out without saying.
* **An axis may move rather than be placed** (`integrate_s`), on the pad. A
deflection is then a speed -- full deflection carries the axis from its rest
to either end of its range in that many seconds -- and let go it stays where
it is. A body-height command is one use. `devices.gamepad.reset`
names the button whose tap puts every such axis back at its rest. The
keyboard places it as it places every other axis: a key held pushes it
towards its end and it is back at rest when let go.
* **A task may keep controls for itself.** The top-level `task:` names controls
no command reads and what the task does with each. Each device binds every
one of them: the pad
one control each (an axis for an `amount`, a d-pad direction for a `press`),
the keyboard one or more keystrokes. The task's own code answers them by
name, in replay a term of its own (`Operator.task_control`) and on the robot
its deploy hook (`deploy/fsm/src/hook.rs`).
"""
from __future__ import annotations
import hashlib
import itertools
import math
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import yaml
# The control dictionary: what a pad and a keyboard can say. Read, never
# repeated -- see `_AXES` below.
from controller import vocabulary as _vocabulary
__all__ = [
"CENTRE",
"SCHEMA",
"Axis",
"Binding",
"Command",
"Controls",
"KeyAxis",
"TaskControl",
"controller_contract",
"key_name",
"load_controls",
"stroke_name",
]
#: The schema this reader implements, and the contract block it writes is
#: `operator_controller/<this>`.
SCHEMA = 2
#: The scaling rule this repository implements. A file asking for another one is
#: refused rather than run under this one.
_SCALING_RULE = "full_deflection_is_range_edge"
#: The gamepad vocabulary, read from the dictionary rather than repeated.
#:
#: `controller/vocabulary.json` is the list and
#: `python -m controller --vocabulary` prints it. This module only decides which
#: of those words a task's controls file may use and what happens when it does
#: -- the meanings, which is the part that belongs to a task.
#:
#: There is no list here on purpose. There were three for a day: this one,
#: `deploy/fsm/src/vocabulary.rs`, and a prose block in a deploy manifest.
_AXES = _vocabulary.axes()
_BUTTONS = frozenset(_vocabulary.buttons())
#: Axis name -> the attribute a `controller` state object carries. A case
#: change and nothing else: `controller/xbox.py` reports the dictionary's names
#: now, and this is only Python's habit of lowercase attributes.
_STICK_FIELD = {name: name.lower() for name in _AXES}
#: Which of the dictionary's axes are sticks (-1..1) and which are triggers
#: (0..1). Read from the ranges the dictionary gives, not from the names.
_BIPOLAR = frozenset(a["name"] for a in _vocabulary.load()["axes"] if a["range"][0] < 0)
#: The four d-pad directions: where a task's `press` control goes on the pad.
#: A rocker is held or not, which is what a `press` reads, and the ten buttons
#: stay the bundle's for switching modes and the operator's for letting go.
_DPAD = frozenset(b["name"] for b in _vocabulary.load()["dpad"])
#: The notched keyboard's other half, retired on 2026-09-27: one key letting go
#: of every stick and trigger at once, needed while a key left its control where
#: the last press put it. A key let up is back at rest and the release lets go
#: of everything, so it did nothing either did not. Named only to refuse a file
#: that still has it.
CENTRE = "centre"
#: The gestures a `shift` implements: `hold`, the layer in for as long as the
#: button is held, and `toggle`, in from one click to the next. `rise` and
#: `fall` are one tick long, which a layer cannot be.
_SHIFT_GESTURES = ("hold", "toggle")
#: The keyboard scheme: keystrokes bound to what they do. See the module
#: docstring.
_KEYBOARD_SCHEME = "keys"
#: What a task control is: `amount`, 0 to 1 as far as it is held -- a trigger,
#: or a key's ramp -- and `press`, 1 while it is held and 0 otherwise.
_TASK_KINDS = ("amount", "press")
@dataclass(frozen=True)
class Axis:
name: str
unit: str
positive: str
#: The command config field holding where a centred stick puts this axis.
#: `None` is zero. A height is the case that needs it -- zero is a
#: body on the floor, and the range it would be clamped to is not where the
#: robot stands.
rest: str | None = None
#: Seconds full deflection takes to carry the axis from its rest to either
#: end of its range: the controls move it rather than place it, and let go
#: it stays where it is. `None` is an axis a control places, as a stick's
#: position places a walking speed.
integrate_s: float | None = None
@dataclass(frozen=True)
class Command:
"""One command term an operator drives. The file has a list of these."""
#: The command term's key in `cfg.commands`.
term: str
feeds: str
frame: str
axes: tuple[Axis, ...]
scaling: dict[str, Any]
@dataclass(frozen=True)
class Binding:
"""One pad control's share of one command axis."""
#: A dictionary axis: `Lx` `Ly` `Rx` `Ry` `LT` `RT`.
source: str
sign: float
#: The part of the control's travel this binding takes, as fractions of
#: full deflection. `(0.0, 0.5)` reaches full at half travel and holds
#: there; `(0.5, 1.0)` is silent until half travel and then climbs. Two
#: more, `(0.0, 0.5, 0.5, 0.75)`, are where it falls back: full until the
#: third, zero from the fourth on -- a binding that gives the control up to
#: the one climbing over the same stretch.
travel: tuple[float, ...] = (0.0, 1.0)
#: Live only while the `shift` button has the other layer in -- held, or
#: toggled, as `shift_gesture` says. A source that has a shifted binding
#: drives **only** that binding while shifted; every other source is
#: unaffected.
shifted: bool = False
@dataclass(frozen=True)
class KeyAxis:
"""One command axis on the keyboard: the keystrokes pushing it each way,
or why none do."""
#: Keystrokes towards the axis's own positive direction, as its `positive:`
#: words it, and away from it. Both empty exactly when `unbound` says why.
plus: tuple[str, ...] = ()
minus: tuple[str, ...] = ()
unbound: str | None = None
@dataclass(frozen=True)
class TaskControl:
"""A control the task answers itself, by name: no command reads it."""
name: str
#: `amount` (0..1, as far as it is held) or `press` (1 while held).
kind: str
#: What the task does with it, in the file's words.
does: str
@dataclass(frozen=True)
class Controls:
"""The specification, parsed: the commands, one operator for all of them."""
commands: tuple[Command, ...]
#: The pad: command axis name -> the controls that drive it, summed.
bindings: dict[str, tuple[Binding, ...]]
#: The button that brings the shifted layer in, or `None` for a file with none.
shift: str | None
release_button: str
#: Seconds a key is held for its direction to reach full deflection, or a
#: task's `amount` to reach 1. It climbs linearly until then and is back at
#: rest when the key comes up.
full_after_s: float
#: The keyboard: command axis name -> its keystrokes, every axis present.
key_axes: dict[str, KeyAxis]
#: Keystrokes that let go of everything, as the pad's release button does.
key_release: tuple[str, ...]
#: The controls the task keeps for itself, by name, in the file's order.
#: Empty for most tasks.
task: dict[str, TaskControl]
#: Task control -> the one pad control it is on.
pad_task: dict[str, str]
#: Task control -> the keystrokes it is on.
key_task: dict[str, tuple[str, ...]]
keyboard: dict[str, Any]
gamepad: dict[str, Any]
source_sha256: str
source: Path
#: How the shift brings its layer in: `hold`, for as long as it is held,
#: or `toggle`, from one click to the next. `None` with no shift.
shift_gesture: str | None = None
#: The button whose tap puts every integrating axis back at its rest, or
#: `None`. A tap is the button pressed and let go with no stick it could
#: have been reaching for moved in between.
reset: str | None = None
def command(self, term: str) -> Command:
for c in self.commands:
if c.term == term:
return c
raise KeyError(
f"{self.source}: describes the commands {[c.term for c in self.commands]}, "
f"not {term!r}. A term driven from a file that does not describe it "
"would take its bindings from nowhere."
)
@property
def sources(self) -> frozenset[str]:
"""Every pad axis some command binding reads."""
return frozenset(b.source for bs in self.bindings.values() for b in bs)
@property
def integrating(self) -> dict[str, float]:
"""Axis name -> `integrate_s`, for every axis the controls move rather
than place."""
return {a.name: a.integrate_s for c in self.commands for a in c.axes
if a.integrate_s is not None}
@property
def keystrokes(self) -> tuple[str, ...]:
"""Every keystroke the keyboard block binds, once each, in file order."""
strokes = [s for k in self.key_axes.values() for s in (*k.plus, *k.minus)]
strokes += self.key_release
strokes += [s for ss in self.key_task.values() for s in ss]
return tuple(dict.fromkeys(strokes))
class _StrictLoader(yaml.SafeLoader):
"""`yaml.safe_load`, except that a key written twice is an error.
PyYAML keeps the last of two equal keys and says nothing, so an axis written
twice in the keyboard's `axes` is one binding silently gone -- the keyboard
then cannot do something and nothing points at the line that took it away.
"""
def _mapping_without_duplicates(loader: yaml.SafeLoader, node: yaml.MappingNode) -> dict:
seen: set[Any] = set()
for key_node, _value in node.value:
key = loader.construct_object(key_node)
if key in seen:
raise ValueError(
f"line {key_node.start_mark.line + 1}: {key!r} is written twice. YAML "
"keeps the last one and drops the other without a word."
)
seen.add(key)
return loader.construct_mapping(node, deep=True)
_StrictLoader.add_constructor(
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _mapping_without_duplicates
)
def _require(doc: Any, path: str, where: Path) -> Any:
node = doc
for part in path.split("."):
if not isinstance(node, dict) or part not in node:
raise ValueError(f"{where}: missing {path}")
node = node[part]
if node is None:
raise ValueError(f"{where}: {path} is null; it has to be decided, not defaulted")
return node
def load_controls(path: Path) -> Controls:
"""Parse a task's `controls.yaml`. See the module docstring."""
if not path.exists():
raise FileNotFoundError(
f"{path} not found. It holds how an operator's input becomes the "
"command the policy observes, and neither play nor the exporter can "
"proceed without it."
)
raw = path.read_bytes()
try:
doc = yaml.load(raw.decode("utf-8"), Loader=_StrictLoader) # a SafeLoader subclass
except ValueError as exc:
raise ValueError(f"{path}: {exc}") from None
version = _require(doc, "schema_version", path)
if version != SCHEMA:
raise ValueError(
f"{path}: schema_version is {version!r}; this reader implements {SCHEMA}. "
"Schema 1 was retired -- regenerate the file with "
".claude/skills/controls/scripts/write_controls.py."
)
return _parse(doc, raw, path)
def _check_button(name: Any, what: str, path: Path) -> None:
# A button name is a contract, not a convention: `view` is on the pad and
# the robot's service does not send it, so binding it works on a bench and
# does nothing on the robot.
if name not in _BUTTONS:
extra = (" `view` is on the pad but the service does not publish it."
if name == "view" else "")
raise ValueError(
f"{path}: {what} is {name!r}, which the robot's gamepad service "
f"does not publish. It publishes {sorted(_BUTTONS)}.{extra}"
)
def _check_stroke(stroke: Any, where: str, path: Path) -> tuple[str | None, str]:
"""A keystroke, or a refusal naming where it was written."""
if not isinstance(stroke, str):
raise ValueError(f"{path}: {where} has {stroke!r}, which is not a keystroke")
try:
return _vocabulary.keystroke(stroke)
except ValueError as exc:
raise ValueError(
f"{path}: {where}: {exc}\n"
f" python -m controller --vocabulary\n"
f" (A key it lacks is added there, with what it also does in "
f"MuJoCo's viewer: the viewer calls the user callback in addition to its "
f"own shortcuts, so most keys do two things at once.)"
) from None
def _positive_seconds(value: Any) -> bool:
return (not isinstance(value, bool) and isinstance(value, (int, float))
and 0 < value < math.inf)
def _travel_is_ordered(travel: list) -> bool:
"""`0 <= from < to <= 1`, and with a fall `to <= back < off <= 1`.
A climb or a fall of zero length would be a step, which a stick's travel
cannot hold still on; the hold between them may be zero, and is, where one
binding's fall starts as the next one's climb does."""
a, z, *fall = travel
if not 0.0 <= a < z <= 1.0:
return False
return not fall or z <= fall[0] < fall[1] <= 1.0
def _field(node: Any, key: str, at: str, path: Path) -> Any:
"""`_require` for a node inside a list, so the message says which entry."""
if not isinstance(node, dict) or key not in node:
raise ValueError(f"{path}: missing {at}.{key}")
if node[key] is None:
raise ValueError(f"{path}: {at}.{key} is null; it has to be decided, not defaulted")
return node[key]
def _parse(doc: dict, raw: bytes, path: Path) -> Controls:
"""Every check here is a control that would drive nothing, drive two things,
or be out of one device's reach, without anything raising."""
commands = _parse_commands(doc, path)
names = [a.name for c in commands for a in c.axes]
task = _parse_task(doc, path)
gamepad = _require(doc, "devices.gamepad", path)
bindings, shift, gesture, release, reset = _parse_pad(gamepad, commands, path)
pad_task = _parse_pad_task(gamepad, task, bindings, {release, shift, reset}, path)
keyboard = _require(doc, "devices.keyboard", path)
full, key_axes, key_release, key_task = _parse_keyboard(keyboard, names, task, path)
return Controls(
commands=commands,
bindings=bindings,
shift=shift,
release_button=release,
full_after_s=float(full),
key_axes=key_axes,
key_release=key_release,
task=task,
pad_task=pad_task,
key_task=key_task,
keyboard=keyboard,
gamepad=gamepad,
source_sha256=hashlib.sha256(raw).hexdigest(),
source=path,
shift_gesture=gesture if shift is not None else None,
reset=reset,
)
def _parse_commands(doc: dict, path: Path) -> tuple[Command, ...]:
listed = _require(doc, "command", path)
if not isinstance(listed, list) or not listed:
raise ValueError(
f"{path}: `command` is a list of the commands an operator "
f"drives, one entry per command term; got {type(listed).__name__}"
)
commands = []
for i, entry in enumerate(listed):
at = f"command[{i}]"
axes = []
for j, a in enumerate(_field(entry, "axes", at, path)):
where = f"{at}.axes[{j}]"
integrate = a.get("integrate_s") if isinstance(a, dict) else None
if isinstance(a, dict) and "integrate_s" in a and not _positive_seconds(integrate):
raise ValueError(
f"{path}: {where}.integrate_s is {integrate!r}; it is the seconds full "
"deflection takes to carry the axis from its rest to either end of its "
"range, a positive number. Leave it out for an axis a control places."
)
axes.append(Axis(
name=_field(a, "name", where, path),
unit=_field(a, "unit", where, path),
positive=_field(a, "positive", where, path),
rest=a.get("rest"),
integrate_s=float(integrate) if integrate is not None else None,
))
if not axes:
raise ValueError(f"{path}: {at} names no axes")
scaling = _field(entry, "scaling", at, path)
if scaling.get("rule") != _SCALING_RULE:
raise ValueError(
f"{path}: {at}.scaling.rule is {scaling.get('rule')!r}; this repository "
f"implements {_SCALING_RULE!r}. Add the rule rather than running the wrong one."
)
commands.append(Command(
term=_field(entry, "term", at, path),
feeds=_field(entry, "feeds", at, path),
frame=_field(entry, "frame", at, path),
axes=tuple(axes),
scaling=scaling,
))
terms = [c.term for c in commands]
if len(set(terms)) != len(terms):
raise ValueError(f"{path}: a command term is listed twice: {terms}")
# Bindings are keyed by axis name, so a name two commands share would be one
# binding driving both.
names = [a.name for c in commands for a in c.axes]
if len(set(names)) != len(names):
raise ValueError(f"{path}: an axis name is used twice across the commands: {names}")
return tuple(commands)
def _parse_task(doc: dict, path: Path) -> dict[str, TaskControl]:
"""The top-level `task:`, the controls the task answers itself."""
declared = doc.get("task")
if declared is None:
if "task" in doc:
raise ValueError(f"{path}: task is null; leave it out for a task that keeps none")
return {}
if not isinstance(declared, dict) or not all(isinstance(n, str) for n in declared):
raise ValueError(
f"{path}: task is a {type(declared).__name__}; it maps each control the task "
"answers itself to its `kind` and what it `does`"
)
task = {}
for name, entry in declared.items():
kind = _field(entry, "kind", f"task.{name}", path)
if kind not in _TASK_KINDS:
raise ValueError(
f"{path}: task.{name}.kind is {kind!r}; a task control is `amount` (0 to "
"1, as far as it is held) or `press` (1 while it is held)"
)
does = entry.get("does")
if not isinstance(does, str) or not does.strip():
raise ValueError(f"{path}: the task keeps {name} and does not say what it does")
task[name] = TaskControl(name, kind, does)
return task
def _parse_pad(gamepad: dict, commands: tuple[Command, ...], path: Path):
"""The pad's command bindings, its shift, its release and its reset."""
names = [a.name for c in commands for a in c.axes]
for field in ("scheme", "layout", "axes", "deadzone", "release_button"):
_require(gamepad, field, path)
if gamepad["scheme"] != "absolute":
raise ValueError(
f"{path}: gamepad scheme is {gamepad['scheme']!r}. This file implements "
"`absolute` only: a stick's position is the command. An axis the controls "
"should move rather than place says `integrate_s` on the axis, for both "
"devices at once."
)
release = gamepad["release_button"]
_check_button(release, "release_button", path)
shift = gesture = None
if "shift" in gamepad:
declared = gamepad["shift"]
shift = _field(declared, "button", "devices.gamepad.shift", path)
_check_button(shift, "shift.button", path)
gesture = _field(declared, "gesture", "devices.gamepad.shift", path)
if gesture not in _vocabulary.gestures():
raise ValueError(
f"{path}: shift.gesture is {gesture!r}; the dictionary's gestures are "
f"{list(_vocabulary.gestures())}"
)
if gesture not in _SHIFT_GESTURES:
raise ValueError(
f"{path}: shift.gesture is {gesture!r}; a shift is "
f"{' or '.join(map(repr, _SHIFT_GESTURES))}. "
"A one-tick gesture brings a layer in for one tick."
)
if shift == release:
raise ValueError(
f"{path}: {shift} is both the shift and the release button. One "
"press would hand the command back and change what a stick means."
)
bound = gamepad["axes"]
stray = sorted(set(bound) - set(names))
if stray:
raise ValueError(
f"{path}: gamepad.axes names {stray}, which no command has. A typo "
f"here is a stick that drives nothing. The axes are {names}"
)
bindings: dict[str, tuple[Binding, ...]] = {}
for name in names:
entry = bound.get(name)
if not entry:
raise ValueError(f"{path}: no control drives {name}; every command axis needs one")
parts = []
for part in entry if isinstance(entry, list) else [entry]:
if not isinstance(part, dict) or "source" not in part:
raise ValueError(f"{path}: {name} has a binding with no source: {part!r}")
source = part["source"]
if source not in _STICK_FIELD:
raise ValueError(
f"{path}: {name} names {source!r}, which the robot's gamepad "
f"service does not publish. It publishes {list(_AXES)}"
)
sign = part.get("sign")
if sign not in (1, -1):
# Not clamped to a default: a sign is the one value here whose
# wrong answer looks exactly like the right one.
raise ValueError(f"{path}: {name} has sign {sign!r}; it must be 1 or -1")
travel = part.get("travel", [0.0, 1.0])
if (
not isinstance(travel, list) or len(travel) not in (2, 4)
or not all(isinstance(t, (int, float)) for t in travel)
or not _travel_is_ordered(travel)
):
raise ValueError(
f"{path}: {name} has travel {travel!r}; it is [from, to] with "
"0 <= from < to <= 1, or [from, to, back, off] with "
"0 <= from < to <= back < off <= 1, as fractions of full deflection"
)
shifted = part.get("shifted", False)
if type(shifted) is not bool:
raise ValueError(f"{path}: {name} has shifted {shifted!r}; it is true or false")
if shifted and shift is None:
raise ValueError(
f"{path}: {name} is on the shifted layer and the file declares no "
"shift button, so nothing can ever reach it"
)
parts.append(Binding(source, float(sign), tuple(float(t) for t in travel), shifted))
bindings[name] = tuple(parts)
# No two bindings may climb over the same stretch of the same control in the
# same layer: one deflection would then move two axes, and each binding
# alone looks correct. A binding's fall is not a claim -- it is the one
# stretch where a deflection may move two axes, because it is how the
# binding gives the control up to the next one.
claims: dict[tuple[str, bool], list[tuple[float, float, str]]] = {}
for name, parts in bindings.items():
for b in parts:
claims.setdefault((b.source, b.shifted), []).append((*b.travel[:2], name))
for (source, layer), spans in claims.items():
spans.sort()
for (_a0, b0, n0), (a1, b1, n1) in itertools.pairwise(spans):
if a1 < b0:
raise ValueError(
f"{path}: {n0} and {n1} both take {source} from {a1:g} to "
f"{min(b0, b1):g} of its travel{' on the shifted layer' if layer else ''}. "
"One deflection would move both; give them separate travel."
)
if shift is not None and not any(b.shifted for parts in bindings.values() for b in parts):
raise ValueError(f"{path}: {shift} is declared as the shift and no binding is shifted")
# The reset: a tap puts every integrating axis back at its rest. With no
# such axis it would be a button that does nothing, bound; on the release
# button it would be a second meaning for one press. It may be the shift --
# a hold can change height and a tap can put it back
# to standing, told apart by whether the stick moved.
reset = gamepad.get("reset")
if reset is not None or "reset" in gamepad:
_check_button(reset, "reset", path)
if reset == release:
raise ValueError(
f"{path}: {reset} is both the reset and the release button. The release "
"already puts every integrating axis back at its rest; a reset on it "
"is the same press read twice."
)
if not any(a.integrate_s is not None for c in commands for a in c.axes):
raise ValueError(
f"{path}: gamepad.reset is {reset}, and no axis has `integrate_s`. It "
"puts an integrating axis back at its rest; with none, it is a button "
"that does nothing and looks bound."
)
return bindings, shift, gesture, release, reset
def _parse_pad_task(
gamepad: dict, task: dict[str, TaskControl], bindings: dict[str, tuple[Binding, ...]],
reserved: set, path: Path,
) -> dict[str, str]:
"""`gamepad.task`: every task control on one pad control of its kind."""
pad_task = _task_block(gamepad, "gamepad", task, path)
commanded = {b.source for parts in bindings.values() for b in parts}
owner: dict[str, str] = {}
for name, control in pad_task.items():
kind = task[name].kind
if kind == "press" and control not in _DPAD:
raise ValueError(
f"{path}: gamepad.task.{name} is {control!r}; a `press` goes on a d-pad "
f"direction ({sorted(_DPAD)}) -- the buttons are the bundle's mode "
"switches and the operator's release"
)
if kind == "amount" and control not in _STICK_FIELD:
raise ValueError(
f"{path}: gamepad.task.{name} is {control!r}; an `amount` goes on an axis "
f"the robot's gamepad service publishes, a trigger or a stick ({list(_AXES)})"
)
if control in commanded:
raise ValueError(
f"{path}: {control} is both the task's ({name}) and a command's; one "
"deflection would do both"
)
if control in reserved:
raise ValueError(
f"{path}: {control} is the task's ({name}) and the release, the shift or "
"the reset button; one press would do both"
)
if control in owner:
raise ValueError(
f"{path}: {control} is on two task controls, {owner[control]} and {name}; "
"the task could not tell them apart"
)
owner[control] = name
return pad_task
def _task_block(device: dict, which: str, task: dict[str, TaskControl], path: Path) -> dict:
"""A device's `task:`, naming exactly the controls the top-level `task:` does."""
block = device.get("task")
if block is None:
block = {}
if not isinstance(block, dict):
raise ValueError(
f"{path}: {which}.task is a {type(block).__name__}; it maps each control the "
"top-level `task:` declares to where it is on this device"
)
stray = sorted(set(block) - set(task))
if stray:
raise ValueError(
f"{path}: {which}.task names {stray}, which the top-level `task:` does not "
f"declare ({sorted(task) or 'nothing'}). A task control is declared once, with "
"its kind and what it does, and bound on each device by that name."
)
missing = [n for n in task if n not in block]
if missing:
raise ValueError(
f"{path}: {which}.task leaves out {missing}. Each device has to be able to "
"reach every control the task keeps on its own."
)
return dict(block)
def _parse_keyboard(keyboard: dict, names: list[str], task: dict[str, TaskControl], path: Path):
"""The keyboard: keystrokes on axis directions, the release and the task's
controls. Every command axis appears, bound or said to be unbound."""
if not isinstance(keyboard, dict):
raise ValueError(f"{path}: devices.keyboard is a {type(keyboard).__name__}")
if "step" in keyboard:
raise ValueError(
f"{path}: the keyboard has a `step`, the notched keyboard retired on "
"2026-09-26 -- one press, a tenth of the travel, kept until pressed back. A "
"key now pushes its direction for as long as it is held and is back at rest "
"when it comes up: write `full_after_s`, the seconds held to full deflection."
)
if CENTRE in keyboard:
raise ValueError(
f"{path}: the keyboard has `{CENTRE}`, retired on 2026-09-27 with the notched "
"keyboard it served: a key let up is back at rest, and the release lets go "
"of everything. Delete it."
)
for field in ("scheme", "full_after_s", "axes", "release"):
_require(keyboard, field, path)
if keyboard["scheme"] != _KEYBOARD_SCHEME:
raise ValueError(
f"{path}: keyboard scheme is {keyboard['scheme']!r}. This file implements "
f"`{_KEYBOARD_SCHEME}`: each keystroke bound to one direction of an axis, "
"the release, or a task control."
)
full = keyboard["full_after_s"]
if not _positive_seconds(full):
raise ValueError(
f"{path}: keyboard full_after_s is {full!r}; it is the seconds a key is held "
"for its direction to reach full deflection, a positive number"
)
# One keystroke, one thing: an axis direction or the release. The task's
# auxiliary controls may share one with each other. Two tools
# both close on Space, and a mode reads only its own side's -- but not with
# an axis or the release, or one key would do both.
owner: dict[str, str] = {}
parsed: dict[str, tuple[str | None, str]] = {}
def claim(stroke: Any, what: str, where: str, exclusive: bool = True) -> str:
parsed[stroke] = _check_stroke(stroke, where, path)
if stroke in owner and (exclusive or not owner[stroke].startswith("task ")):
raise ValueError(
f"{path}: {stroke} is on {owner[stroke]} and on {what}. One keystroke "
"would do both; give each its own."
)
if exclusive:
owner[stroke] = what
else:
owner.setdefault(stroke, what)
return stroke
def strokes(listed: Any, where: str) -> list:
if not isinstance(listed, list) or not listed:
raise ValueError(
f"{path}: {where} is {listed!r}; it is a non-empty list of keystrokes"
)
return listed
declared = keyboard["axes"]
if not isinstance(declared, dict):
raise ValueError(
f"{path}: keyboard.axes is a {type(declared).__name__}; it maps every command "
"axis to its keystrokes"
)
stray = sorted(set(declared) - set(names))
if stray:
raise ValueError(
f"{path}: keyboard.axes names {stray}, which no command has. A typo here is "
f"a key that drives nothing. The axes are {names}"
)
key_axes: dict[str, KeyAxis] = {}
for name in names:
if name not in declared:
raise ValueError(
f"{path}: keyboard.axes leaves out {name}. Every command axis is on the "
"keyboard or says why not (`unbound: <why>`): an axis quietly missing is "
"a keyboard that cannot do something the pad can, and nothing says so."
)
entry = declared[name]
if isinstance(entry, dict) and set(entry) == {"unbound"}:
why = entry["unbound"]
if not isinstance(why, str) or not why.strip():
raise ValueError(
f"{path}: keyboard.axes.{name} is unbound and does not say why"
)
key_axes[name] = KeyAxis(unbound=why)
continue
if not isinstance(entry, dict) or set(entry) != {"+", "-"}:
raise ValueError(
f"{path}: keyboard.axes.{name} is {entry!r}; it is {{\"+\": [...], \"-\": "
"[...]}, the keystrokes towards the axis's positive direction and away "
"from it, or {unbound: <why this axis has no keys>}"
)
plus = [claim(s, f"{name} +", f"keyboard.axes.{name}.+")
for s in strokes(entry["+"], f"keyboard.axes.{name}.+")]
minus = [claim(s, f"{name} -", f"keyboard.axes.{name}.-")
for s in strokes(entry["-"], f"keyboard.axes.{name}.-")]
key_axes[name] = KeyAxis(plus=tuple(plus), minus=tuple(minus))
release = tuple(claim(s, "the release", "keyboard.release")
for s in strokes(keyboard["release"], "keyboard.release"))
key_task = {}
for name, listed in _task_block(keyboard, "keyboard", task, path).items():
where = f"keyboard.task.{name}"
seen = [claim(s, f"task {name}", where, exclusive=False) for s in strokes(listed, where)]
if len(set(seen)) != len(seen):
raise ValueError(f"{path}: {where} lists a keystroke twice: {seen}")
key_task[name] = tuple(seen)
# A modifier held on its own that also modifies a key: holding it for the
# one would do the other while the hand reaches for the key. A key that is
# one of the modifier's own keys (`key_left_shift`) held alone is the same.
modifiers = _vocabulary.modifiers()
chords = {m for m, _k in parsed.values() if m is not None}
alone = {k for m, k in parsed.values() if m is None and k in modifiers}
alone |= {m for m, keys in modifiers.items()
for _m, k in parsed.values() if _m is None and k in keys}
both = sorted(chords & alone)
if both:
raise ValueError(
f"{path}: {', '.join(both)} is bound on its own and also modifies a key "
f"({', '.join(s for s, (m, _k) in parsed.items() if m in both)}). Holding it "
"for the one would do it while reaching for the other."
)
return full, key_axes, release, key_task
def controller_contract(controls: Controls, env_cfg: Any) -> dict[str, Any]:
"""The specification, shaped for an exported policy's contract.
A transcription of the file rather than a description of the code: what a
consumer receives is what `play` read. `source_sha256` is carried so it can
say which revision, and a runner that wants to diff two bundles has a key to
do it with. `deploy/fsm/src/operator.rs` reads it, as
`operator_controller/2`; the name is the schema's, and the version is the
file's, so a reader built for anything else refuses it by name.
`task` is the file's top-level `task:`, kind and words, and `{}` for a task
that keeps none: the controller refuses a hook reading a name it does not
declare. Each device's own `task:` stays in its block, verbatim.
Args:
env_cfg: the **environment** config, whose command terms hold the values
the file names by field -- `rest: neutral_height` becomes the number.
A consumer on the robot has no config to look a field up in, so the
block carries the number, and the name beside it for a reader who
wants to know where it came from. Required: a rest left as a name is
a height the robot would read as zero, which is a body on the floor.
Called at export rather than when the config is built, because a task adds
its other command terms after `velocity_env_cfg` returns -- `velocity_env_cfg`
attaches this as a callable, the way a reference contract is attached.
"""
commands = []
for c in controls.commands:
term_cfg = (getattr(env_cfg, "commands", None) or {}).get(c.term)
axes = []
for i, a in enumerate(c.axes):
entry = {"name": a.name, "index": i, "unit": a.unit, "positive": a.positive,
"range_from": f"command_ranges.{c.term}.{a.name}"}
if a.rest:
if term_cfg is None or not hasattr(term_cfg, a.rest):
raise ValueError(
f"{controls.source}: {c.term}.{a.name} rests at `{a.rest}`, and the "
f"environment config has no command term {c.term!r} carrying it. "
"The block needs the number: a robot has no config to look it up in."
)
entry["rest"] = float(getattr(term_cfg, a.rest))
entry["rest_from"] = a.rest
if a.integrate_s is not None:
entry["integrate_s"] = float(a.integrate_s)
axes.append(entry)
entry = {"term": c.term, "feeds": c.feeds, "frame": c.frame,
"axes": axes, "scaling": dict(c.scaling)}
entry.update(_command_shape(controls, c, term_cfg))
commands.append(entry)
return {
"schema": f"operator_controller/{SCHEMA}",
"source": {
"file": controls.source.name,
"sha256": controls.source_sha256,
"note": "Parsed from the specification play reads. Comments and provenance "
"live in the source file.",
},
"command": commands,
"task": {t.name: {"kind": t.kind, "does": t.does} for t in controls.task.values()},
"devices": {
"gamepad": dict(controls.gamepad),
"keyboard": dict(controls.keyboard),
},
}
def _command_shape(controls: Controls, c: Command, term_cfg: Any) -> dict[str, Any]:
"""What happens to a command between the stick and the policy, if anything.
Two things a command term can carry, and a consumer on the robot has no
config to find either in, so the block carries the numbers:
* **`bands`** -- a narrower band some axes are held to while the velocity
command walks. `command_ranges.<term>` is the standing band, the widest the
command goes and what a full stick reaches; at or above `standing_below`,
the norm of the velocity command's `[vx, vy, wz]`, each axis named in
`moving` is clamped to it. Read off the term's `moving` and
`stand_threshold`: a half-range or a `(low, high)` pair per axis.
* **`max_rate`** -- how fast the command may move, in its own units per
second, because the policy was trained on a command that ramps. Absent is
a command that steps, which is what every other term does.
A term that carries neither gets neither, and the block is what it was. A
`moving` without a threshold, or naming an axis the file does not, raises:
a band nobody can apply is a clamp the robot silently skips.
"""
out: dict[str, Any] = {}
moving = getattr(term_cfg, "moving", None)
if moving is not None:
threshold = getattr(term_cfg, "stand_threshold", None)
velocity = getattr(term_cfg, "velocity_command", None) or getattr(
term_cfg, "velocity_command_name", None
)
if threshold is None or velocity is None:
raise ValueError(
f"{controls.source}: {c.term} has a moving band and no "
"`stand_threshold` / velocity command to decide when it applies"
)
names = {a.name for a in c.axes}
band = {}
for name, value in vars(moving).items():
if name.startswith("_"):
continue
if name not in names:
raise ValueError(
f"{controls.source}: {c.term}'s moving band names {name!r}, "
f"which is not one of the file's axes {sorted(names)}"
)
if isinstance(value, (int, float)) and not isinstance(value, bool):
band[name] = [-float(value), float(value)]
else:
low, high = value
band[name] = [float(low), float(high)]
out["bands"] = {"standing_below": float(threshold), "velocity_term": str(velocity),
"moving": band}
max_rate = getattr(term_cfg, "max_rate", None)
if max_rate is not None:
out["max_rate"] = float(max_rate)
return out
def key_name(code: int) -> str:
"""A GLFW keycode as something a person can read off a banner.
A dictionary key is shown by its own name less the `key_` prefix -- `W`,
`KP8` for `keypad_8` -- and anything else falls back to the number, which
is what a handler would need to remap it.
"""
for name, codes in _vocabulary.key_codes().items():
if codes["glfw"] == code:
if name.startswith("keypad_"):
return "KP" + name[len("keypad_"):]
return name.removeprefix("key_").upper()
return str(code)
def stroke_name(stroke: str) -> str:
"""A keystroke as a banner shows it: `Shift+J`, `Ctrl`, `SPACE`, `KP1`."""
modifier, key = _vocabulary.keystroke(stroke)
shown = (key.capitalize() if key in _vocabulary.modifiers()
else key_name(_vocabulary.key_codes()[key]["glfw"]))
return f"{modifier.capitalize()}+{shown}" if modifier else shown
+292
View File
@@ -0,0 +1,292 @@
"""Finding and reading a Linux evdev gamepad, with no dependencies.
## Why evdev and not joydev
`/dev/input/js*` is simpler to parse, and on this machine it does not exist: the
`joydev` module is not loaded, while `event*` is always there. evdev is also the
one that carries the axis ranges, so a trigger that reports 0..1023 on one pad and
0..255 on another normalises correctly without a table of models.
## The wire format
An evdev event is `struct input_event`: a `timeval` then type, code and value.
On 64-bit Linux that is `llHHi`, 24 bytes, which is asserted at import rather
than assumed -- a wrong size here would not raise, it would silently decode
garbage as button presses.
Ranges come from the `EVIOCGABS` ioctl, one per axis, read once at open. The ioctl
numbers are built with the same arithmetic as the kernel's `_IOR` macro rather
than pasted as magic constants, so the derivation is visible.
"""
from __future__ import annotations
import array
import glob
import os
import select
import struct
import sys
import threading
from dataclasses import dataclass, field
try:
import fcntl
except ImportError: # Windows. There is no evdev to read there and no ioctl
# to read it with; pads there go through `xinput.py`, and nothing in this
# module is called. The import still happens, so it must not take the whole
# package down with it: the tasks import `controller` for its vocabulary, and
# that is on the path of every training run.
fcntl = None
from .xbox import ABS_AXES, GamepadState, normalise
__all__ = ["Gamepad", "GamepadState", "PadInfo", "describe", "find_pads"]
#: `struct input_event` on 64-bit Linux: struct timeval (two longs), then
#: __u16 type, __u16 code, __s32 value.
_EVENT_FORMAT = "llHHi"
_EVENT_SIZE = struct.calcsize(_EVENT_FORMAT)
# Asserted rather than assumed, because a wrong size here would not raise -- it
# would decode the stream misaligned and report button presses that never
# happened. **Only on Linux, which is what the 24 bytes describe**: Windows'
# C `long` is 4 bytes, so the same format measures 16 there and would fail an
# assertion about a layout that platform never uses.
if sys.platform == "linux":
assert _EVENT_SIZE == 24, (
f"struct input_event is {_EVENT_SIZE} bytes here, not 24. Decoding would "
f"silently produce nonsense rather than fail, so this is checked."
)
EV_KEY = 0x01
EV_ABS = 0x03
def _ioc(direction: int, type_: int, nr: int, size: int) -> int:
"""The kernel's `_IOC` macro. dir 2 is `_IOC_READ`."""
return (direction << 30) | (size << 16) | (type_ << 8) | nr
def _eviocgname(length: int) -> int:
return _ioc(2, ord("E"), 0x06, length)
def _eviocgabs(axis: int) -> int:
#: `struct input_absinfo` is six __s32: value, min, max, fuzz, flat, resolution.
return _ioc(2, ord("E"), 0x40 + axis, 24)
@dataclass
class PadInfo:
"""A candidate device and what was decided about it."""
path: str
name: str
readable: bool
reason: str = ""
def _read_name(fd: int) -> str:
buf = array.array("B", b"\x00" * 256)
try:
fcntl.ioctl(fd, _eviocgname(256), buf)
except OSError:
return ""
return bytes(buf).split(b"\x00", 1)[0].decode("utf-8", "replace")
def _read_ranges(fd: int) -> dict[int, tuple[int, int, int]]:
"""Per-axis (min, max, flat) from EVIOCGABS. `flat` is the driver's own idea
of the dead zone, which is better than a number invented here."""
ranges: dict[int, tuple[int, int, int]] = {}
for axis in ABS_AXES:
buf = array.array("i", [0] * 6)
try:
fcntl.ioctl(fd, _eviocgabs(axis), buf)
except OSError:
continue # the pad does not have this axis
_value, lo, hi, _fuzz, flat, _res = buf
if hi > lo:
ranges[axis] = (lo, hi, flat)
return ranges
#: Buttons whose presence marks a device as something to be played with, from
#: udev's own `input_id` rule.
BTN_JOYSTICK = 0x120
BTN_GAMEPAD = 0x130
def _bitmap(path: str) -> set[int]:
"""Parse a sysfs capability bitmap into the set of bits it has.
The kernel prints these as space-separated 64-bit hex words, **most
significant first**, so the last word is bits 0-63. Reading them left to
right, which is the obvious way, gets every bit index wrong.
"""
try:
with open(path) as f:
words = f.read().split()
except OSError:
return set()
bits: set[int] = set()
for i, word in enumerate(reversed(words)):
value = int(word, 16)
base = i * 64
while value:
low = value & -value
bits.add(base + low.bit_length() - 1)
value ^= low
return bits
def find_pads(readable_only: bool = False) -> list[PadInfo]:
"""Every gamepad-looking device, in a stable order.
Discovery is through `/sys/class/input/event*`, applying udev's own rule --
the device has absolute axes and at least one gamepad or joystick button.
**Not through `/dev/input/by-id/*-event-joystick`**, which is what this did
first and which silently finds nothing over Bluetooth: those symlinks are
generated from USB and serial identifiers, and a Bluetooth pad has neither.
Measured here, an Xbox Wireless Controller paired over Bluetooth appears as
`/dev/input/event17` with no `by-id` entry at all, so a working, connected
controller read as absent.
sysfs also answers without opening anything, so a pad the process cannot read
is still *found*, and can be reported as a permissions problem rather than as
nothing at all.
"""
pads: list[PadInfo] = []
for sysdir in sorted(
glob.glob("/sys/class/input/event*"),
key=lambda p: int(os.path.basename(p)[5:] or 0),
):
device = os.path.join(sysdir, "device")
caps = os.path.join(device, "capabilities")
ev = _bitmap(os.path.join(caps, "ev"))
keys = _bitmap(os.path.join(caps, "key"))
if EV_ABS not in ev:
continue
if not (BTN_GAMEPAD in keys or BTN_JOYSTICK in keys):
continue
path = os.path.join("/dev/input", os.path.basename(sysdir))
try:
with open(os.path.join(device, "name")) as f:
name = f.read().strip()
except OSError:
name = os.path.basename(path)
try:
fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
except PermissionError:
pads.append(
PadInfo(path, name, False,
"permission denied -- add yourself to the 'input' group "
"(sudo usermod -aG input \"$USER\", then log out and back "
"in), or log in on the seat so logind grants the ACL")
)
continue
except OSError as exc:
pads.append(PadInfo(path, name, False, str(exc)))
continue
try:
pads.append(PadInfo(path, _read_name(fd) or name, True))
finally:
os.close(fd)
if readable_only:
return [p for p in pads if p.readable]
return pads
def describe() -> str:
"""What was found and, for anything rejected, why. `controller.describe()` is
the entry point; this is its Linux half."""
pads = find_pads()
if not pads:
return (
"no input device has both absolute axes and a gamepad button. "
"Nothing is paired, or it connected as something other than a pad."
)
lines = []
for p in pads:
state = "usable" if p.readable else f"unusable: {p.reason}"
lines.append(f" {p.path} {p.name} -- {state}")
return "\n".join(lines)
@dataclass
class Gamepad:
"""A pad being read on a background thread.
The reader thread only ever writes raw integers into `_raw`; normalisation
happens in `state()`, on the caller's thread. That keeps the axis ranges and
the dead zone out of the hot path and, more usefully, means a caller polling
at 50 Hz sees a consistent snapshot rather than a half-updated one.
"""
info: PadInfo
_raw: dict[tuple[int, int], int] = field(default_factory=dict)
_ranges: dict[int, tuple[int, int, int]] = field(default_factory=dict)
_fd: int = -1
_thread: threading.Thread | None = None
_stop: threading.Event = field(default_factory=threading.Event)
@property
def name(self) -> str:
return self.info.name
def start(self) -> None:
self._fd = os.open(self.info.path, os.O_RDONLY | os.O_NONBLOCK)
self._ranges = _read_ranges(self._fd)
self._thread = threading.Thread(
target=self._run, name="gamepad", daemon=True
)
self._thread.start()
def close(self) -> None:
self._stop.set()
if self._thread is not None:
self._thread.join(timeout=1.0)
if self._fd >= 0:
os.close(self._fd)
self._fd = -1
def _run(self) -> None:
"""Poll rather than block on read, so `close()` is not left waiting for a
stick that nobody is touching."""
while not self._stop.is_set():
ready, _, _ = select.select([self._fd], [], [], 0.1)
if not ready:
continue
try:
data = os.read(self._fd, _EVENT_SIZE * 64)
except OSError:
# The pad went away mid-read: unplugged, or Bluetooth dropped.
# Leave the last state in place and stop; state() keeps working
# and `connected` goes False.
break
for i in range(0, len(data) - _EVENT_SIZE + 1, _EVENT_SIZE):
_s, _us, etype, code, value = struct.unpack_from(
_EVENT_FORMAT, data, i
)
if etype in (EV_KEY, EV_ABS):
self._raw[(etype, code)] = value
self._stop.set()
@property
def connected(self) -> bool:
return self._thread is not None and self._thread.is_alive()
def state(self) -> GamepadState:
"""A normalised snapshot. Never blocks and never raises."""
return normalise(dict(self._raw), self._ranges)
def raw(self) -> dict[tuple[int, int], int]:
"""Everything seen so far, unnormalised, as `{(type, code): value}`.
For working out a pad whose mapping does not match `xbox.py` -- press the
button, see which code moved.
"""
return dict(self._raw)
+192
View File
@@ -0,0 +1,192 @@
"""Map pad deflections and keyboard edges into policy command inputs."""
from __future__ import annotations
from collections.abc import Iterable, Mapping
from typing import Any
from . import vocabulary as _vocabulary
from .controls import Command, Controls, KeyAxis
__all__ = ["Keyboard", "deflections", "scale", "spans"]
# ── The pad's mapping ───────────────────────────────────────────────────────
def _band(s: float, travel: tuple[float, ...]) -> float:
"""A signed deflection through one binding's stretch of travel.
Zero until `from`, full at `to` and held there beyond it. So `(0.0, 0.5)`
is full at half travel, and `(0.5, 1.0)` is silent until half travel. With
a fall, `(0.0, 0.5, 0.5, 0.75)`, it comes back from full at `back` to zero
at `off` and stays there.
"""
lo, hi, *fall = travel
a = abs(s)
x = min(max((a - lo) / (hi - lo), 0.0), 1.0)
if fall:
back, off = fall
x *= min(max((off - a) / (off - back), 0.0), 1.0)
return x if s >= 0.0 else -x
def deflections(
values: Mapping[str, float], shifted: bool, controls: Controls
) -> dict[str, float]:
"""Pad control positions -> each command axis's deflection, -1..1.
`values` is keyed by the dictionary's names (`Lx` ... `RT`). A source that
has a shifted binding drives **only** that binding while shifted; every
other source carries on as it was. The pad's path only: the keyboard has
bindings of its own (`Keyboard`).
"""
layered = {b.source for parts in controls.bindings.values() for b in parts if b.shifted}
out = {}
for axis, parts in controls.bindings.items():
x = 0.0
for b in parts:
live = shifted if b.shifted else not (shifted and b.source in layered)
if live:
x += _band(b.sign * values[b.source], b.travel)
out[axis] = max(-1.0, min(1.0, x))
return out
def scale(x: float, span: tuple[float, float, float]) -> float:
"""`full_deflection_is_range_edge`, measured from the axis's rest.
The two ends are scaled separately. A range like (-0.5, 1.0) gives forward
twice what it gives back, which is what the policy trained on; taking the
larger end for both quietly commands something it never saw.
"""
lo, hi, rest = span
return rest + x * ((hi - rest) if x >= 0.0 else (rest - lo))
def spans(command: Command, cfg: Any) -> dict[str, tuple[float, float, float]]:
"""`(low, high, rest)` per axis, from a live command config.
A range is either a `(low, high)` pair, as the velocity command has, or a
half-range, as the posture command's angles are. The rest is the config
field the file names, or zero.
"""
out = {}
for a in command.axes:
r = getattr(cfg.ranges, a.name)
lo, hi = (-float(r), float(r)) if isinstance(r, (int, float)) else map(float, r)
rest = float(getattr(cfg, a.rest)) if a.rest else 0.0
if not lo <= rest <= hi:
raise ValueError(
f"{command.term}.{a.name} rests at {rest:g}, outside its range "
f"({lo:g}, {hi:g}). A centred stick would command a value the range "
"clamps away -- name the rest in the controls file (`rest: <field>`)."
)
out[a.name] = (lo, hi, rest)
return out
# ── The keyboard's path ─────────────────────────────────────────────────────
class Keyboard:
"""The keyboard's half of the operator: which keystrokes are live, since when.
Fed a key's two edges by the dictionary's name, and asked what an axis or a
task control reads. A keystroke held for `full_after_s` is full deflection
and half that is half; the moment it stops being live it reads nothing.
Several live on one direction are summed, as two keys on one direction of a
stick would push it together, and the two directions of an axis cancel.
"""
def __init__(self, controls: Controls):
self._full = float(controls.full_after_s)
modifiers = _vocabulary.modifiers()
#: Keystroke -> (the modifier held with it or `None`, the key or the
#: modifier it is), every keystroke the file's keyboard block binds.
self._strokes = {s: _vocabulary.keystroke(s) for s in controls.keystrokes}
#: Modifier -> its two keys, for the modifiers this block names at all.
self._modifiers = {
m: keys for m, keys in modifiers.items()
if any(m in (mod, key) for mod, key in self._strokes.values())
}
#: Key -> the modifiers bound in front of it here. While one of them is
#: down, the key alone is dead: Shift + J is not J as well.
self._chorded: dict[str, frozenset[str]] = {}
for mod, key in self._strokes.values():
if mod is not None:
self._chorded[key] = self._chorded.get(key, frozenset()) | {mod}
#: Every dictionary key this block reads: each keystroke's key, and both
#: keys of each modifier it names. Any other key is not this file's.
self.watched = frozenset(
{key for _mod, key in self._strokes.values() if key not in modifiers}
| {k for pair in self._modifiers.values() for k in pair}
)
#: Keys down, as their edges report them.
self._down: set[str] = set()
#: Keys down and pressed since the last `drop`: what drives.
self._fresh: set[str] = set()
#: Live keystroke -> when it last became live, on the operator's clock.
self._live: dict[str, float] = {}
def press(self, key: str, now: float) -> set[str] | None:
"""`key` went down at `now`. The keystrokes that became live, or `None`
for a key already down -- a repeat, which is the same hold."""
if key in self._down:
return None
self._down.add(key)
self._fresh.add(key)
return self._refresh(now)
def lift(self, key: str, now: float) -> set[str]:
"""`key` came up at `now`. The keystrokes that became live: a key whose
modifier was let go of is itself again."""
self._down.discard(key)
self._fresh.discard(key)
return self._refresh(now)
def drop(self) -> None:
"""Hands off: nothing is live, and a key still down drives nothing --
as a key or as a modifier -- until it is pressed again."""
self._fresh.clear()
self._live.clear()
def _refresh(self, now: float) -> set[str]:
held = {m for m, pair in self._modifiers.items() if self._fresh.intersection(pair)}
live = set()
for stroke, (mod, key) in self._strokes.items():
if mod is not None:
on = key in self._fresh and mod in held
elif key in self._modifiers:
on = key in held
else:
on = key in self._fresh and not (self._chorded.get(key, frozenset()) & held)
if on:
live.add(stroke)
became = live - self._live.keys()
self._live = {s: self._live.get(s, now) for s in live}
return became
@property
def live(self) -> frozenset[str]:
return frozenset(self._live)
def ramp(self, stroke: str, now: float) -> float:
"""0..1: how long `stroke` has been live, over `full_after_s`."""
since = self._live.get(stroke)
if since is None:
return 0.0
return min(1.0, max(0.0, (now - since) / self._full))
def deflection(self, axis: KeyAxis, now: float) -> float:
"""One command axis, -1..1: its `+` keystrokes less its `-` ones."""
x = sum(self.ramp(s, now) for s in axis.plus) - sum(self.ramp(s, now) for s in axis.minus)
return max(-1.0, min(1.0, x))
def amount(self, strokes: Iterable[str], now: float) -> float:
"""A task's `amount`: the furthest of its keystrokes' ramps."""
return max((self.ramp(s, now) for s in strokes), default=0.0)
def pressed(self, strokes: Iterable[str]) -> bool:
"""A task's `press`: any of its keystrokes live, at once and in full."""
return any(s in self._live for s in strokes)
+499
View File
@@ -0,0 +1,499 @@
{
"schema": "kk-control-vocabulary/2",
"_what_this_is": [
"Everything a pad or a keyboard can say, and nothing about what any of it",
"means.",
"",
"That split is this package's own rule -- it reports that the left stick is at",
"(0.4, -0.9), that A is down and that keypad_8 is down, and deciding that any",
"of that is 0.2 m/s forward belongs to whoever is driving. So the words live",
"here and the meanings live with the consumer: `[[fsm.button]]` in a deploy",
"manifest, `controls.yaml` in a task.",
"",
"Read by three programs and written by none of them:",
" controller/vocabulary.py python -m controller --vocabulary",
" deploy/fsm/src/vocabulary.rs include_str!, so a board carries no file",
" tasks/.../mdp/controls.py validating a task's controls.yaml",
"",
"It had a copy in each of those for a day. Adding a word meant remembering two",
"other files, and forgetting one leaves a name that reads as supported and is",
"refused on build -- or, worse, the other way round.",
"",
"Was `kk-gamepad-vocabulary/1`, when the pad was the only thing it described.",
"The keys below were a closed list too and were not written down anywhere a",
"consumer could check, so `[[fsm.button]] key = ...` took any string at all: a",
"typo loaded, bound nothing, and looked exactly like a rule that had not fired."
],
"_source": {
"service": "control-pod-svc",
"repo": "robot/embedded-software/soc/apps/control-pod-svc",
"read_from": [
"include/gamepad/gamepad_state.hpp",
"src/gamepad/gamepad_dds_message.cpp"
],
"topic": "RobotControlRaw_ControlRaw",
"note": "It normalises three different pads onto these names, so they mean the same thing on a bench and on the robot. Its own docs put the count at 18: 10 buttons + 2 dpad axes + 6 axes."
},
"buttons": [
{
"name": "A"
},
{
"name": "B"
},
{
"name": "X",
"unresolved": "controller/xbox.py reads evdev 307 as `y` and 308 as `x`; the service's table reads 307 as X and 308 as Y. One is wrong about which physical button it is and pressing it is the only way to find out which."
},
{
"name": "Y",
"unresolved": "see X"
},
{
"name": "LB"
},
{
"name": "RB"
},
{
"name": "menu"
},
{
"name": "home",
"note": "`xbox` in controller/xbox.py. Same button, two names."
},
{
"name": "L3"
},
{
"name": "R3"
}
],
"dpad": [
{
"name": "dpad_up"
},
{
"name": "dpad_down"
},
{
"name": "dpad_left"
},
{
"name": "dpad_right"
}
],
"_dpad_note": "The wire carries dpad_x / dpad_y as -1 / 0 / +1, which no gesture can read, so they are offered as four booleans. The service negates dpad_y so +1 is up; it does not negate Ly, two lines away in the same function.",
"axes": [
{
"name": "Lx",
"range": [
-1,
1
]
},
{
"name": "Ly",
"range": [
-1,
1
],
"note": "Not negated by the service. Forward is the negative end, by evdev's convention -- stated in controller/xbox.py and never measured against this service."
},
{
"name": "Rx",
"range": [
-1,
1
]
},
{
"name": "Ry",
"range": [
-1,
1
]
},
{
"name": "LT",
"range": [
0,
1
],
"note": "Unipolar: a trigger never reports negative."
},
{
"name": "RT",
"range": [
0,
1
]
}
],
"_axes_note": "Sticks carry a 0.15 deadband that is already applied **and rescaled**, so full deflection still reaches ±1. A consumer that applies its own on top narrows what a person can ask for and cannot see that it has.",
"keys": [
{
"name": "keypad_0",
"glfw": 320,
"browser": "Numpad0"
},
{
"name": "keypad_1",
"glfw": 321,
"browser": "Numpad1"
},
{
"name": "keypad_2",
"glfw": 322,
"browser": "Numpad2"
},
{
"name": "keypad_3",
"glfw": 323,
"browser": "Numpad3"
},
{
"name": "keypad_4",
"glfw": 324,
"browser": "Numpad4"
},
{
"name": "keypad_5",
"glfw": 325,
"browser": "Numpad5"
},
{
"name": "keypad_6",
"glfw": 326,
"browser": "Numpad6"
},
{
"name": "keypad_7",
"glfw": 327,
"browser": "Numpad7"
},
{
"name": "keypad_8",
"glfw": 328,
"browser": "Numpad8"
},
{
"name": "keypad_9",
"glfw": 329,
"browser": "Numpad9"
},
{
"name": "keypad_.",
"glfw": 330,
"browser": "NumpadDecimal"
},
{
"name": "keypad_/",
"glfw": 331,
"browser": "NumpadDivide"
},
{
"name": "keypad_*",
"glfw": 332,
"browser": "NumpadMultiply"
},
{
"name": "keypad_-",
"glfw": 333,
"browser": "NumpadSubtract"
},
{
"name": "keypad_+",
"glfw": 334,
"browser": "NumpadAdd"
},
{
"name": "keypad_enter",
"glfw": 335,
"browser": "NumpadEnter"
},
{
"name": "key_w",
"glfw": 87,
"browser": "KeyW"
},
{
"name": "key_s",
"glfw": 83,
"browser": "KeyS"
},
{
"name": "key_a",
"glfw": 65,
"browser": "KeyA"
},
{
"name": "key_d",
"glfw": 68,
"browser": "KeyD"
},
{
"name": "key_j",
"glfw": 74,
"browser": "KeyJ"
},
{
"name": "key_l",
"glfw": 76,
"browser": "KeyL"
},
{
"name": "key_i",
"glfw": 73,
"browser": "KeyI"
},
{
"name": "key_k",
"glfw": 75,
"browser": "KeyK"
},
{
"name": "key_m",
"glfw": 77,
"browser": "KeyM"
},
{
"name": "key_o",
"glfw": 79,
"browser": "KeyO"
},
{
"name": "key_q",
"glfw": 81,
"browser": "KeyQ"
},
{
"name": "key_h",
"glfw": 72,
"browser": "KeyH"
},
{
"name": "key_b",
"glfw": 66,
"browser": "KeyB"
},
{
"name": "key_g",
"glfw": 71,
"browser": "KeyG"
},
{
"name": "key_space",
"glfw": 32,
"browser": "Space"
},
{
"name": "key_right",
"glfw": 262,
"browser": "ArrowRight"
},
{
"name": "key_left",
"glfw": 263,
"browser": "ArrowLeft"
},
{
"name": "key_down",
"glfw": 264,
"browser": "ArrowDown"
},
{
"name": "key_up",
"glfw": 265,
"browser": "ArrowUp"
},
{
"name": "key_left_ctrl",
"glfw": 341,
"browser": "ControlLeft"
},
{
"name": "key_right_ctrl",
"glfw": 345,
"browser": "ControlRight"
},
{
"name": "key_1",
"glfw": 49,
"browser": "Digit1"
},
{
"name": "key_2",
"glfw": 50,
"browser": "Digit2"
},
{
"name": "key_3",
"glfw": 51,
"browser": "Digit3"
},
{
"name": "key_4",
"glfw": 52,
"browser": "Digit4"
},
{
"name": "key_n",
"glfw": 78,
"browser": "KeyN"
},
{
"name": "key_u",
"glfw": 85,
"browser": "KeyU"
},
{
"name": "key_v",
"glfw": 86,
"browser": "KeyV"
},
{
"name": "key_semicolon",
"glfw": 59,
"browser": "Semicolon"
},
{
"name": "key_escape",
"glfw": 256,
"browser": "Escape"
},
{
"name": "key_left_shift",
"glfw": 340,
"browser": "ShiftLeft"
},
{
"name": "key_right_shift",
"glfw": 344,
"browser": "ShiftRight"
},
{
"name": "key_left_alt",
"glfw": 342,
"browser": "AltLeft"
},
{
"name": "key_right_alt",
"glfw": 346,
"browser": "AltRight"
}
],
"modifiers": [
{
"name": "ctrl",
"keys": ["key_left_ctrl", "key_right_ctrl"]
},
{
"name": "shift",
"keys": ["key_left_shift", "key_right_shift"]
},
{
"name": "alt",
"keys": ["key_left_alt", "key_right_alt"]
}
],
"_modifiers_note": [
"A modifier is either of its two keys: `ctrl` is down while the left Ctrl or the",
"right one is. A keyboard binding may name one on its own -- `ctrl`, a key like",
"any other -- or in front of a key, `shift+key_j`, which is J while Shift is",
"held. A key bound both ways answers the modified binding while its modifier is",
"down and the plain one otherwise, so J and Shift + J are two things and never",
"both. A mode switch names one as `with`, as a pad switch names `menu`.",
"",
"Alt costs a browser something: pressed and let go on its own, Windows moves",
"focus to the browser's menu, and the keys after it go there rather than to the",
"page, which has to `preventDefault` it. MuJoCo's viewer uses all three with the",
"mouse and does nothing with any of them alone."
],
"_keys_note": [
"The numeric keypad, and letters as a task asks for them -- each one knowing",
"what it costs.",
"",
"The keypad is the set MuJoCo's viewer does not bind. Everything else on the",
"main keyboard is bound: `Simulate` handles its shortcuts in C++ and calls the",
"user callback **as well as**, never instead, so a letter fires both actions",
"and there is no way to intercept the second.",
"",
"MEASURED against this repository's mujoco, by reading `mjVISSTRING` and",
"`mjRNDSTRING` -- **all twenty-six letters are taken**, one render flag each:",
"",
" A auto-connect B perturb force C contact point D static body",
" E equality F contact force G fog H convex hull",
" I inertia J joint K skybox L additive",
" M centre of mass N island O perturb object P contact split",
" Q camera R reflection S shadow T transparent",
" U actuator V tendon W wireframe X texture",
" Y range finder Z light and ' , / ; \\\\ ` as well",
"",
"The generated locomotion bindings use W A S D and the arrows for planar",
"motion, J L to turn, and Esc to release. These keys still reach the viewer.",
"For example, W also toggles wireframe and the left and right arrows step a",
"paused simulation.",
"",
"Main-row digits 1 to 4 toggle the corresponding geom groups. Keypad digits",
"leave the viewer flags unchanged and can be bound independently. Space",
"pauses the viewer. Legacy FSM mode keys are specified by the bundle.",
"",
"**MuJoCo's viewer reports a key's press and nothing else.** The callback is",
"handed a keycode and no action (`KeyCallbackType = Callable[[int], None]`), on",
"a press only: MuJoCo's GLFW adapter drops the auto-repeat and the release.",
"`mjrl.viewer.keys` puts a GLFW key callback of its own in front of the",
"viewer's and hands every reader both edges, so a key is held from its press to",
"its release, as in a browser, which reports both itself. Until 2026-09-29 the",
"release was inferred from repeats that never came, and every key let go 0.75 s",
"after its press.",
"",
"Browser codes carry over unchanged -- `KeyW` is `KeyW` whatever the viewer",
"does -- so a web consumer pays nothing for this at all."
],
"gestures": [
{
"name": "rise",
"fires": "the tick it goes down, one-shot"
},
{
"name": "fall",
"fires": "the tick it comes back up, one-shot"
},
{
"name": "hold",
"fires": "every tick it is down -- a dead-man switch, not a mode"
},
{
"name": "toggle",
"fires": "down until pressed again -- the default for entering a mode"
},
{
"name": "single",
"fires": "one press and no second inside the click window; switches like `toggle`"
},
{
"name": "double",
"fires": "two presses, each inside the click window of the last; switches like `toggle`"
},
{
"name": "triple",
"fires": "three presses, each inside the click window of the last; switches like `toggle`"
},
{
"name": "quadruple",
"fires": "four presses, each inside the click window of the last; switches like `toggle`"
},
{
"name": "quintuple",
"fires": "five presses, each inside the click window of the last; switches like `toggle`"
}
],
"_gestures_note": "A control may be bound more than once with different gestures: press to arm and release to go is one button, `rise` and `fall`, named twice. The five click gestures count presses of one control -- the same pad button or key, with the same modifier -- and a press counts when it comes within `click_window_ms` (`[fsm]` in the controller's config) of the one before. A count fires the binding for it the moment no longer count on that control is bound, and otherwise once the window has run out with no further press; so `single` fires at once where it is the only click on its control and waits out the window where a `double` shares it. A click switches a mode the way `toggle` does, and an `event` fires once. A control with a click gesture takes no `rise`, `fall`, `hold` or `toggle` -- every click would be one of those first. This note used to say there would be no double click, because counting needs a clock and a clock depends on how often a host ticks. The window is measured on the timestamps every host already hands the controller, not in ticks, so a host at 50 Hz and one at 1 kHz count the same presses the same way; the resolution is the host's rate, far inside any window a person can click to. No long press: nothing has asked for one.",
"absent": [
{
"name": "view",
"why": "The pad has this button and the service deliberately does not publish it. Binding it works on a bench through controller/xbox.py, which does report it, and does nothing at all on the robot."
}
]
}
+161
View File
@@ -0,0 +1,161 @@
"""The gamepad vocabulary: what a pad can say, for looking up.
`vocabulary.json` beside this file is the dictionary. This reads it and nothing
else -- there is no list in this module, deliberately, because a list here would
be the second copy and the one that goes out of date.
python -m controller --vocabulary
Consumers are elsewhere and each attaches its own meanings: a deploy manifest's
`[[fsm.button]]`, a task's `controls.yaml`. That direction is this package's
rule, stated at the top of `__init__.py`: nothing here has an opinion about what
a stick means.
"""
from __future__ import annotations
import functools
import json
from importlib.resources import files
from pathlib import Path
#: The dictionary. Beside this module so `python -m controller` and an editable
#: install find the same bytes, and shipped as package data -- see `pyproject`.
PATH = Path(__file__).resolve().parent / "vocabulary.json"
SCHEMA = "kk-control-vocabulary/2"
@functools.lru_cache(maxsize=1)
def load() -> dict:
doc = json.loads(files("controller").joinpath("vocabulary.json").read_text("utf-8"))
if doc.get("schema") != SCHEMA:
raise ValueError(f"{PATH}: schema is {doc.get('schema')!r}, not {SCHEMA!r}")
return doc
def buttons() -> tuple[str, ...]:
"""Every button a binding may name, the ten plus the four directions."""
doc = load()
return tuple(b["name"] for b in doc["buttons"] + doc["dpad"])
def axes() -> tuple[str, ...]:
"""Every axis, in the order the wire carries them."""
return tuple(a["name"] for a in load()["axes"])
def keys() -> tuple[str, ...]:
"""Every key a binding may name.
The numeric keypad, and the letters a task has asked for. MuJoCo's viewer
calls the user callback **in addition to** its own shortcuts and binds every
letter, so a letter does two things at once there. `_keys_note` in the
dictionary says which, and why a key there is a press and nothing else.
"""
return tuple(k["name"] for k in load()["keys"])
def key_codes() -> dict[str, dict]:
"""`{name: {"glfw": int, "browser": str}}`, for a host that reads one of
those and has to arrive at the contract's name."""
return {k["name"]: {"glfw": k["glfw"], "browser": k["browser"]}
for k in load()["keys"]}
def modifiers() -> dict[str, tuple[str, ...]]:
"""`{name: keys}`: a modifier is down while either of its keys is.
`ctrl`, `shift` and `alt`, each the left and the right key. A keyboard
binding names one on its own or in front of a key -- `shift+key_j` -- and a
mode switch names one as its `with`. `_modifiers_note` has the rule.
"""
return {m["name"]: tuple(m["keys"]) for m in load()["modifiers"]}
def keystroke(spec: str) -> tuple[str | None, str]:
"""`"shift+key_j"` -> `("shift", "key_j")`; `"key_j"` -> `(None, "key_j")`;
`"ctrl"` -> `(None, "ctrl")`.
What a keyboard binding names: a key, a modifier on its own, or a modifier
held with a key. Raises for anything else, naming what it is not -- a key
nobody reports binds nothing, and would look exactly like one that did.
"""
mods = modifiers()
known = frozenset(keys()) | frozenset(mods)
# Whole names first: `keypad_+` is a key whose name ends in a plus, not a
# chord. A modifier's name has no plus, so a chord splits at the first one.
if spec in known:
return None, spec
modifier, _, key = spec.partition("+")
held = {k for ks in mods.values() for k in ks}
if modifier in mods and key in known and key not in mods and key not in held:
return modifier, key
raise ValueError(
f"{spec!r} is not a key, a modifier ({', '.join(mods)}), or a modifier and a key "
f"(`shift+key_j`). The keys are {' '.join(keys())}"
)
def gestures() -> tuple[str, ...]:
return tuple(g["name"] for g in load()["gestures"])
def absent() -> dict[str, str]:
"""Controls the pad has and the service does not publish, and why.
Kept in the dictionary rather than left out of it: a name that is missing
reads as an oversight, and `view` is a decision.
"""
return {a["name"]: a["why"] for a in load()["absent"]}
def describe() -> str:
"""The whole thing, rendered. What `--vocabulary` prints."""
doc = load()
out = ["buttons (a `pad = ` in [[fsm.button]])"]
out.append(" " + " ".join(b["name"] for b in doc["buttons"]))
out.append(" " + " ".join(b["name"] for b in doc["dpad"]))
for name, why in absent().items():
out.append(f" not `{name}`: {why}")
out.append("")
out.append("keys (a `key = ` in [[fsm.button]])")
doc_keys = doc["keys"]
out.append(" " + " ".join(k["name"] for k in doc_keys))
# One line per contiguous run of GLFW codes. The keypad is one run and the
# letters are not -- a single `first-last` span printed "glfw 320-70" once
# they were added, which reads as a range and is not one.
runs, start = [], 0
for i in range(1, len(doc_keys) + 1):
if i == len(doc_keys) or doc_keys[i]["glfw"] != doc_keys[i - 1]["glfw"] + 1:
a, b = doc_keys[start], doc_keys[i - 1]
runs.append(
f"glfw {a['glfw']}" + (f"-{b['glfw']}" if b is not a else "")
+ f" browser {a['browser']}" + (f"..{b['browser']}" if b is not a else "")
)
start = i
out.append(" " + "; ".join(runs) if len(runs) <= 3 else
f" {len(runs)} runs of codes; see `controller/vocabulary.json`")
out.append(" " + "\n ".join(doc["_keys_note"][:5]))
out.append("")
out.append("modifiers (`with = ` on a key switch; `shift+key_j` in a controls.yaml)")
out.append(" " + " ".join(f"{m}: {' '.join(ks)}" for m, ks in modifiers().items()))
out.append("")
out.append("axes (a `source = ` in a task's controls.yaml)")
for a in doc["axes"]:
lo, hi = a["range"]
out.append(f" {a['name']:<8}[{lo}, {hi}]" + (f" {a['note']}" if "note" in a else ""))
out.append(" " + doc["_axes_note"])
out.append("")
out.append("gestures (an `on = ` in [[fsm.button]])")
for g in doc["gestures"]:
out.append(f" {g['name']:<10}{g['fires']}")
out.append(" " + doc["_gestures_note"])
unresolved = [(b["name"], b["unresolved"]) for b in doc["buttons"] if "unresolved" in b]
if unresolved:
out.append("")
out.append("unresolved")
for name, why in unresolved:
out.append(f" {name}: {why}")
return "\n".join(out) + "\n"
+248
View File
@@ -0,0 +1,248 @@
"""The Xbox pad's evdev codes, and normalising them into a `GamepadState`.
The codes are the Linux `xpad` / `xone` drivers', which is what an Xbox controller
presents over both USB and Bluetooth on a modern kernel. They are kernel
constants, not invented here; the names are `linux/input-event-codes.h`.
## Sign conventions are the kernel's, not the robot's
evdev reports **`ABS_Y` positive when the stick is pushed down**, which is
backwards from every intuition about forward. It is left exactly as the kernel
gives it here, and flipped where the meaning is assigned, in the task. Correcting
it in this file would be a robot opinion smuggled into a device driver: a task
mapping the stick to something else, a camera or a gait frequency, would inherit
a flip that only makes sense for velocity.
## There are two Xbox layouts, and the pad has to be asked which
USB through `xpad` and Bluetooth through `hid-generic` do **not** agree, and the
disagreement is not cosmetic -- it puts the right stick where the other one keeps
a trigger:
axis USB (xpad) Bluetooth (hid-generic)
ABS_X, ABS_Y left stick left stick
ABS_Z, ABS_RZ triggers **right stick**
ABS_RX, ABS_RY right stick absent
ABS_GAS, ABS_BRAKE absent triggers
Measured on an Xbox Wireless Controller over Bluetooth here: `ABS_X ABS_Y ABS_Z
ABS_RZ ABS_GAS ABS_BRAKE ABS_HAT0X ABS_HAT0Y`, with the sticks 0..65535 and the
triggers 0..1023. There is no `ABS_RX` on that device at all. Assuming the USB
layout would have read pushing the right stick as squeezing the left trigger --
which is not a crash, just a control that does the wrong thing.
`layout_of` picks between the two by **which axes the device actually reports**,
which is evidence from the device rather than a table of models and firmware
revisions that would need extending every time someone brings a different pad.
## Ranges come from the device
Sticks are -32768..32767 on one of those layouts and 0..65535 on the other, and
triggers are 0..1023 or 0..255. `normalise` is given the ranges read from
`EVIOCGABS` and uses them, so nothing here assumes a model -- the unsigned sticks
above work out because the midpoint is computed rather than assumed to be zero.
The dead zone is the driver's own `flat` for the same reason: a pad that reports
its own slop knows better than a constant would.
"""
from __future__ import annotations
from dataclasses import dataclass, field
__all__ = ["ABS_AXES", "BUTTONS", "GamepadState", "normalise"]
EV_KEY = 0x01
EV_ABS = 0x03
# ── Axes (EV_ABS) ────────────────────────────────────────────────────────
ABS_X = 0x00 # left stick, right positive (both layouts)
ABS_Y = 0x01 # left stick, **down positive** (both layouts)
ABS_Z = 0x02 # USB: left trigger | Bluetooth: right stick X
ABS_RX = 0x03 # USB: right stick X | Bluetooth: absent
ABS_RY = 0x04 # USB: right stick Y | Bluetooth: absent
ABS_RZ = 0x05 # USB: right trigger | Bluetooth: right stick Y
ABS_GAS = 0x09 # Bluetooth: right trigger
ABS_BRAKE = 0x0A # Bluetooth: left trigger
ABS_HAT0X = 0x10 # d-pad, -1 / 0 / +1 (both layouts)
ABS_HAT0Y = 0x11
#: Every axis worth asking the device about, across both layouts. `device.py`
#: queries one range per entry and simply skips the ones the pad does not have,
#: which is also how `layout_of` learns which layout it is looking at.
ABS_AXES = (
ABS_X, ABS_Y, ABS_Z, ABS_RX, ABS_RY, ABS_RZ,
ABS_GAS, ABS_BRAKE, ABS_HAT0X, ABS_HAT0Y,
)
@dataclass(frozen=True)
class Layout:
"""Which code carries which control on this particular pad."""
name: str
rx: int
ry: int
lt: int
rt: int
#: xpad over USB: the right stick is RX/RY and the triggers are the Z pair.
USB_LAYOUT = Layout("usb", rx=ABS_RX, ry=ABS_RY, lt=ABS_Z, rt=ABS_RZ)
#: hid-generic over Bluetooth: the Z pair *is* the right stick, and the triggers
#: moved to the pedal codes.
BLUETOOTH_LAYOUT = Layout("bluetooth", rx=ABS_Z, ry=ABS_RZ, lt=ABS_BRAKE, rt=ABS_GAS)
def layout_of(axes) -> Layout:
"""Pick a layout from the axes the device reports.
`ABS_RX` is the discriminator because it exists on exactly one of the two.
Falling back to USB when neither is conclusive keeps an unknown pad working as
before rather than reading its sticks as triggers.
"""
if ABS_RX in axes and ABS_RY in axes:
return USB_LAYOUT
if ABS_GAS in axes or ABS_BRAKE in axes:
return BLUETOOTH_LAYOUT
return USB_LAYOUT
# ── Buttons (EV_KEY) ─────────────────────────────────────────────────────
BTN_SOUTH = 0x130 # A
BTN_EAST = 0x131 # B
BTN_WEST = 0x134 # X -- note the codes are not in ABXY order
BTN_NORTH = 0x133 # Y
BTN_TL = 0x136 # left bumper
BTN_TR = 0x137 # right bumper
BTN_SELECT = 0x13A # view
BTN_START = 0x13B # menu
BTN_MODE = 0x13C # the Xbox button
BTN_THUMBL = 0x13D
BTN_THUMBR = 0x13E
#: Button code -> the name the **dictionary** gives it.
#:
#: This package's job is to align a pad to those names and stop there; what any
#: of them means belongs to whoever is driving, which is a task. So these are
#: not our own spelling any more -- they are `controller/vocabulary.json`'s, the
#: same names the robot's own pad service publishes and the same ones a
#: `[[fsm.button]]` or a `controls.yaml` may write.
#:
#: `view` is here and not in the dictionary: this pad reports it and the robot's
#: service deliberately does not. Reported rather than dropped, so `--raw` and a
#: person pressing buttons see the truth; `vocabulary.absent()` is where the
#: asymmetry is written down, and nothing downstream may bind it.
#:
#: **X and Y are unresolved.** The dictionary records it: we read 307 as `Y` and
#: 308 as `X`; the service's table reads them the other way round. One of us is
#: wrong about which physical button it is and pressing it settles it. Left
#: disagreeing rather than quietly picked.
BUTTONS = {
BTN_SOUTH: "A", BTN_EAST: "B", BTN_WEST: "X", BTN_NORTH: "Y",
BTN_TL: "LB", BTN_TR: "RB", BTN_SELECT: "view", BTN_START: "menu",
BTN_MODE: "home", BTN_THUMBL: "L3", BTN_THUMBR: "R3",
}
@dataclass(frozen=True)
class GamepadState:
"""One snapshot. Sticks are -1..1, triggers 0..1, buttons bool.
`ly` and `ry` keep the kernel's sign: **positive is down**.
"""
lx: float = 0.0
ly: float = 0.0
rx: float = 0.0
ry: float = 0.0
lt: float = 0.0
rt: float = 0.0
hat_x: int = 0
hat_y: int = 0
buttons: frozenset[str] = field(default_factory=frozenset)
def __getattr__(self, item: str) -> bool:
"""`state.A` for the buttons, so a caller does not have to remember
whether a name is a field or a set member. The names are the
dictionary's."""
if item in BUTTONS.values():
return item in self.buttons
raise AttributeError(item)
@property
def any_input(self) -> bool:
"""Is the operator touching it? What "taking over" is decided on."""
return (
bool(self.buttons)
or self.hat_x != 0
or self.hat_y != 0
or max(abs(self.lx), abs(self.ly), abs(self.rx), abs(self.ry)) > 0.0
or max(self.lt, self.rt) > 0.0
)
def _axis(
raw: dict[tuple[int, int], int],
ranges: dict[int, tuple[int, int, int]],
code: int,
bipolar: bool,
) -> float:
"""One axis, scaled to -1..1 (bipolar) or 0..1, with the dead zone removed.
The dead zone is **rescaled, not just clipped**: an axis that starts moving at
the edge of the zone should start from zero, not jump to the fraction the zone
occupied. Clipping alone leaves a visible step at the point the stick starts to
bite, which reads as a sticky control.
"""
if code not in ranges:
return 0.0
value = raw.get((EV_ABS, code))
if value is None:
return 0.0
lo, hi, flat = ranges[code]
if bipolar:
mid = (lo + hi) / 2.0
span = (hi - lo) / 2.0
if span <= 0:
return 0.0
x = (value - mid) / span
dead = flat / span if span else 0.0
if abs(x) <= dead:
return 0.0
x = (abs(x) - dead) / (1.0 - dead) * (1.0 if x > 0 else -1.0)
return max(-1.0, min(1.0, x))
span = hi - lo
if span <= 0:
return 0.0
x = (value - lo) / span
dead = flat / span
if x <= dead:
return 0.0
return max(0.0, min(1.0, (x - dead) / (1.0 - dead)))
def normalise(
raw: dict[tuple[int, int], int], ranges: dict[int, tuple[int, int, int]]
) -> GamepadState:
"""Raw evdev values plus the device's own ranges -> a `GamepadState`.
The layout is decided from `ranges`, which holds exactly the axes the device
answered `EVIOCGABS` for. See `layout_of`.
"""
pressed = frozenset(
name for code, name in BUTTONS.items() if raw.get((EV_KEY, code), 0)
)
layout = layout_of(ranges)
return GamepadState(
lx=_axis(raw, ranges, ABS_X, True),
ly=_axis(raw, ranges, ABS_Y, True),
rx=_axis(raw, ranges, layout.rx, True),
ry=_axis(raw, ranges, layout.ry, True),
lt=_axis(raw, ranges, layout.lt, False),
rt=_axis(raw, ranges, layout.rt, False),
hat_x=int(raw.get((EV_ABS, ABS_HAT0X), 0)),
hat_y=int(raw.get((EV_ABS, ABS_HAT0Y), 0)),
buttons=pressed,
)
+300
View File
@@ -0,0 +1,300 @@
"""Reading an Xbox pad on Windows through XInput, with no dependencies.
## Why XInput
XInput is a plain C function in a DLL every Windows since 8 ships
(`xinput1_4.dll`), so `ctypes` reaches it and nothing has to be installed -- the
same trade `device.py` makes with evdev. The price is that it reads **only XInput
devices**: Xbox pads and the ones that present themselves as one. A PlayStation or
Switch pad is not one until Steam Input or DS4Windows sits in between.
## It speaks evdev, not a dialect of its own
Each reading is turned into the `{(type, code): value}` table the Linux `xpad`
driver produces for the same pad over USB, with the ranges `xpad` declares, and
then handed to `xbox.normalise` -- the function that reads a Linux pad. The sign
conventions, the dead-zone rescaling and the button names are therefore one piece
of code on both platforms rather than two that have to be kept agreeing, and
`python -m controller` is the same bench check on either.
## The Y axes are flipped here, and nowhere else
XInput reports a stick pushed **up** as positive; evdev, and everything above this
package, reads **down** as positive (`xbox.py` says why that is kept). The flip is
`~y`, which is what `xpad` itself does to the same 16-bit value: it maps
-32768..32767 onto itself exactly, where negation would leave the range at -32768.
Leaving it out would not raise -- it would walk the robot backwards when the stick
says forwards.
## The dead zone is Microsoft's
evdev hands over the driver's `flat`; XInput reports none. Microsoft documents one
instead, `XINPUT_GAMEPAD_LEFT_THUMB_DEADZONE` and its siblings, because a stick does
not return to exactly centre, and that is what goes into `flat` here -- the
platform's own figure, which is the role the driver's `flat` plays on Linux. It is
wider than what a Linux pad reports (xpad's 128 of 32768 over USB; 4095 of 65535
from `hid-generic` over Bluetooth), so the stick travels further before it bites,
and is rescaled like every dead zone here so full deflection still reads 1.
## Buttons are matched by name
XInput's bits name physical buttons, so each is mapped onto the code `xbox.BUTTONS`
gives **that name**, not onto the code `xpad` would send. The physical X button is
`X` here whatever is eventually decided about evdev's 307 and 308, which
`controller/vocabulary.json` records as unresolved.
## The Xbox button needs the undocumented entry point
`XInputGetState` never reports it. Ordinal 100 of the same DLL, `XInputGetStateEx`,
does -- it is what SDL reads -- and it writes four bytes more than the documented
struct holds, so the struct below carries them. It is used when the DLL exports it
and `home` simply never goes down when it does not (`xinput9_1_0.dll`). Windows
also opens the Game Bar on that button unless the Game Bar is told not to.
"""
from __future__ import annotations
import ctypes
import functools
import sys
from collections.abc import Callable
from dataclasses import dataclass, field
from typing import NamedTuple
from .device import PadInfo
from .xbox import (
ABS_HAT0X,
ABS_HAT0Y,
ABS_RX,
ABS_RY,
ABS_RZ,
ABS_X,
ABS_Y,
ABS_Z,
BUTTONS,
EV_ABS,
EV_KEY,
GamepadState,
normalise,
)
__all__ = ["RANGES", "XInputGamepad", "XInputPad", "describe", "find_pads", "read", "translate"]
#: `XUSER_MAX_COUNT`: XInput has four player slots and no more.
SLOTS = 4
ERROR_SUCCESS = 0
# ── The documented constants, from XInput.h ─────────────────────────────
DPAD_UP = 0x0001
DPAD_DOWN = 0x0002
DPAD_LEFT = 0x0004
DPAD_RIGHT = 0x0008
LEFT_THUMB_DEADZONE = 7849
RIGHT_THUMB_DEADZONE = 8689
TRIGGER_THRESHOLD = 30
#: XInput button bit -> the dictionary's name for that physical button. 0x0400 is
#: the Xbox button, which only `XInputGetStateEx` sets.
_BUTTON_NAMES = {
0x1000: "A", 0x2000: "B", 0x4000: "X", 0x8000: "Y",
0x0100: "LB", 0x0200: "RB", 0x0020: "view", 0x0010: "menu",
0x0400: "home", 0x0040: "L3", 0x0080: "R3",
}
_CODE_OF = {name: code for code, name in BUTTONS.items()}
#: XInput button bit -> evdev key code, by way of the name. A name `xbox.BUTTONS`
#: does not know raises here, at import, rather than reading as never pressed.
BUTTON_CODES = {bit: _CODE_OF[name] for bit, name in _BUTTON_NAMES.items()}
#: What `xpad` declares for these axes over USB, with Microsoft's dead zones as the
#: `flat`. Having `ABS_RX`/`ABS_RY` is also what makes `xbox.layout_of` choose the
#: USB layout, so the triggers are read from `ABS_Z`/`ABS_RZ` and not the sticks.
RANGES: dict[int, tuple[int, int, int]] = {
ABS_X: (-32768, 32767, LEFT_THUMB_DEADZONE),
ABS_Y: (-32768, 32767, LEFT_THUMB_DEADZONE),
ABS_RX: (-32768, 32767, RIGHT_THUMB_DEADZONE),
ABS_RY: (-32768, 32767, RIGHT_THUMB_DEADZONE),
ABS_Z: (0, 255, TRIGGER_THRESHOLD),
ABS_RZ: (0, 255, TRIGGER_THRESHOLD),
ABS_HAT0X: (-1, 1, 0),
ABS_HAT0Y: (-1, 1, 0),
}
class XInputGamepad(ctypes.Structure):
"""`XINPUT_GAMEPAD`, plus the trailing DWORD `XInputGetStateEx` writes.
Fixed-width types rather than `ctypes.wintypes`, so the struct -- and
`translate` with it -- exists on every platform and is tested on Linux.
"""
_fields_ = [
("wButtons", ctypes.c_uint16),
("bLeftTrigger", ctypes.c_uint8),
("bRightTrigger", ctypes.c_uint8),
("sThumbLX", ctypes.c_int16),
("sThumbLY", ctypes.c_int16),
("sThumbRX", ctypes.c_int16),
("sThumbRY", ctypes.c_int16),
("dwPaddingReserved", ctypes.c_uint32),
]
class XInputState(ctypes.Structure):
"""`XINPUT_STATE` as `XInputGetStateEx` fills it."""
_fields_ = [("dwPacketNumber", ctypes.c_uint32), ("Gamepad", XInputGamepad)]
# Asserted rather than assumed: a buffer four bytes short of what the Ex call
# writes does not raise, it overwrites whatever ctypes allocated next to it.
assert ctypes.sizeof(XInputGamepad) == 16 and ctypes.sizeof(XInputState) == 20, (
f"XINPUT_STATE_EX is {ctypes.sizeof(XInputState)} bytes here, not 20"
)
def translate(pad: XInputGamepad) -> dict[tuple[int, int], int]:
"""One XInput reading -> the raw evdev table `xpad` would have produced."""
buttons = pad.wButtons
raw = {(EV_KEY, code): int(bool(buttons & bit)) for bit, code in BUTTON_CODES.items()}
raw[(EV_ABS, ABS_X)] = pad.sThumbLX
raw[(EV_ABS, ABS_Y)] = ~pad.sThumbLY # up is positive in XInput, down in evdev
raw[(EV_ABS, ABS_RX)] = pad.sThumbRX
raw[(EV_ABS, ABS_RY)] = ~pad.sThumbRY
raw[(EV_ABS, ABS_Z)] = pad.bLeftTrigger
raw[(EV_ABS, ABS_RZ)] = pad.bRightTrigger
raw[(EV_ABS, ABS_HAT0X)] = bool(buttons & DPAD_RIGHT) - bool(buttons & DPAD_LEFT)
raw[(EV_ABS, ABS_HAT0Y)] = bool(buttons & DPAD_DOWN) - bool(buttons & DPAD_UP)
return raw
class _Api(NamedTuple):
dll: str
get_state: Callable[[int, object], int]
#: Whether `get_state` is `XInputGetStateEx`, i.e. whether `home` can be read.
guide: bool
#: Newest first. 1_4 ships with Windows 8 and later; 1_3 comes with the DirectX
#: runtime; 9_1_0 is Windows 7's and has no ordinal 100.
_DLLS = ("xinput1_4", "xinput1_3", "xinput9_1_0")
@functools.lru_cache(maxsize=1)
def _api() -> _Api | None:
if sys.platform != "win32":
return None
for name in _DLLS:
try:
dll = ctypes.WinDLL(name)
except OSError:
continue
try:
fn, guide = dll[100], True
except AttributeError:
fn, guide = dll.XInputGetState, False
fn.argtypes = (ctypes.c_uint32, ctypes.POINTER(XInputState))
fn.restype = ctypes.c_uint32
return _Api(name, fn, guide)
return None
def read(slot: int) -> XInputGamepad | None:
"""The pad in `slot` now, or None if there is none (or no XInput at all)."""
api = _api()
if api is None:
return None
state = XInputState()
if api.get_state(slot, ctypes.byref(state)) != ERROR_SUCCESS:
return None
return state.Gamepad
def find_pads(readable_only: bool = False) -> list[PadInfo]:
"""Every slot with a pad in it. XInput has no permissions to be refused, so a
pad that answers is one this process can read, and `readable_only` changes
nothing."""
del readable_only
return [
PadInfo(f"xinput:{slot}", f"XInput pad, player {slot + 1}", True)
for slot in range(SLOTS)
if read(slot) is not None
]
def describe() -> str:
"""What was found, or why nothing was."""
api = _api()
if api is None:
return (
"XInput is not available: none of "
+ ", ".join(f"{name}.dll" for name in _DLLS)
+ " could be loaded."
)
pads = find_pads()
if not pads:
return (
"no XInput controller is connected. Windows pads are read through XInput, "
"which sees Xbox pads and pads that present themselves as one; a "
"PlayStation or Switch pad needs Steam Input or DS4Windows in between."
)
lines = [f" {p.path} {p.name} -- usable, through {api.dll}" for p in pads]
if not api.guide:
lines.append(f" {api.dll} has no XInputGetStateEx, so `home` never reads as pressed")
return "\n".join(lines)
@dataclass
class XInputPad:
"""An XInput slot, presenting the same surface as `device.Gamepad`.
Read on the caller's thread, once per `state()`: XInput is a poll API with
nothing to wait on, so a reader thread would only add latency -- up to the
15.6 ms system timer tick on Python 3.10, whose sleeps on Windows round up to
it. When the pad goes away the last reading is kept and `connected` goes False,
as `Gamepad` does, and the empty slot is not polled again: an empty slot is the
slow case of `XInputGetState`, which Microsoft's documentation of it says not to
poll every frame.
"""
info: PadInfo
_raw: dict[tuple[int, int], int] = field(default_factory=dict)
_ranges: dict[int, tuple[int, int, int]] = field(default_factory=lambda: dict(RANGES))
_alive: bool = False
@property
def name(self) -> str:
return self.info.name
@property
def _slot(self) -> int:
return int(self.info.path.rpartition(":")[2])
def start(self) -> None:
self._alive = True
self._poll()
def close(self) -> None:
self._alive = False
@property
def connected(self) -> bool:
return self._alive
def _poll(self) -> None:
if not self._alive:
return
pad = read(self._slot)
if pad is None:
self._alive = False
return
self._raw = translate(pad)
def state(self) -> GamepadState:
"""A normalised snapshot. Never raises."""
self._poll()
return normalise(dict(self._raw), self._ranges)
def raw(self) -> dict[tuple[int, int], int]:
"""The reading as evdev codes, for `python -m controller --raw`."""
self._poll()
return dict(self._raw)
+196
View File
@@ -0,0 +1,196 @@
# Robot application formats
`scripts/deploy.py` writes a directory and a sibling `.app`: an ordinary ZIP with
`bundle.json` at its root. Entry order, timestamps and permissions are normalized.
Readers select a format by its exact `schema` tag, not by guessing from files.
[`app.schema`](app.schema) describes both versions. Python readers validate it
before loading control code or models.
## Self-contained native application — `mjrl-robot-app/1`
This format includes a robot model and supports `rl_onnx`, `ik_mocap`,
`wheeled_velocity`, `cpg_keyframe` and `model_viewer`. The Python runtime runs on
Linux, Windows and macOS with compatible Python and the declared dependencies.
It is a native MuJoCo application; it does not claim a browser, NPU or hardware
binary. Host Python/MuJoCo libraries are dependencies, not embedded interpreters.
```
robot/
bundle.json
app.schema
LICENSE
README.md
model/robot.xml
model/meshes/<digest>.<extension>
model/textures/<digest>.<extension>
controller.toml
<mode>.json
controls/<mode>.yaml
<mode>.trajectory.json
models/<mode>.onnx
<mode>.layout.json
verification.json
verification/<mode>.json
reference.json
manual.en.json
manual.zh.json
runtime/controller.pyz
runtime/requirements.txt
```
Only applicable companions are present: a viewer needs neither a policy nor a
trajectory. A controlled mode requires both native physics and control acceptance.
A geometry viewer claims model compilation/display only. It cannot inherit a
physical or locomotion capability because it successfully opened a file.
Every file except `bundle.json` is inventoried with byte length and SHA-256. Paths
are local relative paths. The reader rejects missing/extra files, mismatched bytes,
links, archive traversal, duplicate entries and excessive archive expansion. The
normalized model has no XML includes or plugin declarations and its mesh/texture
references must remain inside the app. Controller dependencies must be inventoried.
Hashes detect changes; they are not signatures authenticating an outside publisher.
Validation uses the trusted schema shipped with the host package or portable
runtime. The application's `app.schema` is an informational copy and cannot
weaken the reader's rules.
The manifest declares `robot_id`, `model`, `default_mode`, `modes`, `controller`,
`manuals`, `reference`, `verification`, `runtimes`, `capabilities`, `source` and
`files`. Each mode names its `control_mode` and `contract`; it names `controls`,
`trajectory` companions through the contract, and `policy`/`layout` for ONNX.
The declared capability must agree with the controller kind.
`controller.toml` uses `mjrl-app-controller/1` and declares `default_mode`. Native
mode selection is explicit at launch (`--app-mode`) or through
`RobotApplication.select_mode`. Selecting a different mode resets the world to that
mode's verified initial state. It is not a hardware state-machine transition.
### Build and replay
First create a normalized model and controller with `scripts/onboard.py`. Then
verify the exact controller, model and explicit physical response probes:
```bash
python scripts/onboard.py verify --model out/robot/robot.xml \
--config out/robot/verify-input.json --report out/robot/verification.json
python scripts/deploy.py --robot robot --model out/robot/robot.xml \
--control out/robot/controller.json --controls out/robot/controls.yaml \
--verification out/robot/verification.json --bundle out/apps
python scripts/play.py --app out/apps/robot.app --headless --steps 100
python scripts/deploy.py --check-reference out/apps/robot.app
```
An ONNX mode also supplies `--policy <export-directory>` containing `actor.onnx`
and `layout.json`; `--control` identifies the same controller JSON used by verify.
A viewer omits `--control` and `--verification`.
Multiple modes use `--platform-manifest FILE` with `robot_id`, `model`, `modes`,
optional `default_mode`, and optional shared `verification`. Mode entries accept
`controller`, `controls`, `policy`, `layout` and a mode-specific `verification`.
Paths resolve relative to the build manifest. Each controlled mode must match its
own passing report; a report for another controller cannot authorize it.
Existing output directories/archives are refused rather than overwritten.
Concurrent builders targeting the same output are also refused.
Supply asset and policy license or attribution files with repeatable
`--license PATH` options, or the build manifest's `licenses` list. They are copied
under `licenses/`, listed in the application manifest, and covered by file hashes.
The platform's Apache license does not replace the imported asset's license.
A report records the source XML hash, compiled model hash (including dependency
effects), controller hash, controls hash and policy/trajectory dependency hashes.
The bundler checks these before relocation. It compiles the relocated model and
constructs each packaged controller and replays every reference before publishing.
Reports must contain the current initial-pose, finite-state, contact, joint-margin
and sustained-effort checks; a historical pass flag alone is insufficient.
No physical asset values
are guessed to repair a failed bundle.
The application records the verified initial state, control rate, probe inputs,
measured samples and coordinate conventions. `--check-reference` replays these
inputs and compares qpos, qvel, control, actuator force and time. Ordinary playback
is not an acceptance test unless its command and response measurements are checked.
For an independent host, unpack the archive, install `runtime/requirements.txt` in
an isolated Python environment, and run:
```bash
python runtime/controller.pyz --app . --headless --steps 100
```
The packaged Python code is run only by that explicit command. The workbench
loads its installed runtime implementation, not arbitrary code from an opened app.
Use `--reference-out FILE` to record actual playback and `--video FILE.mp4` with
finite `--steps` for a preview. On a headless Linux rendering host, select an
available MuJoCo renderer (for example `MUJOCO_GL=egl`).
On macOS, MuJoCo's interactive viewer requires the `mjpython` launcher instead
of `python`; headless replay uses ordinary Python. Native gamepad polling is
implemented for Linux and Windows; macOS clients can use keyboard or API input.
For a fixed velocity command, `--evaluation-out FILE` writes body-frame mean and
final velocities, MAE/RMSE, peak tilt, resets and warning counts. It does not infer
acceptance thresholds. The preview camera follows a model with one robot root.
## Legacy policy application — `kk-policy-bundle/1`
The existing Rust FSM path remains supported for policies whose contracts its
reader implements. This version contains controller runtimes and policies, but
its simulation world still comes from an installed task. It does not promise a
self-contained model or traditional-control modes.
| Host | Inference | Runtime |
|---|---|---|
| `board` | RKNN | `runtime/board/controller` |
| `web` | ONNX | `runtime/web/controller.wasm` and `controller.js` |
| `mjlab` | ONNX | `runtime/mjlab/<platform>/controller.so` or `.pyd` |
Shared files are `controller.toml`, `<mode>.json`, optional trajectory companions,
`reference.json`, manuals, `README.md`, and `models/<mode>.onnx`/`.rknn`. The
schema's `x-files` inventory describes this legacy format only.
A default legacy build prepares board conversion and cross-builds, checks every
host declared in `runtimes`, produces numerical reference vectors, validates the
schema and packs the archive. `--allow-incomplete` skips unavailable board work
and explicitly records omissions; it is not a board-ready release.
`deploy/manifests.json` starts empty in the generic platform. Populate it with
explicit exports for your hardware, or use repeated `--mode NAME=EXPORT` and a
provided `--fsm FILE`. One mode can use the generated single-mode FSM. No robot,
board address, motor order or hardware-specific mode is selected implicitly.
The Rust contracts map `wire_joint_order`, `obs_joint_order` and
`action_joint_order` by joint name. `default_joint_pos` covers the wire order.
The policy contract records observation terms/history, action transform, rates,
gains, limits and command ranges. Never replace these with positional alignment
or guessed dimensions. Unsupported versioned native action semantics must fail
closed in the legacy reader.
The implementation references are:
| Contract | Reader |
|---|---|
| Bundle and host selection | `fsm/src/bundle.rs` |
| FSM | `fsm/src/config.rs` |
| Named policy layout | `fsm/src/layout.rs` |
| Input mappings | `fsm/src/operator.rs` |
| Reference motion | `fsm/src/trajectory.rs` |
| Recorded reference vectors | `fsm/src/reference.rs` |
| Browser and Python entry points | `fsm/src/web.rs`, `fsm/src/py.rs` |
The generated legacy bundle README, from
[`fsm/BUNDLE_README.md`](fsm/BUNDLE_README.md), documents its actual host API.
## Manuals and translations
Both application formats carry `manual.en.json` (`kk-bundle-manual/1`). The English
manual comes from the controller's bindings. It lists each mode, entry/exit,
keyboard bindings, pad controls and mode switches where implemented.
The bundle-manual skill translates only `source`, `summary`, `enter`, `leave`,
`keyboard`, `pad` and `does`, preserving every identifier and list structure.
```bash
python scripts/deploy.py --translate out/apps/robot \
--language zh --manual /tmp/manual.zh.json
```
The command validates the translation, updates inventory digests and repacks the
archive. Never translate joint names, mode names, file paths or physical key IDs.
+93
View File
@@ -0,0 +1,93 @@
# Export, verify and package a robot application
The deterministic path starts with a normalized model and an explicit controller
contract. The agent skills choose the route and extract metadata; CLI commands
perform transformations, numerical checks and publication.
Read [BUNDLE.md](BUNDLE.md) when implementing a consumer and
[the main tutorial](../docs/TUTORIAL.md) for asset onboarding.
## Native MuJoCo application
```bash
python scripts/onboard.py inspect assets/my_robot
python scripts/onboard.py build-mjcf assets/my_robot --output out/my_robot/model.xml
python scripts/onboard.py setup-tier2 --model out/my_robot/model.xml \
--config assets/my_robot/controller.json --output out/my_robot/control
python scripts/onboard.py verify --model out/my_robot/control/robot.xml \
--config out/my_robot/verify-input.json --report out/my_robot/verification.json
python scripts/deploy.py --robot my_robot --model out/my_robot/control/robot.xml \
--control out/my_robot/control/controller.json --controls out/my_robot/control/controls.yaml \
--verification out/my_robot/verification.json --bundle out/apps
python scripts/play.py --app out/apps/my_robot.app --headless --steps 100
python scripts/deploy.py --check-reference out/apps/my_robot.app
```
Configuration examples for IK, joint sliders, trajectories, wheel mixing, CPG and
policy adapters are in [CONTROLLER_SETUP.md](../docs/CONTROLLER_SETUP.md). Verification
requires explicit expected physical responses. A generated configuration or an
ONNX graph that compiles is not evidence that the robot tracks a command.
Geometry-only assets can publish `model_viewer` apps by omitting `--control` and
`--verification`. They carry no physical/control acceptance claim. A controlled
application is refused unless its verification matches model dependencies,
controller parameters, controls, policy and trajectory bytes.
An ONNX application additionally supplies `--policy <export>` and its verified
controller JSON. The native application contains the normalized model, meshes,
contracts, references and portable Python runtime. It needs host Python and the
listed third-party packages. Mode support is explicitly versioned; do not pass a
native hybrid-action contract to a legacy board reader.
Include each imported asset's license and attribution with repeatable
`--license PATH` arguments. The application preserves these files and their hashes;
the platform license does not replace third-party terms.
## Training export
```bash
python scripts/export.py --task my_robot --checkpoint logs/my_robot/my_robot/run/model_N.pt
```
The exporter validates measurable observations and numerical parity with the
checkpoint. Preserve the emitted joint orders, action transforms, gains, rates,
command ranges and provenance. Use the source revision and robot spec that the
checkpoint was trained with. Reconfiguring the environment after training is not
an adaptation of the old policy.
## Legacy board and web builds
The Rust FSM remains the common controller for its supported legacy policy
contracts. `manifests.json` contains no default robot. Add explicit export paths,
mode switches and a cascade configuration for your machine, or use:
```bash
python scripts/deploy.py --mode walk=<export-directory> --bundle out/legacy
```
A complete legacy build converts ONNX to RKNN, cross-builds the board controller
and Python extensions, builds WebAssembly, checks contracts on each included
host, writes a reference and packs one app. It requires the documented toolchains.
`--allow-incomplete` omits unavailable board components and records omissions;
this flag is for development hosts, not a board-ready release.
- [ONNX to RKNN](convert/README.md)
- [Rust controller and runtime builds](fsm/README.md)
- [DDS wire interface](dds/README.md)
Do not infer a board address or mechanical compatibility. Use the deploy skill's
read-only board preflight with an explicit host before deployment. First open the
app with `--dry-run`, then run `--check-reference`; these checks do not drive a bus.
Actual hardware operation requires the target machine configuration and the
operator's explicit instruction.
## Operator manuals
Every build includes `manual.en.json`. Apply the bundle-manual skill after each
build to create the Chinese translation. Add it through the checked path:
```bash
python scripts/deploy.py --translate out/apps/my_robot --language zh --manual /tmp/manual.zh.json
```
Identifiers must remain unchanged. The command updates hashes and repacks the app.
+64
View File
@@ -0,0 +1,64 @@
<!-- tracks: README.md @ sha256:767fc6409321444a -->
# 导出、验收与机器人应用打包
确定性流程从规范化模型和明确的控制契约开始。Agent 技能负责选择路线并提取元数据;CLI 命令负责转换、数值检查和发布。
实现消费端时请阅读 [BUNDLE.md](BUNDLE.md),资产接入流程见[主教程](../docs/TUTORIAL.zh.md)。
## 原生 MuJoCo 应用
```bash
python scripts/onboard.py inspect assets/my_robot
python scripts/onboard.py build-mjcf assets/my_robot --output out/my_robot/model.xml
python scripts/onboard.py setup-tier2 --model out/my_robot/model.xml \
--config assets/my_robot/controller.json --output out/my_robot/control
python scripts/onboard.py verify --model out/my_robot/control/robot.xml \
--config out/my_robot/verify-input.json --report out/my_robot/verification.json
python scripts/deploy.py --robot my_robot --model out/my_robot/control/robot.xml \
--control out/my_robot/control/controller.json --controls out/my_robot/control/controls.yaml \
--verification out/my_robot/verification.json --bundle out/apps
python scripts/play.py --app out/apps/my_robot.app --headless --steps 100
python scripts/deploy.py --check-reference out/apps/my_robot.app
```
IK、关节滑块、轨迹、轮速混控、CPG 和策略适配的配置示例见 [CONTROLLER_SETUP.md](../docs/CONTROLLER_SETUP.md)。验收必须明确规定预期的物理响应。生成配置或成功编译 ONNX 图,均不能证明机器人能跟踪命令。
仅几何资产可以省略 `--control` 与 `--verification`,发布 `model_viewer` 应用。这类包不声明物理或控制验收通过。受控应用的验收记录必须与模型依赖、控制器参数、操作绑定、策略和轨迹文件完全匹配,否则拒绝打包。
ONNX 应用还须提供 `--policy <export>` 及经过验收的控制器 JSON。原生应用包含规范化模型、网格、契约、参考记录和便携 Python 运行时;主机需要安装 Python 及列出的第三方依赖。控制模式支持按版本明确声明;不能将原生混合动作契约交给旧版板端读取器。
使用可重复的 `--license PATH` 参数加入导入资产的许可证及来源说明。应用会保留这些文件及其摘要;平台许可证不能替代第三方条款。
## 训练导出
```bash
python scripts/export.py --task my_robot --checkpoint logs/my_robot/my_robot/run/model_N.pt
```
导出器校验可测量的观测项,以及与检查点之间的数值一致性。保留输出中的关节顺序、动作变换、增益、频率、命令范围和来源记录。使用训练检查点时的代码版本及机器人规格。训练后改变环境配置,并不等于完成旧策略的适配。
## 旧版板端与网页构建
Rust FSM 继续作为其支持的旧版策略契约的公共控制器。`manifests.json` 不再预置机器人。请为目标机器明确填写导出路径、模式切换及级联配置,或使用:
```bash
python scripts/deploy.py --mode walk=<export-directory> --bundle out/legacy
```
完整的旧版构建会将 ONNX 转为 RKNN,交叉编译板端控制器和 Python 扩展,构建 WebAssembly,在每个包含的主机端校验契约,写入参考记录并打包成一个应用。它需要文档中列出的工具链。`--allow-incomplete` 会省略不可用的板端组件并记录缺失项;该选项用于开发主机,不代表可以上板发布。
- [ONNX 转 RKNN](convert/README.md)
- [Rust 控制器与运行时构建](fsm/README.md)
- [DDS 通信接口](dds/README.md)
不要推测板端地址或机械兼容性。部署前,使用明确的主机地址执行 deploy 技能的只读板端预检。先用 `--dry-run` 打开应用,再运行 `--check-reference`;这些检查不会驱动总线。实际硬件操作需要目标机器配置及操作者的明确指令。
## 操作手册
每次构建都包含 `manual.en.json`。构建后使用 bundle-manual 技能生成中文译本,通过以下校验入口加入:
```bash
python scripts/deploy.py --translate out/apps/my_robot --language zh --manual /tmp/manual.zh.json
```
标识符必须保持不变。命令会更新文件摘要并重新打包应用。
+809
View File
@@ -0,0 +1,809 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "kk-policy-bundle/1",
"title": "A robot application bundle: <name>/ and <name>.app",
"description": "Robot application formats: legacy policy bundles and self-contained native robot applications. Each version declares its actual supported runtimes.",
"x-files": [
{
"path": "^bundle\\.json$",
"required": true,
"format": "json",
"is": "this schema's document: every file, its size and SHA-256, and which host takes what"
},
{
"path": "^README\\.md$",
"required": true,
"format": "markdown",
"is": "the integration guide, from deploy/fsm/BUNDLE_README.md"
},
{
"path": "^controller\\.toml$",
"required": true,
"format": "toml",
"is": "the FSM: modes, the rules that switch between them, the keys and buttons",
"namedBy": "fsm"
},
{
"path": "^manual\\.en\\.json$",
"required": true,
"format": "json",
"schema": "#/$defs/manual",
"is": "every key and button the bundle reads and what it does, in English, generated at build time",
"namedBy": "manuals.en"
},
{
"path": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$",
"required": false,
"format": "json",
"schema": "#/$defs/manual",
"is": "a translation of manual.en.json, added with `deploy.py --translate`",
"namedBy": "manuals.<language>"
},
{
"path": "^reference\\.json$",
"required": false,
"format": "json",
"is": "kk-policy-reference/1: recorded frames every host replays to measure that it agrees; absent only where onnxruntime was not installed at build time",
"namedBy": "reference"
},
{
"path": "^[^/]+\\.trajectory\\.json$",
"required": false,
"format": "json",
"is": "a recording a reference-guided mode is driven by, beside the contracts",
"namedBy": "modes.<mode>.trajectory"
},
{
"path": "^[a-z][a-z0-9_]*\\.json$",
"required": true,
"format": "json",
"is": "one mode's contract (the export's layout.json): observation layout, joint order, gains, controls",
"namedBy": "modes.<mode>.contract"
},
{
"path": "^models/[a-z][a-z0-9_]*\\.onnx$",
"required": true,
"format": "onnx",
"is": "a policy, for the browser and play; one per export, shared by the modes on it",
"namedBy": "modes.<mode>.models.onnx"
},
{
"path": "^models/[a-z][a-z0-9_]*\\.rknn$",
"required": false,
"format": "rknn",
"is": "the same policy converted for the RK3576 NPU; a mode without one runs on a stub on the board",
"namedBy": "modes.<mode>.models.rknn"
},
{
"path": "^runtime/web/controller\\.wasm$",
"required": true,
"format": "wasm",
"is": "the controller for a browser; one build for every CPU",
"namedBy": "runtimes.web.wasm"
},
{
"path": "^runtime/web/controller\\.js$",
"required": true,
"format": "javascript",
"is": "its wasm-bindgen binding",
"namedBy": "runtimes.web.glue"
},
{
"path": "^runtime/mjlab/macosx-[a-z0-9._-]+/controller\\.so$",
"required": false,
"format": "mach-o",
"is": "the same for macOS: macosx-universal2 is cross-built with zig, arm64 and x86_64 in one file, and loaded on any Mac whatever version its Python's platform string carries; a Mac that built the bundle adds its own under its exact platform",
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
},
{
"path": "^runtime/mjlab/[a-z0-9._-]+/controller\\.so$",
"required": false,
"format": "elf-shared-object",
"is": "the controller as a Python extension (abi3) for play --app, one per platform it was built on",
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
},
{
"path": "^runtime/mjlab/win-[a-z0-9_]+/controller\\.pyd$",
"required": false,
"format": "pe-dll",
"is": "the same, for play --app on Windows: cross-built with MinGW, importing python3.dll and nothing outside Windows itself",
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
},
{
"path": "^runtime/board/controller$",
"required": false,
"format": "elf-aarch64",
"is": "the robot's whole program, cross-compiled; absent until deploy/fsm/docker-build.sh has run, which bundle.json's notes then say",
"namedBy": "runtimes.board.file"
}
],
"$defs": {
"mode": {
"type": "object",
"required": [
"models",
"contract",
"observation",
"action"
],
"additionalProperties": false,
"properties": {
"models": {
"description": "The mode's policy in every format a host runs: `onnx` always, `rknn` once converted.",
"type": "object",
"required": [
"onnx"
],
"additionalProperties": false,
"properties": {
"onnx": {
"type": "string",
"pattern": "^models/[a-z][a-z0-9_]*\\.onnx$"
},
"rknn": {
"type": "string",
"pattern": "^models/[a-z][a-z0-9_]*\\.rknn$"
}
}
},
"contract": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*\\.json$"
},
"observation": {
"type": "integer"
},
"action": {
"type": "integer"
},
"trajectory": {
"type": "string",
"pattern": "\\.trajectory\\.json$"
}
}
},
"webRuntime": {
"type": "object",
"required": [
"model",
"glue",
"wasm",
"commit",
"unverifiedByTheConsumer"
],
"properties": {
"model": {
"const": "onnx"
},
"glue": {
"const": "runtime/web/controller.js"
},
"wasm": {
"const": "runtime/web/controller.wasm"
},
"commit": {
"type": "string"
},
"unverifiedByTheConsumer": {
"type": "string"
}
}
},
"mjlabRuntime": {
"type": "object",
"required": [
"model",
"extensions",
"commit"
],
"properties": {
"model": {
"const": "onnx"
},
"extensions": {
"description": "By `sysconfig.get_platform()` of the machine it runs on: the build machine's, and `win-amd64` and `macosx-universal2`, cross-built. macOS is keyed without a version (`mjrl.app_play.extension_for`).",
"type": "object",
"minProperties": 1,
"additionalProperties": {
"type": "object",
"required": [
"file",
"abi"
],
"properties": {
"file": {
"type": "string",
"pattern": "^runtime/mjlab/[a-z0-9._-]+/controller\\.(so|pyd)$"
},
"abi": {
"type": "string"
}
}
}
},
"commit": {
"type": "string"
}
}
},
"boardRuntime": {
"type": "object",
"required": [
"model",
"file",
"platform",
"commit"
],
"properties": {
"model": {
"const": "rknn"
},
"file": {
"const": "runtime/board/controller"
},
"platform": {
"const": "aarch64-unknown-linux-gnu"
},
"commit": {
"type": "string"
}
}
},
"manual": {
"description": "manual.<language>.json. A translation keeps every field of manual.en.json and changes only the text ones: source, summary, enter, leave, keyboard, pad, does. The pad and the keyboard are listed apart, in each mode and among the switches (since 2026-09-29).",
"type": "object",
"required": [
"schema",
"language",
"bundle",
"source",
"modes",
"switches"
],
"additionalProperties": false,
"properties": {
"schema": {
"const": "kk-bundle-manual/1"
},
"language": {
"type": "string",
"pattern": "^[a-z]{2}(-[A-Za-z]+)?$"
},
"bundle": {
"type": "string"
},
"source": {
"type": "string"
},
"modes": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": [
"mode",
"task",
"default",
"summary",
"enter",
"leave",
"keyboard",
"controls",
"keys"
],
"additionalProperties": false,
"properties": {
"mode": {
"type": "string"
},
"task": {
"type": [
"string",
"null"
]
},
"default": {
"type": "boolean"
},
"summary": {
"type": "string"
},
"enter": {
"type": "string"
},
"leave": {
"type": "string"
},
"keyboard": {
"type": "string"
},
"controls": {
"description": "The pad in this mode: each control and what it does.",
"type": "array",
"items": {
"type": "object",
"required": [
"control",
"pad",
"does"
],
"additionalProperties": false,
"properties": {
"control": {
"type": "string"
},
"pad": {
"type": "string"
},
"does": {
"type": "string"
}
}
}
},
"keys": {
"description": "The keyboard in this mode: each keystroke, as printed on the keys, and what it does.",
"type": "array",
"items": {
"type": "object",
"required": [
"key",
"does"
],
"additionalProperties": false,
"properties": {
"key": {
"type": "string"
},
"does": {
"type": "string"
}
}
}
}
}
}
},
"switches": {
"type": "array",
"items": {
"type": "object",
"required": [
"device",
"control",
"with",
"keys",
"pad",
"does",
"from"
],
"additionalProperties": false,
"properties": {
"device": {
"enum": [
"pad",
"keyboard"
]
},
"control": {
"type": "string"
},
"with": {
"type": [
"string",
"null"
]
},
"keys": {
"type": "array",
"items": {
"type": "string"
}
},
"pad": {
"type": [
"string",
"null"
]
},
"does": {
"type": "string"
},
"from": {
"type": [
"array",
"null"
],
"items": {
"type": "string"
}
}
}
}
}
}
},
"policyBundle": {
"type": "object",
"required": [
"schema",
"runtimes",
"fsm",
"modes",
"notes",
"files",
"manuals",
"reference"
],
"additionalProperties": false,
"properties": {
"schema": {
"const": "kk-policy-bundle/1"
},
"runtimes": {
"description": "Each host's build of the controller, keyed by host. A host whose build this bundle does not carry has no entry, and `notes` says how to add it.",
"type": "object",
"required": [
"web"
],
"additionalProperties": false,
"properties": {
"web": {
"$ref": "#/$defs/webRuntime"
},
"mjlab": {
"$ref": "#/$defs/mjlabRuntime"
},
"board": {
"$ref": "#/$defs/boardRuntime"
}
}
},
"fsm": {
"const": "controller.toml"
},
"modes": {
"description": "Every state that runs a model, by the name controller.toml gives it.",
"type": "object",
"minProperties": 1,
"patternProperties": {
"^[a-z][a-z0-9_]*$": {
"$ref": "#/$defs/mode"
}
},
"additionalProperties": false
},
"notes": {
"type": "array",
"items": {
"type": "string"
}
},
"reference": {
"description": "The reference vectors' file, or null when none could be recorded.",
"type": [
"string",
"null"
]
},
"manuals": {
"description": "The manual per language. English is always there; a translation is added after the build.",
"type": "object",
"required": [
"en"
],
"patternProperties": {
"^[a-z]{2}(-[A-Za-z]+)?$": {
"type": "string",
"pattern": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$"
}
},
"additionalProperties": false
},
"built_from": {
"description": "Where every mode came from, for a bundle built from deploy/manifests.json.",
"type": "object",
"additionalProperties": {
"type": "object",
"required": [
"task",
"from",
"export"
],
"properties": {
"task": {
"type": "string"
},
"from": {
"type": "string"
},
"export": {
"type": "string"
}
}
}
},
"files": {
"description": "Every file in the bundle but bundle.json itself, by its path inside it.",
"type": "object",
"minProperties": 1,
"additionalProperties": {
"type": "object",
"required": [
"bytes",
"sha256"
],
"additionalProperties": false,
"properties": {
"bytes": {
"type": "integer"
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
}
}
},
"robotApp": {
"type": "object",
"required": [
"schema",
"robot_id",
"model",
"default_mode",
"modes",
"controller",
"manuals",
"reference",
"verification",
"runtimes",
"capabilities",
"source",
"files"
],
"additionalProperties": false,
"properties": {
"schema": {
"const": "mjrl-robot-app/1"
},
"robot_id": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*$"
},
"model": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
},
"default_mode": {
"type": "string"
},
"modes": {
"type": "object",
"minProperties": 1,
"patternProperties": {
"^[a-z][a-z0-9_]*$": {
"type": "object",
"required": [
"control_mode",
"contract"
],
"additionalProperties": false,
"properties": {
"control_mode": {
"enum": [
"rl_onnx",
"ik_mocap",
"wheeled_velocity",
"cpg_keyframe",
"model_viewer"
]
},
"contract": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
},
"controls": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
},
"policy": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
},
"layout": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
},
"verification": {
"type": "string",
"minLength": 1,
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
}
},
"allOf": [
{
"if": {
"properties": {
"control_mode": {
"const": "rl_onnx"
}
}
},
"then": {
"required": [
"policy",
"layout"
]
}
},
{
"if": {
"properties": {
"control_mode": {
"not": {
"const": "model_viewer"
}
}
}
},
"then": {
"required": [
"verification"
]
}
}
]
}
},
"additionalProperties": false
},
"controller": {
"const": "controller.toml"
},
"manuals": {
"description": "The manual per language. English is always there; a translation is added after the build.",
"type": "object",
"required": [
"en"
],
"patternProperties": {
"^[a-z]{2}(-[A-Za-z]+)?$": {
"type": "string",
"pattern": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$"
}
},
"additionalProperties": false
},
"reference": {
"const": "reference.json"
},
"verification": {
"enum": [
"verification.json",
null
]
},
"runtimes": {
"type": "object",
"required": [
"native"
],
"additionalProperties": false,
"properties": {
"native": {
"type": "object",
"required": [
"file",
"language",
"python",
"dependencies",
"platforms"
],
"additionalProperties": false,
"properties": {
"file": {
"const": "runtime/controller.pyz"
},
"language": {
"const": "python"
},
"python": {
"type": "string"
},
"dependencies": {
"const": "runtime/requirements.txt"
},
"platforms": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {
"enum": [
"linux",
"windows",
"macos"
]
}
}
}
}
}
},
"capabilities": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {
"enum": [
"rl_onnx",
"ik_mocap",
"wheeled_velocity",
"cpg_keyframe",
"model_viewer"
]
}
},
"source": {
"type": "object",
"required": [
"model_sha256"
],
"additionalProperties": false,
"properties": {
"model_sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
},
"files": {
"description": "Every file in the bundle but bundle.json itself, by its path inside it.",
"type": "object",
"minProperties": 1,
"additionalProperties": {
"type": "object",
"required": [
"bytes",
"sha256"
],
"additionalProperties": false,
"properties": {
"bytes": {
"type": "integer"
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
},
"licenses": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {
"type": "string"
},
"description": "Inventoried platform and explicitly supplied asset/policy license and attribution files."
}
}
}
},
"oneOf": [
{
"$ref": "#/$defs/policyBundle"
},
{
"$ref": "#/$defs/robotApp"
}
]
}
+45
View File
@@ -0,0 +1,45 @@
# Conversion image: actor.onnx -> actor.rknn, on a host of any kind.
#
# rknn-toolkit2 publishes Linux wheels only (x86_64 and aarch64, no macOS, no
# Windows), so on anything else the toolkit cannot be installed at all. This
# image is the way those machines convert: Docker Desktop on macOS or Windows
# runs a Linux VM, and the build below produces an image for whatever
# architecture the host runs -- arm64 on Apple Silicon, amd64 elsewhere. Both are
# architectures the toolkit ships for, so the same Dockerfile serves both without
# a `--platform` flag.
#
# Build and run through `docker-convert.sh`, which handles the mount and the file
# ownership; see README.md.
# Debian rather than Alpine: the rknn-toolkit2 wheels are manylinux_2_17, which
# is glibc. On musl they install and then fail to load their .so.
FROM python:3.12-slim
# rknn-toolkit2 declares opencv-python, which links libGL and libglib. Neither is
# in a slim image, and without them `import rknn.api` dies on a missing
# libGL.so.1 -- an error that reads like a broken package rather than a missing
# system library. ~35 MB, and it is the whole reason this is not a one-line
# Dockerfile.
RUN apt-get update \
&& apt-get install -y --no-install-recommends libgl1 libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/convert
# Requirements first, so editing the converter does not re-download ~1.5 GB of
# wheels. The torch pin inside is selected by `platform_machine`, which is what
# lets one file serve both architectures.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY onnx2rknn.py .
# Bundles are mounted here; see docker-convert.sh. Left as the working directory
# so that a relative bundle path on the command line means the same thing inside
# the container as it did outside.
WORKDIR /work
# HOME is set at run time to a writable directory: the container runs as the
# calling user (so output files are not root-owned), and that user has no home
# inside the image. Several libraries write caches to ~ and fail without it.
ENTRYPOINT ["python", "/opt/convert/onnx2rknn.py"]
+175
View File
@@ -0,0 +1,175 @@
# convert — ONNX to RKNN for the RK3576 NPU
The last link in the chain that starts at a checkpoint:
```
model_*.pt ──scripts/export.py──▶ actor.onnx ──deploy/convert/onnx2rknn.py──▶ actor.rknn
layout.json actor.rknn.json
```
`scripts/export.py` writes an exported policy and checks the ONNX against the torch
policy. This directory converts that ONNX for the Rockchip NPU and checks the result
against the ONNX. The two checks together pin the whole chain: a policy that behaves on
the board the way it behaved in training, or a conversion that fails loudly.
## Setup
Two ways in. Docker works everywhere and is the only option off Linux; the virtualenv is
lighter if you are already on Linux.
### Docker — any host
```bash
bash deploy/convert/docker-convert.sh --bundle out/exports/walk
```
Builds the image on first use, then takes the same arguments as `onnx2rknn.py` and passes
them through. `MJRL_RKNN_REBUILD=1` forces a rebuild after editing the converter.
rknn-toolkit2 publishes **Linux wheels only** — x86_64 and aarch64, no macOS, no Windows —
so on a Mac or a Windows box this is not a convenience, it is the only way to convert.
Docker Desktop runs a Linux VM and [`Dockerfile`](Dockerfile) builds for whatever
architecture the host is, arm64 on Apple Silicon and amd64 elsewhere. Both are
architectures the toolkit ships for, so no `--platform` flag is needed.
The wrapper exists because three details are each a wrong answer waiting to happen: the
repository is mounted rather than the bundle (so paths mean the same inside and out), the
container runs as the calling user (or the `.rknn` lands in your tree owned by root), and
`HOME` is redirected (that user has no home in the image, and dependencies write caches
to `~`).
### Virtualenv — Linux only
```bash
bash deploy/convert/setup.sh
```
Builds `deploy/convert/.venv` (~1.5 GB) and verifies the toolkit imports. Python
3.8 – 3.12.
**It is a separate virtualenv on purpose.** rknn-toolkit2 pins `numpy<=1.26.4`,
`torch<=2.4.0`, `onnx==1.16.1` and `setuptools<81`; the training environment is on torch
2.11+cu128, numpy 2.5 and onnx 1.22. Installing the toolkit there downgrades all of them —
including replacing the CUDA build of torch with a CPU one — and pip reports success. The
damage shows up the next time something imports mjlab. Never `pip install rknn-toolkit2`
into the training environment.
[`requirements.txt`](requirements.txt) records why every pin is there; four of them are
bugs or gaps in rknn-toolkit2's own metadata and each was found by a fresh install
failing.
## Converting
```bash
deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py \
--bundle out/exports/walk
```
Writes `actor.rknn` and `actor.rknn.json` beside the bundle's `actor.onnx`. Real output,
from the run the next section measures:
```
[rknn] source .../actor.onnx [1, 74] -> [1, 20]
[rknn] bundle consistent obs=74 act=20
[rknn] test observations drawn from the normaliser baked into the graph (std 0.039 .. 2.623)
[rknn] building target=rk3576 fp16
[rknn] agreement with onnxruntime over 64 observations: max 1.094e-03, mean 2.319e-04,
against outputs up to 1.995
[rknn] done -> .../actor.rknn
```
Useful switches — `--onnx FILE` for a bare ONNX with no contract beside it, `--target`
for another Rockchip part, `--out`, `--samples`, `--seed`, `--tolerance`, `--verbose` for
the toolkit's own log. `--quantize` exists, and `--calib-obs` feeds it a `[N, obs_dim]`
`.npy` of recorded observations instead of sampled ones; read the next section before
reaching for either.
## Why the default is fp16 and not int8
A historical 20-action locomotion actor (74 → 512 → 256 → 128 → 20, ELU)
was tested on an i9-14900KF with rknn-toolkit2 2.3.2 and RK3576. The test
used 64 in-distribution observations and converted action error to joint
angle using that actor's `action_scale = 0.25`.
| build | max abs difference from ONNX | joint angle |
|----------------------|------------------------------|-------------|
| fp16 (default) | 1.1e-3 | 0.016° |
| int8 (`--quantize`) | 2.7e-1 | 3.9° |
The maximum int8 error in this experiment was 13% of the approximately
2-unit output magnitude. This result motivated the fp16 default and the
default tolerance of 2e-2. Verify each actor with its own observations and
acceptance tolerance.
## What the check actually checks
The converted model is run in the toolkit's simulator against onnxruntime on the same
observations, and the outputs compared element-wise. It is built to one side and moved
into place only after it passes, so **a failed check leaves no `.rknn` on disk** — there
is nothing for someone to pick up later without the scrollback that condemned it. Three
more things are worth knowing.
**The observations come from the policy's own training distribution.** The exported
actor carries its `EmpiricalNormalization` inside the graph — the leading `Sub`/`Div`
pair, whose constants are the observation mean and standard deviation measured during
training. `onnx2rknn.py` reads them and samples `mean + std * N(0,1)`. Feeding raw
`N(0,1)` instead puts terms whose training standard deviation is 0.039 about 25σ from
their mean, the activations blow up, and fp16 error grows with them: 5.0e-3 rather than
1.1e-3. That is measuring the test inputs, not the conversion.
**The simulator is not the board.** It models the NPU's arithmetic closely enough to
catch a conversion that changed the graph, which is what this check is for. It is not a
substitute for running the policy once on real hardware before trusting it.
The toolkit prints two warnings during a good conversion, both expected:
- `The config.mean_values is None, zeros will be set for input 0` — correct. The
normalisation is already inside the graph; setting `mean_values` would apply it twice.
This warning is the confirmation that it is not.
- `The 'data_format' is not set, and its default value is 'nhwc'` — harmless for a
`[1, obs_dim]` input, which has no spatial layout to reorder. The 1.1e-3 agreement is
the evidence: a permuted input would not land within 1e-3 of the ONNX.
## On the board
The `.rknn` is loaded by `librknnrt.so`, which is part of the board image rather than
this repository. **It must be at least as new as the toolkit that built the model** —
2.3.2 here. A model built by a newer toolkit than the runtime fails to load, usually
with an unhelpful error:
```bash
strings /usr/lib/librknnrt.so | grep -i 'librknnrt version'
```
Inference itself is `rknn_init` → `rknn_inputs_set` → `rknn_run` → `rknn_outputs_get`
from the C API, or `RKNNLite` from `rknn-toolkit-lite2` in Python. Feed the observation
as a flat `float32` vector of length `observation.dim`, in the order defined by
`layout.json`. Apply the returned action as
the bundle's own `README.md` says — `target = action * action_scale +
default_joint_pos[joint]`, paired joint by joint in `action_joint_order`. Nothing about
the conversion changes that contract.
`actor.rknn.json` records what produced the model: the source ONNX's sha256, the toolkit
version, the target, and the verification numbers. It answers "is this `.rknn` the one
built from that `.onnx`?" without relying on the two files having stayed in the same
directory.
**It records the ONNX's checksum rather than the `.rknn`'s, because the `.rknn` does not
have a stable one.** Converting the same ONNX twice on the same machine, back to back,
produces files of identical length that differ in ~14 kB — always the same byte range,
filled with what look like leaked heap pointers, i.e. uninitialised padding the toolkit
serialises. Both builds reported the same agreement with onnxruntime to four figures, so
the models are equivalent; only the bytes are not. The container and the virtualenv differ
from each other in exactly the same way and for the same reason, so a byte difference
between them means nothing either.
The practical consequence: **do not checksum a `.rknn` to decide whether to redeploy.**
It changes every build. Compare `onnx_sha256` in the sidecar instead.
## Reconverting
`.rknn` files are build products and are not committed (`.gitignore` covers `*.rknn`
alongside `*.onnx`). Reconvert whenever `actor.onnx` changes. `scripts/deploy.py` does
it for you: it reconverts any `.rknn` whose sidecar `onnx_sha256` no longer matches before
bundling. Nothing on the board checks, so a `.rknn` copied there by hand beside a fresh
`.onnx` is still silent.
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
# Run the ONNX -> RKNN conversion inside a container, from any host.
#
# Takes the same arguments as onnx2rknn.py and passes them straight through:
#
# bash deploy/convert/docker-convert.sh --bundle out/exports/walk
# bash deploy/convert/docker-convert.sh --bundle <dir> --target rk3588
#
# The image is built on first use and reused after that. Force a rebuild after
# editing the converter or its requirements:
#
# MJRL_RKNN_REBUILD=1 bash deploy/convert/docker-convert.sh --bundle <dir>
#
# Three details this wraps, each of which is a wrong answer waiting to happen:
#
# - **The repository is mounted, not the bundle.** Bundle paths are given
# relative to the repository root, and mounting only the bundle would break
# every path someone copies out of the README.
# - **The container runs as the calling user.** Docker's default is root, and
# the .rknn it writes would land in the tree owned by root -- on Linux the
# next export, running as you, then cannot overwrite it.
# - **HOME is redirected to /tmp.** That user has no home inside the image, and
# several of the toolkit's dependencies write caches to ~ on import.
#
# On macOS and Windows this is the only way to convert: rknn-toolkit2 has no
# wheel for either. On Linux it works too, but `setup.sh` is lighter.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd "$HERE/../.." && pwd)"
# Tagged with the toolkit version, because that version has to match the board's
# librknnrt.so: two images side by side should be distinguishable at a glance.
TOOLKIT="$(sed -n 's/^rknn-toolkit2==\(.*\)$/\1/p' "$HERE/requirements.txt" | head -1)"
IMAGE="mjrl-rknn:${TOOLKIT:-latest}"
if ! command -v docker >/dev/null 2>&1; then
echo "docker not found. Install Docker Desktop (macOS / Windows) or docker.io (Linux)." >&2
exit 1
fi
if [ -n "${MJRL_RKNN_REBUILD:-}" ] || ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
echo "[docker] building $IMAGE (first run pulls ~1.5 GB of wheels)"
docker build -t "$IMAGE" "$HERE"
fi
# --user keeps output files owned by the caller. On macOS and Windows the bind
# mount already maps ownership and the flag is harmless.
exec docker run --rm -i \
--user "$(id -u):$(id -g)" \
--env HOME=/tmp \
--volume "$REPO:/work" \
--workdir /work \
"$IMAGE" "$@"
+491
View File
@@ -0,0 +1,491 @@
#!/usr/bin/env python3
"""Convert an exported `actor.onnx` into an `.rknn` for the RK3576 NPU.
`scripts/export.py` produces an exported policy -- `actor.onnx` plus
`layout.json`. This takes the ONNX out of that bundle, converts it for the
Rockchip NPU, and **checks the result numerically** before calling it done.
Run it with this directory's own interpreter, never the repository's:
deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py \
--bundle out/exports/walk
rknn-toolkit2 pins `numpy<=1.26.4`, `torch<=2.4.0` and `onnx==1.16.1`.
Installing it into the training environment downgrades all three, so it gets a
virtualenv of its own; `setup.sh` builds it.
## The two silent failures this guards against
**The RKNN API returns error codes, it does not raise.** `build()` answering -1
and the script carrying on leaves the previous `.rknn` on disk -- or no file at
all -- and exits 0. Every toolkit call here is checked.
**A conversion can succeed and still be wrong.** An op fused into something with
different arithmetic, a layout the toolkit guessed, quantization left on: all of
these produce a valid `.rknn` that behaves differently from the ONNX. So the
converted model is run in the toolkit's simulator against onnxruntime on the same
observations and the outputs are compared element-wise. This mirrors
`scripts/export.py::_validate_onnx`, which does the same one link earlier in the
chain; together the two pin torch -> ONNX -> RKNN.
## Which observations the check is run on
The exported actor carries its `EmpiricalNormalization` inside the graph -- the
leading `Sub` / `Div` pair, whose constants are the observation mean and standard
deviation measured during training. This script reads them and draws test
observations from `mean + std * N(0, 1)`, i.e. from the distribution the policy
actually saw.
That is not cosmetic. Feeding raw `N(0, 1)` instead puts terms whose training
standard deviation is 0.039 about 25 sigma from their mean; the activations blow
up and fp16 rounding error grows with them, so the comparison measures the test
inputs rather than the conversion. Measured on the historical actor below: 5.0e-3 max
absolute difference on raw `N(0, 1)`, 1.1e-3 on in-distribution observations.
If the graph does not start with that `Sub` / `Div` pair the script says so and
falls back to `N(0, 1)`, rather than pretending the check is as tight.
## Measurements
A historical locomotion actor (74 -> 512 -> 256 -> 128 -> 20, ELU) was
tested on an i9-14900KF with rknn-toolkit2 2.3.2 and RK3576. The test used
64 in-distribution observations. The last column converts action error
to joint angle using the actor's `action_scale = 0.25`.
| build | max abs difference from ONNX | joint angle |
|-----------------------------|------------------------------|-------------|
| fp16 (the default) | 1.1e-3 | 0.016 deg |
| int8 (`--quantize`) | 2.7e-1 | 3.9 deg |
The int8 error in this experiment was 13% of the approximately 2-unit
output magnitude. This result motivated the fp16 default and the tolerance
of 2e-2. Verify each actor with its own observation distribution.
"""
from __future__ import annotations
import argparse
import hashlib
import importlib.util
import json
import platform
import tempfile
from datetime import datetime, timezone
from pathlib import Path
#: Default agreement required between the RKNN simulator and onnxruntime.
#: Chosen to sit an order of magnitude above the fp16 floor (1.1e-3, measured
#: above) and an order of magnitude below the int8 error (2.7e-1), so it passes
#: rounding noise and fails anything that changed the arithmetic.
DEFAULT_TOLERANCE = 2e-2
#: Calibration samples for `--quantize`. rknn's own examples use a few hundred.
CALIBRATION_SAMPLES = 256
# ──────────────────────────────────────────────────────────────────────
# The host has to be right before anything else is worth trying
# ──────────────────────────────────────────────────────────────────────
def _require_host() -> None:
"""rknn-toolkit2 ships Linux x86_64 wheels only.
Said plainly here because the alternative is an ImportError from a
`.so` several frames deep, which reads like a broken install rather than an
unsupported machine.
"""
if platform.system() != "Linux" or platform.machine() not in ("x86_64", "aarch64"):
raise SystemExit(
f"[rknn] rknn-toolkit2 runs on Linux x86_64 (or aarch64); this is "
f"{platform.system()} {platform.machine()}. Convert on a Linux PC and "
f"copy the .rknn to the board."
)
def _shown(path: Path) -> str:
"""A path as the reader would type it: relative to where they are, if it can be.
The commands in the message below are meant to be copied, and this file sits
two directories down. Naming the directory (`convert/setup.sh`) is wrong from
anywhere, and an absolute path is right but unreadable; relative to the
working directory is what someone standing at the repository root can paste.
"""
try:
return str(path.relative_to(Path.cwd()))
except ValueError:
return str(path)
def _require_toolkit():
"""Import the toolkit, or say which interpreter to use instead.
The likeliest way to reach this is running the script with the repository's
`.venv`, where rknn-toolkit2 is deliberately absent.
"""
if importlib.util.find_spec("rknn") is None:
here = Path(__file__).resolve().parent
raise SystemExit(
f"[rknn] rknn-toolkit2 is not installed in this interpreter.\n"
f" It must not be installed into the training environment -- it "
f"pins numpy<=1.26.4, torch<=2.4.0 and onnx==1.16.1.\n"
f" Build its own environment: bash {_shown(here / 'setup.sh')}\n"
f" Then run: {_shown(here / '.venv/bin/python')} "
f"{_shown(Path(__file__).resolve())} ..."
)
from rknn.api import RKNN
return RKNN
def _check(ret: int, what: str) -> None:
"""Every RKNN call returns a status code instead of raising."""
if ret != 0:
raise SystemExit(f"[rknn] {what} failed (rknn returned {ret}); see the log above")
# ──────────────────────────────────────────────────────────────────────
# Reading what the ONNX and the contract say about themselves
# ──────────────────────────────────────────────────────────────────────
def _io_shapes(model) -> tuple[list[int], list[int]]:
def dims(value):
return [d.dim_value for d in value.type.tensor_type.shape.dim]
return dims(model.graph.input[0]), dims(model.graph.output[0])
def _observation_stats(model, obs_dim: int):
"""The observation mean and standard deviation baked into the graph head.
`rsl_rl`'s `_OnnxMLPModel.forward` runs the `EmpiricalNormalization` before
the MLP, so an exported actor starts with `Sub(obs, mean)` then
`Div(_, std)`. Only the first two nodes are examined: a `Sub` further in is
part of the network and its constants are not observation statistics.
"""
import numpy as np
from onnx import numpy_helper
nodes = list(model.graph.node)
if len(nodes) < 2 or nodes[0].op_type != "Sub" or nodes[1].op_type != "Div":
return None
init = {i.name: numpy_helper.to_array(i) for i in model.graph.initializer}
mean, std = init.get(nodes[0].input[1]), init.get(nodes[1].input[1])
if mean is None or std is None or mean.size != obs_dim or std.size != obs_dim:
return None
return np.asarray(mean, np.float32).reshape(1, obs_dim), np.asarray(
std, np.float32
).reshape(1, obs_dim)
def _sample_observations(n: int, obs_dim: int, stats, seed: int):
import numpy as np
rng = np.random.default_rng(seed)
if stats is None:
return rng.normal(0.0, 1.0, (n, obs_dim)).astype(np.float32)
mean, std = stats
return (mean + std * rng.normal(0.0, 1.0, (n, obs_dim))).astype(np.float32)
def _cross_check_contract(contract: dict, ishape: list[int], oshape: list[int]) -> None:
"""The bundle's contract and its ONNX must agree.
They come out of the same run of `scripts/export.py`, so disagreement means
the directory holds files from two different exports -- the one case where
converting the ONNX would produce a model that does not match the joint order
and gains the deployment will use.
"""
want_obs = contract["observation"]["dim"]
want_act = contract["action"]["dim"]
if ishape[-1] != want_obs or oshape[-1] != want_act:
raise SystemExit(
f"[rknn] the bundle is inconsistent: the layout says "
f"obs={want_obs} act={want_act}, actor.onnx is {ishape} -> {oshape}. "
f"Re-export the bundle."
)
print(f"[rknn] bundle consistent obs={want_obs} act={want_act}")
# ──────────────────────────────────────────────────────────────────────
# Conversion
# ──────────────────────────────────────────────────────────────────────
def _write_calibration(observations, directory: Path) -> Path:
"""rknn takes calibration data as a text file of per-sample file paths."""
import numpy as np
lines = []
for i, obs in enumerate(observations):
path = directory / f"obs_{i:04d}.npy"
np.save(path, obs.reshape(1, -1))
lines.append(str(path))
listing = directory / "dataset.txt"
listing.write_text("\n".join(lines), encoding="utf-8")
return listing
def convert(onnx_path: Path, out_path: Path, target: str, quantize: bool,
calibration, verbose: bool):
"""Build the `.rknn` and return an initialised simulator session."""
RKNN = _require_toolkit()
rknn = RKNN(verbose=verbose)
# mean_values / std_values are deliberately left unset. The normalisation is
# already inside the graph (see the module docstring); setting them here
# would apply it a second time, and the toolkit's "zeros will be set for
# input 0" warning is the confirmation that it is not being applied twice.
rknn.config(target_platform=target)
_check(rknn.load_onnx(model=str(onnx_path)), f"load_onnx({onnx_path})")
with tempfile.TemporaryDirectory(prefix="rknn-calib-") as tmp:
dataset = str(_write_calibration(calibration, Path(tmp))) if quantize else None
_check(rknn.build(do_quantization=quantize, dataset=dataset), "build")
_check(rknn.export_rknn(str(out_path)), f"export_rknn({out_path})")
_check(rknn.init_runtime(), "init_runtime (simulator)")
return rknn
def verify(rknn, onnx_path: Path, observations) -> tuple[float, float, float]:
"""Compare the converted model against onnxruntime, observation by observation.
Returns (max abs difference, mean abs difference, max abs ONNX output). The
third is what makes the first two readable: a 1e-3 difference means one thing
against outputs of order 2 and another against outputs of order 200.
"""
import numpy as np
import onnxruntime as ort
session = ort.InferenceSession(str(onnx_path), providers=["CPUExecutionProvider"])
name = session.get_inputs()[0].name
worst, total, magnitude = 0.0, 0.0, 0.0
for obs in observations:
x = obs.reshape(1, -1)
y_onnx = session.run(None, {name: x})[0]
y_rknn = np.asarray(rknn.inference(inputs=[x])[0]).reshape(y_onnx.shape)
diff = np.abs(y_rknn - y_onnx)
worst = max(worst, float(diff.max()))
total += float(diff.mean())
magnitude = max(magnitude, float(np.abs(y_onnx).max()))
return worst, total / len(observations), magnitude
def _sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
# ──────────────────────────────────────────────────────────────────────
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="onnx2rknn.py",
description=(__doc__ or "").split("\n\n")[0],
formatter_class=argparse.RawDescriptionHelpFormatter,
)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument(
"--bundle", type=Path,
help="an exported policy from scripts/export.py (actor.onnx + layout.json)",
)
source.add_argument("--onnx", type=Path, help="a bare .onnx file, no contract to check against")
parser.add_argument(
"--out", type=Path, default=None,
help="output .rknn; defaults to <bundle>/actor.rknn, or <onnx>.rknn",
)
parser.add_argument(
"--target", default="rk3576",
help="target platform, validated by rknn-toolkit2 (default: rk3576)",
)
parser.add_argument(
"--samples", type=int, default=64, help="observations used for the check (default: 64)"
)
parser.add_argument("--seed", type=int, default=0, help="seed for those observations")
parser.add_argument(
"--tolerance", type=float, default=DEFAULT_TOLERANCE,
help=f"max allowed difference from onnxruntime (default: {DEFAULT_TOLERANCE})",
)
parser.add_argument(
"--quantize", action="store_true",
help="build int8 instead of fp16. Measured at a 13%% error on an actor MLP; "
"read the module docstring before using it",
)
parser.add_argument(
"--calib-obs", type=Path, default=None,
help="[N, obs_dim] float32 .npy of recorded observations for --quantize; "
"without it, calibration is drawn from the baked-in normaliser",
)
parser.add_argument("--verbose", action="store_true", help="the toolkit's own verbose log")
return parser.parse_args()
def main() -> None:
args = _parse_args()
if args.calib_obs is not None and not args.quantize:
# Silently ignoring it would read as "calibration data was used"
raise SystemExit("[rknn] --calib-obs is calibration data for --quantize, which is off")
_require_host()
_require_toolkit()
import numpy as np
import onnx
if args.bundle is not None:
onnx_path = args.bundle / "actor.onnx"
if not onnx_path.is_file():
raise SystemExit(f"[rknn] no actor.onnx in {args.bundle} -- is that a bundle?")
# `contract.json` is what scripts/export.py wrote before the file took the
# name the on-robot importer looks for. Bundles under that name are still
# readable here; nothing regenerates an old export just to rename it.
contract_path = next(
(p for p in (args.bundle / "layout.json", args.bundle / "isaac_layout.json",
args.bundle / "contract.json")
if p.is_file()),
None,
)
else:
onnx_path = args.onnx
if not onnx_path.is_file():
raise SystemExit(f"[rknn] no such file: {onnx_path}")
contract_path = None
out_path = args.out or onnx_path.with_suffix(".rknn")
out_path.parent.mkdir(parents=True, exist_ok=True)
model = onnx.load(str(onnx_path))
ishape, oshape = _io_shapes(model)
if len(ishape) != 2 or ishape[0] != 1:
raise SystemExit(
f"[rknn] expected a [1, obs_dim] input, got {ishape}. This script converts "
f"the single-observation actor that scripts/export.py writes."
)
obs_dim, act_dim = ishape[-1], oshape[-1]
print(f"[rknn] source {onnx_path} {ishape} -> {oshape}")
contract = None
if contract_path is not None and contract_path.is_file():
contract = json.loads(contract_path.read_text(encoding="utf-8"))
_cross_check_contract(contract, ishape, oshape)
stats = _observation_stats(model, obs_dim)
if stats is None:
print(
"[rknn] WARNING: the graph does not start with the Sub/Div normaliser pair, "
"so the check runs on raw N(0,1) observations. That is a looser check -- "
"out-of-distribution inputs inflate fp16 error (5.0e-3 against 1.1e-3, "
"measured); read the tolerance below with that in mind."
)
else:
print(
f"[rknn] test observations drawn from the normaliser baked into the graph "
f"(std {float(stats[1].min()):.3f} .. {float(stats[1].max()):.3f})"
)
observations = _sample_observations(args.samples, obs_dim, stats, args.seed)
if args.quantize and args.calib_obs is not None:
calibration = np.load(args.calib_obs).astype(np.float32)
if calibration.ndim != 2 or calibration.shape[1] != obs_dim:
raise SystemExit(
f"[rknn] --calib-obs must be [N, {obs_dim}], got {calibration.shape}"
)
print(f"[rknn] calibrating on {len(calibration)} recorded observations")
elif args.quantize:
calibration = _sample_observations(
CALIBRATION_SAMPLES, obs_dim, stats, args.seed + 1
)
print(
f"[rknn] WARNING: calibrating on {CALIBRATION_SAMPLES} sampled observations "
f"rather than recorded ones. Pass --calib-obs to do better."
)
else:
calibration = None
print(f"[rknn] building target={args.target} "
f"{'int8 (quantized)' if args.quantize else 'fp16'}")
# Built to one side and moved into place only once it has been checked. A
# model that failed verification must not be left on disk under the name the
# deployment reads: the next person to look sees a plausible .rknn and a
# scrollback they did not write.
staging = out_path.with_name(out_path.name + ".partial")
try:
rknn = convert(onnx_path, staging, args.target, args.quantize, calibration, args.verbose)
try:
worst, mean, magnitude = verify(rknn, onnx_path, observations)
finally:
rknn.release()
print(
f"[rknn] agreement with onnxruntime over {args.samples} observations: "
f"max {worst:.3e}, mean {mean:.3e}, against outputs up to {magnitude:.3f}"
)
if worst > args.tolerance:
raise SystemExit(
f"[rknn] the converted model disagrees with the ONNX by {worst:.3e}, over "
f"the {args.tolerance:.3e} tolerance. Nothing was written.\n"
+ (
" int8 quantization is the likely cause: measured at 2.7e-1 on "
"an actor MLP, which is 3.9 degrees of joint error. Drop --quantize.\n"
if args.quantize
else " fp16 rounding alone measures 1.1e-3; a difference this "
"large means the conversion changed the arithmetic.\n"
)
)
staging.replace(out_path)
except BaseException:
staging.unlink(missing_ok=True)
raise
meta = {
"_source": "deploy/convert/onnx2rknn.py -- reconvert whenever actor.onnx changes",
"created": datetime.now(timezone.utc).isoformat(timespec="seconds"),
"onnx": str(onnx_path),
# So that "which ONNX is this .rknn from?" has an answer that does not
# depend on the two files having stayed in the same directory
"onnx_sha256": _sha256(onnx_path),
"target_platform": args.target,
"quantized": bool(args.quantize),
"precision": "int8" if args.quantize else "float16",
"toolkit_version": _toolkit_version(),
"obs_dim": obs_dim,
"action_dim": act_dim,
"verification": {
"samples": args.samples,
"seed": args.seed,
"observations": "baked_normalizer" if stats is not None else "standard_normal",
"max_abs_diff": worst,
"mean_abs_diff": mean,
"max_abs_output": magnitude,
"tolerance": args.tolerance,
},
**({"task_contract": {"action_joint_order": contract["action_joint_order"]}}
if contract else {}),
}
meta_path = out_path.with_suffix(out_path.suffix + ".json")
meta_path.write_text(json.dumps(meta, indent=2), encoding="utf-8")
print(f"\n[rknn] done -> {out_path}")
for f in (out_path, meta_path):
print(f" {f.name:<20}{f.stat().st_size / 1024:8.1f} KB")
print(
"\n[rknn] the board's librknnrt.so must be at least as new as the toolkit "
f"({_toolkit_version()}); check it with\n"
" strings /usr/lib/librknnrt.so | grep -i 'librknnrt version'"
)
def _toolkit_version() -> str:
from importlib.metadata import PackageNotFoundError, version
try:
return version("rknn-toolkit2")
except PackageNotFoundError:
return "unknown"
if __name__ == "__main__":
main()
+56
View File
@@ -0,0 +1,56 @@
# The conversion environment. Install into deploy/convert/.venv only -- never into the
# repository's .venv. `setup.sh` does that; this file records why each pin is here.
#
# Linux x86_64 (or aarch64) only: rknn-toolkit2 publishes no macOS or Windows wheel.
# Rockchip's converter, newest release on PyPI at the time of writing and the one
# every measurement in README.md was taken with. Pinned rather than floating
# because the version is coupled to the board: a .rknn built here does not load
# under a librknnrt.so older than the toolkit that produced it.
rknn-toolkit2==2.3.2
# ── Pins rknn-toolkit2 declares wrongly or not at all ──────────────────
#
# onnx: the wheel asks for `onnx>=1.16.1`, which resolves to 1.22 and then dies
# inside `load_onnx` with `AttributeError: module 'onnx' has no attribute
# 'mapping'` -- `onnx.mapping` was removed after 1.16. Measured, not guessed.
onnx==1.16.1
# torch: imported at `import rknn.api` time even though nothing here converts a
# torch model, so it cannot be left out. Taken from PyTorch's CPU index because
# the PyPI wheel drags in ~2.5 GB of CUDA libraries this never calls.
#
# Two lines, because the CPU index names the two architectures differently:
# x86_64 gets `torch-2.4.0+cpu-...x86_64.whl`, aarch64 gets plain
# `torch-2.4.0-...aarch64.whl` with no local version. Pinning `2.4.0+cpu`
# everywhere resolves to nothing on aarch64 -- "no matching distribution", after
# the host check has already said the machine is supported. (The aarch64 build
# has no CUDA variant to be confused with, so the bare version is already the
# CPU one.)
--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.4.0+cpu ; platform_machine == "x86_64"
torch==2.4.0 ; platform_machine == "aarch64"
# setuptools: rknn-toolkit2 imports `pkg_resources`, which setuptools removed in
# 81. Undeclared, so a fresh install resolves to the newest setuptools and then
# fails at `RKNN()` with `ModuleNotFoundError: No module named 'pkg_resources'`.
# Measured on a clean venv here: 84.0.0 fails, 80.10.2 works.
setuptools<81
# The rest are rknn-toolkit2's own declared dependencies, listed so that a
# resolver failure names a version rather than "no matching distribution".
numpy==1.26.4
protobuf>=4.21.6,<=4.25.4
psutil>=5.9.0
ruamel.yaml>=0.17.21
scipy>=1.9.3
tqdm>=4.64.1
fast-histogram>=0.11
# opencv-python needs libGL at import. On a container without it, either
# `apt install libgl1` or swap this line for opencv-python-headless -- but note
# that pip will then install opencv-python anyway to satisfy rknn-toolkit2's own
# metadata, leaving two packages providing `cv2`.
opencv-python>=4.5.5.64
# Used by onnx2rknn.py as the reference the converted model is checked against.
onnxruntime>=1.17.0
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Build deploy/convert/.venv, the conversion environment.
#
# It is a separate virtualenv on purpose. rknn-toolkit2 pins numpy<=1.26.4,
# torch<=2.4.0 and (in practice) onnx==1.16.1; installing it into the
# repository's .venv downgrades all three and breaks training -- silently, since
# pip reports success and the damage only shows up the next time something
# imports mjlab.
#
# Usage:
# bash deploy/convert/setup.sh
#
# Then, from the repository root:
# deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py --bundle <bundle dir>
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV="$HERE/.venv"
os=$(uname -s)
arch=$(uname -m)
if [ "$os" != "Linux" ] || { [ "$arch" != "x86_64" ] && [ "$arch" != "aarch64" ]; }; then
echo "rknn-toolkit2 publishes Linux x86_64 / aarch64 wheels only; this is $os $arch." >&2
echo "Convert on a Linux PC and copy the .rknn to the board." >&2
exit 1
fi
# rknn-toolkit2 2.3.2 has wheels for cp36 through cp312. 3.13 finds nothing and
# pip's message ("no matching distribution") does not say why.
py_minor=$(python3 -c 'import sys; print(sys.version_info[1])')
if [ "$py_minor" -gt 12 ] || [ "$py_minor" -lt 8 ]; then
echo "python3.$py_minor found; rknn-toolkit2 2.3.2 ships wheels for 3.8 - 3.12 only." >&2
echo "Point this script at one of those: python3.12 -m venv $VENV, then pip install -r ..." >&2
exit 1
fi
if [ ! -d "$VENV" ]; then
echo "creating $VENV"
python3 -m venv "$VENV"
fi
"$VENV/bin/pip" install --upgrade pip --quiet
"$VENV/bin/pip" install -r "$HERE/requirements.txt"
echo
echo "--- verifying ---"
"$VENV/bin/python" - <<'PY'
import platform
from importlib.metadata import version
from rknn.api import RKNN
print(f"rknn-toolkit2 {version('rknn-toolkit2')} "
f"onnx {version('onnx')} numpy {version('numpy')} torch {version('torch')}")
print(f"host {platform.system()} {platform.machine()} python {platform.python_version()}")
RKNN(verbose=False).release()
print("ok")
PY
echo
echo "Convert a bundle with:"
echo " deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py --bundle <bundle dir>"
+167
View File
@@ -0,0 +1,167 @@
# dds — the wire interface to the robot
Everything needed to talk to the motor controller and the operator, and nothing else.
Copied from the `mbus` middleware (a CycloneDDS wrapper) rather than reimplemented,
because every file here is one half of a contract whose other half is already running on
the robot.
```
idl/ the message types
idl_common.idl shared enums and headers, included by the rest
idl_motor_control.idl MotorControl::Control (out) and ::State (in)
idl_robot_control.idl RobotControl::Control — the operator's commands, interpreted
idl_robot_control_raw.idl
RobotControlRaw::ControlRaw — the gamepad itself
idl_imu.idl Imu::ImuOutput — orientation and rates
config/
mbus_qos.xml QoS per topic
mbus_config.xml CycloneDDS runtime settings (interfaces, discovery)
topics.json topic name + QoS profile + type, per channel
```
## The channels
| direction | topic | type | QoS profile |
|---|---|---|---|
| subscribe | `RobotControlRaw_ControlRaw` | `RobotControlRaw::ControlRaw` | `RobotControlRaw_ControlRaw` |
| subscribe | `mc/motor_state` | `MotorControl::State` | `MotorControl_State` |
| subscribe | `imu/output` | `Imu::ImuOutput` | `Imu_ImuOutput` |
| **publish** | `mc/motor_control` | `MotorControl::Control` | `MotorControl_Control` |
`robot_control` / `RobotControl::Control` is still here and is on the way out. It is
`control-agent`'s interpretation of the pad — sticks already scaled, buttons already
turned into an action enum — and taking it meant the three signs that turn a stick into
a body-frame command existed a **third** time, inside a program this repository cannot
see, agreed with the other two by nobody. `RobotControlRaw_ControlRaw` is the pad itself,
published by `control-pod-svc` at ~50 Hz, and the mapping comes from the policy's own
contract (`controls.yaml`, exported into `layout.json`) exactly as it does in `play` and
in the browser.
Copied from mbus `master` at `d9875988f065` (2026-09-28), every file from that one
revision; "Provenance" below says what that refresh changed and what it means for the
board.
`topics.json` is mbus's own file, trimmed to the channels in the table above plus
`robot_control`, which leaves when its type does. It is a record, not an input: the
topic name `dds.rs` opens is the one in the table above.
`MotorControl::Control` carries `MotorCmd { pos, kp, dq, kd, tau }` per joint — MIT
impedance control, so the position target and the gains go out together and the joint
torque is closed on the motor controller, not here.
**`imu/output` is not optional.** Until mbus `d9875988f065` the motor controller's
embedded IMU reported no orientation at all. `MotorControl::ImuData` now carries a
quaternion (`q`, with `quat_valid` and `imu_err_code`), but nothing here reads it, so a
controller that skips this topic still has no `projected_gravity` and no tilt angle —
both degrade to "perfectly level", silently. `projected_gravity` is an observation
term for every policy this repository trains.
## Why these files are copies and not descriptions
**The IDL is compiled, not read.** `idlc` turns each `.idl` into a
`dds_topic_descriptor_t`, and that descriptor determines the wire layout and the type hash
DDS matches on. Generating it from the same IDL with the same compiler the robot's side
used makes the two byte-compatible by construction. Retyping the structs in Rust and
hoping they line up is the alternative, and it fails at the first `@extensibility` or
padding difference — as a silently unpaired reader, not an error.
**QoS is matched, not negotiated.** A reader whose reliability or durability disagrees
with the writer's is not an error either: DDS simply never pairs them, and the topic stays
empty forever. `mbus_qos.xml` is therefore verbatim, trimmed to the six profiles here —
`common`, one per channel, and `RobotControl_Control`, which nothing here opens any more
and which has not been dropped yet — and the FSM **loads it at run time** through
CycloneDDS's QoS provider rather than constructing QoS in code — so what this side asks
for is literally the file the robot's side was configured with.
The profiles are not uniform, and one of them matters more than it looks:
| profile | reliability | history |
|---|---|---|
| `MotorControl_Control` / `_State`, `Imu_ImuOutput`, `RobotControlRaw_ControlRaw` | BEST_EFFORT | KEEP_LAST 1 |
| **`RobotControl_Control`** | **RELIABLE** | **KEEP_ALL** |
`RobotControl_Control` is what this table was written for. Those operator commands cross a
wireless link, and `mbus_config.xml` puts them on their own domain with retransmission
tuned for it. A reader built with CycloneDDS's defaults still *pairs* with that writer —
BEST_EFFORT requested against RELIABLE offered is legal — it just silently drops the
reliability the tuning exists to provide. That is why the profiles are loaded rather than
assumed, and why a test asserts what comes back.
The pad this controller actually opens is not on that footing.
`RobotControlRaw_ControlRaw` is BEST_EFFORT / KEEP_LAST 1 like the motor and IMU topics —
a 50 Hz snapshot where the newest sample is the only one worth having — and
`control-pod-svc` publishes it on the same domain 0 the controller defaults to. The
argument for loading the file rather than writing QoS in code is unchanged; which channel
is the expensive one to get wrong has moved.
**Three domains, and the channels are split across them.** `mbus_config.xml` defines
domain 0 (loopback only, no multicast — pure on-board IPC), domain 1 (shared memory
over loopback; its wlan0/eth0 lines have been commented out since mbus `09a57d1`), and
domain 2 (wlan0/eth0, SPDP-only multicast) for cross-machine traffic.
All four channels this FSM uses are on-board; `robot_control`, the one that comes over the
air on domain 2, is the one it stopped reading. The domain ids are configuration, not part
of the topic list — `DdsIo::open` takes one for the motor channels and one for the
operator channel, and `controller` passes 0 for both.
## Provenance
Every file here is from mbus master `d9875988f065` (2026-09-28), refreshed on 2026-09-29.
Before that they had arrived in three takes -- `ea627799ffee` (2026-08-23),
`5a934894bd9a` (2026-09-03) and `d375d737b77f` (2026-09-21) -- and each still matched
its recorded revision byte for byte when the refresh compared them. What the refresh
changed:
| file | change | on the wire |
|---|---|---|
| `idl_motor_control.idl` | `ImuData` gains `imu_temperature`, `q`, `quat_timestamp`, `quat_valid`, `imu_err_code`, `raw_count` | **`State` 800 → 832 bytes**, and it is `@extensibility(FINAL)` |
| `idl_common.idl` | `TopicType` gains `ROBOT_CONTROL_CONTROL_RAW` | none: no channel here carries a `TopicType` |
| `idl_robot_control.idl` | `ActionId` and `GripperPose` renamed after the key that fires them rather than the motion (`ACTION_JUMP` → `ACTION_A`, …), with values added to both | `robot_control` only, which nothing here opens |
| `idl_imu.idl` | adds `Imu::YuanjiOutput` | none: `ImuOutput` is unchanged |
| `mbus_config.xml` | domain 1 binds `lo` only | none: every channel here is on domain 0 |
| `topics.json` | gains `robot_control_raw` | none: `dds.rs` names its own topics |
| `mbus_qos.xml` | not touched: the six profiles kept here compare equal to upstream's | none |
| `idl_robot_control_raw.idl` | unchanged | none |
Quote the revision when refreshing. A copy without one is the thing that drifts; a copy
with one is a claim that can be checked:
```bash
git -C <mbus> diff d9875988f065 -- idl/ config/ # what has changed since
```
**Check this against the robot, not against the repository.** The `libmbus` built
into the kkos staging tree on one development machine dates from 2026-06-24 and
predates the `MotorControl::State` extra-register fields by a month, so its `State`
is a different type from the one here. Which build the board actually runs decides
whether the reader pairs at all -- and a mismatch does not raise, it leaves
`mc/motor_state` empty forever, which reads downstream as "no state, hold position".
The 2026-09-29 refresh is that case again. The `ImuData` fields reached mbus master
only in the 2026-09-28 merge, mbus's CHANGELOG has no release entry for them yet, and
the board last read (`deploy/README.md`) ran libmbus 2.4.0, from 2026-09-02. A
controller built from this tree pairs with an `mc_ctrl_driver` / `mc_forwarder` rebuilt
against `d9875988f065` or later, and with nothing older. `check_board.py` compares the
board's `idl_motor_control.h` with the one generated here and says which it is.
## Regenerating
`idlc` comes with CycloneDDS (`idlc <file>.idl` writes `<file>.c` and `<file>.h`). The
build compiles the generated C in, so nothing here is checked in pre-generated: a stale
descriptor beside a changed IDL is exactly the failure this avoids.
Verified on this repository's side: all five IDL files compile standalone with
`idlc` 0.11, every `topics.json` entry resolves to a QoS profile that is present and a
type that is defined, and a Rust round-trip over `MotorControl::Control` through
`libddsc` returns the sample it wrote, field for field. Rechecked at the 2026-09-29
refresh with `idlc` 11.0.1 on the development machine: `cargo test` in `deploy/fsm`
generates all five and passes, the descriptor-size and QoS tests included, and every
`topics.json` entry still resolves.
## If you are adapting this to your own robot
These are one robot's message definitions. Nothing above is a standard — a different
motor controller means a different `idl_motor_control.idl`, and the FSM's DDS layer is
the only part that should need to change with it. What is worth keeping whatever the
hardware is: the joint order comes from the policy's own `layout.json` and is resolved by
name against the robot's joint list, so the wire order and the policy's action order are
never assumed to agree.
+149
View File
@@ -0,0 +1,149 @@
<?xml version="1.0" encoding="UTF-8"?>
<CycloneDDS xmlns="https://cdds.io/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
<Domain Id="0">
<Compatibility>
</Compatibility>
<General>
<!-- 本域只走 lo:domain 0 是纯本机 IPC,跨机流量一律走 domain 2。
只绑 lo 的附带好处是本域彻底不受弱网影响,也不会把本机 topic
暴露到 wlan0/eth0 上。 -->
<Interfaces>
<NetworkInterface name="lo"/>
</Interfaces>
<!-- lo 在 Linux 上没有 IFF_MULTICAST,组播本来就用不了,如实写 false,
免得看配置的人以为本域还有组播发现这条路。 -->
<AllowMulticast>false</AllowMulticast>
</General>
<Discovery>
<!-- 组播在 lo 上不可用,因此这个 peer 是本域唯一的发现手段(不再是
"wlan0/eth0 没起来时的兜底")。PruneDelay=inf 让本机发现端口被
一直探测:否则晚于 InitialLocatorPruneDelay(30s) 启动的进程,
会因为 peer 已被剪掉而永远发现不了先启动的 participant。 -->
<Peers>
<Peer Address="127.0.0.1" PruneDelay="inf"/>
</Peers>
<LeaseDuration>10s</LeaseDuration>
</Discovery>
<Internal>
<HeartbeatInterval>2s</HeartbeatInterval>
</Internal>
<Sizing>
</Sizing>
<Threads>
</Threads>
<Tracing>
<!-- <Verbosity>finer</Verbosity>
<OutputFile>cyclonedds_${CYCLONEDDS_PID}.log</OutputFile> -->
</Tracing>
</Domain>
<!-- Domain 1: Video streaming — shared memory for local, UDP for remote -->
<Domain Id="1">
<SharedMemory>
<Enable>true</Enable>
<LogLevel>info</LogLevel>
</SharedMemory>
<General>
<Interfaces>
<NetworkInterface name="lo"/>
<!-- <NetworkInterface name="wlan0" presence_required="false"/> -->
<!-- <NetworkInterface name="eth0" presence_required="false"/> -->
</Interfaces>
<AllowMulticast>true</AllowMulticast>
</General>
<Discovery>
<!-- Probe local discovery ports forever: a process started before
wlan0/eth0 is up selects only lo, which has no multicast on Linux;
without a never-pruned localhost peer it can never discover
participants started more than InitialLocatorPruneDelay (30s) later. -->
<LeaseDuration>10s</LeaseDuration>
</Discovery>
<Internal>
<HeartbeatInterval>2s</HeartbeatInterval>
</Internal>
<Tracing>
</Tracing>
</Domain>
<!-- Domain 2: 跨机弱网域(只走 wlan0/eth0,无 lo)。robot_control 跑在这里,
QoS 是 RELIABLE + KEEP_ALL(见 mbus_qos.xml 的 RobotControl_Control)。
QoS 只声明"要可达",能不能在下一个控制周期内把丢的帧补回来,取决于本段。 -->
<Domain Id="2">
<Compatibility>
</Compatibility>
<General>
<Interfaces>
<NetworkInterface name="wlan0" presence_required="false"/>
<NetworkInterface name="eth0" presence_required="false"/>
</Interfaces>
<!-- spdp:组播只用于 SPDP 发现,SEDP 和用户数据一律走单播。
Wi-Fi 的组播帧以最低基础速率发送且没有 802.11 层 ARQ(收不到不重发),
弱信号下丢包率远高于单播;RELIABLE 虽然能靠 RTPS 重传补回来,但那是
用一个 NACK 往返去补一条本来就不该丢的帧。改成单播后 802.11 自己的
重传先兜一层,RTPS 重传只处理漏网的。
本域加这一段的初衷是"多播发现",spdp 正好只保留发现所需的组播。 -->
<AllowMulticast>spdp</AllowMulticast>
</General>
<Discovery>
<!-- Probe local discovery ports forever: a process started before
wlan0/eth0 is up selects only lo, which has no multicast on Linux;
without a never-pruned localhost peer it can never discover
participants started more than InitialLocatorPruneDelay (30s) later. -->
<LeaseDuration>10s</LeaseDuration>
</Discovery>
<Internal>
<!-- 弱网重传参数,三个默认值会直接让 RELIABLE 形同虚设:
- HeartbeatInterval:reader 靠 HEARTBEAT 才知道自己缺样本。数据流
连续时 HB 会搭在 DATA 上,但控制流一停(松手静止),最后几帧的
缺失就要等下一个周期心跳 —— 2s 意味着最坏 2s 后才补上,等于
丢了 200 个控制周期。回到 CycloneDDS 默认 100ms,并把周期心跳
收敛到 [20ms, 500ms]。
- NackDelay:reader 发现缺样本后延迟多久发 NACK,默认 100ms
= 10 个控制周期,压到 10ms。
- AutoReschedNackDelay:NACK 或其响应本身丢了之后的重试间隔,
默认 3s,弱网下这是很可能发生的,压到 500ms。
- Watermarks:KEEP_ALL 写端积压超过 WhcHigh 才背压。WhcAdaptive
默认开启,会在重传期间把 WhcHigh 一路收缩到 WhcLow(默认 1kB
≈ 4 帧),正好在丢包时把实时控制环卡在 max_blocking_time 上。
这里关掉自适应并给固定水位:一帧 Control 约 256B(含 RTPS 开销),
WhcHigh 64kB ≈ 250 帧 ≈ 2.5s 重传窗口,再长就是拿过期指令喂运控。
注意 WhcAdaptive=false 时 whc_high 恒等于 WhcHighInit,两者必须
一起设;且 WhcHighInit 必须落在 [WhcLow, WhcHigh] 内,否则报
"Invalid watermark settings",participant 直接建不出来。 -->
<HeartbeatInterval min="20ms" max="500ms">100ms</HeartbeatInterval>
<NackDelay>10ms</NackDelay>
<AutoReschedNackDelay>500ms</AutoReschedNackDelay>
<Watermarks>
<WhcLow>8 kB</WhcLow>
<WhcHigh>64 kB</WhcHigh>
<WhcHighInit>64 kB</WhcHighInit>
<WhcAdaptive>false</WhcAdaptive>
</Watermarks>
</Internal>
<Sizing>
</Sizing>
<Threads>
</Threads>
<Tracing>
<!-- <Verbosity>finer</Verbosity>
<OutputFile>cyclonedds_${CYCLONEDDS_PID}.log</OutputFile> -->
</Tracing>
</Domain>
</CycloneDDS>
+346
View File
@@ -0,0 +1,346 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- Trimmed from mbus/config/mbus_qos.xml to the profiles this FSM uses.
Copied verbatim, never retyped: QoS that does not match the other side
makes DDS silently never pair the reader and the writer. -->
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.omg.org/spec/DDS-XML"
xsi:schemaLocation="http://www.omg.org/spec/DDS-XML http://www.omg.org/spec/DDS-XML/20180501/dds-xml_qos_definitions.xsd">
<qos_library name="mbus">
<qos_profile name="common">
<!-- Domain -->
<domain_participant_qos>
</domain_participant_qos>
<!-- Publisher -->
<publisher_qos>
<presentation>
<access_scope>INSTANCE_PRESENTATION_QOS</access_scope>
<coherent_access>false</coherent_access>
<ordered_access>true</ordered_access>
</presentation>
</publisher_qos>
<!-- Subscriber -->
<subscriber_qos>
<presentation>
<access_scope>INSTANCE_PRESENTATION_QOS</access_scope>
<coherent_access>false</coherent_access>
<ordered_access>true</ordered_access>
</presentation>
</subscriber_qos>
</qos_profile>
<qos_profile name="MotorControl_Control">
<!-- Domain -->
<domain_participant_qos>
</domain_participant_qos>
<!-- Publisher -->
<publisher_qos>
</publisher_qos>
<!-- Subscriber -->
<subscriber_qos>
</subscriber_qos>
<!-- Topic -->
<topic_qos>
<durability>
<kind>VOLATILE_DURABILITY_QOS</kind>
</durability>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
<transport_priority>
<value>10</value>
</transport_priority>
</topic_qos>
<!-- DataWriter -->
<datawriter_qos>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
</datawriter_qos>
<!-- DataReader -->
<datareader_qos>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
</datareader_qos>
</qos_profile>
<qos_profile name="MotorControl_State">
<!-- Domain -->
<domain_participant_qos>
</domain_participant_qos>
<!-- Publisher -->
<publisher_qos>
</publisher_qos>
<!-- Subscriber -->
<subscriber_qos>
</subscriber_qos>
<!-- Topic -->
<topic_qos>
<durability>
<kind>VOLATILE_DURABILITY_QOS</kind>
</durability>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
<transport_priority>
<value>10</value>
</transport_priority>
</topic_qos>
<datawriter_qos>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
</datawriter_qos>
<datareader_qos>
<reliability>
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_LAST_HISTORY_QOS</kind>
<depth>1</depth>
</history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
</datareader_qos>
</qos_profile>
<!-- 机器人控制指令 100Hz(control-agent-cpp -> motion-planner)。
通道里混着边沿触发的一次性事件(action.trigger 脉冲、jump 参数),
丢一帧 = 丢一次跳跃/攻击,所以弱网下要求逐帧可达:
- RELIABLE:丢包走 RTPS 重传补齐,而不是听天由命。
- KEEP_ALL(写端必需):CycloneDDS 的 KEEP_LAST 写端在深度溢出时会把
**尚未被 ACK** 的样本直接从 WHC 里删掉(dds_whc.c whc_default_insert,
hdepth>0 时无条件 prune),depth=1 意味着重传窗口只有一帧 ——
弱网下 RELIABLE+KEEP_LAST(1) 实际退化成 BEST_EFFORT。只有 KEEP_ALL
才会把样本留到所有 reader 确认为止。
- KEEP_ALL(读端必需):一次补包会把积压的多帧一起递上来,
KEEP_LAST(1) 的 RHC 会让旧帧被新帧覆盖,应用照样看不到那一帧。
- max_blocking_time=100ms:KEEP_ALL 写端在未确认积压超过 WhcHigh 时
会阻塞等重传追上(ddsi_transmit.c throttle_writer),超时则本帧被
直接丢弃并返回 TIMEOUT。Domain 2 的 NackDelay 已压到 10ms,一次
重传往返远小于 100ms,够补完;同时封顶在 10 个控制周期内,write()
超时由 wrapper 的 noexcept 路径转成错误码,不会把实时控制环无限拖住。
- 读端 max_samples=512:约 5s@100Hz 的补包缓冲;队列满时 RHC 拒收且
不 ACK,写端持续重传形成背压,而不是静默丢帧。
(写端不受 resource_limits 约束,其积压上限由
Internal/Watermarks/WhcHigh 决定,这里的数值仅作对齐说明。)
本 profile 只声明"要可达",实际能不能补回来还依赖两件事:
1) 收发两端必须同时换上新 profile:旧的 BEST_EFFORT writer 遇上
RELIABLE reader 属于 QoS 不兼容,会直接匹配不上、一帧都收不到。
2) 该 topic 跑在 Domain 2(跨机弱网域),mbus_config.xml 的
Domain 2 已配套调整 AllowMulticast=spdp、HeartbeatInterval、
NackDelay 与 Watermarks —— 详见那边的注释。若换到别的 domain
运行,需要把同一套 Internal 参数带过去,否则 RELIABLE 名存实亡。 -->
<qos_profile name="RobotControl_Control">
<!-- Domain -->
<domain_participant_qos>
</domain_participant_qos>
<!-- Publisher -->
<publisher_qos>
</publisher_qos>
<!-- Subscriber -->
<subscriber_qos>
</subscriber_qos>
<!-- Topic -->
<topic_qos>
<durability>
<kind>VOLATILE_DURABILITY_QOS</kind>
</durability>
<reliability>
<kind>RELIABLE_RELIABILITY_QOS</kind>
<max_blocking_time>
<sec>0</sec>
<nanosec>100000000</nanosec>
</max_blocking_time>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_ALL_HISTORY_QOS</kind>
</history>
<resource_limits>
<max_samples>512</max_samples>
<max_instances>1</max_instances>
<max_samples_per_instance>512</max_samples_per_instance>
</resource_limits>
<transport_priority>
<value>10</value>
</transport_priority>
</topic_qos>
<datawriter_qos>
<durability>
<kind>VOLATILE_DURABILITY_QOS</kind>
</durability>
<reliability>
<kind>RELIABLE_RELIABILITY_QOS</kind>
<max_blocking_time>
<sec>0</sec>
<nanosec>100000000</nanosec>
</max_blocking_time>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_ALL_HISTORY_QOS</kind>
</history>
<resource_limits>
<max_samples>512</max_samples>
<max_instances>1</max_instances>
<max_samples_per_instance>512</max_samples_per_instance>
</resource_limits>
</datawriter_qos>
<datareader_qos>
<durability>
<kind>VOLATILE_DURABILITY_QOS</kind>
</durability>
<reliability>
<kind>RELIABLE_RELIABILITY_QOS</kind>
<max_blocking_time>
<sec>0</sec>
<nanosec>100000000</nanosec>
</max_blocking_time>
</reliability>
<destination_order>
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
</destination_order>
<history>
<kind>KEEP_ALL_HISTORY_QOS</kind>
</history>
<resource_limits>
<max_samples>512</max_samples>
<max_instances>1</max_instances>
<max_samples_per_instance>512</max_samples_per_instance>
</resource_limits>
</datareader_qos>
</qos_profile>
<qos_profile name="Imu_ImuOutput">
<domain_participant_qos></domain_participant_qos>
<publisher_qos></publisher_qos>
<subscriber_qos></subscriber_qos>
<topic_qos>
<durability><kind>VOLATILE_DURABILITY_QOS</kind></durability>
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
<history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>1</depth></history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>10</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
</topic_qos>
<datawriter_qos></datawriter_qos>
<datareader_qos></datareader_qos>
</qos_profile>
<!-- High-rate motion-planner debug stream: best-effort, volatile,
keep-last-1 so a slow subscriber never back-pressures the RT loop. -->
<!-- PhoneControl::Message (手机 App -> control-agent-cpp): 50Hz 控制指令 / ~1Hz 状态回传。
Best-effort, volatile, keep-last-1, 跟上 / 下的机器人控制走同样的 QoS 设定。 -->
<!-- Audio PCM stream: best-effort, volatile, keep-last-8 (~80 ms at
10 ms periods). Depth > 1 so a burst of periods is not collapsed
to a single sample (unlike video KEEP_LAST 1). -->
<!-- RobotControlRaw::ControlRaw (control-pod-svc): 50Hz Xbox-aligned snapshot.
Best-effort, volatile, keep-last-1 — same pattern as VR control frame. -->
<qos_profile name="RobotControlRaw_ControlRaw">
<domain_participant_qos></domain_participant_qos>
<publisher_qos></publisher_qos>
<subscriber_qos></subscriber_qos>
<topic_qos>
<durability><kind>VOLATILE_DURABILITY_QOS</kind></durability>
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
<history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>1</depth></history>
<resource_limits>
<max_samples>10</max_samples>
<max_instances>1</max_instances>
<max_samples_per_instance>10</max_samples_per_instance>
</resource_limits>
<transport_priority><value>10</value></transport_priority>
</topic_qos>
<datawriter_qos>
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
</datawriter_qos>
<datareader_qos>
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
</datareader_qos>
</qos_profile>
</qos_library>
</dds>
+85
View File
@@ -0,0 +1,85 @@
module Common
{
// Topic类型
@extensibility(FINAL)
enum TopicType
{
@value(0)
ROBOT_CONTROL_CONTROL,
MOTOR_CONTROL_CONTROL,
MOTOR_CONTROL_STATE,
WEBRTC_SIGNALING_WEBRTC_SIGNAL,
FAULT_MSG_RAISE,
FAULT_MSG_RELEASE,
IMU_OUTPUT,
MOTION_PLANNER_DEBUG,
BMS_STATUS,
ROBOT_CONTROL_CONTROL_RAW
};
// 姿态角
@extensibility(FINAL)@nested
struct AttitudeAngles
{
float roll;
float pitch;
float yaw;
};
// 速度
@extensibility(FINAL)@nested
struct Velocity3D
{
float x;
float y;
float z;
};
// 加速度
@extensibility(FINAL)@nested
struct Accelerometer3D
{
float x;
float y;
float z;
};
// 磁力计
@extensibility(FINAL)@nested
struct Magnetometer3D
{
float x;
float y;
float z;
};
// 四元数
@extensibility(FINAL)@nested
struct Quaternion
{
float w;
float x;
float y;
float z;
};
// 位置
@extensibility(FINAL)@nested
struct Position3D
{
float x;
float y;
float z;
};
// 模式
@extensibility(FINAL)
enum RobotMode
{
@value(0)
HEXAPOD, // 六足
QUADRUPED, // 四足
DANCE, // 舞蹈
JUMP // 跳跃
};
};
+40
View File
@@ -0,0 +1,40 @@
# include "idl_common.idl"
module Imu
{
@extensibility(FINAL)
struct ImuOutput
{
uint64 timestamp_ns;
Common::Accelerometer3D acc;
Common::Velocity3D vel;
Common::Position3D pos;
Common::Velocity3D gyro;
Common::Quaternion q;
};
// 原极(yuanji) AHRS 单帧原始数据
@extensibility(FINAL)@nested
struct YuanjiAhrs
{
uint32 timer; //us,原极 AHRS 内部时间
float pitch;
float roll;
float yaw;
float ax;
float ay;
float az;
float gx;
float gy;
float gz;
float temp;
};
@extensibility(FINAL)
struct YuanjiOutput
{
// SOC 侧收到该样本的时刻 (CLOCK_REALTIME, Unix 纪元纳秒)。
uint64 timestamp_ns;
YuanjiAhrs ahrs;
};
};
+88
View File
@@ -0,0 +1,88 @@
# include "idl_common.idl"
module MotorControl
{
const uint32 MC_MAX_MOTOR_CNT = 24;
// 扩展寄存器读回(extra-register readback):让选中的少数电机在 MIT 控制的
// 同时额外读回指定寄存器,经 SPI/DDS 回传。最多 MC_MAX_EXTRA_MOTOR 个电机,
// 每个最多 MC_MAX_EXTRA_REG 个寄存器。
const uint32 MC_MAX_EXTRA_MOTOR = 5;
const uint32 MC_MAX_EXTRA_REG = 2;
@extensibility(FINAL)@nested
struct MotorCmd
{
float pos;
float kp;
float dq;
float kd;
float tau;
};
@extensibility(FINAL)@nested
struct MotorState
{
octet isvalid;
float pos;
float dq;
float tau;
octet status;
float fpc_temper;
float pcb_temper;
};
@extensibility(FINAL)@nested
struct ImuData
{
octet imu_data_valid; //imu原始数据就绪标志, 0为未就绪, 1为就绪(下同)
octet mag_data_valid; //mag原始数据就绪标志
uint32 imu_timestamp; //imu原始数据时间戳(us)
uint32 mag_timestamp; //mag原始数据时间戳(us)
Common::Accelerometer3D acc; //加速度原始数据
Common::Velocity3D gyro; //陀螺仪原始数据
Common::Magnetometer3D mag; //地磁原始数据
float imu_temperature; //imu温度, 单位:摄氏度
Common::Quaternion q; //四元数
uint32 quat_timestamp; //四元数时间戳(us)
octet quat_valid; //四元数就绪标志
octet imu_err_code; //imu错误码, 0,无错误 1,imu初始化中 2,imu状态异常 3,imu未启动
octet raw_count;
// raw_count说明
// imu初始化计数, 用于给上层看,当imu异常重启时, 四元数状态会重置,因此需要上层知道imu重启了。
// 验证方法,raw_count与上一次获取的值不一致即为imu重启。
// 数值范围: 0~7 初始化后正常为1,触发复位+1,最大为7,超过7后归0
};
@extensibility(FINAL)
struct Control
{
uint64 timestamp_ns;
uint16 sequence_id;
uint32 motor_count;
MotorCmd motors[MC_MAX_MOTOR_CNT];
};
// 单个电机的扩展寄存器读回条目。addrs/values 一一对应,未用的槽位以 0 填充。
@extensibility(FINAL)@nested
struct MotorExtra
{
octet motor_index; // HCI 电机序号 (0-based);0xFF = 无效条目
octet n_extra; // 该电机实际读回的寄存器个数 0..MC_MAX_EXTRA_REG
octet addrs[MC_MAX_EXTRA_REG]; // 读回的寄存器地址(回显,便于消费者解释)
float values[MC_MAX_EXTRA_REG]; // 对应寄存器的 f32 值
};
@extensibility(FINAL)
struct State
{
uint16 sequence_id;
uint32 motor_count;
MotorState motors[MC_MAX_MOTOR_CNT];
ImuData imu;
uint64 source_timestamp_ns;
octet extra_count; // 有效 MotorExtra 条目数 0..MC_MAX_EXTRA_MOTOR
MotorExtra extra[MC_MAX_EXTRA_MOTOR]; // 扩展寄存器读回(未用条目 motor_index=0xFF)
};
};
+99
View File
@@ -0,0 +1,99 @@
# include "idl_common.idl"
module RobotControl
{
enum ActionId {
@value(0)
ACTION_NONE,
// 方向键(无臂激活时触发)
ACTION_UP, // 方向键 ↑
ACTION_DOWN, // 方向键 ↓
ACTION_LEFT, // 方向键 ←
ACTION_RIGHT, // 方向键 →
// 功能键 → 一次性触发(方向取触发帧 body.linear_velocity.x/y)
ACTION_X, // Xbox X
ACTION_Y, // Xbox Y
ACTION_B, // Xbox B
ACTION_A, // Xbox A 按下触发 / LT、RT 松手触发;参数取 jump.x/y/spin
// Menu 组合键
ACTION_MENU_UP, // Menu + ↑
ACTION_MENU_DOWN, // Menu + ↓
ACTION_MENU_LEFT, // Menu + ←
ACTION_MENU_RIGHT, // Menu + →
ACTION_MENU // 单按 Menu
};
enum GripperPose {
@value(0)
POSE_NONE, // 无预设(回位 / 交给 Rx 俯仰耦合)
POSE_EXTEND_UP, // 前臂操作(方向键 ↑)
POSE_EXTEND_DOWN, // 前臂操作(方向键 ↓)
POSE_EXTEND_LEFT, // 前臂操作(方向键 ←)
POSE_EXTEND_RIGHT, // 前臂操作(方向键 →)
POSE_EXTEND_UP_LEFT, // 前臂操作(方向键 ↑+←)
POSE_EXTEND_UP_RIGHT, // 前臂操作(方向键 ↑+→)
POSE_EXTEND_DOWN_LEFT, // 前臂操作(方向键 ↓+←)
POSE_EXTEND_DOWN_RIGHT // 前臂操作(方向键 ↓+→)
};
@extensibility(FINAL)@nested
struct BodyState
{
Common::AttitudeAngles attitude; // roll / pitch / yaw;pitch=Rx(前后推)· yaw=扭身 twist(|Ry|≤半行程)
Common::Velocity3D linear_velocity; // x=前后 vx / y=左右 vy / z=垂直 vz(VR 身高)
Common::Velocity3D angular_velocity; // z = 转动 yaw_rate(Ry/左右推,|Ry|>半行程);x/y 恒 0
float height; // 目标身高(agent 锁存,[-1,1])
};
@extensibility(FINAL)@nested
struct GripperState
{
boolean is_valid; // 该臂是否激活(Lb/Rb,二者互斥)
Common::Position3D position; // 末端位置目标(VR / 预设位姿数值化)
Common::AttitudeAngles attitude; // 末端姿态目标(VR / 预设位姿数值化)
Common::Velocity3D linear_velocity; // 末端速度(兼容保留)
float opening; // 夹爪开合 [0,1](激活臂的 LT/RT 扳机行程)
GripperPose pose; // 离散预设姿态(方向键;未激活时 NONE)
};
// 跳跃参数(无触发字段,触发统一走 action 通道的 ACTION_A,边沿帧读取本参数)。
// A 按下触发;方向/距离取 x/y(raw,不过死区/fast)。
// LT/RT 松手检测触发,仅无臂激活时有效;按住时长累积转动幅度,
// 松手帧发 ACTION_A 并携带 spin。
@extensibility(FINAL)@nested
struct JumpCommand
{
float x; // 起跳向量 前后 [-1,1](raw 左摇杆纵轴,正=前,不过死区/fast)
float y; // 起跳向量 左右 [-1,1](raw 左摇杆横轴,正=右)
// 方向 = atan2(y,x);距离 = min(|(x,y)|,1),0=原地跳
float spin; // 旋转跳幅度 [-1,1]:负=左旋(LT)/正=右旋(RT),0=直线跳;由 LT/RT 按下时长映射
};
// 离散动作指令。trigger 触发脉冲(连发 3 帧);脉冲期内新按键被忽略(不排队、不覆盖,
// A1 锁存)。ACTION_A 与其它 action_id 同为单次触发、共用同一通道,同一时刻只触发一种。
// action_id=ACTION_A 时参数取 Control.jump(x/y/spin);ACTION_B / ACTION_Y 方向取边沿帧
// body.linear_velocity.x/y(|方向|≈0 → 默认朝机体前方)。
@extensibility(FINAL)@nested
struct ActionCommand
{
float trigger; // 触发脉冲(连发 3 帧):0=空闲,1=触发;下游上升沿只响应一次
ActionId action_id; // 动作编号
};
@extensibility(FINAL)
struct Control
{
uint64 timestamp_ns;
BodyState body;
GripperState left_gripper;
GripperState right_gripper;
JumpCommand jump;
ActionCommand action;
};
@extensibility(FINAL)
struct Debug
{
float time_stamp;
};
};
+31
View File
@@ -0,0 +1,31 @@
module RobotControlRaw
{
@extensibility(FINAL)
struct ControlRaw
{
uint64 timestamp_ns;
uint32 sequence_id;
boolean connected;
uint16 vendor_id;
uint16 product_id;
string profile_id;
boolean A;
boolean B;
boolean X;
boolean Y;
boolean LB;
boolean RB;
boolean menu;
boolean home;
boolean L3;
boolean R3;
int8 dpad_x;
int8 dpad_y;
float Lx;
float Ly;
float Rx;
float Ry;
float LT;
float RT;
};
};
+45
View File
@@ -0,0 +1,45 @@
{
"_source": "mbus idl/topics.json @ master d9875988f065 (2026-09-28) -- only the channels this FSM uses",
"channels": [
{
"key": "robot_control_command",
"topic": "robot_control",
"qos": "RobotControl_Control",
"type": "RobotControl::Control",
"status": "live",
"note": "control-agent-cpp -> motion-planner"
},
{
"key": "motor_control_command",
"topic": "mc/motor_control",
"qos": "MotorControl_Control",
"type": "MotorControl::Control",
"status": "live",
"note": "motion-planner / mc_forwarder -> mc_ctrl_driver"
},
{
"key": "motor_state",
"topic": "mc/motor_state",
"qos": "MotorControl_State",
"type": "MotorControl::State",
"status": "live",
"note": "mc_ctrl_driver / mc_forwarder -> motion-planner"
},
{
"key": "imu_output",
"topic": "imu/output",
"qos": "Imu_ImuOutput",
"type": "Imu::ImuOutput",
"status": "planned",
"note": "IMU stream; QoS defined, no live peer"
},
{
"key": "robot_control_raw",
"topic": "RobotControlRaw_ControlRaw",
"qos": "RobotControlRaw_ControlRaw",
"type": "RobotControlRaw::ControlRaw",
"status": "planned",
"note": "control-pod-svc -> subscribers"
}
]
}
+369
View File
@@ -0,0 +1,369 @@
# {BUNDLE}
This legacy policy bundle carries the robot controller and its policies for
the available board, browser, and simulation hosts.
```
{RUNTIME_FILES}controller.toml the modes, the rules that switch between them, the keys
{MODE_FILES}reference.json recorded frames, so you can measure that you agree
manual.en.json every key and button this bundle reads, and what each does
bundle.json every file, its size and its SHA-256, and which host takes what
```
`manual.en.json` is written by the build from the controller's own account of
its pad and its keys, so it describes exactly the bindings here. A translation travels
beside it as `manual.<language>.json` -- `manual.zh.json` in Chinese -- listed
in `bundle.json`'s `manuals`. The source repository's `deploy/app.schema`
defines the files and metadata this format accepts.
A reference-guided mode carries one more file, named by its contract's
`reference` block. A mode needs its recorded-motion companion before it loads.
**Each host takes its own and leaves the rest.** `bundle.json`'s `runtimes`
says which model format each host loads and where its build of the controller
is, and each mode's `models` names its file in every format:
| host | models | controller |
|---|---|---|
| `board` (RK3576) | `.rknn`, for the NPU | `runtime/board/controller`, aarch64 |
| `web` (a browser) | `.onnx` | `runtime/web/controller.wasm` and its `.js` binding -- the same bytes on any CPU |
| `mjlab` (`play --app`) | `.onnx` | `runtime/mjlab/<platform>/controller.so`, the build machine's, and `runtime/mjlab/win-amd64/controller.pyd` and `runtime/mjlab/macosx-universal2/controller.so` (arm64 and x86_64), cross-built |
A host whose build this bundle does not carry has no entry, and
`bundle.json`'s `notes` says how to add it. Everything else -- the FSM, the
contracts, the recordings, the reference -- is one copy every host reads, so
there is nothing for two hosts to disagree about.
On the board, from this directory:
```bash
./runtime/board/controller --bundle . --dry-run # report, touch no bus
./runtime/board/controller --bundle . --check-reference # replay the frames below
./runtime/board/controller --bundle . --machine /etc/mjrl/machine.toml
```
`--machine` is what this bundle cannot know: the DDS domains, the QoS XML and
the few constants that describe the actuators rather than the policy.
Everything else comes from the contracts here, and `--dry-run` prints what it
will use -- the DDS domains, the QoS file, each mode's controls as its contract
gives them -- and every note and stub.
Replay the bundle from the installed training checkout, with its registered
task and model. The simulator applies the controller's joint targets.
```bash
python scripts/play.py --app <this directory, or the .app beside it>
python scripts/deploy.py --check-reference <this directory>
```
`play --app` takes the controller built for its own machine from here rather
than whatever `mjrl_fsm` is installed, because otherwise it runs a
*different build of the same source*, and the two diverge the moment somebody
rebuilds one of them.
The rest of this file is for a browser.
**The controller here is the one the robot runs.** Not the same logic
reimplemented from the same contract -- the same compiled object, built from
`deploy/fsm` at commit `{COMMIT}`. That is the point of shipping it: three
implementations of one observation agreed by people reading each other's code,
and a sign corrected in one left the other two wrong with every other number
still correct.
**And you can measure that your host runs it the same way.** `reference.json`
is a recorded run of this controller: per frame, the robot state that went in,
the observation it built, the action the policy returned and the joint targets
it published. Replay it on load:
(It says your host agrees with the machine that built the bundle. It says
nothing about whether that machine should be trusted -- see the last section.)
```js
const report = JSON.parse(fsm.checkReference(referenceJson))
// {frames, inferences, observation, target, divergent, modeMismatches}
```
`observation` and `target` should be **0**. They are the same f32 arithmetic in
the same order as every other host, so anything above about 1e-6 is a real
difference, not rounding. `divergent` names terms your host is expected to
source differently from training -- the robot measures nearly everything a
simulator computes -- with the worst difference each one actually showed, which
is normally 0 because the replay feeds every host the same recorded input. This
paragraph used to single out `joint_torque` as reconstructed on the robot from
the commanded PD and differing there by about 2 N*m. **That was wrong**: the
servo reports torque, the controller has always read it, and the term is
measured on the robot too -- the 2 N*m was the gap between a real number and a
reconstruction of it that nothing needed. `modeMismatches` is never acceptable:
a cascade landing in another state is a different robot, not a tolerance.
Your inference backend is a separate question and not covered by that call.
Run each frame's `obs` through your own model and compare against its `act`:
onnxruntime-web should match to ~1e-6, a quantised `.rknn` differs in the third
decimal. Keeping the two apart is deliberate -- a wrong observation and a
quantised model produce the same symptom, a robot that walks slightly wrong.
---
## Loading it
```js
import init, { WebFsm } from "./runtime/web/controller.js";
await init({
module_or_path: await (await fetch("./runtime/web/controller.wasm")).arrayBuffer(),
});
const fsm = new WebFsm(
fsmToml, // controller.toml, as text
JSON.stringify(contracts), // {{ "<mode>": <that mode's .json> }}
JSON.stringify(robot), // below
nowMicroseconds,
JSON.stringify(trajectories), // {{ "<mode>": <its *.trajectory.json text> }}
);
```
`trajectories` is only for a mode whose contract carries a `reference` block --
a recorded motion the policy is driven by. Read the file that block names out of
this bundle and pass its text; omit the argument entirely for a bundle of
ordinary policies. A mode that needs one and does not get it is refused by name,
because the alternative is building its reference terms out of zeros, which is a
motionless recording that a standing robot tracks perfectly.
```js
const trajectories = {{}};
for (const [mode, contract] of Object.entries(contracts)) {{
if (contract.reference) {{
trajectories[mode] = await (await fetch(`./${{contract.reference.file}}`)).text();
}}
}}
```
`robot` is what the *host* knows and no policy does:
```js
{
joint_names: [...], // wire_joint_order, from any mode's contract
joint_pos_lo: [...], // per joint; prefer the contract's joint_limits
joint_pos_hi: [...], // and fall back to your own model's
output_rate_hz: 50, // how often you will call tick()
gait_period: 0.32, // optional; only for a contract whose gait_phase
gait_gate_threshold: 0.05 // carries no clock of its own -- each mode's
// contract names its period or its cadence law
}
```
`joint_limits` is a `{joint: [lo, hi]}` map in each mode's contract, in the
same radians as `default_joint_pos`, and it is what the robot itself clamps
to. Preferring it over your own model keeps the two hosts agreeing about the
one bound that cannot be observed going wrong -- a clamp wide enough never to
fire looks exactly like a clamp that works. Older bundles do not carry it; fall
back to your model then, and say which you used.
Every failure is thrown with the reason in it. There is no fallback: a host
that cannot build this should say so rather than run a robot on defaults
nobody chose.
## One control step
```js
fsm.set_state(q, qd, tau, quat, gyro, nowUs);
fsm.setSignal("base_lin_vel", linVelBody); // everything else you can compute
if (stopPressed) fsm.letGo(nowUs); // your Stop button, Escape -- once
for (const e of keyEvents) fsm.setKeyCode(e.code, e.down, e.repeat, nowUs);
if (pad) { // a real pad, or on-screen sticks
for (const [axis, v] of Object.entries(pad.axes)) fsm.setAxis(axis, v);
for (const b of PAD_BUTTONS) fsm.setPad(b, pad.pressed.includes(b));
fsm.padFrame(nowUs);
}
fsm.set_command(0, 0, 0, nowUs); // keeps the operator fresh
const mode = fsm.tick(nowUs);
if (mode !== undefined) {
fsm.resume(await yourOnnxSession(mode).run(fsm.observation()));
}
publish(fsm.positions()); // joint targets, wire order
```
`tick` returns the mode needing an inference, or nothing when the step is
finished -- a warm start, a mode-switch ramp or a hold still publishes, and
`positions()` is ready either way.
**The split is not an abstraction.** A wasm module cannot call a JavaScript
model and return inside one synchronous call, and the loop publishes faster
than it infers: `tick` only asks when the active mode's `control_hz` says so.
Running the policy at your own rate aliases it *and* rescales the per-tick
joint rate limit.
## What must arrive exactly right, because none of it fails loudly
| argument | what it must be |
|---|---|
| `q` `qd` `tau` | **wire order**, i.e. `joint_names` above. Not your model's order and not the policy's; the three are paired by name, and pairing by index is wrong from the fifth joint on |
| `quat` | body orientation `(w, x, y, z)` |
| `gyro` | angular velocity **in the body frame**. An IMU gives this; a simulator usually gives you the world frame and a rotation matrix, so rotate: `out[a] = m[a]*w[0] + m[3+a]*w[1] + m[6+a]*w[2]` |
| `tau` | applied joint torque. A simulator has the solver's; the robot reads the servo's. If *your* host measures none, reconstruct what your PD commanded and say which you did |
| `vx` `vy` `wz` | never read by a mode with a controls block of its own (`takesRawInput()`); for a mode with none, **physical units**, m/s and rad/s |
| `setAxis` values | the pad as the robot's pad service reports it: sticks `[-1, 1]` with a 0.15 deadband already applied **and rescaled**, triggers `[0, 1]`. No sign, no scale -- those are the bundle's |
| `nowUs` | microseconds, monotonic. The module has no clock; freshness and rate are decided from what you pass |
If your simulator reports a 6-vector velocity, check its order before using it.
MuJoCo's is `(angular, linear)`; reading it the other way gives a gyro that
reports how fast the robot is walking, and nothing anywhere says so.
## Handing over a person's input
**Hand over what you see, and nothing you decided.** What a stick or a key
means is the running mode's: its `controller` block says which control drives
which command axis, how a stick's travel splits, which button holds a layer,
which key pushes which way, where a released axis rests -- and the controller
inside this wasm executes it. The pad and the keyboard are two paths, each
bound independently. The same input can adjust height in one mode and pitch
in another. Send the physical inputs to the controller so it can apply the
active mode's mapping.
So, every frame:
- **Every key, both edges.** `fsm.setKeyCode(event.code, down, repeat, nowUs)`
with the browser's own `KeyboardEvent.code`. The dictionary compiled into
this wasm knows which codes are keys it has a name for; the return value says
whether anything here binds that one, which is what tells you to
`preventDefault` it. `repeat` is the event's own flag, and says the key is
still down: a key held pushes what it drives for as long as it is held --
full after the mode's `full_after_s` -- and lets it back the moment its
`keyup` arrives, so the up edge matters as much as the down. Shift, Alt and
Ctrl are keys like any other, left and right each by its own code; Alt pressed
on its own moves a Windows browser's focus to its menu unless you
`preventDefault` it.
- **The pad, whenever there is one.** `fsm.setAxis(name, value)` for `Lx` `Ly`
`Rx` `Ry` `LT` `RT`, `fsm.setPad(button, down)` for every button, then
`fsm.padFrame(nowUs)`. On-screen sticks are a pad too: report them as `Lx`
`Ly` `Rx` `Ry`. Pad and keyboard are both live and the one touched last
drives; the controller arbitrates, so you need not.
- **Every quantity you can compute.** `fsm.setSignal(name, values)`, whether or
not today's policy observes it -- `base_lin_vel`, body frame, m/s, is the one
read now. A policy that needs something a robot cannot measure is refused at
export, but a web-only bundle may, and a page that passed only what today's
bundles need is a page tomorrow's outgrows without saying so.
- **Your own stop, once.** `fsm.letGo(nowUs)` when your page's Stop button or
Escape is pressed: the operator let go of everything, exactly as the bundle's
release button does. A host control rather than a pad button, because you
have no button of the bundle's to press on somebody's behalf -- which one
releases is the contract's to say. Call it before that step's keys, so a key
pressed after the stop is a new press. A key still down keeps pushing until
its `keyup`, and a page that lost focus mid-press may never be sent one, so
without this a Stop button stops nothing.
`fsm.takesRawInput()` is true when any mode has a controls block. Those modes
never read `set_command`; a mode with none does, and sending it keeps the
operator fresh either way, so sending it every frame is harmless.
`fsm.inputs()` returns `{rawInput, keys: [{name, code}], controls, signals}` for
a page that wants to say which keys do something: `controls` is each mode's own
account of its bindings -- the lines the robot's controller prints before it
enables the motors -- on one line, the modes that read the pad alike named
together: `claw_left, claw_right: … | locomotion: …`. Show it as given;
describing the bundle yourself would be a second copy of its bindings.
`fsm.bindings()` returns `[{name, pad, key, on, with, from, leaves}]` -- the mode
switches, the events and the leaves: everything the operator can ask for
*between* modes, and which control asks it. A binding is one device's, a pad
button or a key, and two with one name -- the pad's `A` and the keyboard's
Space, both `jump` -- share one latch. Those are the same keys and buttons,
reported the same way:
```js
fsm.setKeyCode("KeyW", true, false, nowUs); // the browser's name for it
fsm.setPad("A", down); // the robot pad service's names
```
Both lists come from one dictionary, `controller/vocabulary.json` -- schema
`kk-control-vocabulary/2`, compiled into this wasm, so there is no file to fetch
-- and a binding naming something outside it is refused when the bundle loads.
`fsm.setKey(name, down)` takes the dictionary's own name (`key_w`, `keypad_1`)
for a host that already has it.
Not "was pressed". `on` is one of nine gestures and two of them need the
release:
| `on` | fires |
|---|---|
| `rise` | the tick it goes down |
| `fall` | the tick it comes back up |
| `hold` | every tick it is down |
| `toggle` | down until pressed again |
| `single` `double` `triple` `quadruple` `quintuple` | that many presses, each within `click_window_ms` of the last; switches like `toggle` |
The click gestures are counted on the `nowUs` you hand `tick`, so hand it the
page's real clock (`performance.now() * 1000`): a clock that runs fast or slow
narrows or widens the window a person clicks in.
So press-to-arm and release-to-go is one control bound twice, and a host that
only reported presses could not drive half the vocabulary.
Pad names are `A` `B` `X` `Y` `LB` `RB` `menu` `home` `L3` `R3`, plus
`dpad_up` / `dpad_down` / `dpad_left` / `dpad_right`. Translate your Gamepad API
index into one of those and the two hosts agree about what was pressed. `view`
is on the pad and is deliberately not among them.
`with` names what must be held: a pad button for a pad binding, `ctrl`,
`shift` or `alt` -- either of its two keys -- for a key. A modifier is consumed
by what it modifies, and while it is held a binding without it on the same
control stays quiet, so `menu` alone, `menu`-plus-a-direction and the
direction alone can mean three things without one firing on the way to
another. `from` lists the modes a binding may be pressed in -- pressed
anywhere else it does nothing, though a latch it put on can always be switched
off by it -- and `leaves` the modes a binding lets go of, after which the
cascade falls back to its default mode.
Not every binding switches a mode, and two cases are worth expecting. **A
recorded motion ends.** A jump or a dance is a thing that finishes, and when it
does the controller hands the robot back: `tick` releases that mode's latch for
you and the cascade's last `always` rule catches it, so there is one way out of
a mode rather than two. A toggle you have drawn as lit will go out on its own --
and in a bundle that latches one mode at a time, when another switch is pressed
-- so read `fsm.mode()`, not your own copy of the latch. **And a motion's `go`,
where a bundle has one, is a moment, not a mode.** A push-off is 40 ms wide, so
the button that starts it is read by the mode itself rather than by a
transition rule. It is an ordinary binding otherwise: it is in `bindings()`, it
has a gesture. Report it like every other binding.
## Two things worth putting on screen
`fsm.mode()` is the active mode; `fsm.is_running_policy()` is whether it is
actually producing policy output. They differ through a mode-switch ramp,
which waits on the **measured pose** rather than a timer and can wait
indefinitely if the gains cannot converge. That is deliberate -- the
alternative is starting a policy from a pose it never trained around -- but a
robot holding still looks broken unless you say which it is.
## If you are running this from an upload
**Nothing here has checked this controller.** Your own copy, if you ship one,
is held against your own implementation by whatever test you wrote for it.
This one arrived with the upload. Say so where the person can see it.
Running an uploaded `.onnx` and running an uploaded wasm plus JavaScript are
not the same risk. A graph executed by onnxruntime is arithmetic over tensors;
`controller.js` is a program with your page's authority. If you load it,
**give it an opaque origin** -- an `<iframe sandbox="allow-scripts">` with no
`allow-same-origin`, talking over a `MessagePort`. Measured in Chrome from
inside such a frame: `origin` is `"null"`, `document.cookie`, `localStorage`,
`indexedDB` and `parent.document` all throw `SecurityError`, and a credentialed
`fetch` back to the host origin fails. A round trip of one 411-float
observation had a p95 of 0.1 ms against a 20 ms budget at 50 Hz.
Isolation does not bound CPU or memory. A watchdog that tears the frame down
when it does not answer within a few control periods is the other half.
## One source, compiled three times
`runtime/web/controller.wasm`, a native extension under `runtime/mjlab/`, an
aarch64 binary in `runtime/board/`. Same logic, same commit, which
`bundle.json`'s `runtimes` records for each. The two native ones are
**platform-specific** where the wasm is not, so each records the platform it
was built for, and a host on another one should say so rather than fail with a
loader error nobody can read.
---
Built by `scripts/deploy.py` from source revision `{COMMIT}`.
+587
View File
@@ -0,0 +1,587 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "aho-corasick"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
dependencies = [
"memchr",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "bindgen"
version = "0.70.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f49d8fed880d473ea71efb9bf597651e77201bdd4893efe54c9e5d65ae04ce6f"
dependencies = [
"bitflags",
"cexpr",
"clang-sys",
"itertools",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
"rustc-hash",
"shlex 1.3.0",
"syn 2.0.119",
]
[[package]]
name = "bitflags"
version = "2.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "005ec2760ca554fae18df7a11195552ec576cd665632a881bc011d5bb2fd4d80"
dependencies = [
"find-msvc-tools",
"shlex 2.0.1",
]
[[package]]
name = "cexpr"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766"
dependencies = [
"nom",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "clang-sys"
version = "1.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "157a8ba7b480713b56f4c09fd13fc3e0a22a5dfab8097ba61cbc5feef950788a"
dependencies = [
"glob",
"libc",
"libloading",
]
[[package]]
name = "either"
version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "find-msvc-tools"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
[[package]]
name = "glob"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "indoc"
version = "2.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
dependencies = [
"rustversion",
]
[[package]]
name = "itertools"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
dependencies = [
"either",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "libc"
version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libloading"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55"
dependencies = [
"cfg-if",
"windows-link",
]
[[package]]
name = "log"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "memoffset"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
dependencies = [
"autocfg",
]
[[package]]
name = "minimal-lexical"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "mjrl-fsm"
version = "0.1.0"
dependencies = [
"bindgen",
"cc",
"pyo3",
"serde",
"serde_json",
"toml",
"wasm-bindgen",
]
[[package]]
name = "nom"
version = "7.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
dependencies = [
"memchr",
"minimal-lexical",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "portable-atomic"
version = "1.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn 2.0.119",
]
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "pyo3"
version = "0.22.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f402062616ab18202ae8319da13fa4279883a2b8a9d9f83f20dbade813ce1884"
dependencies = [
"cfg-if",
"indoc",
"libc",
"memoffset",
"once_cell",
"portable-atomic",
"pyo3-build-config",
"pyo3-ffi",
"pyo3-macros",
"unindent",
]
[[package]]
name = "pyo3-build-config"
version = "0.22.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b14b5775b5ff446dd1056212d778012cbe8a0fbffd368029fd9e25b514479c38"
dependencies = [
"once_cell",
"python3-dll-a",
"target-lexicon",
]
[[package]]
name = "pyo3-ffi"
version = "0.22.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ab5bcf04a2cdcbb50c7d6105de943f543f9ed92af55818fd17b660390fc8636"
dependencies = [
"libc",
"pyo3-build-config",
]
[[package]]
name = "pyo3-macros"
version = "0.22.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fd24d897903a9e6d80b968368a34e1525aeb719d568dba8b3d4bfa5dc67d453"
dependencies = [
"proc-macro2",
"pyo3-macros-backend",
"quote",
"syn 2.0.119",
]
[[package]]
name = "pyo3-macros-backend"
version = "0.22.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36c011a03ba1e50152b4b394b479826cad97e7a21eb52df179cd91ac411cbfbe"
dependencies = [
"heck",
"proc-macro2",
"pyo3-build-config",
"quote",
"syn 2.0.119",
]
[[package]]
name = "python3-dll-a"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d80ba7540edb18890d444c5aa8e1f1f99b1bdf26fb26ae383135325f4a36042b"
dependencies = [
"cc",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "regex"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "rustc-hash"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "target-lexicon"
version = "0.12.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1"
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned",
"toml_datetime",
"toml_edit",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap",
"serde",
"serde_spanned",
"toml_datetime",
"toml_write",
"winnow",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unindent"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
[[package]]
name = "wasm-bindgen"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 3.0.5",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e"
dependencies = [
"unicode-ident",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "winnow"
version = "0.7.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
dependencies = [
"memchr",
]
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
+85
View File
@@ -0,0 +1,85 @@
[package]
name = "mjrl-fsm"
version = "0.1.0"
edition = "2021"
description = "Run an exported policy on a Rockchip RK3576 NPU, over the robot's DDS bus"
license = "Apache-2.0"
[lib]
# `cdylib` is what `wasm32` needs to produce a module wasm-bindgen can process;
# `rlib` is what everything else links against, including this crate's own
# tests. Both, so one manifest serves the browser and the robot.
crate-type = ["cdylib", "rlib"]
# The robot's host. `required-features` rather than a `#[cfg]` inside the file:
# a machine without the device feature has no bus and no NPU, and `cargo build`
# there should build the library rather than fail on a binary it cannot use.
[[bin]]
name = "controller"
path = "src/bin/controller.rs"
required-features = ["device"]
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
# Only for the browser host. Optional so the device build and `cargo test` do
# not pull a JS-facing dependency they have no use for.
wasm-bindgen = { version = "0.2", optional = true }
# Only for the `play` host. `abi3-py310` builds one extension that every Python
# this repository supports (3.10-3.13) can import, so the artifact does not have
# to be rebuilt per interpreter.
pyo3 = { version = "0.22", features = ["abi3-py310", "extension-module"], optional = true }
[build-dependencies]
cc = "1"
# The IDL structs are nested and change with the robot; generating the Rust
# mirrors from the headers idlc just emitted is the only way they cannot drift.
bindgen = "0.70"
[features]
# `device` is everything that talks to hardware: the DDS bus and, under `rknn`,
# the NPU. It is on by default so the ordinary build and `cargo test` are
# unchanged -- but it is a **feature** rather than the only way to build, because
# the layers above it have three consumers and only one of them is a robot.
#
# The other two are why this line exists:
#
# cargo check --target wasm32-unknown-unknown --no-default-features
#
# passing is the proof that observation, action, contract and FSM contain no
# syscall. A browser has no DDS and no NPU; if the core cannot be built without
# them then it cannot be the core, and the web simulator would have to reimplement
# it -- which is the second implementation this whole path exists to avoid.
default = ["device"]
device = []
# The RKNN runtime is an aarch64 .so; a host build has no NPU to talk to. Off by
# default so `cargo test` works on a development machine, and turned on by the
# aarch64 profile in .cargo/config.toml. Without it every mode falls back to the
# stub engine, exactly as the C++ controller does when built without RKNN.
rknn = ["device"]
# The browser host: `WebFsm`, the JS-facing wrapper around `Controller`. Off by
# default, because it only makes sense for `--target wasm32-unknown-unknown`.
web = ["dep:wasm-bindgen"]
# The `play` host: an importable extension module wrapping the same `Controller`
# the browser and the robot run. Off by default; `extension-module` links
# against no libpython, so a build with it on cannot be run as an ordinary
# binary and `cargo test` would fail to link.
#
# `generate-import-lib` is for the Windows build, cross-compiled in
# `docker-build.sh`: a Windows DLL has to link against `python3.dll` through an
# import library, which a Linux machine has no Python to take it from, and
# without this the link fails with `cannot find -lpython3`. It makes one from
# the stable ABI's symbol list instead. On every other target it does nothing.
py = ["dep:pyo3", "pyo3/generate-import-lib"]
[profile.release]
# A missed control deadline is a real-time failure, so the loop is built for
# latency rather than size: no unwinding tables to walk, one codegen unit so the
# hot path can inline across modules.
lto = true
codegen-units = 1
panic = "abort"
+603
View File
@@ -0,0 +1,603 @@
# fsm — the shape it is being given
**Status: built.** The core is there -- the premise, the observation port, the
FSM cascade and the two-phase tick -- and so are all three hosts. Each section
below says which. Read [`README.md`](README.md) for the crate as a whole.
---
## The problem
The same three steps — **observation → inference → execution** — now have three
places they have to run:
| | observation from | inference on | execution to |
|---|---|---|---|
| `kk-rl-mjlab`'s `play` | mjlab's simulation | ONNX, on the host | mjlab actuators |
| BEUNLIMITED's web sim | MuJoCo WASM | ONNX, in the browser | the same actuators |
| the robot | servos and IMU | RKNN, on the NPU | DDS to the motor controller |
That logic existed **three times**: this crate in Rust, `rl-wbc-fsm` in C++,
`policy.ts` in TypeScript. They agreed, and they agreed the expensive way — by
people reading each other's code. A sign corrected in one of them left the other
two driving a robot that was wrong with every other number still correct.
The fix is not to pick one and delete the others: two of the three consumers
cannot run the third's language. The fix is that **the logic is one crate and
the three differences are ports**. The C++ is no longer one of them — this crate
ships its own binary, `src/bin/controller.rs`, and `rl-wbc-fsm` is kept as
reading: its priority cascade, its per-mode contracts and its safe-state rule
are all here, and where a decision below says "the C++ does X", that is where X
came from.
## The premise, verified
Everything below assumes the observation/action/FSM logic contains no syscall,
because a browser cannot provide one. That is now checked rather than assumed:
```bash
cargo check --target wasm32-unknown-unknown --no-default-features # passes
cargo check # passes
cargo test # passes
```
`device` is a feature (default on, so the ordinary build is unchanged) covering
`dds` and `rknn`; `build.rs` compiles in the task hooks and then returns without it,
so `idlc`, `bindgen` and `cc` never run for a target that could not use them. `obs.rs`,
`action.rs`, `layout.rs` and `types.rs` — 1282 lines when this was written, 2666 now —
compiled to `wasm32` **unmodified**. They were already clean; the feature only
stopped the two device modules from dragging them down.
If that check ever fails, the core has stopped being the core, and the web
simulator is back to reimplementing it.
---
## Shape
```
controller (pure: no I/O, no clock of its own, one build for every target)
layout the policy's contract, parsed exists
obs RawState -> observation vector exists
action action vector -> joint targets exists
types the values that cross a port exists
source what a state source can supply, and how exists
config modes and transition rules exists
vocabulary the words a pad and a keyboard can say exists
trajectory the recorded motion a reference-guided mode reads exists
fsm the ordered cascade exists
control the two-phase tick, the regimes, the per-mode runtimes exists
operator a task's controls: pad and keys -> each mode's command exists
guide what each control does in each mode, as data exists
hook a task's own code at fixed points of a mode's loop exists
reference a recorded run replayed, and where hosts differ exists
hosts (one per environment, outside the core, each in its own language)
device Rust, this crate's `device` feature exists
web JavaScript, via wasm-bindgen exists
play Python, via pyo3 exists
the bundle format (std::fs, so the one module a browser does not compile)
bundle open a directory, check it, build a host from it exists
```
The core never calls out. A host calls in, is told what to do, and does it. That
is forced by wasm and it is the right shape anyway: it makes the whole of the
control logic a pure function of its inputs, which is the only way the three
environments can be compared at all.
---
## The observation port
### Ports are not interchangeable, and pretending they are is the failure
The obvious design is one trait with three implementations. It is wrong, and the
evidence is a term already in the shipped contract.
`joint_torque` carries one value per observed joint and history frame. The host
determines where each value comes from.
| environment | source |
|---|---|
| mjlab | `data.actuator_force` — what the solver applied |
| MuJoCo WASM | the same |
| the robot | `MotorControl_State.motors[i].tau` — what the servo reports |
That last row used to read `tau = kp*(t − q) − kd*qd`, clamped to
`effort_limit`, with the note that the servo does not report torque. **It does**,
and `dds.rs::take_state` had been reading it into `state.tau` the whole time;
the claim came from the C++ controller (`action.cpp:100-103`) and was never
true of this robot. Declaring the term reconstructed made the controller
re-derive a number it already had, and that is the one term the reference
replay ever disagreed on — 2.146 N·m worst case.
The reconstruction is still there, in `applied_torque`, and is still what a
host without servo torque would use. Which one a host takes is decided by what
its `SourceCapability` declares, not by a branch in the loop.
Two consequences the naive design hides:
1. **"observation → inference → execution" is not a chain.** It has a feedback
edge, and the edge closes in a different place per environment.
2. **A term's name does not tell you what it is.** Two runs can agree on every
dimension, scale and index and still differ in what the numbers mean.
### So a source declares what it can supply, and how
**Built — `src/source.rs`.**
```rust
enum Provenance {
Simulated, // the physics engine computed it: ground truth
Measured, // a sensor read it
Reconstructed, // this controller computed it from what it commanded
Estimated, // a state estimator produced it
}
```
A source declares a `{term -> Provenance}` map. `types.rs` already has half of
this, in the one place it was unavoidable:
```rust
/// Body linear velocity. Almost never available: it needs a state estimator,
/// not an IMU. `has_lin_vel` says whether it means anything.
pub lin_vel: [f32; 3],
pub has_lin_vel: bool,
```
Generalising it is the change. A contract asking for a term no source supplies is
**refused at load**, not discovered at tick 40 000.
### The check produces a list, not a verdict
The valuable output is not "ok". It is the terms whose provenance differs from
the one they had in training:
```
locomotion on `device`:
joint_torque trained Simulated, here Measured
base_lin_vel trained Simulated, here Estimated
(6 terms unchanged)
```
**That list is the sim2real gap, enumerated.** Nobody can see it today. It costs
one field per term in the exported contract — every term kk-rl-mjlab exports is
`Simulated`, so producing it is nearly free, and the value is entirely on the
consuming side.
---
## The inference port
### Two phases, because the host owns the model
In the browser the model runs in onnxruntime-web, which is JavaScript. A wasm
module cannot call it and return within one synchronous call. So the tick splits:
**Built — `src/control.rs`.** The cascade decides *which* mode; this decides what
a tick does inside one.
```rust
enum Step {
/// No inference this tick — warm start, mode-switch ramp, or a hold state.
/// The targets are final; read them from `command()`.
Hold,
/// The host must run `observation()` through this mode's model and call
/// `resume`.
Infer { mode: String },
}
fn tick(&mut self, input: &TickInput<'_>, now: Micros) -> Result<Step, Error>;
fn observation(&self) -> &[f32]; // borrowed: no allocation per tick
fn resume(&mut self, action: &[f32]) -> Result<&MotorCommand, Error>;
fn command(&self) -> &MotorCommand; // valid after either phase
```
`Hold` carries nothing, because the command is readable either way and a payload on
one arm would be a second place to read it from. `TickInput` is the bundle of things
only the host knows — the state, the command, whether each is fresh, the latched
buttons, each mode's operator and its task's controls — so everything derivable from
them is derived here instead of subtly differently in three places.
This is **not** a concession to wasm. `rl-wbc-fsm` already works this way: the
output loop runs at 1 kHz and inference at the active contract's `control_hz`
(50 for locomotion, 200 for jump), with the last decoded target held and an EMA
moving toward it in between. Splitting the tick only writes that fact into the
type instead of leaving it in a timer comparison.
It also makes the decisive property testable: `tick` is a pure function of its
`TickInput` and `now`, so a recorded episode replays bit-exactly. The C++ has
this only for `jump_gate.hpp`, and says why — *"pure, time-injected, unit-tested
in isolation from the DDS control loop"*.
### Rate lives in the core, not the host
`control_hz` comes from the active mode's contract, and the core decides which
ticks infer. A host that decided for itself would be free to get it wrong, and
running a policy at the wrong rate aliases it *and* re-scales
`joint_pos_rate_limit`, which is per-tick.
`Bundle::output_rate_hz` returns the fastest mode's rate for the host loop and
shared output filter. Each mode keeps its inference period. With modes at 50
and 200 Hz, the loop runs at 200 Hz and the slower mode infers every fourth
tick. The output filter derives its coefficient from the cutoff and loop rate.
### A non-inferring engine is a state, not an error
`rl-wbc-fsm` ships a `StubEngine` returning zeros, and `is_stub()` so the loop
refuses to call that "running". The interlock that matters:
> **"Zero action == hold still" is not universal.** It holds only while the
> decode baseline is the home pose. For a reference-residual policy a zero action
> means "track the recording exactly" — a full open-loop jump.
So the stub must be visible to the FSM, not hidden behind the port. It is:
`rknn::make_engine` falls back to a stub **with a reason**, `DeviceHost::stub_modes`
reports every one, `controller` prints them at startup and again the first time
such a mode runs, and `Outcome::Inferred` carries `stub` so the loop cannot call
it running.
And a mode whose decode uses a reference **refuses to run at all** while its
engine is a stub. `DeviceHost::note_stub` hands the fact it owns — this engine
is a stub — to `Controller::interlock_stub_engine`, which owns the rule, because
the contract is there: a `reference` block with `residual_action` bars the mode,
anything else does not. A barred mode takes the path a stale-feedback tick
already takes — keep the mode, hold steadily — rather than inventing a second
way to refuse, and `stub_modes` says which mode and why.
Only a residual is barred. For an ordinary policy a stub's zeros **are** the home
pose held, which is what `stub_modes` promises and is true; barring those too
would make a bundle pending conversion useless for no safety gain. The test
carries both directions.
---
## The execution port
The core produces joint targets in **wire order** and nothing else. What differs:
| | |
|---|---|
| mjlab | write actuator targets; mjlab's `<position>` actuators close the loop |
| web | the same, through the WASM model |
| device | a DDS `MotorCommand` with `pos`/`kp`/`kd` and `tau = 0` — the MCU closes the torque loop |
Two invariants the port may not round off:
- **Wire order is not policy order.** `layout.rs` pairs them by joint *name*;
the two never share a type, deliberately. A wrong order is a robot that runs
and is wrong.
- **The clamps are per-tick.** `joint_pos_rate_limit` is rad *per tick*, so a
host running at a different rate silently changes it. It belongs to the core.
---
## The FSM
### States are data; transitions are an ordered list
`rl-wbc-fsm` has no state enum and no transition table: states are strings in
TOML, and transitions are a priority cascade in `fsm.cpp`, recomputed every tick.
The cascade order is the design — safety before operator intent, recovery before
hand-off — and a `condition -> state` map would lose exactly that.
**Built — `src/config.rs`, `src/fsm.rs`.**
So transitions become an **ordered rule list**, first match wins:
```toml
[[fsm.rule]] when = "feedback_stale" enter = "safe"
[[fsm.rule]] when = "tilted" enter = "safe"
[[fsm.rule]] when = "in_state:safe" enter = "@initial"
[[fsm.rule]] when = "warm_start_reached" enter = "@warm_start_ref"
[[fsm.rule]] when = "in_state:warmstart" enter = "@stay"
[[fsm.rule]] when = "button:jump" enter = "jump"
[[fsm.rule]] when = "button:claw_left" enter = "claw_left"
[[fsm.rule]] when = "always" enter = "locomotion"
```
Two rules about the rules:
- **`when` is a closed vocabulary, not an expression language.** An unknown word
fails the load. An expression language here cannot be validated, cannot be
tested, and would need a parser inside the wasm module.
- **The last rule must be `always`**, checked at load. "No rule matched" then
cannot happen, and a test can assert the cascade is total.
Starting vocabulary — every entry is something the C++ already decides on:
| `when` | true when |
|---|---|
| `always` | always; required as the last rule |
| `feedback_stale` | no state sample within `state_timeout_ms` |
| `tilted` | tilt exceeds `tilt_limit`, unless the active state sets `skip_tilt_check` |
| `command_stale` | no operator command within `command_timeout_ms` |
| `in_state:<name>` | the active state is `<name>` |
| `warm_start_reached` | the active state is a warm start and the measured pose reached its target |
| `button:<id>` | the operator's latched mode equals `<id>` |
| `gripper_active` | either gripper is engaged — see below |
`@initial` and `@warm_start_ref` are indirections into `[fsm]`. `@stay` matches
and changes nothing — needed because a cascade without it quietly claims every
tick ends somewhere new. A warm start ignores the operator until it has finished,
which the C++ says with a bare `return` mid-function; here it is a rule, visible
beside the order it depends on. The first draft had no `@stay` and walked a robot
away mid-ramp, because the cascade fell through to `always`. The test that caught
it is `the_ordinary_path`.
`gripper_active` is the one word that parses and can never be true. Its state came
from `robot_control`, which this controller stopped reading when it started reading
the pad directly, and the pad carries no gripper — every host passes `false`. So a
config *using* it is refused at load, for the same reason a rule naming a button
nothing latches is: a rule that can never fire looks exactly like one that merely
has not. The word stays in the vocabulary, because the vocabulary is what a file
may say and this will be true again when something supplies it.
### Entry conditions stay on the state
`skip_tilt_check` suspends the attitude fallback **only while the state is
already active** — the tick that enters it is still checked, so the robot cannot
launch from an already-tipped pose. That is a guard on the state, not an edge, and
it stays where the C++ put it. A config whose safe state sets it fails to load,
for the reason the C++ gives: *"exempting it would leave no attitude protection
anywhere."*
### Switching modes ramps; it does not blend
Entering a mode with a model:
1. record the pose at entry;
2. direct-PD ramp toward **that model's own default pose**, at the ramp gains,
not the policy's;
3. hand off only when the measured pose is within `pose_reach_tol` of it —
**not when the ramp duration elapses**. Soft gains mean the hand-off waits
indefinitely and the robot soft-holds. That is by design; raise the gains.
Set `mode_switch_ramp_s`, `pose_reach_tol`, `ramp_kp` and `ramp_kd` for the
target's actuator and feedback units. The ramp limits the step in commanded
pose when a mode changes.
### Per-mode, and per-mode only
The split the C++ found, kept verbatim:
> This file holds only what is GLOBAL across policies. Each FSM mode's I/O
> contract (obs/action layout, home pose, action_scale, gains) is loaded at
> runtime from that policy's layout, shipped next to its model.
One mode names one model file. Everything about how that policy is driven comes
from its own contract. The TOML adds only bench knobs, composed rather than
overriding:
```
effective kp = contract.kp * state.kp_scale + state.kp_offset
```
The trained gain stays in the contract; the bench adjustment stays in the config;
neither can be mistaken for the other.
Modes are built **up front**, in `Controller::new`, and so are their engines in
`Bundle::into_host`. This paragraph said "lazily on first entry"; building on
first entry would put every way a contract can be refused — a term this host
cannot supply, a missing recording, a model that will not load — behind an
operator pressing a button, on a robot that is already standing. Start-up is
where a refusal is cheap. A switch costs nothing either way.
### A mode may be driven by a recording
A velocity policy needs nothing but its contract: the command is three numbers
and every observation is measurable now. A reference-guided one is not runnable
without the motion it was trained against, so the motion travels with it — a
`reference` block in the contract, and a companion `<name>.trajectory.json`
beside it that the bundler copies in and `trajectory.rs` reads. The contract
declares whether actions are residuals on the recording's commanded targets
and whether the recording begins on mode entry or on an operator event.
Three things about it are load-time refusals rather than runtime surprises,
because each is silent when wrong. A contract naming a recording that is not
beside it is refused when the bundle opens — the alternative is a mode that loads
and fails at its first tick, which is the right answer at the wrong moment. A
table in a different joint order than the robot's is refused, because a permuted
trajectory is a robot moving the wrong limbs and nothing downstream can see it.
And a residual declared on any baseline other than `q_cmd` is refused rather than
read off `q`. The measured state table `q` feeds the policy preview. The
recorded controller targets in `q_cmd` supply the residual baseline. The two
tables can have different sample rates and lengths.
**A recorded motion ends**, which is the one way it is unlike a gait.
`Controller::take_completed` names the mode whose recording has played out, the
host releases that mode's button on its own latch, and the cascade's final
`always` rule catches the robot. The controller does not leave the mode itself:
that would be a second thing deciding modes, which is the whole complaint against
the C++'s jump latch.
### What a command is differs per mode
From `config.hpp`:
> Policies do not agree on what a command is: locomotion wants
> (lin_vel_x, lin_vel_y, yaw_rate), jump wants (jump_x, jump_y, jump_go).
So a mode may declare its own `command_terms`, falling back to the global list.
The controls half of that has since moved: a task's `controls.yaml` is a list of
commands, one entry per command term an operator drives, and the contract
carries it as `operator_controller/2`. A mode can bind velocity and posture
commands to the same devices. `src/operator.rs` reads those bindings, and the
FSM's `command_terms` still supplies the order of the `commands` observation.
---
## Deliberately not copied
- `walk_enter_speed`, `walk_exit_speed`, `walk_exit_hold_s` — parsed by the C++,
read by nothing. Vestigial stand↔walk hysteresis.
- The jump one-shot latch works by rewriting the operator's mode before the FSM
sees it. It is correct and well-tested, but it means the FSM is not the only
thing deciding modes. Here `button:jump` is an ordinary cascade rule, and the
motion starts when the mode is entered, once the switch-in ramp is done. A
recording that wants a person to pick the moment declares a `go_event` instead:
a binding the mode reads by name rather than a transition, because a push-off
40 ms wide is an event inside a mode and not a mode. Leaving is the cascade's too:
when the recording plays out, `Controller::take_completed` names the mode, the
host releases that button on its own latch, and the final `always` rule catches
the robot. One way out, rather than a second one written beside it.
- The controller reads raw pad input. The deployment manifest defines mode
switches, and each task's `controls.yaml` defines inputs within a mode.
---
## What exists today
`README.md` describes the crate as it stands: a state machine whose states are a
file, three hosts that call it, and `src/bin/controller.rs`, which on the robot is
the whole program. This section used to read "no state machine, no `main.rs`" and
to warn that the README's "two modes" were `rl-wbc-fsm`'s; neither is true now.
Built as of this document:
| | |
|---|---|
| the `device` feature split and the wasm check | `Cargo.toml`, `build.rs`, `lib.rs` |
| provenance, capability, divergence list | `src/source.rs` |
| modes and the ordered rule list, with its load-time checks | `src/config.rs` |
| the words a control may be named, read from the dictionary | `src/vocabulary.rs` |
| the cascade, and the guards that belong to a state | `src/fsm.rs` |
| the two-phase tick, four regimes, per-mode runtimes | `src/control.rs` |
| each mode's reading of the pad and keys, command shaping | `src/operator.rs` |
| the pad's manual as data (`pad_guide`) | `src/guide.rs` |
| a task's deploy hook, compiled in from `tasks/**/deploy/lib.rs` by `build.rs` | `src/hook.rs` |
| the recorded motion a reference-guided mode is driven by | `src/trajectory.rs` |
| a recorded run replayed, and where a host differs | `src/reference.rs` |
| the bundle format, one reader for every host with a filesystem | `src/bundle.rs` |
| the robot's host: freshness, the button latch, engine dispatch | `src/device.rs` |
| the browser's host: the JS-facing `WebFsm` | `src/web.rs` |
| `play`'s host: the pyo3 `Fsm` | `src/py.rs` |
| the robot's program: the loop, the two clocks, the shutdown path | `src/bin/controller.rs` |
156 tests. Two of them are control groups, because both of these modules replace
something that was previously written into code and would pass every obvious
assertion while ignoring the file: `the_cascade_is_the_file_s_rather_than_agreeing_with_it`
moves the tilt rule below the operator's button and requires a tilted robot to
jump, and `the_check_reads_what_training_recorded_rather_than_assuming_it` trains
a term as reconstructed and requires simulation to be reported as the divergence.
`control.rs` earned a third test the hard way. Its first `publish` used
`cmd.pos` as both the standing target and the filtered output, so after one
sample the filter converged on itself and stopped -- at 50 Hz on a 1 kHz loop,
nineteen ticks in twenty did nothing. On a robot that reads as a controller
that feels sluggish, with nothing logged.
`the_output_filter_starts_where_the_robot_is` requires convergence to the
decoded target, and measured 0.0295 against 0.25.
`device.rs`'s two latch tests were run against deliberately broken versions
before being trusted. A latch acting on the trigger's *level* rather than its
rising edge fails `a_held_trigger_is_one_request` -- but only because the press
is four frames long: at three, level-acting toggles an odd number of times and
ends latched, passing for exactly the wrong reason. Dropping `latch.forget()`
on a stale command fails `a_latched_button_does_not_outlive_the_operator`.
## Hosts
`device.rs` is the first, and the one whose I/O already existed. It owns what
the core may not: a clock (so `feedback_stale` and `command_stale` can be
decided), the engines, and the latch that turns the operator's trigger pulses
into a `button:<name>` the rules can match. `pump` is compiled against the real
`DdsIo` -- four calls, and that compile is the check that the shapes still fit.
`[[fsm.button]]` names one thing the operator can ask for and which control asks
it: a pad button or a keyboard key, one device per entry, and two entries with
one name share its latch -- the pad's A and the keyboard's Space are both the
jump. It also carries the gesture (`rise`, `fall`, `hold`, `toggle`, or a click
count `single` … `quintuple` counted inside `[fsm] click_window_ms`), an optional
`with` modifier (a pad button, or `ctrl` / `shift` / `alt` for a key), `from`
(the states it may be pressed in) and `leaves` (latches it lets go of). Pad
buttons and keyboard keys each have their own bindings.
Every one of its load-time checks is a binding that loads and then does nothing,
or does two things: a rule naming a button no entry declares; an entry with
neither a pad button nor a key, or with both; two entries on the same control
with the same gesture; a `pad` or a `with` the robot's service does not publish;
a `key` the dictionary does not list; a `with` that is not a modifier of its
entry's device; a `from` naming no state; a `leaves` that is not a moment or
names nothing that latches; a control used as a modifier that is also bound to
`hold`; a binding no rule reacts to; and any key `[fsm]` does not read.
`[fsm] exclusive` makes the latches one at a time, the last switch pressed
being the mode.
**`event = true` is the exception to the last of those.** A control read by a
*mode* rather than by a rule — a recorded motion's `go`, matched against the
contract's `reference.go_event` — has no rule reacting to it, which is exactly
the shape of a dead binding. So it is declared rather than inferred, and
`Controller::new` checks it in both directions: an `event` no contract names, and
a `go_event` no button declares, are each a control that loads and does nothing.
The words themselves come from `controller/vocabulary.json`, schema
`kk-control-vocabulary/2`, `include_str!`-ed so a board carries no file and a
browser fetches nothing. It was `kk-gamepad-vocabulary/1` while the pad was all
it described; `keys` joined it because `key` was the one field nothing checked —
`pad` and `with` were held against the dictionary from the start and `key` took
any string at all, so a typo loaded, bound nothing, and looked exactly like a
rule that had not fired.
In a bundle the halves come from two files on purpose. `deploy/manifests.json` says
which button or key switches into which mode, and from which modes, and its
top-level `buttons` list names bindings for other events, such as a recording's
`go`. Each task's `controls.yaml` defines the bindings inside a mode. Neither is wrong on its own when both pick the same control, and
neither can catch it, so `Bundle::open` is where the sum is checked
(`operator::check_switches`, which every host's operators run again): opening the
bundle is the first place both exist.
`web.rs` is the second, and the one the two-phase tick exists for: the model runs
in onnxruntime-web, which is JavaScript, so a wasm module cannot call it and
return inside one synchronous call. Everything crosses as a flat `Float32Array`
-- no structs per tick -- and the configuration crosses as text once, because a
browser has no filesystem.
```bash
cargo build --release --target wasm32-unknown-unknown --no-default-features --features web
wasm-bindgen --target web --no-typescript --out-dir <dir> \
target/wasm32-unknown-unknown/release/mjrl_fsm.wasm
```
776 KB of wasm and 19 KB of glue, against MuJoCo's 10.2 MB and onnxruntime's
14 MB in the same page.
**It found a bug on first contact with a real file, which is the point.** Pointed
at an exported `layout.json`, it refused: *observation term 'velocity_commands'
is not implemented*. `rl-wbc-fsm` has mapped that name to the builder's
`commands` since `contract.cpp:27` was written -- locomotion names its command
block `velocity_commands`, the older jumps `jump_command` / `jump_go`, and all
three feed one term. This crate never had the mapping, so it would have refused
every policy in circulation, including `rl-wbc-fsm`'s own
`locomotion.isaac_layout.json`. It went unnoticed because the tests build their
own contracts and named the term `commands` -- what the builder wanted. Two
implementations agreeing with themselves.
`py.rs` is the third, and the one that makes an FSM something you can look at before
you ship it: until it existed, the only way to find out what a `controller.toml` did
was to bundle it and upload it. `Fsm(config, contracts, robot, now_us=0,
trajectories=None, operators=False)` — `trajectories` is `{mode: text}` for the
recordings those contracts name, optional rather than a positional every caller
would pass `{}` for; with `operators`, keys (`set_key`) and whole pad frames
(`set_pad_frame`) are read through each mode's own controls, as on the robot, which
is what `play --app` runs. State crosses as plain sequences of `f32`; errors are
`ValueError` with the reason in them. Buttons are `set_key(key, down)` and
`set_pad(button, down)` — **state, both edges**, because `rise` and `fall` are both
in the vocabulary and a one-shot "was pressed" could only drive half of it — and
`pad_guide()` is what a host draws the pad from; `bindings` lists the mode switches.
The earlier `press_key` and `key_bindings` are gone for exactly that reason: a
one-shot press cannot produce a `fall`.
What `device.rs` does not include is a binary. That is `src/bin/controller.rs`
now: the 1 kHz loop, the two clocks it must not derive one from the other, the
`--dry-run` and `--check-reference` paths, and a shutdown that leaves through the
safe state by telling the FSM the bus has gone stale rather than by writing a
second shutdown behaviour. None of it can be tested off the board.
## Open
- **Provenance in the exported contract.** Needs a field per observation term in
kk-rl-mjlab's exporter. Cheap to produce, and the whole value of the divergence
list depends on it.
- **Per-mode command vocabulary**, above.
+121
View File
@@ -0,0 +1,121 @@
# Cross-build the FSM for the RK3576 board, from a host of any kind.
#
# Cross-compiling, not emulating. Running an arm64 container under qemu needs a
# `binfmt` registration carrying the `F` flag, which is host-wide state a build
# should not be quietly changing -- and it is far slower. This image is x86_64
# and targets aarch64 through a normal cross toolchain.
#
# bash deploy/fsm/docker-build.sh
#
# ## Why ubuntu:22.04 and not something newer
#
# **The base image's glibc is a hard floor on the board.** The RK3576 image is
# Ubuntu 22.04 (glibc 2.35); a binary linked against Debian bookworm's 2.36 dies
# on the board with `version GLIBC_2.36 not found` -- at startup, on the robot,
# nowhere near the build that caused it. 22.04 here matches the rootfs the board
# actually runs.
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential cmake git curl ca-certificates pkg-config \
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-dev-arm64-cross \
clang libclang-dev \
&& rm -rf /var/lib/apt/lists/*
# CycloneDDS is needed twice, for two different reasons:
#
# - **x86_64**, for `idlc`. It is a code generator that runs on the build
# machine, so it must be a host binary whatever the target is.
# - **aarch64**, for `libddsc.so`, which the binary links against.
#
# The ref is an argument because the shared-library soname has to match the
# board's. The board carries `libddsc.so.11`; a CycloneDDS that produces a
# different soname builds fine here and then cannot find its library at runtime.
# `docker-build.sh` checks the built soname rather than trusting this default.
ARG CYCLONEDDS_REF=master
ARG CYCLONEDDS_URL=https://github.com/eclipse-cyclonedds/cyclonedds.git
RUN git clone --depth 1 --branch "${CYCLONEDDS_REF}" "${CYCLONEDDS_URL}" /src/cyclonedds
# 1) Host build -- only idlc is kept.
RUN cmake -S /src/cyclonedds -B /build/host \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/cyclonedds-host \
-DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON \
&& cmake --build /build/host -j"$(nproc)" \
&& cmake --install /build/host \
&& rm -rf /build/host
# 2) Target build -- the library the FSM links against. `-DBUILD_IDLC=OFF`
# because an aarch64 idlc could not run here anyway.
RUN cmake -S /src/cyclonedds -B /build/target \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/cyclonedds-aarch64 \
-DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ \
-DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=OFF \
&& cmake --build /build/target -j"$(nproc)" \
&& cmake --install /build/target \
&& rm -rf /build/target /src/cyclonedds
ENV PATH="/opt/cyclonedds-host/bin:/usr/local/cargo/bin:${PATH}"
ARG RUST_VERSION=1.98.0
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| env CARGO_HOME=/usr/local/cargo RUSTUP_HOME=/usr/local/rustup \
sh -s -- -y --profile minimal --default-toolchain "${RUST_VERSION}" \
--target aarch64-unknown-linux-gnu --no-modify-path
ENV CARGO_HOME=/usr/local/cargo RUSTUP_HOME=/usr/local/rustup
# `build.rs` reads CYCLONEDDS_HOME for the headers and the link path. It points
# at the **aarch64** tree; idlc comes from PATH, which is the host tree above.
ENV CYCLONEDDS_HOME=/opt/cyclonedds-aarch64 \
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar
# bindgen parses the generated headers with libclang, which defaults to the host
# target. Without this it would compute x86_64 struct layouts for an aarch64
# binary -- the two agree here, but silently, and would stop agreeing the moment
# a message gained a pointer or a long.
ENV BINDGEN_EXTRA_CLANG_ARGS_aarch64_unknown_linux_gnu="--target=aarch64-linux-gnu --sysroot=/usr/aarch64-linux-gnu"
# The second cross build: `play --app` on Windows, the controller as a CPython
# extension for `win-amd64`. The GNU target rather than MSVC, because MinGW is
# an apt package and MSVC's libraries come only with a licence to accept. The
# DLL it makes imports `python3.dll` and system DLLs and nothing else: no
# libgcc or winpthread to ship beside it.
#
# Its own layer, after the board's, so that adding it to an image built before
# it reuses every layer above -- CycloneDDS is not compiled again. The macOS
# layer below is added the same way.
RUN apt-get update && apt-get install -y --no-install-recommends gcc-mingw-w64-x86-64 \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add x86_64-pc-windows-gnu
# The third: `play --app` on macOS, one universal2 file carrying arm64 and
# x86_64, so every Mac and every Python build on it -- Homebrew's, conda's,
# python.org's universal2 one, one under Rosetta -- loads the same file and
# dyld picks the slice. Linked by zig through cargo-zigbuild, which carries
# libSystem's stubs, so no Apple SDK is needed and none has a licence to accept;
# rustc's warning that `xcrun` cannot find one is expected and harmless. zig
# also signs the arm64 slice ad hoc, without which Apple Silicon kills the
# process that loads it. Minimum macOS 11.7.1, zig 0.13's floor.
#
# Both pinned: which zig links a binary is not visible in it.
ARG ZIG_VERSION=0.13.0
ARG CARGO_ZIGBUILD_VERSION=0.23.4
RUN curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-linux-x86_64-${ZIG_VERSION}.tar.xz" \
| tar -xJ -C /opt \
&& ln -s "/opt/zig-linux-x86_64-${ZIG_VERSION}/zig" /usr/local/bin/zig \
&& cargo install --locked "cargo-zigbuild@${CARGO_ZIGBUILD_VERSION}" \
&& rm -rf /usr/local/cargo/registry \
&& rustup target add aarch64-apple-darwin x86_64-apple-darwin
WORKDIR /work
ENTRYPOINT ["cargo"]
CMD ["build", "--release", "--target", "aarch64-unknown-linux-gnu", "--features", "rknn"]

Some files were not shown because too many files have changed in this diff Show More