release: v0.1 generic robot training and application platform
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
---
|
||||
name: bundle-manual
|
||||
description: Translate a built bundle's manual.en.json into Chinese as manual.zh.json and pack it into the .app. Use right after `python scripts/deploy.py` writes a bundle -- every bundle an agent builds ships with the Chinese manual as well as the English one -- and whenever a bundle is found with manual.en.json and no manual.zh.json, or when asked for "操作说明", "中文说明", "按键说明" or "manual" of a bundle/app.
|
||||
---
|
||||
|
||||
# The bundle's manual, in Chinese
|
||||
|
||||
`scripts/deploy.py` writes `manual.en.json` (`kk-bundle-manual/1`) into every
|
||||
bundle: each pad control and each key the bundle reads, what it does in each
|
||||
mode, how each mode is reached and left, and every mode switch. It is generated
|
||||
from the controller's own account of its controls (`padGuide`), so it cannot
|
||||
disagree with the bindings. The pad and the keyboard are two paths since
|
||||
2026-09-29 and the manual lists them apart: in each mode `controls` is the pad
|
||||
and `keys` the keyboard, and each switch says which `device` it is on. The
|
||||
Chinese version is a translation of it, and this skill is how it is made: the
|
||||
build cannot write natural Chinese, and a person or an agent can.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Find the bundle directory. `deploy.py` prints it
|
||||
(`[deploy] bundled <commit> -> …/out/bundle_<ts>/<name>`) and then prints the
|
||||
`--translate` command with that path already filled in. It holds
|
||||
`manual.en.json` and `bundle.json`.
|
||||
2. Read `manual.en.json` in full.
|
||||
3. Write `manual.zh.json` somewhere outside the bundle (a scratch file):
|
||||
- the same JSON, field for field, in the same order;
|
||||
- `"language": "zh"`;
|
||||
- translate **only** the text fields: `source`, `summary`, `enter`, `leave`,
|
||||
`keyboard`, `pad`, `does` -- and `does` wherever it is: in each mode's
|
||||
`controls` (the pad), in each mode's `keys` (the keyboard), and in
|
||||
`switches`. A `pad` that is `null` (a keyboard switch) stays `null`;
|
||||
- leave every other value exactly as it is -- `schema`, `bundle`, `mode`,
|
||||
`task`, `default`, `control`, `with`, `device`, `from`, the `key` of each
|
||||
entry in a mode's `keys` (`W`, `;`, `Space`, `↑`), and the `keys`
|
||||
lists of `switches` (`Space`, `Ctrl + 1`, `Num 1`). They are identifiers,
|
||||
and the check below refuses a translation that changes one.
|
||||
4. Add it to the bundle, which checks it and packs the `.app` again:
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --translate out/bundle_<ts>/<name> --language zh --manual <scratch file>
|
||||
```
|
||||
|
||||
It refuses a file whose non-text fields differ from the English one, and
|
||||
holds the result to `deploy/app.schema`. Fix what it names and run it again.
|
||||
5. If the `.app` is going to be committed (bundles are committed under
|
||||
`out/bundle_<ts>/<name>.app`), commit it after this step, not before: the
|
||||
translation changes the archive.
|
||||
|
||||
## How to write it
|
||||
|
||||
For someone holding a gamepad or sitting at a keyboard, in plain Chinese. Keep
|
||||
mode names as identifiers where they name a mode (`进入 jump`), and say what the
|
||||
mode is in the sentence when the English does. The manual lists each mode's
|
||||
keys on their own; say what a key does, never which pad control it is -- the
|
||||
English never says that either.
|
||||
|
||||
| English | 中文 |
|
||||
|---|---|
|
||||
| left stick / right stick | 左摇杆 / 右摇杆 |
|
||||
| up and down / left and right | 上下 / 左右 |
|
||||
| up: …; down: … | 上:……;下:…… |
|
||||
| left trigger (LT) / right trigger (RT) | 左扳机(LT)/ 右扳机(RT) |
|
||||
| left bumper (LB) / right bumper (RB) | 左肩键(LB)/ 右肩键(RB) |
|
||||
| right stick click (R3) | 右摇杆按下(R3) |
|
||||
| D-pad / D-pad up | 十字键 / 十字键上 |
|
||||
| hold Menu, then D-pad up | 按住 Menu 再按十字键上 |
|
||||
| Menu, pressed and let go on its own | 单独按下再松开 Menu |
|
||||
| walk forward / walk back | 前进 / 后退 |
|
||||
| step left / step right | 左移 / 右移 |
|
||||
| turn left / turn right | 左转 / 右转 |
|
||||
| twist left / twist right | 左扭 / 右扭 |
|
||||
| nose down / nose up | 低头 / 抬头 |
|
||||
| roll, right side down / left side down | 横滚,右侧低 / 横滚,左侧低 |
|
||||
| stand taller / crouch lower | 升高 / 降低 |
|
||||
| (moves it; let go, it stays) | (持续移动,松开即停留) |
|
||||
| with right stick click (R3) held -- … | 按住右摇杆(R3)时——…… |
|
||||
| hold for the other stick layer | 按住切换到摇杆的另一层 |
|
||||
| tapped -- pressed and let go without the stick moving -- puts the moved axes back at rest | 轻点(按下后摇杆不动就松开):被移动的量回到静止 |
|
||||
| let go of everything: every control back to rest | 全部松开:所有操作回到静止 |
|
||||
| switch to X; press again to switch back | 进入 X;再按一次退出 |
|
||||
| (from X, Y only) | (仅在 X、Y 中有效) |
|
||||
| leave X, Y -- whichever is on -- back to locomotion | 退出 X、Y 中正在进行的那一个,回到 locomotion |
|
||||
| click A once / double-click A / triple-click A | 单击 A / 双击 A / 三连击 A |
|
||||
| click A four times / click A five times | 四连击 A / 五连击 A |
|
||||
| switch to X; double-click again to switch back | 进入 X;再双击一次退出 |
|
||||
|
||||
A task's own words -- `closes the left claw as far as it is held`, from its
|
||||
`controls.yaml` -- are translated like any other text: `按多深,左爪就夹多紧`;
|
||||
`holds the carried arm straight out, thumb up, while held` is
|
||||
`按住时手臂向前伸直,拇指朝上`, and thumb-web up is `虎口朝上`. A claw mode lists
|
||||
only its own side's claw (the left claw's lists `LT` and Space, not `RT`), so
|
||||
translate what is there and add nothing.
|
||||
|
||||
## Why a separate step
|
||||
|
||||
The English is the build's and the Chinese is a translation of it, never the
|
||||
other way round: a manual written in Chinese first would have no generator to
|
||||
hold it to the bindings. `--translate` is what keeps the two the same manual in
|
||||
two languages -- it compares every identifier -- and it is also why a bundle
|
||||
built without an agent still ships the English one.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: controls
|
||||
description: Configure operator keys, pad bindings, releases, or legacy FSM mode switches using the shared control vocabulary and schema.
|
||||
---
|
||||
|
||||
# Bind operator input
|
||||
|
||||
Read [Controls](../../../docs/CONTROLS.md). Use
|
||||
`python -m controller --vocabulary` as the source of device names.
|
||||
|
||||
For the locked locomotion templates, generate the ordered body-frame
|
||||
velocity binding in the robot's asset directory:
|
||||
|
||||
```bash
|
||||
python .claude/skills/controls/scripts/write_controls.py --task <robot_id>
|
||||
```
|
||||
|
||||
Use `--profile differential` for `wheeled_biped`. W/S drive forward and
|
||||
backward and A/D turn left and right. The keyboard leaves lateral motion
|
||||
unbound. The required Lx pad channel stays in the layout and has zero range.
|
||||
The default `omni` profile keeps A/D for lateral motion and J/L for turning.
|
||||
Use `--out` to specify another destination file. Keep all three velocity
|
||||
channels in order. Use `--bind AXIS=SOURCE+` or `--bind AXIS=SOURCE-` for
|
||||
named changes and `--release BUTTON` when specified. Point `robot_spec.py`
|
||||
at this file.
|
||||
For deterministic modes, generate the binding with `onboard.py setup-tier2`.
|
||||
|
||||
Validate the final YAML with `controller.controls.load_controls`. Exercise
|
||||
both ends of the active axes and release on the selected model. Repeat native
|
||||
verification after changing bindings that a portable app will package.
|
||||
|
||||
A legacy FSM mode switch belongs in the deployment manifest. Check it against
|
||||
the mode's operator bindings and validate the legacy bundle. Native app mode
|
||||
selection uses `--app-mode`.
|
||||
@@ -0,0 +1,293 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Write a locomotion binding using the shared keyboard and gamepad vocabulary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[4]
|
||||
sys.path.insert(0, str(REPO))
|
||||
|
||||
# After the path insert, necessarily: this runs from a skill directory, not from
|
||||
# an installed package, and `controller` is at the repository root.
|
||||
from controller import vocabulary # noqa: E402
|
||||
|
||||
#: Command axis -> (unit, which way is positive).
|
||||
#:
|
||||
#: The velocity-tracking vocabulary the observation term uses; the directions
|
||||
#: are the body frame's, ROS REP-103.
|
||||
AXIS_MEANING = {
|
||||
"lin_vel_x": ("m/s", "forward"),
|
||||
"lin_vel_y": ("m/s", "left"),
|
||||
"ang_vel_z": ("rad/s", "counter-clockwise"),
|
||||
}
|
||||
|
||||
#: Which stick drives which axis, by default, and with what sign.
|
||||
#:
|
||||
#: evdev reports positive stick motion to the right and downward.
|
||||
#: Confirm these signs on the intended input device during verification.
|
||||
DEFAULT_BINDING = {
|
||||
"lin_vel_x": ("Ly", -1),
|
||||
"lin_vel_y": ("Lx", -1),
|
||||
"ang_vel_z": ("Rx", -1),
|
||||
}
|
||||
|
||||
#: The keyboard: command axis -> (the keys towards its positive direction, the
|
||||
#: keys away from it). W S and the arrows walk forward and back, A D and the
|
||||
#: arrows step left and right (the arrows asked for on 2026-09-28), J L turn --
|
||||
#: the layout as the
|
||||
#: user decided it on 2026-09-29. Two keys on one direction are summed while
|
||||
#: both are held. Letting go of the keys is taking your hands off; there is no
|
||||
#: key for it (`centre`, retired 2026-09-27).
|
||||
#:
|
||||
#: Every letter is also one of MuJoCo's viewer shortcuts -- W flips the scene to
|
||||
#: wireframe -- and that was accepted for a keyboard laid out for the hands; the
|
||||
#: dictionary's `_keys_note` has the table.
|
||||
KEYS_FOR = {
|
||||
"lin_vel_x": (("key_w", "key_up"), ("key_s", "key_down")),
|
||||
"lin_vel_y": (("key_a", "key_left"), ("key_d", "key_right")),
|
||||
"ang_vel_z": (("key_j",), ("key_l",)),
|
||||
}
|
||||
|
||||
DIFFERENTIAL_KEYS = {
|
||||
"lin_vel_x": KEYS_FOR["lin_vel_x"],
|
||||
"lin_vel_y": None,
|
||||
"ang_vel_z": (("key_a", "key_left"), ("key_d", "key_right")),
|
||||
}
|
||||
|
||||
KeyDirections = tuple[tuple[str, ...], tuple[str, ...]]
|
||||
|
||||
#: The keys that let go of everything, as the pad's release button does: Esc.
|
||||
#: The keyboard's own, whatever button `--release` gives the pad. It was B until
|
||||
#: Control-agent 3.1 put the right claw there (2026-09-29).
|
||||
RELEASE_KEYS = ("key_escape",)
|
||||
|
||||
#: Seconds a key is held for its direction to reach full deflection; it is back
|
||||
#: at rest when the key comes up. Asked for on 2026-09-26, measured against nothing.
|
||||
FULL_AFTER_S = 2.0
|
||||
|
||||
|
||||
def parse_bind(entry: str) -> tuple[str, str, int]:
|
||||
"""`lin_vel_x=Ly-` -> ("lin_vel_x", "Ly", -1)."""
|
||||
if "=" not in entry:
|
||||
raise SystemExit(f"[controls] --bind wants AXIS=SOURCE[+|-], got {entry!r}")
|
||||
axis, source = entry.split("=", 1)
|
||||
if not source.endswith(("+", "-")):
|
||||
raise SystemExit(
|
||||
f"[controls] --bind {entry!r} has no sign. Write `{axis}={source}-` or "
|
||||
f"`{axis}={source}+`.\n"
|
||||
f" Not defaulted: a sign is the one value here whose wrong "
|
||||
f"answer looks exactly like the right one."
|
||||
)
|
||||
return axis, source[:-1], (-1 if source.endswith("-") else 1)
|
||||
|
||||
|
||||
def check(name: str, kind: str, allowed) -> None:
|
||||
if name in allowed:
|
||||
return
|
||||
absent = vocabulary.absent()
|
||||
extra = f"\n {absent[name]}" if name in absent else ""
|
||||
raise SystemExit(
|
||||
f"[controls] {name!r} is not {kind} the robot's pad service publishes.\n"
|
||||
f" It publishes: {' '.join(allowed)}{extra}\n"
|
||||
f" python -m controller --vocabulary"
|
||||
)
|
||||
|
||||
|
||||
def keyboard_for(axes: list[str], profile: str) -> dict[str, KeyDirections | None]:
|
||||
"""Check each selected keystroke against the dictionary."""
|
||||
mapping = DIFFERENTIAL_KEYS if profile == "differential" else KEYS_FOR
|
||||
out = {}
|
||||
for axis in axes:
|
||||
directions = mapping[axis]
|
||||
if directions is not None:
|
||||
for stroke in (*directions[0], *directions[1]):
|
||||
try:
|
||||
vocabulary.keystroke(stroke)
|
||||
except ValueError as exc:
|
||||
raise SystemExit(f"[controls] {exc}\n python -m controller "
|
||||
"--vocabulary") from None
|
||||
out[axis] = directions
|
||||
return out
|
||||
|
||||
|
||||
def render(task: str, axes: list[str], bind: dict[str, tuple[str, int]],
|
||||
release: str, keys: dict[str, KeyDirections | None]) -> str:
|
||||
out = [
|
||||
f"# {task} operator controls -- how a person's input becomes the command the",
|
||||
"# policy observes.",
|
||||
"#",
|
||||
"# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes",
|
||||
"# every control name from `controller/vocabulary.json`. Run",
|
||||
"# `python -m controller --vocabulary` to see the list; it is the same one the",
|
||||
"# deployed controller refuses a binding against.",
|
||||
"#",
|
||||
"# The pad and the keyboard are two paths, each bound straight to the command",
|
||||
"# axes -- a stick with a sign, a key with the direction it pushes -- and",
|
||||
"# neither names a control of the other. Either device drives alone; the one",
|
||||
"# touched last drives.",
|
||||
"#",
|
||||
"# Two rules this file inherits from the specification it replaced:",
|
||||
"# * Every value carries its source. A number without one cannot be told",
|
||||
"# apart from a guess.",
|
||||
"# * A consumer fails on a missing or malformed value rather than",
|
||||
"# substituting a default. A sign that defaults to +1 is a robot that walks",
|
||||
"# backwards while the screen looks perfectly normal.",
|
||||
"",
|
||||
"schema_version: 2",
|
||||
"",
|
||||
"command:",
|
||||
" - term: twist",
|
||||
" # The observation term these numbers are written into.",
|
||||
" feeds: velocity_commands",
|
||||
" # The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).",
|
||||
" frame: body",
|
||||
" # Order is the order the command term writes them, and therefore the order",
|
||||
" # they appear in the observation.",
|
||||
" axes:",
|
||||
]
|
||||
for axis in axes:
|
||||
unit, positive = AXIS_MEANING[axis]
|
||||
out += [f" - name: {axis}", f" unit: {unit}", f" positive: {positive}"]
|
||||
out += [
|
||||
" scaling:",
|
||||
" # Full deflection is the edge of the range the policy was **trained** on,",
|
||||
" # not an absolute speed -- so a stick means \"as fast as this policy was",
|
||||
" # ever asked to go\" and cannot ask for something off-distribution.",
|
||||
" rule: full_deflection_is_range_edge",
|
||||
" formula: axis * (range_max if axis >= 0 else -range_min)",
|
||||
" # A range like (-0.5, 1.0) gives forward twice what it gives back, which is",
|
||||
" # what the policy trained on; taking the larger end for both quietly",
|
||||
" # commands something it never saw.",
|
||||
" ends_scaled_separately: true",
|
||||
"",
|
||||
"devices:",
|
||||
" gamepad:",
|
||||
" # The stick position *is* the command.",
|
||||
" scheme: absolute",
|
||||
" layout: xbox",
|
||||
" axes:",
|
||||
]
|
||||
for axis in axes:
|
||||
source, sign = bind[axis]
|
||||
if keys[axis] is None:
|
||||
out.append(" # The wheeled template scales this required pad channel to zero.")
|
||||
out.append(f" {axis}: {{ source: {source}, sign: {sign} }}")
|
||||
out += [
|
||||
" # MEASURED: the signs are not measured against the robot's pad service.",
|
||||
" # `controller/xbox.py` states the same evdev convention -- ABS_Y positive",
|
||||
" # is **down**, ABS_X positive is **right** -- and the service passes both",
|
||||
" # through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on",
|
||||
" # the robot is what would settle it.",
|
||||
" #",
|
||||
" # The service applies a 0.15 deadband and **rescales**, so full deflection",
|
||||
" # still reaches ±1. A consumer that applies its own on top narrows the",
|
||||
" # range a person can ask for and cannot tell that it has.",
|
||||
" deadzone: device_reported_rescaled",
|
||||
" # Hands the command back to the random sampler in `play --task`; on a robot, which",
|
||||
" # has no sampler, it lets go of everything.",
|
||||
f" release_button: {release}",
|
||||
"",
|
||||
" keyboard:",
|
||||
" # Keystrokes bound to what they do: each pushes one direction of one axis,",
|
||||
" # \"+\" being the axis's own positive direction as `positive:` above words",
|
||||
" # it, so the keyboard has no signs of its own -- there is no stick for a",
|
||||
" # sign to be relative to. A virtual pad read through the pad's mapping",
|
||||
" # until 2026-09-29.",
|
||||
" scheme: keys",
|
||||
" # Seconds a key is held for full deflection: the axis climbs linearly",
|
||||
" # until then and is back at rest the moment the key comes up, as a stick",
|
||||
" # let go of. Asked for on 2026-09-26; measured against nothing.",
|
||||
f" full_after_s: {FULL_AFTER_S}",
|
||||
" axes:",
|
||||
]
|
||||
for axis, directions in keys.items():
|
||||
if directions is None:
|
||||
out.append(f" {axis}: {{unbound: lateral command range is zero}}")
|
||||
else:
|
||||
plus, minus = directions
|
||||
out.append(f' {axis}: {{ "+": [{", ".join(plus)}], "-": [{", ".join(minus)}] }}')
|
||||
out += [
|
||||
" # Lets go of everything, as the pad's release button does.",
|
||||
f" release: [{', '.join(RELEASE_KEYS)}]",
|
||||
"",
|
||||
]
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(prog="write_controls.py", description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--task", required=True, help="robot id, e.g. sample_biped")
|
||||
ap.add_argument(
|
||||
"--profile", choices=("omni", "differential"), default="omni",
|
||||
help="keyboard profile. Use differential for the wheeled-biped template",
|
||||
)
|
||||
ap.add_argument("--axes", default=",".join(AXIS_MEANING),
|
||||
help="command axes in order (default: all three)")
|
||||
ap.add_argument("--bind", action="append", default=[], metavar="AXIS=SOURCE[+|-]",
|
||||
help="which stick drives an axis and with what sign. Repeat. "
|
||||
"Omitted axes take the default binding.")
|
||||
ap.add_argument("--release", default="B",
|
||||
help="the button that hands control back to the sampler")
|
||||
ap.add_argument("--out", type=Path, help="output (default: assets/<robot_id>/controls.yaml)")
|
||||
ap.add_argument("--force", action="store_true", help="overwrite an existing file")
|
||||
ap.add_argument("--print", action="store_true", help="write nothing; print it")
|
||||
args = ap.parse_args()
|
||||
if re.fullmatch(r"[a-z][a-z0-9_]*", args.task) is None:
|
||||
ap.error("--task must be a lowercase robot id")
|
||||
|
||||
axes = [a.strip() for a in args.axes.split(",") if a.strip()]
|
||||
if args.profile == "differential" and axes != list(AXIS_MEANING):
|
||||
ap.error("the differential profile requires all three command axes in template order")
|
||||
for axis in axes:
|
||||
if axis not in AXIS_MEANING:
|
||||
raise SystemExit(
|
||||
f"[controls] {axis!r} is not a command axis this environment writes.\n"
|
||||
f" It writes: {' '.join(AXIS_MEANING)}"
|
||||
)
|
||||
|
||||
known_axes = vocabulary.axes()
|
||||
bind = {a: DEFAULT_BINDING[a] for a in axes}
|
||||
for entry in args.bind:
|
||||
axis, source, sign = parse_bind(entry)
|
||||
if axis not in axes:
|
||||
raise SystemExit(f"[controls] --bind names {axis!r}, which is not in --axes")
|
||||
if args.profile == "differential" and axis == "lin_vel_y":
|
||||
ap.error("the differential profile has a zero lateral command range")
|
||||
check(source, "an axis", known_axes)
|
||||
bind[axis] = (source, sign)
|
||||
|
||||
driven = [bind[a][0] for a in axes]
|
||||
if len(set(driven)) != len(driven):
|
||||
raise SystemExit(
|
||||
f"[controls] two command axes share a stick: {driven}. Splitting a stick's "
|
||||
f"travel between two axes is an explicit schema edit."
|
||||
)
|
||||
check(args.release, "a button", vocabulary.buttons())
|
||||
|
||||
text = render(args.task, axes, bind, args.release, keyboard_for(axes, args.profile))
|
||||
if args.print:
|
||||
print(text, end="")
|
||||
return 0
|
||||
|
||||
out = args.out or REPO / "assets" / args.task / "controls.yaml"
|
||||
if out.exists() and not args.force:
|
||||
raise SystemExit(f"[controls] {out} exists; pass --force to replace it, or --out")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(text, "utf-8")
|
||||
shown = out.relative_to(REPO) if out.is_relative_to(REPO) else out
|
||||
print(f"[controls] wrote {shown}")
|
||||
for axis in axes:
|
||||
source, sign = bind[axis]
|
||||
disabled = " (zero range)" if args.profile == "differential" and axis == "lin_vel_y" else ""
|
||||
print(f" {axis:<12}{'-' if sign < 0 else '+'}{source}{disabled}")
|
||||
print(f" release on {args.release}, and on the keyboard "
|
||||
f"{' '.join(RELEASE_KEYS)}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: deploy
|
||||
description: Export, verify and package a robot controller as a native MuJoCo .app or a supported legacy board/web bundle. Use for deployment contracts, ONNX/RKNN conversion, runtime parity, bundle creation, or an explicitly requested hardware deployment.
|
||||
---
|
||||
|
||||
# Verified application packaging
|
||||
|
||||
Read [deploy/README.md](../../../deploy/README.md) and
|
||||
[deploy/BUNDLE.md](../../../deploy/BUNDLE.md). A directory containing an ONNX file
|
||||
is an export, not an accepted application.
|
||||
|
||||
## Native application
|
||||
|
||||
1. Identify the robot ID, normalized MJCF and control mode. Use the onboarding
|
||||
skill for source assets. Do not infer a home pose or motor mapping from order.
|
||||
2. For a trained or adapted policy, inspect its `layout.json`, numerical parity
|
||||
report and source provenance. Preserve wire/observation/action named orders,
|
||||
term histories, transforms, rates and gain semantics.
|
||||
3. Write explicit native physical-response probes and run `onboard.py verify`.
|
||||
Require both physics and control acceptance for a controlled capability.
|
||||
Changing tolerances solely to turn a failure green is not acceptance.
|
||||
4. Run `deploy.py --robot <id> --model <normalized.xml> --control
|
||||
<controller.json> --controls <controls.yaml> --verification <report.json>
|
||||
--bundle <output-parent>`. Add `--policy <export>` for an ONNX mode.
|
||||
Geometry-only display omits controller/verification and claims `model_viewer`.
|
||||
5. Replay the emitted app and run `deploy.py --check-reference <app>`. Record
|
||||
actual measured results and supported runtime, including missing capabilities.
|
||||
6. Follow [bundle-manual](../bundle-manual/SKILL.md) to add the Chinese manual
|
||||
through `deploy.py --translate`; report the resulting archive path.
|
||||
|
||||
The portable native runtime needs host Python and declared dependencies. It is
|
||||
not a browser or board executable. Do not mark an unsupported host as available.
|
||||
|
||||
## Legacy Rust board/web application
|
||||
|
||||
1. Export the checkpoint through `scripts/export.py`. Refuse contracts the target
|
||||
reader does not implement; native hybrid position/velocity contracts are not
|
||||
legacy scalar-action contracts.
|
||||
2. Choose explicit export paths, cascade and switches in `deploy/manifests.json`
|
||||
or repeated `--mode` arguments. The generic repository defines no default robot.
|
||||
Use the [controls skill](../controls/SKILL.md) for binding changes.
|
||||
3. For a real board, run the read-only preflight script with the explicitly supplied
|
||||
`--host` and machine configuration. Check software ABI, DDS/QoS and named motor
|
||||
order before a build or transfer. No address is inferred.
|
||||
4. Build through `scripts/deploy.py`. Its default legacy path performs RKNN
|
||||
conversion, cross-builds, contract checks and reference generation. Follow
|
||||
`deploy/convert/README.md` and `deploy/fsm/README.md` for toolchains.
|
||||
5. `--allow-incomplete` is a development build with recorded omissions. Do not
|
||||
describe one as board-ready. Add its Chinese manual before handoff too.
|
||||
6. Hardware operation is separate from building. When explicitly authorized,
|
||||
run the board controller's `--dry-run` then `--check-reference` before the
|
||||
operational command with the machine's configuration. Stop on mismatch.
|
||||
|
||||
## Input conflicts
|
||||
|
||||
A mode's controls and the FSM's switches share a vocabulary and must not consume
|
||||
one input twice. The legacy reader reports:
|
||||
|
||||
```
|
||||
switches on the pad's ... controls use too: one press would do both.
|
||||
Pick another control, or say with `from` that the switch is not pressed in
|
||||
```
|
||||
|
||||
Change the binding or explicitly restrict the switch's source modes. Do not remove
|
||||
validation or widen a contract to hide the conflict. Unknown observation terms,
|
||||
joint names, unsupported versions and missing trajectory companions fail closed.
|
||||
@@ -0,0 +1,250 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read-only preflight: does this board match what we are about to build for it?
|
||||
|
||||
Every mismatch this looks for is one that **does not raise on the robot**. A
|
||||
CycloneDDS soname that differs makes the binary fail to load; an IDL that differs
|
||||
makes the DDS reader never pair, leaving the topic empty forever; a joint order
|
||||
that differs drives the wrong joints. None of them announce themselves, and the
|
||||
last one moves a real machine.
|
||||
|
||||
Standard library plus `ssh` only, so it runs without this skill and without the
|
||||
repository's virtualenv:
|
||||
|
||||
python3 .claude/skills/deploy/scripts/check_board.py --host <user>@<board>
|
||||
python3 .claude/skills/deploy/scripts/check_board.py --host <user>@<board> \\
|
||||
--bundle out/my_robot/<name>
|
||||
|
||||
`--host` has no default: a board's address belongs to the network it is on, not
|
||||
to this repository.
|
||||
|
||||
**Read-only. It runs no binary on the board and publishes nothing.** The robot's
|
||||
own controller usually runs while this does, and two controllers on one motor bus
|
||||
is not a thing to find out about experimentally.
|
||||
|
||||
Exit status: 0 all checks passed, 1 a check failed, 2 could not reach the board.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[4]
|
||||
|
||||
#: What this repository builds against. Each is compared, not assumed.
|
||||
EXPECT_RKNN = "2.3.2" # deploy/fsm/vendor/rknpu2 and convert/requirements.txt
|
||||
EXPECT_DDS_SONAME = "libddsc.so.11"
|
||||
|
||||
|
||||
class Result:
|
||||
def __init__(self) -> None:
|
||||
self.rows: list[tuple[str, bool | None, str]] = []
|
||||
|
||||
def add(self, name: str, ok: bool | None, detail: str) -> None:
|
||||
self.rows.append((name, ok, detail))
|
||||
|
||||
def report(self) -> int:
|
||||
width = max(len(n) for n, _, _ in self.rows)
|
||||
for name, ok, detail in self.rows:
|
||||
mark = {True: "ok ", False: "FAIL", None: "-- "}[ok]
|
||||
print(f" [{mark}] {name:<{width}} {detail}")
|
||||
failed = [n for n, ok, _ in self.rows if ok is False]
|
||||
skipped = [n for n, ok, _ in self.rows if ok is None]
|
||||
print()
|
||||
if failed:
|
||||
print(f"FAILED: {', '.join(failed)}")
|
||||
return 1
|
||||
print("all checks passed" + (f" ({len(skipped)} skipped)" if skipped else ""))
|
||||
return 0
|
||||
|
||||
|
||||
def ssh(host: str, command: str, timeout: int = 30) -> tuple[int, str]:
|
||||
"""One read-only command on the board."""
|
||||
proc = subprocess.run(
|
||||
["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8", host, command],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
return proc.returncode, proc.stdout.strip()
|
||||
|
||||
|
||||
def check_reachable(host: str) -> str | None:
|
||||
code, out = ssh(host, "uname -srm; . /etc/os-release 2>/dev/null && echo $VERSION")
|
||||
if code != 0:
|
||||
return None
|
||||
return out
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||
ap.add_argument("--host", required=True, help="ssh target, <user>@<board>")
|
||||
ap.add_argument(
|
||||
"--bundle",
|
||||
type=Path,
|
||||
help="an exported policy to check the board's joint order against "
|
||||
"(the directory holding layout.json)",
|
||||
)
|
||||
ap.add_argument("--config", default=None,
|
||||
help="a file on the board holding [robot] joint_names, to check the "
|
||||
"bundle's wire order against. There is no default: the path this "
|
||||
"had belonged to a controller that is not part of the product, "
|
||||
"and a default pointing at it made the check look like a step "
|
||||
"rather than a thing you can do if you happen to have such a file")
|
||||
args = ap.parse_args()
|
||||
|
||||
if shutil.which("ssh") is None:
|
||||
print("ssh not found", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
print(f"board: {args.host}\n")
|
||||
banner = check_reachable(args.host)
|
||||
if banner is None:
|
||||
print(f"cannot reach {args.host} over ssh (key auth, BatchMode).", file=sys.stderr)
|
||||
print(" Check the address, the route, and that your key is authorised.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
r = Result()
|
||||
r.add("reachable", True, banner.replace("\n", " "))
|
||||
|
||||
# ── glibc: a floor, not a ceiling ──────────────────────────────────
|
||||
# A binary built against a newer glibc than the board's dies at startup
|
||||
# with "version GLIBC_2.xx not found", nowhere near the build that caused it.
|
||||
_, out = ssh(args.host, "ldd --version | head -1")
|
||||
m = re.search(r"(\d+\.\d+)\s*$", out)
|
||||
r.add("glibc", m is not None, f"{m.group(1) if m else out} (fsm/Dockerfile builds on 22.04 = 2.35)")
|
||||
|
||||
# ── The NPU runtime must be at least as new as the toolkit that built
|
||||
# the .rknn. Older refuses to load the model.
|
||||
_, out = ssh(args.host, "grep -a -o 'librknnrt version: [0-9.]*' /usr/lib/librknnrt.so | head -1")
|
||||
got = out.split(":")[-1].strip() if ":" in out else ""
|
||||
r.add("librknnrt", bool(got) and got >= EXPECT_RKNN,
|
||||
f"{got or '<not found>'} (need >= {EXPECT_RKNN}, what convert/ builds with)")
|
||||
|
||||
# ── The soname the binary was linked against has to exist on the board.
|
||||
_, out = ssh(args.host, "ls /usr/lib/libddsc.so.* 2>/dev/null | head -3")
|
||||
r.add("cyclonedds", EXPECT_DDS_SONAME in out, f"{out.replace(chr(10), ' ') or '<none>'}")
|
||||
|
||||
_, out = ssh(args.host, "dpkg -l 2>/dev/null | awk '/libmbus/{print $3}'")
|
||||
r.add("libmbus", bool(out), out or "<not installed>")
|
||||
|
||||
# ── The IDL is the wire contract. Compared by substance: idlc stamps the
|
||||
# source path into the header comment and the include guard, so those
|
||||
# differ between any two builds and mean nothing.
|
||||
ours = REPO / "deploy" / "dds" / "idl"
|
||||
r.add(*check_idl(args.host, ours))
|
||||
|
||||
# ── Joint order. The one that moves the robot if it is wrong.
|
||||
if args.bundle and args.config:
|
||||
r.add(*check_joints(args.host, args.config, args.bundle))
|
||||
elif args.bundle:
|
||||
r.add("joint order", None, "pass --config <file on the board> to check it")
|
||||
else:
|
||||
r.add("joint order", None, "pass --bundle <dir> and --config <file> to check it")
|
||||
|
||||
print()
|
||||
return r.report()
|
||||
|
||||
|
||||
def check_idl(host: str, ours: Path) -> tuple[str, bool | None, str]:
|
||||
"""Compare the board's generated headers with ours, ignoring path noise.
|
||||
|
||||
The repository ships `.idl`, the board ships the `.h` its middleware was
|
||||
built from, so ours are generated here with `idlc` and the two compared.
|
||||
`idlc` stamps the source path into both the header comment and the include
|
||||
guard, and those differ between any two builds -- filtering them is what
|
||||
makes this comparison mean "same types" rather than "same file".
|
||||
"""
|
||||
def strip(text: str) -> list[str]:
|
||||
skip = re.compile(r"File name:|Source:|DDSC_|Cyclone DDS IDL compiler")
|
||||
return [ln for ln in text.splitlines() if ln.strip() and not skip.search(ln)]
|
||||
|
||||
names = ["idl_common", "idl_imu", "idl_motor_control", "idl_robot_control"]
|
||||
if not ours.is_dir():
|
||||
return "IDL types", None, f"{ours} not found"
|
||||
if shutil.which("idlc") is None:
|
||||
return ("IDL types", None,
|
||||
"idlc not on PATH (it ships with CycloneDDS); cannot generate ours to compare")
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="mjrl-idl-") as tmp:
|
||||
tmpdir = Path(tmp)
|
||||
for f in ours.glob("*.idl"):
|
||||
shutil.copy(f, tmpdir / f.name)
|
||||
differing, absent = [], []
|
||||
for n in names:
|
||||
gen = subprocess.run(["idlc", f"{n}.idl"], cwd=tmpdir,
|
||||
capture_output=True, text=True)
|
||||
if gen.returncode != 0 or not (tmpdir / f"{n}.h").is_file():
|
||||
return "IDL types", None, f"idlc failed on {n}.idl"
|
||||
code, board = ssh(host, f"cat /usr/include/{n}.h 2>/dev/null")
|
||||
if code != 0 or not board:
|
||||
absent.append(n)
|
||||
continue
|
||||
if strip(board) != strip((tmpdir / f"{n}.h").read_text()):
|
||||
differing.append(n)
|
||||
|
||||
if absent:
|
||||
return "IDL types", None, f"not on the board: {', '.join(absent)}"
|
||||
if differing:
|
||||
return ("IDL types", False,
|
||||
f"differ: {', '.join(differing)} -- the DDS reader will never pair, "
|
||||
f"and the topic stays empty rather than erroring")
|
||||
return "IDL types", True, f"{len(names)} headers identical in substance"
|
||||
|
||||
|
||||
def check_joints(host: str, config: str, bundle: Path) -> tuple[str, bool, str]:
|
||||
"""The board's wire order against the bundle's, by name."""
|
||||
layout_path = next(
|
||||
(bundle / n for n in ("layout.json", "isaac_layout.json", "contract.json")
|
||||
if (bundle / n).is_file()),
|
||||
None,
|
||||
)
|
||||
if layout_path is None:
|
||||
return "joint order", None, f"no layout.json in {bundle}"
|
||||
|
||||
code, out = ssh(host, f"sed -n '/^joint_names = \\[/,/^\\]/p' {config}")
|
||||
board = re.findall(r'"([^"]+)"', out)
|
||||
if code != 0 or not board:
|
||||
return "joint order", None, f"no [robot] joint_names in {config}"
|
||||
|
||||
layout = json.loads(layout_path.read_text())
|
||||
wire = layout.get("wire_joint_order")
|
||||
if wire is None:
|
||||
# Bundles from before wire_joint_order existed key the home pose by it.
|
||||
wire = list(layout.get("default_joint_pos", {}))
|
||||
if board != wire:
|
||||
for i, (a, b) in enumerate(zip(board, wire)):
|
||||
if a != b:
|
||||
return ("joint order", False,
|
||||
f"differ at index {i}: board={a} contract={b}")
|
||||
return ("joint order", False,
|
||||
f"different lengths: board {len(board)}, contract {len(wire)}")
|
||||
|
||||
index = {n: i for i, n in enumerate(board)}
|
||||
missing = [n for n in layout["obs_joint_order"] + layout["action_joint_order"]
|
||||
if n not in index]
|
||||
if missing:
|
||||
return "joint order", False, f"not on the board: {missing}"
|
||||
uncovered = [n for n in board if n not in layout.get("default_joint_pos", {})]
|
||||
if uncovered:
|
||||
return ("joint order", False,
|
||||
f"default_joint_pos misses {uncovered}; every wire joint needs a home")
|
||||
obs_idx = [index[n] for n in layout["obs_joint_order"]]
|
||||
gaps = [i for i in range(len(board)) if i not in obs_idx]
|
||||
return ("joint order", True,
|
||||
f"{len(board)} joints, same order; observation covers {len(obs_idx)}, "
|
||||
f"skipping wire {gaps} -- resolved by name")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
sys.exit(main())
|
||||
except subprocess.TimeoutExpired:
|
||||
print("timed out talking to the board", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
@@ -0,0 +1,33 @@
|
||||
---
|
||||
name: new-task
|
||||
description: Register a new legged robot for training by generating a data-only robot_spec.py against a locked morphology template.
|
||||
---
|
||||
|
||||
# Register a robot
|
||||
|
||||
Read [Usage](../../../docs/USAGE.md#robot-specification) for the schema.
|
||||
Select `biped`, `quadruped`, `wheeled_biped`, or `multipod` using inspected
|
||||
model facts and control intent. Reuse the exact model that passed the
|
||||
stance checks. Keep robot assets and optional controls outside `tasks/`.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py scaffold-tier3 --config robot.json --calibration calibration.json
|
||||
python scripts/train.py --list
|
||||
python scripts/train.py --task <robot_id> --dry-run
|
||||
python -m pytest tests/test_registry.py tests/test_template_lock.py -q
|
||||
```
|
||||
|
||||
The scaffold writes `tasks/<robot_id>/robot_spec.py`. It carries the model and
|
||||
controls paths, home joint values and base height, gains and effort limits,
|
||||
action scales, and contact names. With `--calibration`, the JSON supplies the
|
||||
robot ID, family, model path, action scales, and any optional contacts,
|
||||
controls, or wheel joint specifications not supplied by the report.
|
||||
|
||||
For operator input, apply the [controls skill](../controls/SKILL.md) before
|
||||
scaffolding and set `controls_path` in the scaffold JSON relative to that
|
||||
file. Use `--profile differential` when generating bindings for
|
||||
`wheeled_biped`.
|
||||
Onboarding must not modify the template files, the registry source, the
|
||||
entry points, or the lock. The only generated Python file is the literal
|
||||
robot specification. If the asset requires a new family or behavior, treat
|
||||
it as an explicit template revision.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: onboard-asset
|
||||
description: Inspect a new robot URDF, MJCF, mesh, or project directory, normalize it for MuJoCo, and select a viewer, existing policy, deterministic control, or legged training route.
|
||||
---
|
||||
|
||||
# Onboard a robot asset
|
||||
|
||||
Read [Workflow](../../../docs/WORKFLOWS.md) and [Hardware](../../../docs/HARDWARE.md).
|
||||
|
||||
1. Run `python scripts/onboard.py inspect <source> --report <new-report.json>`.
|
||||
Pass `--profile`, `--mesh-root`, `--policy`, or `--metadata` when supplied.
|
||||
Review the selected model, missing dependencies, model checks, declared
|
||||
control intent, and recommended tier. Resolve ambiguous inputs explicitly.
|
||||
Keep candidate training configs associated with the actor that used them.
|
||||
A Tier 1 recommendation with `needs_metadata` goes to
|
||||
[tier1-adapt](../tier1-adapt/SKILL.md) for extraction and review.
|
||||
2. Preserve the source and record the exact model selected. For URDF, MJCF,
|
||||
or a standalone mesh, run `build-mjcf` with a new `--output` path. A
|
||||
directory works when it contains one model, or one mesh if there is no model. Add
|
||||
`--floating-base` only when the robot requires a free root. If generating
|
||||
visual fallback collisions, record the approximation and inspect the result.
|
||||
3. Check compilation and inspect joint names, units, axes, limits, inertias,
|
||||
collision geometry, and frames. For a viewer use `scripts/deploy.py --robot
|
||||
<id> --model <xml>` and run a finite headless replay.
|
||||
4. Route an existing policy to [tier1-adapt](../tier1-adapt/SKILL.md),
|
||||
deterministic control to [tier2-control](../tier2-control/SKILL.md), and
|
||||
explicit legged training to [tier3-train](../tier3-train/SKILL.md).
|
||||
|
||||
A viewer pass establishes that the model loads. Carry separate measured
|
||||
physics and command evidence into every controlled mode.
|
||||
@@ -0,0 +1,160 @@
|
||||
---
|
||||
name: setup-env
|
||||
description: Set up, verify or repair the mjrl-lab development environment on Linux, Windows or macOS, with or without an NVIDIA GPU -- the Python environment and the Rust toolchain the controller (deploy/fsm) builds with. Detects the machine, picks the right torch build, finds which of cargo, the wasm target, wasm-bindgen and CycloneDDS are missing, installs, and verifies Python and optional legacy targets. Use this whenever someone is getting this repository running on a new machine, onboarding, or hitting environment trouble: "ModuleNotFoundError: No module named 'mjlab'", torch not seeing the GPU, warp reporting only ['cpu'], a registered robot missing from `--list`, mujoco version conflicts, `cargo` not found, `idlc not found`, a wasm-bindgen version mismatch, the controller's tests skipping, or a suspicion that the wrong Python or the wrong checkout is being used. Reach for it before hand-rolling pip or cargo commands, because the failures here are quiet -- an environment that imports everything and trains happily can still be running another checkout's code or ignoring the GPU entirely, and a test suite on a machine without cargo is green because it skipped the controller.
|
||||
---
|
||||
|
||||
# Setting up mjrl-lab
|
||||
|
||||
The reference procedure is [`docs/AGENT_SETUP.md`](../../../docs/AGENT_SETUP.md) — about 600 lines,
|
||||
written to be read by an agent, with the full platform tables and a catalogue of known failures.
|
||||
**This skill does not restate it.** It sequences it, automates the two parts worth automating,
|
||||
and knows when to stop and read.
|
||||
|
||||
Two scripts carry the weight. Both are plain standard-library Python and run without this skill,
|
||||
so an agent that cannot load skills can still use them:
|
||||
|
||||
| Script | Interpreter | Answers |
|
||||
|---|---|---|
|
||||
| `scripts/detect.py` | whatever the machine already has | what this machine is, and the exact commands that follow from it |
|
||||
| `scripts/gates.py` | the environment's own | whether the environment actually works, and what to fix |
|
||||
|
||||
## The shape of the problem
|
||||
|
||||
Almost everything here fails **quietly**. Installing a cu126 torch on a Blackwell card succeeds
|
||||
and dies later inside CUDA. An editable install pointing at a stale checkout imports cleanly and
|
||||
runs somebody else's code. A `pip` into the wrong interpreter reports success and changes
|
||||
nothing. A machine without cargo runs the test suite green, because the tests that build the
|
||||
controller skip. None of these announce themselves, and all of them surface much later as
|
||||
something that looks unrelated.
|
||||
|
||||
That is why this measures before deciding and verifies after acting, rather than running a
|
||||
sequence of commands and trusting that silence means success.
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Measure, before deciding anything
|
||||
|
||||
```bash
|
||||
python3 .claude/skills/setup-env/scripts/detect.py
|
||||
```
|
||||
|
||||
Run it with the machine's existing interpreter — establishing whether that interpreter is usable
|
||||
at all is part of what it answers. It reports OS, architecture, Python version, GPU and compute
|
||||
capability, any environment that already exists, which parts of the controller's toolchain are
|
||||
already here, and it ends with the exact command sequence for this machine.
|
||||
|
||||
Read its output before continuing. In particular:
|
||||
|
||||
- **Python out of range** (needs 3.10–3.13): install a compliant interpreter *first*. Installing
|
||||
dependencies on an old one produces failures much later that read as unrelated.
|
||||
- **An environment already exists**: go to step 3 and verify it. The common case after the first
|
||||
run is not a bare machine but a broken environment, and building a second one hides the fault
|
||||
rather than fixing it.
|
||||
|
||||
### 2. Install
|
||||
|
||||
Follow the commands `detect.py` printed. They are section 1 of the reference doc with the
|
||||
branches already resolved for this machine; consult the doc when something does not match — the
|
||||
Debian `ensurepip` split, the Windows execution policy, and the MSVC prerequisite each have a
|
||||
paragraph there.
|
||||
|
||||
Only one step genuinely branches: **which torch wheel**. `detect.py` chose it from the measured
|
||||
compute capability, and its reasoning is on the `torch_plan` function if you need to check it.
|
||||
|
||||
Before any `pip`, confirm the interpreter is the intended one:
|
||||
|
||||
```bash
|
||||
python -c "import sys; print(sys.prefix)"
|
||||
```
|
||||
|
||||
It must be the environment you just made. Pip reports success regardless of where it installs, so
|
||||
this is the cheapest place to catch a mistake that otherwise surfaces three steps later.
|
||||
|
||||
`pip install -e .` is not optional and is not a convenience: the vendored `mjlab` and `rsl_rl`
|
||||
live under `rl/`, which is not on `sys.path`. That editable install *is* the mechanism that makes
|
||||
them importable.
|
||||
|
||||
**The legacy controller toolchain is required for its browser and board targets** — sections 2.1 and 2.2
|
||||
of the reference doc. It is additive rather than per-checkout, so `detect.py` prints only the
|
||||
steps this machine is missing, and "nothing missing" is a normal answer on a second checkout.
|
||||
Two of its parts are easy to get subtly wrong:
|
||||
|
||||
- **wasm-bindgen's version comes from `deploy/fsm/Cargo.lock`**, exactly — not from
|
||||
`Cargo.toml`'s `"0.2"`, a range every 0.2.x satisfies and `cargo install --version` refuses.
|
||||
Take the command `detect.py` printed; `scripts/deploy.py` refuses any other version.
|
||||
- **CycloneDDS is a source build** (11.0.1, the soname the board carries) and needs
|
||||
`sudo ldconfig` after installing into `/usr/local`. It is not needed for `deploy.py`, but once
|
||||
cargo exists `tests/test_deploy_sources.py` stops skipping and fails without it.
|
||||
- **Rockchip's NPU header is per checkout, not per machine.** `bash
|
||||
deploy/fsm/vendor/rknpu2/fetch.sh` puts it in the repository, git-ignored because its licence
|
||||
is Rockchip's. `detect.py` prints that step whenever it is missing, a second checkout included.
|
||||
|
||||
### 3. Verify
|
||||
|
||||
```bash
|
||||
.venv/bin/python .claude/skills/setup-env/scripts/gates.py --python-only
|
||||
```
|
||||
|
||||
Use `--python-only` for Python onboarding, training, and native apps. Omit it to verify the legacy controller toolchain as well. The exit code counts failures. Each prints what it measured and, on failure, the
|
||||
remedy and where the long form lives:
|
||||
|
||||
- **A — warp device enumeration.** On a GPU machine `cuda:0` must appear in the device list.
|
||||
`wp.init()` also prints a banner naming the card; that banner appears even when CUDA is
|
||||
unusable, so the list is the criterion and the banner is not.
|
||||
- **B — native MuJoCo's batch interface.** `native:cpu` is built on `mujoco.rollout`, so
|
||||
importing `mujoco` is not sufficient evidence.
|
||||
- **C — the vendored copies are this checkout's.** Stricter than the doc's version, which only
|
||||
rules out `site-packages`. An editable install left pointing at an older copy of the repository
|
||||
satisfies that and still executes another checkout's code — measured on a training machine on
|
||||
2026-09-04, where `tasks` came from the current repository and `mjlab` from a snapshot three
|
||||
days stale. Everything imported; the seam being edited simply was not the seam being run.
|
||||
- **D — the task registry.** Loads no simulation dependencies, so it passes even where the GPU is
|
||||
broken. A failure here is a packaging problem, not a simulation one — which is worth knowing
|
||||
before looking in the wrong place.
|
||||
- **E — the Rust toolchain builds the controller's host targets.** `cargo check` of the browser's
|
||||
wasm and of `play --app`'s extension, and the wasm-bindgen CLI against `Cargo.lock`. It builds
|
||||
rather than reading versions, because a missing target or linker passes `cargo --version`.
|
||||
- **F — the device build links and loads.** `cargo test --lib -- --list` with the default
|
||||
features: CycloneDDS, libclang, Rockchip's NPU header and a `libddsc` the loader finds. `--list` runs the binary
|
||||
without running a test, so a failing crate test is not reported as an environment fault.
|
||||
|
||||
E and F take about 30 s together, cold, on the i9-14900KF with the crates already downloaded; a
|
||||
first run also downloads them.
|
||||
|
||||
Do not proceed past a failure. Each remedy is printed; the long forms are section 6 of the
|
||||
reference doc.
|
||||
|
||||
### 4. Smoke test
|
||||
|
||||
With the relevant gates green, smoke-test the backend you intend to use.
|
||||
Choose a registered robot from `scripts/train.py --list`. If the registry is
|
||||
empty, use the model viewer example in the root README.
|
||||
|
||||
```bash
|
||||
python scripts/train.py --task <robot_id> --backend warp --device cuda --num_envs 4 --max-iterations 1 --headless
|
||||
python scripts/train.py --task <robot_id> --backend native --device cpu --num_envs 4 --max-iterations 1 --headless
|
||||
```
|
||||
|
||||
Check the backend in the run banner and confirm the PPO update completes.
|
||||
|
||||
## When something fails
|
||||
|
||||
Work from the measurement, not from the symptom. `detect.py` and `gates.py` both print what they
|
||||
observed, and the reference doc's section 6 is indexed by the exact error text — search it for the
|
||||
message rather than reasoning from the traceback, because several of these have causes that the
|
||||
traceback does not mention (the `njmax` overflow, the wandb `start_method` rejection, and
|
||||
`select_gpus` raising `IndexError` are all in that class).
|
||||
|
||||
Two rules survive every platform, and both are in section 7 of the doc:
|
||||
|
||||
- **Never `pip install mjlab` or `rsl-rl-lib`.** They exist as modifiable copies under `rl/`.
|
||||
A PyPI copy in the same environment shadows them ambiguously and is very hard to notice.
|
||||
- **Never install into an existing Isaac Lab environment.** mjlab needs `mujoco~=3.11`, Isaac Lab
|
||||
ships 3.10, and they break each other.
|
||||
|
||||
## Reporting back
|
||||
|
||||
Say which branch the machine took, which gates passed, and what the smoke test measured. If a
|
||||
gate was skipped or a step was worked around, say so plainly — an environment reported as ready
|
||||
when one gate was quietly skipped is worse than one reported as broken, because the next failure
|
||||
will be attributed to the code instead of the setup.
|
||||
@@ -0,0 +1,445 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report what this machine is, and which install path follows from it.
|
||||
|
||||
Run with **whatever interpreter the machine already has** -- that is the point.
|
||||
It answers "is this Python usable at all", so it cannot assume a modern one, and
|
||||
it cannot import anything that is not in the standard library of Python 3.6.
|
||||
|
||||
Every later step depends on these four facts, and each of them changes what the
|
||||
install has to do:
|
||||
|
||||
OS + arch -> whether a C++ compiler is a prerequisite (Windows)
|
||||
Python version -> whether an interpreter has to be installed first
|
||||
GPU present -> which torch wheel index, and whether a driver is required
|
||||
compute cap -> which CUDA build; a cu126 torch dies on Blackwell
|
||||
|
||||
and, for the controller (`deploy/fsm`, Rust), what of its toolchain is already
|
||||
here -- that part is additive, so only the missing steps are printed.
|
||||
|
||||
Prints a plain report and, at the end, the exact commands for this machine.
|
||||
"""
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import glob
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
MIN_PY = (3, 10)
|
||||
MAX_PY = (3, 13) # inclusive; 3.14 is excluded by pyproject
|
||||
|
||||
# The highest `rust-version` among the packages `deploy/fsm/Cargo.lock` pins,
|
||||
# read from their manifests on 2026-09-29: hashbrown 0.17.1 and indexmap 2.14.2,
|
||||
# both through `toml`. A floor for the report only -- gate E builds the crate,
|
||||
# which is the criterion, and says so when the lock moves past this.
|
||||
MIN_RUST = (1, 85)
|
||||
|
||||
# The CycloneDDS the device build is known to work with: what the development
|
||||
# machine (i9-14900KF, Ubuntu 24.04) runs, `idlc` 11.0.1, and a release whose
|
||||
# soname, libddsc.so.11, is the board's.
|
||||
CYCLONEDDS_TAG = "11.0.1"
|
||||
|
||||
|
||||
def sh(cmd):
|
||||
"""Run a command, returning (ok, output). Never raises: a missing tool is an
|
||||
answer, not an error."""
|
||||
try:
|
||||
out = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
|
||||
return True, out.decode("utf-8", "replace").strip()
|
||||
except Exception as e: # noqa: BLE001 - any failure means "not available"
|
||||
return False, str(e)
|
||||
|
||||
|
||||
def detect_gpu():
|
||||
"""(has_gpu, name, compute_cap). No nvidia-smi means no usable NVIDIA GPU.
|
||||
|
||||
macOS is decided without asking: Apple Silicon has no CUDA and there is no
|
||||
workaround, so a stray nvidia-smi there would be misleading.
|
||||
"""
|
||||
if sys.platform == "darwin":
|
||||
return False, None, None
|
||||
ok, out = sh(["nvidia-smi", "--query-gpu=name,compute_cap",
|
||||
"--format=csv,noheader"])
|
||||
if not ok or not out:
|
||||
return False, None, None
|
||||
first = out.splitlines()[0]
|
||||
parts = [p.strip() for p in first.split(",")]
|
||||
name = parts[0] if parts else None
|
||||
cap = parts[1] if len(parts) > 1 else None
|
||||
return True, name, cap
|
||||
|
||||
|
||||
def torch_plan(has_gpu, cap):
|
||||
"""Which torch wheel to install, and why.
|
||||
|
||||
cu128 is the recommendation whenever the capability is unknown or >= 8.0: it
|
||||
is required on Blackwell (12.0) and backward compatible with Ada and Ampere,
|
||||
so choosing it cannot be wrong in that range -- whereas guessing cu126 on a
|
||||
Blackwell card installs cleanly and then dies when CUDA initialises, which
|
||||
is a far more expensive mistake than an unnecessarily new wheel.
|
||||
"""
|
||||
if not has_gpu:
|
||||
return ("pip install torch torchvision",
|
||||
"CPU build -- no --index-url. native:cpu is the training backend here.")
|
||||
try:
|
||||
capf = float(cap)
|
||||
except (TypeError, ValueError):
|
||||
capf = None
|
||||
if capf is not None and capf < 8.0:
|
||||
return ("# consult https://pytorch.org for a build matching compute capability %s" % cap,
|
||||
"compute capability %s is below 8.0; the right wheel needs looking up." % cap)
|
||||
why = "cu128: required on Blackwell (12.0), backward compatible with Ada/Ampere."
|
||||
if capf is not None:
|
||||
why = "compute capability %s -> %s" % (cap, why)
|
||||
return ("pip install torch torchvision --index-url "
|
||||
"https://download.pytorch.org/whl/cu128", why)
|
||||
|
||||
|
||||
def python_verdict():
|
||||
v = sys.version_info[:3]
|
||||
ok = MIN_PY <= v[:2] <= MAX_PY
|
||||
return ok, ".".join(str(x) for x in v)
|
||||
|
||||
|
||||
def interpreter_advice(system):
|
||||
"""What to install when the interpreter is out of range."""
|
||||
return {
|
||||
"Linux": ("Ubuntu 22.04/24.04 already ship 3.10/3.12. On older releases:\n"
|
||||
" sudo add-apt-repository ppa:deadsnakes/ppa\n"
|
||||
" sudo apt install -y python3.11 python3.11-venv\n"
|
||||
" then create the venv with python3.11"),
|
||||
"Darwin": "brew install python@3.12, then create the venv with python3.12",
|
||||
"Windows": ("Installer from python.org with \"Add python.exe to PATH\" ticked, "
|
||||
"then create the venv with py -3.11"),
|
||||
}.get(system, "Install a CPython between 3.10 and 3.13.")
|
||||
|
||||
|
||||
def find_repo():
|
||||
"""The repository root, if we are standing in it.
|
||||
|
||||
Identified by pyproject naming this project rather than by the directory
|
||||
name, which is routinely different (kk-rl-mjlab, mjrl-lab, a worktree path).
|
||||
"""
|
||||
here = os.path.abspath(os.getcwd())
|
||||
while True:
|
||||
candidate = os.path.join(here, "pyproject.toml")
|
||||
if os.path.isfile(candidate):
|
||||
try:
|
||||
with open(candidate, "rb") as fh:
|
||||
text = fh.read().decode("utf-8", "replace")
|
||||
if 'name = "mjrl-lab"' in text:
|
||||
return here
|
||||
except OSError:
|
||||
pass
|
||||
parent = os.path.dirname(here)
|
||||
if parent == here:
|
||||
return None
|
||||
here = parent
|
||||
|
||||
|
||||
def which(name):
|
||||
"""A tool on PATH, then in `$CARGO_HOME/bin`, where rustup and `cargo
|
||||
install` put things -- on the PATH of a login shell, and often not on that
|
||||
of whatever launched this."""
|
||||
found = shutil.which(name)
|
||||
if found:
|
||||
return found
|
||||
home = os.environ.get("CARGO_HOME") or os.path.join(os.path.expanduser("~"), ".cargo")
|
||||
for exe in (name, name + ".exe"):
|
||||
candidate = os.path.join(home, "bin", exe)
|
||||
if os.path.isfile(candidate) and os.access(candidate, os.X_OK):
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def version_of(path, flag="--version"):
|
||||
"""(text, (major, minor, patch)) from `<tool> --version`, or (None, None).
|
||||
|
||||
`flag` because idlc has no `--version`; it answers `-v`.
|
||||
"""
|
||||
if not path:
|
||||
return None, None
|
||||
ok, out = sh([path, flag])
|
||||
m = re.search(r"(\d+)\.(\d+)\.(\d+)", out) if ok else None
|
||||
if not m:
|
||||
return None, None
|
||||
return m.group(0), tuple(int(x) for x in m.groups())
|
||||
|
||||
|
||||
def locked_version(repo, package):
|
||||
"""The version of `package` that `deploy/fsm/Cargo.lock` pins."""
|
||||
if not repo:
|
||||
return None
|
||||
try:
|
||||
with open(os.path.join(repo, "deploy", "fsm", "Cargo.lock"), "rb") as fh:
|
||||
text = fh.read().decode("utf-8", "replace")
|
||||
except OSError:
|
||||
return None
|
||||
m = re.search(r'^name = "%s"\nversion = "([^"]+)"' % re.escape(package), text, re.MULTILINE)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def detect_rust(repo):
|
||||
"""What of the controller's toolchain is already on this machine.
|
||||
|
||||
`rustup` is asked for separately from `cargo` because a distribution's
|
||||
cargo (`apt install cargo`) answers `cargo --version` and cannot add the
|
||||
wasm target -- it has no rustup to ask -- and is usually older than this
|
||||
lock needs. The rest of the install goes through rustup.
|
||||
"""
|
||||
cargo = which("cargo")
|
||||
rustup = which("rustup")
|
||||
rustc_text, rustc = version_of(which("rustc"))
|
||||
targets = []
|
||||
if rustup:
|
||||
ok, out = sh([rustup, "target", "list", "--installed"])
|
||||
targets = out.split() if ok else []
|
||||
bindgen_text, _ = version_of(which("wasm-bindgen"))
|
||||
home = os.environ.get("CYCLONEDDS_HOME", "/usr/local")
|
||||
return {
|
||||
"cargo": cargo,
|
||||
"on_path": shutil.which("cargo") is not None,
|
||||
"rustup": rustup,
|
||||
"rustc_text": rustc_text,
|
||||
"rustc": rustc,
|
||||
"wasm32": "wasm32-unknown-unknown" in targets,
|
||||
"wasm_bindgen": bindgen_text,
|
||||
"wasm_bindgen_wanted": locked_version(repo, "wasm-bindgen"),
|
||||
"cc": which("cc"),
|
||||
"idlc": which("idlc"),
|
||||
"idlc_text": version_of(which("idlc"), "-v")[0],
|
||||
"dds_home": home,
|
||||
"dds_headers": os.path.isfile(os.path.join(home, "include", "dds", "dds.h")),
|
||||
"libclang": find_libclang(),
|
||||
# Per checkout, unlike everything above: Rockchip's header is fetched into
|
||||
# the repository (deploy/fsm/vendor/rknpu2/README.md). None when there is
|
||||
# no repository to look in.
|
||||
"npu_header": (os.path.isfile(os.path.join(
|
||||
repo, "deploy", "fsm", "vendor", "rknpu2", "include", "rknn_api.h"))
|
||||
if repo else None),
|
||||
}
|
||||
|
||||
|
||||
def find_libclang():
|
||||
"""Where bindgen will find libclang, on Linux; None where it will not.
|
||||
|
||||
The directories clang-sys searches after `LIBCLANG_PATH`, not all of them:
|
||||
enough to tell a machine that has it from one that needs
|
||||
`libclang-dev`. Elsewhere it ships with the compiler (Xcode's command-line
|
||||
tools, an LLVM install on Windows), and gate F is what finds out.
|
||||
"""
|
||||
if not sys.platform.startswith("linux"):
|
||||
return "not checked on this OS"
|
||||
dirs = [os.environ.get("LIBCLANG_PATH", "")] + glob.glob("/usr/lib/llvm-*/lib") + [
|
||||
"/usr/lib", "/usr/lib64", "/usr/lib/x86_64-linux-gnu", "/usr/lib/aarch64-linux-gnu",
|
||||
"/usr/local/lib"]
|
||||
for d in dirs:
|
||||
hits = sorted(glob.glob(os.path.join(d, "libclang*.so*"))) if d else []
|
||||
hits = [h for h in hits if "libclang-cpp" not in h]
|
||||
if hits:
|
||||
return hits[0]
|
||||
return None
|
||||
|
||||
|
||||
def rust_commands(system, rust):
|
||||
"""The steps still missing for the controller, in order, for this OS.
|
||||
|
||||
Only what is missing: unlike the Python environment this is not built
|
||||
fresh per checkout, and re-running an installer that is already satisfied
|
||||
is noise at best.
|
||||
"""
|
||||
steps = []
|
||||
fresh = rust["rustup"] is None
|
||||
if system == "Windows":
|
||||
if fresh:
|
||||
steps.append("winget install --id Rustlang.Rustup -e # then open a new terminal;"
|
||||
" it links with MSVC (section 1.5)")
|
||||
else:
|
||||
if rust["cc"] is None:
|
||||
steps.append("sudo apt install -y build-essential curl" if system == "Linux"
|
||||
else "xcode-select --install")
|
||||
if fresh:
|
||||
if rust["cargo"]:
|
||||
steps.append("# cargo here is not rustup's (a distribution package?): it cannot "
|
||||
"add the wasm target")
|
||||
steps.append("curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs"
|
||||
" | sh -s -- -y --profile minimal")
|
||||
steps.append('. "$HOME/.cargo/env"')
|
||||
if not fresh and rust["rustc"] and rust["rustc"][:2] < MIN_RUST:
|
||||
steps.append("rustup update stable")
|
||||
if fresh or not rust["wasm32"]:
|
||||
steps.append("rustup target add wasm32-unknown-unknown")
|
||||
wanted = rust["wasm_bindgen_wanted"]
|
||||
if wanted and rust["wasm_bindgen"] != wanted:
|
||||
steps.append("cargo install wasm-bindgen-cli --version %s --locked" % wanted)
|
||||
return steps
|
||||
|
||||
|
||||
def device_commands(system, rust):
|
||||
"""The device build's prerequisites: CycloneDDS (idlc, headers, libddsc) and
|
||||
libclang for bindgen. The source build is the Dockerfile's host half, at the
|
||||
release this machine family runs. Rockchip's NPU header is the one step that
|
||||
belongs to the checkout rather than the machine, so a second checkout can be
|
||||
missing it on a machine that has everything else."""
|
||||
steps = []
|
||||
if rust["npu_header"] is False:
|
||||
steps.append("bash deploy/fsm/vendor/rknpu2/fetch.sh # Rockchip's NPU header, once per checkout")
|
||||
if system == "Linux" and rust["libclang"] is None:
|
||||
steps.append("sudo apt install -y libclang-dev")
|
||||
if rust["idlc"] and not rust["idlc_text"] and system == "Linux":
|
||||
# Installed and not running: idlc links libddsc, so this is the loader,
|
||||
# and rebuilding CycloneDDS would not be the fix.
|
||||
steps.append("sudo ldconfig # idlc is installed and does not run: libddsc is not "
|
||||
"in the loader's cache")
|
||||
return steps
|
||||
if rust["idlc"] and rust["dds_headers"]:
|
||||
return steps
|
||||
src = "/tmp/cyclonedds"
|
||||
if system == "Darwin":
|
||||
# The same source build into a prefix of one's own: no sudo, nothing under
|
||||
# /usr/local, and no ldconfig, which macOS does not have -- the crate writes
|
||||
# the prefix's lib/ into the test binary's rpath from CYCLONEDDS_HOME
|
||||
# instead, so the exports belong in the shell that runs cargo. Measured on
|
||||
# an M3 Max on 2026-10-08 (section 2.2 of docs/AGENT_SETUP.md).
|
||||
prefix = "$HOME/.local/opt/cyclonedds-%s" % CYCLONEDDS_TAG
|
||||
return steps + [
|
||||
"brew install cmake",
|
||||
"git clone --depth 1 --branch %s https://github.com/eclipse-cyclonedds/cyclonedds.git %s"
|
||||
% (CYCLONEDDS_TAG, src),
|
||||
"cmake -S %s -B %s/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%s"
|
||||
" -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON" % (src, src, prefix),
|
||||
"cmake --build %s/build -j" % src,
|
||||
"cmake --install %s/build" % src,
|
||||
"export CYCLONEDDS_HOME=%s PATH=%s/bin:$PATH"
|
||||
" # in the shell that builds: the crate takes its rpath from it" % (prefix, prefix),
|
||||
]
|
||||
if system != "Linux":
|
||||
return steps + ["# CycloneDDS %s from source with cmake, CYCLONEDDS_HOME at its prefix,"
|
||||
" idlc on PATH" % CYCLONEDDS_TAG,
|
||||
"# -- section 2.2 of docs/AGENT_SETUP.md; measured on Linux and macOS only"]
|
||||
return steps + [
|
||||
"sudo apt install -y cmake git",
|
||||
"git clone --depth 1 --branch %s https://github.com/eclipse-cyclonedds/cyclonedds.git %s"
|
||||
% (CYCLONEDDS_TAG, src),
|
||||
"cmake -S %s -B %s/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local"
|
||||
" -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON" % (src, src),
|
||||
"cmake --build %s/build -j" % src,
|
||||
"sudo cmake --install %s/build && sudo ldconfig" % src,
|
||||
]
|
||||
|
||||
|
||||
def existing_env(repo):
|
||||
"""An environment that already exists, and whether it is this repo's .venv.
|
||||
|
||||
Reported rather than assumed away, because the common case after the first
|
||||
run is not a bare machine -- it is an environment that exists and has gone
|
||||
wrong, and blindly creating a second one hides the problem instead of
|
||||
fixing it.
|
||||
"""
|
||||
rows = []
|
||||
if repo:
|
||||
for rel in (".venv/bin/python", ".venv/Scripts/python.exe"):
|
||||
p = os.path.join(repo, rel)
|
||||
if os.path.isfile(p):
|
||||
rows.append(("repo .venv", p))
|
||||
in_venv = sys.prefix != getattr(sys, "base_prefix", sys.prefix)
|
||||
if in_venv or os.environ.get("CONDA_PREFIX"):
|
||||
kind = "conda env" if os.environ.get("CONDA_PREFIX") else "active venv"
|
||||
rows.append((kind + " (currently active)", sys.executable))
|
||||
return rows
|
||||
|
||||
|
||||
def main():
|
||||
system = platform.system()
|
||||
py_ok, py_ver = python_verdict()
|
||||
has_gpu, gpu_name, cap = detect_gpu()
|
||||
repo = find_repo()
|
||||
cmd, why = torch_plan(has_gpu, cap)
|
||||
|
||||
print("== machine ==")
|
||||
print(" os : %s (%s)" % (system, platform.machine()))
|
||||
print(" python : %s %s" % (py_ver, "OK" if py_ok else "OUT OF RANGE (need 3.10-3.13)"))
|
||||
print(" interpreter : %s" % sys.executable)
|
||||
if has_gpu:
|
||||
print(" nvidia gpu : %s (compute capability %s)" % (gpu_name, cap))
|
||||
elif system == "Darwin":
|
||||
print(" nvidia gpu : none -- macOS has no CUDA, no exceptions")
|
||||
else:
|
||||
print(" nvidia gpu : none detected (no nvidia-smi, or it failed)")
|
||||
print(" repository : %s" % (repo or "NOT FOUND -- run this from inside the repo"))
|
||||
|
||||
envs = existing_env(repo)
|
||||
print("\n== environments already present ==")
|
||||
if envs:
|
||||
for label, path in envs:
|
||||
print(" %-28s %s" % (label, path))
|
||||
print(" -> One exists. Verify it with gates.py before building another;")
|
||||
print(" a second environment hides the fault instead of fixing it.")
|
||||
else:
|
||||
print(" none")
|
||||
|
||||
print("\n== what follows ==")
|
||||
if not py_ok:
|
||||
print(" ! Install a 3.10-3.13 interpreter FIRST. Do not install dependencies")
|
||||
print(" on this one -- the failures come much later and read as unrelated.")
|
||||
print(" %s" % interpreter_advice(system))
|
||||
print(" backends available here : %s" % ("warp:cuda (primary), native:cpu"
|
||||
if has_gpu else "native:cpu"))
|
||||
if has_gpu:
|
||||
print(" prerequisite : NVIDIA driver (nvidia-smi already answers, so it is present)")
|
||||
if system == "Windows":
|
||||
print(" prerequisite : a C++ compiler (MSVC Build Tools) -- warp compiles kernels")
|
||||
print(" torch : %s" % why)
|
||||
|
||||
rust = detect_rust(repo)
|
||||
rustc_ok = rust["rustc"] is not None and rust["rustc"][:2] >= MIN_RUST
|
||||
wanted = rust["wasm_bindgen_wanted"]
|
||||
print("\n== the controller's toolchain (deploy/fsm, Rust) ==")
|
||||
print(" cargo : %s" % (rust["cargo"] or "not found"))
|
||||
if rust["cargo"] and not rust["on_path"]:
|
||||
print(" not on PATH -- a new shell, or . \"$HOME/.cargo/env\"")
|
||||
print(" rustup : %s" % (rust["rustup"] or "not found"))
|
||||
print(" rustc : %s %s" % (rust["rustc_text"] or "-", "OK" if rustc_ok else
|
||||
"NEED >= %d.%d" % MIN_RUST))
|
||||
print(" wasm32 target : %s" % ("installed" if rust["wasm32"] else "missing"))
|
||||
print(" wasm-bindgen : %s %s" % (rust["wasm_bindgen"] or "not found",
|
||||
"OK" if rust["wasm_bindgen"] == wanted
|
||||
else "Cargo.lock pins %s" % wanted))
|
||||
if system != "Windows":
|
||||
print(" C linker (cc) : %s" % (rust["cc"] or "not found"))
|
||||
idlc = rust["idlc_text"] or ("at %s, and it does not run" % rust["idlc"] if rust["idlc"]
|
||||
else "not found")
|
||||
print(" CycloneDDS : idlc %s, headers %s under %s" % (
|
||||
idlc, "present" if rust["dds_headers"] else "missing", rust["dds_home"]))
|
||||
print(" libclang : %s" % (rust["libclang"] or "not found"))
|
||||
print(" NPU header : %s" % {True: "fetched", False: "missing (once per checkout)",
|
||||
None: "-"}[rust["npu_header"]])
|
||||
|
||||
print("\n== commands ==")
|
||||
venv_py = "python3" if system != "Windows" else "py -3.11"
|
||||
activate = (".venv\\Scripts\\Activate.ps1" if system == "Windows"
|
||||
else "source .venv/bin/activate")
|
||||
print(" %s -m venv .venv" % venv_py)
|
||||
print(" %s" % activate)
|
||||
print(" %s" % cmd)
|
||||
print(" pip install -e .")
|
||||
steps = rust_commands(system, rust)
|
||||
print("\n # the controller's toolchain (section 2.1)%s" % ("" if steps else
|
||||
": nothing missing"))
|
||||
for step in steps:
|
||||
print(" %s" % step)
|
||||
steps = device_commands(system, rust)
|
||||
print("\n # the device build: CycloneDDS, libclang, the NPU header (section 2.2)%s" % (
|
||||
"" if steps else ": nothing missing"))
|
||||
for step in steps:
|
||||
print(" %s" % step)
|
||||
print("\n Then verify: python <skill>/scripts/gates.py")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,426 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that an mjrl-lab environment actually works, and say what to fix.
|
||||
|
||||
Run this **with the environment's own interpreter** (`.venv/bin/python gates.py`,
|
||||
or whatever interpreter the environment uses) from anywhere; pass `--repo` if the
|
||||
working directory is not inside the repository.
|
||||
|
||||
Each gate exists because its failure is quiet. An environment can import
|
||||
everything, start training, and produce numbers, while the GPU is unused, or the
|
||||
vendored code being executed belongs to a different checkout. So each gate prints
|
||||
what it measured, not merely OK, and names the remedy -- an agent reading only
|
||||
"FAIL" would have to go and rediscover it.
|
||||
|
||||
Exit code is the number of failed gates, so this drops into a script:
|
||||
|
||||
python gates.py || echo "environment is not ready"
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
DOC = "docs/AGENT_SETUP.md"
|
||||
CRATE = Path("deploy") / "fsm"
|
||||
|
||||
#: What cargo says when the toolchain, not the source, is the problem, and what
|
||||
#: to do about it. Matched against the tail of stderr; the tail is printed too,
|
||||
#: so a message nobody listed here still reaches the reader.
|
||||
RUST_HINTS = (
|
||||
("may not be installed", "rustup target add wasm32-unknown-unknown"),
|
||||
("linker `", ("no C linker: build-essential (Linux), xcode-select --install (macOS),\n"
|
||||
f"MSVC Build Tools (Windows, section 1.5 of {DOC})")),
|
||||
("is not supported by the following package", "rustup update stable"),
|
||||
("does not understand this lock file", "rustup update stable"),
|
||||
("feature `edition2024` is required", "rustup update stable"),
|
||||
("no Python 3.x interpreter found",
|
||||
"pyo3's build script needs a python3 on PATH: activate the environment, or set PYO3_PYTHON"),
|
||||
("lock file needs to be updated",
|
||||
("Cargo.toml and Cargo.lock disagree in this checkout -- a source problem, "
|
||||
"not the environment")),
|
||||
("idlc not found",
|
||||
f"CycloneDDS is not installed, or idlc is not on PATH: section 2.2 of {DOC}"),
|
||||
("dds/dds.h", ("CycloneDDS headers are not under $CYCLONEDDS_HOME/include "
|
||||
f"(default /usr/local): section 2.2 of {DOC}")),
|
||||
("Unable to find libclang", "bindgen needs libclang: sudo apt install -y libclang-dev"),
|
||||
("rknn_api.h is missing",
|
||||
("Rockchip's NPU header is fetched, not committed, once per checkout: "
|
||||
"bash deploy/fsm/vendor/rknpu2/fetch.sh")),
|
||||
("-lddsc", "libddsc is not under $CYCLONEDDS_HOME/lib (default /usr/local)"),
|
||||
("error while loading shared libraries",
|
||||
("built, and the loader cannot find libddsc: sudo ldconfig, or add its lib/ to "
|
||||
"LD_LIBRARY_PATH")),
|
||||
("Library not loaded: @rpath/libddsc",
|
||||
("built, and the macOS loader cannot find libddsc: the crate writes its rpath from "
|
||||
"$CYCLONEDDS_HOME at build time, so export it and rebuild (cargo clean -p mjrl-fsm), "
|
||||
"or add its lib/ to DYLD_LIBRARY_PATH for one run")),
|
||||
)
|
||||
|
||||
|
||||
def which(name: str) -> str | None:
|
||||
"""A tool on PATH, then where `cargo install` and rustup put it.
|
||||
|
||||
The same rule `scripts/deploy.py::which` follows, so this gate and the
|
||||
script agree on whether a tool is installed. `$CARGO_HOME/bin` is on the
|
||||
PATH of a login shell and often not on that of an editor or a service, and
|
||||
a tool found only there is installed, not missing.
|
||||
"""
|
||||
found = shutil.which(name)
|
||||
if found:
|
||||
return found
|
||||
home = Path(os.environ.get("CARGO_HOME") or (Path.home() / ".cargo"))
|
||||
for candidate in (home / "bin" / name, home / "bin" / f"{name}.exe"):
|
||||
if candidate.is_file() and os.access(candidate, os.X_OK):
|
||||
return str(candidate)
|
||||
return None
|
||||
|
||||
|
||||
def locked_version(repo: Path, package: str) -> str | None:
|
||||
"""The version of `package` that `deploy/fsm/Cargo.lock` pins."""
|
||||
lock = repo / CRATE / "Cargo.lock"
|
||||
if not lock.is_file():
|
||||
return None
|
||||
m = re.search(rf'^name = "{re.escape(package)}"\nversion = "([^"]+)"',
|
||||
lock.read_text(encoding="utf-8"), re.MULTILINE)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def find_repo(start: Path | None = None) -> Path | None:
|
||||
"""The repository root, identified by pyproject rather than directory name.
|
||||
|
||||
The directory is routinely named something else -- kk-rl-mjlab, a git
|
||||
worktree path -- so matching on the name would fail on the machines this is
|
||||
most needed on.
|
||||
"""
|
||||
here = (start or Path.cwd()).resolve()
|
||||
for candidate in [here, *here.parents]:
|
||||
pyproject = candidate / "pyproject.toml"
|
||||
if pyproject.is_file() and 'name = "mjrl-lab"' in pyproject.read_text(
|
||||
encoding="utf-8", errors="replace"
|
||||
):
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
class Gates:
|
||||
def __init__(self, repo: Path) -> None:
|
||||
self.repo = repo
|
||||
self.failed: list[str] = []
|
||||
|
||||
def report(self, name: str, ok: bool, detail: str, remedy: str = "") -> None:
|
||||
print(f"[{'PASS' if ok else 'FAIL'}] {name}")
|
||||
for line in detail.splitlines():
|
||||
print(f" {line}")
|
||||
if not ok:
|
||||
self.failed.append(name)
|
||||
if remedy:
|
||||
for line in remedy.splitlines():
|
||||
print(f" -> {line}")
|
||||
print()
|
||||
|
||||
# ── A ───────────────────────────────────────────────────────────────
|
||||
def gate_a_warp(self) -> None:
|
||||
"""Whether warp can enumerate devices.
|
||||
|
||||
The criterion is `cuda:0` in the device list on a GPU machine. wp.init()
|
||||
also prints a banner to stderr naming the card -- that banner appears
|
||||
even when CUDA is unusable, so reading it instead of the list is how a
|
||||
broken GPU gets mistaken for a working one.
|
||||
"""
|
||||
try:
|
||||
import warp as wp
|
||||
|
||||
wp.init()
|
||||
devices = [str(d) for d in wp.get_devices()]
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.report("A warp device enumeration", False, f"{type(e).__name__}: {e}",
|
||||
f"warp is missing or broken; see section 2 of {DOC}")
|
||||
return
|
||||
|
||||
has_gpu = self._nvidia_present()
|
||||
detail = f"devices: {devices}"
|
||||
if has_gpu:
|
||||
self.report("A warp device enumeration", "cuda:0" in devices,
|
||||
detail + " (GPU machine: cuda:0 required)",
|
||||
"the driver or CUDA toolchain is wrong -- warp:cuda is unavailable.\n"
|
||||
f"see 'torch.cuda.is_available() returns False' in section 6 of {DOC}")
|
||||
else:
|
||||
self.report("A warp device enumeration", True,
|
||||
detail + " (no GPU: ['cpu'] alone is the expected result)")
|
||||
|
||||
@staticmethod
|
||||
def _nvidia_present() -> bool:
|
||||
if sys.platform == "darwin":
|
||||
return False
|
||||
try:
|
||||
subprocess.check_output(["nvidia-smi"], stderr=subprocess.STDOUT)
|
||||
return True
|
||||
except Exception: # noqa: BLE001
|
||||
return False
|
||||
|
||||
# ── B ───────────────────────────────────────────────────────────────
|
||||
def gate_b_mujoco(self) -> None:
|
||||
"""native:cpu is built on `mujoco.rollout`, so importing mujoco is not enough."""
|
||||
try:
|
||||
import mujoco
|
||||
from mujoco import rollout # noqa: F401
|
||||
|
||||
v = mujoco.__version__
|
||||
ok = v.startswith(("3.11", "3.12"))
|
||||
self.report("B native MuJoCo batch interface", ok, f"mujoco {v}",
|
||||
"expected 3.11.x or 3.12.x; pyproject pins mujoco~=3.11.0.\n"
|
||||
"an Isaac Lab environment carries 3.10 and the two break each other")
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.report("B native MuJoCo batch interface", False, f"{type(e).__name__}: {e}",
|
||||
"pip install -e . was not run, or ran into the wrong environment")
|
||||
|
||||
# ── C ───────────────────────────────────────────────────────────────
|
||||
def gate_c_vendored(self) -> None:
|
||||
"""The vendored copies must come from **this** checkout's `rl/`.
|
||||
|
||||
The published gate only rules out site-packages, which is not enough. An
|
||||
editable install left pointing at an older copy of the repository
|
||||
satisfies "not site-packages" while executing code from a different
|
||||
checkout -- measured on a training machine on 2026-09-04, where `tasks`
|
||||
resolved from the current repository and `mjlab` from a snapshot three
|
||||
days stale. Everything imported; the seam simply was not the one being
|
||||
edited. So compare against the repository root, not against a substring.
|
||||
"""
|
||||
try:
|
||||
import mjlab
|
||||
import rsl_rl
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.report("C vendored copies are this checkout's", False,
|
||||
f"{type(e).__name__}: {e}",
|
||||
"pip install -e . creates the mapping that makes rl/ importable;\n"
|
||||
f"without it nothing under rl/ is on sys.path. See section 2 of {DOC}")
|
||||
return
|
||||
|
||||
want = (self.repo / "rl").resolve()
|
||||
rows, ok = [], True
|
||||
for mod in (mjlab, rsl_rl):
|
||||
path = Path(mod.__file__).resolve().parent
|
||||
inside = path == want / path.name
|
||||
ok = ok and inside
|
||||
rows.append(f"{mod.__name__:7s} {path}{'' if inside else ' <-- not this repo'}")
|
||||
self.report("C vendored copies are this checkout's", ok,
|
||||
"\n".join(rows) + f"\nexpected under: {want}",
|
||||
"pip uninstall -y mjlab rsl-rl-lib # if they came from PyPI\n"
|
||||
f"pip install -e . # from {self.repo}\n"
|
||||
"then re-run this gate: an editable install can point at another checkout")
|
||||
|
||||
# ── D ───────────────────────────────────────────────────────────────
|
||||
def gate_d_registry(self) -> None:
|
||||
"""The registry, exercised the way a user meets it.
|
||||
|
||||
This loads no simulation dependencies -- tasks register through lazy
|
||||
factories -- so it passes even where the GPU is broken. A failure here is
|
||||
a packaging problem, not a simulation one, and that distinction saves
|
||||
looking in the wrong place.
|
||||
"""
|
||||
train = self.repo / "scripts" / "train.py"
|
||||
try:
|
||||
out = subprocess.run([sys.executable, str(train), "--list"],
|
||||
cwd=self.repo, capture_output=True, text=True, timeout=180)
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.report("D task registry", False, f"{type(e).__name__}: {e}")
|
||||
return
|
||||
expected = {path.parent.name for path in (self.repo / "tasks").glob("*/robot_spec.py")}
|
||||
first_section = out.stdout.split("\n\n", 1)[0]
|
||||
lines = first_section.splitlines()[2:]
|
||||
listed = {line.split()[0] for line in lines if line and not line[0].isspace()}
|
||||
ok = out.returncode == 0 and (
|
||||
listed == expected if expected else out.stdout.strip() == "The registry is empty."
|
||||
)
|
||||
detail = f"{len(listed)} robots: {', '.join(sorted(listed))}" if listed else "no robots registered"
|
||||
if not ok:
|
||||
detail += f"\nExpected robot IDs: {', '.join(sorted(expected)) or '(none)'}"
|
||||
detail += f"\n{(out.stderr or out.stdout).strip()[:400]}"
|
||||
self.report("D task registry", ok, detail,
|
||||
"verify the editable install and the literal tasks/<robot_id>/robot_spec.py files")
|
||||
|
||||
# ── E ───────────────────────────────────────────────────────────────
|
||||
def cargo(self, cargo: str, *args: str,
|
||||
passthrough: tuple[str, ...] = ()) -> subprocess.CompletedProcess:
|
||||
"""Run cargo on the controller's crate.
|
||||
|
||||
`--locked` always: a gate must not rewrite `Cargo.lock`, and the
|
||||
toolchain floor this setup states was measured against the locked
|
||||
versions, not against whatever resolves today. `passthrough` goes
|
||||
after `--`, to the test binary, so it has to follow the options that
|
||||
are cargo's.
|
||||
"""
|
||||
return subprocess.run(
|
||||
[cargo, *args, "--locked", "--manifest-path", str(self.repo / CRATE / "Cargo.toml"),
|
||||
*(("--", *passthrough) if passthrough else ())],
|
||||
cwd=self.repo, capture_output=True, text=True, timeout=900, check=False)
|
||||
|
||||
@staticmethod
|
||||
def errors(stderr: str) -> list[str]:
|
||||
"""The lines of cargo's stderr worth reading: its errors, the last few.
|
||||
|
||||
Not simply the tail. A link failure ends in the linker's command line,
|
||||
one line of several kilobytes that buries the one naming the library.
|
||||
"""
|
||||
lines = stderr.strip().splitlines()
|
||||
shown = [ln for ln in lines if "error" in ln.lower()][-6:] or lines[-6:]
|
||||
return [" " + (ln if len(ln) <= 160 else ln[:157] + "...") for ln in shown]
|
||||
|
||||
@staticmethod
|
||||
def hints(stderr: str) -> list[str]:
|
||||
"""The remedy for every message in `RUST_HINTS` that stderr contains."""
|
||||
return list(dict.fromkeys(fix for needle, fix in RUST_HINTS if needle in stderr))
|
||||
|
||||
def gate_e_rust(self) -> None:
|
||||
"""The controller's host builds: cargo, the wasm target, pyo3, wasm-bindgen.
|
||||
|
||||
`deploy/fsm` is the controller on all three hosts, and `scripts/deploy.py`
|
||||
builds two of them here with cargo -- the browser's wasm and `play
|
||||
--app`'s extension (the board's binary and the Windows extension are
|
||||
cross-built in Docker, not by this toolchain). The failure this gate is
|
||||
for is quiet at the level of the test suite: without cargo,
|
||||
`tests/test_fsm_extension.py` and `tests/test_deploy_sources.py` skip,
|
||||
and `pytest tests/` is green on a machine that has never built the
|
||||
controller.
|
||||
|
||||
So it builds rather than reading versions. `cargo check` of each host's
|
||||
features proves the target is installed and that build scripts and
|
||||
proc macros link on this machine, which `cargo --version` does not.
|
||||
|
||||
The wasm-bindgen CLI is held to `Cargo.lock`, as `scripts/deploy.py`
|
||||
holds it: the CLI and the crate share an ABI and version separately,
|
||||
and `Cargo.toml`'s `"0.2"` is a range every 0.2.x satisfies. Checked
|
||||
here so a mismatch is found while setting up, not at the first bundle.
|
||||
"""
|
||||
name = "E Rust toolchain builds the controller's host targets"
|
||||
cargo = which("cargo")
|
||||
if cargo is None:
|
||||
self.report(name, False, "cargo: not on PATH and not in $CARGO_HOME/bin",
|
||||
f"install rustup: section 2.1 of {DOC}")
|
||||
return
|
||||
|
||||
rows, ok, remedies = [], True, []
|
||||
version = subprocess.run([cargo, "--version"], capture_output=True, text=True,
|
||||
check=False).stdout.strip()
|
||||
on_path = shutil.which("cargo") is not None
|
||||
rows.append(f"{version or 'cargo --version failed'} ({cargo})")
|
||||
if not on_path:
|
||||
rows.append(" not on PATH: deploy.py and the tests look in $CARGO_HOME/bin too; "
|
||||
"a shell will not until it sources ~/.cargo/env")
|
||||
|
||||
for label, args in (
|
||||
("web (wasm32)", ["check", "--lib", "--target", "wasm32-unknown-unknown",
|
||||
"--no-default-features", "--features", "web"]),
|
||||
("py (play --app)", ["check", "--lib", "--no-default-features",
|
||||
"--features", "py"]),
|
||||
):
|
||||
try:
|
||||
out = self.cargo(cargo, *args)
|
||||
except Exception as e: # noqa: BLE001
|
||||
ok = False
|
||||
rows.append(f"{label}: {type(e).__name__}: {e}")
|
||||
continue
|
||||
ok = ok and out.returncode == 0
|
||||
rows.append(f"{label}: {'builds' if out.returncode == 0 else 'FAILS'}")
|
||||
if out.returncode != 0:
|
||||
rows += self.errors(out.stderr)
|
||||
remedies += self.hints(out.stderr)
|
||||
|
||||
wanted = locked_version(self.repo, "wasm-bindgen")
|
||||
cli = which("wasm-bindgen")
|
||||
have = None
|
||||
if cli:
|
||||
have = subprocess.run([cli, "--version"], capture_output=True, text=True,
|
||||
check=False).stdout.split()[-1:]
|
||||
have = have[0] if have else None
|
||||
match = cli is not None and have == wanted
|
||||
ok = ok and match
|
||||
rows.append(f"wasm-bindgen CLI: {have or 'not found'}, Cargo.lock pins {wanted}")
|
||||
if not match:
|
||||
remedies.append(f"cargo install wasm-bindgen-cli --version {wanted} --locked")
|
||||
|
||||
self.report(name, ok, "\n".join(rows), "\n".join(dict.fromkeys(remedies)))
|
||||
|
||||
# ── F ───────────────────────────────────────────────────────────────
|
||||
def gate_f_device(self) -> None:
|
||||
"""The device build: CycloneDDS, libclang, and a libddsc the loader finds.
|
||||
|
||||
The crate's default feature is `device`, the robot's DDS bus, and it is
|
||||
what a plain `cargo test` and `tests/test_deploy_sources.py` build.
|
||||
`build.rs` runs `idlc`, compiles its output against `$CYCLONEDDS_HOME`
|
||||
(default /usr/local), generates bindings through libclang, links libddsc
|
||||
and compiles a probe against Rockchip's NPU header, which each checkout
|
||||
fetches (deploy/fsm/vendor/rknpu2/fetch.sh). deploy.py's host builds need
|
||||
none of it, and the board's build
|
||||
has its own inside the Docker image -- but once cargo is installed,
|
||||
test_deploy_sources stops skipping and fails without it.
|
||||
|
||||
Built and **run**: `--list` executes the test binary without running a
|
||||
test, so a libddsc that links and then cannot be loaded fails here, as
|
||||
`cargo check` never would. A failing crate test is not an environment
|
||||
fault, and listing keeps this gate from reporting one.
|
||||
"""
|
||||
name = "F device build (CycloneDDS, libclang)"
|
||||
cargo = which("cargo")
|
||||
if cargo is None:
|
||||
self.report(name, False, "cargo: not found (gate E)", f"section 2.1 of {DOC} first")
|
||||
return
|
||||
idlc = which("idlc")
|
||||
home = os.environ.get("CYCLONEDDS_HOME", "/usr/local")
|
||||
detail = [f"idlc: {idlc or 'not on PATH'}", f"CYCLONEDDS_HOME: {home}"]
|
||||
try:
|
||||
out = self.cargo(cargo, "test", "--lib", passthrough=("--list",))
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.report(name, False, "\n".join(detail + [f"{type(e).__name__}: {e}"]))
|
||||
return
|
||||
listed = sum(1 for ln in out.stdout.splitlines() if ln.endswith(": test"))
|
||||
ok = out.returncode == 0 and listed > 0
|
||||
if ok:
|
||||
detail.append(f"built, linked and loaded; {listed} tests listed")
|
||||
else:
|
||||
detail += ["cargo test --lib: FAILS", *self.errors(out.stderr)]
|
||||
self.report(name, ok, "\n".join(detail),
|
||||
"\n".join([*self.hints(out.stderr), f"see section 2.2 of {DOC}"]))
|
||||
|
||||
def run(self, *, python_only: bool = False) -> int:
|
||||
print(f"repository: {self.repo}")
|
||||
print(f"interpreter: {sys.executable}\n")
|
||||
self.gate_a_warp()
|
||||
self.gate_b_mujoco()
|
||||
self.gate_c_vendored()
|
||||
self.gate_d_registry()
|
||||
if not python_only:
|
||||
self.gate_e_rust()
|
||||
self.gate_f_device()
|
||||
if self.failed:
|
||||
print(f"{len(self.failed)} gate(s) failed: {', '.join(self.failed)}")
|
||||
print(f"Remedies are above; the long forms are in section 6 of {DOC}.")
|
||||
else:
|
||||
scope = "four Python" if python_only else "six"
|
||||
print(f"All {scope} gates passed. Next: the smoke test in section 4 of {DOC}.")
|
||||
return len(self.failed)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--repo", type=Path, default=None,
|
||||
help="repository root; found from the working directory if omitted")
|
||||
ap.add_argument("--python-only", action="store_true",
|
||||
help="check onboarding, training and native apps without legacy Rust targets")
|
||||
args = ap.parse_args()
|
||||
repo = args.repo.resolve() if args.repo else find_repo()
|
||||
if repo is None or not (repo / "pyproject.toml").is_file():
|
||||
print("error: not inside an mjrl-lab checkout; pass --repo", file=sys.stderr)
|
||||
return 2
|
||||
return Gates(repo).run(python_only=args.python_only)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: tier1-adapt
|
||||
description: Recover explicit policy semantics from upstream training configs or unified metadata, adapt an ONNX or TorchScript actor, and verify parity and native response.
|
||||
---
|
||||
|
||||
# Adapt an existing policy
|
||||
|
||||
1. Use [onboard-asset](../onboard-asset/SKILL.md) to identify the matching
|
||||
model, actor and training configs. Treat a Tier 1 recommendation with
|
||||
`needs_metadata` as a request to recover policy semantics.
|
||||
2. If the source supplies a complete `mjrl-policy-adapter/1` document, use it
|
||||
directly. Otherwise extract evidence from the matched config files.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
|
||||
--output metadata-draft
|
||||
```
|
||||
|
||||
Review `metadata.json` and the provenance, conflicts, `missing_fields` and
|
||||
`unresolved` entries in `manifest.json`. An incomplete extraction writes
|
||||
the draft and exits 1. Python inputs are read as static syntax only.
|
||||
Resolve unknown imports, inheritance or expressions from source evidence
|
||||
and record the confirmed values in a JSON or YAML overrides file. The
|
||||
actor's observation and action definitions establish channel order.
|
||||
A joint-name inventory, a dictionary of default angles, MuJoCo joint order
|
||||
and graph widths each establish different facts.
|
||||
3. Rerun extraction with `--overrides reviewed.json --output metadata-ready`.
|
||||
Require `ready: true` in the manifest. Review ordered observation terms,
|
||||
history, joint and action order, home pose, scaling, gains, limits and rate.
|
||||
Run the adapter with the exact model.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py adapt-tier1 actor.onnx --metadata metadata-ready/metadata.json \
|
||||
--model robot.xml --output policy --report adapter-report.json
|
||||
```
|
||||
|
||||
After review, `--config env.yaml --config agent.yaml --overrides reviewed.json`
|
||||
can replace `--metadata` and run the same extraction during adaptation.
|
||||
4. Review the adapter parity result and layout against the supplied semantics.
|
||||
Build a verification config for the ONNX controller with `kind: rl_onnx`,
|
||||
its relative `policy` and `layout` paths, and the robot root when needed.
|
||||
Run native command probes with meaningful expected motion and tolerances.
|
||||
5. Package a compatible controlled mode only with passing verification for
|
||||
those files. Use [deploy](../deploy/SKILL.md) and replay the resulting app.
|
||||
|
||||
A graph comparison verifies the transformation. Native command response
|
||||
establishes how the adapted controller moves that model.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
name: tier2-control
|
||||
description: Set up a deterministic native MuJoCo controller for a complete robot model, including direct actuation, joint sliders, IK, differential or omnidirectional wheels, trajectories, and CPG.
|
||||
---
|
||||
|
||||
# Set up deterministic control
|
||||
|
||||
1. Inspect the model with [onboard-asset](../onboard-asset/SKILL.md).
|
||||
Choose a controller from the actual kinematics and intended motion.
|
||||
Confirm joint axes and limits, actuator semantics, effort, and rates.
|
||||
2. Declare a JSON controller config with its `kind`. Use
|
||||
`joint_sliders` for named position joints, `ik` for a
|
||||
named chain and end-effector site or body, `differential` or `omni` for
|
||||
measured wheel geometry, and `trajectory` or `cpg` for explicit motions.
|
||||
The `direct` API can inspect authored actuator inputs, but a packaged
|
||||
`model_viewer` must have `actuators: []` and carries no control claim. Choose
|
||||
one of the verified control kinds above for a controlled application.
|
||||
3. Run setup to a fresh path.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py setup-tier2 --model robot.xml --config setup.json --output control
|
||||
```
|
||||
|
||||
Use the resulting `robot.xml` and `controller.json`. Inspect the optional
|
||||
`controls.yaml` with the [controls skill](../controls/SKILL.md).
|
||||
4. Run `onboard.py verify` with a probe JSON that references the generated
|
||||
controller and controls and declares expected measured responses. Use
|
||||
[Tutorial](../../../docs/TUTORIAL.md) for a runnable joint-slider example.
|
||||
Read both the physics and control stages and repeat after any relevant edit.
|
||||
5. Package with the exact evidence and replay the app with finite steps.
|
||||
Report observed responses, limits, and any failing probe.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
name: tier3-train
|
||||
description: Calibrate a floating-base biped, quadruped, wheeled biped, or multipod for locked-template locomotion training, then scaffold and run measured replay.
|
||||
---
|
||||
|
||||
# Calibrate and train a legged robot
|
||||
|
||||
1. Run [onboard-asset](../onboard-asset/SKILL.md). Establish the family, named
|
||||
supports, single floating root, explicit locomotion intent, and joint
|
||||
control semantics. Wheeled bipeds require named unlimited wheel hinges.
|
||||
2. Prepare stance JSON with the seed pose, named supports, optional height
|
||||
adjustments, and measured effort limits. Run
|
||||
`python scripts/onboard.py calibrate --model robot.xml --config stance.json
|
||||
--report calibration.json`. Inspect the measured contact loads, drift,
|
||||
limits, effort headroom, and complete report before using it.
|
||||
3. Follow [new-task](../new-task/SKILL.md) to generate the literal robot spec.
|
||||
Supply explicit action scales and reuse the passing report for the same
|
||||
XML and compiled dependencies. If operator input is requested, generate
|
||||
bindings before scaffolding and set `controls_path` in the scaffold JSON.
|
||||
Use `--profile differential` for a wheeled biped so A/D command yaw.
|
||||
Keep robot-specific changes in data.
|
||||
4. Follow [train](../train/SKILL.md). Verify discovery and a backend smoke
|
||||
before allocating a learning run. Measure the selected checkpoint on
|
||||
trained ranges with the required command and observation-noise setting.
|
||||
5. If packaging is requested, export with `--native-layout` and verify the
|
||||
exported model, controller and bindings. The native layout supports the
|
||||
wheeled-biped template's hybrid position and velocity targets. Follow
|
||||
[deploy](../deploy/SKILL.md) for the selected runtime.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: train
|
||||
description: Train or resume a locomotion policy through a locked template, validate backend and run size, replay the checkpoint, and report measured results.
|
||||
---
|
||||
|
||||
# Train and measure
|
||||
|
||||
1. Confirm the robot appears in `scripts/train.py --list`. Use
|
||||
[new-task](../new-task/SKILL.md) when the robot still needs registration.
|
||||
Run `.claude/skills/setup-env/scripts/detect.py` and verify the environment
|
||||
with the [setup skill](../setup-env/SKILL.md) when needed.
|
||||
2. Resolve with `--dry-run`. Use `native:cpu` for the CPU path or `warp:cuda`
|
||||
for a supported CUDA installation. A small environment count and one
|
||||
iteration exercise the path. Choose the learning run size from the
|
||||
backend's available memory and measured throughput. If Warp reports an
|
||||
overflow, raise `--warp-max-constraints N` for per-world constraints or
|
||||
`--warp-max-contacts N` for the shared contact pool and rerun the smoke.
|
||||
Use positive integers. Training saves the selected and effective capacities
|
||||
in `warp-allocation.json` beside the checkpoints. Warp resume, replay, and
|
||||
export use that record with the current task or scene capacity floor.
|
||||
Explicit flags override the saved setting and floor.
|
||||
3. Run `scripts/train.py --task <robot_id>` with the chosen backend, device
|
||||
and environment count. Use `--headless` for unattended runs. Watch
|
||||
tracking rewards, episode lengths, termination counts, action spread,
|
||||
and curriculum level. Use `--resume` for the latest run or
|
||||
`--checkpoint <path>` for a specific one. On resume, an iteration override
|
||||
counts additional iterations.
|
||||
4. Run `scripts/play.py --task <robot_id> --checkpoint <path> --headless
|
||||
--steps <count> --speed 0 --measure`. Use `--checkpoint-command-ranges`
|
||||
for the saved curriculum range. For a fixed command inside that range,
|
||||
add `--command <vx> <vy> <yaw_rate> --evaluation-out <json>`. Match replay
|
||||
physics to training and report the observation noise setting.
|
||||
5. Report the checkpoint, backend, environment count, iteration count,
|
||||
evaluation command and duration, measured tracking, and resets. Describe
|
||||
a one-iteration smoke as an execution check. Record when replay was not
|
||||
run or results are unavailable.
|
||||
|
||||
See the [tutorial](../../../docs/TUTORIAL.md) for the end-to-end sequence.
|
||||
@@ -0,0 +1,15 @@
|
||||
# Optional local defaults: copy to .env and customize.
|
||||
# Blank values use CLI built-in defaults. Shell variables and CLI flags win.
|
||||
MJRL_TASK=
|
||||
MJRL_MODEL=
|
||||
MJRL_SCENE=
|
||||
MJRL_TERRAIN_ROW=
|
||||
MJRL_TERRAIN_COL=
|
||||
MJRL_BACKEND=
|
||||
MJRL_DEVICE=
|
||||
MJRL_NUM_ENVS=
|
||||
MJRL_PLAY_NUM_ENVS=
|
||||
MJRL_CPU_THREADS=
|
||||
MJRL_STRIP_VISUAL=
|
||||
MJRL_TENSORBOARD=
|
||||
MJRL_TB_PORT=
|
||||
@@ -0,0 +1,108 @@
|
||||
name: Checks
|
||||
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
layout-and-translations:
|
||||
name: Layout, translation and Colab checks
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
# These checks parse files, validate translations and exercise Colab backup
|
||||
# contracts with short-lived subprocesses. They need no MuJoCo or GPU stack.
|
||||
- name: Install test runner
|
||||
run: python -m pip install --disable-pip-version-check pytest
|
||||
|
||||
- name: Check layout, translations and Colab helpers
|
||||
run: python -m pytest tests/test_layout.py tests/test_translations.py tests/test_colab.py tests/test_colab_metrics.py tests/test_colab_backup_inspection.py tests/test_colab_notebook_workflow.py -q
|
||||
|
||||
lint:
|
||||
name: Ruff
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
# The rule set is written into pyproject.toml, so the ruff version is not
|
||||
# what decides what passes here; CONTRIBUTING.md asks for this command
|
||||
# before handing a change over, and this is the same command.
|
||||
- name: Install ruff
|
||||
run: python -m pip install --disable-pip-version-check ruff
|
||||
|
||||
- name: Lint
|
||||
run: python -m ruff check .
|
||||
|
||||
framework-tests:
|
||||
name: Framework tests (native backend, CPU)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: pyproject.toml
|
||||
|
||||
# The whole suite on the native CPU backend: the seam, the vendored
|
||||
# copies, the registry, the task contracts, export and the bundle checks.
|
||||
# No GPU, so CUDA-specific checks skip. Current Rust/wasm runtimes are
|
||||
# built from temporary deterministic test exports, not old applications
|
||||
# under out/. The CPU torch wheel limits installation disk usage.
|
||||
#
|
||||
# mjlab sets MUJOCO_GL=egl on Linux before mujoco is imported, so every
|
||||
# test that imports a task needs an EGL the loader can find, GPU or not:
|
||||
# Mesa's. Without it collection fails on 29 files with
|
||||
# `'NoneType' object has no attribute 'eglQueryString'` (measured in a
|
||||
# python:3.12-slim container on 2026-10-08; the packages below fixed it).
|
||||
- name: Install Mesa EGL for headless rendering
|
||||
run: sudo apt-get update -qq && sudo apt-get install -y -qq libegl1 libegl-mesa0 libgl1-mesa-dri
|
||||
|
||||
- name: Install PyTorch (CPU build)
|
||||
run: python -m pip install --disable-pip-version-check torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
- name: Install this repository
|
||||
run: python -m pip install --disable-pip-version-check -e ".[dev]"
|
||||
|
||||
- name: Install portable controller build tools
|
||||
run: |
|
||||
rustup target add wasm32-unknown-unknown
|
||||
cargo install wasm-bindgen-cli --version 0.2.128 --locked
|
||||
|
||||
- name: Run framework and onboarding checks
|
||||
run: python -m pytest tests/ --ignore=tests/test_replay.py -q
|
||||
|
||||
- name: Run all locked-template replay checks
|
||||
run: python -m pytest tests/test_replay.py -q
|
||||
|
||||
controller-core:
|
||||
name: Portable Rust controller
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.4.0
|
||||
- name: Test the hardware-independent controller
|
||||
run: cargo test --manifest-path deploy/fsm/Cargo.toml --no-default-features
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
venv/
|
||||
build/
|
||||
dist/
|
||||
|
||||
# Training artefacts
|
||||
logs/
|
||||
runs/
|
||||
outputs/
|
||||
# scripts/export.py's default output directory
|
||||
out/
|
||||
*.pt
|
||||
*.onnx
|
||||
# deploy/onnx2rknn.py's output, and the record of what built it
|
||||
*.rknn
|
||||
*.rknn.json
|
||||
wandb/
|
||||
|
||||
# Rust build output. No leading slash: it matches at every depth, so
|
||||
# deploy/fsm/target/ is covered without naming it.
|
||||
target/
|
||||
# cargo-zigbuild probes zig by compiling an empty file into the working
|
||||
# directory -- the crate's, in docker-build.sh -- on every macOS build.
|
||||
# `docker-build.sh` removes it; ignored as well, because `deploy.py` reads
|
||||
# `git status` to decide whether the source is clean, and one left by a failed
|
||||
# build would mark every later one `-dirty`.
|
||||
.intentionally-empty-file.o
|
||||
|
||||
# Warp kernel cache
|
||||
.warp/
|
||||
|
||||
# Rockchip's NPU header and runtime, fetched by deploy/fsm/vendor/rknpu2/fetch.sh.
|
||||
# They are under Rockchip's own licence, which this repository cannot re-grant.
|
||||
deploy/fsm/vendor/rknpu2/include/
|
||||
deploy/fsm/vendor/rknpu2/lib/
|
||||
|
||||
# Derived caches for user assets.
|
||||
assets/*/.cache/
|
||||
|
||||
# The log MuJoCo leaves in the working directory at runtime
|
||||
MUJOCO_LOG.TXT
|
||||
|
||||
# Editors / OS
|
||||
.vscode/
|
||||
.idea/
|
||||
.DS_Store
|
||||
|
||||
# Local configuration and secrets
|
||||
# Copy .env.example to .env for local defaults; never commit local secrets.
|
||||
.env
|
||||
.env.local
|
||||
.claude/launch.json
|
||||
@@ -0,0 +1,41 @@
|
||||
# Repository instructions
|
||||
|
||||
Read [CLAUDE.md](CLAUDE.md) for working rules and [Workflow](docs/WORKFLOWS.md)
|
||||
to choose the robot intake route. The repository contains onboarding, native
|
||||
control, simulation training, and packaging. Use the local sources and the
|
||||
asset's supplied documentation to establish its model and control assumptions.
|
||||
|
||||
Use lightweight models for bounded reading and documentation, standard models
|
||||
for routine changes, and advanced models for architecture or difficult
|
||||
debugging. Delegate independent tasks when appropriate and preserve validation.
|
||||
|
||||
| Work | Reference |
|
||||
|---|---|
|
||||
| Commands and configuration | [Usage](docs/USAGE.md) |
|
||||
| Environment setup or repair | [Agent setup](docs/AGENT_SETUP.md) |
|
||||
| Model units, frames, collision, and calibration | [Hardware](docs/HARDWARE.md) |
|
||||
| Controller and input bindings | [Controls](docs/CONTROLS.md) |
|
||||
| Application build or policy deployment | [Deploy guide](deploy/README.md) and [bundle format](deploy/BUNDLE.md) |
|
||||
| Repository conventions and checks | [Contributing](CONTRIBUTING.md) |
|
||||
|
||||
## Skills
|
||||
|
||||
Read an applicable `SKILL.md` in full before starting the work.
|
||||
|
||||
| Skill | Use for |
|
||||
|---|---|
|
||||
| [setup-env](.claude/skills/setup-env/SKILL.md) | Establish or repair the Python and controller build environment. |
|
||||
| [onboard-asset](.claude/skills/onboard-asset/SKILL.md) | Inspect assets, normalize geometry, choose an intake tier, and verify physics. |
|
||||
| [tier1-adapt](.claude/skills/tier1-adapt/SKILL.md) | Adapt an existing policy from declarative metadata and check numerical parity. |
|
||||
| [tier2-control](.claude/skills/tier2-control/SKILL.md) | Configure and verify deterministic robot controls. |
|
||||
| [tier3-train](.claude/skills/tier3-train/SKILL.md) | Calibrate and train a robot using a locked locomotion template. |
|
||||
| [new-task](.claude/skills/new-task/SKILL.md) | Create the data-only robot specification and verify discovery. |
|
||||
| [train](.claude/skills/train/SKILL.md) | Run or resume training and report measured replay results. |
|
||||
| [controls](.claude/skills/controls/SKILL.md) | Bind keys and pad controls using the shared vocabulary. |
|
||||
| [deploy](.claude/skills/deploy/SKILL.md) | Export and package policies for the supported host. |
|
||||
| [bundle-manual](.claude/skills/bundle-manual/SKILL.md) | Translate the built application's operator manual. |
|
||||
|
||||
Every new skill belongs in this table. The template directories carry their own
|
||||
lock instructions. Onboarding writes one literal `robot_spec.py` per robot and
|
||||
keeps the calibrated asset, controller, reports, and controls outside the locked
|
||||
templates.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Repository working rules
|
||||
|
||||
The project is a generic robotics onboarding, simulation, and training platform.
|
||||
Choose an intake route in [Workflow](docs/WORKFLOWS.md). Use
|
||||
[Usage](docs/USAGE.md) for commands and [Contributing](CONTRIBUTING.md) for layout.
|
||||
|
||||
## Checks
|
||||
|
||||
Run from the repository root with the editable install active.
|
||||
|
||||
```bash
|
||||
python -m pip install -e ".[dev]"
|
||||
python -m pytest tests/ -q
|
||||
python -m ruff check .
|
||||
python scripts/train.py --list
|
||||
```
|
||||
|
||||
For a host with unrelated pytest plugins on its Python path, run pytest in the
|
||||
project environment with `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1`. Use
|
||||
[agent setup](docs/AGENT_SETUP.md) if imports resolve to a different checkout.
|
||||
|
||||
## Architecture
|
||||
|
||||
- The root and `rl/` are separate package roots. `rl/` has no `__init__.py`.
|
||||
Keep imports from `mjrl`, `mjlab`, and `rsl_rl` in their existing form and
|
||||
use the editable install to resolve them.
|
||||
- `mjrl.backend.select.use_backend` must run before creating an environment.
|
||||
The manager resolves mjlab's simulation class when it builds the environment.
|
||||
Native MuJoCo implements the batched simulator interface that Warp provides.
|
||||
- Native rollout scratch data is per worker. Restore each environment's state,
|
||||
controls, and warm starts from its own buffers. Constant recomputation uses
|
||||
dedicated scratch data and preserves visual statistics.
|
||||
- `tasks/registry.py` discovers literal `tasks/<robot_id>/robot_spec.py`
|
||||
files. Listing tasks must stay free of simulation imports. The selected
|
||||
family resolves to a locked template in `tasks/templates/`.
|
||||
- Keep physical facts in the robot specification and asset. Keep reward and
|
||||
observation logic, randomization, curriculum, and PPO in the templates.
|
||||
Adding a robot requires no changes to a template or a command entry point.
|
||||
- Onboarding and verification are in `rl/mjrl/onboarding/`. Deterministic
|
||||
controls live in `rl/mjrl/controllers/`, and the portable native runtime
|
||||
lives in `rl/mjrl/platform/`. The legacy Rust controller and target build
|
||||
tools remain in `deploy/`.
|
||||
|
||||
## Changes and verification
|
||||
|
||||
- Preserve the lock in `tests/test_template_lock.py`. Review an authorized
|
||||
template revision before updating the recorded hashes, including shared
|
||||
dependencies that change template behavior.
|
||||
- Verify physics and control response separately. Report measured outcomes with
|
||||
the model, backend, and run size. A shape check, a visual preview, or a
|
||||
completed iteration alone cannot establish behavior or hardware readiness.
|
||||
- Keep changes to vendored libraries identifiable with an `[mjrl]` marker,
|
||||
reason, and upstream version. See [Vendoring](docs/VENDOR.md).
|
||||
- Write repository material in English. A Chinese `*.zh.md` translation
|
||||
carries the digest of its English source. Update both before refreshing the
|
||||
marker with `tools/checks/stamp_translations.py`.
|
||||
- Exercise tests against the quiet failure they claim to prevent. When an
|
||||
assertion could pass vacuously, include a control that demonstrates detection.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Contributing
|
||||
|
||||
Run checks from the repository root with an editable install.
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python -m pip install -e ".[dev]"
|
||||
python -m pytest tests/ -q
|
||||
python -m ruff check .
|
||||
```
|
||||
|
||||
Use the [agent setup guide](docs/AGENT_SETUP.md) for platform-specific
|
||||
installation and [usage](docs/USAGE.md) for the workflow commands.
|
||||
|
||||
## Directory layout
|
||||
|
||||
```
|
||||
mjrl-lab/
|
||||
├── assets/ # robot models and source assets
|
||||
├── controller/ # input vocabulary, parsing, and device access
|
||||
├── deploy/ # native app contract and legacy target builders
|
||||
│ ├── convert/ # ONNX to RKNN conversion
|
||||
│ ├── dds/ # legacy wire interface
|
||||
│ └── fsm/ # legacy Rust controller
|
||||
├── docs/ # manuals and design notes
|
||||
├── rl/
|
||||
│ ├── mjlab/ # vendored simulator and manager library
|
||||
│ ├── mjrl/
|
||||
│ │ ├── backend/ # native and Warp integration
|
||||
│ │ ├── controllers/ # deterministic controller models
|
||||
│ │ ├── onboarding/ # inspect, build, calibrate, verify
|
||||
│ │ ├── platform/ # native app assembly and runtime
|
||||
│ │ └── tasks/ # reusable MDP and curriculum code
|
||||
│ └── rsl_rl/ # vendored learning algorithms
|
||||
├── scenes/ # built-in world configurations
|
||||
├── scripts/ # onboard, train, play, export, deploy
|
||||
├── tasks/
|
||||
│ └── templates/ # locked family recipes
|
||||
├── tests/ # regression checks and synthetic fixtures
|
||||
└── tools/ # development and validation utilities
|
||||
```
|
||||
|
||||
Add robot-specific inputs under `assets/<robot_id>/` or reference an existing
|
||||
asset checkout. Use onboarding to generate `tasks/<robot_id>/robot_spec.py`.
|
||||
Model-specific tools belong beside their asset. Reusable tools belong in
|
||||
`tools/` or the relevant `mjrl` package.
|
||||
|
||||
Tests should pin shared contracts using neutral fixtures. Exercise a native
|
||||
build and command response when changing model or controller logic. Run the
|
||||
lock test and verify both backends when changing template behavior. Follow
|
||||
[CLAUDE.md](CLAUDE.md) for authorized lock revisions and translations.
|
||||
@@ -0,0 +1,202 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,53 @@
|
||||
Robot asset and control platform
|
||||
Copyright 2026 KingKong Robotics.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (see LICENSE).
|
||||
|
||||
This repository includes the third-party material below, each part under its
|
||||
own licence. The full texts are in the files named.
|
||||
|
||||
rl/mjlab/
|
||||
mjlab 1.6.0 (https://github.com/mujocolab/mjlab)
|
||||
Copyright 2025, The mjlab Developers
|
||||
Apache License 2.0 -- rl/mjlab/LICENSE
|
||||
Modified: five files, each marked "[mjrl]" with the reason for the change
|
||||
(listed in docs/VENDOR.md).
|
||||
|
||||
rl/mjlab/utils/lab_api/
|
||||
Derived from Isaac Lab (https://github.com/isaac-sim/IsaacLab)
|
||||
Copyright (c) 2022-2025, The Isaac Lab Project Developers
|
||||
BSD 3-Clause License -- licenses/IsaacLab.txt
|
||||
|
||||
rl/mjlab/asset_zoo/robots/unitree_go1/
|
||||
rl/mjlab/asset_zoo/robots/unitree_g1/
|
||||
assets/go1_example/go1.xml
|
||||
assets/g1_example/g1.xml
|
||||
From MuJoCo Menagerie (https://github.com/google-deepmind/mujoco_menagerie)
|
||||
Copyright (c) 2016-2022 (Go1), 2016-2023 (G1)
|
||||
HangZhou YuShu TECHNOLOGY CO.,LTD. ("Unitree Robotics")
|
||||
BSD 3-Clause License -- licenses/mujoco_menagerie-unitree_go1.txt,
|
||||
licenses/mujoco_menagerie-unitree_g1.txt
|
||||
|
||||
rl/mjlab/asset_zoo/robots/i2rt_yam/
|
||||
From MuJoCo Menagerie, apart from xmls/assets/d405.stl, which is not in
|
||||
Menagerie and comes with mjlab
|
||||
Copyright (c) 2025 i2rt robotics
|
||||
MIT License -- licenses/mujoco_menagerie-i2rt_yam.txt
|
||||
|
||||
rl/rsl_rl/
|
||||
rsl_rl 5.4.2 (https://github.com/leggedrobotics/rsl_rl)
|
||||
Copyright (c) 2026, ETH Zurich
|
||||
Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES
|
||||
BSD 3-Clause License -- rl/rsl_rl/LICENSE
|
||||
|
||||
rl/rsl_rl/modules/normalization.py
|
||||
Contains code from pfrl (https://github.com/pfnet/pfrl)
|
||||
Copyright (c) 2020 Preferred Networks, Inc.
|
||||
MIT License -- licenses/pfrl.txt
|
||||
|
||||
Not included in this repository
|
||||
|
||||
deploy/fsm/vendor/rknpu2/
|
||||
Filled by fetch.sh with Rockchip's rknn_api.h and librknnrt.so, which are
|
||||
under Rockchip's RKNN SDK License and are not covered by this repository's
|
||||
licence. See deploy/fsm/vendor/rknpu2/README.md.
|
||||
@@ -0,0 +1,80 @@
|
||||
# MJRL Robotics Platform
|
||||
|
||||
[简体中文](README.zh.md) · [Workflow](docs/WORKFLOWS.md) · [Command reference](docs/USAGE.md)
|
||||
|
||||
Bring a robot model into MuJoCo, verify its dynamics and controls, and package it
|
||||
as a native simulation application. The platform also adapts compatible
|
||||
external policies and trains locomotion policies with the same environment
|
||||
on native MuJoCo CPU or MuJoCo Warp GPU.
|
||||
|
||||
| Tier | Starting point | Output |
|
||||
|---|---|---|
|
||||
| 0 | Mesh, URDF, or an MJCF without verified controls | Compiled model and viewer application |
|
||||
| 1 | Compatible policy and explicit observation, action, and joint metadata | Adapted ONNX policy and layout with numerical parity results |
|
||||
| 2 | Complete robot model and known kinematics | Deterministic controller and native verification report |
|
||||
| 3 | Legged robot with a calibrated stance and explicit control intent | Robot specification and training through a locked morphology template |
|
||||
|
||||
## Run the model viewer
|
||||
|
||||
From the checkout, with Python 3.10–3.13:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python -m pip install -e ".[dev]"
|
||||
python scripts/onboard.py inspect assets/smoke/smoke_biped.xml
|
||||
python scripts/deploy.py --robot viewer_smoke --model assets/smoke/smoke_biped.xml --bundle out/viewer_demo
|
||||
python scripts/play.py --app out/viewer_demo/viewer_smoke.app --headless --steps 20
|
||||
```
|
||||
|
||||
Choose an unused output directory. The smoke model exercises loading and
|
||||
packaging. For a new robot, replace the input with your own model and follow
|
||||
the report's recommended route. On Windows, activate with
|
||||
`.venv\Scripts\Activate.ps1`.
|
||||
|
||||
## Bring your own robot
|
||||
|
||||
1. Run `scripts/onboard.py inspect` with the source model and any available
|
||||
profile or policy metadata. Use `build-mjcf` for URDF and standalone meshes.
|
||||
2. Choose the route in [Workflow](docs/WORKFLOWS.md). Simulate both physics and
|
||||
command response before building a controlled application.
|
||||
3. For legged locomotion, calibrate the stance and run `scaffold-tier3`. It
|
||||
generates `tasks/<robot_id>/robot_spec.py`. The biped, quadruped, wheeled
|
||||
biped, and multipod templates supply observations, rewards, domain
|
||||
randomization, curriculum, and PPO.
|
||||
4. Run `python scripts/train.py --list` to discover registered robots and
|
||||
`python scripts/train.py --task <robot_id> --dry-run` to check resolution.
|
||||
|
||||
A native `.app` contains the model, its dependencies, controller or policy, and
|
||||
a Python runtime entry point. The host supplies Python and the listed runtime
|
||||
libraries. The existing Rust browser and board workflow uses the legacy bundle
|
||||
format. The native exporter and runtime support wheeled-biped hybrid
|
||||
actions. The legacy Rust runtime accepts its position-action layout.
|
||||
|
||||
The four templates completed one PPO update each on native CPU and Warp GPU
|
||||
using synthetic robot fixtures. These runs validate the execution paths.
|
||||
|
||||
The [Go1 example](assets/go1_example/README.md) also completed 1,000 PPO updates
|
||||
on an RTX 5080. Its selected checkpoint averaged 0.401 m/s at a 0.400 m/s
|
||||
forward command over 18 seconds, with 0.022 m/s mean absolute error and no
|
||||
resets. Native checkpoint and ONNX replay passed forward, reverse, lateral,
|
||||
turning and idle probes. These are flat-floor simulation results, not hardware
|
||||
qualification or evidence that every robot will learn with the same run size.
|
||||
|
||||
The [wheeled-biped example](assets/wheeled_biped_example/README.md) verifies
|
||||
hybrid leg-position and wheel-velocity control. Its selected policy passed
|
||||
native forward, reverse, yaw and idle probes, with keyboard and gamepad tests
|
||||
at its declared ±0.91 m/s and ±0.77 rad/s ranges. The
|
||||
[G1 biped example](assets/g1_example/README.md) passed seven isolated native
|
||||
probes and full-range keyboard tests after 7,800 Warp updates. The
|
||||
[development record](docs/REFACTOR_STATUS.md) includes accepted applications
|
||||
and unsuccessful trials, with their measured reasons.
|
||||
|
||||
## Reference
|
||||
|
||||
- [Project guide](docs/PROJECT_GUIDE.md) for navigation and supported boundaries.
|
||||
- [Tutorial](docs/TUTORIAL.md) for verification, training, and measured replay.
|
||||
- [Colab](docs/COLAB.md) for GPU training with your own registered robot.
|
||||
- [Hardware and model facts](docs/HARDWARE.md) for calibration and actuator inputs.
|
||||
- [Deployment format](deploy/BUNDLE.md) for application contents and runtime contracts.
|
||||
- [Contributing](CONTRIBUTING.md) for repository layout and checks.
|
||||
@@ -0,0 +1,73 @@
|
||||
# MJRL 机器人平台
|
||||
|
||||
<!-- tracks: README.md @ sha256:5e58614ff2c467bd -->
|
||||
|
||||
[English](README.md) · [接入流程](docs/WORKFLOWS.md) · [命令手册](docs/USAGE.zh.md)
|
||||
|
||||
把机器人模型导入 MuJoCo,验证动力学和控制响应,再打包为原生仿真应用。
|
||||
平台可以适配兼容的外部策略,也可以通过同一训练环境在原生 MuJoCo CPU 或
|
||||
MuJoCo Warp GPU 上训练运动策略。
|
||||
|
||||
| 层级 | 起点 | 产物 |
|
||||
|---|---|---|
|
||||
| 0 | 网格、URDF 或缺少已验证控制的 MJCF | 可编译模型和查看应用 |
|
||||
| 1 | 兼容策略与明确的观测、动作和关节元数据 | 适配后的 ONNX、布局及数值一致性结果 |
|
||||
| 2 | 完整模型与已知运动学 | 确定性控制器及原生验证报告 |
|
||||
| 3 | 站姿已标定且控制意图明确的足式机器人 | 机器人规格及锁定形态模板训练 |
|
||||
|
||||
## 运行模型查看应用
|
||||
|
||||
在代码库根目录中使用 Python 3.10–3.13:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python -m pip install -e ".[dev]"
|
||||
python scripts/onboard.py inspect assets/smoke/smoke_biped.xml
|
||||
python scripts/deploy.py --robot viewer_smoke --model assets/smoke/smoke_biped.xml --bundle out/viewer_demo
|
||||
python scripts/play.py --app out/viewer_demo/viewer_smoke.app --headless --steps 20
|
||||
```
|
||||
|
||||
选择尚未使用的输出目录。Windows 上使用 `.venv\Scripts\Activate.ps1` 激活。
|
||||
该模型用于检验加载与打包。接入自己的机器人时,替换为实际模型并按检查报告的建议继续。
|
||||
|
||||
## 接入自己的机器人
|
||||
|
||||
1. 用 `scripts/onboard.py inspect` 检查模型和可用的配置或策略元数据。URDF
|
||||
和独立网格可用 `build-mjcf` 转换。
|
||||
2. 根据[接入流程](docs/WORKFLOWS.md)选择层级。打包可控制应用前,分别验证
|
||||
动力学和命令响应。
|
||||
3. 足式机器人先标定站姿,再用 `scaffold-tier3` 生成
|
||||
`tasks/<robot_id>/robot_spec.py`。双足、四足、轮足双足和多足模板负责
|
||||
观测、奖励、域随机化、课程和 PPO。
|
||||
4. 用 `python scripts/train.py --list` 查看注册机器人,用
|
||||
`python scripts/train.py --task <robot_id> --dry-run` 检查参数解析。
|
||||
|
||||
原生 `.app` 包含模型、资产依赖、控制器或策略以及 Python 入口。宿主提供
|
||||
Python 和所列依赖。现有 Rust 浏览器与板端走独立的旧版包格式。轮足双足的混合
|
||||
动作可导出并在原生运行时回放,旧版 Rust 使用其位置动作布局。
|
||||
|
||||
四个模板分别在原生 CPU 和 Warp GPU 上用合成模型完成过一次 PPO 更新,
|
||||
用于验证执行路径。
|
||||
|
||||
[Go1 示例](assets/go1_example/README.md)还在 RTX 5080 上完成了 1,000 次 PPO
|
||||
更新。选定检查点在 0.400 m/s 前进命令下,18 秒回放平均速度为 0.401 m/s,
|
||||
平均绝对误差为 0.022 m/s,无重置。原生检查点与 ONNX 回放通过了前进、后退、
|
||||
横移、转向及静止探测。这些是平地仿真结果,不是硬件资格认证,也不代表每个
|
||||
机器人都能用相同训练规模学会控制。
|
||||
|
||||
[轮足双足示例](assets/wheeled_biped_example/README.md)验证了腿部位置与车轮速度
|
||||
混合控制。所选策略通过了原生前进、后退、转向和静止探测,并在其声明的
|
||||
±0.91 m/s 和 ±0.77 rad/s 范围内完成键盘及手柄测试。
|
||||
[G1 双足示例](assets/g1_example/README.md)在完成 7,800 次 Warp 更新后,通过了
|
||||
七项独立原生探测和满量程键盘测试。
|
||||
[研发记录](docs/REFACTOR_STATUS.md)汇总已验收应用、未通过的试验及实测原因。
|
||||
|
||||
## 参考
|
||||
|
||||
- [项目指南](docs/PROJECT_GUIDE.zh.md)
|
||||
- [教程](docs/TUTORIAL.zh.md)
|
||||
- [Colab](docs/COLAB.zh.md) 用于在 GPU 上训练自己注册的机器人
|
||||
- [硬件与模型参数](docs/HARDWARE.zh.md)
|
||||
- [部署包格式](deploy/BUNDLE.md)
|
||||
- [贡献指南](CONTRIBUTING.md)
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"schema": "mjrl-controller/1",
|
||||
"kind": "cpg",
|
||||
"joints": ["fl", "fr", "rl", "rr"],
|
||||
"frequency_hz": 0.6,
|
||||
"amplitude": [0.08, 0.08, 0.08, 0.08],
|
||||
"phase": [0, 3.141592653589793, 3.141592653589793, 0],
|
||||
"kp": 40,
|
||||
"kd": 4,
|
||||
"effort_limit": 4
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
<mujoco model="cpg_stance">
|
||||
<option timestep="0.002" integrator="implicitfast"/>
|
||||
<default>
|
||||
<joint axis="0 1 0" range="-30 30" armature="0.08" damping="0.15"/>
|
||||
<default class="leg">
|
||||
<geom type="capsule" size="0.013" fromto="0 0 0 0 0 -0.13" mass="0.07"
|
||||
rgba="0.21 0.57 0.74 1"/>
|
||||
</default>
|
||||
<default class="foot">
|
||||
<geom type="sphere" pos="0 0 -0.13" size="0.03" mass="0.05"
|
||||
friction="0.7 0.01 0.001" rgba="0.8 0.44 0.13 1"/>
|
||||
</default>
|
||||
</default>
|
||||
<worldbody>
|
||||
<light pos="0 -1 2" dir="0 0 -1"/>
|
||||
<geom name="floor" type="plane" size="5 5 0.1" rgba="0.8 0.82 0.86 1"/>
|
||||
<body name="base" pos="0 0 0.183">
|
||||
<freejoint name="root"/>
|
||||
<geom type="box" size="0.13 0.1 0.025" mass="1.2" rgba="0.16 0.25 0.33 1"/>
|
||||
<body name="front_left" pos="0.12 0.09 -0.025">
|
||||
<joint name="fl"/>
|
||||
<geom class="leg" mass="0.08"/>
|
||||
<geom class="foot" name="fl_foot"/>
|
||||
</body>
|
||||
<body name="front_right" pos="0.12 -0.09 -0.025">
|
||||
<joint name="fr"/>
|
||||
<geom class="leg"/>
|
||||
<geom class="foot" name="fr_foot"/>
|
||||
</body>
|
||||
<body name="rear_left" pos="-0.12 0.09 -0.025">
|
||||
<joint name="rl"/>
|
||||
<geom class="leg"/>
|
||||
<geom class="foot" name="rl_foot"/>
|
||||
</body>
|
||||
<body name="rear_right" pos="-0.12 -0.09 -0.025">
|
||||
<joint name="rr"/>
|
||||
<geom class="leg"/>
|
||||
<geom class="foot" name="rr_foot"/>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
<keyframe>
|
||||
<!-- A two-degree roll perturbation makes the startup measurement nontrivial. -->
|
||||
<key name="start" qpos="0 0 0.183 0.999847695156 0.017452406437 0 0 0 0 0 0"/>
|
||||
</keyframe>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"root_body": "base",
|
||||
"keyframe": "start",
|
||||
"ground_geom": "floor",
|
||||
"foot_geoms": ["fl_foot", "fr_foot", "rl_foot", "rr_foot"],
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"steps": 5200,
|
||||
"thresholds": {
|
||||
"max_root_tilt_rad": 0.08,
|
||||
"max_root_drop_m": 0.015,
|
||||
"max_root_translation_m": 0.03,
|
||||
"position_atol": 0.035
|
||||
},
|
||||
"probes": [{
|
||||
"command": {"amplitude_scale": 1},
|
||||
"actuator_controls": {
|
||||
"mjrl_fl": 1.2063715789784806,
|
||||
"mjrl_fr": -1.2063715789784806,
|
||||
"mjrl_rl": -1.2063715789784806,
|
||||
"mjrl_rr": 1.2063715789784806
|
||||
},
|
||||
"joint_positions": {"fl": 0.079842138, "fr": -0.079842138, "rl": -0.079842138, "rr": 0.079842138}
|
||||
}]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"kind": "differential",
|
||||
"left_joint": "left_wheel",
|
||||
"right_joint": "right_wheel",
|
||||
"wheel_radius": 0.1,
|
||||
"track_width": 0.26,
|
||||
"max_wheel_speed": 8,
|
||||
"kv": 5,
|
||||
"effort_limit": 5
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
<mujoco model="differential_drive">
|
||||
<option timestep="0.002" integrator="implicitfast"/>
|
||||
<default>
|
||||
<joint damping="0.02" armature="0.01"/>
|
||||
<geom friction="1 0.01 0.001"/>
|
||||
</default>
|
||||
<worldbody>
|
||||
<light pos="0 -1 2" dir="0 0 -1"/>
|
||||
<geom name="floor" type="plane" size="2 2 0.1" rgba="0.8 0.82 0.86 1"/>
|
||||
<body name="chassis" pos="0 0 0.12">
|
||||
<freejoint name="root"/>
|
||||
<geom type="box" size="0.14 0.1 0.025" mass="1.5" rgba="0.24 0.53 0.68 1"/>
|
||||
<geom type="sphere" pos="-0.13 0 -0.095" size="0.025" mass="0.05"
|
||||
friction="0.03 0.01 0.001" rgba="0.28 0.30 0.34 1"/>
|
||||
<body name="right" pos="0 -0.13 -0.02">
|
||||
<joint name="right_wheel" axis="0 1 0"/>
|
||||
<geom name="right_tire" type="cylinder" euler="90 0 0" size="0.1 0.02"
|
||||
mass="0.25" rgba="0.19 0.22 0.25 1"/>
|
||||
</body>
|
||||
<body name="left" pos="0 0.13 -0.02">
|
||||
<joint name="left_wheel" axis="0 1 0"/>
|
||||
<geom name="left_tire" type="cylinder" euler="90 0 0" size="0.1 0.02"
|
||||
mass="0.25" rgba="0.19 0.22 0.25 1"/>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"root_body": "chassis",
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"ground_geom": "floor",
|
||||
"foot_geoms": ["left_tire", "right_tire"],
|
||||
"steps": 750,
|
||||
"thresholds": {
|
||||
"base_displacement_atol_m": 0.055,
|
||||
"base_angular_velocity_atol_rad_s": 0.18
|
||||
},
|
||||
"probes": [
|
||||
{
|
||||
"pad": {"Ly": -0.7},
|
||||
"actuator_controls": {"mjrl_left_wheel": 3.5, "mjrl_right_wheel": 3.5},
|
||||
"joint_velocities": {"left_wheel": 3.5, "right_wheel": 3.5},
|
||||
"base_displacement_m": [0.525, 0, 0]
|
||||
},
|
||||
{
|
||||
"pad": {"Rx": -1},
|
||||
"actuator_controls": {"mjrl_left_wheel": -1.3, "mjrl_right_wheel": 1.3},
|
||||
"base_angular_velocity_rad_s": [0, 0, 1]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
# Unitree G1 biped training example
|
||||
|
||||
This 29-joint humanoid uses the Unitree G1 model and STL meshes from the
|
||||
local [mjlab asset zoo](../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/g1.xml),
|
||||
introduced in commit `004afda1ac1dafc3aedd4e9ca8c6d2242b399f57`. The files
|
||||
come from MuJoCo Menagerie and are copyright 2016–2023 HangZhou YuShu
|
||||
TECHNOLOGY CO., LTD. (Unitree Robotics), under the
|
||||
[BSD 3-Clause License](../../licenses/mujoco_menagerie-unitree_g1.txt).
|
||||
The repository [NOTICE](../../NOTICE) and [provenance](model-provenance.json)
|
||||
identify the files and their hashes.
|
||||
|
||||
`g1.xml` rebases mesh paths and applies the reflected motor armatures from
|
||||
the vendored [G1 constants](../../rl/mjlab/asset_zoo/robots/unitree_g1/g1_constants.py).
|
||||
The compiled masses, joint limits, mesh vertices and reference pose match
|
||||
the source exactly. The MJCF round trip changes collision geom sizes by
|
||||
at most 4.4e−8 m and quaternion components by at most 4.2e−7. Several
|
||||
articulated links carry visual meshes and inertia only. Each named ankle
|
||||
roll body carries seven collision capsules that represent its sole.
|
||||
|
||||
## Stance calibration
|
||||
|
||||
`stance.json` specifies a symmetric, slightly bent stance with a 0.24 rad
|
||||
knee angle and −0.12 rad hip and ankle pitch. It uses the vendored effort
|
||||
limits. The calibration estimates each joint's apparent inertia as
|
||||
`1 / diagonal(inverse(M))`, using the full mass matrix `M` of the floating
|
||||
robot at that pose. This captures the inertia when other coordinates can
|
||||
move in response to the applied joint effort. Gains use 14 Hz, 28 Hz at the
|
||||
ankles and a damping ratio of 0.85. In the reference hold, the floating-base
|
||||
model with mass-matrix diagonal gains and the vendored home pose fell.
|
||||
|
||||
The two-second native hold completed 1,000 steps. The peak base drift was
|
||||
2.30 mm and the peak base rotation was 0.00410 rad. The soles supported
|
||||
99.9998% of the robot's 327 N weight on average. The maximum joint tracking
|
||||
error was 0.0126 rad and peak effort was 12.6% of its configured limit. There
|
||||
were zero self contacts and zero MuJoCo warnings.
|
||||
|
||||
`robot.json` specifies an action scale of 60% of effort limit divided by
|
||||
proportional gain per joint. Limb contacts carry a penalty. Pelvis, torso
|
||||
and head contacts terminate the episode. The generated
|
||||
[robot specification](../../tasks/g1_example/robot_spec.py) is the only
|
||||
training task file and uses the locked biped template.
|
||||
|
||||
From the checkout root with the Python environment activated, use fresh
|
||||
output paths for the reports. The scaffold command writes a new robot
|
||||
specification when its destination is empty.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/g1_example/g1.xml \
|
||||
--profile assets/g1_example/profile.json --report out/g1_example/intake.json
|
||||
python scripts/onboard.py calibrate --model assets/g1_example/g1.xml \
|
||||
--config assets/g1_example/stance.json --report out/g1_example/calibration.json
|
||||
python scripts/onboard.py scaffold-tier3 --config assets/g1_example/robot.json \
|
||||
--calibration out/g1_example/calibration.json --tasks-root out/g1_example/scaffold
|
||||
python scripts/train.py --task g1_example --backend native --device cpu \
|
||||
--num-envs 4 --headless --no-tensorboard --max-iterations 1
|
||||
```
|
||||
|
||||
## Warp constraint allocation
|
||||
|
||||
A 1,024-environment run with the default Warp allocation aborted before
|
||||
completing its first update. It reported that the 64-row constraint buffer
|
||||
was too small during contacts. A 64-environment smoke also logged overflows.
|
||||
The model needs room for the constraints from its soles and contacts made
|
||||
during falls. The following run uses the vendored G1 flat configuration's
|
||||
300-row allocation while leaving model contacts and template behavior as
|
||||
configured. Training saves the allocation for matching Warp replay.
|
||||
|
||||
```bash
|
||||
python scripts/train.py --task g1_example --backend warp --device cuda \
|
||||
--num-envs 1024 --warp-max-constraints 300 --headless --no-tensorboard \
|
||||
--max-iterations 1200
|
||||
```
|
||||
|
||||
## Measured training and replay
|
||||
|
||||
The first run completed 1,200 updates and 29,491,200 environment steps on
|
||||
an RTX 5080. Physics ran at 400 Hz and policy control at 50 Hz. The Warp
|
||||
allocation record reported 300 constraints per environment and 49,152
|
||||
contacts shared across 1,024 environments. The replay used the saved
|
||||
command ranges and disabled observation noise.
|
||||
|
||||
In a 12-second Warp replay with a +0.25 m/s held command, the first
|
||||
checkpoint averaged +0.0002 m/s with 0.250 m/s mean absolute error.
|
||||
Checkpoint 1199 averaged +0.218 m/s with 0.070 m/s mean absolute error.
|
||||
The same checkpoint averaged −0.185 m/s under a −0.25 m/s command with
|
||||
0.066 m/s error. All seven held-command and idle replays had zero episode
|
||||
resets on both Warp and native MuJoCo. The lateral and yaw tests missed
|
||||
their targets. For example, Warp measured +0.007 m/s against +0.12 m/s
|
||||
lateral and +0.047 rad/s against +0.30 rad/s yaw. The exported native ONNX
|
||||
actor passed the physics stage and failed the tracking stage on the same
|
||||
seven commands.
|
||||
|
||||
The held-command checks use forward and reverse at 0.25 m/s, left and
|
||||
right at 0.12 m/s, yaw in both directions at 0.30 rad/s, and idle. Each
|
||||
replay lasts 12 seconds. The ONNX acceptance requires a mean error and
|
||||
mean absolute error within 0.12 m/s forward, 0.06 m/s lateral and
|
||||
0.15 rad/s yaw, along with native physics checks. For the Warp forward
|
||||
measurement, substitute the actual checkpoint path from the run.
|
||||
|
||||
```bash
|
||||
python scripts/play.py --task g1_example --checkpoint path/to/model_1199.pt \
|
||||
--backend warp --device cuda --warp-max-constraints 300 \
|
||||
--headless --steps 600 --speed 0 --measure --checkpoint-command-ranges \
|
||||
--no-obs-noise --command 0.25 0 0 --evaluation-out out/g1_example/forward.json
|
||||
```
|
||||
|
||||
A continuation from checkpoint 1199 initially widened the command range
|
||||
before collecting a complete fresh episode. The runner had randomized
|
||||
starting episode ages, so the restored curriculum clock admitted biased
|
||||
tracking evidence. That run was stopped and excluded. After the resume
|
||||
fix, training restarted from checkpoint 1199 and kept the saved curriculum
|
||||
level through the fresh 1,000-step window.
|
||||
|
||||
The checkpoint 1800 forward preview shows a short shuffle. Native ONNX
|
||||
contact measurements between 2 and 12 seconds found both soles loaded in
|
||||
65.8% of samples and one sole loaded in the remainder. Peak sole clearance
|
||||
was 5.7 mm. Loaded ground contact points slid at mean speeds of 0.111 and
|
||||
0.127 m/s on the two soles. Both soles stayed loaded throughout lateral
|
||||
and yaw replay. These are simulator gait diagnostics.
|
||||
|
||||
The corrected continuation added 1,600 updates for a total of 68,812,800
|
||||
environment steps across both valid runs. Checkpoint 2798 stayed at
|
||||
curriculum level 0. In Warp it averaged +0.257 m/s forward and −0.210 m/s
|
||||
reverse, with 0.036 and 0.046 m/s mean absolute errors. All seven native
|
||||
and Warp replays completed with zero resets. The standalone lateral and
|
||||
yaw commands still missed their targets. Native ONNX replay passed physics
|
||||
and failed tracking acceptance.
|
||||
|
||||
Mixed commands showed that the policy could control all three axes when
|
||||
the forward command was present. Under `[0.25, 0.12, 0.30]`, native replay
|
||||
averaged `[0.244, 0.107, 0.289]` in body-frame m/s, m/s and rad/s. Under
|
||||
`[0.25, -0.12, -0.30]`, it averaged `[0.290, -0.112, -0.303]`. Switching
|
||||
from forward motion to a side-only or yaw-only command brought the robot
|
||||
back to a planted stance. The example currently provides the calibrated
|
||||
model and training task. The ONNX export is retained as a diagnostic.
|
||||
|
||||
A second continuation added 3,000 updates, bringing the total to 5,800
|
||||
updates and 142,540,800 environment steps. The run stayed at curriculum
|
||||
level 0 and logged no Warp capacity overflows. Checkpoint 5797 returned
|
||||
these 12-second results on the training backend with observation noise
|
||||
turned off.
|
||||
|
||||
| Command | Target | Mean | Mean absolute error |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Forward, m/s | +0.25 | +0.240 | 0.017 |
|
||||
| Reverse, m/s | −0.25 | −0.240 | 0.013 |
|
||||
| Left, m/s | +0.12 | +0.003 | 0.117 |
|
||||
| Right, m/s | −0.12 | −0.115 | 0.017 |
|
||||
| Yaw left, rad/s | +0.30 | +0.225 | 0.089 |
|
||||
| Yaw right, rad/s | −0.30 | −0.264 | 0.059 |
|
||||
|
||||
All seven replays, including idle, completed without resets on both Warp
|
||||
and native MuJoCo. The native ONNX policy passed physics verification. It
|
||||
still failed the leftward command and exceeded the 0.15 rad/s yaw-error
|
||||
limit by 0.00025 rad/s during the forward probe. In the ONNX forward gait,
|
||||
peak sole clearance remained 5.5 mm. Loaded contact slip decreased to
|
||||
0.057 and 0.089 m/s. Both soles remained loaded during the left command.
|
||||
This checkpoint is retained for diagnosis without a release app. The
|
||||
compliant-gain trial trained a fresh policy with the same mechanical model
|
||||
and softer hip and knee gains, but failed acceptance. Its task and source assets
|
||||
were removed; the failure record remains in
|
||||
[the refactor report](../../docs/REFACTOR_STATUS.md#g1-learning-and-selection).
|
||||
|
||||
## Verified checkpoint 7796
|
||||
|
||||
The last continuation added 2,000 updates. Across the four valid runs, the
|
||||
original-gain model completed 7,800 updates and 191,692,800 environment
|
||||
steps. The saved curriculum level was 0, which provides command ranges
|
||||
of ±0.28 m/s forward, ±0.14 m/s lateral and ±0.32 rad/s yaw.
|
||||
|
||||
Checkpoint 7796 passed the original seven isolated-command probes in
|
||||
native ONNX replay. The table gives 12-second body-frame measurements for
|
||||
the commanded axis. The corresponding Warp and native checkpoint replays
|
||||
had zero resets, and the ONNX verifier recorded zero physics warnings.
|
||||
|
||||
| Command | Target | Warp mean | Native ONNX mean | ONNX mean absolute error |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Forward, m/s | +0.25 | +0.259 | +0.258 | 0.020 |
|
||||
| Reverse, m/s | −0.25 | −0.241 | −0.242 | 0.015 |
|
||||
| Left, m/s | +0.12 | +0.105 | +0.105 | 0.028 |
|
||||
| Right, m/s | −0.12 | −0.152 | −0.152 | 0.034 |
|
||||
| Yaw left, rad/s | +0.30 | +0.217 | +0.215 | 0.108 |
|
||||
| Yaw right, rad/s | −0.30 | −0.296 | −0.297 | 0.068 |
|
||||
|
||||
The idle ONNX replay held both soles loaded, with mean body-frame speeds
|
||||
of +0.00002 m/s forward, −0.0022 m/s lateral and +0.0046 rad/s yaw. In a
|
||||
separate test, each of W, S, A, D, J and L was held for 12 seconds, including
|
||||
the configured two-second ramp to its saved command limit. The highest
|
||||
per-axis mean absolute error across the six tests was 0.023 m/s forward,
|
||||
0.037 m/s lateral and 0.102 rad/s yaw. Escape brought the robot back to idle
|
||||
in each test, with zero physics warnings.
|
||||
|
||||
The final forward gait still used short, low steps. Both soles carried load
|
||||
in 51% of post-transient samples, and peak sole clearance was 5.7 mm. Left
|
||||
motion unloaded one sole in 49% of samples, with peak sole clearance 9.0 mm.
|
||||
These results describe simulated operation on the training plane.
|
||||
@@ -0,0 +1,97 @@
|
||||
# g1_example operator controls -- how a person's input becomes the command the
|
||||
# policy observes.
|
||||
#
|
||||
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
|
||||
# every control name from `controller/vocabulary.json`. Run
|
||||
# `python -m controller --vocabulary` to see the list; it is the same one the
|
||||
# deployed controller refuses a binding against.
|
||||
#
|
||||
# The pad and the keyboard are two paths, each bound straight to the command
|
||||
# axes -- a stick with a sign, a key with the direction it pushes -- and
|
||||
# neither names a control of the other. Either device drives alone; the one
|
||||
# touched last drives.
|
||||
#
|
||||
# Two rules this file inherits from the specification it replaced:
|
||||
# * Every value carries its source. A number without one cannot be told
|
||||
# apart from a guess.
|
||||
# * A consumer fails on a missing or malformed value rather than
|
||||
# substituting a default. A sign that defaults to +1 is a robot that walks
|
||||
# backwards while the screen looks perfectly normal.
|
||||
|
||||
schema_version: 2
|
||||
|
||||
command:
|
||||
- term: twist
|
||||
# The observation term these numbers are written into.
|
||||
feeds: velocity_commands
|
||||
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
|
||||
frame: body
|
||||
# Order is the order the command term writes them, and therefore the order
|
||||
# they appear in the observation.
|
||||
axes:
|
||||
- name: lin_vel_x
|
||||
unit: m/s
|
||||
positive: forward
|
||||
- name: lin_vel_y
|
||||
unit: m/s
|
||||
positive: left
|
||||
- name: ang_vel_z
|
||||
unit: rad/s
|
||||
positive: counter-clockwise
|
||||
scaling:
|
||||
# Full deflection is the edge of the range the policy was **trained** on,
|
||||
# not an absolute speed -- so a stick means "as fast as this policy was
|
||||
# ever asked to go" and cannot ask for something off-distribution.
|
||||
rule: full_deflection_is_range_edge
|
||||
formula: axis * (range_max if axis >= 0 else -range_min)
|
||||
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
|
||||
# what the policy trained on; taking the larger end for both quietly
|
||||
# commands something it never saw.
|
||||
ends_scaled_separately: true
|
||||
|
||||
devices:
|
||||
gamepad:
|
||||
# The stick position *is* the command.
|
||||
scheme: absolute
|
||||
layout: xbox
|
||||
axes:
|
||||
lin_vel_x: { source: Ly, sign: -1 }
|
||||
lin_vel_y: { source: Lx, sign: -1 }
|
||||
ang_vel_z: { source: Rx, sign: -1 }
|
||||
# MEASURED: the signs are not measured against the robot's pad service.
|
||||
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
|
||||
# is **down**, ABS_X positive is **right** -- and the service passes both
|
||||
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
|
||||
# the robot is what would settle it.
|
||||
#
|
||||
# The service applies a 0.15 deadband and **rescales**, so full deflection
|
||||
# still reaches ±1. A consumer that applies its own on top narrows the
|
||||
# range a person can ask for and cannot tell that it has.
|
||||
deadzone: device_reported_rescaled
|
||||
# Hands the command back to the random sampler in `play --task`; on a robot, which
|
||||
# has no sampler, it lets go of everything.
|
||||
release_button: B
|
||||
|
||||
keyboard:
|
||||
# Keystrokes bound to what they do: each pushes one direction of one axis,
|
||||
# "+" being the axis's own positive direction as `positive:` above words
|
||||
# it, so the keyboard has no signs of its own -- there is no stick for a
|
||||
# sign to be relative to. A virtual pad read through the pad's mapping
|
||||
# until 2026-09-29.
|
||||
scheme: keys
|
||||
# Seconds a key is held for full deflection: the axis climbs linearly
|
||||
# until then and is back at rest the moment the key comes up, as a stick
|
||||
# let go of. Asked for on 2026-09-26; measured against nothing.
|
||||
full_after_s: 2.0
|
||||
axes:
|
||||
# W S and the arrows walk, A D and the arrows step sideways, J L turn:
|
||||
# the layout as the user
|
||||
# decided it on 2026-09-29. Every letter is also a MuJoCo viewer
|
||||
# shortcut -- W flips the scene to wireframe -- which was accepted for a
|
||||
# keyboard laid out for the hands; left and right also step a paused
|
||||
# scene.
|
||||
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
|
||||
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
|
||||
ang_vel_z: { "+": [key_j], "-": [key_l] }
|
||||
# Lets go of everything, as the pad's release button does.
|
||||
release: [key_escape]
|
||||
@@ -0,0 +1,267 @@
|
||||
<mujoco model="g1_29dof_rev_1_0">
|
||||
<compiler angle="radian" />
|
||||
<default>
|
||||
<default class="g1">
|
||||
<site group="5" rgba="1 0 0 1" />
|
||||
<default class="visual">
|
||||
<geom type="mesh" contype="0" conaffinity="0" group="2" density="0" material="silver" />
|
||||
</default>
|
||||
<default class="collision">
|
||||
<geom type="capsule" group="3" rgba="0.2 0.6 0.2 0.3" />
|
||||
<default class="foot_capsule">
|
||||
<geom size="0.01 0 0" />
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
<asset>
|
||||
<material name="silver" rgba="0.7 0.7 0.7 1" />
|
||||
<material name="black" rgba="0.2 0.2 0.2 1" />
|
||||
<mesh name="pelvis" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/pelvis.STL" />
|
||||
<mesh name="pelvis_contour_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/pelvis_contour_link.STL" />
|
||||
<mesh name="left_hip_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_pitch_link.STL" />
|
||||
<mesh name="left_hip_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_roll_link.STL" />
|
||||
<mesh name="left_hip_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_hip_yaw_link.STL" />
|
||||
<mesh name="left_knee_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_knee_link.STL" />
|
||||
<mesh name="left_ankle_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_ankle_pitch_link.STL" />
|
||||
<mesh name="left_ankle_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_ankle_roll_link.STL" />
|
||||
<mesh name="right_hip_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_pitch_link.STL" />
|
||||
<mesh name="right_hip_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_roll_link.STL" />
|
||||
<mesh name="right_hip_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_hip_yaw_link.STL" />
|
||||
<mesh name="right_knee_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_knee_link.STL" />
|
||||
<mesh name="right_ankle_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_ankle_pitch_link.STL" />
|
||||
<mesh name="right_ankle_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_ankle_roll_link.STL" />
|
||||
<mesh name="waist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/waist_yaw_link_rev_1_0.STL" />
|
||||
<mesh name="waist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/waist_roll_link_rev_1_0.STL" />
|
||||
<mesh name="torso_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/torso_link_rev_1_0.STL" />
|
||||
<mesh name="logo_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/logo_link.STL" />
|
||||
<mesh name="head_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/head_link.STL" />
|
||||
<mesh name="left_shoulder_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_pitch_link.STL" />
|
||||
<mesh name="left_shoulder_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_roll_link.STL" />
|
||||
<mesh name="left_shoulder_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_shoulder_yaw_link.STL" />
|
||||
<mesh name="left_elbow_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_elbow_link.STL" />
|
||||
<mesh name="left_wrist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_roll_link.STL" />
|
||||
<mesh name="left_wrist_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_pitch_link.STL" />
|
||||
<mesh name="left_wrist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_wrist_yaw_link.STL" />
|
||||
<mesh name="left_rubber_hand" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/left_rubber_hand.STL" />
|
||||
<mesh name="right_shoulder_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_pitch_link.STL" />
|
||||
<mesh name="right_shoulder_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_roll_link.STL" />
|
||||
<mesh name="right_shoulder_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_shoulder_yaw_link.STL" />
|
||||
<mesh name="right_elbow_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_elbow_link.STL" />
|
||||
<mesh name="right_wrist_roll_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_roll_link.STL" />
|
||||
<mesh name="right_wrist_pitch_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_pitch_link.STL" />
|
||||
<mesh name="right_wrist_yaw_link" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_wrist_yaw_link.STL" />
|
||||
<mesh name="right_rubber_hand" file="../../rl/mjlab/asset_zoo/robots/unitree_g1/xmls/assets/right_rubber_hand.STL" />
|
||||
</asset>
|
||||
<worldbody>
|
||||
<body name="pelvis" childclass="g1" pos="0 0 0.793">
|
||||
<inertial pos="0 0 -0.07605" quat="1 0 -0.000399148 0" mass="3.813" diaginertia="0.010549 0.0093089 0.0079184" />
|
||||
<joint name="floating_base_joint" type="free" />
|
||||
<geom class="visual" material="black" mesh="pelvis" />
|
||||
<geom class="visual" mesh="pelvis_contour_link" />
|
||||
<geom name="pelvis_collision" class="collision" size="0.07" pos="0 0 -0.08" type="sphere" />
|
||||
<site name="imu_in_pelvis" pos="0.04525 0 -0.08339" size="0.01" />
|
||||
<camera name="tracking" pos="1.734 -1.135 0.35" quat="0.683353 0.555871 0.298684 0.367183" mode="trackcom" />
|
||||
<light pos="0 0 2" dir="0 0 -1" castshadow="false" mode="trackcom" />
|
||||
<body name="left_hip_pitch_link" pos="0 0.064452 -0.1027">
|
||||
<inertial pos="0.002741 0.047791 -0.02606" quat="0.954862 0.293964 0.0302556 0.030122" mass="1.35" diaginertia="0.00181517 0.00153422 0.00116212" />
|
||||
<joint name="left_hip_pitch_joint" pos="0 0 0" axis="0 1 0" range="-2.5307 2.8798" armature="0.0101775" />
|
||||
<geom class="visual" material="black" mesh="left_hip_pitch_link" />
|
||||
<body name="left_hip_roll_link" pos="0 0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
|
||||
<inertial pos="0.029812 -0.001045 -0.087934" quat="0.977808 -1.97119e-05 0.205576 -0.0403793" mass="1.52" diaginertia="0.00254986 0.00241169 0.00148755" />
|
||||
<joint name="left_hip_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.5236 2.9671" armature="0.0251019" />
|
||||
<geom class="visual" mesh="left_hip_roll_link" />
|
||||
<geom name="left_hip_collision" class="collision" size="0.06 0.04" pos="0.02 0 -0.04" />
|
||||
<body name="left_hip_yaw_link" pos="0.025001 0 -0.12412">
|
||||
<inertial pos="-0.057709 -0.010981 -0.15078" quat="0.600598 0.15832 0.223482 0.751181" mass="1.702" diaginertia="0.00776166 0.00717575 0.00160139" />
|
||||
<joint name="left_hip_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.7576 2.7576" armature="0.0101775" />
|
||||
<geom class="visual" mesh="left_hip_yaw_link" />
|
||||
<geom name="left_thigh_collision" class="collision" size="0.055 0.0761577" pos="-0.03 0 -0.1" quat="0.979578 0 0.201066 0" />
|
||||
<body name="left_knee_link" pos="-0.078273 0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
|
||||
<inertial pos="0.005457 0.003964 -0.12074" quat="0.923418 -0.0327699 0.0158246 0.382067" mass="1.932" diaginertia="0.0113804 0.0112778 0.00146458" />
|
||||
<joint name="left_knee_joint" pos="0 0 0" axis="0 1 0" range="-0.087267 2.8798" armature="0.0251019" />
|
||||
<geom class="visual" mesh="left_knee_link" />
|
||||
<geom name="left_shin_collision" class="collision" size="0.045 0.075" pos="0.01 0 -0.075" />
|
||||
<geom name="left_linkage_brace_collision" class="collision" size="0.03 0.04" pos="0.01 0 -0.24" />
|
||||
<body name="left_ankle_pitch_link" pos="0 -9.4445e-05 -0.30001">
|
||||
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074" diaginertia="1.89e-05 1.40805e-05 6.9195e-06" />
|
||||
<joint name="left_ankle_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.87267 0.5236" armature="0.00721945" />
|
||||
<geom class="visual" mesh="left_ankle_pitch_link" />
|
||||
<body name="left_ankle_roll_link" pos="0 0 -0.017558">
|
||||
<inertial pos="0.026505 0 -0.016425" quat="-0.000481092 0.728482 -0.000618967 0.685065" mass="0.608" diaginertia="0.00167218 0.0016161 0.000217621" />
|
||||
<joint name="left_ankle_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.2618 0.2618" armature="0.00721945" />
|
||||
<geom class="visual" material="black" mesh="left_ankle_roll_link" />
|
||||
<geom name="left_foot1_collision" class="foot_capsule" size="0.01 0.025005" pos="0.075 -0.0265 -0.025" quat="0.707107 -0.0141393 0.706965 0" />
|
||||
<geom name="left_foot2_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 -0.018 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="left_foot3_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 -0.01 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="left_foot4_collision" class="foot_capsule" size="0.01 0.093" pos="0.039 0 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="left_foot5_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 0.01 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="left_foot6_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 0.018 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="left_foot7_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 0.026 -0.025" quat="0.707107 0 0.707107 0" />
|
||||
<site name="left_foot" pos="0.04 0 -0.037" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_hip_pitch_link" pos="0 -0.064452 -0.1027">
|
||||
<inertial pos="0.002741 -0.047791 -0.02606" quat="0.954862 -0.293964 0.0302556 -0.030122" mass="1.35" diaginertia="0.00181517 0.00153422 0.00116212" />
|
||||
<joint name="right_hip_pitch_joint" pos="0 0 0" axis="0 1 0" range="-2.5307 2.8798" armature="0.0101775" />
|
||||
<geom class="visual" material="black" mesh="right_hip_pitch_link" />
|
||||
<body name="right_hip_roll_link" pos="0 -0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
|
||||
<inertial pos="0.029812 0.001045 -0.087934" quat="0.977808 1.97119e-05 0.205576 0.0403793" mass="1.52" diaginertia="0.00254986 0.00241169 0.00148755" />
|
||||
<joint name="right_hip_roll_joint" pos="0 0 0" axis="1 0 0" range="-2.9671 0.5236" armature="0.0251019" />
|
||||
<geom class="visual" mesh="right_hip_roll_link" />
|
||||
<geom name="right_hip_collision" class="collision" size="0.06 0.04" pos="0.02 0 -0.04" />
|
||||
<body name="right_hip_yaw_link" pos="0.025001 0 -0.12412">
|
||||
<inertial pos="-0.057709 0.010981 -0.15078" quat="0.751181 0.223482 0.15832 0.600598" mass="1.702" diaginertia="0.00776166 0.00717575 0.00160139" />
|
||||
<joint name="right_hip_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.7576 2.7576" armature="0.0101775" />
|
||||
<geom class="visual" mesh="right_hip_yaw_link" />
|
||||
<geom name="right_thigh_collision" class="collision" size="0.055 0.0761577" pos="-0.03 0 -0.1" quat="0.979578 0 0.201066 0" />
|
||||
<body name="right_knee_link" pos="-0.078273 -0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
|
||||
<inertial pos="0.005457 -0.003964 -0.12074" quat="0.923439 0.0345276 0.0116333 -0.382012" mass="1.932" diaginertia="0.011374 0.0112843 0.00146452" />
|
||||
<joint name="right_knee_joint" pos="0 0 0" axis="0 1 0" range="-0.087267 2.8798" armature="0.0251019" />
|
||||
<geom class="visual" mesh="right_knee_link" />
|
||||
<geom name="right_shin_collision" class="collision" size="0.045 0.075" pos="0.01 0 -0.075" />
|
||||
<geom name="right_linkage_brace_collision" class="collision" size="0.03 0.04" pos="0.01 0 -0.24" />
|
||||
<body name="right_ankle_pitch_link" pos="0 9.4445e-05 -0.30001">
|
||||
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074" diaginertia="1.89e-05 1.40805e-05 6.9195e-06" />
|
||||
<joint name="right_ankle_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.87267 0.5236" armature="0.00721945" />
|
||||
<geom class="visual" mesh="right_ankle_pitch_link" />
|
||||
<body name="right_ankle_roll_link" pos="0 0 -0.017558">
|
||||
<inertial pos="0.026505 0 -0.016425" quat="0.000481092 0.728482 0.000618967 0.685065" mass="0.608" diaginertia="0.00167218 0.0016161 0.000217621" />
|
||||
<joint name="right_ankle_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.2618 0.2618" armature="0.00721945" />
|
||||
<geom class="visual" material="black" mesh="right_ankle_roll_link" />
|
||||
<geom name="right_foot1_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 -0.026 -0.025" quat="0.707107 0 0.707107 0" />
|
||||
<geom name="right_foot2_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 -0.018 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="right_foot3_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 -0.01 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="right_foot4_collision" class="foot_capsule" size="0.01 0.093" pos="0.039 0 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="right_foot5_collision" class="foot_capsule" size="0.01 0.091" pos="0.039 0.01 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="right_foot6_collision" class="foot_capsule" size="0.01 0.0835" pos="0.0395 0.018 -0.025" quat="0.707107 0 -0.707107 0" />
|
||||
<geom name="right_foot7_collision" class="foot_capsule" size="0.01 0.025" pos="0.075 0.026 -0.025" quat="0.707107 0 0.707107 0" />
|
||||
<site name="right_foot" pos="0.04 0 -0.037" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="waist_yaw_link">
|
||||
<inertial pos="0.003494 0.000233 0.018034" quat="0.289697 0.591001 -0.337795 0.672821" mass="0.214" diaginertia="0.000163531 0.000107714 0.000102205" />
|
||||
<joint name="waist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.0101775" />
|
||||
<geom class="visual" mesh="waist_yaw_link" />
|
||||
<body name="waist_roll_link" pos="-0.0039635 0 0.044">
|
||||
<inertial pos="0 2.3e-05 0" quat="0.5 0.5 -0.5 0.5" mass="0.086" diaginertia="8.245e-06 7.079e-06 6.339e-06" />
|
||||
<joint name="waist_roll_joint" pos="0 0 0" axis="1 0 0" range="-0.52 0.52" armature="0.00721945" />
|
||||
<geom class="visual" mesh="waist_roll_link" />
|
||||
<body name="torso_link">
|
||||
<inertial pos="0.00203158 0.000339683 0.184568" quat="0.999803 -6.03319e-05 0.0198256 0.00131986" mass="7.818" diaginertia="0.121847 0.109825 0.0273735" />
|
||||
<joint name="waist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-0.52 0.52" armature="0.00721945" />
|
||||
<geom class="visual" mesh="torso_link" />
|
||||
<geom class="visual" pos="0.0039635 0 -0.044" quat="1 0 0 0" material="black" mesh="logo_link" />
|
||||
<geom class="visual" pos="0.0039635 0 -0.044" material="black" mesh="head_link" />
|
||||
<geom name="torso_collision" class="collision" size="0.09 0.06" pos="0.01 0 0.14" quat="0 1 0 0" />
|
||||
<geom name="head_collision" class="collision" size="0.06" pos="0 0 0.43" type="sphere" />
|
||||
<site name="imu_in_torso" pos="-0.03959 -0.00224 0.14792" size="0.01" />
|
||||
<body name="left_shoulder_pitch_link" pos="0.0039563 0.10022 0.24778" quat="0.990264 0.139201 1.38722e-05 -9.86868e-05">
|
||||
<inertial pos="0 0.035892 -0.011628" quat="0.654152 0.0130458 -0.326267 0.68225" mass="0.718" diaginertia="0.000465864 0.000432842 0.000406394" />
|
||||
<joint name="left_shoulder_pitch_joint" pos="0 0 0" axis="0 1 0" range="-3.0892 2.6704" armature="0.00360972" />
|
||||
<geom class="visual" mesh="left_shoulder_pitch_link" />
|
||||
<body name="left_shoulder_roll_link" pos="0 0.038 -0.013831" quat="0.990268 -0.139172 0 0">
|
||||
<inertial pos="-0.000227 0.00727 -0.063243" quat="0.701256 -0.0196223 -0.00710317 0.712604" mass="0.643" diaginertia="0.000691311 0.000618011 0.000388977" />
|
||||
<joint name="left_shoulder_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.5882 2.2515" armature="0.00360972" />
|
||||
<geom class="visual" mesh="left_shoulder_roll_link" />
|
||||
<body name="left_shoulder_yaw_link" pos="0 0.00624 -0.1032">
|
||||
<inertial pos="0.010773 -0.002949 -0.072009" quat="0.716879 -0.0964829 -0.0679942 0.687134" mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661" />
|
||||
<joint name="left_shoulder_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.00360972" />
|
||||
<geom class="visual" mesh="left_shoulder_yaw_link" />
|
||||
<geom name="left_shoulder_yaw_collision" class="collision" size="0.035 0.065" pos="0 0 -0.015" quat="0 1 0 0" />
|
||||
<body name="left_elbow_link" pos="0.015783 0 -0.080518">
|
||||
<inertial pos="0.064956 0.004454 -0.010062" quat="0.541765 0.636132 0.388821 0.388129" mass="0.6" diaginertia="0.000443035 0.000421612 0.000259353" />
|
||||
<joint name="left_elbow_joint" pos="0 0 0" axis="0 1 0" range="-1.0472 2.0944" armature="0.00360972" />
|
||||
<geom class="visual" mesh="left_elbow_link" />
|
||||
<geom name="left_elbow_yaw_collision" class="collision" size="0.035 0.045" pos="0.035 0 -0.01" quat="0.707107 0 -0.707107 0" />
|
||||
<body name="left_wrist_roll_link" pos="0.1 0.00188791 -0.01">
|
||||
<inertial pos="0.0171394 0.000537591 4.8864e-07" quat="0.575338 0.411667 -0.574906 0.411094" mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05" />
|
||||
<joint name="left_wrist_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.97222 1.97222" armature="0.00360972" />
|
||||
<geom class="visual" mesh="left_wrist_roll_link" />
|
||||
<body name="left_wrist_pitch_link" pos="0.038 0 0">
|
||||
<inertial pos="0.0229999 -0.00111685 -0.00111658" quat="0.249998 0.661363 0.293036 0.643608" mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648" />
|
||||
<joint name="left_wrist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-1.61443 1.61443" armature="0.00425" />
|
||||
<geom class="visual" mesh="left_wrist_pitch_link" />
|
||||
<geom name="left_wrist_collision" class="collision" size="0.035 0.035" pos="0.025 0 0" quat="0.707107 0 -0.707107 0" />
|
||||
<body name="left_wrist_yaw_link" pos="0.046 0 0">
|
||||
<inertial pos="0.0708244 0.000191745 0.00161742" quat="0.510571 0.526295 0.468078 0.493188" mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566" />
|
||||
<joint name="left_wrist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-1.61443 1.61443" armature="0.00425" />
|
||||
<geom class="visual" mesh="left_wrist_yaw_link" />
|
||||
<geom class="visual" pos="0.0415 0.003 0" quat="1 0 0 0" material="black" mesh="left_rubber_hand" />
|
||||
<geom name="left_hand_collision" class="collision" size="0.035 0.0412311" pos="0.11 -0.01 0" quat="0.707107 -0.171499 -0.685994 0" />
|
||||
<site name="left_palm" pos="0.08 0 0" size="0.01" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_shoulder_pitch_link" pos="0.0039563 -0.10021 0.24778" quat="0.990264 -0.139201 1.38722e-05 9.86868e-05">
|
||||
<inertial pos="0 -0.035892 -0.011628" quat="0.68225 -0.326267 0.0130458 0.654152" mass="0.718" diaginertia="0.000465864 0.000432842 0.000406394" />
|
||||
<joint name="right_shoulder_pitch_joint" pos="0 0 0" axis="0 1 0" range="-3.0892 2.6704" armature="0.00360972" />
|
||||
<geom class="visual" mesh="right_shoulder_pitch_link" />
|
||||
<body name="right_shoulder_roll_link" pos="0 -0.038 -0.013831" quat="0.990268 0.139172 0 0">
|
||||
<inertial pos="-0.000227 -0.00727 -0.063243" quat="0.712604 -0.00710317 -0.0196223 0.701256" mass="0.643" diaginertia="0.000691311 0.000618011 0.000388977" />
|
||||
<joint name="right_shoulder_roll_joint" pos="0 0 0" axis="1 0 0" range="-2.2515 1.5882" armature="0.00360972" />
|
||||
<geom class="visual" mesh="right_shoulder_roll_link" />
|
||||
<body name="right_shoulder_yaw_link" pos="0 -0.00624 -0.1032">
|
||||
<inertial pos="0.010773 0.002949 -0.072009" quat="0.687134 -0.0679942 -0.0964829 0.716879" mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661" />
|
||||
<joint name="right_shoulder_yaw_joint" pos="0 0 0" axis="0 0 1" range="-2.618 2.618" armature="0.00360972" />
|
||||
<geom class="visual" mesh="right_shoulder_yaw_link" />
|
||||
<geom name="right_shoulder_yaw_collision" class="collision" size="0.035 0.065" pos="0 0 -0.015" quat="0 1 0 0" />
|
||||
<body name="right_elbow_link" pos="0.015783 0 -0.080518">
|
||||
<inertial pos="0.064956 -0.004454 -0.010062" quat="0.388129 0.388821 0.636132 0.541765" mass="0.6" diaginertia="0.000443035 0.000421612 0.000259353" />
|
||||
<joint name="right_elbow_joint" pos="0 0 0" axis="0 1 0" range="-1.0472 2.0944" armature="0.00360972" />
|
||||
<geom class="visual" mesh="right_elbow_link" />
|
||||
<geom name="right_elbow_yaw_collision" class="collision" size="0.035 0.045" pos="0.035 0 -0.01" quat="0.707107 0 -0.707107 0" />
|
||||
<body name="right_wrist_roll_link" pos="0.1 -0.00188791 -0.01">
|
||||
<inertial pos="0.0171394 -0.000537591 4.8864e-07" quat="0.411667 0.575338 -0.411094 0.574906" mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05" />
|
||||
<joint name="right_wrist_roll_joint" pos="0 0 0" axis="1 0 0" range="-1.97222 1.97222" armature="0.00360972" />
|
||||
<geom class="visual" mesh="right_wrist_roll_link" />
|
||||
<body name="right_wrist_pitch_link" pos="0.038 0 0">
|
||||
<inertial pos="0.0229999 0.00111685 -0.00111658" quat="0.643608 0.293036 0.661363 0.249998" mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648" />
|
||||
<joint name="right_wrist_pitch_joint" pos="0 0 0" axis="0 1 0" range="-1.61443 1.61443" armature="0.00425" />
|
||||
<geom class="visual" mesh="right_wrist_pitch_link" />
|
||||
<geom name="right_wrist_collision" class="collision" size="0.035 0.035" pos="0.025 0 0" quat="0.707107 0 -0.707107 0" />
|
||||
<body name="right_wrist_yaw_link" pos="0.046 0 0">
|
||||
<inertial pos="0.0708244 -0.000191745 0.00161742" quat="0.493188 0.468078 0.526295 0.510571" mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566" />
|
||||
<joint name="right_wrist_yaw_joint" pos="0 0 0" axis="0 0 1" range="-1.61443 1.61443" armature="0.00425" />
|
||||
<geom class="visual" mesh="right_wrist_yaw_link" />
|
||||
<geom class="visual" pos="0.0415 -0.003 0" quat="1 0 0 0" material="black" mesh="right_rubber_hand" />
|
||||
<geom name="right_hand_collision" class="collision" size="0.035 0.0412311" pos="0.11 0.01 0" quat="0.707107 0.171499 -0.685994 0" />
|
||||
<site name="right_palm" pos="0.08 0 0" size="0.01" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
<contact>
|
||||
<exclude body1="pelvis" body2="left_hip_roll_link" />
|
||||
<exclude body1="pelvis" body2="right_hip_roll_link" />
|
||||
<exclude body1="left_elbow_link" body2="left_wrist_pitch_link" />
|
||||
<exclude body1="right_elbow_link" body2="right_wrist_pitch_link" />
|
||||
</contact>
|
||||
<sensor>
|
||||
<gyro site="imu_in_pelvis" name="imu_ang_vel" />
|
||||
<velocimeter site="imu_in_pelvis" name="imu_lin_vel" />
|
||||
<accelerometer site="imu_in_pelvis" name="imu_lin_acc" />
|
||||
<framezaxis objtype="body" objname="world" reftype="site" refname="imu_in_pelvis" name="imu_upvector" />
|
||||
<subtreeangmom body="pelvis" name="root_angmom" />
|
||||
</sensor>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,138 @@
|
||||
{
|
||||
"schema": "g1_example_asset_provenance/1",
|
||||
"local_source_commit": "004afda1ac1dafc3aedd4e9ca8c6d2242b399f57",
|
||||
"source_xml": "rl/mjlab/asset_zoo/robots/unitree_g1/xmls/g1.xml",
|
||||
"source_xml_sha256": "febdcbeffbbf84051556ae41a5ac1b43fb479a5d76bdb3f54824dbc2721c20aa",
|
||||
"normalized_xml": "assets/g1_example/g1.xml",
|
||||
"normalized_xml_sha256": "a29a91395e727135341377c71ec3c27c55e9849a07c4b5df3daf7fc7ddf77cf7",
|
||||
"mesh_sha256": {
|
||||
"head_link.STL": "005fb67fbd3eff94aa8bf4a6e83238174e9f91b6721f7111594322f223724411",
|
||||
"left_ankle_pitch_link.STL": "d49e3abc6f5b12e532062cd575b87b5ef40cd2a3fc18f54a1ca5bba4f773d51d",
|
||||
"left_ankle_roll_link.STL": "c4092af943141d4d9f74232f3cfa345afc6565f46a077793b8ae0e68b39dc33f",
|
||||
"left_elbow_link.STL": "fa752198accd104d5c4c3a01382e45165b944fbbc5acce085059223324e5bed3",
|
||||
"left_hip_pitch_link.STL": "4725168105ee768ee31638ef22b53f6be2d7641bfd7cfefe803488d884776fa4",
|
||||
"left_hip_roll_link.STL": "91f25922ee8a7c3152790051bebad17b4d9cd243569c38fe340285ff93a97acf",
|
||||
"left_hip_yaw_link.STL": "a16d88aa6ddac8083aa7ad55ed317bea44b1fa003d314fba88965b7ed0f3b55b",
|
||||
"left_knee_link.STL": "8d92b9e3d3a636761150bb8025e32514c4602b91c7028d523ee42b3e632de477",
|
||||
"left_rubber_hand.STL": "cff2221a690fa69303f61fce68f2d155c1517b52efb6ca9262dd56e0bc6e70fe",
|
||||
"left_shoulder_pitch_link.STL": "f0d1cfd02fcf0d42f95e678eeca33da3afbcc366ffba5c052847773ec4f31d52",
|
||||
"left_shoulder_roll_link.STL": "fb9df21687773522598dc384f1a2945c7519f11cbc8bd372a49170316d6eee88",
|
||||
"left_shoulder_yaw_link.STL": "1aa97e9748e924336567992181f78c7cd0652fd52a4afcca3df6b2ef6f9e712e",
|
||||
"left_wrist_pitch_link.STL": "b251d8e05047f695d0f536cd78c11973cfa4e78d08cfe82759336cc3471de3a9",
|
||||
"left_wrist_roll_link.STL": "edc387c9a0ba8c2237e9b296d32531426fabeb6f53e58df45c76106bca74148c",
|
||||
"left_wrist_yaw_link.STL": "83f8fb3a726bf9613d65dd14f0f447cb918c3c95b3938042a0c9c09749267d3b",
|
||||
"logo_link.STL": "8571a0a19bc4916fa55f91449f51d5fdefd751000054865a842449429d5f155b",
|
||||
"pelvis.STL": "5ba6bbc888e630550140d3c26763f10206da8c8bd30ed886b8ede41c61f57a31",
|
||||
"pelvis_contour_link.STL": "5cc5c2c7a312329e3feeb2b03d3fc09fc29705bd01864f6767e51be959662420",
|
||||
"right_ankle_pitch_link.STL": "15be426539ec1be70246d4d82a168806db64a41301af8b35c197a33348c787a9",
|
||||
"right_ankle_roll_link.STL": "4b66222ea56653e627711b56d0a8949b4920da5df091da0ceb343f54e884e3a5",
|
||||
"right_elbow_link.STL": "1be925d7aa268bb8fddf5362b9173066890c7d32092c05638608126e59d1e2ab",
|
||||
"right_hip_pitch_link.STL": "e4f3c99d7f4a7d34eadbef9461fc66e3486cb5442db1ec50c86317d459f1a9c6",
|
||||
"right_hip_roll_link.STL": "4c254ef66a356f492947f360dd931965477b631e3fcc841f91ccc46d945d54f6",
|
||||
"right_hip_yaw_link.STL": "e479c2936ca2057e9eb2f7dff6c189b7419d7b8484dea0b298cbb36a2a6aa668",
|
||||
"right_knee_link.STL": "63c4008449c9bbe701a6e2b557b7a252e90cf3a5abcf54cee46862b9a69f8ec8",
|
||||
"right_rubber_hand.STL": "99533b778bca6246144fa511bb9d4e555e075c641f2a0251e04372869cd99d67",
|
||||
"right_shoulder_pitch_link.STL": "24cdb387e0128dfe602770a81c56cdce3a0181d34d039a11d1aaf8819b7b8c02",
|
||||
"right_shoulder_roll_link.STL": "962b97c48f9ce9e8399f45dd9522e0865d19aa9fd299406b2d475a8fc4a53e81",
|
||||
"right_shoulder_yaw_link.STL": "a0b76489271da0c72461a344c9ffb0f0c6e64f019ea5014c1624886c442a2fe5",
|
||||
"right_wrist_pitch_link.STL": "d22f8f3b3127f15a63e5be1ee273cd5075786c3142f1c3d9f76cbf43d2a26477",
|
||||
"right_wrist_roll_link.STL": "a7ee9212ff5b94d6cb7f52bb1bbf3f352194d5b598acff74f4c77d340c5b344f",
|
||||
"right_wrist_yaw_link.STL": "bc9dece2d12509707e0057ba2e48df8f3d56db0c79410212963a25e8a50f61a6",
|
||||
"torso_link_rev_1_0.STL": "11ddb46f2098efbbd8816b1d65893632d8e78be936376c7cdcd6771899ccc723",
|
||||
"waist_roll_link_rev_1_0.STL": "b67c347a05abc3e8ddae600d98a082bee273bb39f8e651647708b0a7140a8a97",
|
||||
"waist_yaw_link_rev_1_0.STL": "ec6db442b11f25eed898b5add07940c85d804f300de24dcbd264ccd8be7d554c"
|
||||
},
|
||||
"license": "licenses/mujoco_menagerie-unitree_g1.txt",
|
||||
"nq": 36,
|
||||
"nv": 35,
|
||||
"nu": 0,
|
||||
"mass_kg": 33.341142000000005,
|
||||
"roundtrip_max_absolute_differences": {
|
||||
"body_mass": 0.0,
|
||||
"body_inertia": 0.0,
|
||||
"body_pos": 0.0,
|
||||
"body_quat": 0.0,
|
||||
"jnt_range": 0.0,
|
||||
"jnt_axis": 0.0,
|
||||
"geom_pos": 2.7755575615628914e-17,
|
||||
"geom_quat": 4.161878089103954e-07,
|
||||
"geom_size": 4.3743823403830095e-08,
|
||||
"geom_contype": 0.0,
|
||||
"geom_conaffinity": 0.0,
|
||||
"qpos0": 0.0
|
||||
},
|
||||
"armature_before": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"armature_after": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0101775,
|
||||
0.0251019,
|
||||
0.0101775,
|
||||
0.0251019,
|
||||
0.00721945,
|
||||
0.00721945,
|
||||
0.0101775,
|
||||
0.0251019,
|
||||
0.0101775,
|
||||
0.0251019,
|
||||
0.00721945,
|
||||
0.00721945,
|
||||
0.0101775,
|
||||
0.00721945,
|
||||
0.00721945,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00425,
|
||||
0.00425,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00360972,
|
||||
0.00425,
|
||||
0.00425
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"base_body": "pelvis",
|
||||
"control_intent": "learned",
|
||||
"family": "biped",
|
||||
"foot_bodies": [
|
||||
"left_ankle_roll_link",
|
||||
"right_ankle_roll_link"
|
||||
],
|
||||
"position_joints": [
|
||||
"left_hip_pitch_joint",
|
||||
"left_hip_roll_joint",
|
||||
"left_hip_yaw_joint",
|
||||
"left_knee_joint",
|
||||
"left_ankle_pitch_joint",
|
||||
"left_ankle_roll_joint",
|
||||
"right_hip_pitch_joint",
|
||||
"right_hip_roll_joint",
|
||||
"right_hip_yaw_joint",
|
||||
"right_knee_joint",
|
||||
"right_ankle_pitch_joint",
|
||||
"right_ankle_roll_joint",
|
||||
"waist_yaw_joint",
|
||||
"waist_roll_joint",
|
||||
"waist_pitch_joint",
|
||||
"left_shoulder_pitch_joint",
|
||||
"left_shoulder_roll_joint",
|
||||
"left_shoulder_yaw_joint",
|
||||
"left_elbow_joint",
|
||||
"left_wrist_roll_joint",
|
||||
"left_wrist_pitch_joint",
|
||||
"left_wrist_yaw_joint",
|
||||
"right_shoulder_pitch_joint",
|
||||
"right_shoulder_roll_joint",
|
||||
"right_shoulder_yaw_joint",
|
||||
"right_elbow_joint",
|
||||
"right_wrist_roll_joint",
|
||||
"right_wrist_pitch_joint",
|
||||
"right_wrist_yaw_joint"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"action_scales": {
|
||||
"left_ankle_pitch_joint": 0.102076,
|
||||
"left_ankle_roll_joint": 0.12836,
|
||||
"left_elbow_joint": 0.170088,
|
||||
"left_hip_pitch_joint": 0.126324,
|
||||
"left_hip_roll_joint": 0.092448,
|
||||
"left_hip_yaw_joint": 0.355179,
|
||||
"left_knee_joint": 0.191022,
|
||||
"left_shoulder_pitch_joint": 0.053176,
|
||||
"left_shoulder_roll_joint": 0.039043,
|
||||
"left_shoulder_yaw_joint": 0.239303,
|
||||
"left_wrist_pitch_joint": 0.063044,
|
||||
"left_wrist_roll_joint": 0.489431,
|
||||
"left_wrist_yaw_joint": 0.070853,
|
||||
"right_ankle_pitch_joint": 0.102076,
|
||||
"right_ankle_roll_joint": 0.12836,
|
||||
"right_elbow_joint": 0.170127,
|
||||
"right_hip_pitch_joint": 0.126313,
|
||||
"right_hip_roll_joint": 0.092451,
|
||||
"right_hip_yaw_joint": 0.355601,
|
||||
"right_knee_joint": 0.191022,
|
||||
"right_shoulder_pitch_joint": 0.053254,
|
||||
"right_shoulder_roll_joint": 0.039045,
|
||||
"right_shoulder_yaw_joint": 0.239308,
|
||||
"right_wrist_pitch_joint": 0.063045,
|
||||
"right_wrist_roll_joint": 0.489431,
|
||||
"right_wrist_yaw_joint": 0.070853,
|
||||
"waist_pitch_joint": 0.099043,
|
||||
"waist_roll_joint": 0.028443,
|
||||
"waist_yaw_joint": 0.133551
|
||||
},
|
||||
"controls_path": "controls.yaml",
|
||||
"family": "biped",
|
||||
"model_path": "g1.xml",
|
||||
"penalized_contact_geoms": [
|
||||
"left_hip_collision",
|
||||
"left_thigh_collision",
|
||||
"left_shin_collision",
|
||||
"left_linkage_brace_collision",
|
||||
"right_hip_collision",
|
||||
"right_thigh_collision",
|
||||
"right_shin_collision",
|
||||
"right_linkage_brace_collision",
|
||||
"left_shoulder_yaw_collision",
|
||||
"left_elbow_yaw_collision",
|
||||
"left_wrist_collision",
|
||||
"left_hand_collision",
|
||||
"right_shoulder_yaw_collision",
|
||||
"right_elbow_yaw_collision",
|
||||
"right_wrist_collision",
|
||||
"right_hand_collision"
|
||||
],
|
||||
"robot_id": "g1_example",
|
||||
"termination_contact_geoms": [
|
||||
"pelvis_collision",
|
||||
"torso_collision",
|
||||
"head_collision"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
{
|
||||
"damping_ratio": 0.85,
|
||||
"effort_limits": {
|
||||
"left_ankle_pitch_joint": 50.0,
|
||||
"left_ankle_roll_joint": 50.0,
|
||||
"left_elbow_joint": 25.0,
|
||||
"left_hip_pitch_joint": 88.0,
|
||||
"left_hip_roll_joint": 139.0,
|
||||
"left_hip_yaw_joint": 88.0,
|
||||
"left_knee_joint": 139.0,
|
||||
"left_shoulder_pitch_joint": 25.0,
|
||||
"left_shoulder_roll_joint": 25.0,
|
||||
"left_shoulder_yaw_joint": 25.0,
|
||||
"left_wrist_pitch_joint": 5.0,
|
||||
"left_wrist_roll_joint": 25.0,
|
||||
"left_wrist_yaw_joint": 5.0,
|
||||
"right_ankle_pitch_joint": 50.0,
|
||||
"right_ankle_roll_joint": 50.0,
|
||||
"right_elbow_joint": 25.0,
|
||||
"right_hip_pitch_joint": 88.0,
|
||||
"right_hip_roll_joint": 139.0,
|
||||
"right_hip_yaw_joint": 88.0,
|
||||
"right_knee_joint": 139.0,
|
||||
"right_shoulder_pitch_joint": 25.0,
|
||||
"right_shoulder_roll_joint": 25.0,
|
||||
"right_shoulder_yaw_joint": 25.0,
|
||||
"right_wrist_pitch_joint": 5.0,
|
||||
"right_wrist_roll_joint": 25.0,
|
||||
"right_wrist_yaw_joint": 5.0,
|
||||
"waist_pitch_joint": 50.0,
|
||||
"waist_roll_joint": 50.0,
|
||||
"waist_yaw_joint": 88.0
|
||||
},
|
||||
"foot_bodies": [
|
||||
"left_ankle_roll_link",
|
||||
"right_ankle_roll_link"
|
||||
],
|
||||
"frequency_hz": 14.0,
|
||||
"inertia_mode": "coupled",
|
||||
"joint_frequency_hz": {
|
||||
"left_ankle_pitch_joint": 28.0,
|
||||
"left_ankle_roll_joint": 28.0,
|
||||
"right_ankle_pitch_joint": 28.0,
|
||||
"right_ankle_roll_joint": 28.0
|
||||
},
|
||||
"pose": {
|
||||
"left_ankle_pitch_joint": -0.12,
|
||||
"left_ankle_roll_joint": 0.0,
|
||||
"left_elbow_joint": 0.8,
|
||||
"left_hip_pitch_joint": -0.12,
|
||||
"left_hip_roll_joint": 0.0,
|
||||
"left_hip_yaw_joint": 0.0,
|
||||
"left_knee_joint": 0.24,
|
||||
"left_shoulder_pitch_joint": 0.2,
|
||||
"left_shoulder_roll_joint": 0.35,
|
||||
"left_shoulder_yaw_joint": 0.0,
|
||||
"left_wrist_pitch_joint": 0.0,
|
||||
"left_wrist_roll_joint": 0.0,
|
||||
"left_wrist_yaw_joint": 0.0,
|
||||
"right_ankle_pitch_joint": -0.12,
|
||||
"right_ankle_roll_joint": 0.0,
|
||||
"right_elbow_joint": 0.8,
|
||||
"right_hip_pitch_joint": -0.12,
|
||||
"right_hip_roll_joint": 0.0,
|
||||
"right_hip_yaw_joint": 0.0,
|
||||
"right_knee_joint": 0.24,
|
||||
"right_shoulder_pitch_joint": 0.2,
|
||||
"right_shoulder_roll_joint": -0.35,
|
||||
"right_shoulder_yaw_joint": 0.0,
|
||||
"right_wrist_pitch_joint": 0.0,
|
||||
"right_wrist_roll_joint": 0.0,
|
||||
"right_wrist_yaw_joint": 0.0,
|
||||
"waist_pitch_joint": 0.0,
|
||||
"waist_roll_joint": 0.0,
|
||||
"waist_yaw_joint": 0.0
|
||||
},
|
||||
"root_body": "pelvis",
|
||||
"seconds": 2
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
# Unitree Go1 quadruped training example
|
||||
|
||||
The model comes from the local mjlab asset zoo at
|
||||
[unitree_go1/xmls/go1.xml](../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/go1.xml).
|
||||
The vendored source entered this checkout in commit
|
||||
`004afda1ac1dafc3aedd4e9ca8c6d2242b399f57`. The source XML SHA-256 is
|
||||
`2f1856b0c83994c009cafbad1f5eac38d09efa1a64db73681ff72c9237822db3`.
|
||||
The five STL assets are referenced in place. The model and meshes are from
|
||||
MuJoCo Menagerie, copyright 2016–2022 HangZhou YuShu TECHNOLOGY CO., LTD.
|
||||
(Unitree Robotics), under the
|
||||
[BSD 3-Clause License](../../licenses/mujoco_menagerie-unitree_go1.txt).
|
||||
See the repository [NOTICE](../../NOTICE).
|
||||
|
||||
`go1.xml` reuses the source geometry, masses, joint stops, collision geoms and
|
||||
reference frames. It adds reflected rotor inertia from the vendored
|
||||
[go1_constants.py](../../rl/mjlab/asset_zoo/robots/unitree_go1/go1_constants.py)
|
||||
and rebases mesh paths. `stance.json` uses the same source for the seed joint
|
||||
positions and effort limits. Its gains come from the model inertia at the
|
||||
calibrated pose, using 14 Hz and a damping ratio of 1.25. At that setting a
|
||||
2-second native hold supported the robot on all four named feet. Peak base
|
||||
displacement was 16.1 mm, peak joint error was 0.0562 rad, and peak effort
|
||||
reached 20.0% of its joint limit. The original vendored gains let the stance
|
||||
settle onto a calf in the same test.
|
||||
|
||||
`robot.json` sets each policy action scale to 60% of its effort limit divided
|
||||
by its calibrated proportional gain. This is an exploration scale for the
|
||||
simulator. Thigh and calf ground contacts carry a penalty. Trunk and head
|
||||
ground contacts terminate the episode. The only task file is the calibrated
|
||||
[robot specification](../../tasks/go1_example/robot_spec.py), which uses the
|
||||
locked quadruped template.
|
||||
|
||||
## Reproduce the intake
|
||||
|
||||
Run from the checkout root with its installed Python environment and fresh
|
||||
report paths.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/go1_example/go1.xml \
|
||||
--profile assets/go1_example/profile.json --report out/go1_example/intake.json
|
||||
python scripts/onboard.py calibrate --model assets/go1_example/go1.xml \
|
||||
--config assets/go1_example/stance.json --report out/go1_example/calibration.json
|
||||
python scripts/train.py --task go1_example --backend warp --device cuda \
|
||||
--num-envs 2048 --headless --no-tensorboard --max-iterations 1000
|
||||
```
|
||||
|
||||
The measured stance and any learned policy results are simulation evidence.
|
||||
|
||||
## Measured training run
|
||||
|
||||
The RTX 5080 completed 1,000 Warp PPO updates with 2,048 environments and 24
|
||||
steps per environment per update, totaling 49,152,000 environment steps. The
|
||||
run reached all three curriculum levels. Checkpoint 900 was selected from
|
||||
that run after its four feet remained loaded for the final three seconds of
|
||||
a 12-second idle test. Checkpoint 999 tracked motion but unloaded its
|
||||
front-right foot during the same idle check.
|
||||
|
||||
With a +0.40 m/s command in an 18-second Warp replay, checkpoint 900
|
||||
averaged +0.401 m/s with 0.022 m/s mean absolute forward error and no resets.
|
||||
The first saved checkpoint was nearly stationary with 0.400 m/s mean
|
||||
absolute forward error under the same command. Both used the saved curriculum
|
||||
ranges, 400 Hz physics, 50 Hz control and observation noise disabled.
|
||||
|
||||
The following 12-second tests used held body-frame commands, native MuJoCo
|
||||
physics at 400 Hz and 50 Hz control. Both columns use checkpoint 900 with
|
||||
observation noise disabled. The checkpoint replay had zero resets in each
|
||||
test. The standalone ONNX replay had zero MuJoCo warnings.
|
||||
|
||||
| Commanded axis and target | Checkpoint mean | ONNX mean | ONNX mean absolute error |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Forward +0.40 m/s | +0.400 | +0.400 | 0.022 |
|
||||
| Reverse −0.35 m/s | −0.337 | −0.337 | 0.018 |
|
||||
| Left +0.25 m/s | +0.232 | +0.232 | 0.019 |
|
||||
| Right −0.25 m/s | −0.235 | −0.235 | 0.018 |
|
||||
| Yaw left +0.60 rad/s | +0.587 | +0.587 | 0.039 |
|
||||
| Yaw right −0.60 rad/s | −0.593 | −0.591 | 0.040 |
|
||||
|
||||
With an idle command, ONNX replay averaged +0.0002 m/s forward, −0.0014 m/s
|
||||
lateral and −0.0014 rad/s yaw. A separate keyboard test held W, S, A, D, J and
|
||||
L for 12 seconds each, including the configured two-second ramp to the saved
|
||||
command limit. Peak per-axis mean absolute tracking error across these tests
|
||||
was 0.038 m/s for translation and 0.082 rad/s for yaw. Each test stayed upright
|
||||
and settled after Escape.
|
||||
|
||||
The packaged runtime reproduced seven 12-second probes, including sampled
|
||||
joint state and per-axis body-frame tracking metrics. The app includes the
|
||||
asset license and provenance along with English and Chinese operator manuals.
|
||||
After building the app, use the repository's native reference checker.
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --check-reference out/platform_release_20261104/go1_example.app
|
||||
python scripts/play.py --app out/platform_release_20261104/go1_example.app --headless \
|
||||
--steps 200 --command 0.4 0 0
|
||||
```
|
||||
@@ -0,0 +1,97 @@
|
||||
# go1_example operator controls -- how a person's input becomes the command the
|
||||
# policy observes.
|
||||
#
|
||||
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
|
||||
# every control name from `controller/vocabulary.json`. Run
|
||||
# `python -m controller --vocabulary` to see the list; it is the same one the
|
||||
# deployed controller refuses a binding against.
|
||||
#
|
||||
# The pad and the keyboard are two paths, each bound straight to the command
|
||||
# axes -- a stick with a sign, a key with the direction it pushes -- and
|
||||
# neither names a control of the other. Either device drives alone; the one
|
||||
# touched last drives.
|
||||
#
|
||||
# Two rules this file inherits from the specification it replaced:
|
||||
# * Every value carries its source. A number without one cannot be told
|
||||
# apart from a guess.
|
||||
# * A consumer fails on a missing or malformed value rather than
|
||||
# substituting a default. A sign that defaults to +1 is a robot that walks
|
||||
# backwards while the screen looks perfectly normal.
|
||||
|
||||
schema_version: 2
|
||||
|
||||
command:
|
||||
- term: twist
|
||||
# The observation term these numbers are written into.
|
||||
feeds: velocity_commands
|
||||
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
|
||||
frame: body
|
||||
# Order is the order the command term writes them, and therefore the order
|
||||
# they appear in the observation.
|
||||
axes:
|
||||
- name: lin_vel_x
|
||||
unit: m/s
|
||||
positive: forward
|
||||
- name: lin_vel_y
|
||||
unit: m/s
|
||||
positive: left
|
||||
- name: ang_vel_z
|
||||
unit: rad/s
|
||||
positive: counter-clockwise
|
||||
scaling:
|
||||
# Full deflection is the edge of the range the policy was **trained** on,
|
||||
# not an absolute speed -- so a stick means "as fast as this policy was
|
||||
# ever asked to go" and cannot ask for something off-distribution.
|
||||
rule: full_deflection_is_range_edge
|
||||
formula: axis * (range_max if axis >= 0 else -range_min)
|
||||
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
|
||||
# what the policy trained on; taking the larger end for both quietly
|
||||
# commands something it never saw.
|
||||
ends_scaled_separately: true
|
||||
|
||||
devices:
|
||||
gamepad:
|
||||
# The stick position *is* the command.
|
||||
scheme: absolute
|
||||
layout: xbox
|
||||
axes:
|
||||
lin_vel_x: { source: Ly, sign: -1 }
|
||||
lin_vel_y: { source: Lx, sign: -1 }
|
||||
ang_vel_z: { source: Rx, sign: -1 }
|
||||
# MEASURED: the signs are not measured against the robot's pad service.
|
||||
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
|
||||
# is **down**, ABS_X positive is **right** -- and the service passes both
|
||||
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
|
||||
# the robot is what would settle it.
|
||||
#
|
||||
# The service applies a 0.15 deadband and **rescales**, so full deflection
|
||||
# still reaches ±1. A consumer that applies its own on top narrows the
|
||||
# range a person can ask for and cannot tell that it has.
|
||||
deadzone: device_reported_rescaled
|
||||
# Hands the command back to the random sampler in `play --task`; on a robot, which
|
||||
# has no sampler, it lets go of everything.
|
||||
release_button: B
|
||||
|
||||
keyboard:
|
||||
# Keystrokes bound to what they do: each pushes one direction of one axis,
|
||||
# "+" being the axis's own positive direction as `positive:` above words
|
||||
# it, so the keyboard has no signs of its own -- there is no stick for a
|
||||
# sign to be relative to. A virtual pad read through the pad's mapping
|
||||
# until 2026-09-29.
|
||||
scheme: keys
|
||||
# Seconds a key is held for full deflection: the axis climbs linearly
|
||||
# until then and is back at rest the moment the key comes up, as a stick
|
||||
# let go of. Asked for on 2026-09-26; measured against nothing.
|
||||
full_after_s: 2.0
|
||||
axes:
|
||||
# W S and the arrows walk, A D and the arrows step sideways, J L turn:
|
||||
# the layout as the user
|
||||
# decided it on 2026-09-29. Every letter is also a MuJoCo viewer
|
||||
# shortcut -- W flips the scene to wireframe -- which was accepted for a
|
||||
# keyboard laid out for the hands; left and right also step a paused
|
||||
# scene.
|
||||
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
|
||||
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
|
||||
ang_vel_z: { "+": [key_j], "-": [key_l] }
|
||||
# Lets go of everything, as the pad's release button does.
|
||||
release: [key_escape]
|
||||
@@ -0,0 +1,154 @@
|
||||
<mujoco model="go1">
|
||||
<compiler angle="radian" />
|
||||
<default>
|
||||
<default class="go1">
|
||||
<default class="visual">
|
||||
<geom type="mesh" contype="0" conaffinity="0" group="2" material="dark" />
|
||||
</default>
|
||||
<default class="collision">
|
||||
<geom type="capsule" group="3" />
|
||||
<default class="thigh1">
|
||||
<geom size="0.015 0 0" />
|
||||
</default>
|
||||
<default class="thigh2">
|
||||
<geom size="0.015 0 0" />
|
||||
</default>
|
||||
<default class="thigh3">
|
||||
<geom size="0.015 0 0" />
|
||||
</default>
|
||||
<default class="calf1">
|
||||
<geom size="0.01 0 0" />
|
||||
</default>
|
||||
<default class="calf2">
|
||||
<geom size="0.01 0 0" />
|
||||
</default>
|
||||
<default class="foot">
|
||||
<geom size="0.023 0 0" type="sphere" />
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
<asset>
|
||||
<material name="dark" rgba="0.2 0.2 0.2 1" />
|
||||
<mesh name="trunk" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/trunk.stl" />
|
||||
<mesh name="hip" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/hip.stl" />
|
||||
<mesh name="thigh_mirror" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/thigh_mirror.stl" />
|
||||
<mesh name="calf" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/calf.stl" />
|
||||
<mesh name="thigh" class="go1" file="../../rl/mjlab/asset_zoo/robots/unitree_go1/xmls/assets/thigh.stl" />
|
||||
</asset>
|
||||
<worldbody>
|
||||
<body name="trunk" childclass="go1" pos="0 0 0.445">
|
||||
<inertial pos="0.0223 0.002 -0.0005" quat="-0.00342088 0.705204 0.000106698 0.708996" mass="5.204" diaginertia="0.0716565 0.0630105 0.0168101" />
|
||||
<joint name="floating_base_joint" type="free" />
|
||||
<geom class="visual" mesh="trunk" />
|
||||
<geom name="trunk_collision" class="collision" size="0.13 0.09 0.05" type="box" />
|
||||
<geom name="head_collision" class="collision" size="0.06" pos="0.24 0 0" type="sphere" />
|
||||
<site name="head" pos="0.3 0 0" size="0.02" group="5" rgba="1 0 0 1" />
|
||||
<site name="imu" pos="-0.01592 -0.06659 -0.00617" group="5" />
|
||||
<camera name="tracking" pos="0.846 -1.3 0.316" quat="0.791164 0.554139 0.148485 0.211997" mode="trackcom" />
|
||||
<camera name="top" pos="-1 0 1" quat="0.653281 0.270598 -0.270598 -0.653281" mode="trackcom" />
|
||||
<camera name="side" pos="0 -1 0.3" quat="0.850651 0.525731 0 0" mode="trackcom" />
|
||||
<camera name="back" pos="-1 0 0.3" quat="0.601501 0.371748 -0.371748 -0.601501" mode="trackcom" />
|
||||
<light pos="0 0 2" dir="0 0 -1" mode="trackcom" />
|
||||
<body name="FR_hip" pos="0.1881 -0.04675 0">
|
||||
<inertial pos="-0.0049166 0.00762615 -8.865e-05" quat="0.507341 0.514169 0.495027 0.482891" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
|
||||
<joint name="FR_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
|
||||
<geom class="visual" mesh="hip" />
|
||||
<geom name="FR_hip_collision" class="collision" size="0.05 0.01" pos="0 -0.03 0" quat="0.707107 -0.707107 0 0" />
|
||||
<body name="FR_thigh" pos="0 -0.08 0">
|
||||
<inertial pos="-0.00304722 0.019315 -0.0305004" quat="0.65243 -0.0272313 0.0775126 0.753383" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
|
||||
<joint name="FR_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
|
||||
<geom class="visual" mesh="thigh_mirror" />
|
||||
<geom name="FR_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
|
||||
<geom name="FR_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
|
||||
<geom name="FR_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
|
||||
<body name="FR_calf" pos="0 0 -0.213">
|
||||
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
|
||||
<joint name="FR_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
|
||||
<geom class="visual" mesh="calf" />
|
||||
<geom name="FR_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
|
||||
<geom name="FR_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
|
||||
<geom name="FR_foot_collision" class="foot" pos="0 0 -0.213" />
|
||||
<site name="FR" pos="0 0 -0.213" size="0.023" group="5" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="FL_hip" pos="0.1881 0.04675 0">
|
||||
<inertial pos="-0.0049166 -0.00762615 -8.865e-05" quat="0.482891 0.495027 0.514169 0.507341" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
|
||||
<joint name="FL_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
|
||||
<geom class="visual" mesh="hip" />
|
||||
<geom name="FL_hip_collision" class="collision" size="0.05 0.01" pos="0 0.03 0" quat="0.707107 0.707107 0 0" />
|
||||
<body name="FL_thigh" pos="0 0.08 0">
|
||||
<inertial pos="-0.00304722 -0.019315 -0.0305004" quat="0.753383 0.0775126 -0.0272313 0.65243" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
|
||||
<joint name="FL_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
|
||||
<geom class="visual" mesh="thigh" />
|
||||
<geom name="FL_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
|
||||
<geom name="FL_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
|
||||
<geom name="FL_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
|
||||
<body name="FL_calf" pos="0 0 -0.213">
|
||||
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
|
||||
<joint name="FL_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
|
||||
<geom class="visual" mesh="calf" />
|
||||
<geom name="FL_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
|
||||
<geom name="FL_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
|
||||
<geom name="FL_foot_collision" class="foot" pos="0 0 -0.213" />
|
||||
<site name="FL" pos="0 0 -0.213" size="0.023" group="5" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="RR_hip" pos="-0.1881 -0.04675 0">
|
||||
<inertial pos="0.0049166 0.00762615 -8.865e-05" quat="0.495027 0.482891 0.507341 0.514169" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
|
||||
<joint name="RR_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
|
||||
<geom class="visual" quat="0 0 0 -1" mesh="hip" />
|
||||
<geom name="RR_hip_collision" class="collision" size="0.05 0.01" pos="0 -0.03 0" quat="0.707107 -0.707107 0 0" />
|
||||
<body name="RR_thigh" pos="0 -0.08 0">
|
||||
<inertial pos="-0.00304722 0.019315 -0.0305004" quat="0.65243 -0.0272313 0.0775126 0.753383" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
|
||||
<joint name="RR_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
|
||||
<geom class="visual" mesh="thigh_mirror" />
|
||||
<geom name="RR_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
|
||||
<geom name="RR_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
|
||||
<geom name="RR_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
|
||||
<body name="RR_calf" pos="0 0 -0.213">
|
||||
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
|
||||
<joint name="RR_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
|
||||
<geom class="visual" mesh="calf" />
|
||||
<geom name="RR_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
|
||||
<geom name="RR_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
|
||||
<geom name="RR_foot_collision" class="foot" pos="0 0 -0.213" />
|
||||
<site name="RR" pos="0 0 -0.213" size="0.023" group="5" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="RL_hip" pos="-0.1881 0.04675 0">
|
||||
<inertial pos="0.0049166 -0.00762615 -8.865e-05" quat="0.514169 0.507341 0.482891 0.495027" mass="0.68" diaginertia="0.000734064 0.000468438 0.000398719" />
|
||||
<joint name="RL_hip_joint" pos="0 0 0" axis="1 0 0" range="-0.863 0.863" armature="0.00402631" />
|
||||
<geom class="visual" quat="0 0 1 0" mesh="hip" />
|
||||
<geom name="RL_hip_collision" class="collision" size="0.05 0.01" pos="0 0.03 0" quat="0.707107 0.707107 0 0" />
|
||||
<body name="RL_thigh" pos="0 0.08 0">
|
||||
<inertial pos="-0.00304722 -0.019315 -0.0305004" quat="0.753383 0.0775126 -0.0272313 0.65243" mass="1.009" diaginertia="0.00478717 0.00460903 0.000709268" />
|
||||
<joint name="RL_thigh_joint" pos="0 0 0" axis="0 1 0" range="-0.686 4.501" armature="0.00402631" />
|
||||
<geom class="visual" mesh="thigh" />
|
||||
<geom name="RL_thigh_collision1" class="thigh1" size="0.015 0.08" pos="-0.02 0 -0.08" />
|
||||
<geom name="RL_thigh_collision2" class="thigh2" size="0.015 0.0509902" pos="-0.01 0 -0.05" quat="0.995133 0 0.0985376 0" />
|
||||
<geom name="RL_thigh_collision3" class="thigh3" size="0.015 0.0223607" pos="-0.01 0 -0.18" quat="0.973249 0 -0.229753 0" />
|
||||
<body name="RL_calf" pos="0 0 -0.213">
|
||||
<inertial pos="0.00429862 0.000976676 -0.146197" quat="0.691246 0.00357467 0.00511118 0.722592" mass="0.195862" diaginertia="0.00149767 0.00148468 3.58427e-05" />
|
||||
<joint name="RL_calf_joint" pos="0 0 0" axis="0 1 0" range="-2.818 -0.888" armature="0.0090592" />
|
||||
<geom class="visual" mesh="calf" />
|
||||
<geom name="RL_calf_collision1" class="calf1" size="0.01 0.0657647" pos="0.01 0 -0.065" quat="0.997089 0 -0.0762506 0" />
|
||||
<geom name="RL_calf_collision2" class="calf2" size="0.01 0.0364005" pos="0.01 0 -0.165" quat="0.990334 0 0.138701 0" />
|
||||
<geom name="RL_foot_collision" class="foot" pos="0 0 -0.213" />
|
||||
<site name="RL" pos="0 0 -0.213" size="0.023" group="5" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
<sensor>
|
||||
<gyro site="imu" name="imu_ang_vel" />
|
||||
<velocimeter site="imu" name="imu_lin_vel" />
|
||||
<accelerometer site="imu" name="imu_lin_acc" />
|
||||
<framezaxis objtype="body" objname="world" reftype="site" refname="imu" name="imu_upvector" />
|
||||
<subtreeangmom body="trunk" name="root_angmom" />
|
||||
</sensor>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"base_body": "trunk",
|
||||
"control_intent": "learned",
|
||||
"family": "quadruped",
|
||||
"foot_geoms": [
|
||||
"FR_foot_collision",
|
||||
"FL_foot_collision",
|
||||
"RR_foot_collision",
|
||||
"RL_foot_collision"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"action_scales": {
|
||||
"FL_calf_joint": 0.18693,
|
||||
"FL_hip_joint": 0.083974,
|
||||
"FL_thigh_joint": 0.087029,
|
||||
"FR_calf_joint": 0.18693,
|
||||
"FR_hip_joint": 0.084209,
|
||||
"FR_thigh_joint": 0.087029,
|
||||
"RL_calf_joint": 0.18693,
|
||||
"RL_hip_joint": 0.083974,
|
||||
"RL_thigh_joint": 0.087029,
|
||||
"RR_calf_joint": 0.18693,
|
||||
"RR_hip_joint": 0.084209,
|
||||
"RR_thigh_joint": 0.087029
|
||||
},
|
||||
"controls_path": "controls.yaml",
|
||||
"family": "quadruped",
|
||||
"model_path": "go1.xml",
|
||||
"penalized_contact_geoms": [
|
||||
"FR_thigh_collision1",
|
||||
"FR_thigh_collision2",
|
||||
"FR_thigh_collision3",
|
||||
"FR_calf_collision1",
|
||||
"FR_calf_collision2",
|
||||
"FL_thigh_collision1",
|
||||
"FL_thigh_collision2",
|
||||
"FL_thigh_collision3",
|
||||
"FL_calf_collision1",
|
||||
"FL_calf_collision2",
|
||||
"RR_thigh_collision1",
|
||||
"RR_thigh_collision2",
|
||||
"RR_thigh_collision3",
|
||||
"RR_calf_collision1",
|
||||
"RR_calf_collision2",
|
||||
"RL_thigh_collision1",
|
||||
"RL_thigh_collision2",
|
||||
"RL_thigh_collision3",
|
||||
"RL_calf_collision1",
|
||||
"RL_calf_collision2"
|
||||
],
|
||||
"robot_id": "go1_example",
|
||||
"termination_contact_geoms": [
|
||||
"trunk_collision",
|
||||
"head_collision"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"damping_ratio": 1.25,
|
||||
"effort_limits": {
|
||||
"FL_calf_joint": 35.55,
|
||||
"FL_hip_joint": 23.7,
|
||||
"FL_thigh_joint": 23.7,
|
||||
"FR_calf_joint": 35.55,
|
||||
"FR_hip_joint": 23.7,
|
||||
"FR_thigh_joint": 23.7,
|
||||
"RL_calf_joint": 35.55,
|
||||
"RL_hip_joint": 23.7,
|
||||
"RL_thigh_joint": 23.7,
|
||||
"RR_calf_joint": 35.55,
|
||||
"RR_hip_joint": 23.7,
|
||||
"RR_thigh_joint": 23.7
|
||||
},
|
||||
"foot_geoms": [
|
||||
"FR_foot_collision",
|
||||
"FL_foot_collision",
|
||||
"RR_foot_collision",
|
||||
"RL_foot_collision"
|
||||
],
|
||||
"frequency_hz": 14,
|
||||
"pose": {
|
||||
"FL_calf_joint": -1.8,
|
||||
"FL_hip_joint": -0.1,
|
||||
"FL_thigh_joint": 0.9,
|
||||
"FR_calf_joint": -1.8,
|
||||
"FR_hip_joint": 0.1,
|
||||
"FR_thigh_joint": 0.9,
|
||||
"RL_calf_joint": -1.8,
|
||||
"RL_hip_joint": -0.1,
|
||||
"RL_thigh_joint": 0.9,
|
||||
"RR_calf_joint": -1.8,
|
||||
"RR_hip_joint": 0.1,
|
||||
"RR_thigh_joint": 0.9
|
||||
},
|
||||
"root_body": "trunk",
|
||||
"seconds": 2
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
# hybrid_wheeled_smoke operator controls -- how a person's input becomes the command the
|
||||
# policy observes.
|
||||
#
|
||||
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
|
||||
# every control name from `controller/vocabulary.json`. Run
|
||||
# `python -m controller --vocabulary` to see the list; it is the same one the
|
||||
# deployed controller refuses a binding against.
|
||||
#
|
||||
# The pad and the keyboard are two paths, each bound straight to the command
|
||||
# axes -- a stick with a sign, a key with the direction it pushes -- and
|
||||
# neither names a control of the other. Either device drives alone; the one
|
||||
# touched last drives.
|
||||
#
|
||||
# Two rules this file inherits from the specification it replaced:
|
||||
# * Every value carries its source. A number without one cannot be told
|
||||
# apart from a guess.
|
||||
# * A consumer fails on a missing or malformed value rather than
|
||||
# substituting a default. A sign that defaults to +1 is a robot that walks
|
||||
# backwards while the screen looks perfectly normal.
|
||||
|
||||
schema_version: 2
|
||||
|
||||
command:
|
||||
- term: twist
|
||||
# The observation term these numbers are written into.
|
||||
feeds: velocity_commands
|
||||
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
|
||||
frame: body
|
||||
# Order is the order the command term writes them, and therefore the order
|
||||
# they appear in the observation.
|
||||
axes:
|
||||
- name: lin_vel_x
|
||||
unit: m/s
|
||||
positive: forward
|
||||
- name: lin_vel_y
|
||||
unit: m/s
|
||||
positive: left
|
||||
- name: ang_vel_z
|
||||
unit: rad/s
|
||||
positive: counter-clockwise
|
||||
scaling:
|
||||
# Full deflection is the edge of the range the policy was **trained** on,
|
||||
# not an absolute speed -- so a stick means "as fast as this policy was
|
||||
# ever asked to go" and cannot ask for something off-distribution.
|
||||
rule: full_deflection_is_range_edge
|
||||
formula: axis * (range_max if axis >= 0 else -range_min)
|
||||
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
|
||||
# what the policy trained on; taking the larger end for both quietly
|
||||
# commands something it never saw.
|
||||
ends_scaled_separately: true
|
||||
|
||||
devices:
|
||||
gamepad:
|
||||
# The stick position *is* the command.
|
||||
scheme: absolute
|
||||
layout: xbox
|
||||
axes:
|
||||
lin_vel_x: { source: Ly, sign: -1 }
|
||||
# The wheeled template scales this required pad channel to zero.
|
||||
lin_vel_y: { source: Lx, sign: -1 }
|
||||
ang_vel_z: { source: Rx, sign: -1 }
|
||||
# MEASURED: the signs are not measured against the robot's pad service.
|
||||
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
|
||||
# is **down**, ABS_X positive is **right** -- and the service passes both
|
||||
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
|
||||
# the robot is what would settle it.
|
||||
#
|
||||
# The service applies a 0.15 deadband and **rescales**, so full deflection
|
||||
# still reaches ±1. A consumer that applies its own on top narrows the
|
||||
# range a person can ask for and cannot tell that it has.
|
||||
deadzone: device_reported_rescaled
|
||||
# Hands the command back to the random sampler in `play --task`; on a robot, which
|
||||
# has no sampler, it lets go of everything.
|
||||
release_button: B
|
||||
|
||||
keyboard:
|
||||
# Keystrokes bound to what they do: each pushes one direction of one axis,
|
||||
# "+" being the axis's own positive direction as `positive:` above words
|
||||
# it, so the keyboard has no signs of its own -- there is no stick for a
|
||||
# sign to be relative to. A virtual pad read through the pad's mapping
|
||||
# until 2026-09-29.
|
||||
scheme: keys
|
||||
# Seconds a key is held for full deflection: the axis climbs linearly
|
||||
# until then and is back at rest the moment the key comes up, as a stick
|
||||
# let go of. Asked for on 2026-09-26; measured against nothing.
|
||||
full_after_s: 2.0
|
||||
axes:
|
||||
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
|
||||
lin_vel_y: {unbound: lateral command range is zero}
|
||||
ang_vel_z: { "+": [key_a, key_left], "-": [key_d, key_right] }
|
||||
# Lets go of everything, as the pad's release button does.
|
||||
release: [key_escape]
|
||||
@@ -0,0 +1,27 @@
|
||||
<mujoco model="hybrid_wheeled_smoke">
|
||||
<compiler angle="radian"/>
|
||||
<worldbody>
|
||||
<body name="base" pos="0 0 .27">
|
||||
<freejoint name="root"/>
|
||||
<geom name="trunk" type="box" size=".13 .09 .025" mass="1.5"/>
|
||||
<geom name="back_caster" type="sphere" pos="-.11 0 -.245" size=".025" mass=".04" friction=".03 .01 .001"/>
|
||||
<geom name="front_caster" type="sphere" pos=".11 0 -.245" size=".025" mass=".04" friction=".03 .01 .001"/>
|
||||
<body name="left_leg" pos="0 .13 0">
|
||||
<joint name="left_hip" axis="0 1 0" range="-.5 .5"/>
|
||||
<geom type="capsule" fromto="0 0 -.025 0 0 -.13" size=".018" mass=".12"/>
|
||||
<body name="left_wheel" pos="0 0 -.17">
|
||||
<joint name="left_wheel_joint" axis="0 1 0"/>
|
||||
<geom name="left_tire" type="cylinder" euler="1.570796326795 0 0" size=".1 .02" mass=".25"/>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_leg" pos="0 -.13 0">
|
||||
<joint name="right_hip" axis="0 1 0" range="-.5 .5"/>
|
||||
<geom type="capsule" fromto="0 0 -.025 0 0 -.13" size=".018" mass=".12"/>
|
||||
<body name="right_wheel" pos="0 0 -.17">
|
||||
<joint name="right_wheel_joint" axis="0 1 0"/>
|
||||
<geom name="right_tire" type="cylinder" euler="1.570796326795 0 0" size=".1 .02" mass=".25"/>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,21 @@
|
||||
# Authored example cube, metres; repository Apache-2.0 license.
|
||||
v -0.1 -0.1 -0.1
|
||||
v 0.1 -0.1 -0.1
|
||||
v 0.1 0.1 -0.1
|
||||
v -0.1 0.1 -0.1
|
||||
v -0.1 -0.1 0.1
|
||||
v 0.1 -0.1 0.1
|
||||
v 0.1 0.1 0.1
|
||||
v -0.1 0.1 0.1
|
||||
f 1 3 2
|
||||
f 1 4 3
|
||||
f 5 6 7
|
||||
f 5 7 8
|
||||
f 1 2 6
|
||||
f 1 6 5
|
||||
f 2 3 7
|
||||
f 2 7 6
|
||||
f 3 4 8
|
||||
f 3 8 7
|
||||
f 4 1 5
|
||||
f 4 5 8
|
||||
@@ -0,0 +1,28 @@
|
||||
# Six-legged locomotion example
|
||||
|
||||
This synthetic MuJoCo robot has six radial legs and 18 actuated hinges.
|
||||
The coxa yaw joint sweeps each leg horizontally. The femur and tibia pitch
|
||||
joints lift and extend its foot. The body and links use box, capsule and
|
||||
sphere collision geometry with explicitly specified masses and inertias.
|
||||
The model's front is +X and its total mass is 5.14 kg.
|
||||
|
||||
`profile.json` identifies the six named supports and the locomotion intent.
|
||||
`stance.json` supplies the nominal joint angles and trial PD parameters
|
||||
used for native stance calibration. `robot.json` supplies joint action
|
||||
scales and contact selections for the scaffold. The assets use the
|
||||
repository's Apache 2.0 license.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/multipod_example \
|
||||
--profile assets/multipod_example/profile.json
|
||||
python scripts/onboard.py calibrate --model assets/multipod_example/multipod.xml \
|
||||
--config assets/multipod_example/stance.json --report out/multipod-calibration.json
|
||||
python scripts/onboard.py scaffold-tier3 --config assets/multipod_example/robot.json \
|
||||
--calibration out/multipod-calibration.json --tasks-root out/multipod-scaffold
|
||||
python scripts/train.py --task multipod_example --backend native --device cpu \
|
||||
--num_envs 4 --max-iterations 1 --headless --no-tensorboard
|
||||
```
|
||||
|
||||
The checked-in robot specification is the calibrated task used by the
|
||||
training command. The scaffold example writes a separate copy for inspection.
|
||||
W/S command forward and reverse, A/D command left and right, and J/L command yaw.
|
||||
@@ -0,0 +1,91 @@
|
||||
# multipod_example operator controls -- how a person's input becomes the command the
|
||||
# policy observes.
|
||||
#
|
||||
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
|
||||
# every control name from `controller/vocabulary.json`. Run
|
||||
# `python -m controller --vocabulary` to see the list; it is the same one the
|
||||
# deployed controller refuses a binding against.
|
||||
#
|
||||
# The pad and the keyboard are two paths, each bound straight to the command
|
||||
# axes -- a stick with a sign, a key with the direction it pushes -- and
|
||||
# neither names a control of the other. Either device drives alone; the one
|
||||
# touched last drives.
|
||||
#
|
||||
# Two rules this file inherits from the specification it replaced:
|
||||
# * Every value carries its source. A number without one cannot be told
|
||||
# apart from a guess.
|
||||
# * A consumer fails on a missing or malformed value rather than
|
||||
# substituting a default. A sign that defaults to +1 is a robot that walks
|
||||
# backwards while the screen looks perfectly normal.
|
||||
|
||||
schema_version: 2
|
||||
|
||||
command:
|
||||
- term: twist
|
||||
# The observation term these numbers are written into.
|
||||
feeds: velocity_commands
|
||||
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
|
||||
frame: body
|
||||
# Order is the order the command term writes them, and therefore the order
|
||||
# they appear in the observation.
|
||||
axes:
|
||||
- name: lin_vel_x
|
||||
unit: m/s
|
||||
positive: forward
|
||||
- name: lin_vel_y
|
||||
unit: m/s
|
||||
positive: left
|
||||
- name: ang_vel_z
|
||||
unit: rad/s
|
||||
positive: counter-clockwise
|
||||
scaling:
|
||||
# Full deflection is the edge of the range the policy was **trained** on,
|
||||
# not an absolute speed -- so a stick means "as fast as this policy was
|
||||
# ever asked to go" and cannot ask for something off-distribution.
|
||||
rule: full_deflection_is_range_edge
|
||||
formula: axis * (range_max if axis >= 0 else -range_min)
|
||||
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
|
||||
# what the policy trained on; taking the larger end for both quietly
|
||||
# commands something it never saw.
|
||||
ends_scaled_separately: true
|
||||
|
||||
devices:
|
||||
gamepad:
|
||||
# The stick position *is* the command.
|
||||
scheme: absolute
|
||||
layout: xbox
|
||||
axes:
|
||||
lin_vel_x: { source: Ly, sign: -1 }
|
||||
lin_vel_y: { source: Lx, sign: -1 }
|
||||
ang_vel_z: { source: Rx, sign: -1 }
|
||||
# MEASURED: the signs are not measured against the robot's pad service.
|
||||
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
|
||||
# is **down**, ABS_X positive is **right** -- and the service passes both
|
||||
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
|
||||
# the robot is what would settle it.
|
||||
#
|
||||
# The service applies a 0.15 deadband and **rescales**, so full deflection
|
||||
# still reaches ±1. A consumer that applies its own on top narrows the
|
||||
# range a person can ask for and cannot tell that it has.
|
||||
deadzone: device_reported_rescaled
|
||||
# Hands the command back to the random sampler in `play --task`; on a robot, which
|
||||
# has no sampler, it lets go of everything.
|
||||
release_button: B
|
||||
|
||||
keyboard:
|
||||
# Keystrokes bound to what they do: each pushes one direction of one axis,
|
||||
# "+" being the axis's own positive direction as `positive:` above words
|
||||
# it, so the keyboard has no signs of its own -- there is no stick for a
|
||||
# sign to be relative to. A virtual pad read through the pad's mapping
|
||||
# until 2026-09-29.
|
||||
scheme: keys
|
||||
# Seconds a key is held for full deflection: the axis climbs linearly
|
||||
# until then and is back at rest the moment the key comes up, as a stick
|
||||
# let go of. Asked for on 2026-09-26; measured against nothing.
|
||||
full_after_s: 2.0
|
||||
axes:
|
||||
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
|
||||
lin_vel_y: { "+": [key_a, key_left], "-": [key_d, key_right] }
|
||||
ang_vel_z: { "+": [key_j], "-": [key_l] }
|
||||
# Lets go of everything, as the pad's release button does.
|
||||
release: [key_escape]
|
||||
@@ -0,0 +1,108 @@
|
||||
<mujoco model="multipod_example">
|
||||
<!-- Six radial legs with an outward coxa, pitch femur, and pitch tibia. -->
|
||||
<compiler angle="radian" inertiafromgeom="false" />
|
||||
<option timestep="0.0025" integrator="implicitfast" />
|
||||
<worldbody>
|
||||
<body name="base" pos="0 0 0.25">
|
||||
<freejoint name="base_free" />
|
||||
<inertial pos="0 0 0" mass="2.8" diaginertia="0.0114 0.0385 0.0476" />
|
||||
<geom name="base_collision" type="box" size="0.20 0.105 0.035" rgba="0.16 0.28 0.40 1" />
|
||||
<body name="left_front_coxa" pos="0.15 0.105 0" euler="0 0 0.7853981633974483">
|
||||
<joint name="left_front_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="left_front_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="left_front_femur" pos="0.085 0 0">
|
||||
<joint name="left_front_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="left_front_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="left_front_tibia" pos="0.12 0 0">
|
||||
<joint name="left_front_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="left_front_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="left_front_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="left_middle_coxa" pos="0 0.105 0" euler="0 0 1.5707963267948966">
|
||||
<joint name="left_middle_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="left_middle_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="left_middle_femur" pos="0.085 0 0">
|
||||
<joint name="left_middle_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="left_middle_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="left_middle_tibia" pos="0.12 0 0">
|
||||
<joint name="left_middle_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="left_middle_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="left_middle_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="left_rear_coxa" pos="-0.15 0.105 0" euler="0 0 2.356194490192345">
|
||||
<joint name="left_rear_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="left_rear_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="left_rear_femur" pos="0.085 0 0">
|
||||
<joint name="left_rear_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="left_rear_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="left_rear_tibia" pos="0.12 0 0">
|
||||
<joint name="left_rear_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="left_rear_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="left_rear_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_front_coxa" pos="0.15 -0.105 0" euler="0 0 -0.7853981633974483">
|
||||
<joint name="right_front_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="right_front_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="right_front_femur" pos="0.085 0 0">
|
||||
<joint name="right_front_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="right_front_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="right_front_tibia" pos="0.12 0 0">
|
||||
<joint name="right_front_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="right_front_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="right_front_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_middle_coxa" pos="0 -0.105 0" euler="0 0 -1.5707963267948966">
|
||||
<joint name="right_middle_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="right_middle_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="right_middle_femur" pos="0.085 0 0">
|
||||
<joint name="right_middle_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="right_middle_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="right_middle_tibia" pos="0.12 0 0">
|
||||
<joint name="right_middle_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="right_middle_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="right_middle_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_rear_coxa" pos="-0.15 -0.105 0" euler="0 0 -2.356194490192345">
|
||||
<joint name="right_rear_coxa_joint" axis="0 0 1" range="-0.75 0.75" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.0425 0 0" mass="0.07" diaginertia="0.000008 0.000050 0.000050" />
|
||||
<geom name="right_rear_coxa_collision" type="capsule" size="0.014" fromto="0.015 0 0 0.070 0 0" rgba="0.52 0.64 0.72 1" />
|
||||
<body name="right_rear_femur" pos="0.085 0 0">
|
||||
<joint name="right_rear_femur_joint" axis="0 1 0" range="-0.65 1.25" armature="0.006" damping="0.045" />
|
||||
<inertial pos="0.06 0 0" mass="0.17" diaginertia="0.000028 0.000225 0.000225" />
|
||||
<geom name="right_rear_femur_collision" type="capsule" size="0.017" fromto="0.022 0 0 0.098 0 0" rgba="0.32 0.55 0.70 1" />
|
||||
<body name="right_rear_tibia" pos="0.12 0 0">
|
||||
<joint name="right_rear_tibia_joint" axis="0 1 0" range="-0.30 1.90" armature="0.004" damping="0.035" />
|
||||
<inertial pos="0.095 0 0" mass="0.15" diaginertia="0.000030 0.000460 0.000460" />
|
||||
<geom name="right_rear_tibia_collision" type="capsule" size="0.014" fromto="0.020 0 0 0.153 0 0" rgba="0.22 0.39 0.54 1" />
|
||||
<geom name="right_rear_foot" type="sphere" pos="0.18 0 0" size="0.026" friction="0.95 0.015 0.003" rgba="0.90 0.62 0.20 1" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"base_body": "base",
|
||||
"control_intent": "wasd",
|
||||
"family": "multipod",
|
||||
"foot_geoms": [
|
||||
"left_front_foot",
|
||||
"left_middle_foot",
|
||||
"left_rear_foot",
|
||||
"right_front_foot",
|
||||
"right_middle_foot",
|
||||
"right_rear_foot"
|
||||
],
|
||||
"position_joints": [
|
||||
"left_front_coxa_joint",
|
||||
"left_front_femur_joint",
|
||||
"left_front_tibia_joint",
|
||||
"left_middle_coxa_joint",
|
||||
"left_middle_femur_joint",
|
||||
"left_middle_tibia_joint",
|
||||
"left_rear_coxa_joint",
|
||||
"left_rear_femur_joint",
|
||||
"left_rear_tibia_joint",
|
||||
"right_front_coxa_joint",
|
||||
"right_front_femur_joint",
|
||||
"right_front_tibia_joint",
|
||||
"right_middle_coxa_joint",
|
||||
"right_middle_femur_joint",
|
||||
"right_middle_tibia_joint",
|
||||
"right_rear_coxa_joint",
|
||||
"right_rear_femur_joint",
|
||||
"right_rear_tibia_joint"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"action_scales": {
|
||||
"left_front_coxa_joint": 0.26,
|
||||
"left_front_femur_joint": 0.32,
|
||||
"left_front_tibia_joint": 0.44,
|
||||
"left_middle_coxa_joint": 0.26,
|
||||
"left_middle_femur_joint": 0.32,
|
||||
"left_middle_tibia_joint": 0.44,
|
||||
"left_rear_coxa_joint": 0.26,
|
||||
"left_rear_femur_joint": 0.32,
|
||||
"left_rear_tibia_joint": 0.44,
|
||||
"right_front_coxa_joint": 0.26,
|
||||
"right_front_femur_joint": 0.32,
|
||||
"right_front_tibia_joint": 0.44,
|
||||
"right_middle_coxa_joint": 0.26,
|
||||
"right_middle_femur_joint": 0.32,
|
||||
"right_middle_tibia_joint": 0.44,
|
||||
"right_rear_coxa_joint": 0.26,
|
||||
"right_rear_femur_joint": 0.32,
|
||||
"right_rear_tibia_joint": 0.44
|
||||
},
|
||||
"controls_path": "controls.yaml",
|
||||
"family": "multipod",
|
||||
"model_path": "multipod.xml",
|
||||
"penalized_contact_geoms": [
|
||||
"left_front_coxa_collision",
|
||||
"left_front_femur_collision",
|
||||
"left_front_tibia_collision",
|
||||
"left_middle_coxa_collision",
|
||||
"left_middle_femur_collision",
|
||||
"left_middle_tibia_collision",
|
||||
"left_rear_coxa_collision",
|
||||
"left_rear_femur_collision",
|
||||
"left_rear_tibia_collision",
|
||||
"right_front_coxa_collision",
|
||||
"right_front_femur_collision",
|
||||
"right_front_tibia_collision",
|
||||
"right_middle_coxa_collision",
|
||||
"right_middle_femur_collision",
|
||||
"right_middle_tibia_collision",
|
||||
"right_rear_coxa_collision",
|
||||
"right_rear_femur_collision",
|
||||
"right_rear_tibia_collision"
|
||||
],
|
||||
"robot_id": "multipod_example",
|
||||
"termination_contact_geoms": [
|
||||
"base_collision"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"effort_limits": {
|
||||
"left_front_coxa_joint": 6,
|
||||
"left_front_femur_joint": 9,
|
||||
"left_front_tibia_joint": 9,
|
||||
"left_middle_coxa_joint": 6,
|
||||
"left_middle_femur_joint": 9,
|
||||
"left_middle_tibia_joint": 9,
|
||||
"left_rear_coxa_joint": 6,
|
||||
"left_rear_femur_joint": 9,
|
||||
"left_rear_tibia_joint": 9,
|
||||
"right_front_coxa_joint": 6,
|
||||
"right_front_femur_joint": 9,
|
||||
"right_front_tibia_joint": 9,
|
||||
"right_middle_coxa_joint": 6,
|
||||
"right_middle_femur_joint": 9,
|
||||
"right_middle_tibia_joint": 9,
|
||||
"right_rear_coxa_joint": 6,
|
||||
"right_rear_femur_joint": 9,
|
||||
"right_rear_tibia_joint": 9
|
||||
},
|
||||
"foot_geoms": [
|
||||
"left_front_foot",
|
||||
"left_middle_foot",
|
||||
"left_rear_foot",
|
||||
"right_front_foot",
|
||||
"right_middle_foot",
|
||||
"right_rear_foot"
|
||||
],
|
||||
"height_joints": {
|
||||
"left_front_foot": "left_front_tibia_joint",
|
||||
"left_middle_foot": "left_middle_tibia_joint",
|
||||
"left_rear_foot": "left_rear_tibia_joint",
|
||||
"right_front_foot": "right_front_tibia_joint",
|
||||
"right_middle_foot": "right_middle_tibia_joint",
|
||||
"right_rear_foot": "right_rear_tibia_joint"
|
||||
},
|
||||
"kd": {
|
||||
"left_front_coxa_joint": 0.42,
|
||||
"left_front_femur_joint": 0.55,
|
||||
"left_front_tibia_joint": 0.65,
|
||||
"left_middle_coxa_joint": 0.42,
|
||||
"left_middle_femur_joint": 0.55,
|
||||
"left_middle_tibia_joint": 0.65,
|
||||
"left_rear_coxa_joint": 0.42,
|
||||
"left_rear_femur_joint": 0.55,
|
||||
"left_rear_tibia_joint": 0.65,
|
||||
"right_front_coxa_joint": 0.42,
|
||||
"right_front_femur_joint": 0.55,
|
||||
"right_front_tibia_joint": 0.65,
|
||||
"right_middle_coxa_joint": 0.42,
|
||||
"right_middle_femur_joint": 0.55,
|
||||
"right_middle_tibia_joint": 0.65,
|
||||
"right_rear_coxa_joint": 0.42,
|
||||
"right_rear_femur_joint": 0.55,
|
||||
"right_rear_tibia_joint": 0.65
|
||||
},
|
||||
"kp": {
|
||||
"left_front_coxa_joint": 11,
|
||||
"left_front_femur_joint": 17,
|
||||
"left_front_tibia_joint": 19,
|
||||
"left_middle_coxa_joint": 11,
|
||||
"left_middle_femur_joint": 17,
|
||||
"left_middle_tibia_joint": 19,
|
||||
"left_rear_coxa_joint": 11,
|
||||
"left_rear_femur_joint": 17,
|
||||
"left_rear_tibia_joint": 19,
|
||||
"right_front_coxa_joint": 11,
|
||||
"right_front_femur_joint": 17,
|
||||
"right_front_tibia_joint": 19,
|
||||
"right_middle_coxa_joint": 11,
|
||||
"right_middle_femur_joint": 17,
|
||||
"right_middle_tibia_joint": 19,
|
||||
"right_rear_coxa_joint": 11,
|
||||
"right_rear_femur_joint": 17,
|
||||
"right_rear_tibia_joint": 19
|
||||
},
|
||||
"max_adjust": 0.1,
|
||||
"pose": {
|
||||
"left_front_coxa_joint": 0.0,
|
||||
"left_front_femur_joint": 0.42,
|
||||
"left_front_tibia_joint": 0.95,
|
||||
"left_middle_coxa_joint": 0.0,
|
||||
"left_middle_femur_joint": 0.42,
|
||||
"left_middle_tibia_joint": 0.95,
|
||||
"left_rear_coxa_joint": 0.0,
|
||||
"left_rear_femur_joint": 0.42,
|
||||
"left_rear_tibia_joint": 0.95,
|
||||
"right_front_coxa_joint": 0.0,
|
||||
"right_front_femur_joint": 0.42,
|
||||
"right_front_tibia_joint": 0.95,
|
||||
"right_middle_coxa_joint": 0.0,
|
||||
"right_middle_femur_joint": 0.42,
|
||||
"right_middle_tibia_joint": 0.95,
|
||||
"right_rear_coxa_joint": 0.0,
|
||||
"right_rear_femur_joint": 0.42,
|
||||
"right_rear_tibia_joint": 0.95
|
||||
},
|
||||
"root_body": "base",
|
||||
"seconds": 2
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"kind": "ik",
|
||||
"joints": ["shoulder", "elbow"],
|
||||
"end_effector_body": "tool_tip",
|
||||
"kp": 20,
|
||||
"kd": 0.75,
|
||||
"effort_limit": 10
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"kind": "joint_sliders",
|
||||
"joints": ["shoulder", "elbow"],
|
||||
"kp": 20,
|
||||
"kd": 0.75,
|
||||
"effort_limit": 10
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
<?xml version="1.0"?>
|
||||
<robot name="planar_arm">
|
||||
<material name="base_color"><color rgba="0.20 0.27 0.34 1"/></material>
|
||||
<material name="link_color"><color rgba="0.25 0.63 0.78 1"/></material>
|
||||
<material name="tip_color"><color rgba="0.91 0.68 0.32 1"/></material>
|
||||
<link name="base">
|
||||
<inertial>
|
||||
<origin xyz="0 0 0.04"/>
|
||||
<mass value="1.8"/>
|
||||
<inertia ixx="0.004" iyy="0.004" izz="0.006" ixy="0" ixz="0" iyz="0"/>
|
||||
</inertial>
|
||||
<visual><origin xyz="0 0 0.04"/><geometry><box size="0.14 0.14 0.08"/></geometry>
|
||||
<material name="base_color"/></visual>
|
||||
<collision><origin xyz="0 0 0.04"/><geometry><box size="0.14 0.14 0.08"/></geometry></collision>
|
||||
</link>
|
||||
<link name="upper_link">
|
||||
<inertial>
|
||||
<origin xyz="0.11 0 0"/>
|
||||
<mass value="0.4"/>
|
||||
<inertia ixx="0.0005" iyy="0.002" izz="0.002" ixy="0" ixz="0" iyz="0"/>
|
||||
</inertial>
|
||||
<visual><origin xyz="0.11 0 0"/><geometry><box size="0.20 0.038 0.04"/></geometry>
|
||||
<material name="link_color"/></visual>
|
||||
<collision><origin xyz="0.11 0 0"/><geometry><box size="0.20 0.038 0.04"/></geometry></collision>
|
||||
</link>
|
||||
<link name="lower_link">
|
||||
<inertial>
|
||||
<origin xyz="0.09 0 0"/>
|
||||
<mass value="0.25"/>
|
||||
<inertia ixx="0.0003" iyy="0.001" izz="0.001" ixy="0" ixz="0" iyz="0"/>
|
||||
</inertial>
|
||||
<visual><origin xyz="0.09 0 0"/><geometry><box size="0.16 0.03 0.035"/></geometry>
|
||||
<material name="link_color"/></visual>
|
||||
<collision><origin xyz="0.09 0 0"/><geometry><box size="0.16 0.03 0.035"/></geometry></collision>
|
||||
</link>
|
||||
<link name="tool_tip">
|
||||
<inertial>
|
||||
<mass value="0.025"/>
|
||||
<inertia ixx="0.00001" iyy="0.00001" izz="0.00001" ixy="0" ixz="0" iyz="0"/>
|
||||
</inertial>
|
||||
<visual><geometry><sphere radius="0.017"/></geometry><material name="tip_color"/></visual>
|
||||
<collision><geometry><sphere radius="0.017"/></geometry></collision>
|
||||
</link>
|
||||
<joint name="shoulder" type="revolute">
|
||||
<parent link="base"/><child link="upper_link"/><origin xyz="0 0 0.16"/>
|
||||
<axis xyz="0 0 1"/><limit lower="-1.5" upper="1.5" effort="10" velocity="4"/>
|
||||
<dynamics damping="0.2"/>
|
||||
</joint>
|
||||
<joint name="elbow" type="revolute">
|
||||
<parent link="upper_link"/><child link="lower_link"/><origin xyz="0.22 0 0"/>
|
||||
<axis xyz="0 0 1"/><limit lower="-2" upper="2" effort="10" velocity="4"/>
|
||||
<dynamics damping="0.2"/>
|
||||
</joint>
|
||||
<joint name="tip_mount" type="fixed">
|
||||
<parent link="lower_link"/><child link="tool_tip"/><origin xyz="0.18 0 0"/>
|
||||
</joint>
|
||||
</robot>
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"kind": "trajectory",
|
||||
"joints": ["shoulder", "elbow"],
|
||||
"kp": 20,
|
||||
"kd": 0.75,
|
||||
"effort_limit": 10,
|
||||
"trajectory": {
|
||||
"schema": "mjrl-trajectory/1",
|
||||
"joint_order": ["elbow", "shoulder"],
|
||||
"time_s": [0, 0.4, 0.8, 1.2],
|
||||
"q": [[0, 0], [-0.45, 0.25], [0.35, -0.2], [-0.4, 0.2]]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"root_body": "base",
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"steps": 750,
|
||||
"thresholds": {"site_atol_m": 0.003},
|
||||
"probes": [{
|
||||
"command": {"target_x": -0.08, "target_y": 0.08},
|
||||
"site_positions": {"mjrl_ee": [0.32, 0.08, 0.16]}
|
||||
}]
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"root_body": "base",
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"steps": 750,
|
||||
"thresholds": {"position_atol": 0.008},
|
||||
"probes": [{
|
||||
"command": {"shoulder": 0.25, "elbow": -0.4},
|
||||
"actuator_controls": {"mjrl_shoulder": 0.25, "mjrl_elbow": -0.4},
|
||||
"joint_positions": {"shoulder": 0.25, "elbow": -0.4}
|
||||
}]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"root_body": "base",
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"steps": 1000,
|
||||
"thresholds": {"position_atol": 0.008},
|
||||
"probes": [{
|
||||
"joint_positions": {"shoulder": 0.2, "elbow": -0.4}
|
||||
}]
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
<!--
|
||||
A minimal model for the framework's own smoke tests, unrelated to any real
|
||||
robot.
|
||||
|
||||
The design goal is not to resemble anything, but to cover every path the
|
||||
framework has to validate at the smallest possible scale:
|
||||
· freejoint -- reading and writing a floating base's root state
|
||||
· contact -- feet against the ground, exercising the contact solver
|
||||
· position actuators -- the same PD control form as a real legged robot
|
||||
· sensor -- framequat / gyro / velocimeter, through the batched
|
||||
sensordata path
|
||||
· left-right symmetry -- convenient for validating indexing and mirroring
|
||||
|
||||
4 actuated degrees of freedom plus 6 for the base, with primitive geometry
|
||||
throughout (no meshes), so it simulates extremely fast -- suitable as the
|
||||
control group for backend-consistency regressions and throughput benchmarks.
|
||||
-->
|
||||
<mujoco model="smoke_biped">
|
||||
<compiler angle="radian" autolimits="true"/>
|
||||
<option timestep="0.002" integrator="implicitfast"/>
|
||||
|
||||
<default>
|
||||
<default class="robot">
|
||||
<joint damping="0.05" armature="0.01"/>
|
||||
<default class="motor">
|
||||
<position kp="60" kv="2.0" forcerange="-20 20"/>
|
||||
</default>
|
||||
<default class="link">
|
||||
<geom type="capsule" size="0.02" density="800" rgba="0.55 0.6 0.68 1"/>
|
||||
</default>
|
||||
<default class="foot">
|
||||
<geom type="sphere" size="0.028" density="800" rgba="0.85 0.6 0.25 1"
|
||||
friction="1.0 0.01 0.01" priority="1" condim="3"/>
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
|
||||
<worldbody>
|
||||
<light pos="0 0 2.5" dir="0 0 -1" directional="true"/>
|
||||
<geom name="floor" type="plane" size="0 0 0.05" rgba="0.3 0.33 0.36 1"
|
||||
friction="1.0 0.01 0.01" condim="3"/>
|
||||
|
||||
<body name="base" pos="0 0 0.42" childclass="robot">
|
||||
<freejoint name="root"/>
|
||||
<inertial pos="0 0 0" mass="2.0" diaginertia="0.012 0.016 0.010"/>
|
||||
<geom name="torso" type="box" size="0.06 0.045 0.035" rgba="0.42 0.47 0.55 1"/>
|
||||
<site name="imu" pos="0 0 0" size="0.008"/>
|
||||
|
||||
<body name="L_thigh" pos="0 0.05 -0.035">
|
||||
<joint name="L_hip" axis="0 1 0" range="-1.4 1.4"/>
|
||||
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
|
||||
<body name="L_calf" pos="0 0 -0.16">
|
||||
<joint name="L_knee" axis="0 1 0" range="-2.2 0"/>
|
||||
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
|
||||
<body name="L_foot" pos="0 0 -0.16">
|
||||
<geom name="L_foot_geom" class="foot"/>
|
||||
<site name="L_foot_site" size="0.008"/>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
|
||||
<body name="R_thigh" pos="0 -0.05 -0.035">
|
||||
<joint name="R_hip" axis="0 1 0" range="-1.4 1.4"/>
|
||||
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
|
||||
<body name="R_calf" pos="0 0 -0.16">
|
||||
<joint name="R_knee" axis="0 1 0" range="-2.2 0"/>
|
||||
<geom class="link" fromto="0 0 0 0 0 -0.16"/>
|
||||
<body name="R_foot" pos="0 0 -0.16">
|
||||
<geom name="R_foot_geom" class="foot"/>
|
||||
<site name="R_foot_site" size="0.008"/>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
|
||||
<actuator>
|
||||
<position class="motor" name="L_hip_ctrl" joint="L_hip" ctrlrange="-1.4 1.4"/>
|
||||
<position class="motor" name="L_knee_ctrl" joint="L_knee" ctrlrange="-2.2 0"/>
|
||||
<position class="motor" name="R_hip_ctrl" joint="R_hip" ctrlrange="-1.4 1.4"/>
|
||||
<position class="motor" name="R_knee_ctrl" joint="R_knee" ctrlrange="-2.2 0"/>
|
||||
</actuator>
|
||||
|
||||
<sensor>
|
||||
<framequat name="base_quat" objtype="site" objname="imu"/>
|
||||
<gyro name="base_gyro" site="imu"/>
|
||||
<velocimeter name="base_vel" site="imu"/>
|
||||
<framepos name="L_foot_pos" objtype="site" objname="L_foot_site"/>
|
||||
<framepos name="R_foot_pos" objtype="site" objname="R_foot_site"/>
|
||||
</sensor>
|
||||
|
||||
<keyframe>
|
||||
<!--
|
||||
Standing pose: hip +0.3 / knee -0.6 (the knee's limit is [-2.2, 0], so it
|
||||
only bends negative).
|
||||
Vertically: 0.035 (half the base height) + 0.16*cos(0.3)*2 + 0.028 (foot
|
||||
radius) ~= 0.369. The hip's forward swing 0.16*sin(0.3) and the knee's
|
||||
return 0.16*sin(-0.3) cancel, so the feet land directly under the base.
|
||||
-->
|
||||
<key name="home"
|
||||
qpos="0 0 0.369 1 0 0 0 0.3 -0.6 0.3 -0.6"
|
||||
ctrl="0.3 -0.6 0.3 -0.6"/>
|
||||
</keyframe>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,67 @@
|
||||
# Wheeled biped example
|
||||
|
||||
This synthetic MuJoCo robot has two articulated legs, each with a pitch
|
||||
hip, pitch knee and continuous wheel hinge. The two tires provide the
|
||||
ground support. The battery case hangs below the upper frame and lowers the
|
||||
center of mass. At the calibrated neutral pose the 7.62 kg robot has a
|
||||
0.421 m root height, a 0.19 m wheel radius and a 0.35 m track.
|
||||
|
||||
The locked wheeled-biped template applies position PD to the four leg joints
|
||||
and velocity feedback to the two wheels. The model declares 12 N·m hip,
|
||||
16 N·m knee and 5 N·m wheel effort limits. Forward, reverse and yaw controls
|
||||
use the shared keyboard and gamepad vocabulary. The template's lateral
|
||||
command range is zero.
|
||||
|
||||
These commands reproduce the intake and stance checks. The last command
|
||||
writes a review copy of the generated task specification.
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/wheeled_biped_example \
|
||||
--profile assets/wheeled_biped_example/profile.json
|
||||
python scripts/onboard.py calibrate --model assets/wheeled_biped_example/robot.xml \
|
||||
--config assets/wheeled_biped_example/stance.json --report out/wheeled_calibration.json
|
||||
python scripts/onboard.py scaffold-tier3 --config assets/wheeled_biped_example/robot.json \
|
||||
--calibration out/wheeled_calibration.json --tasks-root out/wheeled_specs
|
||||
```
|
||||
|
||||
In the two-second native stance check, both tires maintained support over
|
||||
the final measurement window, carrying 74.75 N of total normal load. The
|
||||
base settled 2.12 mm with no measured rotation, self-contact or engine warning.
|
||||
Calibration uses a bounded position hold for the wheels. Training and
|
||||
replay drive wheel velocity. In a separate hold using zero wheel velocity
|
||||
targets, an initial pitch of ±0.03 rad settled to within 0.00066 rad in two
|
||||
seconds. The peak effort per hip, knee and wheel was 0.291, 0.171 and
|
||||
0.076 N·m respectively. Only the tires touched the ground.
|
||||
|
||||
Copy `verify_policy.json` beside the native export's `controller.json` and
|
||||
`controls.yaml`. Its forward, reverse, left yaw, right yaw and idle probes
|
||||
evaluate tracking after a one-second warmup. A policy export needs to pass those
|
||||
physical response checks before it can be packaged as a controlled app.
|
||||
|
||||
```bash
|
||||
python scripts/train.py --task wheeled_biped_example --backend native --device cpu \
|
||||
--num-envs 4 --max-iterations 1 --headless --no-tensorboard
|
||||
python scripts/export.py --task wheeled_biped_example --backend native --device cpu \
|
||||
--checkpoint <checkpoint.pt> --native-layout --out out/wheeled_policy
|
||||
cp assets/wheeled_biped_example/verify_policy.json out/wheeled_policy/verify-input.json
|
||||
python scripts/onboard.py verify --model out/wheeled_policy/robot.xml \
|
||||
--config out/wheeled_policy/verify-input.json --report out/wheeled_policy/verification.json
|
||||
```
|
||||
|
||||
One training iteration exercises the pipeline. Use a trained checkpoint
|
||||
for the physical tracking probes.
|
||||
|
||||
A Warp run on an RTX 5080 used 1,024 environments and saved the selected
|
||||
checkpoint after 201 updates. With observation noise off, six-second Warp
|
||||
replays averaged +0.146 and −0.184 m/s for forward and reverse commands of
|
||||
±0.18 m/s, and +0.308 and −0.347 rad/s for yaw commands of ±0.35 rad/s. All
|
||||
five replays, including idle, ran without resets. The exported actor passed
|
||||
the matching native verification probes.
|
||||
|
||||
The selected checkpoint exposes ±0.91 m/s forward and reverse and
|
||||
±0.77 rad/s yaw. At full keyboard deflection, the native app averaged
|
||||
+0.866 and −0.927 m/s and +0.769 and −0.773 rad/s. The 0.19 m tires and
|
||||
5 rad/s wheel target limit give a nominal rolling speed of 0.95 m/s.
|
||||
The final checkpoint after 1,000 updates averaged +0.899 m/s on a +1.30 m/s
|
||||
command. The published app uses the earlier checkpoint and its measured
|
||||
command range.
|
||||
@@ -0,0 +1,92 @@
|
||||
# wheeled_biped_example operator controls -- how a person's input becomes the command the
|
||||
# policy observes.
|
||||
#
|
||||
# Generated by `.claude/skills/controls/scripts/write_controls.py`, which takes
|
||||
# every control name from `controller/vocabulary.json`. Run
|
||||
# `python -m controller --vocabulary` to see the list; it is the same one the
|
||||
# deployed controller refuses a binding against.
|
||||
#
|
||||
# The pad and the keyboard are two paths, each bound straight to the command
|
||||
# axes -- a stick with a sign, a key with the direction it pushes -- and
|
||||
# neither names a control of the other. Either device drives alone; the one
|
||||
# touched last drives.
|
||||
#
|
||||
# Two rules this file inherits from the specification it replaced:
|
||||
# * Every value carries its source. A number without one cannot be told
|
||||
# apart from a guess.
|
||||
# * A consumer fails on a missing or malformed value rather than
|
||||
# substituting a default. A sign that defaults to +1 is a robot that walks
|
||||
# backwards while the screen looks perfectly normal.
|
||||
|
||||
schema_version: 2
|
||||
|
||||
command:
|
||||
- term: twist
|
||||
# The observation term these numbers are written into.
|
||||
feeds: velocity_commands
|
||||
# The robot's own frame: +X forward, +Y left, +Z up (ROS REP-103).
|
||||
frame: body
|
||||
# Order is the order the command term writes them, and therefore the order
|
||||
# they appear in the observation.
|
||||
axes:
|
||||
- name: lin_vel_x
|
||||
unit: m/s
|
||||
positive: forward
|
||||
- name: lin_vel_y
|
||||
unit: m/s
|
||||
positive: left
|
||||
- name: ang_vel_z
|
||||
unit: rad/s
|
||||
positive: counter-clockwise
|
||||
scaling:
|
||||
# Full deflection is the edge of the range the policy was **trained** on,
|
||||
# not an absolute speed -- so a stick means "as fast as this policy was
|
||||
# ever asked to go" and cannot ask for something off-distribution.
|
||||
rule: full_deflection_is_range_edge
|
||||
formula: axis * (range_max if axis >= 0 else -range_min)
|
||||
# A range like (-0.5, 1.0) gives forward twice what it gives back, which is
|
||||
# what the policy trained on; taking the larger end for both quietly
|
||||
# commands something it never saw.
|
||||
ends_scaled_separately: true
|
||||
|
||||
devices:
|
||||
gamepad:
|
||||
# The stick position *is* the command.
|
||||
scheme: absolute
|
||||
layout: xbox
|
||||
axes:
|
||||
lin_vel_x: { source: Ly, sign: -1 }
|
||||
# The wheeled template scales this required pad channel to zero.
|
||||
lin_vel_y: { source: Lx, sign: -1 }
|
||||
ang_vel_z: { source: Rx, sign: -1 }
|
||||
# MEASURED: the signs are not measured against the robot's pad service.
|
||||
# `controller/xbox.py` states the same evdev convention -- ABS_Y positive
|
||||
# is **down**, ABS_X positive is **right** -- and the service passes both
|
||||
# through unchanged, negating `dpad_y` and not `Ly`. Pushing the stick on
|
||||
# the robot is what would settle it.
|
||||
#
|
||||
# The service applies a 0.15 deadband and **rescales**, so full deflection
|
||||
# still reaches ±1. A consumer that applies its own on top narrows the
|
||||
# range a person can ask for and cannot tell that it has.
|
||||
deadzone: device_reported_rescaled
|
||||
# Hands the command back to the random sampler in `play --task`; on a robot, which
|
||||
# has no sampler, it lets go of everything.
|
||||
release_button: B
|
||||
|
||||
keyboard:
|
||||
# Keystrokes bound to what they do: each pushes one direction of one axis,
|
||||
# "+" being the axis's own positive direction as `positive:` above words
|
||||
# it, so the keyboard has no signs of its own -- there is no stick for a
|
||||
# sign to be relative to. A virtual pad read through the pad's mapping
|
||||
# until 2026-09-29.
|
||||
scheme: keys
|
||||
# Seconds a key is held for full deflection: the axis climbs linearly
|
||||
# until then and is back at rest the moment the key comes up, as a stick
|
||||
# let go of. Asked for on 2026-09-26; measured against nothing.
|
||||
full_after_s: 2.0
|
||||
axes:
|
||||
lin_vel_x: { "+": [key_w, key_up], "-": [key_s, key_down] }
|
||||
lin_vel_y: {unbound: lateral command range is zero}
|
||||
ang_vel_z: { "+": [key_a, key_left], "-": [key_d, key_right] }
|
||||
# Lets go of everything, as the pad's release button does.
|
||||
release: [key_escape]
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"family": "wheeled_biped",
|
||||
"control_intent": "wasd",
|
||||
"base_body": "base",
|
||||
"foot_geoms": ["left_tire", "right_tire"],
|
||||
"position_joints": ["left_hip", "left_knee", "right_hip", "right_knee"],
|
||||
"velocity_joints": ["left_wheel_joint", "right_wheel_joint"]
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"robot_id": "wheeled_biped_example",
|
||||
"family": "wheeled_biped",
|
||||
"model_path": "robot.xml",
|
||||
"controls_path": "controls.yaml",
|
||||
"action_scales": {
|
||||
"left_hip": 0.09,
|
||||
"left_knee": 0.10,
|
||||
"right_hip": 0.09,
|
||||
"right_knee": 0.10
|
||||
},
|
||||
"velocity_joints": [
|
||||
{"name": "left_wheel_joint", "kp": 0, "kd": 1.1, "effort_limit": 5, "action_scale": 5, "armature": 0.01},
|
||||
{"name": "right_wheel_joint", "kp": 0, "kd": 1.1, "effort_limit": 5, "action_scale": 5, "armature": 0.01}
|
||||
],
|
||||
"penalized_contact_geoms": [
|
||||
"left_thigh_collision", "left_shank_collision",
|
||||
"right_thigh_collision", "right_shank_collision"
|
||||
],
|
||||
"termination_contact_geoms": ["upper_housing", "spine", "battery_case"]
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
<mujoco model="wheeled_biped_example">
|
||||
<compiler angle="radian"/>
|
||||
<option timestep="0.0025" integrator="implicitfast"/>
|
||||
<default>
|
||||
<joint damping="0.04" armature="0.012"/>
|
||||
<geom friction="0.9 0.01 0.0004"/>
|
||||
</default>
|
||||
<worldbody>
|
||||
<body name="base" pos="0 0 0.421">
|
||||
<freejoint name="root"/>
|
||||
<geom name="upper_housing" type="box" size="0.085 0.125 0.026"
|
||||
mass="0.65" rgba="0.19 0.36 0.48 1"/>
|
||||
<geom name="spine" type="box" pos="0 0 -0.145" size="0.027 0.040 0.125"
|
||||
mass="0.35" rgba="0.25 0.49 0.63 1"/>
|
||||
<geom name="battery_case" type="box" pos="0 0 -0.349"
|
||||
size="0.140 0.090 0.033" mass="5.2" rgba="0.24 0.29 0.35 1"/>
|
||||
<body name="left_thigh" pos="0 0.125 0">
|
||||
<joint name="left_hip" axis="0 1 0" range="-0.7 0.7" actuatorfrcrange="-12 12"/>
|
||||
<geom name="left_thigh_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
|
||||
size="0.016" mass="0.24" rgba="0.89 0.47 0.19 1"/>
|
||||
<body name="left_shank" pos="0 0 -0.115">
|
||||
<joint name="left_knee" axis="0 1 0" range="-0.55 0.8" actuatorfrcrange="-16 16"/>
|
||||
<geom name="left_shank_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
|
||||
size="0.015" mass="0.21" rgba="0.89 0.62 0.25 1"/>
|
||||
<body name="left_wheel" pos="0 0.050 -0.115">
|
||||
<joint name="left_wheel_joint" axis="0 1 0" limited="false"
|
||||
damping="0.015" armature="0.010" actuatorfrcrange="-5 5"/>
|
||||
<geom name="left_tire" type="cylinder" euler="1.5707963267948966 0 0"
|
||||
size="0.190 0.024" mass="0.24" rgba="0.12 0.13 0.16 1"/>
|
||||
<geom name="left_hub" type="box" pos="0 0.024 0" size="0.071 0.004 0.012"
|
||||
mass="0.02" contype="0" conaffinity="0" rgba="0.68 0.70 0.75 1"/>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
<body name="right_thigh" pos="0 -0.125 0">
|
||||
<joint name="right_hip" axis="0 1 0" range="-0.7 0.7" actuatorfrcrange="-12 12"/>
|
||||
<geom name="right_thigh_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
|
||||
size="0.016" mass="0.24" rgba="0.89 0.47 0.19 1"/>
|
||||
<body name="right_shank" pos="0 0 -0.115">
|
||||
<joint name="right_knee" axis="0 1 0" range="-0.55 0.8" actuatorfrcrange="-16 16"/>
|
||||
<geom name="right_shank_collision" type="capsule" fromto="0 0 0 0 0 -0.115"
|
||||
size="0.015" mass="0.21" rgba="0.89 0.62 0.25 1"/>
|
||||
<body name="right_wheel" pos="0 -0.050 -0.115">
|
||||
<joint name="right_wheel_joint" axis="0 1 0" limited="false"
|
||||
damping="0.015" armature="0.010" actuatorfrcrange="-5 5"/>
|
||||
<geom name="right_tire" type="cylinder" euler="1.5707963267948966 0 0"
|
||||
size="0.190 0.024" mass="0.24" rgba="0.12 0.13 0.16 1"/>
|
||||
<geom name="right_hub" type="box" pos="0 -0.024 0" size="0.071 0.004 0.012"
|
||||
mass="0.02" contype="0" conaffinity="0" rgba="0.68 0.70 0.75 1"/>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
</mujoco>
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"root_body": "base",
|
||||
"foot_geoms": ["left_tire", "right_tire"],
|
||||
"pose": {
|
||||
"left_hip": 0,
|
||||
"left_knee": 0,
|
||||
"left_wheel_joint": 0,
|
||||
"right_hip": 0,
|
||||
"right_knee": 0,
|
||||
"right_wheel_joint": 0
|
||||
},
|
||||
"kp": {
|
||||
"left_hip": 130,
|
||||
"left_knee": 150,
|
||||
"left_wheel_joint": 1,
|
||||
"right_hip": 130,
|
||||
"right_knee": 150,
|
||||
"right_wheel_joint": 1
|
||||
},
|
||||
"kd": {
|
||||
"left_hip": 3.4,
|
||||
"left_knee": 3.7,
|
||||
"left_wheel_joint": 1.1,
|
||||
"right_hip": 3.4,
|
||||
"right_knee": 3.7,
|
||||
"right_wheel_joint": 1.1
|
||||
},
|
||||
"effort_limits": {
|
||||
"left_hip": 12,
|
||||
"left_knee": 16,
|
||||
"left_wheel_joint": 5,
|
||||
"right_hip": 12,
|
||||
"right_knee": 16,
|
||||
"right_wheel_joint": 5
|
||||
},
|
||||
"seconds": 2
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
{
|
||||
"root_body": "robot/base",
|
||||
"controller": "controller.json",
|
||||
"controls": "controls.yaml",
|
||||
"ground_geom": "terrain",
|
||||
"foot_geoms": [
|
||||
"robot/left_tire",
|
||||
"robot/right_tire"
|
||||
],
|
||||
"steps": 2400,
|
||||
"probes": [
|
||||
{
|
||||
"command": {
|
||||
"lin_vel_x": 0.18,
|
||||
"lin_vel_y": 0,
|
||||
"ang_vel_z": 0
|
||||
},
|
||||
"evaluation_start_s": 1,
|
||||
"expect": {
|
||||
"base_velocity_mean_m_s": {
|
||||
"target": [0.18, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_velocity_tracking_mae_m_s": {
|
||||
"target": [0.18, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_angular_velocity_mean_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
},
|
||||
"base_angular_velocity_tracking_mae_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"command": {
|
||||
"lin_vel_x": -0.18,
|
||||
"lin_vel_y": 0,
|
||||
"ang_vel_z": 0
|
||||
},
|
||||
"evaluation_start_s": 1,
|
||||
"expect": {
|
||||
"base_velocity_mean_m_s": {
|
||||
"target": [-0.18, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_velocity_tracking_mae_m_s": {
|
||||
"target": [-0.18, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_angular_velocity_mean_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
},
|
||||
"base_angular_velocity_tracking_mae_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"command": {
|
||||
"lin_vel_x": 0,
|
||||
"lin_vel_y": 0,
|
||||
"ang_vel_z": 0.35
|
||||
},
|
||||
"evaluation_start_s": 1,
|
||||
"expect": {
|
||||
"base_velocity_mean_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_velocity_tracking_mae_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_angular_velocity_mean_rad_s": {
|
||||
"target": [0, 0, 0.35],
|
||||
"tolerance": [null, null, 0.12]
|
||||
},
|
||||
"base_angular_velocity_tracking_mae_rad_s": {
|
||||
"target": [0, 0, 0.35],
|
||||
"tolerance": [null, null, 0.12]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"command": {
|
||||
"lin_vel_x": 0,
|
||||
"lin_vel_y": 0,
|
||||
"ang_vel_z": -0.35
|
||||
},
|
||||
"evaluation_start_s": 1,
|
||||
"expect": {
|
||||
"base_velocity_mean_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_velocity_tracking_mae_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_angular_velocity_mean_rad_s": {
|
||||
"target": [0, 0, -0.35],
|
||||
"tolerance": [null, null, 0.12]
|
||||
},
|
||||
"base_angular_velocity_tracking_mae_rad_s": {
|
||||
"target": [0, 0, -0.35],
|
||||
"tolerance": [null, null, 0.12]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"command": {
|
||||
"lin_vel_x": 0,
|
||||
"lin_vel_y": 0,
|
||||
"ang_vel_z": 0
|
||||
},
|
||||
"evaluation_start_s": 1,
|
||||
"expect": {
|
||||
"base_velocity_mean_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_velocity_tracking_mae_m_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [0.08, 0.04, null]
|
||||
},
|
||||
"base_angular_velocity_mean_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
},
|
||||
"base_angular_velocity_tracking_mae_rad_s": {
|
||||
"target": [0, 0, 0],
|
||||
"tolerance": [null, null, 0.12]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
"""Game controllers, as their own axis.
|
||||
|
||||
A fourth thing alongside the framework, the tasks and the scenes, and separate for
|
||||
the same reason a scene is: **it knows nothing about robots.** Nothing here has an
|
||||
opinion about what a stick means. It reports that the left stick is at (0.4, -0.9)
|
||||
and that A is down; deciding that this is 0.2 m/s forward belongs to the task, in
|
||||
`mjrl.tasks.operator` or the native controller, the same way a key's meaning does.
|
||||
|
||||
controller/ this package: find a pad, read it, name it
|
||||
vocabulary.json what a pad can say -- the dictionary, and the only
|
||||
copy of that list anywhere
|
||||
mjrl/tasks/operator.py training command semantics from controls.yaml
|
||||
deploy/fsm/ the same dictionary, for the robot and the browser
|
||||
scripts/ replay forwards input to the selected controller
|
||||
|
||||
`python -m controller --vocabulary` prints the dictionary. Three programs read
|
||||
it and none of them writes it: this package, a task validating its
|
||||
`controls.yaml`, and the deployed controller refusing a `[[fsm.button]]`.
|
||||
|
||||
## What it is for
|
||||
|
||||
`play.py` samples random velocity commands. That is right for seeing a policy
|
||||
across its training distribution and useless for "walk it into the corner and see
|
||||
what happens". The keyboard already covers that (`mjrl.viewer.keys` plus a task's
|
||||
teleop term); a stick covers it better, because walking is a continuous thing and
|
||||
a key press is not.
|
||||
|
||||
## Using it
|
||||
|
||||
import controller
|
||||
|
||||
pad = controller.open() # None if nothing is plugged in
|
||||
if pad is not None:
|
||||
s = pad.state() # a snapshot; never blocks
|
||||
print(s.lx, s.ly, s.a)
|
||||
|
||||
`open()` returns None rather than raising when there is no pad, because "no
|
||||
controller today" is the normal case and not an error. It *does* print why when
|
||||
there is a device it cannot use -- a pad that is plugged in but unreadable is a
|
||||
problem to be told about, and the cause is almost always permissions rather than
|
||||
anything to do with this code.
|
||||
|
||||
## Permissions, which is where this actually fails on Linux
|
||||
|
||||
`/dev/input/event*` is `crw-rw---- root input`, so reading it needs membership of
|
||||
the `input` group:
|
||||
|
||||
sudo usermod -aG input "$USER" # then log out and back in
|
||||
|
||||
Until that is done a paired, working, blinking controller reads as absent. This
|
||||
package says so explicitly rather than looking like it simply found nothing --
|
||||
`describe()` reports what it saw and why each candidate was rejected.
|
||||
|
||||
## A new top-level package needs the editable install refreshed
|
||||
|
||||
`pip install -e .` writes a **strict name-to-path table**, not a path entry, so a
|
||||
package added after the install is invisible to anything that does not happen to
|
||||
be run from the repository root. The symptom is confusing: `python -c "import
|
||||
controller"` works (the current directory is on `sys.path`) while
|
||||
`python scripts/play.py` raises `ModuleNotFoundError`, because a script's
|
||||
`sys.path[0]` is `scripts/`. `pip install -e .` again, once, fixes it -- the same
|
||||
thing was true of `scenes` when it arrived.
|
||||
|
||||
## Platform
|
||||
|
||||
Linux through evdev (`device.py`), Windows through XInput (`xinput.py`), both with
|
||||
no dependency beyond the standard library. The Windows reader presents each
|
||||
reading as the evdev table Linux's `xpad` driver would have produced, so both
|
||||
end in the same `xbox.normalise` and a stick means the same number on either.
|
||||
macOS exposes nothing equivalent without a package, and adding one for a
|
||||
convenience feature is the wrong trade; `open()` returns None there, `describe()`
|
||||
says it is the platform, and everything that uses it carries on without a pad.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from . import device, xinput
|
||||
from .device import Gamepad, GamepadState, PadInfo
|
||||
from .xinput import XInputPad
|
||||
|
||||
__all__ = [
|
||||
"Gamepad", "GamepadState", "XInputPad", "available", "describe", "find_pads", "open",
|
||||
]
|
||||
|
||||
|
||||
def _backend():
|
||||
"""The module that reads pads on this platform, or None where nothing does."""
|
||||
if sys.platform == "linux":
|
||||
return device
|
||||
if sys.platform == "win32":
|
||||
return xinput
|
||||
return None
|
||||
|
||||
|
||||
def find_pads(readable_only: bool = False) -> list[PadInfo]:
|
||||
"""Every pad this platform's reader can see, in a stable order."""
|
||||
backend = _backend()
|
||||
return backend.find_pads(readable_only) if backend is not None else []
|
||||
|
||||
|
||||
def describe() -> str:
|
||||
"""What was found and, for anything rejected, why. For diagnosing "the pad is
|
||||
connected but nothing happens", which is otherwise indistinguishable from
|
||||
"no pad"."""
|
||||
backend = _backend()
|
||||
if backend is None:
|
||||
return (
|
||||
f"pads are read on Linux (evdev) and Windows (XInput), not on "
|
||||
f"{sys.platform}; the keyboard drives."
|
||||
)
|
||||
return backend.describe()
|
||||
|
||||
|
||||
def available() -> bool:
|
||||
"""Is there a controller this process can actually read?
|
||||
|
||||
Cheap: it stats the device nodes and checks readability (on Windows, asks each
|
||||
XInput slot once), and never opens a reader thread. Safe to call while building
|
||||
a config.
|
||||
"""
|
||||
return bool(find_pads(readable_only=True))
|
||||
|
||||
|
||||
def open(name: str | None = None) -> Gamepad | XInputPad | None:
|
||||
"""Open the first usable controller, or None.
|
||||
|
||||
Args:
|
||||
name: substring of the device name to require, for a machine with more
|
||||
than one pad. None takes the first.
|
||||
"""
|
||||
backend = _backend()
|
||||
if backend is None:
|
||||
return None
|
||||
pads = backend.find_pads(readable_only=True)
|
||||
if name is not None:
|
||||
pads = [p for p in pads if name.lower() in p.name.lower()]
|
||||
if not pads:
|
||||
return None
|
||||
pad = (XInputPad if backend is xinput else Gamepad)(pads[0])
|
||||
pad.start()
|
||||
return pad
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Live readout of the connected pad: `python -m controller`.
|
||||
|
||||
The one thing no test on this side can settle is whether pushing a stick forward
|
||||
reads as forward. Everything else about a controller can be checked from synthetic
|
||||
values, but the mapping from a physical direction to a number needs a hand on the
|
||||
pad -- so this exists to make that a five-second check rather than a guess, and to
|
||||
be the first thing to run when a pad "does nothing".
|
||||
|
||||
python -m controller # live values
|
||||
python -m controller --raw # every (type, code) the pad emits
|
||||
python -m controller --vocabulary # the words a binding may use
|
||||
|
||||
`--raw` is for a pad whose layout `xbox.py` does not know: press the control, see
|
||||
which code moves, and add it there.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import time
|
||||
|
||||
import controller
|
||||
from controller.xbox import layout_of
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if "--vocabulary" in sys.argv:
|
||||
# No pad needed: this is the dictionary, not a reading of one.
|
||||
from . import vocabulary
|
||||
|
||||
print(vocabulary.describe(), end="")
|
||||
return 0
|
||||
raw_mode = "--raw" in sys.argv
|
||||
|
||||
print(controller.describe())
|
||||
pad = controller.open()
|
||||
if pad is None:
|
||||
print("\nnothing to read. If a pad is listed above as unusable, the reason "
|
||||
"is on that line.")
|
||||
return 1
|
||||
|
||||
layout = layout_of(pad._ranges)
|
||||
print(f"\n{pad.name}: {layout.name} layout, "
|
||||
f"axes {sorted(pad._ranges)}")
|
||||
print("\nPush each control and watch the numbers. Ctrl-C to stop.")
|
||||
print("Expected: left stick up -> ly negative (evdev's Y is positive down),")
|
||||
print(" left stick left -> lx negative, right stick right -> rx positive\n")
|
||||
|
||||
try:
|
||||
while True:
|
||||
if raw_mode:
|
||||
items = sorted(pad.raw().items())
|
||||
line = " ".join(f"{t:x}:{c:x}={v}" for (t, c), v in items if v)
|
||||
else:
|
||||
s = pad.state()
|
||||
pressed = " ".join(sorted(s.buttons)) or "-"
|
||||
line = (
|
||||
f"L({s.lx:+.2f},{s.ly:+.2f}) R({s.rx:+.2f},{s.ry:+.2f}) "
|
||||
f"LT {s.lt:.2f} RT {s.rt:.2f} "
|
||||
f"hat({s.hat_x:+d},{s.hat_y:+d}) [{pressed}]"
|
||||
)
|
||||
print(f"\r{line:<100}", end="", flush=True)
|
||||
time.sleep(0.05)
|
||||
except KeyboardInterrupt:
|
||||
print()
|
||||
finally:
|
||||
pad.close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,991 @@
|
||||
"""Read a task's `controls.yaml`, the operator-control specification.
|
||||
|
||||
**The specification is the source of truth and this is the only reader.** The
|
||||
signs, the splits and the bindings live in the file; `mjrl.tasks.operator` drives
|
||||
`replay from what this returns, `scripts/export.py` embeds it in the deployment
|
||||
contract, and `deploy/fsm/src/operator.rs` drives the robot and the browser from
|
||||
that. Programs that must agree about which way is forward agree by reading one
|
||||
file rather than by people reading each other's docstrings.
|
||||
|
||||
Everything here fails loudly, for the reason `load_servo_curve` gives about the
|
||||
motor spec: every quiet alternative is worse than a stopped import. A missing
|
||||
axis, a sign left out, a `sign: 0`, a binding naming a key that does not exist --
|
||||
each would otherwise become a default, and a default sign is a robot that walks
|
||||
the wrong way while the screen looks entirely normal.
|
||||
|
||||
## What the file says
|
||||
|
||||
`command` is a list, one entry per command term an operator drives -- one for a
|
||||
velocity-tracking task and one more if the policy takes posture input. **The pad and the keyboard are two
|
||||
paths**, each bound straight to what it does, and neither names a control of
|
||||
the other (2026-09-29):
|
||||
|
||||
* **The pad** binds a stick or a trigger to a command axis, with a sign. A stick
|
||||
can carry more than one axis: a binding may take part of its travel
|
||||
(`travel: [0.0, 0.5]`), hand it on to the next one (`[0.0, 0.5, 0.5, 0.75]`
|
||||
climbs over the first half and falls back to zero by three quarters), sit on
|
||||
a layer a button brings in (`shifted: true`), or be one of several controls
|
||||
summed into one axis (a list). The loader refuses two bindings that would
|
||||
climb over the same part of the same stick, which is the one way this could go
|
||||
quietly wrong.
|
||||
* **The keyboard** binds keystrokes to one direction of an axis -- `"+"` is the
|
||||
axis's own positive direction, as its `positive:` words it, so the keyboard
|
||||
has no signs: there is no stick for a sign to be relative to. A keystroke is a
|
||||
key (`key_j`), a modifier on its own (`shift`), or a modifier held with a key
|
||||
(`shift+key_j`), which is a different keystroke from the key alone: J and
|
||||
Shift + J are two things and never both (`controller/vocabulary.py`). A key
|
||||
held pushes its direction further the longer it is held, full after
|
||||
`full_after_s`, and let go it is back at rest. An axis a task deliberately
|
||||
leaves off the keyboard says why (`unbound:`).
|
||||
|
||||
Until 2026-09-29 the keyboard was a *virtual pad*: a key held moved one of
|
||||
the pad's controls, read through the pad's own mapping. That made the two
|
||||
devices agree by construction, and it made a keyboard that could not lay
|
||||
itself out for hands: a key could only be a stick, so J could not turn while
|
||||
Shift + J twisted, and Shift, Alt and Ctrl could not hold an arm out. The
|
||||
layout Control-agent 3.1 asks for needs both, so the keyboard became a path
|
||||
of its own, and agreement is the file's: both paths name the same axes, and
|
||||
`load_controls` refuses a command axis either one leaves out without saying.
|
||||
* **An axis may move rather than be placed** (`integrate_s`), on the pad. A
|
||||
deflection is then a speed -- full deflection carries the axis from its rest
|
||||
to either end of its range in that many seconds -- and let go it stays where
|
||||
it is. A body-height command is one use. `devices.gamepad.reset`
|
||||
names the button whose tap puts every such axis back at its rest. The
|
||||
keyboard places it as it places every other axis: a key held pushes it
|
||||
towards its end and it is back at rest when let go.
|
||||
* **A task may keep controls for itself.** The top-level `task:` names controls
|
||||
no command reads and what the task does with each. Each device binds every
|
||||
one of them: the pad
|
||||
one control each (an axis for an `amount`, a d-pad direction for a `press`),
|
||||
the keyboard one or more keystrokes. The task's own code answers them by
|
||||
name, in replay a term of its own (`Operator.task_control`) and on the robot
|
||||
its deploy hook (`deploy/fsm/src/hook.rs`).
|
||||
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import itertools
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
# The control dictionary: what a pad and a keyboard can say. Read, never
|
||||
# repeated -- see `_AXES` below.
|
||||
from controller import vocabulary as _vocabulary
|
||||
|
||||
__all__ = [
|
||||
"CENTRE",
|
||||
"SCHEMA",
|
||||
"Axis",
|
||||
"Binding",
|
||||
"Command",
|
||||
"Controls",
|
||||
"KeyAxis",
|
||||
"TaskControl",
|
||||
"controller_contract",
|
||||
"key_name",
|
||||
"load_controls",
|
||||
"stroke_name",
|
||||
]
|
||||
|
||||
#: The schema this reader implements, and the contract block it writes is
|
||||
#: `operator_controller/<this>`.
|
||||
SCHEMA = 2
|
||||
|
||||
#: The scaling rule this repository implements. A file asking for another one is
|
||||
#: refused rather than run under this one.
|
||||
_SCALING_RULE = "full_deflection_is_range_edge"
|
||||
|
||||
#: The gamepad vocabulary, read from the dictionary rather than repeated.
|
||||
#:
|
||||
#: `controller/vocabulary.json` is the list and
|
||||
#: `python -m controller --vocabulary` prints it. This module only decides which
|
||||
#: of those words a task's controls file may use and what happens when it does
|
||||
#: -- the meanings, which is the part that belongs to a task.
|
||||
#:
|
||||
#: There is no list here on purpose. There were three for a day: this one,
|
||||
#: `deploy/fsm/src/vocabulary.rs`, and a prose block in a deploy manifest.
|
||||
_AXES = _vocabulary.axes()
|
||||
_BUTTONS = frozenset(_vocabulary.buttons())
|
||||
|
||||
#: Axis name -> the attribute a `controller` state object carries. A case
|
||||
#: change and nothing else: `controller/xbox.py` reports the dictionary's names
|
||||
#: now, and this is only Python's habit of lowercase attributes.
|
||||
_STICK_FIELD = {name: name.lower() for name in _AXES}
|
||||
|
||||
#: Which of the dictionary's axes are sticks (-1..1) and which are triggers
|
||||
#: (0..1). Read from the ranges the dictionary gives, not from the names.
|
||||
_BIPOLAR = frozenset(a["name"] for a in _vocabulary.load()["axes"] if a["range"][0] < 0)
|
||||
|
||||
#: The four d-pad directions: where a task's `press` control goes on the pad.
|
||||
#: A rocker is held or not, which is what a `press` reads, and the ten buttons
|
||||
#: stay the bundle's for switching modes and the operator's for letting go.
|
||||
_DPAD = frozenset(b["name"] for b in _vocabulary.load()["dpad"])
|
||||
|
||||
#: The notched keyboard's other half, retired on 2026-09-27: one key letting go
|
||||
#: of every stick and trigger at once, needed while a key left its control where
|
||||
#: the last press put it. A key let up is back at rest and the release lets go
|
||||
#: of everything, so it did nothing either did not. Named only to refuse a file
|
||||
#: that still has it.
|
||||
CENTRE = "centre"
|
||||
|
||||
#: The gestures a `shift` implements: `hold`, the layer in for as long as the
|
||||
#: button is held, and `toggle`, in from one click to the next. `rise` and
|
||||
#: `fall` are one tick long, which a layer cannot be.
|
||||
_SHIFT_GESTURES = ("hold", "toggle")
|
||||
|
||||
#: The keyboard scheme: keystrokes bound to what they do. See the module
|
||||
#: docstring.
|
||||
_KEYBOARD_SCHEME = "keys"
|
||||
|
||||
#: What a task control is: `amount`, 0 to 1 as far as it is held -- a trigger,
|
||||
#: or a key's ramp -- and `press`, 1 while it is held and 0 otherwise.
|
||||
_TASK_KINDS = ("amount", "press")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Axis:
|
||||
name: str
|
||||
unit: str
|
||||
positive: str
|
||||
#: The command config field holding where a centred stick puts this axis.
|
||||
#: `None` is zero. A height is the case that needs it -- zero is a
|
||||
#: body on the floor, and the range it would be clamped to is not where the
|
||||
#: robot stands.
|
||||
rest: str | None = None
|
||||
#: Seconds full deflection takes to carry the axis from its rest to either
|
||||
#: end of its range: the controls move it rather than place it, and let go
|
||||
#: it stays where it is. `None` is an axis a control places, as a stick's
|
||||
#: position places a walking speed.
|
||||
integrate_s: float | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Command:
|
||||
"""One command term an operator drives. The file has a list of these."""
|
||||
|
||||
#: The command term's key in `cfg.commands`.
|
||||
term: str
|
||||
feeds: str
|
||||
frame: str
|
||||
axes: tuple[Axis, ...]
|
||||
scaling: dict[str, Any]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Binding:
|
||||
"""One pad control's share of one command axis."""
|
||||
|
||||
#: A dictionary axis: `Lx` `Ly` `Rx` `Ry` `LT` `RT`.
|
||||
source: str
|
||||
sign: float
|
||||
#: The part of the control's travel this binding takes, as fractions of
|
||||
#: full deflection. `(0.0, 0.5)` reaches full at half travel and holds
|
||||
#: there; `(0.5, 1.0)` is silent until half travel and then climbs. Two
|
||||
#: more, `(0.0, 0.5, 0.5, 0.75)`, are where it falls back: full until the
|
||||
#: third, zero from the fourth on -- a binding that gives the control up to
|
||||
#: the one climbing over the same stretch.
|
||||
travel: tuple[float, ...] = (0.0, 1.0)
|
||||
#: Live only while the `shift` button has the other layer in -- held, or
|
||||
#: toggled, as `shift_gesture` says. A source that has a shifted binding
|
||||
#: drives **only** that binding while shifted; every other source is
|
||||
#: unaffected.
|
||||
shifted: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class KeyAxis:
|
||||
"""One command axis on the keyboard: the keystrokes pushing it each way,
|
||||
or why none do."""
|
||||
|
||||
#: Keystrokes towards the axis's own positive direction, as its `positive:`
|
||||
#: words it, and away from it. Both empty exactly when `unbound` says why.
|
||||
plus: tuple[str, ...] = ()
|
||||
minus: tuple[str, ...] = ()
|
||||
unbound: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TaskControl:
|
||||
"""A control the task answers itself, by name: no command reads it."""
|
||||
|
||||
name: str
|
||||
#: `amount` (0..1, as far as it is held) or `press` (1 while held).
|
||||
kind: str
|
||||
#: What the task does with it, in the file's words.
|
||||
does: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Controls:
|
||||
"""The specification, parsed: the commands, one operator for all of them."""
|
||||
|
||||
commands: tuple[Command, ...]
|
||||
#: The pad: command axis name -> the controls that drive it, summed.
|
||||
bindings: dict[str, tuple[Binding, ...]]
|
||||
#: The button that brings the shifted layer in, or `None` for a file with none.
|
||||
shift: str | None
|
||||
release_button: str
|
||||
#: Seconds a key is held for its direction to reach full deflection, or a
|
||||
#: task's `amount` to reach 1. It climbs linearly until then and is back at
|
||||
#: rest when the key comes up.
|
||||
full_after_s: float
|
||||
#: The keyboard: command axis name -> its keystrokes, every axis present.
|
||||
key_axes: dict[str, KeyAxis]
|
||||
#: Keystrokes that let go of everything, as the pad's release button does.
|
||||
key_release: tuple[str, ...]
|
||||
#: The controls the task keeps for itself, by name, in the file's order.
|
||||
#: Empty for most tasks.
|
||||
task: dict[str, TaskControl]
|
||||
#: Task control -> the one pad control it is on.
|
||||
pad_task: dict[str, str]
|
||||
#: Task control -> the keystrokes it is on.
|
||||
key_task: dict[str, tuple[str, ...]]
|
||||
keyboard: dict[str, Any]
|
||||
gamepad: dict[str, Any]
|
||||
source_sha256: str
|
||||
source: Path
|
||||
#: How the shift brings its layer in: `hold`, for as long as it is held,
|
||||
#: or `toggle`, from one click to the next. `None` with no shift.
|
||||
shift_gesture: str | None = None
|
||||
#: The button whose tap puts every integrating axis back at its rest, or
|
||||
#: `None`. A tap is the button pressed and let go with no stick it could
|
||||
#: have been reaching for moved in between.
|
||||
reset: str | None = None
|
||||
|
||||
def command(self, term: str) -> Command:
|
||||
for c in self.commands:
|
||||
if c.term == term:
|
||||
return c
|
||||
raise KeyError(
|
||||
f"{self.source}: describes the commands {[c.term for c in self.commands]}, "
|
||||
f"not {term!r}. A term driven from a file that does not describe it "
|
||||
"would take its bindings from nowhere."
|
||||
)
|
||||
|
||||
@property
|
||||
def sources(self) -> frozenset[str]:
|
||||
"""Every pad axis some command binding reads."""
|
||||
return frozenset(b.source for bs in self.bindings.values() for b in bs)
|
||||
|
||||
@property
|
||||
def integrating(self) -> dict[str, float]:
|
||||
"""Axis name -> `integrate_s`, for every axis the controls move rather
|
||||
than place."""
|
||||
return {a.name: a.integrate_s for c in self.commands for a in c.axes
|
||||
if a.integrate_s is not None}
|
||||
|
||||
@property
|
||||
def keystrokes(self) -> tuple[str, ...]:
|
||||
"""Every keystroke the keyboard block binds, once each, in file order."""
|
||||
strokes = [s for k in self.key_axes.values() for s in (*k.plus, *k.minus)]
|
||||
strokes += self.key_release
|
||||
strokes += [s for ss in self.key_task.values() for s in ss]
|
||||
return tuple(dict.fromkeys(strokes))
|
||||
|
||||
|
||||
class _StrictLoader(yaml.SafeLoader):
|
||||
"""`yaml.safe_load`, except that a key written twice is an error.
|
||||
|
||||
PyYAML keeps the last of two equal keys and says nothing, so an axis written
|
||||
twice in the keyboard's `axes` is one binding silently gone -- the keyboard
|
||||
then cannot do something and nothing points at the line that took it away.
|
||||
"""
|
||||
|
||||
|
||||
def _mapping_without_duplicates(loader: yaml.SafeLoader, node: yaml.MappingNode) -> dict:
|
||||
seen: set[Any] = set()
|
||||
for key_node, _value in node.value:
|
||||
key = loader.construct_object(key_node)
|
||||
if key in seen:
|
||||
raise ValueError(
|
||||
f"line {key_node.start_mark.line + 1}: {key!r} is written twice. YAML "
|
||||
"keeps the last one and drops the other without a word."
|
||||
)
|
||||
seen.add(key)
|
||||
return loader.construct_mapping(node, deep=True)
|
||||
|
||||
|
||||
_StrictLoader.add_constructor(
|
||||
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _mapping_without_duplicates
|
||||
)
|
||||
|
||||
|
||||
def _require(doc: Any, path: str, where: Path) -> Any:
|
||||
node = doc
|
||||
for part in path.split("."):
|
||||
if not isinstance(node, dict) or part not in node:
|
||||
raise ValueError(f"{where}: missing {path}")
|
||||
node = node[part]
|
||||
if node is None:
|
||||
raise ValueError(f"{where}: {path} is null; it has to be decided, not defaulted")
|
||||
return node
|
||||
|
||||
|
||||
def load_controls(path: Path) -> Controls:
|
||||
"""Parse a task's `controls.yaml`. See the module docstring."""
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"{path} not found. It holds how an operator's input becomes the "
|
||||
"command the policy observes, and neither play nor the exporter can "
|
||||
"proceed without it."
|
||||
)
|
||||
raw = path.read_bytes()
|
||||
try:
|
||||
doc = yaml.load(raw.decode("utf-8"), Loader=_StrictLoader) # a SafeLoader subclass
|
||||
except ValueError as exc:
|
||||
raise ValueError(f"{path}: {exc}") from None
|
||||
|
||||
version = _require(doc, "schema_version", path)
|
||||
if version != SCHEMA:
|
||||
raise ValueError(
|
||||
f"{path}: schema_version is {version!r}; this reader implements {SCHEMA}. "
|
||||
"Schema 1 was retired -- regenerate the file with "
|
||||
".claude/skills/controls/scripts/write_controls.py."
|
||||
)
|
||||
return _parse(doc, raw, path)
|
||||
|
||||
|
||||
def _check_button(name: Any, what: str, path: Path) -> None:
|
||||
# A button name is a contract, not a convention: `view` is on the pad and
|
||||
# the robot's service does not send it, so binding it works on a bench and
|
||||
# does nothing on the robot.
|
||||
if name not in _BUTTONS:
|
||||
extra = (" `view` is on the pad but the service does not publish it."
|
||||
if name == "view" else "")
|
||||
raise ValueError(
|
||||
f"{path}: {what} is {name!r}, which the robot's gamepad service "
|
||||
f"does not publish. It publishes {sorted(_BUTTONS)}.{extra}"
|
||||
)
|
||||
|
||||
|
||||
def _check_stroke(stroke: Any, where: str, path: Path) -> tuple[str | None, str]:
|
||||
"""A keystroke, or a refusal naming where it was written."""
|
||||
if not isinstance(stroke, str):
|
||||
raise ValueError(f"{path}: {where} has {stroke!r}, which is not a keystroke")
|
||||
try:
|
||||
return _vocabulary.keystroke(stroke)
|
||||
except ValueError as exc:
|
||||
raise ValueError(
|
||||
f"{path}: {where}: {exc}\n"
|
||||
f" python -m controller --vocabulary\n"
|
||||
f" (A key it lacks is added there, with what it also does in "
|
||||
f"MuJoCo's viewer: the viewer calls the user callback in addition to its "
|
||||
f"own shortcuts, so most keys do two things at once.)"
|
||||
) from None
|
||||
|
||||
|
||||
def _positive_seconds(value: Any) -> bool:
|
||||
return (not isinstance(value, bool) and isinstance(value, (int, float))
|
||||
and 0 < value < math.inf)
|
||||
|
||||
|
||||
def _travel_is_ordered(travel: list) -> bool:
|
||||
"""`0 <= from < to <= 1`, and with a fall `to <= back < off <= 1`.
|
||||
|
||||
A climb or a fall of zero length would be a step, which a stick's travel
|
||||
cannot hold still on; the hold between them may be zero, and is, where one
|
||||
binding's fall starts as the next one's climb does."""
|
||||
a, z, *fall = travel
|
||||
if not 0.0 <= a < z <= 1.0:
|
||||
return False
|
||||
return not fall or z <= fall[0] < fall[1] <= 1.0
|
||||
|
||||
|
||||
def _field(node: Any, key: str, at: str, path: Path) -> Any:
|
||||
"""`_require` for a node inside a list, so the message says which entry."""
|
||||
if not isinstance(node, dict) or key not in node:
|
||||
raise ValueError(f"{path}: missing {at}.{key}")
|
||||
if node[key] is None:
|
||||
raise ValueError(f"{path}: {at}.{key} is null; it has to be decided, not defaulted")
|
||||
return node[key]
|
||||
|
||||
|
||||
def _parse(doc: dict, raw: bytes, path: Path) -> Controls:
|
||||
"""Every check here is a control that would drive nothing, drive two things,
|
||||
or be out of one device's reach, without anything raising."""
|
||||
commands = _parse_commands(doc, path)
|
||||
names = [a.name for c in commands for a in c.axes]
|
||||
task = _parse_task(doc, path)
|
||||
|
||||
gamepad = _require(doc, "devices.gamepad", path)
|
||||
bindings, shift, gesture, release, reset = _parse_pad(gamepad, commands, path)
|
||||
pad_task = _parse_pad_task(gamepad, task, bindings, {release, shift, reset}, path)
|
||||
|
||||
keyboard = _require(doc, "devices.keyboard", path)
|
||||
full, key_axes, key_release, key_task = _parse_keyboard(keyboard, names, task, path)
|
||||
|
||||
return Controls(
|
||||
commands=commands,
|
||||
bindings=bindings,
|
||||
shift=shift,
|
||||
release_button=release,
|
||||
full_after_s=float(full),
|
||||
key_axes=key_axes,
|
||||
key_release=key_release,
|
||||
task=task,
|
||||
pad_task=pad_task,
|
||||
key_task=key_task,
|
||||
keyboard=keyboard,
|
||||
gamepad=gamepad,
|
||||
source_sha256=hashlib.sha256(raw).hexdigest(),
|
||||
source=path,
|
||||
shift_gesture=gesture if shift is not None else None,
|
||||
reset=reset,
|
||||
)
|
||||
|
||||
|
||||
def _parse_commands(doc: dict, path: Path) -> tuple[Command, ...]:
|
||||
listed = _require(doc, "command", path)
|
||||
if not isinstance(listed, list) or not listed:
|
||||
raise ValueError(
|
||||
f"{path}: `command` is a list of the commands an operator "
|
||||
f"drives, one entry per command term; got {type(listed).__name__}"
|
||||
)
|
||||
commands = []
|
||||
for i, entry in enumerate(listed):
|
||||
at = f"command[{i}]"
|
||||
axes = []
|
||||
for j, a in enumerate(_field(entry, "axes", at, path)):
|
||||
where = f"{at}.axes[{j}]"
|
||||
integrate = a.get("integrate_s") if isinstance(a, dict) else None
|
||||
if isinstance(a, dict) and "integrate_s" in a and not _positive_seconds(integrate):
|
||||
raise ValueError(
|
||||
f"{path}: {where}.integrate_s is {integrate!r}; it is the seconds full "
|
||||
"deflection takes to carry the axis from its rest to either end of its "
|
||||
"range, a positive number. Leave it out for an axis a control places."
|
||||
)
|
||||
axes.append(Axis(
|
||||
name=_field(a, "name", where, path),
|
||||
unit=_field(a, "unit", where, path),
|
||||
positive=_field(a, "positive", where, path),
|
||||
rest=a.get("rest"),
|
||||
integrate_s=float(integrate) if integrate is not None else None,
|
||||
))
|
||||
if not axes:
|
||||
raise ValueError(f"{path}: {at} names no axes")
|
||||
scaling = _field(entry, "scaling", at, path)
|
||||
if scaling.get("rule") != _SCALING_RULE:
|
||||
raise ValueError(
|
||||
f"{path}: {at}.scaling.rule is {scaling.get('rule')!r}; this repository "
|
||||
f"implements {_SCALING_RULE!r}. Add the rule rather than running the wrong one."
|
||||
)
|
||||
commands.append(Command(
|
||||
term=_field(entry, "term", at, path),
|
||||
feeds=_field(entry, "feeds", at, path),
|
||||
frame=_field(entry, "frame", at, path),
|
||||
axes=tuple(axes),
|
||||
scaling=scaling,
|
||||
))
|
||||
terms = [c.term for c in commands]
|
||||
if len(set(terms)) != len(terms):
|
||||
raise ValueError(f"{path}: a command term is listed twice: {terms}")
|
||||
# Bindings are keyed by axis name, so a name two commands share would be one
|
||||
# binding driving both.
|
||||
names = [a.name for c in commands for a in c.axes]
|
||||
if len(set(names)) != len(names):
|
||||
raise ValueError(f"{path}: an axis name is used twice across the commands: {names}")
|
||||
return tuple(commands)
|
||||
|
||||
|
||||
def _parse_task(doc: dict, path: Path) -> dict[str, TaskControl]:
|
||||
"""The top-level `task:`, the controls the task answers itself."""
|
||||
declared = doc.get("task")
|
||||
if declared is None:
|
||||
if "task" in doc:
|
||||
raise ValueError(f"{path}: task is null; leave it out for a task that keeps none")
|
||||
return {}
|
||||
if not isinstance(declared, dict) or not all(isinstance(n, str) for n in declared):
|
||||
raise ValueError(
|
||||
f"{path}: task is a {type(declared).__name__}; it maps each control the task "
|
||||
"answers itself to its `kind` and what it `does`"
|
||||
)
|
||||
task = {}
|
||||
for name, entry in declared.items():
|
||||
kind = _field(entry, "kind", f"task.{name}", path)
|
||||
if kind not in _TASK_KINDS:
|
||||
raise ValueError(
|
||||
f"{path}: task.{name}.kind is {kind!r}; a task control is `amount` (0 to "
|
||||
"1, as far as it is held) or `press` (1 while it is held)"
|
||||
)
|
||||
does = entry.get("does")
|
||||
if not isinstance(does, str) or not does.strip():
|
||||
raise ValueError(f"{path}: the task keeps {name} and does not say what it does")
|
||||
task[name] = TaskControl(name, kind, does)
|
||||
return task
|
||||
|
||||
|
||||
def _parse_pad(gamepad: dict, commands: tuple[Command, ...], path: Path):
|
||||
"""The pad's command bindings, its shift, its release and its reset."""
|
||||
names = [a.name for c in commands for a in c.axes]
|
||||
for field in ("scheme", "layout", "axes", "deadzone", "release_button"):
|
||||
_require(gamepad, field, path)
|
||||
if gamepad["scheme"] != "absolute":
|
||||
raise ValueError(
|
||||
f"{path}: gamepad scheme is {gamepad['scheme']!r}. This file implements "
|
||||
"`absolute` only: a stick's position is the command. An axis the controls "
|
||||
"should move rather than place says `integrate_s` on the axis, for both "
|
||||
"devices at once."
|
||||
)
|
||||
release = gamepad["release_button"]
|
||||
_check_button(release, "release_button", path)
|
||||
|
||||
shift = gesture = None
|
||||
if "shift" in gamepad:
|
||||
declared = gamepad["shift"]
|
||||
shift = _field(declared, "button", "devices.gamepad.shift", path)
|
||||
_check_button(shift, "shift.button", path)
|
||||
gesture = _field(declared, "gesture", "devices.gamepad.shift", path)
|
||||
if gesture not in _vocabulary.gestures():
|
||||
raise ValueError(
|
||||
f"{path}: shift.gesture is {gesture!r}; the dictionary's gestures are "
|
||||
f"{list(_vocabulary.gestures())}"
|
||||
)
|
||||
if gesture not in _SHIFT_GESTURES:
|
||||
raise ValueError(
|
||||
f"{path}: shift.gesture is {gesture!r}; a shift is "
|
||||
f"{' or '.join(map(repr, _SHIFT_GESTURES))}. "
|
||||
"A one-tick gesture brings a layer in for one tick."
|
||||
)
|
||||
if shift == release:
|
||||
raise ValueError(
|
||||
f"{path}: {shift} is both the shift and the release button. One "
|
||||
"press would hand the command back and change what a stick means."
|
||||
)
|
||||
|
||||
bound = gamepad["axes"]
|
||||
stray = sorted(set(bound) - set(names))
|
||||
if stray:
|
||||
raise ValueError(
|
||||
f"{path}: gamepad.axes names {stray}, which no command has. A typo "
|
||||
f"here is a stick that drives nothing. The axes are {names}"
|
||||
)
|
||||
bindings: dict[str, tuple[Binding, ...]] = {}
|
||||
for name in names:
|
||||
entry = bound.get(name)
|
||||
if not entry:
|
||||
raise ValueError(f"{path}: no control drives {name}; every command axis needs one")
|
||||
parts = []
|
||||
for part in entry if isinstance(entry, list) else [entry]:
|
||||
if not isinstance(part, dict) or "source" not in part:
|
||||
raise ValueError(f"{path}: {name} has a binding with no source: {part!r}")
|
||||
source = part["source"]
|
||||
if source not in _STICK_FIELD:
|
||||
raise ValueError(
|
||||
f"{path}: {name} names {source!r}, which the robot's gamepad "
|
||||
f"service does not publish. It publishes {list(_AXES)}"
|
||||
)
|
||||
sign = part.get("sign")
|
||||
if sign not in (1, -1):
|
||||
# Not clamped to a default: a sign is the one value here whose
|
||||
# wrong answer looks exactly like the right one.
|
||||
raise ValueError(f"{path}: {name} has sign {sign!r}; it must be 1 or -1")
|
||||
travel = part.get("travel", [0.0, 1.0])
|
||||
if (
|
||||
not isinstance(travel, list) or len(travel) not in (2, 4)
|
||||
or not all(isinstance(t, (int, float)) for t in travel)
|
||||
or not _travel_is_ordered(travel)
|
||||
):
|
||||
raise ValueError(
|
||||
f"{path}: {name} has travel {travel!r}; it is [from, to] with "
|
||||
"0 <= from < to <= 1, or [from, to, back, off] with "
|
||||
"0 <= from < to <= back < off <= 1, as fractions of full deflection"
|
||||
)
|
||||
shifted = part.get("shifted", False)
|
||||
if type(shifted) is not bool:
|
||||
raise ValueError(f"{path}: {name} has shifted {shifted!r}; it is true or false")
|
||||
if shifted and shift is None:
|
||||
raise ValueError(
|
||||
f"{path}: {name} is on the shifted layer and the file declares no "
|
||||
"shift button, so nothing can ever reach it"
|
||||
)
|
||||
parts.append(Binding(source, float(sign), tuple(float(t) for t in travel), shifted))
|
||||
bindings[name] = tuple(parts)
|
||||
|
||||
# No two bindings may climb over the same stretch of the same control in the
|
||||
# same layer: one deflection would then move two axes, and each binding
|
||||
# alone looks correct. A binding's fall is not a claim -- it is the one
|
||||
# stretch where a deflection may move two axes, because it is how the
|
||||
# binding gives the control up to the next one.
|
||||
claims: dict[tuple[str, bool], list[tuple[float, float, str]]] = {}
|
||||
for name, parts in bindings.items():
|
||||
for b in parts:
|
||||
claims.setdefault((b.source, b.shifted), []).append((*b.travel[:2], name))
|
||||
for (source, layer), spans in claims.items():
|
||||
spans.sort()
|
||||
for (_a0, b0, n0), (a1, b1, n1) in itertools.pairwise(spans):
|
||||
if a1 < b0:
|
||||
raise ValueError(
|
||||
f"{path}: {n0} and {n1} both take {source} from {a1:g} to "
|
||||
f"{min(b0, b1):g} of its travel{' on the shifted layer' if layer else ''}. "
|
||||
"One deflection would move both; give them separate travel."
|
||||
)
|
||||
if shift is not None and not any(b.shifted for parts in bindings.values() for b in parts):
|
||||
raise ValueError(f"{path}: {shift} is declared as the shift and no binding is shifted")
|
||||
|
||||
# The reset: a tap puts every integrating axis back at its rest. With no
|
||||
# such axis it would be a button that does nothing, bound; on the release
|
||||
# button it would be a second meaning for one press. It may be the shift --
|
||||
# a hold can change height and a tap can put it back
|
||||
# to standing, told apart by whether the stick moved.
|
||||
reset = gamepad.get("reset")
|
||||
if reset is not None or "reset" in gamepad:
|
||||
_check_button(reset, "reset", path)
|
||||
if reset == release:
|
||||
raise ValueError(
|
||||
f"{path}: {reset} is both the reset and the release button. The release "
|
||||
"already puts every integrating axis back at its rest; a reset on it "
|
||||
"is the same press read twice."
|
||||
)
|
||||
if not any(a.integrate_s is not None for c in commands for a in c.axes):
|
||||
raise ValueError(
|
||||
f"{path}: gamepad.reset is {reset}, and no axis has `integrate_s`. It "
|
||||
"puts an integrating axis back at its rest; with none, it is a button "
|
||||
"that does nothing and looks bound."
|
||||
)
|
||||
return bindings, shift, gesture, release, reset
|
||||
|
||||
|
||||
def _parse_pad_task(
|
||||
gamepad: dict, task: dict[str, TaskControl], bindings: dict[str, tuple[Binding, ...]],
|
||||
reserved: set, path: Path,
|
||||
) -> dict[str, str]:
|
||||
"""`gamepad.task`: every task control on one pad control of its kind."""
|
||||
pad_task = _task_block(gamepad, "gamepad", task, path)
|
||||
commanded = {b.source for parts in bindings.values() for b in parts}
|
||||
owner: dict[str, str] = {}
|
||||
for name, control in pad_task.items():
|
||||
kind = task[name].kind
|
||||
if kind == "press" and control not in _DPAD:
|
||||
raise ValueError(
|
||||
f"{path}: gamepad.task.{name} is {control!r}; a `press` goes on a d-pad "
|
||||
f"direction ({sorted(_DPAD)}) -- the buttons are the bundle's mode "
|
||||
"switches and the operator's release"
|
||||
)
|
||||
if kind == "amount" and control not in _STICK_FIELD:
|
||||
raise ValueError(
|
||||
f"{path}: gamepad.task.{name} is {control!r}; an `amount` goes on an axis "
|
||||
f"the robot's gamepad service publishes, a trigger or a stick ({list(_AXES)})"
|
||||
)
|
||||
if control in commanded:
|
||||
raise ValueError(
|
||||
f"{path}: {control} is both the task's ({name}) and a command's; one "
|
||||
"deflection would do both"
|
||||
)
|
||||
if control in reserved:
|
||||
raise ValueError(
|
||||
f"{path}: {control} is the task's ({name}) and the release, the shift or "
|
||||
"the reset button; one press would do both"
|
||||
)
|
||||
if control in owner:
|
||||
raise ValueError(
|
||||
f"{path}: {control} is on two task controls, {owner[control]} and {name}; "
|
||||
"the task could not tell them apart"
|
||||
)
|
||||
owner[control] = name
|
||||
return pad_task
|
||||
|
||||
|
||||
def _task_block(device: dict, which: str, task: dict[str, TaskControl], path: Path) -> dict:
|
||||
"""A device's `task:`, naming exactly the controls the top-level `task:` does."""
|
||||
block = device.get("task")
|
||||
if block is None:
|
||||
block = {}
|
||||
if not isinstance(block, dict):
|
||||
raise ValueError(
|
||||
f"{path}: {which}.task is a {type(block).__name__}; it maps each control the "
|
||||
"top-level `task:` declares to where it is on this device"
|
||||
)
|
||||
stray = sorted(set(block) - set(task))
|
||||
if stray:
|
||||
raise ValueError(
|
||||
f"{path}: {which}.task names {stray}, which the top-level `task:` does not "
|
||||
f"declare ({sorted(task) or 'nothing'}). A task control is declared once, with "
|
||||
"its kind and what it does, and bound on each device by that name."
|
||||
)
|
||||
missing = [n for n in task if n not in block]
|
||||
if missing:
|
||||
raise ValueError(
|
||||
f"{path}: {which}.task leaves out {missing}. Each device has to be able to "
|
||||
"reach every control the task keeps on its own."
|
||||
)
|
||||
return dict(block)
|
||||
|
||||
|
||||
def _parse_keyboard(keyboard: dict, names: list[str], task: dict[str, TaskControl], path: Path):
|
||||
"""The keyboard: keystrokes on axis directions, the release and the task's
|
||||
controls. Every command axis appears, bound or said to be unbound."""
|
||||
if not isinstance(keyboard, dict):
|
||||
raise ValueError(f"{path}: devices.keyboard is a {type(keyboard).__name__}")
|
||||
if "step" in keyboard:
|
||||
raise ValueError(
|
||||
f"{path}: the keyboard has a `step`, the notched keyboard retired on "
|
||||
"2026-09-26 -- one press, a tenth of the travel, kept until pressed back. A "
|
||||
"key now pushes its direction for as long as it is held and is back at rest "
|
||||
"when it comes up: write `full_after_s`, the seconds held to full deflection."
|
||||
)
|
||||
if CENTRE in keyboard:
|
||||
raise ValueError(
|
||||
f"{path}: the keyboard has `{CENTRE}`, retired on 2026-09-27 with the notched "
|
||||
"keyboard it served: a key let up is back at rest, and the release lets go "
|
||||
"of everything. Delete it."
|
||||
)
|
||||
for field in ("scheme", "full_after_s", "axes", "release"):
|
||||
_require(keyboard, field, path)
|
||||
if keyboard["scheme"] != _KEYBOARD_SCHEME:
|
||||
raise ValueError(
|
||||
f"{path}: keyboard scheme is {keyboard['scheme']!r}. This file implements "
|
||||
f"`{_KEYBOARD_SCHEME}`: each keystroke bound to one direction of an axis, "
|
||||
"the release, or a task control."
|
||||
)
|
||||
full = keyboard["full_after_s"]
|
||||
if not _positive_seconds(full):
|
||||
raise ValueError(
|
||||
f"{path}: keyboard full_after_s is {full!r}; it is the seconds a key is held "
|
||||
"for its direction to reach full deflection, a positive number"
|
||||
)
|
||||
|
||||
# One keystroke, one thing: an axis direction or the release. The task's
|
||||
# auxiliary controls may share one with each other. Two tools
|
||||
# both close on Space, and a mode reads only its own side's -- but not with
|
||||
# an axis or the release, or one key would do both.
|
||||
owner: dict[str, str] = {}
|
||||
parsed: dict[str, tuple[str | None, str]] = {}
|
||||
|
||||
def claim(stroke: Any, what: str, where: str, exclusive: bool = True) -> str:
|
||||
parsed[stroke] = _check_stroke(stroke, where, path)
|
||||
if stroke in owner and (exclusive or not owner[stroke].startswith("task ")):
|
||||
raise ValueError(
|
||||
f"{path}: {stroke} is on {owner[stroke]} and on {what}. One keystroke "
|
||||
"would do both; give each its own."
|
||||
)
|
||||
if exclusive:
|
||||
owner[stroke] = what
|
||||
else:
|
||||
owner.setdefault(stroke, what)
|
||||
return stroke
|
||||
|
||||
def strokes(listed: Any, where: str) -> list:
|
||||
if not isinstance(listed, list) or not listed:
|
||||
raise ValueError(
|
||||
f"{path}: {where} is {listed!r}; it is a non-empty list of keystrokes"
|
||||
)
|
||||
return listed
|
||||
|
||||
declared = keyboard["axes"]
|
||||
if not isinstance(declared, dict):
|
||||
raise ValueError(
|
||||
f"{path}: keyboard.axes is a {type(declared).__name__}; it maps every command "
|
||||
"axis to its keystrokes"
|
||||
)
|
||||
stray = sorted(set(declared) - set(names))
|
||||
if stray:
|
||||
raise ValueError(
|
||||
f"{path}: keyboard.axes names {stray}, which no command has. A typo here is "
|
||||
f"a key that drives nothing. The axes are {names}"
|
||||
)
|
||||
key_axes: dict[str, KeyAxis] = {}
|
||||
for name in names:
|
||||
if name not in declared:
|
||||
raise ValueError(
|
||||
f"{path}: keyboard.axes leaves out {name}. Every command axis is on the "
|
||||
"keyboard or says why not (`unbound: <why>`): an axis quietly missing is "
|
||||
"a keyboard that cannot do something the pad can, and nothing says so."
|
||||
)
|
||||
entry = declared[name]
|
||||
if isinstance(entry, dict) and set(entry) == {"unbound"}:
|
||||
why = entry["unbound"]
|
||||
if not isinstance(why, str) or not why.strip():
|
||||
raise ValueError(
|
||||
f"{path}: keyboard.axes.{name} is unbound and does not say why"
|
||||
)
|
||||
key_axes[name] = KeyAxis(unbound=why)
|
||||
continue
|
||||
if not isinstance(entry, dict) or set(entry) != {"+", "-"}:
|
||||
raise ValueError(
|
||||
f"{path}: keyboard.axes.{name} is {entry!r}; it is {{\"+\": [...], \"-\": "
|
||||
"[...]}, the keystrokes towards the axis's positive direction and away "
|
||||
"from it, or {unbound: <why this axis has no keys>}"
|
||||
)
|
||||
plus = [claim(s, f"{name} +", f"keyboard.axes.{name}.+")
|
||||
for s in strokes(entry["+"], f"keyboard.axes.{name}.+")]
|
||||
minus = [claim(s, f"{name} -", f"keyboard.axes.{name}.-")
|
||||
for s in strokes(entry["-"], f"keyboard.axes.{name}.-")]
|
||||
key_axes[name] = KeyAxis(plus=tuple(plus), minus=tuple(minus))
|
||||
|
||||
release = tuple(claim(s, "the release", "keyboard.release")
|
||||
for s in strokes(keyboard["release"], "keyboard.release"))
|
||||
|
||||
key_task = {}
|
||||
for name, listed in _task_block(keyboard, "keyboard", task, path).items():
|
||||
where = f"keyboard.task.{name}"
|
||||
seen = [claim(s, f"task {name}", where, exclusive=False) for s in strokes(listed, where)]
|
||||
if len(set(seen)) != len(seen):
|
||||
raise ValueError(f"{path}: {where} lists a keystroke twice: {seen}")
|
||||
key_task[name] = tuple(seen)
|
||||
|
||||
# A modifier held on its own that also modifies a key: holding it for the
|
||||
# one would do the other while the hand reaches for the key. A key that is
|
||||
# one of the modifier's own keys (`key_left_shift`) held alone is the same.
|
||||
modifiers = _vocabulary.modifiers()
|
||||
chords = {m for m, _k in parsed.values() if m is not None}
|
||||
alone = {k for m, k in parsed.values() if m is None and k in modifiers}
|
||||
alone |= {m for m, keys in modifiers.items()
|
||||
for _m, k in parsed.values() if _m is None and k in keys}
|
||||
both = sorted(chords & alone)
|
||||
if both:
|
||||
raise ValueError(
|
||||
f"{path}: {', '.join(both)} is bound on its own and also modifies a key "
|
||||
f"({', '.join(s for s, (m, _k) in parsed.items() if m in both)}). Holding it "
|
||||
"for the one would do it while reaching for the other."
|
||||
)
|
||||
return full, key_axes, release, key_task
|
||||
|
||||
|
||||
def controller_contract(controls: Controls, env_cfg: Any) -> dict[str, Any]:
|
||||
"""The specification, shaped for an exported policy's contract.
|
||||
|
||||
A transcription of the file rather than a description of the code: what a
|
||||
consumer receives is what `play` read. `source_sha256` is carried so it can
|
||||
say which revision, and a runner that wants to diff two bundles has a key to
|
||||
do it with. `deploy/fsm/src/operator.rs` reads it, as
|
||||
`operator_controller/2`; the name is the schema's, and the version is the
|
||||
file's, so a reader built for anything else refuses it by name.
|
||||
|
||||
`task` is the file's top-level `task:`, kind and words, and `{}` for a task
|
||||
that keeps none: the controller refuses a hook reading a name it does not
|
||||
declare. Each device's own `task:` stays in its block, verbatim.
|
||||
|
||||
Args:
|
||||
env_cfg: the **environment** config, whose command terms hold the values
|
||||
the file names by field -- `rest: neutral_height` becomes the number.
|
||||
A consumer on the robot has no config to look a field up in, so the
|
||||
block carries the number, and the name beside it for a reader who
|
||||
wants to know where it came from. Required: a rest left as a name is
|
||||
a height the robot would read as zero, which is a body on the floor.
|
||||
|
||||
Called at export rather than when the config is built, because a task adds
|
||||
its other command terms after `velocity_env_cfg` returns -- `velocity_env_cfg`
|
||||
attaches this as a callable, the way a reference contract is attached.
|
||||
"""
|
||||
commands = []
|
||||
for c in controls.commands:
|
||||
term_cfg = (getattr(env_cfg, "commands", None) or {}).get(c.term)
|
||||
axes = []
|
||||
for i, a in enumerate(c.axes):
|
||||
entry = {"name": a.name, "index": i, "unit": a.unit, "positive": a.positive,
|
||||
"range_from": f"command_ranges.{c.term}.{a.name}"}
|
||||
if a.rest:
|
||||
if term_cfg is None or not hasattr(term_cfg, a.rest):
|
||||
raise ValueError(
|
||||
f"{controls.source}: {c.term}.{a.name} rests at `{a.rest}`, and the "
|
||||
f"environment config has no command term {c.term!r} carrying it. "
|
||||
"The block needs the number: a robot has no config to look it up in."
|
||||
)
|
||||
entry["rest"] = float(getattr(term_cfg, a.rest))
|
||||
entry["rest_from"] = a.rest
|
||||
if a.integrate_s is not None:
|
||||
entry["integrate_s"] = float(a.integrate_s)
|
||||
axes.append(entry)
|
||||
entry = {"term": c.term, "feeds": c.feeds, "frame": c.frame,
|
||||
"axes": axes, "scaling": dict(c.scaling)}
|
||||
entry.update(_command_shape(controls, c, term_cfg))
|
||||
commands.append(entry)
|
||||
return {
|
||||
"schema": f"operator_controller/{SCHEMA}",
|
||||
"source": {
|
||||
"file": controls.source.name,
|
||||
"sha256": controls.source_sha256,
|
||||
"note": "Parsed from the specification play reads. Comments and provenance "
|
||||
"live in the source file.",
|
||||
},
|
||||
"command": commands,
|
||||
"task": {t.name: {"kind": t.kind, "does": t.does} for t in controls.task.values()},
|
||||
"devices": {
|
||||
"gamepad": dict(controls.gamepad),
|
||||
"keyboard": dict(controls.keyboard),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _command_shape(controls: Controls, c: Command, term_cfg: Any) -> dict[str, Any]:
|
||||
"""What happens to a command between the stick and the policy, if anything.
|
||||
|
||||
Two things a command term can carry, and a consumer on the robot has no
|
||||
config to find either in, so the block carries the numbers:
|
||||
|
||||
* **`bands`** -- a narrower band some axes are held to while the velocity
|
||||
command walks. `command_ranges.<term>` is the standing band, the widest the
|
||||
command goes and what a full stick reaches; at or above `standing_below`,
|
||||
the norm of the velocity command's `[vx, vy, wz]`, each axis named in
|
||||
`moving` is clamped to it. Read off the term's `moving` and
|
||||
`stand_threshold`: a half-range or a `(low, high)` pair per axis.
|
||||
* **`max_rate`** -- how fast the command may move, in its own units per
|
||||
second, because the policy was trained on a command that ramps. Absent is
|
||||
a command that steps, which is what every other term does.
|
||||
|
||||
A term that carries neither gets neither, and the block is what it was. A
|
||||
`moving` without a threshold, or naming an axis the file does not, raises:
|
||||
a band nobody can apply is a clamp the robot silently skips.
|
||||
"""
|
||||
out: dict[str, Any] = {}
|
||||
moving = getattr(term_cfg, "moving", None)
|
||||
if moving is not None:
|
||||
threshold = getattr(term_cfg, "stand_threshold", None)
|
||||
velocity = getattr(term_cfg, "velocity_command", None) or getattr(
|
||||
term_cfg, "velocity_command_name", None
|
||||
)
|
||||
if threshold is None or velocity is None:
|
||||
raise ValueError(
|
||||
f"{controls.source}: {c.term} has a moving band and no "
|
||||
"`stand_threshold` / velocity command to decide when it applies"
|
||||
)
|
||||
names = {a.name for a in c.axes}
|
||||
band = {}
|
||||
for name, value in vars(moving).items():
|
||||
if name.startswith("_"):
|
||||
continue
|
||||
if name not in names:
|
||||
raise ValueError(
|
||||
f"{controls.source}: {c.term}'s moving band names {name!r}, "
|
||||
f"which is not one of the file's axes {sorted(names)}"
|
||||
)
|
||||
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
||||
band[name] = [-float(value), float(value)]
|
||||
else:
|
||||
low, high = value
|
||||
band[name] = [float(low), float(high)]
|
||||
out["bands"] = {"standing_below": float(threshold), "velocity_term": str(velocity),
|
||||
"moving": band}
|
||||
max_rate = getattr(term_cfg, "max_rate", None)
|
||||
if max_rate is not None:
|
||||
out["max_rate"] = float(max_rate)
|
||||
return out
|
||||
|
||||
|
||||
def key_name(code: int) -> str:
|
||||
"""A GLFW keycode as something a person can read off a banner.
|
||||
|
||||
A dictionary key is shown by its own name less the `key_` prefix -- `W`,
|
||||
`KP8` for `keypad_8` -- and anything else falls back to the number, which
|
||||
is what a handler would need to remap it.
|
||||
"""
|
||||
for name, codes in _vocabulary.key_codes().items():
|
||||
if codes["glfw"] == code:
|
||||
if name.startswith("keypad_"):
|
||||
return "KP" + name[len("keypad_"):]
|
||||
return name.removeprefix("key_").upper()
|
||||
return str(code)
|
||||
|
||||
|
||||
def stroke_name(stroke: str) -> str:
|
||||
"""A keystroke as a banner shows it: `Shift+J`, `Ctrl`, `SPACE`, `KP1`."""
|
||||
modifier, key = _vocabulary.keystroke(stroke)
|
||||
shown = (key.capitalize() if key in _vocabulary.modifiers()
|
||||
else key_name(_vocabulary.key_codes()[key]["glfw"]))
|
||||
return f"{modifier.capitalize()}+{shown}" if modifier else shown
|
||||
@@ -0,0 +1,292 @@
|
||||
"""Finding and reading a Linux evdev gamepad, with no dependencies.
|
||||
|
||||
## Why evdev and not joydev
|
||||
|
||||
`/dev/input/js*` is simpler to parse, and on this machine it does not exist: the
|
||||
`joydev` module is not loaded, while `event*` is always there. evdev is also the
|
||||
one that carries the axis ranges, so a trigger that reports 0..1023 on one pad and
|
||||
0..255 on another normalises correctly without a table of models.
|
||||
|
||||
## The wire format
|
||||
|
||||
An evdev event is `struct input_event`: a `timeval` then type, code and value.
|
||||
On 64-bit Linux that is `llHHi`, 24 bytes, which is asserted at import rather
|
||||
than assumed -- a wrong size here would not raise, it would silently decode
|
||||
garbage as button presses.
|
||||
|
||||
Ranges come from the `EVIOCGABS` ioctl, one per axis, read once at open. The ioctl
|
||||
numbers are built with the same arithmetic as the kernel's `_IOR` macro rather
|
||||
than pasted as magic constants, so the derivation is visible.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import array
|
||||
import glob
|
||||
import os
|
||||
import select
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
try:
|
||||
import fcntl
|
||||
except ImportError: # Windows. There is no evdev to read there and no ioctl
|
||||
# to read it with; pads there go through `xinput.py`, and nothing in this
|
||||
# module is called. The import still happens, so it must not take the whole
|
||||
# package down with it: the tasks import `controller` for its vocabulary, and
|
||||
# that is on the path of every training run.
|
||||
fcntl = None
|
||||
|
||||
from .xbox import ABS_AXES, GamepadState, normalise
|
||||
|
||||
__all__ = ["Gamepad", "GamepadState", "PadInfo", "describe", "find_pads"]
|
||||
|
||||
#: `struct input_event` on 64-bit Linux: struct timeval (two longs), then
|
||||
#: __u16 type, __u16 code, __s32 value.
|
||||
_EVENT_FORMAT = "llHHi"
|
||||
_EVENT_SIZE = struct.calcsize(_EVENT_FORMAT)
|
||||
# Asserted rather than assumed, because a wrong size here would not raise -- it
|
||||
# would decode the stream misaligned and report button presses that never
|
||||
# happened. **Only on Linux, which is what the 24 bytes describe**: Windows'
|
||||
# C `long` is 4 bytes, so the same format measures 16 there and would fail an
|
||||
# assertion about a layout that platform never uses.
|
||||
if sys.platform == "linux":
|
||||
assert _EVENT_SIZE == 24, (
|
||||
f"struct input_event is {_EVENT_SIZE} bytes here, not 24. Decoding would "
|
||||
f"silently produce nonsense rather than fail, so this is checked."
|
||||
)
|
||||
|
||||
EV_KEY = 0x01
|
||||
EV_ABS = 0x03
|
||||
|
||||
|
||||
def _ioc(direction: int, type_: int, nr: int, size: int) -> int:
|
||||
"""The kernel's `_IOC` macro. dir 2 is `_IOC_READ`."""
|
||||
return (direction << 30) | (size << 16) | (type_ << 8) | nr
|
||||
|
||||
|
||||
def _eviocgname(length: int) -> int:
|
||||
return _ioc(2, ord("E"), 0x06, length)
|
||||
|
||||
|
||||
def _eviocgabs(axis: int) -> int:
|
||||
#: `struct input_absinfo` is six __s32: value, min, max, fuzz, flat, resolution.
|
||||
return _ioc(2, ord("E"), 0x40 + axis, 24)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PadInfo:
|
||||
"""A candidate device and what was decided about it."""
|
||||
|
||||
path: str
|
||||
name: str
|
||||
readable: bool
|
||||
reason: str = ""
|
||||
|
||||
|
||||
def _read_name(fd: int) -> str:
|
||||
buf = array.array("B", b"\x00" * 256)
|
||||
try:
|
||||
fcntl.ioctl(fd, _eviocgname(256), buf)
|
||||
except OSError:
|
||||
return ""
|
||||
return bytes(buf).split(b"\x00", 1)[0].decode("utf-8", "replace")
|
||||
|
||||
|
||||
def _read_ranges(fd: int) -> dict[int, tuple[int, int, int]]:
|
||||
"""Per-axis (min, max, flat) from EVIOCGABS. `flat` is the driver's own idea
|
||||
of the dead zone, which is better than a number invented here."""
|
||||
ranges: dict[int, tuple[int, int, int]] = {}
|
||||
for axis in ABS_AXES:
|
||||
buf = array.array("i", [0] * 6)
|
||||
try:
|
||||
fcntl.ioctl(fd, _eviocgabs(axis), buf)
|
||||
except OSError:
|
||||
continue # the pad does not have this axis
|
||||
_value, lo, hi, _fuzz, flat, _res = buf
|
||||
if hi > lo:
|
||||
ranges[axis] = (lo, hi, flat)
|
||||
return ranges
|
||||
|
||||
|
||||
#: Buttons whose presence marks a device as something to be played with, from
|
||||
#: udev's own `input_id` rule.
|
||||
BTN_JOYSTICK = 0x120
|
||||
BTN_GAMEPAD = 0x130
|
||||
|
||||
|
||||
def _bitmap(path: str) -> set[int]:
|
||||
"""Parse a sysfs capability bitmap into the set of bits it has.
|
||||
|
||||
The kernel prints these as space-separated 64-bit hex words, **most
|
||||
significant first**, so the last word is bits 0-63. Reading them left to
|
||||
right, which is the obvious way, gets every bit index wrong.
|
||||
"""
|
||||
try:
|
||||
with open(path) as f:
|
||||
words = f.read().split()
|
||||
except OSError:
|
||||
return set()
|
||||
bits: set[int] = set()
|
||||
for i, word in enumerate(reversed(words)):
|
||||
value = int(word, 16)
|
||||
base = i * 64
|
||||
while value:
|
||||
low = value & -value
|
||||
bits.add(base + low.bit_length() - 1)
|
||||
value ^= low
|
||||
return bits
|
||||
|
||||
|
||||
def find_pads(readable_only: bool = False) -> list[PadInfo]:
|
||||
"""Every gamepad-looking device, in a stable order.
|
||||
|
||||
Discovery is through `/sys/class/input/event*`, applying udev's own rule --
|
||||
the device has absolute axes and at least one gamepad or joystick button.
|
||||
|
||||
**Not through `/dev/input/by-id/*-event-joystick`**, which is what this did
|
||||
first and which silently finds nothing over Bluetooth: those symlinks are
|
||||
generated from USB and serial identifiers, and a Bluetooth pad has neither.
|
||||
Measured here, an Xbox Wireless Controller paired over Bluetooth appears as
|
||||
`/dev/input/event17` with no `by-id` entry at all, so a working, connected
|
||||
controller read as absent.
|
||||
|
||||
sysfs also answers without opening anything, so a pad the process cannot read
|
||||
is still *found*, and can be reported as a permissions problem rather than as
|
||||
nothing at all.
|
||||
"""
|
||||
pads: list[PadInfo] = []
|
||||
for sysdir in sorted(
|
||||
glob.glob("/sys/class/input/event*"),
|
||||
key=lambda p: int(os.path.basename(p)[5:] or 0),
|
||||
):
|
||||
device = os.path.join(sysdir, "device")
|
||||
caps = os.path.join(device, "capabilities")
|
||||
ev = _bitmap(os.path.join(caps, "ev"))
|
||||
keys = _bitmap(os.path.join(caps, "key"))
|
||||
if EV_ABS not in ev:
|
||||
continue
|
||||
if not (BTN_GAMEPAD in keys or BTN_JOYSTICK in keys):
|
||||
continue
|
||||
|
||||
path = os.path.join("/dev/input", os.path.basename(sysdir))
|
||||
try:
|
||||
with open(os.path.join(device, "name")) as f:
|
||||
name = f.read().strip()
|
||||
except OSError:
|
||||
name = os.path.basename(path)
|
||||
|
||||
try:
|
||||
fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
|
||||
except PermissionError:
|
||||
pads.append(
|
||||
PadInfo(path, name, False,
|
||||
"permission denied -- add yourself to the 'input' group "
|
||||
"(sudo usermod -aG input \"$USER\", then log out and back "
|
||||
"in), or log in on the seat so logind grants the ACL")
|
||||
)
|
||||
continue
|
||||
except OSError as exc:
|
||||
pads.append(PadInfo(path, name, False, str(exc)))
|
||||
continue
|
||||
try:
|
||||
pads.append(PadInfo(path, _read_name(fd) or name, True))
|
||||
finally:
|
||||
os.close(fd)
|
||||
if readable_only:
|
||||
return [p for p in pads if p.readable]
|
||||
return pads
|
||||
|
||||
|
||||
def describe() -> str:
|
||||
"""What was found and, for anything rejected, why. `controller.describe()` is
|
||||
the entry point; this is its Linux half."""
|
||||
pads = find_pads()
|
||||
if not pads:
|
||||
return (
|
||||
"no input device has both absolute axes and a gamepad button. "
|
||||
"Nothing is paired, or it connected as something other than a pad."
|
||||
)
|
||||
lines = []
|
||||
for p in pads:
|
||||
state = "usable" if p.readable else f"unusable: {p.reason}"
|
||||
lines.append(f" {p.path} {p.name} -- {state}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Gamepad:
|
||||
"""A pad being read on a background thread.
|
||||
|
||||
The reader thread only ever writes raw integers into `_raw`; normalisation
|
||||
happens in `state()`, on the caller's thread. That keeps the axis ranges and
|
||||
the dead zone out of the hot path and, more usefully, means a caller polling
|
||||
at 50 Hz sees a consistent snapshot rather than a half-updated one.
|
||||
"""
|
||||
|
||||
info: PadInfo
|
||||
_raw: dict[tuple[int, int], int] = field(default_factory=dict)
|
||||
_ranges: dict[int, tuple[int, int, int]] = field(default_factory=dict)
|
||||
_fd: int = -1
|
||||
_thread: threading.Thread | None = None
|
||||
_stop: threading.Event = field(default_factory=threading.Event)
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return self.info.name
|
||||
|
||||
def start(self) -> None:
|
||||
self._fd = os.open(self.info.path, os.O_RDONLY | os.O_NONBLOCK)
|
||||
self._ranges = _read_ranges(self._fd)
|
||||
self._thread = threading.Thread(
|
||||
target=self._run, name="gamepad", daemon=True
|
||||
)
|
||||
self._thread.start()
|
||||
|
||||
def close(self) -> None:
|
||||
self._stop.set()
|
||||
if self._thread is not None:
|
||||
self._thread.join(timeout=1.0)
|
||||
if self._fd >= 0:
|
||||
os.close(self._fd)
|
||||
self._fd = -1
|
||||
|
||||
def _run(self) -> None:
|
||||
"""Poll rather than block on read, so `close()` is not left waiting for a
|
||||
stick that nobody is touching."""
|
||||
while not self._stop.is_set():
|
||||
ready, _, _ = select.select([self._fd], [], [], 0.1)
|
||||
if not ready:
|
||||
continue
|
||||
try:
|
||||
data = os.read(self._fd, _EVENT_SIZE * 64)
|
||||
except OSError:
|
||||
# The pad went away mid-read: unplugged, or Bluetooth dropped.
|
||||
# Leave the last state in place and stop; state() keeps working
|
||||
# and `connected` goes False.
|
||||
break
|
||||
for i in range(0, len(data) - _EVENT_SIZE + 1, _EVENT_SIZE):
|
||||
_s, _us, etype, code, value = struct.unpack_from(
|
||||
_EVENT_FORMAT, data, i
|
||||
)
|
||||
if etype in (EV_KEY, EV_ABS):
|
||||
self._raw[(etype, code)] = value
|
||||
self._stop.set()
|
||||
|
||||
@property
|
||||
def connected(self) -> bool:
|
||||
return self._thread is not None and self._thread.is_alive()
|
||||
|
||||
def state(self) -> GamepadState:
|
||||
"""A normalised snapshot. Never blocks and never raises."""
|
||||
return normalise(dict(self._raw), self._ranges)
|
||||
|
||||
def raw(self) -> dict[tuple[int, int], int]:
|
||||
"""Everything seen so far, unnormalised, as `{(type, code): value}`.
|
||||
|
||||
For working out a pad whose mapping does not match `xbox.py` -- press the
|
||||
button, see which code moved.
|
||||
"""
|
||||
return dict(self._raw)
|
||||
@@ -0,0 +1,192 @@
|
||||
"""Map pad deflections and keyboard edges into policy command inputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable, Mapping
|
||||
from typing import Any
|
||||
|
||||
from . import vocabulary as _vocabulary
|
||||
from .controls import Command, Controls, KeyAxis
|
||||
|
||||
__all__ = ["Keyboard", "deflections", "scale", "spans"]
|
||||
|
||||
|
||||
# ── The pad's mapping ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _band(s: float, travel: tuple[float, ...]) -> float:
|
||||
"""A signed deflection through one binding's stretch of travel.
|
||||
|
||||
Zero until `from`, full at `to` and held there beyond it. So `(0.0, 0.5)`
|
||||
is full at half travel, and `(0.5, 1.0)` is silent until half travel. With
|
||||
a fall, `(0.0, 0.5, 0.5, 0.75)`, it comes back from full at `back` to zero
|
||||
at `off` and stays there.
|
||||
"""
|
||||
lo, hi, *fall = travel
|
||||
a = abs(s)
|
||||
x = min(max((a - lo) / (hi - lo), 0.0), 1.0)
|
||||
if fall:
|
||||
back, off = fall
|
||||
x *= min(max((off - a) / (off - back), 0.0), 1.0)
|
||||
return x if s >= 0.0 else -x
|
||||
|
||||
|
||||
def deflections(
|
||||
values: Mapping[str, float], shifted: bool, controls: Controls
|
||||
) -> dict[str, float]:
|
||||
"""Pad control positions -> each command axis's deflection, -1..1.
|
||||
|
||||
`values` is keyed by the dictionary's names (`Lx` ... `RT`). A source that
|
||||
has a shifted binding drives **only** that binding while shifted; every
|
||||
other source carries on as it was. The pad's path only: the keyboard has
|
||||
bindings of its own (`Keyboard`).
|
||||
"""
|
||||
layered = {b.source for parts in controls.bindings.values() for b in parts if b.shifted}
|
||||
out = {}
|
||||
for axis, parts in controls.bindings.items():
|
||||
x = 0.0
|
||||
for b in parts:
|
||||
live = shifted if b.shifted else not (shifted and b.source in layered)
|
||||
if live:
|
||||
x += _band(b.sign * values[b.source], b.travel)
|
||||
out[axis] = max(-1.0, min(1.0, x))
|
||||
return out
|
||||
|
||||
|
||||
def scale(x: float, span: tuple[float, float, float]) -> float:
|
||||
"""`full_deflection_is_range_edge`, measured from the axis's rest.
|
||||
|
||||
The two ends are scaled separately. A range like (-0.5, 1.0) gives forward
|
||||
twice what it gives back, which is what the policy trained on; taking the
|
||||
larger end for both quietly commands something it never saw.
|
||||
"""
|
||||
lo, hi, rest = span
|
||||
return rest + x * ((hi - rest) if x >= 0.0 else (rest - lo))
|
||||
|
||||
|
||||
def spans(command: Command, cfg: Any) -> dict[str, tuple[float, float, float]]:
|
||||
"""`(low, high, rest)` per axis, from a live command config.
|
||||
|
||||
A range is either a `(low, high)` pair, as the velocity command has, or a
|
||||
half-range, as the posture command's angles are. The rest is the config
|
||||
field the file names, or zero.
|
||||
"""
|
||||
out = {}
|
||||
for a in command.axes:
|
||||
r = getattr(cfg.ranges, a.name)
|
||||
lo, hi = (-float(r), float(r)) if isinstance(r, (int, float)) else map(float, r)
|
||||
rest = float(getattr(cfg, a.rest)) if a.rest else 0.0
|
||||
if not lo <= rest <= hi:
|
||||
raise ValueError(
|
||||
f"{command.term}.{a.name} rests at {rest:g}, outside its range "
|
||||
f"({lo:g}, {hi:g}). A centred stick would command a value the range "
|
||||
"clamps away -- name the rest in the controls file (`rest: <field>`)."
|
||||
)
|
||||
out[a.name] = (lo, hi, rest)
|
||||
return out
|
||||
|
||||
|
||||
# ── The keyboard's path ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class Keyboard:
|
||||
"""The keyboard's half of the operator: which keystrokes are live, since when.
|
||||
|
||||
Fed a key's two edges by the dictionary's name, and asked what an axis or a
|
||||
task control reads. A keystroke held for `full_after_s` is full deflection
|
||||
and half that is half; the moment it stops being live it reads nothing.
|
||||
Several live on one direction are summed, as two keys on one direction of a
|
||||
stick would push it together, and the two directions of an axis cancel.
|
||||
"""
|
||||
|
||||
def __init__(self, controls: Controls):
|
||||
self._full = float(controls.full_after_s)
|
||||
modifiers = _vocabulary.modifiers()
|
||||
#: Keystroke -> (the modifier held with it or `None`, the key or the
|
||||
#: modifier it is), every keystroke the file's keyboard block binds.
|
||||
self._strokes = {s: _vocabulary.keystroke(s) for s in controls.keystrokes}
|
||||
#: Modifier -> its two keys, for the modifiers this block names at all.
|
||||
self._modifiers = {
|
||||
m: keys for m, keys in modifiers.items()
|
||||
if any(m in (mod, key) for mod, key in self._strokes.values())
|
||||
}
|
||||
#: Key -> the modifiers bound in front of it here. While one of them is
|
||||
#: down, the key alone is dead: Shift + J is not J as well.
|
||||
self._chorded: dict[str, frozenset[str]] = {}
|
||||
for mod, key in self._strokes.values():
|
||||
if mod is not None:
|
||||
self._chorded[key] = self._chorded.get(key, frozenset()) | {mod}
|
||||
#: Every dictionary key this block reads: each keystroke's key, and both
|
||||
#: keys of each modifier it names. Any other key is not this file's.
|
||||
self.watched = frozenset(
|
||||
{key for _mod, key in self._strokes.values() if key not in modifiers}
|
||||
| {k for pair in self._modifiers.values() for k in pair}
|
||||
)
|
||||
#: Keys down, as their edges report them.
|
||||
self._down: set[str] = set()
|
||||
#: Keys down and pressed since the last `drop`: what drives.
|
||||
self._fresh: set[str] = set()
|
||||
#: Live keystroke -> when it last became live, on the operator's clock.
|
||||
self._live: dict[str, float] = {}
|
||||
|
||||
def press(self, key: str, now: float) -> set[str] | None:
|
||||
"""`key` went down at `now`. The keystrokes that became live, or `None`
|
||||
for a key already down -- a repeat, which is the same hold."""
|
||||
if key in self._down:
|
||||
return None
|
||||
self._down.add(key)
|
||||
self._fresh.add(key)
|
||||
return self._refresh(now)
|
||||
|
||||
def lift(self, key: str, now: float) -> set[str]:
|
||||
"""`key` came up at `now`. The keystrokes that became live: a key whose
|
||||
modifier was let go of is itself again."""
|
||||
self._down.discard(key)
|
||||
self._fresh.discard(key)
|
||||
return self._refresh(now)
|
||||
|
||||
def drop(self) -> None:
|
||||
"""Hands off: nothing is live, and a key still down drives nothing --
|
||||
as a key or as a modifier -- until it is pressed again."""
|
||||
self._fresh.clear()
|
||||
self._live.clear()
|
||||
|
||||
def _refresh(self, now: float) -> set[str]:
|
||||
held = {m for m, pair in self._modifiers.items() if self._fresh.intersection(pair)}
|
||||
live = set()
|
||||
for stroke, (mod, key) in self._strokes.items():
|
||||
if mod is not None:
|
||||
on = key in self._fresh and mod in held
|
||||
elif key in self._modifiers:
|
||||
on = key in held
|
||||
else:
|
||||
on = key in self._fresh and not (self._chorded.get(key, frozenset()) & held)
|
||||
if on:
|
||||
live.add(stroke)
|
||||
became = live - self._live.keys()
|
||||
self._live = {s: self._live.get(s, now) for s in live}
|
||||
return became
|
||||
|
||||
@property
|
||||
def live(self) -> frozenset[str]:
|
||||
return frozenset(self._live)
|
||||
|
||||
def ramp(self, stroke: str, now: float) -> float:
|
||||
"""0..1: how long `stroke` has been live, over `full_after_s`."""
|
||||
since = self._live.get(stroke)
|
||||
if since is None:
|
||||
return 0.0
|
||||
return min(1.0, max(0.0, (now - since) / self._full))
|
||||
|
||||
def deflection(self, axis: KeyAxis, now: float) -> float:
|
||||
"""One command axis, -1..1: its `+` keystrokes less its `-` ones."""
|
||||
x = sum(self.ramp(s, now) for s in axis.plus) - sum(self.ramp(s, now) for s in axis.minus)
|
||||
return max(-1.0, min(1.0, x))
|
||||
|
||||
def amount(self, strokes: Iterable[str], now: float) -> float:
|
||||
"""A task's `amount`: the furthest of its keystrokes' ramps."""
|
||||
return max((self.ramp(s, now) for s in strokes), default=0.0)
|
||||
|
||||
def pressed(self, strokes: Iterable[str]) -> bool:
|
||||
"""A task's `press`: any of its keystrokes live, at once and in full."""
|
||||
return any(s in self._live for s in strokes)
|
||||
@@ -0,0 +1,499 @@
|
||||
{
|
||||
"schema": "kk-control-vocabulary/2",
|
||||
"_what_this_is": [
|
||||
"Everything a pad or a keyboard can say, and nothing about what any of it",
|
||||
"means.",
|
||||
"",
|
||||
"That split is this package's own rule -- it reports that the left stick is at",
|
||||
"(0.4, -0.9), that A is down and that keypad_8 is down, and deciding that any",
|
||||
"of that is 0.2 m/s forward belongs to whoever is driving. So the words live",
|
||||
"here and the meanings live with the consumer: `[[fsm.button]]` in a deploy",
|
||||
"manifest, `controls.yaml` in a task.",
|
||||
"",
|
||||
"Read by three programs and written by none of them:",
|
||||
" controller/vocabulary.py python -m controller --vocabulary",
|
||||
" deploy/fsm/src/vocabulary.rs include_str!, so a board carries no file",
|
||||
" tasks/.../mdp/controls.py validating a task's controls.yaml",
|
||||
"",
|
||||
"It had a copy in each of those for a day. Adding a word meant remembering two",
|
||||
"other files, and forgetting one leaves a name that reads as supported and is",
|
||||
"refused on build -- or, worse, the other way round.",
|
||||
"",
|
||||
"Was `kk-gamepad-vocabulary/1`, when the pad was the only thing it described.",
|
||||
"The keys below were a closed list too and were not written down anywhere a",
|
||||
"consumer could check, so `[[fsm.button]] key = ...` took any string at all: a",
|
||||
"typo loaded, bound nothing, and looked exactly like a rule that had not fired."
|
||||
],
|
||||
"_source": {
|
||||
"service": "control-pod-svc",
|
||||
"repo": "robot/embedded-software/soc/apps/control-pod-svc",
|
||||
"read_from": [
|
||||
"include/gamepad/gamepad_state.hpp",
|
||||
"src/gamepad/gamepad_dds_message.cpp"
|
||||
],
|
||||
"topic": "RobotControlRaw_ControlRaw",
|
||||
"note": "It normalises three different pads onto these names, so they mean the same thing on a bench and on the robot. Its own docs put the count at 18: 10 buttons + 2 dpad axes + 6 axes."
|
||||
},
|
||||
"buttons": [
|
||||
{
|
||||
"name": "A"
|
||||
},
|
||||
{
|
||||
"name": "B"
|
||||
},
|
||||
{
|
||||
"name": "X",
|
||||
"unresolved": "controller/xbox.py reads evdev 307 as `y` and 308 as `x`; the service's table reads 307 as X and 308 as Y. One is wrong about which physical button it is and pressing it is the only way to find out which."
|
||||
},
|
||||
{
|
||||
"name": "Y",
|
||||
"unresolved": "see X"
|
||||
},
|
||||
{
|
||||
"name": "LB"
|
||||
},
|
||||
{
|
||||
"name": "RB"
|
||||
},
|
||||
{
|
||||
"name": "menu"
|
||||
},
|
||||
{
|
||||
"name": "home",
|
||||
"note": "`xbox` in controller/xbox.py. Same button, two names."
|
||||
},
|
||||
{
|
||||
"name": "L3"
|
||||
},
|
||||
{
|
||||
"name": "R3"
|
||||
}
|
||||
],
|
||||
"dpad": [
|
||||
{
|
||||
"name": "dpad_up"
|
||||
},
|
||||
{
|
||||
"name": "dpad_down"
|
||||
},
|
||||
{
|
||||
"name": "dpad_left"
|
||||
},
|
||||
{
|
||||
"name": "dpad_right"
|
||||
}
|
||||
],
|
||||
"_dpad_note": "The wire carries dpad_x / dpad_y as -1 / 0 / +1, which no gesture can read, so they are offered as four booleans. The service negates dpad_y so +1 is up; it does not negate Ly, two lines away in the same function.",
|
||||
"axes": [
|
||||
{
|
||||
"name": "Lx",
|
||||
"range": [
|
||||
-1,
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Ly",
|
||||
"range": [
|
||||
-1,
|
||||
1
|
||||
],
|
||||
"note": "Not negated by the service. Forward is the negative end, by evdev's convention -- stated in controller/xbox.py and never measured against this service."
|
||||
},
|
||||
{
|
||||
"name": "Rx",
|
||||
"range": [
|
||||
-1,
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Ry",
|
||||
"range": [
|
||||
-1,
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "LT",
|
||||
"range": [
|
||||
0,
|
||||
1
|
||||
],
|
||||
"note": "Unipolar: a trigger never reports negative."
|
||||
},
|
||||
{
|
||||
"name": "RT",
|
||||
"range": [
|
||||
0,
|
||||
1
|
||||
]
|
||||
}
|
||||
],
|
||||
"_axes_note": "Sticks carry a 0.15 deadband that is already applied **and rescaled**, so full deflection still reaches ±1. A consumer that applies its own on top narrows what a person can ask for and cannot see that it has.",
|
||||
"keys": [
|
||||
{
|
||||
"name": "keypad_0",
|
||||
"glfw": 320,
|
||||
"browser": "Numpad0"
|
||||
},
|
||||
{
|
||||
"name": "keypad_1",
|
||||
"glfw": 321,
|
||||
"browser": "Numpad1"
|
||||
},
|
||||
{
|
||||
"name": "keypad_2",
|
||||
"glfw": 322,
|
||||
"browser": "Numpad2"
|
||||
},
|
||||
{
|
||||
"name": "keypad_3",
|
||||
"glfw": 323,
|
||||
"browser": "Numpad3"
|
||||
},
|
||||
{
|
||||
"name": "keypad_4",
|
||||
"glfw": 324,
|
||||
"browser": "Numpad4"
|
||||
},
|
||||
{
|
||||
"name": "keypad_5",
|
||||
"glfw": 325,
|
||||
"browser": "Numpad5"
|
||||
},
|
||||
{
|
||||
"name": "keypad_6",
|
||||
"glfw": 326,
|
||||
"browser": "Numpad6"
|
||||
},
|
||||
{
|
||||
"name": "keypad_7",
|
||||
"glfw": 327,
|
||||
"browser": "Numpad7"
|
||||
},
|
||||
{
|
||||
"name": "keypad_8",
|
||||
"glfw": 328,
|
||||
"browser": "Numpad8"
|
||||
},
|
||||
{
|
||||
"name": "keypad_9",
|
||||
"glfw": 329,
|
||||
"browser": "Numpad9"
|
||||
},
|
||||
{
|
||||
"name": "keypad_.",
|
||||
"glfw": 330,
|
||||
"browser": "NumpadDecimal"
|
||||
},
|
||||
{
|
||||
"name": "keypad_/",
|
||||
"glfw": 331,
|
||||
"browser": "NumpadDivide"
|
||||
},
|
||||
{
|
||||
"name": "keypad_*",
|
||||
"glfw": 332,
|
||||
"browser": "NumpadMultiply"
|
||||
},
|
||||
{
|
||||
"name": "keypad_-",
|
||||
"glfw": 333,
|
||||
"browser": "NumpadSubtract"
|
||||
},
|
||||
{
|
||||
"name": "keypad_+",
|
||||
"glfw": 334,
|
||||
"browser": "NumpadAdd"
|
||||
},
|
||||
{
|
||||
"name": "keypad_enter",
|
||||
"glfw": 335,
|
||||
"browser": "NumpadEnter"
|
||||
},
|
||||
{
|
||||
"name": "key_w",
|
||||
"glfw": 87,
|
||||
"browser": "KeyW"
|
||||
},
|
||||
{
|
||||
"name": "key_s",
|
||||
"glfw": 83,
|
||||
"browser": "KeyS"
|
||||
},
|
||||
{
|
||||
"name": "key_a",
|
||||
"glfw": 65,
|
||||
"browser": "KeyA"
|
||||
},
|
||||
{
|
||||
"name": "key_d",
|
||||
"glfw": 68,
|
||||
"browser": "KeyD"
|
||||
},
|
||||
{
|
||||
"name": "key_j",
|
||||
"glfw": 74,
|
||||
"browser": "KeyJ"
|
||||
},
|
||||
{
|
||||
"name": "key_l",
|
||||
"glfw": 76,
|
||||
"browser": "KeyL"
|
||||
},
|
||||
{
|
||||
"name": "key_i",
|
||||
"glfw": 73,
|
||||
"browser": "KeyI"
|
||||
},
|
||||
{
|
||||
"name": "key_k",
|
||||
"glfw": 75,
|
||||
"browser": "KeyK"
|
||||
},
|
||||
{
|
||||
"name": "key_m",
|
||||
"glfw": 77,
|
||||
"browser": "KeyM"
|
||||
},
|
||||
{
|
||||
"name": "key_o",
|
||||
"glfw": 79,
|
||||
"browser": "KeyO"
|
||||
},
|
||||
{
|
||||
"name": "key_q",
|
||||
"glfw": 81,
|
||||
"browser": "KeyQ"
|
||||
},
|
||||
{
|
||||
"name": "key_h",
|
||||
"glfw": 72,
|
||||
"browser": "KeyH"
|
||||
},
|
||||
{
|
||||
"name": "key_b",
|
||||
"glfw": 66,
|
||||
"browser": "KeyB"
|
||||
},
|
||||
{
|
||||
"name": "key_g",
|
||||
"glfw": 71,
|
||||
"browser": "KeyG"
|
||||
},
|
||||
{
|
||||
"name": "key_space",
|
||||
"glfw": 32,
|
||||
"browser": "Space"
|
||||
},
|
||||
{
|
||||
"name": "key_right",
|
||||
"glfw": 262,
|
||||
"browser": "ArrowRight"
|
||||
},
|
||||
{
|
||||
"name": "key_left",
|
||||
"glfw": 263,
|
||||
"browser": "ArrowLeft"
|
||||
},
|
||||
{
|
||||
"name": "key_down",
|
||||
"glfw": 264,
|
||||
"browser": "ArrowDown"
|
||||
},
|
||||
{
|
||||
"name": "key_up",
|
||||
"glfw": 265,
|
||||
"browser": "ArrowUp"
|
||||
},
|
||||
{
|
||||
"name": "key_left_ctrl",
|
||||
"glfw": 341,
|
||||
"browser": "ControlLeft"
|
||||
},
|
||||
{
|
||||
"name": "key_right_ctrl",
|
||||
"glfw": 345,
|
||||
"browser": "ControlRight"
|
||||
},
|
||||
{
|
||||
"name": "key_1",
|
||||
"glfw": 49,
|
||||
"browser": "Digit1"
|
||||
},
|
||||
{
|
||||
"name": "key_2",
|
||||
"glfw": 50,
|
||||
"browser": "Digit2"
|
||||
},
|
||||
{
|
||||
"name": "key_3",
|
||||
"glfw": 51,
|
||||
"browser": "Digit3"
|
||||
},
|
||||
{
|
||||
"name": "key_4",
|
||||
"glfw": 52,
|
||||
"browser": "Digit4"
|
||||
},
|
||||
{
|
||||
"name": "key_n",
|
||||
"glfw": 78,
|
||||
"browser": "KeyN"
|
||||
},
|
||||
{
|
||||
"name": "key_u",
|
||||
"glfw": 85,
|
||||
"browser": "KeyU"
|
||||
},
|
||||
{
|
||||
"name": "key_v",
|
||||
"glfw": 86,
|
||||
"browser": "KeyV"
|
||||
},
|
||||
{
|
||||
"name": "key_semicolon",
|
||||
"glfw": 59,
|
||||
"browser": "Semicolon"
|
||||
},
|
||||
{
|
||||
"name": "key_escape",
|
||||
"glfw": 256,
|
||||
"browser": "Escape"
|
||||
},
|
||||
{
|
||||
"name": "key_left_shift",
|
||||
"glfw": 340,
|
||||
"browser": "ShiftLeft"
|
||||
},
|
||||
{
|
||||
"name": "key_right_shift",
|
||||
"glfw": 344,
|
||||
"browser": "ShiftRight"
|
||||
},
|
||||
{
|
||||
"name": "key_left_alt",
|
||||
"glfw": 342,
|
||||
"browser": "AltLeft"
|
||||
},
|
||||
{
|
||||
"name": "key_right_alt",
|
||||
"glfw": 346,
|
||||
"browser": "AltRight"
|
||||
}
|
||||
],
|
||||
"modifiers": [
|
||||
{
|
||||
"name": "ctrl",
|
||||
"keys": ["key_left_ctrl", "key_right_ctrl"]
|
||||
},
|
||||
{
|
||||
"name": "shift",
|
||||
"keys": ["key_left_shift", "key_right_shift"]
|
||||
},
|
||||
{
|
||||
"name": "alt",
|
||||
"keys": ["key_left_alt", "key_right_alt"]
|
||||
}
|
||||
],
|
||||
"_modifiers_note": [
|
||||
"A modifier is either of its two keys: `ctrl` is down while the left Ctrl or the",
|
||||
"right one is. A keyboard binding may name one on its own -- `ctrl`, a key like",
|
||||
"any other -- or in front of a key, `shift+key_j`, which is J while Shift is",
|
||||
"held. A key bound both ways answers the modified binding while its modifier is",
|
||||
"down and the plain one otherwise, so J and Shift + J are two things and never",
|
||||
"both. A mode switch names one as `with`, as a pad switch names `menu`.",
|
||||
"",
|
||||
"Alt costs a browser something: pressed and let go on its own, Windows moves",
|
||||
"focus to the browser's menu, and the keys after it go there rather than to the",
|
||||
"page, which has to `preventDefault` it. MuJoCo's viewer uses all three with the",
|
||||
"mouse and does nothing with any of them alone."
|
||||
],
|
||||
"_keys_note": [
|
||||
"The numeric keypad, and letters as a task asks for them -- each one knowing",
|
||||
"what it costs.",
|
||||
"",
|
||||
"The keypad is the set MuJoCo's viewer does not bind. Everything else on the",
|
||||
"main keyboard is bound: `Simulate` handles its shortcuts in C++ and calls the",
|
||||
"user callback **as well as**, never instead, so a letter fires both actions",
|
||||
"and there is no way to intercept the second.",
|
||||
"",
|
||||
"MEASURED against this repository's mujoco, by reading `mjVISSTRING` and",
|
||||
"`mjRNDSTRING` -- **all twenty-six letters are taken**, one render flag each:",
|
||||
"",
|
||||
" A auto-connect B perturb force C contact point D static body",
|
||||
" E equality F contact force G fog H convex hull",
|
||||
" I inertia J joint K skybox L additive",
|
||||
" M centre of mass N island O perturb object P contact split",
|
||||
" Q camera R reflection S shadow T transparent",
|
||||
" U actuator V tendon W wireframe X texture",
|
||||
" Y range finder Z light and ' , / ; \\\\ ` as well",
|
||||
"",
|
||||
"The generated locomotion bindings use W A S D and the arrows for planar",
|
||||
"motion, J L to turn, and Esc to release. These keys still reach the viewer.",
|
||||
"For example, W also toggles wireframe and the left and right arrows step a",
|
||||
"paused simulation.",
|
||||
"",
|
||||
"Main-row digits 1 to 4 toggle the corresponding geom groups. Keypad digits",
|
||||
"leave the viewer flags unchanged and can be bound independently. Space",
|
||||
"pauses the viewer. Legacy FSM mode keys are specified by the bundle.",
|
||||
"",
|
||||
"**MuJoCo's viewer reports a key's press and nothing else.** The callback is",
|
||||
"handed a keycode and no action (`KeyCallbackType = Callable[[int], None]`), on",
|
||||
"a press only: MuJoCo's GLFW adapter drops the auto-repeat and the release.",
|
||||
"`mjrl.viewer.keys` puts a GLFW key callback of its own in front of the",
|
||||
"viewer's and hands every reader both edges, so a key is held from its press to",
|
||||
"its release, as in a browser, which reports both itself. Until 2026-09-29 the",
|
||||
"release was inferred from repeats that never came, and every key let go 0.75 s",
|
||||
"after its press.",
|
||||
"",
|
||||
"Browser codes carry over unchanged -- `KeyW` is `KeyW` whatever the viewer",
|
||||
"does -- so a web consumer pays nothing for this at all."
|
||||
],
|
||||
"gestures": [
|
||||
{
|
||||
"name": "rise",
|
||||
"fires": "the tick it goes down, one-shot"
|
||||
},
|
||||
{
|
||||
"name": "fall",
|
||||
"fires": "the tick it comes back up, one-shot"
|
||||
},
|
||||
{
|
||||
"name": "hold",
|
||||
"fires": "every tick it is down -- a dead-man switch, not a mode"
|
||||
},
|
||||
{
|
||||
"name": "toggle",
|
||||
"fires": "down until pressed again -- the default for entering a mode"
|
||||
},
|
||||
{
|
||||
"name": "single",
|
||||
"fires": "one press and no second inside the click window; switches like `toggle`"
|
||||
},
|
||||
{
|
||||
"name": "double",
|
||||
"fires": "two presses, each inside the click window of the last; switches like `toggle`"
|
||||
},
|
||||
{
|
||||
"name": "triple",
|
||||
"fires": "three presses, each inside the click window of the last; switches like `toggle`"
|
||||
},
|
||||
{
|
||||
"name": "quadruple",
|
||||
"fires": "four presses, each inside the click window of the last; switches like `toggle`"
|
||||
},
|
||||
{
|
||||
"name": "quintuple",
|
||||
"fires": "five presses, each inside the click window of the last; switches like `toggle`"
|
||||
}
|
||||
],
|
||||
"_gestures_note": "A control may be bound more than once with different gestures: press to arm and release to go is one button, `rise` and `fall`, named twice. The five click gestures count presses of one control -- the same pad button or key, with the same modifier -- and a press counts when it comes within `click_window_ms` (`[fsm]` in the controller's config) of the one before. A count fires the binding for it the moment no longer count on that control is bound, and otherwise once the window has run out with no further press; so `single` fires at once where it is the only click on its control and waits out the window where a `double` shares it. A click switches a mode the way `toggle` does, and an `event` fires once. A control with a click gesture takes no `rise`, `fall`, `hold` or `toggle` -- every click would be one of those first. This note used to say there would be no double click, because counting needs a clock and a clock depends on how often a host ticks. The window is measured on the timestamps every host already hands the controller, not in ticks, so a host at 50 Hz and one at 1 kHz count the same presses the same way; the resolution is the host's rate, far inside any window a person can click to. No long press: nothing has asked for one.",
|
||||
"absent": [
|
||||
{
|
||||
"name": "view",
|
||||
"why": "The pad has this button and the service deliberately does not publish it. Binding it works on a bench through controller/xbox.py, which does report it, and does nothing at all on the robot."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
"""The gamepad vocabulary: what a pad can say, for looking up.
|
||||
|
||||
`vocabulary.json` beside this file is the dictionary. This reads it and nothing
|
||||
else -- there is no list in this module, deliberately, because a list here would
|
||||
be the second copy and the one that goes out of date.
|
||||
|
||||
python -m controller --vocabulary
|
||||
|
||||
Consumers are elsewhere and each attaches its own meanings: a deploy manifest's
|
||||
`[[fsm.button]]`, a task's `controls.yaml`. That direction is this package's
|
||||
rule, stated at the top of `__init__.py`: nothing here has an opinion about what
|
||||
a stick means.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import json
|
||||
from importlib.resources import files
|
||||
from pathlib import Path
|
||||
|
||||
#: The dictionary. Beside this module so `python -m controller` and an editable
|
||||
#: install find the same bytes, and shipped as package data -- see `pyproject`.
|
||||
PATH = Path(__file__).resolve().parent / "vocabulary.json"
|
||||
|
||||
SCHEMA = "kk-control-vocabulary/2"
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=1)
|
||||
def load() -> dict:
|
||||
doc = json.loads(files("controller").joinpath("vocabulary.json").read_text("utf-8"))
|
||||
if doc.get("schema") != SCHEMA:
|
||||
raise ValueError(f"{PATH}: schema is {doc.get('schema')!r}, not {SCHEMA!r}")
|
||||
return doc
|
||||
|
||||
|
||||
def buttons() -> tuple[str, ...]:
|
||||
"""Every button a binding may name, the ten plus the four directions."""
|
||||
doc = load()
|
||||
return tuple(b["name"] for b in doc["buttons"] + doc["dpad"])
|
||||
|
||||
|
||||
def axes() -> tuple[str, ...]:
|
||||
"""Every axis, in the order the wire carries them."""
|
||||
return tuple(a["name"] for a in load()["axes"])
|
||||
|
||||
|
||||
def keys() -> tuple[str, ...]:
|
||||
"""Every key a binding may name.
|
||||
|
||||
The numeric keypad, and the letters a task has asked for. MuJoCo's viewer
|
||||
calls the user callback **in addition to** its own shortcuts and binds every
|
||||
letter, so a letter does two things at once there. `_keys_note` in the
|
||||
dictionary says which, and why a key there is a press and nothing else.
|
||||
"""
|
||||
return tuple(k["name"] for k in load()["keys"])
|
||||
|
||||
|
||||
def key_codes() -> dict[str, dict]:
|
||||
"""`{name: {"glfw": int, "browser": str}}`, for a host that reads one of
|
||||
those and has to arrive at the contract's name."""
|
||||
return {k["name"]: {"glfw": k["glfw"], "browser": k["browser"]}
|
||||
for k in load()["keys"]}
|
||||
|
||||
|
||||
def modifiers() -> dict[str, tuple[str, ...]]:
|
||||
"""`{name: keys}`: a modifier is down while either of its keys is.
|
||||
|
||||
`ctrl`, `shift` and `alt`, each the left and the right key. A keyboard
|
||||
binding names one on its own or in front of a key -- `shift+key_j` -- and a
|
||||
mode switch names one as its `with`. `_modifiers_note` has the rule.
|
||||
"""
|
||||
return {m["name"]: tuple(m["keys"]) for m in load()["modifiers"]}
|
||||
|
||||
|
||||
def keystroke(spec: str) -> tuple[str | None, str]:
|
||||
"""`"shift+key_j"` -> `("shift", "key_j")`; `"key_j"` -> `(None, "key_j")`;
|
||||
`"ctrl"` -> `(None, "ctrl")`.
|
||||
|
||||
What a keyboard binding names: a key, a modifier on its own, or a modifier
|
||||
held with a key. Raises for anything else, naming what it is not -- a key
|
||||
nobody reports binds nothing, and would look exactly like one that did.
|
||||
"""
|
||||
mods = modifiers()
|
||||
known = frozenset(keys()) | frozenset(mods)
|
||||
# Whole names first: `keypad_+` is a key whose name ends in a plus, not a
|
||||
# chord. A modifier's name has no plus, so a chord splits at the first one.
|
||||
if spec in known:
|
||||
return None, spec
|
||||
modifier, _, key = spec.partition("+")
|
||||
held = {k for ks in mods.values() for k in ks}
|
||||
if modifier in mods and key in known and key not in mods and key not in held:
|
||||
return modifier, key
|
||||
raise ValueError(
|
||||
f"{spec!r} is not a key, a modifier ({', '.join(mods)}), or a modifier and a key "
|
||||
f"(`shift+key_j`). The keys are {' '.join(keys())}"
|
||||
)
|
||||
|
||||
|
||||
def gestures() -> tuple[str, ...]:
|
||||
return tuple(g["name"] for g in load()["gestures"])
|
||||
|
||||
|
||||
def absent() -> dict[str, str]:
|
||||
"""Controls the pad has and the service does not publish, and why.
|
||||
|
||||
Kept in the dictionary rather than left out of it: a name that is missing
|
||||
reads as an oversight, and `view` is a decision.
|
||||
"""
|
||||
return {a["name"]: a["why"] for a in load()["absent"]}
|
||||
|
||||
|
||||
def describe() -> str:
|
||||
"""The whole thing, rendered. What `--vocabulary` prints."""
|
||||
doc = load()
|
||||
out = ["buttons (a `pad = ` in [[fsm.button]])"]
|
||||
out.append(" " + " ".join(b["name"] for b in doc["buttons"]))
|
||||
out.append(" " + " ".join(b["name"] for b in doc["dpad"]))
|
||||
for name, why in absent().items():
|
||||
out.append(f" not `{name}`: {why}")
|
||||
out.append("")
|
||||
out.append("keys (a `key = ` in [[fsm.button]])")
|
||||
doc_keys = doc["keys"]
|
||||
out.append(" " + " ".join(k["name"] for k in doc_keys))
|
||||
# One line per contiguous run of GLFW codes. The keypad is one run and the
|
||||
# letters are not -- a single `first-last` span printed "glfw 320-70" once
|
||||
# they were added, which reads as a range and is not one.
|
||||
runs, start = [], 0
|
||||
for i in range(1, len(doc_keys) + 1):
|
||||
if i == len(doc_keys) or doc_keys[i]["glfw"] != doc_keys[i - 1]["glfw"] + 1:
|
||||
a, b = doc_keys[start], doc_keys[i - 1]
|
||||
runs.append(
|
||||
f"glfw {a['glfw']}" + (f"-{b['glfw']}" if b is not a else "")
|
||||
+ f" browser {a['browser']}" + (f"..{b['browser']}" if b is not a else "")
|
||||
)
|
||||
start = i
|
||||
out.append(" " + "; ".join(runs) if len(runs) <= 3 else
|
||||
f" {len(runs)} runs of codes; see `controller/vocabulary.json`")
|
||||
out.append(" " + "\n ".join(doc["_keys_note"][:5]))
|
||||
out.append("")
|
||||
out.append("modifiers (`with = ` on a key switch; `shift+key_j` in a controls.yaml)")
|
||||
out.append(" " + " ".join(f"{m}: {' '.join(ks)}" for m, ks in modifiers().items()))
|
||||
out.append("")
|
||||
out.append("axes (a `source = ` in a task's controls.yaml)")
|
||||
for a in doc["axes"]:
|
||||
lo, hi = a["range"]
|
||||
out.append(f" {a['name']:<8}[{lo}, {hi}]" + (f" {a['note']}" if "note" in a else ""))
|
||||
out.append(" " + doc["_axes_note"])
|
||||
out.append("")
|
||||
out.append("gestures (an `on = ` in [[fsm.button]])")
|
||||
for g in doc["gestures"]:
|
||||
out.append(f" {g['name']:<10}{g['fires']}")
|
||||
out.append(" " + doc["_gestures_note"])
|
||||
|
||||
unresolved = [(b["name"], b["unresolved"]) for b in doc["buttons"] if "unresolved" in b]
|
||||
if unresolved:
|
||||
out.append("")
|
||||
out.append("unresolved")
|
||||
for name, why in unresolved:
|
||||
out.append(f" {name}: {why}")
|
||||
return "\n".join(out) + "\n"
|
||||
@@ -0,0 +1,248 @@
|
||||
"""The Xbox pad's evdev codes, and normalising them into a `GamepadState`.
|
||||
|
||||
The codes are the Linux `xpad` / `xone` drivers', which is what an Xbox controller
|
||||
presents over both USB and Bluetooth on a modern kernel. They are kernel
|
||||
constants, not invented here; the names are `linux/input-event-codes.h`.
|
||||
|
||||
## Sign conventions are the kernel's, not the robot's
|
||||
|
||||
evdev reports **`ABS_Y` positive when the stick is pushed down**, which is
|
||||
backwards from every intuition about forward. It is left exactly as the kernel
|
||||
gives it here, and flipped where the meaning is assigned, in the task. Correcting
|
||||
it in this file would be a robot opinion smuggled into a device driver: a task
|
||||
mapping the stick to something else, a camera or a gait frequency, would inherit
|
||||
a flip that only makes sense for velocity.
|
||||
|
||||
## There are two Xbox layouts, and the pad has to be asked which
|
||||
|
||||
USB through `xpad` and Bluetooth through `hid-generic` do **not** agree, and the
|
||||
disagreement is not cosmetic -- it puts the right stick where the other one keeps
|
||||
a trigger:
|
||||
|
||||
axis USB (xpad) Bluetooth (hid-generic)
|
||||
ABS_X, ABS_Y left stick left stick
|
||||
ABS_Z, ABS_RZ triggers **right stick**
|
||||
ABS_RX, ABS_RY right stick absent
|
||||
ABS_GAS, ABS_BRAKE absent triggers
|
||||
|
||||
Measured on an Xbox Wireless Controller over Bluetooth here: `ABS_X ABS_Y ABS_Z
|
||||
ABS_RZ ABS_GAS ABS_BRAKE ABS_HAT0X ABS_HAT0Y`, with the sticks 0..65535 and the
|
||||
triggers 0..1023. There is no `ABS_RX` on that device at all. Assuming the USB
|
||||
layout would have read pushing the right stick as squeezing the left trigger --
|
||||
which is not a crash, just a control that does the wrong thing.
|
||||
|
||||
`layout_of` picks between the two by **which axes the device actually reports**,
|
||||
which is evidence from the device rather than a table of models and firmware
|
||||
revisions that would need extending every time someone brings a different pad.
|
||||
|
||||
## Ranges come from the device
|
||||
|
||||
Sticks are -32768..32767 on one of those layouts and 0..65535 on the other, and
|
||||
triggers are 0..1023 or 0..255. `normalise` is given the ranges read from
|
||||
`EVIOCGABS` and uses them, so nothing here assumes a model -- the unsigned sticks
|
||||
above work out because the midpoint is computed rather than assumed to be zero.
|
||||
The dead zone is the driver's own `flat` for the same reason: a pad that reports
|
||||
its own slop knows better than a constant would.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
__all__ = ["ABS_AXES", "BUTTONS", "GamepadState", "normalise"]
|
||||
|
||||
EV_KEY = 0x01
|
||||
EV_ABS = 0x03
|
||||
|
||||
# ── Axes (EV_ABS) ────────────────────────────────────────────────────────
|
||||
ABS_X = 0x00 # left stick, right positive (both layouts)
|
||||
ABS_Y = 0x01 # left stick, **down positive** (both layouts)
|
||||
ABS_Z = 0x02 # USB: left trigger | Bluetooth: right stick X
|
||||
ABS_RX = 0x03 # USB: right stick X | Bluetooth: absent
|
||||
ABS_RY = 0x04 # USB: right stick Y | Bluetooth: absent
|
||||
ABS_RZ = 0x05 # USB: right trigger | Bluetooth: right stick Y
|
||||
ABS_GAS = 0x09 # Bluetooth: right trigger
|
||||
ABS_BRAKE = 0x0A # Bluetooth: left trigger
|
||||
ABS_HAT0X = 0x10 # d-pad, -1 / 0 / +1 (both layouts)
|
||||
ABS_HAT0Y = 0x11
|
||||
|
||||
#: Every axis worth asking the device about, across both layouts. `device.py`
|
||||
#: queries one range per entry and simply skips the ones the pad does not have,
|
||||
#: which is also how `layout_of` learns which layout it is looking at.
|
||||
ABS_AXES = (
|
||||
ABS_X, ABS_Y, ABS_Z, ABS_RX, ABS_RY, ABS_RZ,
|
||||
ABS_GAS, ABS_BRAKE, ABS_HAT0X, ABS_HAT0Y,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Layout:
|
||||
"""Which code carries which control on this particular pad."""
|
||||
|
||||
name: str
|
||||
rx: int
|
||||
ry: int
|
||||
lt: int
|
||||
rt: int
|
||||
|
||||
|
||||
#: xpad over USB: the right stick is RX/RY and the triggers are the Z pair.
|
||||
USB_LAYOUT = Layout("usb", rx=ABS_RX, ry=ABS_RY, lt=ABS_Z, rt=ABS_RZ)
|
||||
|
||||
#: hid-generic over Bluetooth: the Z pair *is* the right stick, and the triggers
|
||||
#: moved to the pedal codes.
|
||||
BLUETOOTH_LAYOUT = Layout("bluetooth", rx=ABS_Z, ry=ABS_RZ, lt=ABS_BRAKE, rt=ABS_GAS)
|
||||
|
||||
|
||||
def layout_of(axes) -> Layout:
|
||||
"""Pick a layout from the axes the device reports.
|
||||
|
||||
`ABS_RX` is the discriminator because it exists on exactly one of the two.
|
||||
Falling back to USB when neither is conclusive keeps an unknown pad working as
|
||||
before rather than reading its sticks as triggers.
|
||||
"""
|
||||
if ABS_RX in axes and ABS_RY in axes:
|
||||
return USB_LAYOUT
|
||||
if ABS_GAS in axes or ABS_BRAKE in axes:
|
||||
return BLUETOOTH_LAYOUT
|
||||
return USB_LAYOUT
|
||||
|
||||
# ── Buttons (EV_KEY) ─────────────────────────────────────────────────────
|
||||
BTN_SOUTH = 0x130 # A
|
||||
BTN_EAST = 0x131 # B
|
||||
BTN_WEST = 0x134 # X -- note the codes are not in ABXY order
|
||||
BTN_NORTH = 0x133 # Y
|
||||
BTN_TL = 0x136 # left bumper
|
||||
BTN_TR = 0x137 # right bumper
|
||||
BTN_SELECT = 0x13A # view
|
||||
BTN_START = 0x13B # menu
|
||||
BTN_MODE = 0x13C # the Xbox button
|
||||
BTN_THUMBL = 0x13D
|
||||
BTN_THUMBR = 0x13E
|
||||
|
||||
#: Button code -> the name the **dictionary** gives it.
|
||||
#:
|
||||
#: This package's job is to align a pad to those names and stop there; what any
|
||||
#: of them means belongs to whoever is driving, which is a task. So these are
|
||||
#: not our own spelling any more -- they are `controller/vocabulary.json`'s, the
|
||||
#: same names the robot's own pad service publishes and the same ones a
|
||||
#: `[[fsm.button]]` or a `controls.yaml` may write.
|
||||
#:
|
||||
#: `view` is here and not in the dictionary: this pad reports it and the robot's
|
||||
#: service deliberately does not. Reported rather than dropped, so `--raw` and a
|
||||
#: person pressing buttons see the truth; `vocabulary.absent()` is where the
|
||||
#: asymmetry is written down, and nothing downstream may bind it.
|
||||
#:
|
||||
#: **X and Y are unresolved.** The dictionary records it: we read 307 as `Y` and
|
||||
#: 308 as `X`; the service's table reads them the other way round. One of us is
|
||||
#: wrong about which physical button it is and pressing it settles it. Left
|
||||
#: disagreeing rather than quietly picked.
|
||||
BUTTONS = {
|
||||
BTN_SOUTH: "A", BTN_EAST: "B", BTN_WEST: "X", BTN_NORTH: "Y",
|
||||
BTN_TL: "LB", BTN_TR: "RB", BTN_SELECT: "view", BTN_START: "menu",
|
||||
BTN_MODE: "home", BTN_THUMBL: "L3", BTN_THUMBR: "R3",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GamepadState:
|
||||
"""One snapshot. Sticks are -1..1, triggers 0..1, buttons bool.
|
||||
|
||||
`ly` and `ry` keep the kernel's sign: **positive is down**.
|
||||
"""
|
||||
|
||||
lx: float = 0.0
|
||||
ly: float = 0.0
|
||||
rx: float = 0.0
|
||||
ry: float = 0.0
|
||||
lt: float = 0.0
|
||||
rt: float = 0.0
|
||||
hat_x: int = 0
|
||||
hat_y: int = 0
|
||||
buttons: frozenset[str] = field(default_factory=frozenset)
|
||||
|
||||
def __getattr__(self, item: str) -> bool:
|
||||
"""`state.A` for the buttons, so a caller does not have to remember
|
||||
whether a name is a field or a set member. The names are the
|
||||
dictionary's."""
|
||||
if item in BUTTONS.values():
|
||||
return item in self.buttons
|
||||
raise AttributeError(item)
|
||||
|
||||
@property
|
||||
def any_input(self) -> bool:
|
||||
"""Is the operator touching it? What "taking over" is decided on."""
|
||||
return (
|
||||
bool(self.buttons)
|
||||
or self.hat_x != 0
|
||||
or self.hat_y != 0
|
||||
or max(abs(self.lx), abs(self.ly), abs(self.rx), abs(self.ry)) > 0.0
|
||||
or max(self.lt, self.rt) > 0.0
|
||||
)
|
||||
|
||||
|
||||
def _axis(
|
||||
raw: dict[tuple[int, int], int],
|
||||
ranges: dict[int, tuple[int, int, int]],
|
||||
code: int,
|
||||
bipolar: bool,
|
||||
) -> float:
|
||||
"""One axis, scaled to -1..1 (bipolar) or 0..1, with the dead zone removed.
|
||||
|
||||
The dead zone is **rescaled, not just clipped**: an axis that starts moving at
|
||||
the edge of the zone should start from zero, not jump to the fraction the zone
|
||||
occupied. Clipping alone leaves a visible step at the point the stick starts to
|
||||
bite, which reads as a sticky control.
|
||||
"""
|
||||
if code not in ranges:
|
||||
return 0.0
|
||||
value = raw.get((EV_ABS, code))
|
||||
if value is None:
|
||||
return 0.0
|
||||
lo, hi, flat = ranges[code]
|
||||
|
||||
if bipolar:
|
||||
mid = (lo + hi) / 2.0
|
||||
span = (hi - lo) / 2.0
|
||||
if span <= 0:
|
||||
return 0.0
|
||||
x = (value - mid) / span
|
||||
dead = flat / span if span else 0.0
|
||||
if abs(x) <= dead:
|
||||
return 0.0
|
||||
x = (abs(x) - dead) / (1.0 - dead) * (1.0 if x > 0 else -1.0)
|
||||
return max(-1.0, min(1.0, x))
|
||||
|
||||
span = hi - lo
|
||||
if span <= 0:
|
||||
return 0.0
|
||||
x = (value - lo) / span
|
||||
dead = flat / span
|
||||
if x <= dead:
|
||||
return 0.0
|
||||
return max(0.0, min(1.0, (x - dead) / (1.0 - dead)))
|
||||
|
||||
|
||||
def normalise(
|
||||
raw: dict[tuple[int, int], int], ranges: dict[int, tuple[int, int, int]]
|
||||
) -> GamepadState:
|
||||
"""Raw evdev values plus the device's own ranges -> a `GamepadState`.
|
||||
|
||||
The layout is decided from `ranges`, which holds exactly the axes the device
|
||||
answered `EVIOCGABS` for. See `layout_of`.
|
||||
"""
|
||||
pressed = frozenset(
|
||||
name for code, name in BUTTONS.items() if raw.get((EV_KEY, code), 0)
|
||||
)
|
||||
layout = layout_of(ranges)
|
||||
return GamepadState(
|
||||
lx=_axis(raw, ranges, ABS_X, True),
|
||||
ly=_axis(raw, ranges, ABS_Y, True),
|
||||
rx=_axis(raw, ranges, layout.rx, True),
|
||||
ry=_axis(raw, ranges, layout.ry, True),
|
||||
lt=_axis(raw, ranges, layout.lt, False),
|
||||
rt=_axis(raw, ranges, layout.rt, False),
|
||||
hat_x=int(raw.get((EV_ABS, ABS_HAT0X), 0)),
|
||||
hat_y=int(raw.get((EV_ABS, ABS_HAT0Y), 0)),
|
||||
buttons=pressed,
|
||||
)
|
||||
@@ -0,0 +1,300 @@
|
||||
"""Reading an Xbox pad on Windows through XInput, with no dependencies.
|
||||
|
||||
## Why XInput
|
||||
|
||||
XInput is a plain C function in a DLL every Windows since 8 ships
|
||||
(`xinput1_4.dll`), so `ctypes` reaches it and nothing has to be installed -- the
|
||||
same trade `device.py` makes with evdev. The price is that it reads **only XInput
|
||||
devices**: Xbox pads and the ones that present themselves as one. A PlayStation or
|
||||
Switch pad is not one until Steam Input or DS4Windows sits in between.
|
||||
|
||||
## It speaks evdev, not a dialect of its own
|
||||
|
||||
Each reading is turned into the `{(type, code): value}` table the Linux `xpad`
|
||||
driver produces for the same pad over USB, with the ranges `xpad` declares, and
|
||||
then handed to `xbox.normalise` -- the function that reads a Linux pad. The sign
|
||||
conventions, the dead-zone rescaling and the button names are therefore one piece
|
||||
of code on both platforms rather than two that have to be kept agreeing, and
|
||||
`python -m controller` is the same bench check on either.
|
||||
|
||||
## The Y axes are flipped here, and nowhere else
|
||||
|
||||
XInput reports a stick pushed **up** as positive; evdev, and everything above this
|
||||
package, reads **down** as positive (`xbox.py` says why that is kept). The flip is
|
||||
`~y`, which is what `xpad` itself does to the same 16-bit value: it maps
|
||||
-32768..32767 onto itself exactly, where negation would leave the range at -32768.
|
||||
Leaving it out would not raise -- it would walk the robot backwards when the stick
|
||||
says forwards.
|
||||
|
||||
## The dead zone is Microsoft's
|
||||
|
||||
evdev hands over the driver's `flat`; XInput reports none. Microsoft documents one
|
||||
instead, `XINPUT_GAMEPAD_LEFT_THUMB_DEADZONE` and its siblings, because a stick does
|
||||
not return to exactly centre, and that is what goes into `flat` here -- the
|
||||
platform's own figure, which is the role the driver's `flat` plays on Linux. It is
|
||||
wider than what a Linux pad reports (xpad's 128 of 32768 over USB; 4095 of 65535
|
||||
from `hid-generic` over Bluetooth), so the stick travels further before it bites,
|
||||
and is rescaled like every dead zone here so full deflection still reads 1.
|
||||
|
||||
## Buttons are matched by name
|
||||
|
||||
XInput's bits name physical buttons, so each is mapped onto the code `xbox.BUTTONS`
|
||||
gives **that name**, not onto the code `xpad` would send. The physical X button is
|
||||
`X` here whatever is eventually decided about evdev's 307 and 308, which
|
||||
`controller/vocabulary.json` records as unresolved.
|
||||
|
||||
## The Xbox button needs the undocumented entry point
|
||||
|
||||
`XInputGetState` never reports it. Ordinal 100 of the same DLL, `XInputGetStateEx`,
|
||||
does -- it is what SDL reads -- and it writes four bytes more than the documented
|
||||
struct holds, so the struct below carries them. It is used when the DLL exports it
|
||||
and `home` simply never goes down when it does not (`xinput9_1_0.dll`). Windows
|
||||
also opens the Game Bar on that button unless the Game Bar is told not to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ctypes
|
||||
import functools
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from typing import NamedTuple
|
||||
|
||||
from .device import PadInfo
|
||||
from .xbox import (
|
||||
ABS_HAT0X,
|
||||
ABS_HAT0Y,
|
||||
ABS_RX,
|
||||
ABS_RY,
|
||||
ABS_RZ,
|
||||
ABS_X,
|
||||
ABS_Y,
|
||||
ABS_Z,
|
||||
BUTTONS,
|
||||
EV_ABS,
|
||||
EV_KEY,
|
||||
GamepadState,
|
||||
normalise,
|
||||
)
|
||||
|
||||
__all__ = ["RANGES", "XInputGamepad", "XInputPad", "describe", "find_pads", "read", "translate"]
|
||||
|
||||
#: `XUSER_MAX_COUNT`: XInput has four player slots and no more.
|
||||
SLOTS = 4
|
||||
ERROR_SUCCESS = 0
|
||||
|
||||
# ── The documented constants, from XInput.h ─────────────────────────────
|
||||
DPAD_UP = 0x0001
|
||||
DPAD_DOWN = 0x0002
|
||||
DPAD_LEFT = 0x0004
|
||||
DPAD_RIGHT = 0x0008
|
||||
LEFT_THUMB_DEADZONE = 7849
|
||||
RIGHT_THUMB_DEADZONE = 8689
|
||||
TRIGGER_THRESHOLD = 30
|
||||
|
||||
#: XInput button bit -> the dictionary's name for that physical button. 0x0400 is
|
||||
#: the Xbox button, which only `XInputGetStateEx` sets.
|
||||
_BUTTON_NAMES = {
|
||||
0x1000: "A", 0x2000: "B", 0x4000: "X", 0x8000: "Y",
|
||||
0x0100: "LB", 0x0200: "RB", 0x0020: "view", 0x0010: "menu",
|
||||
0x0400: "home", 0x0040: "L3", 0x0080: "R3",
|
||||
}
|
||||
_CODE_OF = {name: code for code, name in BUTTONS.items()}
|
||||
#: XInput button bit -> evdev key code, by way of the name. A name `xbox.BUTTONS`
|
||||
#: does not know raises here, at import, rather than reading as never pressed.
|
||||
BUTTON_CODES = {bit: _CODE_OF[name] for bit, name in _BUTTON_NAMES.items()}
|
||||
|
||||
#: What `xpad` declares for these axes over USB, with Microsoft's dead zones as the
|
||||
#: `flat`. Having `ABS_RX`/`ABS_RY` is also what makes `xbox.layout_of` choose the
|
||||
#: USB layout, so the triggers are read from `ABS_Z`/`ABS_RZ` and not the sticks.
|
||||
RANGES: dict[int, tuple[int, int, int]] = {
|
||||
ABS_X: (-32768, 32767, LEFT_THUMB_DEADZONE),
|
||||
ABS_Y: (-32768, 32767, LEFT_THUMB_DEADZONE),
|
||||
ABS_RX: (-32768, 32767, RIGHT_THUMB_DEADZONE),
|
||||
ABS_RY: (-32768, 32767, RIGHT_THUMB_DEADZONE),
|
||||
ABS_Z: (0, 255, TRIGGER_THRESHOLD),
|
||||
ABS_RZ: (0, 255, TRIGGER_THRESHOLD),
|
||||
ABS_HAT0X: (-1, 1, 0),
|
||||
ABS_HAT0Y: (-1, 1, 0),
|
||||
}
|
||||
|
||||
|
||||
class XInputGamepad(ctypes.Structure):
|
||||
"""`XINPUT_GAMEPAD`, plus the trailing DWORD `XInputGetStateEx` writes.
|
||||
|
||||
Fixed-width types rather than `ctypes.wintypes`, so the struct -- and
|
||||
`translate` with it -- exists on every platform and is tested on Linux.
|
||||
"""
|
||||
|
||||
_fields_ = [
|
||||
("wButtons", ctypes.c_uint16),
|
||||
("bLeftTrigger", ctypes.c_uint8),
|
||||
("bRightTrigger", ctypes.c_uint8),
|
||||
("sThumbLX", ctypes.c_int16),
|
||||
("sThumbLY", ctypes.c_int16),
|
||||
("sThumbRX", ctypes.c_int16),
|
||||
("sThumbRY", ctypes.c_int16),
|
||||
("dwPaddingReserved", ctypes.c_uint32),
|
||||
]
|
||||
|
||||
|
||||
class XInputState(ctypes.Structure):
|
||||
"""`XINPUT_STATE` as `XInputGetStateEx` fills it."""
|
||||
|
||||
_fields_ = [("dwPacketNumber", ctypes.c_uint32), ("Gamepad", XInputGamepad)]
|
||||
|
||||
|
||||
# Asserted rather than assumed: a buffer four bytes short of what the Ex call
|
||||
# writes does not raise, it overwrites whatever ctypes allocated next to it.
|
||||
assert ctypes.sizeof(XInputGamepad) == 16 and ctypes.sizeof(XInputState) == 20, (
|
||||
f"XINPUT_STATE_EX is {ctypes.sizeof(XInputState)} bytes here, not 20"
|
||||
)
|
||||
|
||||
|
||||
def translate(pad: XInputGamepad) -> dict[tuple[int, int], int]:
|
||||
"""One XInput reading -> the raw evdev table `xpad` would have produced."""
|
||||
buttons = pad.wButtons
|
||||
raw = {(EV_KEY, code): int(bool(buttons & bit)) for bit, code in BUTTON_CODES.items()}
|
||||
raw[(EV_ABS, ABS_X)] = pad.sThumbLX
|
||||
raw[(EV_ABS, ABS_Y)] = ~pad.sThumbLY # up is positive in XInput, down in evdev
|
||||
raw[(EV_ABS, ABS_RX)] = pad.sThumbRX
|
||||
raw[(EV_ABS, ABS_RY)] = ~pad.sThumbRY
|
||||
raw[(EV_ABS, ABS_Z)] = pad.bLeftTrigger
|
||||
raw[(EV_ABS, ABS_RZ)] = pad.bRightTrigger
|
||||
raw[(EV_ABS, ABS_HAT0X)] = bool(buttons & DPAD_RIGHT) - bool(buttons & DPAD_LEFT)
|
||||
raw[(EV_ABS, ABS_HAT0Y)] = bool(buttons & DPAD_DOWN) - bool(buttons & DPAD_UP)
|
||||
return raw
|
||||
|
||||
|
||||
class _Api(NamedTuple):
|
||||
dll: str
|
||||
get_state: Callable[[int, object], int]
|
||||
#: Whether `get_state` is `XInputGetStateEx`, i.e. whether `home` can be read.
|
||||
guide: bool
|
||||
|
||||
|
||||
#: Newest first. 1_4 ships with Windows 8 and later; 1_3 comes with the DirectX
|
||||
#: runtime; 9_1_0 is Windows 7's and has no ordinal 100.
|
||||
_DLLS = ("xinput1_4", "xinput1_3", "xinput9_1_0")
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=1)
|
||||
def _api() -> _Api | None:
|
||||
if sys.platform != "win32":
|
||||
return None
|
||||
for name in _DLLS:
|
||||
try:
|
||||
dll = ctypes.WinDLL(name)
|
||||
except OSError:
|
||||
continue
|
||||
try:
|
||||
fn, guide = dll[100], True
|
||||
except AttributeError:
|
||||
fn, guide = dll.XInputGetState, False
|
||||
fn.argtypes = (ctypes.c_uint32, ctypes.POINTER(XInputState))
|
||||
fn.restype = ctypes.c_uint32
|
||||
return _Api(name, fn, guide)
|
||||
return None
|
||||
|
||||
|
||||
def read(slot: int) -> XInputGamepad | None:
|
||||
"""The pad in `slot` now, or None if there is none (or no XInput at all)."""
|
||||
api = _api()
|
||||
if api is None:
|
||||
return None
|
||||
state = XInputState()
|
||||
if api.get_state(slot, ctypes.byref(state)) != ERROR_SUCCESS:
|
||||
return None
|
||||
return state.Gamepad
|
||||
|
||||
|
||||
def find_pads(readable_only: bool = False) -> list[PadInfo]:
|
||||
"""Every slot with a pad in it. XInput has no permissions to be refused, so a
|
||||
pad that answers is one this process can read, and `readable_only` changes
|
||||
nothing."""
|
||||
del readable_only
|
||||
return [
|
||||
PadInfo(f"xinput:{slot}", f"XInput pad, player {slot + 1}", True)
|
||||
for slot in range(SLOTS)
|
||||
if read(slot) is not None
|
||||
]
|
||||
|
||||
|
||||
def describe() -> str:
|
||||
"""What was found, or why nothing was."""
|
||||
api = _api()
|
||||
if api is None:
|
||||
return (
|
||||
"XInput is not available: none of "
|
||||
+ ", ".join(f"{name}.dll" for name in _DLLS)
|
||||
+ " could be loaded."
|
||||
)
|
||||
pads = find_pads()
|
||||
if not pads:
|
||||
return (
|
||||
"no XInput controller is connected. Windows pads are read through XInput, "
|
||||
"which sees Xbox pads and pads that present themselves as one; a "
|
||||
"PlayStation or Switch pad needs Steam Input or DS4Windows in between."
|
||||
)
|
||||
lines = [f" {p.path} {p.name} -- usable, through {api.dll}" for p in pads]
|
||||
if not api.guide:
|
||||
lines.append(f" {api.dll} has no XInputGetStateEx, so `home` never reads as pressed")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
@dataclass
|
||||
class XInputPad:
|
||||
"""An XInput slot, presenting the same surface as `device.Gamepad`.
|
||||
|
||||
Read on the caller's thread, once per `state()`: XInput is a poll API with
|
||||
nothing to wait on, so a reader thread would only add latency -- up to the
|
||||
15.6 ms system timer tick on Python 3.10, whose sleeps on Windows round up to
|
||||
it. When the pad goes away the last reading is kept and `connected` goes False,
|
||||
as `Gamepad` does, and the empty slot is not polled again: an empty slot is the
|
||||
slow case of `XInputGetState`, which Microsoft's documentation of it says not to
|
||||
poll every frame.
|
||||
"""
|
||||
|
||||
info: PadInfo
|
||||
_raw: dict[tuple[int, int], int] = field(default_factory=dict)
|
||||
_ranges: dict[int, tuple[int, int, int]] = field(default_factory=lambda: dict(RANGES))
|
||||
_alive: bool = False
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return self.info.name
|
||||
|
||||
@property
|
||||
def _slot(self) -> int:
|
||||
return int(self.info.path.rpartition(":")[2])
|
||||
|
||||
def start(self) -> None:
|
||||
self._alive = True
|
||||
self._poll()
|
||||
|
||||
def close(self) -> None:
|
||||
self._alive = False
|
||||
|
||||
@property
|
||||
def connected(self) -> bool:
|
||||
return self._alive
|
||||
|
||||
def _poll(self) -> None:
|
||||
if not self._alive:
|
||||
return
|
||||
pad = read(self._slot)
|
||||
if pad is None:
|
||||
self._alive = False
|
||||
return
|
||||
self._raw = translate(pad)
|
||||
|
||||
def state(self) -> GamepadState:
|
||||
"""A normalised snapshot. Never raises."""
|
||||
self._poll()
|
||||
return normalise(dict(self._raw), self._ranges)
|
||||
|
||||
def raw(self) -> dict[tuple[int, int], int]:
|
||||
"""The reading as evdev codes, for `python -m controller --raw`."""
|
||||
self._poll()
|
||||
return dict(self._raw)
|
||||
@@ -0,0 +1,196 @@
|
||||
# Robot application formats
|
||||
|
||||
`scripts/deploy.py` writes a directory and a sibling `.app`: an ordinary ZIP with
|
||||
`bundle.json` at its root. Entry order, timestamps and permissions are normalized.
|
||||
Readers select a format by its exact `schema` tag, not by guessing from files.
|
||||
[`app.schema`](app.schema) describes both versions. Python readers validate it
|
||||
before loading control code or models.
|
||||
|
||||
## Self-contained native application — `mjrl-robot-app/1`
|
||||
|
||||
This format includes a robot model and supports `rl_onnx`, `ik_mocap`,
|
||||
`wheeled_velocity`, `cpg_keyframe` and `model_viewer`. The Python runtime runs on
|
||||
Linux, Windows and macOS with compatible Python and the declared dependencies.
|
||||
It is a native MuJoCo application; it does not claim a browser, NPU or hardware
|
||||
binary. Host Python/MuJoCo libraries are dependencies, not embedded interpreters.
|
||||
|
||||
```
|
||||
robot/
|
||||
bundle.json
|
||||
app.schema
|
||||
LICENSE
|
||||
README.md
|
||||
model/robot.xml
|
||||
model/meshes/<digest>.<extension>
|
||||
model/textures/<digest>.<extension>
|
||||
controller.toml
|
||||
<mode>.json
|
||||
controls/<mode>.yaml
|
||||
<mode>.trajectory.json
|
||||
models/<mode>.onnx
|
||||
<mode>.layout.json
|
||||
verification.json
|
||||
verification/<mode>.json
|
||||
reference.json
|
||||
manual.en.json
|
||||
manual.zh.json
|
||||
runtime/controller.pyz
|
||||
runtime/requirements.txt
|
||||
```
|
||||
|
||||
Only applicable companions are present: a viewer needs neither a policy nor a
|
||||
trajectory. A controlled mode requires both native physics and control acceptance.
|
||||
A geometry viewer claims model compilation/display only. It cannot inherit a
|
||||
physical or locomotion capability because it successfully opened a file.
|
||||
|
||||
Every file except `bundle.json` is inventoried with byte length and SHA-256. Paths
|
||||
are local relative paths. The reader rejects missing/extra files, mismatched bytes,
|
||||
links, archive traversal, duplicate entries and excessive archive expansion. The
|
||||
normalized model has no XML includes or plugin declarations and its mesh/texture
|
||||
references must remain inside the app. Controller dependencies must be inventoried.
|
||||
Hashes detect changes; they are not signatures authenticating an outside publisher.
|
||||
Validation uses the trusted schema shipped with the host package or portable
|
||||
runtime. The application's `app.schema` is an informational copy and cannot
|
||||
weaken the reader's rules.
|
||||
|
||||
The manifest declares `robot_id`, `model`, `default_mode`, `modes`, `controller`,
|
||||
`manuals`, `reference`, `verification`, `runtimes`, `capabilities`, `source` and
|
||||
`files`. Each mode names its `control_mode` and `contract`; it names `controls`,
|
||||
`trajectory` companions through the contract, and `policy`/`layout` for ONNX.
|
||||
The declared capability must agree with the controller kind.
|
||||
|
||||
`controller.toml` uses `mjrl-app-controller/1` and declares `default_mode`. Native
|
||||
mode selection is explicit at launch (`--app-mode`) or through
|
||||
`RobotApplication.select_mode`. Selecting a different mode resets the world to that
|
||||
mode's verified initial state. It is not a hardware state-machine transition.
|
||||
|
||||
### Build and replay
|
||||
|
||||
First create a normalized model and controller with `scripts/onboard.py`. Then
|
||||
verify the exact controller, model and explicit physical response probes:
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py verify --model out/robot/robot.xml \
|
||||
--config out/robot/verify-input.json --report out/robot/verification.json
|
||||
python scripts/deploy.py --robot robot --model out/robot/robot.xml \
|
||||
--control out/robot/controller.json --controls out/robot/controls.yaml \
|
||||
--verification out/robot/verification.json --bundle out/apps
|
||||
python scripts/play.py --app out/apps/robot.app --headless --steps 100
|
||||
python scripts/deploy.py --check-reference out/apps/robot.app
|
||||
```
|
||||
|
||||
An ONNX mode also supplies `--policy <export-directory>` containing `actor.onnx`
|
||||
and `layout.json`; `--control` identifies the same controller JSON used by verify.
|
||||
A viewer omits `--control` and `--verification`.
|
||||
|
||||
Multiple modes use `--platform-manifest FILE` with `robot_id`, `model`, `modes`,
|
||||
optional `default_mode`, and optional shared `verification`. Mode entries accept
|
||||
`controller`, `controls`, `policy`, `layout` and a mode-specific `verification`.
|
||||
Paths resolve relative to the build manifest. Each controlled mode must match its
|
||||
own passing report; a report for another controller cannot authorize it.
|
||||
Existing output directories/archives are refused rather than overwritten.
|
||||
Concurrent builders targeting the same output are also refused.
|
||||
|
||||
Supply asset and policy license or attribution files with repeatable
|
||||
`--license PATH` options, or the build manifest's `licenses` list. They are copied
|
||||
under `licenses/`, listed in the application manifest, and covered by file hashes.
|
||||
The platform's Apache license does not replace the imported asset's license.
|
||||
|
||||
A report records the source XML hash, compiled model hash (including dependency
|
||||
effects), controller hash, controls hash and policy/trajectory dependency hashes.
|
||||
The bundler checks these before relocation. It compiles the relocated model and
|
||||
constructs each packaged controller and replays every reference before publishing.
|
||||
Reports must contain the current initial-pose, finite-state, contact, joint-margin
|
||||
and sustained-effort checks; a historical pass flag alone is insufficient.
|
||||
No physical asset values
|
||||
are guessed to repair a failed bundle.
|
||||
|
||||
The application records the verified initial state, control rate, probe inputs,
|
||||
measured samples and coordinate conventions. `--check-reference` replays these
|
||||
inputs and compares qpos, qvel, control, actuator force and time. Ordinary playback
|
||||
is not an acceptance test unless its command and response measurements are checked.
|
||||
|
||||
For an independent host, unpack the archive, install `runtime/requirements.txt` in
|
||||
an isolated Python environment, and run:
|
||||
|
||||
```bash
|
||||
python runtime/controller.pyz --app . --headless --steps 100
|
||||
```
|
||||
|
||||
The packaged Python code is run only by that explicit command. The workbench
|
||||
loads its installed runtime implementation, not arbitrary code from an opened app.
|
||||
Use `--reference-out FILE` to record actual playback and `--video FILE.mp4` with
|
||||
finite `--steps` for a preview. On a headless Linux rendering host, select an
|
||||
available MuJoCo renderer (for example `MUJOCO_GL=egl`).
|
||||
On macOS, MuJoCo's interactive viewer requires the `mjpython` launcher instead
|
||||
of `python`; headless replay uses ordinary Python. Native gamepad polling is
|
||||
implemented for Linux and Windows; macOS clients can use keyboard or API input.
|
||||
For a fixed velocity command, `--evaluation-out FILE` writes body-frame mean and
|
||||
final velocities, MAE/RMSE, peak tilt, resets and warning counts. It does not infer
|
||||
acceptance thresholds. The preview camera follows a model with one robot root.
|
||||
|
||||
## Legacy policy application — `kk-policy-bundle/1`
|
||||
|
||||
The existing Rust FSM path remains supported for policies whose contracts its
|
||||
reader implements. This version contains controller runtimes and policies, but
|
||||
its simulation world still comes from an installed task. It does not promise a
|
||||
self-contained model or traditional-control modes.
|
||||
|
||||
| Host | Inference | Runtime |
|
||||
|---|---|---|
|
||||
| `board` | RKNN | `runtime/board/controller` |
|
||||
| `web` | ONNX | `runtime/web/controller.wasm` and `controller.js` |
|
||||
| `mjlab` | ONNX | `runtime/mjlab/<platform>/controller.so` or `.pyd` |
|
||||
|
||||
Shared files are `controller.toml`, `<mode>.json`, optional trajectory companions,
|
||||
`reference.json`, manuals, `README.md`, and `models/<mode>.onnx`/`.rknn`. The
|
||||
schema's `x-files` inventory describes this legacy format only.
|
||||
|
||||
A default legacy build prepares board conversion and cross-builds, checks every
|
||||
host declared in `runtimes`, produces numerical reference vectors, validates the
|
||||
schema and packs the archive. `--allow-incomplete` skips unavailable board work
|
||||
and explicitly records omissions; it is not a board-ready release.
|
||||
|
||||
`deploy/manifests.json` starts empty in the generic platform. Populate it with
|
||||
explicit exports for your hardware, or use repeated `--mode NAME=EXPORT` and a
|
||||
provided `--fsm FILE`. One mode can use the generated single-mode FSM. No robot,
|
||||
board address, motor order or hardware-specific mode is selected implicitly.
|
||||
|
||||
The Rust contracts map `wire_joint_order`, `obs_joint_order` and
|
||||
`action_joint_order` by joint name. `default_joint_pos` covers the wire order.
|
||||
The policy contract records observation terms/history, action transform, rates,
|
||||
gains, limits and command ranges. Never replace these with positional alignment
|
||||
or guessed dimensions. Unsupported versioned native action semantics must fail
|
||||
closed in the legacy reader.
|
||||
|
||||
The implementation references are:
|
||||
|
||||
| Contract | Reader |
|
||||
|---|---|
|
||||
| Bundle and host selection | `fsm/src/bundle.rs` |
|
||||
| FSM | `fsm/src/config.rs` |
|
||||
| Named policy layout | `fsm/src/layout.rs` |
|
||||
| Input mappings | `fsm/src/operator.rs` |
|
||||
| Reference motion | `fsm/src/trajectory.rs` |
|
||||
| Recorded reference vectors | `fsm/src/reference.rs` |
|
||||
| Browser and Python entry points | `fsm/src/web.rs`, `fsm/src/py.rs` |
|
||||
|
||||
The generated legacy bundle README, from
|
||||
[`fsm/BUNDLE_README.md`](fsm/BUNDLE_README.md), documents its actual host API.
|
||||
|
||||
## Manuals and translations
|
||||
|
||||
Both application formats carry `manual.en.json` (`kk-bundle-manual/1`). The English
|
||||
manual comes from the controller's bindings. It lists each mode, entry/exit,
|
||||
keyboard bindings, pad controls and mode switches where implemented.
|
||||
|
||||
The bundle-manual skill translates only `source`, `summary`, `enter`, `leave`,
|
||||
`keyboard`, `pad` and `does`, preserving every identifier and list structure.
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --translate out/apps/robot \
|
||||
--language zh --manual /tmp/manual.zh.json
|
||||
```
|
||||
|
||||
The command validates the translation, updates inventory digests and repacks the
|
||||
archive. Never translate joint names, mode names, file paths or physical key IDs.
|
||||
@@ -0,0 +1,93 @@
|
||||
# Export, verify and package a robot application
|
||||
|
||||
The deterministic path starts with a normalized model and an explicit controller
|
||||
contract. The agent skills choose the route and extract metadata; CLI commands
|
||||
perform transformations, numerical checks and publication.
|
||||
|
||||
Read [BUNDLE.md](BUNDLE.md) when implementing a consumer and
|
||||
[the main tutorial](../docs/TUTORIAL.md) for asset onboarding.
|
||||
|
||||
## Native MuJoCo application
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/my_robot
|
||||
python scripts/onboard.py build-mjcf assets/my_robot --output out/my_robot/model.xml
|
||||
python scripts/onboard.py setup-tier2 --model out/my_robot/model.xml \
|
||||
--config assets/my_robot/controller.json --output out/my_robot/control
|
||||
python scripts/onboard.py verify --model out/my_robot/control/robot.xml \
|
||||
--config out/my_robot/verify-input.json --report out/my_robot/verification.json
|
||||
python scripts/deploy.py --robot my_robot --model out/my_robot/control/robot.xml \
|
||||
--control out/my_robot/control/controller.json --controls out/my_robot/control/controls.yaml \
|
||||
--verification out/my_robot/verification.json --bundle out/apps
|
||||
python scripts/play.py --app out/apps/my_robot.app --headless --steps 100
|
||||
python scripts/deploy.py --check-reference out/apps/my_robot.app
|
||||
```
|
||||
|
||||
Configuration examples for IK, joint sliders, trajectories, wheel mixing, CPG and
|
||||
policy adapters are in [CONTROLLER_SETUP.md](../docs/CONTROLLER_SETUP.md). Verification
|
||||
requires explicit expected physical responses. A generated configuration or an
|
||||
ONNX graph that compiles is not evidence that the robot tracks a command.
|
||||
|
||||
Geometry-only assets can publish `model_viewer` apps by omitting `--control` and
|
||||
`--verification`. They carry no physical/control acceptance claim. A controlled
|
||||
application is refused unless its verification matches model dependencies,
|
||||
controller parameters, controls, policy and trajectory bytes.
|
||||
|
||||
An ONNX application additionally supplies `--policy <export>` and its verified
|
||||
controller JSON. The native application contains the normalized model, meshes,
|
||||
contracts, references and portable Python runtime. It needs host Python and the
|
||||
listed third-party packages. Mode support is explicitly versioned; do not pass a
|
||||
native hybrid-action contract to a legacy board reader.
|
||||
|
||||
Include each imported asset's license and attribution with repeatable
|
||||
`--license PATH` arguments. The application preserves these files and their hashes;
|
||||
the platform license does not replace third-party terms.
|
||||
|
||||
## Training export
|
||||
|
||||
```bash
|
||||
python scripts/export.py --task my_robot --checkpoint logs/my_robot/my_robot/run/model_N.pt
|
||||
```
|
||||
|
||||
The exporter validates measurable observations and numerical parity with the
|
||||
checkpoint. Preserve the emitted joint orders, action transforms, gains, rates,
|
||||
command ranges and provenance. Use the source revision and robot spec that the
|
||||
checkpoint was trained with. Reconfiguring the environment after training is not
|
||||
an adaptation of the old policy.
|
||||
|
||||
## Legacy board and web builds
|
||||
|
||||
The Rust FSM remains the common controller for its supported legacy policy
|
||||
contracts. `manifests.json` contains no default robot. Add explicit export paths,
|
||||
mode switches and a cascade configuration for your machine, or use:
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --mode walk=<export-directory> --bundle out/legacy
|
||||
```
|
||||
|
||||
A complete legacy build converts ONNX to RKNN, cross-builds the board controller
|
||||
and Python extensions, builds WebAssembly, checks contracts on each included
|
||||
host, writes a reference and packs one app. It requires the documented toolchains.
|
||||
`--allow-incomplete` omits unavailable board components and records omissions;
|
||||
this flag is for development hosts, not a board-ready release.
|
||||
|
||||
- [ONNX to RKNN](convert/README.md)
|
||||
- [Rust controller and runtime builds](fsm/README.md)
|
||||
- [DDS wire interface](dds/README.md)
|
||||
|
||||
Do not infer a board address or mechanical compatibility. Use the deploy skill's
|
||||
read-only board preflight with an explicit host before deployment. First open the
|
||||
app with `--dry-run`, then run `--check-reference`; these checks do not drive a bus.
|
||||
Actual hardware operation requires the target machine configuration and the
|
||||
operator's explicit instruction.
|
||||
|
||||
## Operator manuals
|
||||
|
||||
Every build includes `manual.en.json`. Apply the bundle-manual skill after each
|
||||
build to create the Chinese translation. Add it through the checked path:
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --translate out/apps/my_robot --language zh --manual /tmp/manual.zh.json
|
||||
```
|
||||
|
||||
Identifiers must remain unchanged. The command updates hashes and repacks the app.
|
||||
@@ -0,0 +1,64 @@
|
||||
<!-- tracks: README.md @ sha256:767fc6409321444a -->
|
||||
# 导出、验收与机器人应用打包
|
||||
|
||||
确定性流程从规范化模型和明确的控制契约开始。Agent 技能负责选择路线并提取元数据;CLI 命令负责转换、数值检查和发布。
|
||||
|
||||
实现消费端时请阅读 [BUNDLE.md](BUNDLE.md),资产接入流程见[主教程](../docs/TUTORIAL.zh.md)。
|
||||
|
||||
## 原生 MuJoCo 应用
|
||||
|
||||
```bash
|
||||
python scripts/onboard.py inspect assets/my_robot
|
||||
python scripts/onboard.py build-mjcf assets/my_robot --output out/my_robot/model.xml
|
||||
python scripts/onboard.py setup-tier2 --model out/my_robot/model.xml \
|
||||
--config assets/my_robot/controller.json --output out/my_robot/control
|
||||
python scripts/onboard.py verify --model out/my_robot/control/robot.xml \
|
||||
--config out/my_robot/verify-input.json --report out/my_robot/verification.json
|
||||
python scripts/deploy.py --robot my_robot --model out/my_robot/control/robot.xml \
|
||||
--control out/my_robot/control/controller.json --controls out/my_robot/control/controls.yaml \
|
||||
--verification out/my_robot/verification.json --bundle out/apps
|
||||
python scripts/play.py --app out/apps/my_robot.app --headless --steps 100
|
||||
python scripts/deploy.py --check-reference out/apps/my_robot.app
|
||||
```
|
||||
|
||||
IK、关节滑块、轨迹、轮速混控、CPG 和策略适配的配置示例见 [CONTROLLER_SETUP.md](../docs/CONTROLLER_SETUP.md)。验收必须明确规定预期的物理响应。生成配置或成功编译 ONNX 图,均不能证明机器人能跟踪命令。
|
||||
|
||||
仅几何资产可以省略 `--control` 与 `--verification`,发布 `model_viewer` 应用。这类包不声明物理或控制验收通过。受控应用的验收记录必须与模型依赖、控制器参数、操作绑定、策略和轨迹文件完全匹配,否则拒绝打包。
|
||||
|
||||
ONNX 应用还须提供 `--policy <export>` 及经过验收的控制器 JSON。原生应用包含规范化模型、网格、契约、参考记录和便携 Python 运行时;主机需要安装 Python 及列出的第三方依赖。控制模式支持按版本明确声明;不能将原生混合动作契约交给旧版板端读取器。
|
||||
|
||||
使用可重复的 `--license PATH` 参数加入导入资产的许可证及来源说明。应用会保留这些文件及其摘要;平台许可证不能替代第三方条款。
|
||||
|
||||
## 训练导出
|
||||
|
||||
```bash
|
||||
python scripts/export.py --task my_robot --checkpoint logs/my_robot/my_robot/run/model_N.pt
|
||||
```
|
||||
|
||||
导出器校验可测量的观测项,以及与检查点之间的数值一致性。保留输出中的关节顺序、动作变换、增益、频率、命令范围和来源记录。使用训练检查点时的代码版本及机器人规格。训练后改变环境配置,并不等于完成旧策略的适配。
|
||||
|
||||
## 旧版板端与网页构建
|
||||
|
||||
Rust FSM 继续作为其支持的旧版策略契约的公共控制器。`manifests.json` 不再预置机器人。请为目标机器明确填写导出路径、模式切换及级联配置,或使用:
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --mode walk=<export-directory> --bundle out/legacy
|
||||
```
|
||||
|
||||
完整的旧版构建会将 ONNX 转为 RKNN,交叉编译板端控制器和 Python 扩展,构建 WebAssembly,在每个包含的主机端校验契约,写入参考记录并打包成一个应用。它需要文档中列出的工具链。`--allow-incomplete` 会省略不可用的板端组件并记录缺失项;该选项用于开发主机,不代表可以上板发布。
|
||||
|
||||
- [ONNX 转 RKNN](convert/README.md)
|
||||
- [Rust 控制器与运行时构建](fsm/README.md)
|
||||
- [DDS 通信接口](dds/README.md)
|
||||
|
||||
不要推测板端地址或机械兼容性。部署前,使用明确的主机地址执行 deploy 技能的只读板端预检。先用 `--dry-run` 打开应用,再运行 `--check-reference`;这些检查不会驱动总线。实际硬件操作需要目标机器配置及操作者的明确指令。
|
||||
|
||||
## 操作手册
|
||||
|
||||
每次构建都包含 `manual.en.json`。构建后使用 bundle-manual 技能生成中文译本,通过以下校验入口加入:
|
||||
|
||||
```bash
|
||||
python scripts/deploy.py --translate out/apps/my_robot --language zh --manual /tmp/manual.zh.json
|
||||
```
|
||||
|
||||
标识符必须保持不变。命令会更新文件摘要并重新打包应用。
|
||||
@@ -0,0 +1,809 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "kk-policy-bundle/1",
|
||||
"title": "A robot application bundle: <name>/ and <name>.app",
|
||||
"description": "Robot application formats: legacy policy bundles and self-contained native robot applications. Each version declares its actual supported runtimes.",
|
||||
"x-files": [
|
||||
{
|
||||
"path": "^bundle\\.json$",
|
||||
"required": true,
|
||||
"format": "json",
|
||||
"is": "this schema's document: every file, its size and SHA-256, and which host takes what"
|
||||
},
|
||||
{
|
||||
"path": "^README\\.md$",
|
||||
"required": true,
|
||||
"format": "markdown",
|
||||
"is": "the integration guide, from deploy/fsm/BUNDLE_README.md"
|
||||
},
|
||||
{
|
||||
"path": "^controller\\.toml$",
|
||||
"required": true,
|
||||
"format": "toml",
|
||||
"is": "the FSM: modes, the rules that switch between them, the keys and buttons",
|
||||
"namedBy": "fsm"
|
||||
},
|
||||
{
|
||||
"path": "^manual\\.en\\.json$",
|
||||
"required": true,
|
||||
"format": "json",
|
||||
"schema": "#/$defs/manual",
|
||||
"is": "every key and button the bundle reads and what it does, in English, generated at build time",
|
||||
"namedBy": "manuals.en"
|
||||
},
|
||||
{
|
||||
"path": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$",
|
||||
"required": false,
|
||||
"format": "json",
|
||||
"schema": "#/$defs/manual",
|
||||
"is": "a translation of manual.en.json, added with `deploy.py --translate`",
|
||||
"namedBy": "manuals.<language>"
|
||||
},
|
||||
{
|
||||
"path": "^reference\\.json$",
|
||||
"required": false,
|
||||
"format": "json",
|
||||
"is": "kk-policy-reference/1: recorded frames every host replays to measure that it agrees; absent only where onnxruntime was not installed at build time",
|
||||
"namedBy": "reference"
|
||||
},
|
||||
{
|
||||
"path": "^[^/]+\\.trajectory\\.json$",
|
||||
"required": false,
|
||||
"format": "json",
|
||||
"is": "a recording a reference-guided mode is driven by, beside the contracts",
|
||||
"namedBy": "modes.<mode>.trajectory"
|
||||
},
|
||||
{
|
||||
"path": "^[a-z][a-z0-9_]*\\.json$",
|
||||
"required": true,
|
||||
"format": "json",
|
||||
"is": "one mode's contract (the export's layout.json): observation layout, joint order, gains, controls",
|
||||
"namedBy": "modes.<mode>.contract"
|
||||
},
|
||||
{
|
||||
"path": "^models/[a-z][a-z0-9_]*\\.onnx$",
|
||||
"required": true,
|
||||
"format": "onnx",
|
||||
"is": "a policy, for the browser and play; one per export, shared by the modes on it",
|
||||
"namedBy": "modes.<mode>.models.onnx"
|
||||
},
|
||||
{
|
||||
"path": "^models/[a-z][a-z0-9_]*\\.rknn$",
|
||||
"required": false,
|
||||
"format": "rknn",
|
||||
"is": "the same policy converted for the RK3576 NPU; a mode without one runs on a stub on the board",
|
||||
"namedBy": "modes.<mode>.models.rknn"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/web/controller\\.wasm$",
|
||||
"required": true,
|
||||
"format": "wasm",
|
||||
"is": "the controller for a browser; one build for every CPU",
|
||||
"namedBy": "runtimes.web.wasm"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/web/controller\\.js$",
|
||||
"required": true,
|
||||
"format": "javascript",
|
||||
"is": "its wasm-bindgen binding",
|
||||
"namedBy": "runtimes.web.glue"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/mjlab/macosx-[a-z0-9._-]+/controller\\.so$",
|
||||
"required": false,
|
||||
"format": "mach-o",
|
||||
"is": "the same for macOS: macosx-universal2 is cross-built with zig, arm64 and x86_64 in one file, and loaded on any Mac whatever version its Python's platform string carries; a Mac that built the bundle adds its own under its exact platform",
|
||||
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/mjlab/[a-z0-9._-]+/controller\\.so$",
|
||||
"required": false,
|
||||
"format": "elf-shared-object",
|
||||
"is": "the controller as a Python extension (abi3) for play --app, one per platform it was built on",
|
||||
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/mjlab/win-[a-z0-9_]+/controller\\.pyd$",
|
||||
"required": false,
|
||||
"format": "pe-dll",
|
||||
"is": "the same, for play --app on Windows: cross-built with MinGW, importing python3.dll and nothing outside Windows itself",
|
||||
"namedBy": "runtimes.mjlab.extensions.<platform>.file"
|
||||
},
|
||||
{
|
||||
"path": "^runtime/board/controller$",
|
||||
"required": false,
|
||||
"format": "elf-aarch64",
|
||||
"is": "the robot's whole program, cross-compiled; absent until deploy/fsm/docker-build.sh has run, which bundle.json's notes then say",
|
||||
"namedBy": "runtimes.board.file"
|
||||
}
|
||||
],
|
||||
"$defs": {
|
||||
"mode": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"models",
|
||||
"contract",
|
||||
"observation",
|
||||
"action"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"models": {
|
||||
"description": "The mode's policy in every format a host runs: `onnx` always, `rknn` once converted.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"onnx"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"onnx": {
|
||||
"type": "string",
|
||||
"pattern": "^models/[a-z][a-z0-9_]*\\.onnx$"
|
||||
},
|
||||
"rknn": {
|
||||
"type": "string",
|
||||
"pattern": "^models/[a-z][a-z0-9_]*\\.rknn$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"contract": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9_]*\\.json$"
|
||||
},
|
||||
"observation": {
|
||||
"type": "integer"
|
||||
},
|
||||
"action": {
|
||||
"type": "integer"
|
||||
},
|
||||
"trajectory": {
|
||||
"type": "string",
|
||||
"pattern": "\\.trajectory\\.json$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"webRuntime": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"model",
|
||||
"glue",
|
||||
"wasm",
|
||||
"commit",
|
||||
"unverifiedByTheConsumer"
|
||||
],
|
||||
"properties": {
|
||||
"model": {
|
||||
"const": "onnx"
|
||||
},
|
||||
"glue": {
|
||||
"const": "runtime/web/controller.js"
|
||||
},
|
||||
"wasm": {
|
||||
"const": "runtime/web/controller.wasm"
|
||||
},
|
||||
"commit": {
|
||||
"type": "string"
|
||||
},
|
||||
"unverifiedByTheConsumer": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"mjlabRuntime": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"model",
|
||||
"extensions",
|
||||
"commit"
|
||||
],
|
||||
"properties": {
|
||||
"model": {
|
||||
"const": "onnx"
|
||||
},
|
||||
"extensions": {
|
||||
"description": "By `sysconfig.get_platform()` of the machine it runs on: the build machine's, and `win-amd64` and `macosx-universal2`, cross-built. macOS is keyed without a version (`mjrl.app_play.extension_for`).",
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"file",
|
||||
"abi"
|
||||
],
|
||||
"properties": {
|
||||
"file": {
|
||||
"type": "string",
|
||||
"pattern": "^runtime/mjlab/[a-z0-9._-]+/controller\\.(so|pyd)$"
|
||||
},
|
||||
"abi": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"commit": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"boardRuntime": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"model",
|
||||
"file",
|
||||
"platform",
|
||||
"commit"
|
||||
],
|
||||
"properties": {
|
||||
"model": {
|
||||
"const": "rknn"
|
||||
},
|
||||
"file": {
|
||||
"const": "runtime/board/controller"
|
||||
},
|
||||
"platform": {
|
||||
"const": "aarch64-unknown-linux-gnu"
|
||||
},
|
||||
"commit": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"manual": {
|
||||
"description": "manual.<language>.json. A translation keeps every field of manual.en.json and changes only the text ones: source, summary, enter, leave, keyboard, pad, does. The pad and the keyboard are listed apart, in each mode and among the switches (since 2026-09-29).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema",
|
||||
"language",
|
||||
"bundle",
|
||||
"source",
|
||||
"modes",
|
||||
"switches"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"schema": {
|
||||
"const": "kk-bundle-manual/1"
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z]{2}(-[A-Za-z]+)?$"
|
||||
},
|
||||
"bundle": {
|
||||
"type": "string"
|
||||
},
|
||||
"source": {
|
||||
"type": "string"
|
||||
},
|
||||
"modes": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"mode",
|
||||
"task",
|
||||
"default",
|
||||
"summary",
|
||||
"enter",
|
||||
"leave",
|
||||
"keyboard",
|
||||
"controls",
|
||||
"keys"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"mode": {
|
||||
"type": "string"
|
||||
},
|
||||
"task": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"default": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"summary": {
|
||||
"type": "string"
|
||||
},
|
||||
"enter": {
|
||||
"type": "string"
|
||||
},
|
||||
"leave": {
|
||||
"type": "string"
|
||||
},
|
||||
"keyboard": {
|
||||
"type": "string"
|
||||
},
|
||||
"controls": {
|
||||
"description": "The pad in this mode: each control and what it does.",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"control",
|
||||
"pad",
|
||||
"does"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"control": {
|
||||
"type": "string"
|
||||
},
|
||||
"pad": {
|
||||
"type": "string"
|
||||
},
|
||||
"does": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"keys": {
|
||||
"description": "The keyboard in this mode: each keystroke, as printed on the keys, and what it does.",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"key",
|
||||
"does"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"does": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"switches": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"device",
|
||||
"control",
|
||||
"with",
|
||||
"keys",
|
||||
"pad",
|
||||
"does",
|
||||
"from"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"device": {
|
||||
"enum": [
|
||||
"pad",
|
||||
"keyboard"
|
||||
]
|
||||
},
|
||||
"control": {
|
||||
"type": "string"
|
||||
},
|
||||
"with": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"keys": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"pad": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"does": {
|
||||
"type": "string"
|
||||
},
|
||||
"from": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"policyBundle": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema",
|
||||
"runtimes",
|
||||
"fsm",
|
||||
"modes",
|
||||
"notes",
|
||||
"files",
|
||||
"manuals",
|
||||
"reference"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"schema": {
|
||||
"const": "kk-policy-bundle/1"
|
||||
},
|
||||
"runtimes": {
|
||||
"description": "Each host's build of the controller, keyed by host. A host whose build this bundle does not carry has no entry, and `notes` says how to add it.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"web"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"web": {
|
||||
"$ref": "#/$defs/webRuntime"
|
||||
},
|
||||
"mjlab": {
|
||||
"$ref": "#/$defs/mjlabRuntime"
|
||||
},
|
||||
"board": {
|
||||
"$ref": "#/$defs/boardRuntime"
|
||||
}
|
||||
}
|
||||
},
|
||||
"fsm": {
|
||||
"const": "controller.toml"
|
||||
},
|
||||
"modes": {
|
||||
"description": "Every state that runs a model, by the name controller.toml gives it.",
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"patternProperties": {
|
||||
"^[a-z][a-z0-9_]*$": {
|
||||
"$ref": "#/$defs/mode"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"notes": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"reference": {
|
||||
"description": "The reference vectors' file, or null when none could be recorded.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"manuals": {
|
||||
"description": "The manual per language. English is always there; a translation is added after the build.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"en"
|
||||
],
|
||||
"patternProperties": {
|
||||
"^[a-z]{2}(-[A-Za-z]+)?$": {
|
||||
"type": "string",
|
||||
"pattern": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"built_from": {
|
||||
"description": "Where every mode came from, for a bundle built from deploy/manifests.json.",
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"task",
|
||||
"from",
|
||||
"export"
|
||||
],
|
||||
"properties": {
|
||||
"task": {
|
||||
"type": "string"
|
||||
},
|
||||
"from": {
|
||||
"type": "string"
|
||||
},
|
||||
"export": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"files": {
|
||||
"description": "Every file in the bundle but bundle.json itself, by its path inside it.",
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bytes",
|
||||
"sha256"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"type": "integer"
|
||||
},
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"robotApp": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema",
|
||||
"robot_id",
|
||||
"model",
|
||||
"default_mode",
|
||||
"modes",
|
||||
"controller",
|
||||
"manuals",
|
||||
"reference",
|
||||
"verification",
|
||||
"runtimes",
|
||||
"capabilities",
|
||||
"source",
|
||||
"files"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"schema": {
|
||||
"const": "mjrl-robot-app/1"
|
||||
},
|
||||
"robot_id": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z][a-z0-9_]*$"
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
},
|
||||
"default_mode": {
|
||||
"type": "string"
|
||||
},
|
||||
"modes": {
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"patternProperties": {
|
||||
"^[a-z][a-z0-9_]*$": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"control_mode",
|
||||
"contract"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"control_mode": {
|
||||
"enum": [
|
||||
"rl_onnx",
|
||||
"ik_mocap",
|
||||
"wheeled_velocity",
|
||||
"cpg_keyframe",
|
||||
"model_viewer"
|
||||
]
|
||||
},
|
||||
"contract": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
},
|
||||
"controls": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
},
|
||||
"policy": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
},
|
||||
"layout": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
},
|
||||
"verification": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*[\\\\:]).+$"
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"control_mode": {
|
||||
"const": "rl_onnx"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"required": [
|
||||
"policy",
|
||||
"layout"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"control_mode": {
|
||||
"not": {
|
||||
"const": "model_viewer"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"required": [
|
||||
"verification"
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"controller": {
|
||||
"const": "controller.toml"
|
||||
},
|
||||
"manuals": {
|
||||
"description": "The manual per language. English is always there; a translation is added after the build.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"en"
|
||||
],
|
||||
"patternProperties": {
|
||||
"^[a-z]{2}(-[A-Za-z]+)?$": {
|
||||
"type": "string",
|
||||
"pattern": "^manual\\.[a-z]{2}(-[A-Za-z]+)?\\.json$"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"reference": {
|
||||
"const": "reference.json"
|
||||
},
|
||||
"verification": {
|
||||
"enum": [
|
||||
"verification.json",
|
||||
null
|
||||
]
|
||||
},
|
||||
"runtimes": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"native"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"native": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"file",
|
||||
"language",
|
||||
"python",
|
||||
"dependencies",
|
||||
"platforms"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"file": {
|
||||
"const": "runtime/controller.pyz"
|
||||
},
|
||||
"language": {
|
||||
"const": "python"
|
||||
},
|
||||
"python": {
|
||||
"type": "string"
|
||||
},
|
||||
"dependencies": {
|
||||
"const": "runtime/requirements.txt"
|
||||
},
|
||||
"platforms": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"linux",
|
||||
"windows",
|
||||
"macos"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"capabilities": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"rl_onnx",
|
||||
"ik_mocap",
|
||||
"wheeled_velocity",
|
||||
"cpg_keyframe",
|
||||
"model_viewer"
|
||||
]
|
||||
}
|
||||
},
|
||||
"source": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"model_sha256"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"model_sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"files": {
|
||||
"description": "Every file in the bundle but bundle.json itself, by its path inside it.",
|
||||
"type": "object",
|
||||
"minProperties": 1,
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bytes",
|
||||
"sha256"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"bytes": {
|
||||
"type": "integer"
|
||||
},
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"licenses": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Inventoried platform and explicitly supplied asset/policy license and attribution files."
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/policyBundle"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/robotApp"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
# Conversion image: actor.onnx -> actor.rknn, on a host of any kind.
|
||||
#
|
||||
# rknn-toolkit2 publishes Linux wheels only (x86_64 and aarch64, no macOS, no
|
||||
# Windows), so on anything else the toolkit cannot be installed at all. This
|
||||
# image is the way those machines convert: Docker Desktop on macOS or Windows
|
||||
# runs a Linux VM, and the build below produces an image for whatever
|
||||
# architecture the host runs -- arm64 on Apple Silicon, amd64 elsewhere. Both are
|
||||
# architectures the toolkit ships for, so the same Dockerfile serves both without
|
||||
# a `--platform` flag.
|
||||
#
|
||||
# Build and run through `docker-convert.sh`, which handles the mount and the file
|
||||
# ownership; see README.md.
|
||||
|
||||
# Debian rather than Alpine: the rknn-toolkit2 wheels are manylinux_2_17, which
|
||||
# is glibc. On musl they install and then fail to load their .so.
|
||||
FROM python:3.12-slim
|
||||
|
||||
# rknn-toolkit2 declares opencv-python, which links libGL and libglib. Neither is
|
||||
# in a slim image, and without them `import rknn.api` dies on a missing
|
||||
# libGL.so.1 -- an error that reads like a broken package rather than a missing
|
||||
# system library. ~35 MB, and it is the whole reason this is not a one-line
|
||||
# Dockerfile.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends libgl1 libglib2.0-0 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /opt/convert
|
||||
|
||||
# Requirements first, so editing the converter does not re-download ~1.5 GB of
|
||||
# wheels. The torch pin inside is selected by `platform_machine`, which is what
|
||||
# lets one file serve both architectures.
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY onnx2rknn.py .
|
||||
|
||||
# Bundles are mounted here; see docker-convert.sh. Left as the working directory
|
||||
# so that a relative bundle path on the command line means the same thing inside
|
||||
# the container as it did outside.
|
||||
WORKDIR /work
|
||||
|
||||
# HOME is set at run time to a writable directory: the container runs as the
|
||||
# calling user (so output files are not root-owned), and that user has no home
|
||||
# inside the image. Several libraries write caches to ~ and fail without it.
|
||||
ENTRYPOINT ["python", "/opt/convert/onnx2rknn.py"]
|
||||
@@ -0,0 +1,175 @@
|
||||
# convert — ONNX to RKNN for the RK3576 NPU
|
||||
|
||||
The last link in the chain that starts at a checkpoint:
|
||||
|
||||
```
|
||||
model_*.pt ──scripts/export.py──▶ actor.onnx ──deploy/convert/onnx2rknn.py──▶ actor.rknn
|
||||
layout.json actor.rknn.json
|
||||
```
|
||||
|
||||
`scripts/export.py` writes an exported policy and checks the ONNX against the torch
|
||||
policy. This directory converts that ONNX for the Rockchip NPU and checks the result
|
||||
against the ONNX. The two checks together pin the whole chain: a policy that behaves on
|
||||
the board the way it behaved in training, or a conversion that fails loudly.
|
||||
|
||||
## Setup
|
||||
|
||||
Two ways in. Docker works everywhere and is the only option off Linux; the virtualenv is
|
||||
lighter if you are already on Linux.
|
||||
|
||||
### Docker — any host
|
||||
|
||||
```bash
|
||||
bash deploy/convert/docker-convert.sh --bundle out/exports/walk
|
||||
```
|
||||
|
||||
Builds the image on first use, then takes the same arguments as `onnx2rknn.py` and passes
|
||||
them through. `MJRL_RKNN_REBUILD=1` forces a rebuild after editing the converter.
|
||||
|
||||
rknn-toolkit2 publishes **Linux wheels only** — x86_64 and aarch64, no macOS, no Windows —
|
||||
so on a Mac or a Windows box this is not a convenience, it is the only way to convert.
|
||||
Docker Desktop runs a Linux VM and [`Dockerfile`](Dockerfile) builds for whatever
|
||||
architecture the host is, arm64 on Apple Silicon and amd64 elsewhere. Both are
|
||||
architectures the toolkit ships for, so no `--platform` flag is needed.
|
||||
|
||||
The wrapper exists because three details are each a wrong answer waiting to happen: the
|
||||
repository is mounted rather than the bundle (so paths mean the same inside and out), the
|
||||
container runs as the calling user (or the `.rknn` lands in your tree owned by root), and
|
||||
`HOME` is redirected (that user has no home in the image, and dependencies write caches
|
||||
to `~`).
|
||||
|
||||
### Virtualenv — Linux only
|
||||
|
||||
```bash
|
||||
bash deploy/convert/setup.sh
|
||||
```
|
||||
|
||||
Builds `deploy/convert/.venv` (~1.5 GB) and verifies the toolkit imports. Python
|
||||
3.8 – 3.12.
|
||||
|
||||
**It is a separate virtualenv on purpose.** rknn-toolkit2 pins `numpy<=1.26.4`,
|
||||
`torch<=2.4.0`, `onnx==1.16.1` and `setuptools<81`; the training environment is on torch
|
||||
2.11+cu128, numpy 2.5 and onnx 1.22. Installing the toolkit there downgrades all of them —
|
||||
including replacing the CUDA build of torch with a CPU one — and pip reports success. The
|
||||
damage shows up the next time something imports mjlab. Never `pip install rknn-toolkit2`
|
||||
into the training environment.
|
||||
|
||||
[`requirements.txt`](requirements.txt) records why every pin is there; four of them are
|
||||
bugs or gaps in rknn-toolkit2's own metadata and each was found by a fresh install
|
||||
failing.
|
||||
|
||||
## Converting
|
||||
|
||||
```bash
|
||||
deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py \
|
||||
--bundle out/exports/walk
|
||||
```
|
||||
|
||||
Writes `actor.rknn` and `actor.rknn.json` beside the bundle's `actor.onnx`. Real output,
|
||||
from the run the next section measures:
|
||||
|
||||
```
|
||||
[rknn] source .../actor.onnx [1, 74] -> [1, 20]
|
||||
[rknn] bundle consistent obs=74 act=20
|
||||
[rknn] test observations drawn from the normaliser baked into the graph (std 0.039 .. 2.623)
|
||||
[rknn] building target=rk3576 fp16
|
||||
[rknn] agreement with onnxruntime over 64 observations: max 1.094e-03, mean 2.319e-04,
|
||||
against outputs up to 1.995
|
||||
[rknn] done -> .../actor.rknn
|
||||
```
|
||||
|
||||
Useful switches — `--onnx FILE` for a bare ONNX with no contract beside it, `--target`
|
||||
for another Rockchip part, `--out`, `--samples`, `--seed`, `--tolerance`, `--verbose` for
|
||||
the toolkit's own log. `--quantize` exists, and `--calib-obs` feeds it a `[N, obs_dim]`
|
||||
`.npy` of recorded observations instead of sampled ones; read the next section before
|
||||
reaching for either.
|
||||
|
||||
## Why the default is fp16 and not int8
|
||||
|
||||
A historical 20-action locomotion actor (74 → 512 → 256 → 128 → 20, ELU)
|
||||
was tested on an i9-14900KF with rknn-toolkit2 2.3.2 and RK3576. The test
|
||||
used 64 in-distribution observations and converted action error to joint
|
||||
angle using that actor's `action_scale = 0.25`.
|
||||
|
||||
| build | max abs difference from ONNX | joint angle |
|
||||
|----------------------|------------------------------|-------------|
|
||||
| fp16 (default) | 1.1e-3 | 0.016° |
|
||||
| int8 (`--quantize`) | 2.7e-1 | 3.9° |
|
||||
|
||||
The maximum int8 error in this experiment was 13% of the approximately
|
||||
2-unit output magnitude. This result motivated the fp16 default and the
|
||||
default tolerance of 2e-2. Verify each actor with its own observations and
|
||||
acceptance tolerance.
|
||||
|
||||
## What the check actually checks
|
||||
|
||||
The converted model is run in the toolkit's simulator against onnxruntime on the same
|
||||
observations, and the outputs compared element-wise. It is built to one side and moved
|
||||
into place only after it passes, so **a failed check leaves no `.rknn` on disk** — there
|
||||
is nothing for someone to pick up later without the scrollback that condemned it. Three
|
||||
more things are worth knowing.
|
||||
|
||||
**The observations come from the policy's own training distribution.** The exported
|
||||
actor carries its `EmpiricalNormalization` inside the graph — the leading `Sub`/`Div`
|
||||
pair, whose constants are the observation mean and standard deviation measured during
|
||||
training. `onnx2rknn.py` reads them and samples `mean + std * N(0,1)`. Feeding raw
|
||||
`N(0,1)` instead puts terms whose training standard deviation is 0.039 about 25σ from
|
||||
their mean, the activations blow up, and fp16 error grows with them: 5.0e-3 rather than
|
||||
1.1e-3. That is measuring the test inputs, not the conversion.
|
||||
|
||||
**The simulator is not the board.** It models the NPU's arithmetic closely enough to
|
||||
catch a conversion that changed the graph, which is what this check is for. It is not a
|
||||
substitute for running the policy once on real hardware before trusting it.
|
||||
|
||||
The toolkit prints two warnings during a good conversion, both expected:
|
||||
|
||||
- `The config.mean_values is None, zeros will be set for input 0` — correct. The
|
||||
normalisation is already inside the graph; setting `mean_values` would apply it twice.
|
||||
This warning is the confirmation that it is not.
|
||||
- `The 'data_format' is not set, and its default value is 'nhwc'` — harmless for a
|
||||
`[1, obs_dim]` input, which has no spatial layout to reorder. The 1.1e-3 agreement is
|
||||
the evidence: a permuted input would not land within 1e-3 of the ONNX.
|
||||
|
||||
## On the board
|
||||
|
||||
The `.rknn` is loaded by `librknnrt.so`, which is part of the board image rather than
|
||||
this repository. **It must be at least as new as the toolkit that built the model** —
|
||||
2.3.2 here. A model built by a newer toolkit than the runtime fails to load, usually
|
||||
with an unhelpful error:
|
||||
|
||||
```bash
|
||||
strings /usr/lib/librknnrt.so | grep -i 'librknnrt version'
|
||||
```
|
||||
|
||||
Inference itself is `rknn_init` → `rknn_inputs_set` → `rknn_run` → `rknn_outputs_get`
|
||||
from the C API, or `RKNNLite` from `rknn-toolkit-lite2` in Python. Feed the observation
|
||||
as a flat `float32` vector of length `observation.dim`, in the order defined by
|
||||
`layout.json`. Apply the returned action as
|
||||
the bundle's own `README.md` says — `target = action * action_scale +
|
||||
default_joint_pos[joint]`, paired joint by joint in `action_joint_order`. Nothing about
|
||||
the conversion changes that contract.
|
||||
|
||||
`actor.rknn.json` records what produced the model: the source ONNX's sha256, the toolkit
|
||||
version, the target, and the verification numbers. It answers "is this `.rknn` the one
|
||||
built from that `.onnx`?" without relying on the two files having stayed in the same
|
||||
directory.
|
||||
|
||||
**It records the ONNX's checksum rather than the `.rknn`'s, because the `.rknn` does not
|
||||
have a stable one.** Converting the same ONNX twice on the same machine, back to back,
|
||||
produces files of identical length that differ in ~14 kB — always the same byte range,
|
||||
filled with what look like leaked heap pointers, i.e. uninitialised padding the toolkit
|
||||
serialises. Both builds reported the same agreement with onnxruntime to four figures, so
|
||||
the models are equivalent; only the bytes are not. The container and the virtualenv differ
|
||||
from each other in exactly the same way and for the same reason, so a byte difference
|
||||
between them means nothing either.
|
||||
|
||||
The practical consequence: **do not checksum a `.rknn` to decide whether to redeploy.**
|
||||
It changes every build. Compare `onnx_sha256` in the sidecar instead.
|
||||
|
||||
## Reconverting
|
||||
|
||||
`.rknn` files are build products and are not committed (`.gitignore` covers `*.rknn`
|
||||
alongside `*.onnx`). Reconvert whenever `actor.onnx` changes. `scripts/deploy.py` does
|
||||
it for you: it reconverts any `.rknn` whose sidecar `onnx_sha256` no longer matches before
|
||||
bundling. Nothing on the board checks, so a `.rknn` copied there by hand beside a fresh
|
||||
`.onnx` is still silent.
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run the ONNX -> RKNN conversion inside a container, from any host.
|
||||
#
|
||||
# Takes the same arguments as onnx2rknn.py and passes them straight through:
|
||||
#
|
||||
# bash deploy/convert/docker-convert.sh --bundle out/exports/walk
|
||||
# bash deploy/convert/docker-convert.sh --bundle <dir> --target rk3588
|
||||
#
|
||||
# The image is built on first use and reused after that. Force a rebuild after
|
||||
# editing the converter or its requirements:
|
||||
#
|
||||
# MJRL_RKNN_REBUILD=1 bash deploy/convert/docker-convert.sh --bundle <dir>
|
||||
#
|
||||
# Three details this wraps, each of which is a wrong answer waiting to happen:
|
||||
#
|
||||
# - **The repository is mounted, not the bundle.** Bundle paths are given
|
||||
# relative to the repository root, and mounting only the bundle would break
|
||||
# every path someone copies out of the README.
|
||||
# - **The container runs as the calling user.** Docker's default is root, and
|
||||
# the .rknn it writes would land in the tree owned by root -- on Linux the
|
||||
# next export, running as you, then cannot overwrite it.
|
||||
# - **HOME is redirected to /tmp.** That user has no home inside the image, and
|
||||
# several of the toolkit's dependencies write caches to ~ on import.
|
||||
#
|
||||
# On macOS and Windows this is the only way to convert: rknn-toolkit2 has no
|
||||
# wheel for either. On Linux it works too, but `setup.sh` is lighter.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO="$(cd "$HERE/../.." && pwd)"
|
||||
|
||||
# Tagged with the toolkit version, because that version has to match the board's
|
||||
# librknnrt.so: two images side by side should be distinguishable at a glance.
|
||||
TOOLKIT="$(sed -n 's/^rknn-toolkit2==\(.*\)$/\1/p' "$HERE/requirements.txt" | head -1)"
|
||||
IMAGE="mjrl-rknn:${TOOLKIT:-latest}"
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "docker not found. Install Docker Desktop (macOS / Windows) or docker.io (Linux)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -n "${MJRL_RKNN_REBUILD:-}" ] || ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
|
||||
echo "[docker] building $IMAGE (first run pulls ~1.5 GB of wheels)"
|
||||
docker build -t "$IMAGE" "$HERE"
|
||||
fi
|
||||
|
||||
# --user keeps output files owned by the caller. On macOS and Windows the bind
|
||||
# mount already maps ownership and the flag is harmless.
|
||||
exec docker run --rm -i \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--env HOME=/tmp \
|
||||
--volume "$REPO:/work" \
|
||||
--workdir /work \
|
||||
"$IMAGE" "$@"
|
||||
Executable
+491
@@ -0,0 +1,491 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Convert an exported `actor.onnx` into an `.rknn` for the RK3576 NPU.
|
||||
|
||||
`scripts/export.py` produces an exported policy -- `actor.onnx` plus
|
||||
`layout.json`. This takes the ONNX out of that bundle, converts it for the
|
||||
Rockchip NPU, and **checks the result numerically** before calling it done.
|
||||
|
||||
Run it with this directory's own interpreter, never the repository's:
|
||||
|
||||
deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py \
|
||||
--bundle out/exports/walk
|
||||
|
||||
rknn-toolkit2 pins `numpy<=1.26.4`, `torch<=2.4.0` and `onnx==1.16.1`.
|
||||
Installing it into the training environment downgrades all three, so it gets a
|
||||
virtualenv of its own; `setup.sh` builds it.
|
||||
|
||||
## The two silent failures this guards against
|
||||
|
||||
**The RKNN API returns error codes, it does not raise.** `build()` answering -1
|
||||
and the script carrying on leaves the previous `.rknn` on disk -- or no file at
|
||||
all -- and exits 0. Every toolkit call here is checked.
|
||||
|
||||
**A conversion can succeed and still be wrong.** An op fused into something with
|
||||
different arithmetic, a layout the toolkit guessed, quantization left on: all of
|
||||
these produce a valid `.rknn` that behaves differently from the ONNX. So the
|
||||
converted model is run in the toolkit's simulator against onnxruntime on the same
|
||||
observations and the outputs are compared element-wise. This mirrors
|
||||
`scripts/export.py::_validate_onnx`, which does the same one link earlier in the
|
||||
chain; together the two pin torch -> ONNX -> RKNN.
|
||||
|
||||
## Which observations the check is run on
|
||||
|
||||
The exported actor carries its `EmpiricalNormalization` inside the graph -- the
|
||||
leading `Sub` / `Div` pair, whose constants are the observation mean and standard
|
||||
deviation measured during training. This script reads them and draws test
|
||||
observations from `mean + std * N(0, 1)`, i.e. from the distribution the policy
|
||||
actually saw.
|
||||
|
||||
That is not cosmetic. Feeding raw `N(0, 1)` instead puts terms whose training
|
||||
standard deviation is 0.039 about 25 sigma from their mean; the activations blow
|
||||
up and fp16 rounding error grows with them, so the comparison measures the test
|
||||
inputs rather than the conversion. Measured on the historical actor below: 5.0e-3 max
|
||||
absolute difference on raw `N(0, 1)`, 1.1e-3 on in-distribution observations.
|
||||
|
||||
If the graph does not start with that `Sub` / `Div` pair the script says so and
|
||||
falls back to `N(0, 1)`, rather than pretending the check is as tight.
|
||||
|
||||
## Measurements
|
||||
|
||||
A historical locomotion actor (74 -> 512 -> 256 -> 128 -> 20, ELU) was
|
||||
tested on an i9-14900KF with rknn-toolkit2 2.3.2 and RK3576. The test used
|
||||
64 in-distribution observations. The last column converts action error
|
||||
to joint angle using the actor's `action_scale = 0.25`.
|
||||
|
||||
| build | max abs difference from ONNX | joint angle |
|
||||
|-----------------------------|------------------------------|-------------|
|
||||
| fp16 (the default) | 1.1e-3 | 0.016 deg |
|
||||
| int8 (`--quantize`) | 2.7e-1 | 3.9 deg |
|
||||
|
||||
The int8 error in this experiment was 13% of the approximately 2-unit
|
||||
output magnitude. This result motivated the fp16 default and the tolerance
|
||||
of 2e-2. Verify each actor with its own observation distribution.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import platform
|
||||
import tempfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
#: Default agreement required between the RKNN simulator and onnxruntime.
|
||||
#: Chosen to sit an order of magnitude above the fp16 floor (1.1e-3, measured
|
||||
#: above) and an order of magnitude below the int8 error (2.7e-1), so it passes
|
||||
#: rounding noise and fails anything that changed the arithmetic.
|
||||
DEFAULT_TOLERANCE = 2e-2
|
||||
|
||||
#: Calibration samples for `--quantize`. rknn's own examples use a few hundred.
|
||||
CALIBRATION_SAMPLES = 256
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
# The host has to be right before anything else is worth trying
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _require_host() -> None:
|
||||
"""rknn-toolkit2 ships Linux x86_64 wheels only.
|
||||
|
||||
Said plainly here because the alternative is an ImportError from a
|
||||
`.so` several frames deep, which reads like a broken install rather than an
|
||||
unsupported machine.
|
||||
"""
|
||||
if platform.system() != "Linux" or platform.machine() not in ("x86_64", "aarch64"):
|
||||
raise SystemExit(
|
||||
f"[rknn] rknn-toolkit2 runs on Linux x86_64 (or aarch64); this is "
|
||||
f"{platform.system()} {platform.machine()}. Convert on a Linux PC and "
|
||||
f"copy the .rknn to the board."
|
||||
)
|
||||
|
||||
|
||||
def _shown(path: Path) -> str:
|
||||
"""A path as the reader would type it: relative to where they are, if it can be.
|
||||
|
||||
The commands in the message below are meant to be copied, and this file sits
|
||||
two directories down. Naming the directory (`convert/setup.sh`) is wrong from
|
||||
anywhere, and an absolute path is right but unreadable; relative to the
|
||||
working directory is what someone standing at the repository root can paste.
|
||||
"""
|
||||
try:
|
||||
return str(path.relative_to(Path.cwd()))
|
||||
except ValueError:
|
||||
return str(path)
|
||||
|
||||
|
||||
def _require_toolkit():
|
||||
"""Import the toolkit, or say which interpreter to use instead.
|
||||
|
||||
The likeliest way to reach this is running the script with the repository's
|
||||
`.venv`, where rknn-toolkit2 is deliberately absent.
|
||||
"""
|
||||
if importlib.util.find_spec("rknn") is None:
|
||||
here = Path(__file__).resolve().parent
|
||||
raise SystemExit(
|
||||
f"[rknn] rknn-toolkit2 is not installed in this interpreter.\n"
|
||||
f" It must not be installed into the training environment -- it "
|
||||
f"pins numpy<=1.26.4, torch<=2.4.0 and onnx==1.16.1.\n"
|
||||
f" Build its own environment: bash {_shown(here / 'setup.sh')}\n"
|
||||
f" Then run: {_shown(here / '.venv/bin/python')} "
|
||||
f"{_shown(Path(__file__).resolve())} ..."
|
||||
)
|
||||
from rknn.api import RKNN
|
||||
|
||||
return RKNN
|
||||
|
||||
|
||||
def _check(ret: int, what: str) -> None:
|
||||
"""Every RKNN call returns a status code instead of raising."""
|
||||
if ret != 0:
|
||||
raise SystemExit(f"[rknn] {what} failed (rknn returned {ret}); see the log above")
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
# Reading what the ONNX and the contract say about themselves
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _io_shapes(model) -> tuple[list[int], list[int]]:
|
||||
def dims(value):
|
||||
return [d.dim_value for d in value.type.tensor_type.shape.dim]
|
||||
|
||||
return dims(model.graph.input[0]), dims(model.graph.output[0])
|
||||
|
||||
|
||||
def _observation_stats(model, obs_dim: int):
|
||||
"""The observation mean and standard deviation baked into the graph head.
|
||||
|
||||
`rsl_rl`'s `_OnnxMLPModel.forward` runs the `EmpiricalNormalization` before
|
||||
the MLP, so an exported actor starts with `Sub(obs, mean)` then
|
||||
`Div(_, std)`. Only the first two nodes are examined: a `Sub` further in is
|
||||
part of the network and its constants are not observation statistics.
|
||||
"""
|
||||
import numpy as np
|
||||
from onnx import numpy_helper
|
||||
|
||||
nodes = list(model.graph.node)
|
||||
if len(nodes) < 2 or nodes[0].op_type != "Sub" or nodes[1].op_type != "Div":
|
||||
return None
|
||||
init = {i.name: numpy_helper.to_array(i) for i in model.graph.initializer}
|
||||
mean, std = init.get(nodes[0].input[1]), init.get(nodes[1].input[1])
|
||||
if mean is None or std is None or mean.size != obs_dim or std.size != obs_dim:
|
||||
return None
|
||||
return np.asarray(mean, np.float32).reshape(1, obs_dim), np.asarray(
|
||||
std, np.float32
|
||||
).reshape(1, obs_dim)
|
||||
|
||||
|
||||
def _sample_observations(n: int, obs_dim: int, stats, seed: int):
|
||||
import numpy as np
|
||||
|
||||
rng = np.random.default_rng(seed)
|
||||
if stats is None:
|
||||
return rng.normal(0.0, 1.0, (n, obs_dim)).astype(np.float32)
|
||||
mean, std = stats
|
||||
return (mean + std * rng.normal(0.0, 1.0, (n, obs_dim))).astype(np.float32)
|
||||
|
||||
|
||||
def _cross_check_contract(contract: dict, ishape: list[int], oshape: list[int]) -> None:
|
||||
"""The bundle's contract and its ONNX must agree.
|
||||
|
||||
They come out of the same run of `scripts/export.py`, so disagreement means
|
||||
the directory holds files from two different exports -- the one case where
|
||||
converting the ONNX would produce a model that does not match the joint order
|
||||
and gains the deployment will use.
|
||||
"""
|
||||
want_obs = contract["observation"]["dim"]
|
||||
want_act = contract["action"]["dim"]
|
||||
if ishape[-1] != want_obs or oshape[-1] != want_act:
|
||||
raise SystemExit(
|
||||
f"[rknn] the bundle is inconsistent: the layout says "
|
||||
f"obs={want_obs} act={want_act}, actor.onnx is {ishape} -> {oshape}. "
|
||||
f"Re-export the bundle."
|
||||
)
|
||||
print(f"[rknn] bundle consistent obs={want_obs} act={want_act}")
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
# Conversion
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _write_calibration(observations, directory: Path) -> Path:
|
||||
"""rknn takes calibration data as a text file of per-sample file paths."""
|
||||
import numpy as np
|
||||
|
||||
lines = []
|
||||
for i, obs in enumerate(observations):
|
||||
path = directory / f"obs_{i:04d}.npy"
|
||||
np.save(path, obs.reshape(1, -1))
|
||||
lines.append(str(path))
|
||||
listing = directory / "dataset.txt"
|
||||
listing.write_text("\n".join(lines), encoding="utf-8")
|
||||
return listing
|
||||
|
||||
|
||||
def convert(onnx_path: Path, out_path: Path, target: str, quantize: bool,
|
||||
calibration, verbose: bool):
|
||||
"""Build the `.rknn` and return an initialised simulator session."""
|
||||
RKNN = _require_toolkit()
|
||||
rknn = RKNN(verbose=verbose)
|
||||
|
||||
# mean_values / std_values are deliberately left unset. The normalisation is
|
||||
# already inside the graph (see the module docstring); setting them here
|
||||
# would apply it a second time, and the toolkit's "zeros will be set for
|
||||
# input 0" warning is the confirmation that it is not being applied twice.
|
||||
rknn.config(target_platform=target)
|
||||
_check(rknn.load_onnx(model=str(onnx_path)), f"load_onnx({onnx_path})")
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="rknn-calib-") as tmp:
|
||||
dataset = str(_write_calibration(calibration, Path(tmp))) if quantize else None
|
||||
_check(rknn.build(do_quantization=quantize, dataset=dataset), "build")
|
||||
|
||||
_check(rknn.export_rknn(str(out_path)), f"export_rknn({out_path})")
|
||||
_check(rknn.init_runtime(), "init_runtime (simulator)")
|
||||
return rknn
|
||||
|
||||
|
||||
def verify(rknn, onnx_path: Path, observations) -> tuple[float, float, float]:
|
||||
"""Compare the converted model against onnxruntime, observation by observation.
|
||||
|
||||
Returns (max abs difference, mean abs difference, max abs ONNX output). The
|
||||
third is what makes the first two readable: a 1e-3 difference means one thing
|
||||
against outputs of order 2 and another against outputs of order 200.
|
||||
"""
|
||||
import numpy as np
|
||||
import onnxruntime as ort
|
||||
|
||||
session = ort.InferenceSession(str(onnx_path), providers=["CPUExecutionProvider"])
|
||||
name = session.get_inputs()[0].name
|
||||
worst, total, magnitude = 0.0, 0.0, 0.0
|
||||
for obs in observations:
|
||||
x = obs.reshape(1, -1)
|
||||
y_onnx = session.run(None, {name: x})[0]
|
||||
y_rknn = np.asarray(rknn.inference(inputs=[x])[0]).reshape(y_onnx.shape)
|
||||
diff = np.abs(y_rknn - y_onnx)
|
||||
worst = max(worst, float(diff.max()))
|
||||
total += float(diff.mean())
|
||||
magnitude = max(magnitude, float(np.abs(y_onnx).max()))
|
||||
return worst, total / len(observations), magnitude
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="onnx2rknn.py",
|
||||
description=(__doc__ or "").split("\n\n")[0],
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
source = parser.add_mutually_exclusive_group(required=True)
|
||||
source.add_argument(
|
||||
"--bundle", type=Path,
|
||||
help="an exported policy from scripts/export.py (actor.onnx + layout.json)",
|
||||
)
|
||||
source.add_argument("--onnx", type=Path, help="a bare .onnx file, no contract to check against")
|
||||
parser.add_argument(
|
||||
"--out", type=Path, default=None,
|
||||
help="output .rknn; defaults to <bundle>/actor.rknn, or <onnx>.rknn",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--target", default="rk3576",
|
||||
help="target platform, validated by rknn-toolkit2 (default: rk3576)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--samples", type=int, default=64, help="observations used for the check (default: 64)"
|
||||
)
|
||||
parser.add_argument("--seed", type=int, default=0, help="seed for those observations")
|
||||
parser.add_argument(
|
||||
"--tolerance", type=float, default=DEFAULT_TOLERANCE,
|
||||
help=f"max allowed difference from onnxruntime (default: {DEFAULT_TOLERANCE})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quantize", action="store_true",
|
||||
help="build int8 instead of fp16. Measured at a 13%% error on an actor MLP; "
|
||||
"read the module docstring before using it",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--calib-obs", type=Path, default=None,
|
||||
help="[N, obs_dim] float32 .npy of recorded observations for --quantize; "
|
||||
"without it, calibration is drawn from the baked-in normaliser",
|
||||
)
|
||||
parser.add_argument("--verbose", action="store_true", help="the toolkit's own verbose log")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = _parse_args()
|
||||
if args.calib_obs is not None and not args.quantize:
|
||||
# Silently ignoring it would read as "calibration data was used"
|
||||
raise SystemExit("[rknn] --calib-obs is calibration data for --quantize, which is off")
|
||||
_require_host()
|
||||
_require_toolkit()
|
||||
|
||||
import numpy as np
|
||||
import onnx
|
||||
|
||||
if args.bundle is not None:
|
||||
onnx_path = args.bundle / "actor.onnx"
|
||||
if not onnx_path.is_file():
|
||||
raise SystemExit(f"[rknn] no actor.onnx in {args.bundle} -- is that a bundle?")
|
||||
# `contract.json` is what scripts/export.py wrote before the file took the
|
||||
# name the on-robot importer looks for. Bundles under that name are still
|
||||
# readable here; nothing regenerates an old export just to rename it.
|
||||
contract_path = next(
|
||||
(p for p in (args.bundle / "layout.json", args.bundle / "isaac_layout.json",
|
||||
args.bundle / "contract.json")
|
||||
if p.is_file()),
|
||||
None,
|
||||
)
|
||||
else:
|
||||
onnx_path = args.onnx
|
||||
if not onnx_path.is_file():
|
||||
raise SystemExit(f"[rknn] no such file: {onnx_path}")
|
||||
contract_path = None
|
||||
|
||||
out_path = args.out or onnx_path.with_suffix(".rknn")
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
model = onnx.load(str(onnx_path))
|
||||
ishape, oshape = _io_shapes(model)
|
||||
if len(ishape) != 2 or ishape[0] != 1:
|
||||
raise SystemExit(
|
||||
f"[rknn] expected a [1, obs_dim] input, got {ishape}. This script converts "
|
||||
f"the single-observation actor that scripts/export.py writes."
|
||||
)
|
||||
obs_dim, act_dim = ishape[-1], oshape[-1]
|
||||
print(f"[rknn] source {onnx_path} {ishape} -> {oshape}")
|
||||
|
||||
contract = None
|
||||
if contract_path is not None and contract_path.is_file():
|
||||
contract = json.loads(contract_path.read_text(encoding="utf-8"))
|
||||
_cross_check_contract(contract, ishape, oshape)
|
||||
|
||||
stats = _observation_stats(model, obs_dim)
|
||||
if stats is None:
|
||||
print(
|
||||
"[rknn] WARNING: the graph does not start with the Sub/Div normaliser pair, "
|
||||
"so the check runs on raw N(0,1) observations. That is a looser check -- "
|
||||
"out-of-distribution inputs inflate fp16 error (5.0e-3 against 1.1e-3, "
|
||||
"measured); read the tolerance below with that in mind."
|
||||
)
|
||||
else:
|
||||
print(
|
||||
f"[rknn] test observations drawn from the normaliser baked into the graph "
|
||||
f"(std {float(stats[1].min()):.3f} .. {float(stats[1].max()):.3f})"
|
||||
)
|
||||
|
||||
observations = _sample_observations(args.samples, obs_dim, stats, args.seed)
|
||||
|
||||
if args.quantize and args.calib_obs is not None:
|
||||
calibration = np.load(args.calib_obs).astype(np.float32)
|
||||
if calibration.ndim != 2 or calibration.shape[1] != obs_dim:
|
||||
raise SystemExit(
|
||||
f"[rknn] --calib-obs must be [N, {obs_dim}], got {calibration.shape}"
|
||||
)
|
||||
print(f"[rknn] calibrating on {len(calibration)} recorded observations")
|
||||
elif args.quantize:
|
||||
calibration = _sample_observations(
|
||||
CALIBRATION_SAMPLES, obs_dim, stats, args.seed + 1
|
||||
)
|
||||
print(
|
||||
f"[rknn] WARNING: calibrating on {CALIBRATION_SAMPLES} sampled observations "
|
||||
f"rather than recorded ones. Pass --calib-obs to do better."
|
||||
)
|
||||
else:
|
||||
calibration = None
|
||||
|
||||
print(f"[rknn] building target={args.target} "
|
||||
f"{'int8 (quantized)' if args.quantize else 'fp16'}")
|
||||
|
||||
# Built to one side and moved into place only once it has been checked. A
|
||||
# model that failed verification must not be left on disk under the name the
|
||||
# deployment reads: the next person to look sees a plausible .rknn and a
|
||||
# scrollback they did not write.
|
||||
staging = out_path.with_name(out_path.name + ".partial")
|
||||
try:
|
||||
rknn = convert(onnx_path, staging, args.target, args.quantize, calibration, args.verbose)
|
||||
try:
|
||||
worst, mean, magnitude = verify(rknn, onnx_path, observations)
|
||||
finally:
|
||||
rknn.release()
|
||||
|
||||
print(
|
||||
f"[rknn] agreement with onnxruntime over {args.samples} observations: "
|
||||
f"max {worst:.3e}, mean {mean:.3e}, against outputs up to {magnitude:.3f}"
|
||||
)
|
||||
if worst > args.tolerance:
|
||||
raise SystemExit(
|
||||
f"[rknn] the converted model disagrees with the ONNX by {worst:.3e}, over "
|
||||
f"the {args.tolerance:.3e} tolerance. Nothing was written.\n"
|
||||
+ (
|
||||
" int8 quantization is the likely cause: measured at 2.7e-1 on "
|
||||
"an actor MLP, which is 3.9 degrees of joint error. Drop --quantize.\n"
|
||||
if args.quantize
|
||||
else " fp16 rounding alone measures 1.1e-3; a difference this "
|
||||
"large means the conversion changed the arithmetic.\n"
|
||||
)
|
||||
)
|
||||
staging.replace(out_path)
|
||||
except BaseException:
|
||||
staging.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
meta = {
|
||||
"_source": "deploy/convert/onnx2rknn.py -- reconvert whenever actor.onnx changes",
|
||||
"created": datetime.now(timezone.utc).isoformat(timespec="seconds"),
|
||||
"onnx": str(onnx_path),
|
||||
# So that "which ONNX is this .rknn from?" has an answer that does not
|
||||
# depend on the two files having stayed in the same directory
|
||||
"onnx_sha256": _sha256(onnx_path),
|
||||
"target_platform": args.target,
|
||||
"quantized": bool(args.quantize),
|
||||
"precision": "int8" if args.quantize else "float16",
|
||||
"toolkit_version": _toolkit_version(),
|
||||
"obs_dim": obs_dim,
|
||||
"action_dim": act_dim,
|
||||
"verification": {
|
||||
"samples": args.samples,
|
||||
"seed": args.seed,
|
||||
"observations": "baked_normalizer" if stats is not None else "standard_normal",
|
||||
"max_abs_diff": worst,
|
||||
"mean_abs_diff": mean,
|
||||
"max_abs_output": magnitude,
|
||||
"tolerance": args.tolerance,
|
||||
},
|
||||
**({"task_contract": {"action_joint_order": contract["action_joint_order"]}}
|
||||
if contract else {}),
|
||||
}
|
||||
meta_path = out_path.with_suffix(out_path.suffix + ".json")
|
||||
meta_path.write_text(json.dumps(meta, indent=2), encoding="utf-8")
|
||||
|
||||
print(f"\n[rknn] done -> {out_path}")
|
||||
for f in (out_path, meta_path):
|
||||
print(f" {f.name:<20}{f.stat().st_size / 1024:8.1f} KB")
|
||||
print(
|
||||
"\n[rknn] the board's librknnrt.so must be at least as new as the toolkit "
|
||||
f"({_toolkit_version()}); check it with\n"
|
||||
" strings /usr/lib/librknnrt.so | grep -i 'librknnrt version'"
|
||||
)
|
||||
|
||||
|
||||
def _toolkit_version() -> str:
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
|
||||
try:
|
||||
return version("rknn-toolkit2")
|
||||
except PackageNotFoundError:
|
||||
return "unknown"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,56 @@
|
||||
# The conversion environment. Install into deploy/convert/.venv only -- never into the
|
||||
# repository's .venv. `setup.sh` does that; this file records why each pin is here.
|
||||
#
|
||||
# Linux x86_64 (or aarch64) only: rknn-toolkit2 publishes no macOS or Windows wheel.
|
||||
|
||||
# Rockchip's converter, newest release on PyPI at the time of writing and the one
|
||||
# every measurement in README.md was taken with. Pinned rather than floating
|
||||
# because the version is coupled to the board: a .rknn built here does not load
|
||||
# under a librknnrt.so older than the toolkit that produced it.
|
||||
rknn-toolkit2==2.3.2
|
||||
|
||||
# ── Pins rknn-toolkit2 declares wrongly or not at all ──────────────────
|
||||
#
|
||||
# onnx: the wheel asks for `onnx>=1.16.1`, which resolves to 1.22 and then dies
|
||||
# inside `load_onnx` with `AttributeError: module 'onnx' has no attribute
|
||||
# 'mapping'` -- `onnx.mapping` was removed after 1.16. Measured, not guessed.
|
||||
onnx==1.16.1
|
||||
|
||||
# torch: imported at `import rknn.api` time even though nothing here converts a
|
||||
# torch model, so it cannot be left out. Taken from PyTorch's CPU index because
|
||||
# the PyPI wheel drags in ~2.5 GB of CUDA libraries this never calls.
|
||||
#
|
||||
# Two lines, because the CPU index names the two architectures differently:
|
||||
# x86_64 gets `torch-2.4.0+cpu-...x86_64.whl`, aarch64 gets plain
|
||||
# `torch-2.4.0-...aarch64.whl` with no local version. Pinning `2.4.0+cpu`
|
||||
# everywhere resolves to nothing on aarch64 -- "no matching distribution", after
|
||||
# the host check has already said the machine is supported. (The aarch64 build
|
||||
# has no CUDA variant to be confused with, so the bare version is already the
|
||||
# CPU one.)
|
||||
--extra-index-url https://download.pytorch.org/whl/cpu
|
||||
torch==2.4.0+cpu ; platform_machine == "x86_64"
|
||||
torch==2.4.0 ; platform_machine == "aarch64"
|
||||
|
||||
# setuptools: rknn-toolkit2 imports `pkg_resources`, which setuptools removed in
|
||||
# 81. Undeclared, so a fresh install resolves to the newest setuptools and then
|
||||
# fails at `RKNN()` with `ModuleNotFoundError: No module named 'pkg_resources'`.
|
||||
# Measured on a clean venv here: 84.0.0 fails, 80.10.2 works.
|
||||
setuptools<81
|
||||
|
||||
# The rest are rknn-toolkit2's own declared dependencies, listed so that a
|
||||
# resolver failure names a version rather than "no matching distribution".
|
||||
numpy==1.26.4
|
||||
protobuf>=4.21.6,<=4.25.4
|
||||
psutil>=5.9.0
|
||||
ruamel.yaml>=0.17.21
|
||||
scipy>=1.9.3
|
||||
tqdm>=4.64.1
|
||||
fast-histogram>=0.11
|
||||
# opencv-python needs libGL at import. On a container without it, either
|
||||
# `apt install libgl1` or swap this line for opencv-python-headless -- but note
|
||||
# that pip will then install opencv-python anyway to satisfy rknn-toolkit2's own
|
||||
# metadata, leaving two packages providing `cv2`.
|
||||
opencv-python>=4.5.5.64
|
||||
|
||||
# Used by onnx2rknn.py as the reference the converted model is checked against.
|
||||
onnxruntime>=1.17.0
|
||||
Executable
+62
@@ -0,0 +1,62 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build deploy/convert/.venv, the conversion environment.
|
||||
#
|
||||
# It is a separate virtualenv on purpose. rknn-toolkit2 pins numpy<=1.26.4,
|
||||
# torch<=2.4.0 and (in practice) onnx==1.16.1; installing it into the
|
||||
# repository's .venv downgrades all three and breaks training -- silently, since
|
||||
# pip reports success and the damage only shows up the next time something
|
||||
# imports mjlab.
|
||||
#
|
||||
# Usage:
|
||||
# bash deploy/convert/setup.sh
|
||||
#
|
||||
# Then, from the repository root:
|
||||
# deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py --bundle <bundle dir>
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
VENV="$HERE/.venv"
|
||||
|
||||
os=$(uname -s)
|
||||
arch=$(uname -m)
|
||||
if [ "$os" != "Linux" ] || { [ "$arch" != "x86_64" ] && [ "$arch" != "aarch64" ]; }; then
|
||||
echo "rknn-toolkit2 publishes Linux x86_64 / aarch64 wheels only; this is $os $arch." >&2
|
||||
echo "Convert on a Linux PC and copy the .rknn to the board." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# rknn-toolkit2 2.3.2 has wheels for cp36 through cp312. 3.13 finds nothing and
|
||||
# pip's message ("no matching distribution") does not say why.
|
||||
py_minor=$(python3 -c 'import sys; print(sys.version_info[1])')
|
||||
if [ "$py_minor" -gt 12 ] || [ "$py_minor" -lt 8 ]; then
|
||||
echo "python3.$py_minor found; rknn-toolkit2 2.3.2 ships wheels for 3.8 - 3.12 only." >&2
|
||||
echo "Point this script at one of those: python3.12 -m venv $VENV, then pip install -r ..." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -d "$VENV" ]; then
|
||||
echo "creating $VENV"
|
||||
python3 -m venv "$VENV"
|
||||
fi
|
||||
|
||||
"$VENV/bin/pip" install --upgrade pip --quiet
|
||||
"$VENV/bin/pip" install -r "$HERE/requirements.txt"
|
||||
|
||||
echo
|
||||
echo "--- verifying ---"
|
||||
"$VENV/bin/python" - <<'PY'
|
||||
import platform
|
||||
from importlib.metadata import version
|
||||
from rknn.api import RKNN
|
||||
|
||||
print(f"rknn-toolkit2 {version('rknn-toolkit2')} "
|
||||
f"onnx {version('onnx')} numpy {version('numpy')} torch {version('torch')}")
|
||||
print(f"host {platform.system()} {platform.machine()} python {platform.python_version()}")
|
||||
RKNN(verbose=False).release()
|
||||
print("ok")
|
||||
PY
|
||||
|
||||
echo
|
||||
echo "Convert a bundle with:"
|
||||
echo " deploy/convert/.venv/bin/python deploy/convert/onnx2rknn.py --bundle <bundle dir>"
|
||||
@@ -0,0 +1,167 @@
|
||||
# dds — the wire interface to the robot
|
||||
|
||||
Everything needed to talk to the motor controller and the operator, and nothing else.
|
||||
Copied from the `mbus` middleware (a CycloneDDS wrapper) rather than reimplemented,
|
||||
because every file here is one half of a contract whose other half is already running on
|
||||
the robot.
|
||||
|
||||
```
|
||||
idl/ the message types
|
||||
idl_common.idl shared enums and headers, included by the rest
|
||||
idl_motor_control.idl MotorControl::Control (out) and ::State (in)
|
||||
idl_robot_control.idl RobotControl::Control — the operator's commands, interpreted
|
||||
idl_robot_control_raw.idl
|
||||
RobotControlRaw::ControlRaw — the gamepad itself
|
||||
idl_imu.idl Imu::ImuOutput — orientation and rates
|
||||
config/
|
||||
mbus_qos.xml QoS per topic
|
||||
mbus_config.xml CycloneDDS runtime settings (interfaces, discovery)
|
||||
topics.json topic name + QoS profile + type, per channel
|
||||
```
|
||||
|
||||
## The channels
|
||||
|
||||
| direction | topic | type | QoS profile |
|
||||
|---|---|---|---|
|
||||
| subscribe | `RobotControlRaw_ControlRaw` | `RobotControlRaw::ControlRaw` | `RobotControlRaw_ControlRaw` |
|
||||
| subscribe | `mc/motor_state` | `MotorControl::State` | `MotorControl_State` |
|
||||
| subscribe | `imu/output` | `Imu::ImuOutput` | `Imu_ImuOutput` |
|
||||
| **publish** | `mc/motor_control` | `MotorControl::Control` | `MotorControl_Control` |
|
||||
|
||||
`robot_control` / `RobotControl::Control` is still here and is on the way out. It is
|
||||
`control-agent`'s interpretation of the pad — sticks already scaled, buttons already
|
||||
turned into an action enum — and taking it meant the three signs that turn a stick into
|
||||
a body-frame command existed a **third** time, inside a program this repository cannot
|
||||
see, agreed with the other two by nobody. `RobotControlRaw_ControlRaw` is the pad itself,
|
||||
published by `control-pod-svc` at ~50 Hz, and the mapping comes from the policy's own
|
||||
contract (`controls.yaml`, exported into `layout.json`) exactly as it does in `play` and
|
||||
in the browser.
|
||||
|
||||
Copied from mbus `master` at `d9875988f065` (2026-09-28), every file from that one
|
||||
revision; "Provenance" below says what that refresh changed and what it means for the
|
||||
board.
|
||||
|
||||
`topics.json` is mbus's own file, trimmed to the channels in the table above plus
|
||||
`robot_control`, which leaves when its type does. It is a record, not an input: the
|
||||
topic name `dds.rs` opens is the one in the table above.
|
||||
|
||||
`MotorControl::Control` carries `MotorCmd { pos, kp, dq, kd, tau }` per joint — MIT
|
||||
impedance control, so the position target and the gains go out together and the joint
|
||||
torque is closed on the motor controller, not here.
|
||||
|
||||
**`imu/output` is not optional.** Until mbus `d9875988f065` the motor controller's
|
||||
embedded IMU reported no orientation at all. `MotorControl::ImuData` now carries a
|
||||
quaternion (`q`, with `quat_valid` and `imu_err_code`), but nothing here reads it, so a
|
||||
controller that skips this topic still has no `projected_gravity` and no tilt angle —
|
||||
both degrade to "perfectly level", silently. `projected_gravity` is an observation
|
||||
term for every policy this repository trains.
|
||||
|
||||
## Why these files are copies and not descriptions
|
||||
|
||||
**The IDL is compiled, not read.** `idlc` turns each `.idl` into a
|
||||
`dds_topic_descriptor_t`, and that descriptor determines the wire layout and the type hash
|
||||
DDS matches on. Generating it from the same IDL with the same compiler the robot's side
|
||||
used makes the two byte-compatible by construction. Retyping the structs in Rust and
|
||||
hoping they line up is the alternative, and it fails at the first `@extensibility` or
|
||||
padding difference — as a silently unpaired reader, not an error.
|
||||
|
||||
**QoS is matched, not negotiated.** A reader whose reliability or durability disagrees
|
||||
with the writer's is not an error either: DDS simply never pairs them, and the topic stays
|
||||
empty forever. `mbus_qos.xml` is therefore verbatim, trimmed to the six profiles here —
|
||||
`common`, one per channel, and `RobotControl_Control`, which nothing here opens any more
|
||||
and which has not been dropped yet — and the FSM **loads it at run time** through
|
||||
CycloneDDS's QoS provider rather than constructing QoS in code — so what this side asks
|
||||
for is literally the file the robot's side was configured with.
|
||||
|
||||
The profiles are not uniform, and one of them matters more than it looks:
|
||||
|
||||
| profile | reliability | history |
|
||||
|---|---|---|
|
||||
| `MotorControl_Control` / `_State`, `Imu_ImuOutput`, `RobotControlRaw_ControlRaw` | BEST_EFFORT | KEEP_LAST 1 |
|
||||
| **`RobotControl_Control`** | **RELIABLE** | **KEEP_ALL** |
|
||||
|
||||
`RobotControl_Control` is what this table was written for. Those operator commands cross a
|
||||
wireless link, and `mbus_config.xml` puts them on their own domain with retransmission
|
||||
tuned for it. A reader built with CycloneDDS's defaults still *pairs* with that writer —
|
||||
BEST_EFFORT requested against RELIABLE offered is legal — it just silently drops the
|
||||
reliability the tuning exists to provide. That is why the profiles are loaded rather than
|
||||
assumed, and why a test asserts what comes back.
|
||||
|
||||
The pad this controller actually opens is not on that footing.
|
||||
`RobotControlRaw_ControlRaw` is BEST_EFFORT / KEEP_LAST 1 like the motor and IMU topics —
|
||||
a 50 Hz snapshot where the newest sample is the only one worth having — and
|
||||
`control-pod-svc` publishes it on the same domain 0 the controller defaults to. The
|
||||
argument for loading the file rather than writing QoS in code is unchanged; which channel
|
||||
is the expensive one to get wrong has moved.
|
||||
|
||||
**Three domains, and the channels are split across them.** `mbus_config.xml` defines
|
||||
domain 0 (loopback only, no multicast — pure on-board IPC), domain 1 (shared memory
|
||||
over loopback; its wlan0/eth0 lines have been commented out since mbus `09a57d1`), and
|
||||
domain 2 (wlan0/eth0, SPDP-only multicast) for cross-machine traffic.
|
||||
All four channels this FSM uses are on-board; `robot_control`, the one that comes over the
|
||||
air on domain 2, is the one it stopped reading. The domain ids are configuration, not part
|
||||
of the topic list — `DdsIo::open` takes one for the motor channels and one for the
|
||||
operator channel, and `controller` passes 0 for both.
|
||||
|
||||
## Provenance
|
||||
|
||||
Every file here is from mbus master `d9875988f065` (2026-09-28), refreshed on 2026-09-29.
|
||||
Before that they had arrived in three takes -- `ea627799ffee` (2026-08-23),
|
||||
`5a934894bd9a` (2026-09-03) and `d375d737b77f` (2026-09-21) -- and each still matched
|
||||
its recorded revision byte for byte when the refresh compared them. What the refresh
|
||||
changed:
|
||||
|
||||
| file | change | on the wire |
|
||||
|---|---|---|
|
||||
| `idl_motor_control.idl` | `ImuData` gains `imu_temperature`, `q`, `quat_timestamp`, `quat_valid`, `imu_err_code`, `raw_count` | **`State` 800 → 832 bytes**, and it is `@extensibility(FINAL)` |
|
||||
| `idl_common.idl` | `TopicType` gains `ROBOT_CONTROL_CONTROL_RAW` | none: no channel here carries a `TopicType` |
|
||||
| `idl_robot_control.idl` | `ActionId` and `GripperPose` renamed after the key that fires them rather than the motion (`ACTION_JUMP` → `ACTION_A`, …), with values added to both | `robot_control` only, which nothing here opens |
|
||||
| `idl_imu.idl` | adds `Imu::YuanjiOutput` | none: `ImuOutput` is unchanged |
|
||||
| `mbus_config.xml` | domain 1 binds `lo` only | none: every channel here is on domain 0 |
|
||||
| `topics.json` | gains `robot_control_raw` | none: `dds.rs` names its own topics |
|
||||
| `mbus_qos.xml` | not touched: the six profiles kept here compare equal to upstream's | none |
|
||||
| `idl_robot_control_raw.idl` | unchanged | none |
|
||||
|
||||
Quote the revision when refreshing. A copy without one is the thing that drifts; a copy
|
||||
with one is a claim that can be checked:
|
||||
|
||||
```bash
|
||||
git -C <mbus> diff d9875988f065 -- idl/ config/ # what has changed since
|
||||
```
|
||||
|
||||
**Check this against the robot, not against the repository.** The `libmbus` built
|
||||
into the kkos staging tree on one development machine dates from 2026-06-24 and
|
||||
predates the `MotorControl::State` extra-register fields by a month, so its `State`
|
||||
is a different type from the one here. Which build the board actually runs decides
|
||||
whether the reader pairs at all -- and a mismatch does not raise, it leaves
|
||||
`mc/motor_state` empty forever, which reads downstream as "no state, hold position".
|
||||
|
||||
The 2026-09-29 refresh is that case again. The `ImuData` fields reached mbus master
|
||||
only in the 2026-09-28 merge, mbus's CHANGELOG has no release entry for them yet, and
|
||||
the board last read (`deploy/README.md`) ran libmbus 2.4.0, from 2026-09-02. A
|
||||
controller built from this tree pairs with an `mc_ctrl_driver` / `mc_forwarder` rebuilt
|
||||
against `d9875988f065` or later, and with nothing older. `check_board.py` compares the
|
||||
board's `idl_motor_control.h` with the one generated here and says which it is.
|
||||
|
||||
## Regenerating
|
||||
|
||||
`idlc` comes with CycloneDDS (`idlc <file>.idl` writes `<file>.c` and `<file>.h`). The
|
||||
build compiles the generated C in, so nothing here is checked in pre-generated: a stale
|
||||
descriptor beside a changed IDL is exactly the failure this avoids.
|
||||
|
||||
Verified on this repository's side: all five IDL files compile standalone with
|
||||
`idlc` 0.11, every `topics.json` entry resolves to a QoS profile that is present and a
|
||||
type that is defined, and a Rust round-trip over `MotorControl::Control` through
|
||||
`libddsc` returns the sample it wrote, field for field. Rechecked at the 2026-09-29
|
||||
refresh with `idlc` 11.0.1 on the development machine: `cargo test` in `deploy/fsm`
|
||||
generates all five and passes, the descriptor-size and QoS tests included, and every
|
||||
`topics.json` entry still resolves.
|
||||
|
||||
## If you are adapting this to your own robot
|
||||
|
||||
These are one robot's message definitions. Nothing above is a standard — a different
|
||||
motor controller means a different `idl_motor_control.idl`, and the FSM's DDS layer is
|
||||
the only part that should need to change with it. What is worth keeping whatever the
|
||||
hardware is: the joint order comes from the policy's own `layout.json` and is resolved by
|
||||
name against the robot's joint list, so the wire order and the policy's action order are
|
||||
never assumed to agree.
|
||||
@@ -0,0 +1,149 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<CycloneDDS xmlns="https://cdds.io/config"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
|
||||
<Domain Id="0">
|
||||
<Compatibility>
|
||||
</Compatibility>
|
||||
|
||||
<General>
|
||||
<!-- 本域只走 lo:domain 0 是纯本机 IPC,跨机流量一律走 domain 2。
|
||||
只绑 lo 的附带好处是本域彻底不受弱网影响,也不会把本机 topic
|
||||
暴露到 wlan0/eth0 上。 -->
|
||||
<Interfaces>
|
||||
<NetworkInterface name="lo"/>
|
||||
</Interfaces>
|
||||
<!-- lo 在 Linux 上没有 IFF_MULTICAST,组播本来就用不了,如实写 false,
|
||||
免得看配置的人以为本域还有组播发现这条路。 -->
|
||||
<AllowMulticast>false</AllowMulticast>
|
||||
</General>
|
||||
|
||||
<Discovery>
|
||||
<!-- 组播在 lo 上不可用,因此这个 peer 是本域唯一的发现手段(不再是
|
||||
"wlan0/eth0 没起来时的兜底")。PruneDelay=inf 让本机发现端口被
|
||||
一直探测:否则晚于 InitialLocatorPruneDelay(30s) 启动的进程,
|
||||
会因为 peer 已被剪掉而永远发现不了先启动的 participant。 -->
|
||||
<Peers>
|
||||
<Peer Address="127.0.0.1" PruneDelay="inf"/>
|
||||
</Peers>
|
||||
<LeaseDuration>10s</LeaseDuration>
|
||||
</Discovery>
|
||||
|
||||
<Internal>
|
||||
<HeartbeatInterval>2s</HeartbeatInterval>
|
||||
</Internal>
|
||||
|
||||
<Sizing>
|
||||
</Sizing>
|
||||
|
||||
<Threads>
|
||||
</Threads>
|
||||
|
||||
<Tracing>
|
||||
<!-- <Verbosity>finer</Verbosity>
|
||||
<OutputFile>cyclonedds_${CYCLONEDDS_PID}.log</OutputFile> -->
|
||||
</Tracing>
|
||||
</Domain>
|
||||
|
||||
<!-- Domain 1: Video streaming — shared memory for local, UDP for remote -->
|
||||
<Domain Id="1">
|
||||
<SharedMemory>
|
||||
<Enable>true</Enable>
|
||||
<LogLevel>info</LogLevel>
|
||||
</SharedMemory>
|
||||
|
||||
<General>
|
||||
<Interfaces>
|
||||
<NetworkInterface name="lo"/>
|
||||
<!-- <NetworkInterface name="wlan0" presence_required="false"/> -->
|
||||
<!-- <NetworkInterface name="eth0" presence_required="false"/> -->
|
||||
</Interfaces>
|
||||
<AllowMulticast>true</AllowMulticast>
|
||||
</General>
|
||||
|
||||
<Discovery>
|
||||
<!-- Probe local discovery ports forever: a process started before
|
||||
wlan0/eth0 is up selects only lo, which has no multicast on Linux;
|
||||
without a never-pruned localhost peer it can never discover
|
||||
participants started more than InitialLocatorPruneDelay (30s) later. -->
|
||||
<LeaseDuration>10s</LeaseDuration>
|
||||
</Discovery>
|
||||
|
||||
<Internal>
|
||||
<HeartbeatInterval>2s</HeartbeatInterval>
|
||||
</Internal>
|
||||
|
||||
<Tracing>
|
||||
</Tracing>
|
||||
</Domain>
|
||||
<!-- Domain 2: 跨机弱网域(只走 wlan0/eth0,无 lo)。robot_control 跑在这里,
|
||||
QoS 是 RELIABLE + KEEP_ALL(见 mbus_qos.xml 的 RobotControl_Control)。
|
||||
QoS 只声明"要可达",能不能在下一个控制周期内把丢的帧补回来,取决于本段。 -->
|
||||
<Domain Id="2">
|
||||
<Compatibility>
|
||||
</Compatibility>
|
||||
|
||||
<General>
|
||||
<Interfaces>
|
||||
<NetworkInterface name="wlan0" presence_required="false"/>
|
||||
<NetworkInterface name="eth0" presence_required="false"/>
|
||||
</Interfaces>
|
||||
<!-- spdp:组播只用于 SPDP 发现,SEDP 和用户数据一律走单播。
|
||||
Wi-Fi 的组播帧以最低基础速率发送且没有 802.11 层 ARQ(收不到不重发),
|
||||
弱信号下丢包率远高于单播;RELIABLE 虽然能靠 RTPS 重传补回来,但那是
|
||||
用一个 NACK 往返去补一条本来就不该丢的帧。改成单播后 802.11 自己的
|
||||
重传先兜一层,RTPS 重传只处理漏网的。
|
||||
本域加这一段的初衷是"多播发现",spdp 正好只保留发现所需的组播。 -->
|
||||
<AllowMulticast>spdp</AllowMulticast>
|
||||
</General>
|
||||
|
||||
<Discovery>
|
||||
<!-- Probe local discovery ports forever: a process started before
|
||||
wlan0/eth0 is up selects only lo, which has no multicast on Linux;
|
||||
without a never-pruned localhost peer it can never discover
|
||||
participants started more than InitialLocatorPruneDelay (30s) later. -->
|
||||
<LeaseDuration>10s</LeaseDuration>
|
||||
</Discovery>
|
||||
|
||||
<Internal>
|
||||
<!-- 弱网重传参数,三个默认值会直接让 RELIABLE 形同虚设:
|
||||
- HeartbeatInterval:reader 靠 HEARTBEAT 才知道自己缺样本。数据流
|
||||
连续时 HB 会搭在 DATA 上,但控制流一停(松手静止),最后几帧的
|
||||
缺失就要等下一个周期心跳 —— 2s 意味着最坏 2s 后才补上,等于
|
||||
丢了 200 个控制周期。回到 CycloneDDS 默认 100ms,并把周期心跳
|
||||
收敛到 [20ms, 500ms]。
|
||||
- NackDelay:reader 发现缺样本后延迟多久发 NACK,默认 100ms
|
||||
= 10 个控制周期,压到 10ms。
|
||||
- AutoReschedNackDelay:NACK 或其响应本身丢了之后的重试间隔,
|
||||
默认 3s,弱网下这是很可能发生的,压到 500ms。
|
||||
- Watermarks:KEEP_ALL 写端积压超过 WhcHigh 才背压。WhcAdaptive
|
||||
默认开启,会在重传期间把 WhcHigh 一路收缩到 WhcLow(默认 1kB
|
||||
≈ 4 帧),正好在丢包时把实时控制环卡在 max_blocking_time 上。
|
||||
这里关掉自适应并给固定水位:一帧 Control 约 256B(含 RTPS 开销),
|
||||
WhcHigh 64kB ≈ 250 帧 ≈ 2.5s 重传窗口,再长就是拿过期指令喂运控。
|
||||
注意 WhcAdaptive=false 时 whc_high 恒等于 WhcHighInit,两者必须
|
||||
一起设;且 WhcHighInit 必须落在 [WhcLow, WhcHigh] 内,否则报
|
||||
"Invalid watermark settings",participant 直接建不出来。 -->
|
||||
<HeartbeatInterval min="20ms" max="500ms">100ms</HeartbeatInterval>
|
||||
<NackDelay>10ms</NackDelay>
|
||||
<AutoReschedNackDelay>500ms</AutoReschedNackDelay>
|
||||
<Watermarks>
|
||||
<WhcLow>8 kB</WhcLow>
|
||||
<WhcHigh>64 kB</WhcHigh>
|
||||
<WhcHighInit>64 kB</WhcHighInit>
|
||||
<WhcAdaptive>false</WhcAdaptive>
|
||||
</Watermarks>
|
||||
</Internal>
|
||||
|
||||
<Sizing>
|
||||
</Sizing>
|
||||
|
||||
<Threads>
|
||||
</Threads>
|
||||
|
||||
<Tracing>
|
||||
<!-- <Verbosity>finer</Verbosity>
|
||||
<OutputFile>cyclonedds_${CYCLONEDDS_PID}.log</OutputFile> -->
|
||||
</Tracing>
|
||||
</Domain>
|
||||
</CycloneDDS>
|
||||
@@ -0,0 +1,346 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- Trimmed from mbus/config/mbus_qos.xml to the profiles this FSM uses.
|
||||
Copied verbatim, never retyped: QoS that does not match the other side
|
||||
makes DDS silently never pair the reader and the writer. -->
|
||||
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xmlns="http://www.omg.org/spec/DDS-XML"
|
||||
xsi:schemaLocation="http://www.omg.org/spec/DDS-XML http://www.omg.org/spec/DDS-XML/20180501/dds-xml_qos_definitions.xsd">
|
||||
<qos_library name="mbus">
|
||||
<qos_profile name="common">
|
||||
<!-- Domain -->
|
||||
<domain_participant_qos>
|
||||
</domain_participant_qos>
|
||||
<!-- Publisher -->
|
||||
<publisher_qos>
|
||||
<presentation>
|
||||
<access_scope>INSTANCE_PRESENTATION_QOS</access_scope>
|
||||
<coherent_access>false</coherent_access>
|
||||
<ordered_access>true</ordered_access>
|
||||
</presentation>
|
||||
</publisher_qos>
|
||||
<!-- Subscriber -->
|
||||
<subscriber_qos>
|
||||
<presentation>
|
||||
<access_scope>INSTANCE_PRESENTATION_QOS</access_scope>
|
||||
<coherent_access>false</coherent_access>
|
||||
<ordered_access>true</ordered_access>
|
||||
</presentation>
|
||||
</subscriber_qos>
|
||||
</qos_profile>
|
||||
|
||||
|
||||
|
||||
<qos_profile name="MotorControl_Control">
|
||||
<!-- Domain -->
|
||||
<domain_participant_qos>
|
||||
</domain_participant_qos>
|
||||
<!-- Publisher -->
|
||||
<publisher_qos>
|
||||
</publisher_qos>
|
||||
<!-- Subscriber -->
|
||||
<subscriber_qos>
|
||||
</subscriber_qos>
|
||||
<!-- Topic -->
|
||||
<topic_qos>
|
||||
<durability>
|
||||
<kind>VOLATILE_DURABILITY_QOS</kind>
|
||||
</durability>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
<transport_priority>
|
||||
<value>10</value>
|
||||
</transport_priority>
|
||||
</topic_qos>
|
||||
<!-- DataWriter -->
|
||||
<datawriter_qos>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datawriter_qos>
|
||||
<!-- DataReader -->
|
||||
<datareader_qos>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datareader_qos>
|
||||
</qos_profile>
|
||||
|
||||
<qos_profile name="MotorControl_State">
|
||||
<!-- Domain -->
|
||||
<domain_participant_qos>
|
||||
</domain_participant_qos>
|
||||
<!-- Publisher -->
|
||||
<publisher_qos>
|
||||
</publisher_qos>
|
||||
<!-- Subscriber -->
|
||||
<subscriber_qos>
|
||||
</subscriber_qos>
|
||||
<!-- Topic -->
|
||||
<topic_qos>
|
||||
<durability>
|
||||
<kind>VOLATILE_DURABILITY_QOS</kind>
|
||||
</durability>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
<transport_priority>
|
||||
<value>10</value>
|
||||
</transport_priority>
|
||||
</topic_qos>
|
||||
<datawriter_qos>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datawriter_qos>
|
||||
<datareader_qos>
|
||||
<reliability>
|
||||
<kind>BEST_EFFORT_RELIABILITY_QOS</kind>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_LAST_HISTORY_QOS</kind>
|
||||
<depth>1</depth>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datareader_qos>
|
||||
</qos_profile>
|
||||
|
||||
<!-- 机器人控制指令 100Hz(control-agent-cpp -> motion-planner)。
|
||||
通道里混着边沿触发的一次性事件(action.trigger 脉冲、jump 参数),
|
||||
丢一帧 = 丢一次跳跃/攻击,所以弱网下要求逐帧可达:
|
||||
|
||||
- RELIABLE:丢包走 RTPS 重传补齐,而不是听天由命。
|
||||
- KEEP_ALL(写端必需):CycloneDDS 的 KEEP_LAST 写端在深度溢出时会把
|
||||
**尚未被 ACK** 的样本直接从 WHC 里删掉(dds_whc.c whc_default_insert,
|
||||
hdepth>0 时无条件 prune),depth=1 意味着重传窗口只有一帧 ——
|
||||
弱网下 RELIABLE+KEEP_LAST(1) 实际退化成 BEST_EFFORT。只有 KEEP_ALL
|
||||
才会把样本留到所有 reader 确认为止。
|
||||
- KEEP_ALL(读端必需):一次补包会把积压的多帧一起递上来,
|
||||
KEEP_LAST(1) 的 RHC 会让旧帧被新帧覆盖,应用照样看不到那一帧。
|
||||
- max_blocking_time=100ms:KEEP_ALL 写端在未确认积压超过 WhcHigh 时
|
||||
会阻塞等重传追上(ddsi_transmit.c throttle_writer),超时则本帧被
|
||||
直接丢弃并返回 TIMEOUT。Domain 2 的 NackDelay 已压到 10ms,一次
|
||||
重传往返远小于 100ms,够补完;同时封顶在 10 个控制周期内,write()
|
||||
超时由 wrapper 的 noexcept 路径转成错误码,不会把实时控制环无限拖住。
|
||||
- 读端 max_samples=512:约 5s@100Hz 的补包缓冲;队列满时 RHC 拒收且
|
||||
不 ACK,写端持续重传形成背压,而不是静默丢帧。
|
||||
(写端不受 resource_limits 约束,其积压上限由
|
||||
Internal/Watermarks/WhcHigh 决定,这里的数值仅作对齐说明。)
|
||||
|
||||
本 profile 只声明"要可达",实际能不能补回来还依赖两件事:
|
||||
1) 收发两端必须同时换上新 profile:旧的 BEST_EFFORT writer 遇上
|
||||
RELIABLE reader 属于 QoS 不兼容,会直接匹配不上、一帧都收不到。
|
||||
2) 该 topic 跑在 Domain 2(跨机弱网域),mbus_config.xml 的
|
||||
Domain 2 已配套调整 AllowMulticast=spdp、HeartbeatInterval、
|
||||
NackDelay 与 Watermarks —— 详见那边的注释。若换到别的 domain
|
||||
运行,需要把同一套 Internal 参数带过去,否则 RELIABLE 名存实亡。 -->
|
||||
<qos_profile name="RobotControl_Control">
|
||||
<!-- Domain -->
|
||||
<domain_participant_qos>
|
||||
</domain_participant_qos>
|
||||
<!-- Publisher -->
|
||||
<publisher_qos>
|
||||
</publisher_qos>
|
||||
<!-- Subscriber -->
|
||||
<subscriber_qos>
|
||||
</subscriber_qos>
|
||||
<!-- Topic -->
|
||||
<topic_qos>
|
||||
<durability>
|
||||
<kind>VOLATILE_DURABILITY_QOS</kind>
|
||||
</durability>
|
||||
<reliability>
|
||||
<kind>RELIABLE_RELIABILITY_QOS</kind>
|
||||
<max_blocking_time>
|
||||
<sec>0</sec>
|
||||
<nanosec>100000000</nanosec>
|
||||
</max_blocking_time>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_ALL_HISTORY_QOS</kind>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>512</max_samples>
|
||||
<max_instances>1</max_instances>
|
||||
<max_samples_per_instance>512</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
<transport_priority>
|
||||
<value>10</value>
|
||||
</transport_priority>
|
||||
</topic_qos>
|
||||
<datawriter_qos>
|
||||
<durability>
|
||||
<kind>VOLATILE_DURABILITY_QOS</kind>
|
||||
</durability>
|
||||
<reliability>
|
||||
<kind>RELIABLE_RELIABILITY_QOS</kind>
|
||||
<max_blocking_time>
|
||||
<sec>0</sec>
|
||||
<nanosec>100000000</nanosec>
|
||||
</max_blocking_time>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_ALL_HISTORY_QOS</kind>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>512</max_samples>
|
||||
<max_instances>1</max_instances>
|
||||
<max_samples_per_instance>512</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datawriter_qos>
|
||||
<datareader_qos>
|
||||
<durability>
|
||||
<kind>VOLATILE_DURABILITY_QOS</kind>
|
||||
</durability>
|
||||
<reliability>
|
||||
<kind>RELIABLE_RELIABILITY_QOS</kind>
|
||||
<max_blocking_time>
|
||||
<sec>0</sec>
|
||||
<nanosec>100000000</nanosec>
|
||||
</max_blocking_time>
|
||||
</reliability>
|
||||
<destination_order>
|
||||
<kind>BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS</kind>
|
||||
</destination_order>
|
||||
<history>
|
||||
<kind>KEEP_ALL_HISTORY_QOS</kind>
|
||||
</history>
|
||||
<resource_limits>
|
||||
<max_samples>512</max_samples>
|
||||
<max_instances>1</max_instances>
|
||||
<max_samples_per_instance>512</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</datareader_qos>
|
||||
</qos_profile>
|
||||
|
||||
|
||||
|
||||
<qos_profile name="Imu_ImuOutput">
|
||||
<domain_participant_qos></domain_participant_qos>
|
||||
<publisher_qos></publisher_qos>
|
||||
<subscriber_qos></subscriber_qos>
|
||||
<topic_qos>
|
||||
<durability><kind>VOLATILE_DURABILITY_QOS</kind></durability>
|
||||
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
|
||||
<history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>1</depth></history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>10</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
</topic_qos>
|
||||
<datawriter_qos></datawriter_qos>
|
||||
<datareader_qos></datareader_qos>
|
||||
</qos_profile>
|
||||
|
||||
|
||||
|
||||
<!-- High-rate motion-planner debug stream: best-effort, volatile,
|
||||
keep-last-1 so a slow subscriber never back-pressures the RT loop. -->
|
||||
|
||||
<!-- PhoneControl::Message (手机 App -> control-agent-cpp): 50Hz 控制指令 / ~1Hz 状态回传。
|
||||
Best-effort, volatile, keep-last-1, 跟上 / 下的机器人控制走同样的 QoS 设定。 -->
|
||||
|
||||
<!-- Audio PCM stream: best-effort, volatile, keep-last-8 (~80 ms at
|
||||
10 ms periods). Depth > 1 so a burst of periods is not collapsed
|
||||
to a single sample (unlike video KEEP_LAST 1). -->
|
||||
<!-- RobotControlRaw::ControlRaw (control-pod-svc): 50Hz Xbox-aligned snapshot.
|
||||
Best-effort, volatile, keep-last-1 — same pattern as VR control frame. -->
|
||||
<qos_profile name="RobotControlRaw_ControlRaw">
|
||||
<domain_participant_qos></domain_participant_qos>
|
||||
<publisher_qos></publisher_qos>
|
||||
<subscriber_qos></subscriber_qos>
|
||||
<topic_qos>
|
||||
<durability><kind>VOLATILE_DURABILITY_QOS</kind></durability>
|
||||
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
|
||||
<history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>1</depth></history>
|
||||
<resource_limits>
|
||||
<max_samples>10</max_samples>
|
||||
<max_instances>1</max_instances>
|
||||
<max_samples_per_instance>10</max_samples_per_instance>
|
||||
</resource_limits>
|
||||
<transport_priority><value>10</value></transport_priority>
|
||||
</topic_qos>
|
||||
<datawriter_qos>
|
||||
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
|
||||
</datawriter_qos>
|
||||
<datareader_qos>
|
||||
<reliability><kind>BEST_EFFORT_RELIABILITY_QOS</kind></reliability>
|
||||
</datareader_qos>
|
||||
</qos_profile>
|
||||
|
||||
</qos_library>
|
||||
</dds>
|
||||
@@ -0,0 +1,85 @@
|
||||
module Common
|
||||
{
|
||||
// Topic类型
|
||||
@extensibility(FINAL)
|
||||
enum TopicType
|
||||
{
|
||||
@value(0)
|
||||
ROBOT_CONTROL_CONTROL,
|
||||
MOTOR_CONTROL_CONTROL,
|
||||
MOTOR_CONTROL_STATE,
|
||||
WEBRTC_SIGNALING_WEBRTC_SIGNAL,
|
||||
FAULT_MSG_RAISE,
|
||||
FAULT_MSG_RELEASE,
|
||||
IMU_OUTPUT,
|
||||
MOTION_PLANNER_DEBUG,
|
||||
BMS_STATUS,
|
||||
ROBOT_CONTROL_CONTROL_RAW
|
||||
};
|
||||
|
||||
// 姿态角
|
||||
@extensibility(FINAL)@nested
|
||||
struct AttitudeAngles
|
||||
{
|
||||
float roll;
|
||||
float pitch;
|
||||
float yaw;
|
||||
};
|
||||
|
||||
// 速度
|
||||
@extensibility(FINAL)@nested
|
||||
struct Velocity3D
|
||||
{
|
||||
float x;
|
||||
float y;
|
||||
float z;
|
||||
};
|
||||
|
||||
// 加速度
|
||||
@extensibility(FINAL)@nested
|
||||
struct Accelerometer3D
|
||||
{
|
||||
float x;
|
||||
float y;
|
||||
float z;
|
||||
};
|
||||
|
||||
// 磁力计
|
||||
@extensibility(FINAL)@nested
|
||||
struct Magnetometer3D
|
||||
{
|
||||
float x;
|
||||
float y;
|
||||
float z;
|
||||
};
|
||||
|
||||
// 四元数
|
||||
@extensibility(FINAL)@nested
|
||||
struct Quaternion
|
||||
{
|
||||
float w;
|
||||
float x;
|
||||
float y;
|
||||
float z;
|
||||
};
|
||||
|
||||
// 位置
|
||||
@extensibility(FINAL)@nested
|
||||
struct Position3D
|
||||
{
|
||||
float x;
|
||||
float y;
|
||||
float z;
|
||||
};
|
||||
|
||||
// 模式
|
||||
@extensibility(FINAL)
|
||||
enum RobotMode
|
||||
{
|
||||
@value(0)
|
||||
HEXAPOD, // 六足
|
||||
QUADRUPED, // 四足
|
||||
DANCE, // 舞蹈
|
||||
JUMP // 跳跃
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,40 @@
|
||||
# include "idl_common.idl"
|
||||
|
||||
module Imu
|
||||
{
|
||||
@extensibility(FINAL)
|
||||
struct ImuOutput
|
||||
{
|
||||
uint64 timestamp_ns;
|
||||
Common::Accelerometer3D acc;
|
||||
Common::Velocity3D vel;
|
||||
Common::Position3D pos;
|
||||
Common::Velocity3D gyro;
|
||||
Common::Quaternion q;
|
||||
};
|
||||
|
||||
// 原极(yuanji) AHRS 单帧原始数据
|
||||
@extensibility(FINAL)@nested
|
||||
struct YuanjiAhrs
|
||||
{
|
||||
uint32 timer; //us,原极 AHRS 内部时间
|
||||
float pitch;
|
||||
float roll;
|
||||
float yaw;
|
||||
float ax;
|
||||
float ay;
|
||||
float az;
|
||||
float gx;
|
||||
float gy;
|
||||
float gz;
|
||||
float temp;
|
||||
};
|
||||
|
||||
@extensibility(FINAL)
|
||||
struct YuanjiOutput
|
||||
{
|
||||
// SOC 侧收到该样本的时刻 (CLOCK_REALTIME, Unix 纪元纳秒)。
|
||||
uint64 timestamp_ns;
|
||||
YuanjiAhrs ahrs;
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,88 @@
|
||||
# include "idl_common.idl"
|
||||
|
||||
module MotorControl
|
||||
{
|
||||
const uint32 MC_MAX_MOTOR_CNT = 24;
|
||||
|
||||
// 扩展寄存器读回(extra-register readback):让选中的少数电机在 MIT 控制的
|
||||
// 同时额外读回指定寄存器,经 SPI/DDS 回传。最多 MC_MAX_EXTRA_MOTOR 个电机,
|
||||
// 每个最多 MC_MAX_EXTRA_REG 个寄存器。
|
||||
const uint32 MC_MAX_EXTRA_MOTOR = 5;
|
||||
const uint32 MC_MAX_EXTRA_REG = 2;
|
||||
|
||||
@extensibility(FINAL)@nested
|
||||
struct MotorCmd
|
||||
{
|
||||
float pos;
|
||||
float kp;
|
||||
float dq;
|
||||
float kd;
|
||||
float tau;
|
||||
};
|
||||
|
||||
@extensibility(FINAL)@nested
|
||||
struct MotorState
|
||||
{
|
||||
octet isvalid;
|
||||
float pos;
|
||||
float dq;
|
||||
float tau;
|
||||
octet status;
|
||||
float fpc_temper;
|
||||
float pcb_temper;
|
||||
};
|
||||
|
||||
@extensibility(FINAL)@nested
|
||||
struct ImuData
|
||||
{
|
||||
octet imu_data_valid; //imu原始数据就绪标志, 0为未就绪, 1为就绪(下同)
|
||||
octet mag_data_valid; //mag原始数据就绪标志
|
||||
uint32 imu_timestamp; //imu原始数据时间戳(us)
|
||||
uint32 mag_timestamp; //mag原始数据时间戳(us)
|
||||
Common::Accelerometer3D acc; //加速度原始数据
|
||||
Common::Velocity3D gyro; //陀螺仪原始数据
|
||||
Common::Magnetometer3D mag; //地磁原始数据
|
||||
float imu_temperature; //imu温度, 单位:摄氏度
|
||||
Common::Quaternion q; //四元数
|
||||
uint32 quat_timestamp; //四元数时间戳(us)
|
||||
octet quat_valid; //四元数就绪标志
|
||||
octet imu_err_code; //imu错误码, 0,无错误 1,imu初始化中 2,imu状态异常 3,imu未启动
|
||||
octet raw_count;
|
||||
// raw_count说明
|
||||
// imu初始化计数, 用于给上层看,当imu异常重启时, 四元数状态会重置,因此需要上层知道imu重启了。
|
||||
// 验证方法,raw_count与上一次获取的值不一致即为imu重启。
|
||||
// 数值范围: 0~7 初始化后正常为1,触发复位+1,最大为7,超过7后归0
|
||||
};
|
||||
|
||||
|
||||
@extensibility(FINAL)
|
||||
struct Control
|
||||
{
|
||||
uint64 timestamp_ns;
|
||||
uint16 sequence_id;
|
||||
uint32 motor_count;
|
||||
MotorCmd motors[MC_MAX_MOTOR_CNT];
|
||||
};
|
||||
|
||||
// 单个电机的扩展寄存器读回条目。addrs/values 一一对应,未用的槽位以 0 填充。
|
||||
@extensibility(FINAL)@nested
|
||||
struct MotorExtra
|
||||
{
|
||||
octet motor_index; // HCI 电机序号 (0-based);0xFF = 无效条目
|
||||
octet n_extra; // 该电机实际读回的寄存器个数 0..MC_MAX_EXTRA_REG
|
||||
octet addrs[MC_MAX_EXTRA_REG]; // 读回的寄存器地址(回显,便于消费者解释)
|
||||
float values[MC_MAX_EXTRA_REG]; // 对应寄存器的 f32 值
|
||||
};
|
||||
|
||||
@extensibility(FINAL)
|
||||
struct State
|
||||
{
|
||||
uint16 sequence_id;
|
||||
uint32 motor_count;
|
||||
MotorState motors[MC_MAX_MOTOR_CNT];
|
||||
ImuData imu;
|
||||
uint64 source_timestamp_ns;
|
||||
octet extra_count; // 有效 MotorExtra 条目数 0..MC_MAX_EXTRA_MOTOR
|
||||
MotorExtra extra[MC_MAX_EXTRA_MOTOR]; // 扩展寄存器读回(未用条目 motor_index=0xFF)
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,99 @@
|
||||
# include "idl_common.idl"
|
||||
|
||||
module RobotControl
|
||||
{
|
||||
enum ActionId {
|
||||
@value(0)
|
||||
ACTION_NONE,
|
||||
// 方向键(无臂激活时触发)
|
||||
ACTION_UP, // 方向键 ↑
|
||||
ACTION_DOWN, // 方向键 ↓
|
||||
ACTION_LEFT, // 方向键 ←
|
||||
ACTION_RIGHT, // 方向键 →
|
||||
// 功能键 → 一次性触发(方向取触发帧 body.linear_velocity.x/y)
|
||||
ACTION_X, // Xbox X
|
||||
ACTION_Y, // Xbox Y
|
||||
ACTION_B, // Xbox B
|
||||
ACTION_A, // Xbox A 按下触发 / LT、RT 松手触发;参数取 jump.x/y/spin
|
||||
// Menu 组合键
|
||||
ACTION_MENU_UP, // Menu + ↑
|
||||
ACTION_MENU_DOWN, // Menu + ↓
|
||||
ACTION_MENU_LEFT, // Menu + ←
|
||||
ACTION_MENU_RIGHT, // Menu + →
|
||||
ACTION_MENU // 单按 Menu
|
||||
};
|
||||
|
||||
enum GripperPose {
|
||||
@value(0)
|
||||
POSE_NONE, // 无预设(回位 / 交给 Rx 俯仰耦合)
|
||||
POSE_EXTEND_UP, // 前臂操作(方向键 ↑)
|
||||
POSE_EXTEND_DOWN, // 前臂操作(方向键 ↓)
|
||||
POSE_EXTEND_LEFT, // 前臂操作(方向键 ←)
|
||||
POSE_EXTEND_RIGHT, // 前臂操作(方向键 →)
|
||||
POSE_EXTEND_UP_LEFT, // 前臂操作(方向键 ↑+←)
|
||||
POSE_EXTEND_UP_RIGHT, // 前臂操作(方向键 ↑+→)
|
||||
POSE_EXTEND_DOWN_LEFT, // 前臂操作(方向键 ↓+←)
|
||||
POSE_EXTEND_DOWN_RIGHT // 前臂操作(方向键 ↓+→)
|
||||
};
|
||||
|
||||
@extensibility(FINAL)@nested
|
||||
struct BodyState
|
||||
{
|
||||
Common::AttitudeAngles attitude; // roll / pitch / yaw;pitch=Rx(前后推)· yaw=扭身 twist(|Ry|≤半行程)
|
||||
Common::Velocity3D linear_velocity; // x=前后 vx / y=左右 vy / z=垂直 vz(VR 身高)
|
||||
Common::Velocity3D angular_velocity; // z = 转动 yaw_rate(Ry/左右推,|Ry|>半行程);x/y 恒 0
|
||||
float height; // 目标身高(agent 锁存,[-1,1])
|
||||
};
|
||||
|
||||
@extensibility(FINAL)@nested
|
||||
struct GripperState
|
||||
{
|
||||
boolean is_valid; // 该臂是否激活(Lb/Rb,二者互斥)
|
||||
Common::Position3D position; // 末端位置目标(VR / 预设位姿数值化)
|
||||
Common::AttitudeAngles attitude; // 末端姿态目标(VR / 预设位姿数值化)
|
||||
Common::Velocity3D linear_velocity; // 末端速度(兼容保留)
|
||||
float opening; // 夹爪开合 [0,1](激活臂的 LT/RT 扳机行程)
|
||||
GripperPose pose; // 离散预设姿态(方向键;未激活时 NONE)
|
||||
};
|
||||
|
||||
// 跳跃参数(无触发字段,触发统一走 action 通道的 ACTION_A,边沿帧读取本参数)。
|
||||
// A 按下触发;方向/距离取 x/y(raw,不过死区/fast)。
|
||||
// LT/RT 松手检测触发,仅无臂激活时有效;按住时长累积转动幅度,
|
||||
// 松手帧发 ACTION_A 并携带 spin。
|
||||
@extensibility(FINAL)@nested
|
||||
struct JumpCommand
|
||||
{
|
||||
float x; // 起跳向量 前后 [-1,1](raw 左摇杆纵轴,正=前,不过死区/fast)
|
||||
float y; // 起跳向量 左右 [-1,1](raw 左摇杆横轴,正=右)
|
||||
// 方向 = atan2(y,x);距离 = min(|(x,y)|,1),0=原地跳
|
||||
float spin; // 旋转跳幅度 [-1,1]:负=左旋(LT)/正=右旋(RT),0=直线跳;由 LT/RT 按下时长映射
|
||||
};
|
||||
|
||||
// 离散动作指令。trigger 触发脉冲(连发 3 帧);脉冲期内新按键被忽略(不排队、不覆盖,
|
||||
// A1 锁存)。ACTION_A 与其它 action_id 同为单次触发、共用同一通道,同一时刻只触发一种。
|
||||
// action_id=ACTION_A 时参数取 Control.jump(x/y/spin);ACTION_B / ACTION_Y 方向取边沿帧
|
||||
// body.linear_velocity.x/y(|方向|≈0 → 默认朝机体前方)。
|
||||
@extensibility(FINAL)@nested
|
||||
struct ActionCommand
|
||||
{
|
||||
float trigger; // 触发脉冲(连发 3 帧):0=空闲,1=触发;下游上升沿只响应一次
|
||||
ActionId action_id; // 动作编号
|
||||
};
|
||||
|
||||
@extensibility(FINAL)
|
||||
struct Control
|
||||
{
|
||||
uint64 timestamp_ns;
|
||||
BodyState body;
|
||||
GripperState left_gripper;
|
||||
GripperState right_gripper;
|
||||
JumpCommand jump;
|
||||
ActionCommand action;
|
||||
};
|
||||
|
||||
@extensibility(FINAL)
|
||||
struct Debug
|
||||
{
|
||||
float time_stamp;
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,31 @@
|
||||
module RobotControlRaw
|
||||
{
|
||||
@extensibility(FINAL)
|
||||
struct ControlRaw
|
||||
{
|
||||
uint64 timestamp_ns;
|
||||
uint32 sequence_id;
|
||||
boolean connected;
|
||||
uint16 vendor_id;
|
||||
uint16 product_id;
|
||||
string profile_id;
|
||||
boolean A;
|
||||
boolean B;
|
||||
boolean X;
|
||||
boolean Y;
|
||||
boolean LB;
|
||||
boolean RB;
|
||||
boolean menu;
|
||||
boolean home;
|
||||
boolean L3;
|
||||
boolean R3;
|
||||
int8 dpad_x;
|
||||
int8 dpad_y;
|
||||
float Lx;
|
||||
float Ly;
|
||||
float Rx;
|
||||
float Ry;
|
||||
float LT;
|
||||
float RT;
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,45 @@
|
||||
{
|
||||
"_source": "mbus idl/topics.json @ master d9875988f065 (2026-09-28) -- only the channels this FSM uses",
|
||||
"channels": [
|
||||
{
|
||||
"key": "robot_control_command",
|
||||
"topic": "robot_control",
|
||||
"qos": "RobotControl_Control",
|
||||
"type": "RobotControl::Control",
|
||||
"status": "live",
|
||||
"note": "control-agent-cpp -> motion-planner"
|
||||
},
|
||||
{
|
||||
"key": "motor_control_command",
|
||||
"topic": "mc/motor_control",
|
||||
"qos": "MotorControl_Control",
|
||||
"type": "MotorControl::Control",
|
||||
"status": "live",
|
||||
"note": "motion-planner / mc_forwarder -> mc_ctrl_driver"
|
||||
},
|
||||
{
|
||||
"key": "motor_state",
|
||||
"topic": "mc/motor_state",
|
||||
"qos": "MotorControl_State",
|
||||
"type": "MotorControl::State",
|
||||
"status": "live",
|
||||
"note": "mc_ctrl_driver / mc_forwarder -> motion-planner"
|
||||
},
|
||||
{
|
||||
"key": "imu_output",
|
||||
"topic": "imu/output",
|
||||
"qos": "Imu_ImuOutput",
|
||||
"type": "Imu::ImuOutput",
|
||||
"status": "planned",
|
||||
"note": "IMU stream; QoS defined, no live peer"
|
||||
},
|
||||
{
|
||||
"key": "robot_control_raw",
|
||||
"topic": "RobotControlRaw_ControlRaw",
|
||||
"qos": "RobotControlRaw_ControlRaw",
|
||||
"type": "RobotControlRaw::ControlRaw",
|
||||
"status": "planned",
|
||||
"note": "control-pod-svc -> subscribers"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,369 @@
|
||||
# {BUNDLE}
|
||||
|
||||
This legacy policy bundle carries the robot controller and its policies for
|
||||
the available board, browser, and simulation hosts.
|
||||
|
||||
```
|
||||
{RUNTIME_FILES}controller.toml the modes, the rules that switch between them, the keys
|
||||
{MODE_FILES}reference.json recorded frames, so you can measure that you agree
|
||||
manual.en.json every key and button this bundle reads, and what each does
|
||||
bundle.json every file, its size and its SHA-256, and which host takes what
|
||||
```
|
||||
|
||||
`manual.en.json` is written by the build from the controller's own account of
|
||||
its pad and its keys, so it describes exactly the bindings here. A translation travels
|
||||
beside it as `manual.<language>.json` -- `manual.zh.json` in Chinese -- listed
|
||||
in `bundle.json`'s `manuals`. The source repository's `deploy/app.schema`
|
||||
defines the files and metadata this format accepts.
|
||||
|
||||
A reference-guided mode carries one more file, named by its contract's
|
||||
`reference` block. A mode needs its recorded-motion companion before it loads.
|
||||
|
||||
**Each host takes its own and leaves the rest.** `bundle.json`'s `runtimes`
|
||||
says which model format each host loads and where its build of the controller
|
||||
is, and each mode's `models` names its file in every format:
|
||||
|
||||
| host | models | controller |
|
||||
|---|---|---|
|
||||
| `board` (RK3576) | `.rknn`, for the NPU | `runtime/board/controller`, aarch64 |
|
||||
| `web` (a browser) | `.onnx` | `runtime/web/controller.wasm` and its `.js` binding -- the same bytes on any CPU |
|
||||
| `mjlab` (`play --app`) | `.onnx` | `runtime/mjlab/<platform>/controller.so`, the build machine's, and `runtime/mjlab/win-amd64/controller.pyd` and `runtime/mjlab/macosx-universal2/controller.so` (arm64 and x86_64), cross-built |
|
||||
|
||||
A host whose build this bundle does not carry has no entry, and
|
||||
`bundle.json`'s `notes` says how to add it. Everything else -- the FSM, the
|
||||
contracts, the recordings, the reference -- is one copy every host reads, so
|
||||
there is nothing for two hosts to disagree about.
|
||||
|
||||
On the board, from this directory:
|
||||
|
||||
```bash
|
||||
./runtime/board/controller --bundle . --dry-run # report, touch no bus
|
||||
./runtime/board/controller --bundle . --check-reference # replay the frames below
|
||||
./runtime/board/controller --bundle . --machine /etc/mjrl/machine.toml
|
||||
```
|
||||
|
||||
`--machine` is what this bundle cannot know: the DDS domains, the QoS XML and
|
||||
the few constants that describe the actuators rather than the policy.
|
||||
Everything else comes from the contracts here, and `--dry-run` prints what it
|
||||
will use -- the DDS domains, the QoS file, each mode's controls as its contract
|
||||
gives them -- and every note and stub.
|
||||
|
||||
Replay the bundle from the installed training checkout, with its registered
|
||||
task and model. The simulator applies the controller's joint targets.
|
||||
|
||||
```bash
|
||||
python scripts/play.py --app <this directory, or the .app beside it>
|
||||
python scripts/deploy.py --check-reference <this directory>
|
||||
```
|
||||
|
||||
`play --app` takes the controller built for its own machine from here rather
|
||||
than whatever `mjrl_fsm` is installed, because otherwise it runs a
|
||||
*different build of the same source*, and the two diverge the moment somebody
|
||||
rebuilds one of them.
|
||||
|
||||
The rest of this file is for a browser.
|
||||
|
||||
**The controller here is the one the robot runs.** Not the same logic
|
||||
reimplemented from the same contract -- the same compiled object, built from
|
||||
`deploy/fsm` at commit `{COMMIT}`. That is the point of shipping it: three
|
||||
implementations of one observation agreed by people reading each other's code,
|
||||
and a sign corrected in one left the other two wrong with every other number
|
||||
still correct.
|
||||
|
||||
**And you can measure that your host runs it the same way.** `reference.json`
|
||||
is a recorded run of this controller: per frame, the robot state that went in,
|
||||
the observation it built, the action the policy returned and the joint targets
|
||||
it published. Replay it on load:
|
||||
|
||||
(It says your host agrees with the machine that built the bundle. It says
|
||||
nothing about whether that machine should be trusted -- see the last section.)
|
||||
|
||||
```js
|
||||
const report = JSON.parse(fsm.checkReference(referenceJson))
|
||||
// {frames, inferences, observation, target, divergent, modeMismatches}
|
||||
```
|
||||
|
||||
`observation` and `target` should be **0**. They are the same f32 arithmetic in
|
||||
the same order as every other host, so anything above about 1e-6 is a real
|
||||
difference, not rounding. `divergent` names terms your host is expected to
|
||||
source differently from training -- the robot measures nearly everything a
|
||||
simulator computes -- with the worst difference each one actually showed, which
|
||||
is normally 0 because the replay feeds every host the same recorded input. This
|
||||
paragraph used to single out `joint_torque` as reconstructed on the robot from
|
||||
the commanded PD and differing there by about 2 N*m. **That was wrong**: the
|
||||
servo reports torque, the controller has always read it, and the term is
|
||||
measured on the robot too -- the 2 N*m was the gap between a real number and a
|
||||
reconstruction of it that nothing needed. `modeMismatches` is never acceptable:
|
||||
a cascade landing in another state is a different robot, not a tolerance.
|
||||
|
||||
Your inference backend is a separate question and not covered by that call.
|
||||
Run each frame's `obs` through your own model and compare against its `act`:
|
||||
onnxruntime-web should match to ~1e-6, a quantised `.rknn` differs in the third
|
||||
decimal. Keeping the two apart is deliberate -- a wrong observation and a
|
||||
quantised model produce the same symptom, a robot that walks slightly wrong.
|
||||
|
||||
---
|
||||
|
||||
## Loading it
|
||||
|
||||
```js
|
||||
import init, { WebFsm } from "./runtime/web/controller.js";
|
||||
|
||||
await init({
|
||||
module_or_path: await (await fetch("./runtime/web/controller.wasm")).arrayBuffer(),
|
||||
});
|
||||
|
||||
const fsm = new WebFsm(
|
||||
fsmToml, // controller.toml, as text
|
||||
JSON.stringify(contracts), // {{ "<mode>": <that mode's .json> }}
|
||||
JSON.stringify(robot), // below
|
||||
nowMicroseconds,
|
||||
JSON.stringify(trajectories), // {{ "<mode>": <its *.trajectory.json text> }}
|
||||
);
|
||||
```
|
||||
|
||||
`trajectories` is only for a mode whose contract carries a `reference` block --
|
||||
a recorded motion the policy is driven by. Read the file that block names out of
|
||||
this bundle and pass its text; omit the argument entirely for a bundle of
|
||||
ordinary policies. A mode that needs one and does not get it is refused by name,
|
||||
because the alternative is building its reference terms out of zeros, which is a
|
||||
motionless recording that a standing robot tracks perfectly.
|
||||
|
||||
```js
|
||||
const trajectories = {{}};
|
||||
for (const [mode, contract] of Object.entries(contracts)) {{
|
||||
if (contract.reference) {{
|
||||
trajectories[mode] = await (await fetch(`./${{contract.reference.file}}`)).text();
|
||||
}}
|
||||
}}
|
||||
```
|
||||
|
||||
`robot` is what the *host* knows and no policy does:
|
||||
|
||||
```js
|
||||
{
|
||||
joint_names: [...], // wire_joint_order, from any mode's contract
|
||||
joint_pos_lo: [...], // per joint; prefer the contract's joint_limits
|
||||
joint_pos_hi: [...], // and fall back to your own model's
|
||||
output_rate_hz: 50, // how often you will call tick()
|
||||
gait_period: 0.32, // optional; only for a contract whose gait_phase
|
||||
gait_gate_threshold: 0.05 // carries no clock of its own -- each mode's
|
||||
// contract names its period or its cadence law
|
||||
}
|
||||
```
|
||||
|
||||
`joint_limits` is a `{joint: [lo, hi]}` map in each mode's contract, in the
|
||||
same radians as `default_joint_pos`, and it is what the robot itself clamps
|
||||
to. Preferring it over your own model keeps the two hosts agreeing about the
|
||||
one bound that cannot be observed going wrong -- a clamp wide enough never to
|
||||
fire looks exactly like a clamp that works. Older bundles do not carry it; fall
|
||||
back to your model then, and say which you used.
|
||||
|
||||
Every failure is thrown with the reason in it. There is no fallback: a host
|
||||
that cannot build this should say so rather than run a robot on defaults
|
||||
nobody chose.
|
||||
|
||||
## One control step
|
||||
|
||||
```js
|
||||
fsm.set_state(q, qd, tau, quat, gyro, nowUs);
|
||||
fsm.setSignal("base_lin_vel", linVelBody); // everything else you can compute
|
||||
if (stopPressed) fsm.letGo(nowUs); // your Stop button, Escape -- once
|
||||
for (const e of keyEvents) fsm.setKeyCode(e.code, e.down, e.repeat, nowUs);
|
||||
if (pad) { // a real pad, or on-screen sticks
|
||||
for (const [axis, v] of Object.entries(pad.axes)) fsm.setAxis(axis, v);
|
||||
for (const b of PAD_BUTTONS) fsm.setPad(b, pad.pressed.includes(b));
|
||||
fsm.padFrame(nowUs);
|
||||
}
|
||||
fsm.set_command(0, 0, 0, nowUs); // keeps the operator fresh
|
||||
|
||||
const mode = fsm.tick(nowUs);
|
||||
if (mode !== undefined) {
|
||||
fsm.resume(await yourOnnxSession(mode).run(fsm.observation()));
|
||||
}
|
||||
publish(fsm.positions()); // joint targets, wire order
|
||||
```
|
||||
|
||||
`tick` returns the mode needing an inference, or nothing when the step is
|
||||
finished -- a warm start, a mode-switch ramp or a hold still publishes, and
|
||||
`positions()` is ready either way.
|
||||
|
||||
**The split is not an abstraction.** A wasm module cannot call a JavaScript
|
||||
model and return inside one synchronous call, and the loop publishes faster
|
||||
than it infers: `tick` only asks when the active mode's `control_hz` says so.
|
||||
Running the policy at your own rate aliases it *and* rescales the per-tick
|
||||
joint rate limit.
|
||||
|
||||
## What must arrive exactly right, because none of it fails loudly
|
||||
|
||||
| argument | what it must be |
|
||||
|---|---|
|
||||
| `q` `qd` `tau` | **wire order**, i.e. `joint_names` above. Not your model's order and not the policy's; the three are paired by name, and pairing by index is wrong from the fifth joint on |
|
||||
| `quat` | body orientation `(w, x, y, z)` |
|
||||
| `gyro` | angular velocity **in the body frame**. An IMU gives this; a simulator usually gives you the world frame and a rotation matrix, so rotate: `out[a] = m[a]*w[0] + m[3+a]*w[1] + m[6+a]*w[2]` |
|
||||
| `tau` | applied joint torque. A simulator has the solver's; the robot reads the servo's. If *your* host measures none, reconstruct what your PD commanded and say which you did |
|
||||
| `vx` `vy` `wz` | never read by a mode with a controls block of its own (`takesRawInput()`); for a mode with none, **physical units**, m/s and rad/s |
|
||||
| `setAxis` values | the pad as the robot's pad service reports it: sticks `[-1, 1]` with a 0.15 deadband already applied **and rescaled**, triggers `[0, 1]`. No sign, no scale -- those are the bundle's |
|
||||
| `nowUs` | microseconds, monotonic. The module has no clock; freshness and rate are decided from what you pass |
|
||||
|
||||
If your simulator reports a 6-vector velocity, check its order before using it.
|
||||
MuJoCo's is `(angular, linear)`; reading it the other way gives a gyro that
|
||||
reports how fast the robot is walking, and nothing anywhere says so.
|
||||
|
||||
## Handing over a person's input
|
||||
|
||||
**Hand over what you see, and nothing you decided.** What a stick or a key
|
||||
means is the running mode's: its `controller` block says which control drives
|
||||
which command axis, how a stick's travel splits, which button holds a layer,
|
||||
which key pushes which way, where a released axis rests -- and the controller
|
||||
inside this wasm executes it. The pad and the keyboard are two paths, each
|
||||
bound independently. The same input can adjust height in one mode and pitch
|
||||
in another. Send the physical inputs to the controller so it can apply the
|
||||
active mode's mapping.
|
||||
|
||||
So, every frame:
|
||||
|
||||
- **Every key, both edges.** `fsm.setKeyCode(event.code, down, repeat, nowUs)`
|
||||
with the browser's own `KeyboardEvent.code`. The dictionary compiled into
|
||||
this wasm knows which codes are keys it has a name for; the return value says
|
||||
whether anything here binds that one, which is what tells you to
|
||||
`preventDefault` it. `repeat` is the event's own flag, and says the key is
|
||||
still down: a key held pushes what it drives for as long as it is held --
|
||||
full after the mode's `full_after_s` -- and lets it back the moment its
|
||||
`keyup` arrives, so the up edge matters as much as the down. Shift, Alt and
|
||||
Ctrl are keys like any other, left and right each by its own code; Alt pressed
|
||||
on its own moves a Windows browser's focus to its menu unless you
|
||||
`preventDefault` it.
|
||||
- **The pad, whenever there is one.** `fsm.setAxis(name, value)` for `Lx` `Ly`
|
||||
`Rx` `Ry` `LT` `RT`, `fsm.setPad(button, down)` for every button, then
|
||||
`fsm.padFrame(nowUs)`. On-screen sticks are a pad too: report them as `Lx`
|
||||
`Ly` `Rx` `Ry`. Pad and keyboard are both live and the one touched last
|
||||
drives; the controller arbitrates, so you need not.
|
||||
- **Every quantity you can compute.** `fsm.setSignal(name, values)`, whether or
|
||||
not today's policy observes it -- `base_lin_vel`, body frame, m/s, is the one
|
||||
read now. A policy that needs something a robot cannot measure is refused at
|
||||
export, but a web-only bundle may, and a page that passed only what today's
|
||||
bundles need is a page tomorrow's outgrows without saying so.
|
||||
|
||||
- **Your own stop, once.** `fsm.letGo(nowUs)` when your page's Stop button or
|
||||
Escape is pressed: the operator let go of everything, exactly as the bundle's
|
||||
release button does. A host control rather than a pad button, because you
|
||||
have no button of the bundle's to press on somebody's behalf -- which one
|
||||
releases is the contract's to say. Call it before that step's keys, so a key
|
||||
pressed after the stop is a new press. A key still down keeps pushing until
|
||||
its `keyup`, and a page that lost focus mid-press may never be sent one, so
|
||||
without this a Stop button stops nothing.
|
||||
|
||||
`fsm.takesRawInput()` is true when any mode has a controls block. Those modes
|
||||
never read `set_command`; a mode with none does, and sending it keeps the
|
||||
operator fresh either way, so sending it every frame is harmless.
|
||||
`fsm.inputs()` returns `{rawInput, keys: [{name, code}], controls, signals}` for
|
||||
a page that wants to say which keys do something: `controls` is each mode's own
|
||||
account of its bindings -- the lines the robot's controller prints before it
|
||||
enables the motors -- on one line, the modes that read the pad alike named
|
||||
together: `claw_left, claw_right: … | locomotion: …`. Show it as given;
|
||||
describing the bundle yourself would be a second copy of its bindings.
|
||||
|
||||
`fsm.bindings()` returns `[{name, pad, key, on, with, from, leaves}]` -- the mode
|
||||
switches, the events and the leaves: everything the operator can ask for
|
||||
*between* modes, and which control asks it. A binding is one device's, a pad
|
||||
button or a key, and two with one name -- the pad's `A` and the keyboard's
|
||||
Space, both `jump` -- share one latch. Those are the same keys and buttons,
|
||||
reported the same way:
|
||||
|
||||
```js
|
||||
fsm.setKeyCode("KeyW", true, false, nowUs); // the browser's name for it
|
||||
fsm.setPad("A", down); // the robot pad service's names
|
||||
```
|
||||
|
||||
Both lists come from one dictionary, `controller/vocabulary.json` -- schema
|
||||
`kk-control-vocabulary/2`, compiled into this wasm, so there is no file to fetch
|
||||
-- and a binding naming something outside it is refused when the bundle loads.
|
||||
`fsm.setKey(name, down)` takes the dictionary's own name (`key_w`, `keypad_1`)
|
||||
for a host that already has it.
|
||||
|
||||
Not "was pressed". `on` is one of nine gestures and two of them need the
|
||||
release:
|
||||
|
||||
| `on` | fires |
|
||||
|---|---|
|
||||
| `rise` | the tick it goes down |
|
||||
| `fall` | the tick it comes back up |
|
||||
| `hold` | every tick it is down |
|
||||
| `toggle` | down until pressed again |
|
||||
| `single` `double` `triple` `quadruple` `quintuple` | that many presses, each within `click_window_ms` of the last; switches like `toggle` |
|
||||
|
||||
The click gestures are counted on the `nowUs` you hand `tick`, so hand it the
|
||||
page's real clock (`performance.now() * 1000`): a clock that runs fast or slow
|
||||
narrows or widens the window a person clicks in.
|
||||
|
||||
So press-to-arm and release-to-go is one control bound twice, and a host that
|
||||
only reported presses could not drive half the vocabulary.
|
||||
|
||||
Pad names are `A` `B` `X` `Y` `LB` `RB` `menu` `home` `L3` `R3`, plus
|
||||
`dpad_up` / `dpad_down` / `dpad_left` / `dpad_right`. Translate your Gamepad API
|
||||
index into one of those and the two hosts agree about what was pressed. `view`
|
||||
is on the pad and is deliberately not among them.
|
||||
|
||||
`with` names what must be held: a pad button for a pad binding, `ctrl`,
|
||||
`shift` or `alt` -- either of its two keys -- for a key. A modifier is consumed
|
||||
by what it modifies, and while it is held a binding without it on the same
|
||||
control stays quiet, so `menu` alone, `menu`-plus-a-direction and the
|
||||
direction alone can mean three things without one firing on the way to
|
||||
another. `from` lists the modes a binding may be pressed in -- pressed
|
||||
anywhere else it does nothing, though a latch it put on can always be switched
|
||||
off by it -- and `leaves` the modes a binding lets go of, after which the
|
||||
cascade falls back to its default mode.
|
||||
|
||||
Not every binding switches a mode, and two cases are worth expecting. **A
|
||||
recorded motion ends.** A jump or a dance is a thing that finishes, and when it
|
||||
does the controller hands the robot back: `tick` releases that mode's latch for
|
||||
you and the cascade's last `always` rule catches it, so there is one way out of
|
||||
a mode rather than two. A toggle you have drawn as lit will go out on its own --
|
||||
and in a bundle that latches one mode at a time, when another switch is pressed
|
||||
-- so read `fsm.mode()`, not your own copy of the latch. **And a motion's `go`,
|
||||
where a bundle has one, is a moment, not a mode.** A push-off is 40 ms wide, so
|
||||
the button that starts it is read by the mode itself rather than by a
|
||||
transition rule. It is an ordinary binding otherwise: it is in `bindings()`, it
|
||||
has a gesture. Report it like every other binding.
|
||||
|
||||
## Two things worth putting on screen
|
||||
|
||||
`fsm.mode()` is the active mode; `fsm.is_running_policy()` is whether it is
|
||||
actually producing policy output. They differ through a mode-switch ramp,
|
||||
which waits on the **measured pose** rather than a timer and can wait
|
||||
indefinitely if the gains cannot converge. That is deliberate -- the
|
||||
alternative is starting a policy from a pose it never trained around -- but a
|
||||
robot holding still looks broken unless you say which it is.
|
||||
|
||||
## If you are running this from an upload
|
||||
|
||||
**Nothing here has checked this controller.** Your own copy, if you ship one,
|
||||
is held against your own implementation by whatever test you wrote for it.
|
||||
This one arrived with the upload. Say so where the person can see it.
|
||||
|
||||
Running an uploaded `.onnx` and running an uploaded wasm plus JavaScript are
|
||||
not the same risk. A graph executed by onnxruntime is arithmetic over tensors;
|
||||
`controller.js` is a program with your page's authority. If you load it,
|
||||
**give it an opaque origin** -- an `<iframe sandbox="allow-scripts">` with no
|
||||
`allow-same-origin`, talking over a `MessagePort`. Measured in Chrome from
|
||||
inside such a frame: `origin` is `"null"`, `document.cookie`, `localStorage`,
|
||||
`indexedDB` and `parent.document` all throw `SecurityError`, and a credentialed
|
||||
`fetch` back to the host origin fails. A round trip of one 411-float
|
||||
observation had a p95 of 0.1 ms against a 20 ms budget at 50 Hz.
|
||||
|
||||
Isolation does not bound CPU or memory. A watchdog that tears the frame down
|
||||
when it does not answer within a few control periods is the other half.
|
||||
|
||||
## One source, compiled three times
|
||||
|
||||
`runtime/web/controller.wasm`, a native extension under `runtime/mjlab/`, an
|
||||
aarch64 binary in `runtime/board/`. Same logic, same commit, which
|
||||
`bundle.json`'s `runtimes` records for each. The two native ones are
|
||||
**platform-specific** where the wasm is not, so each records the platform it
|
||||
was built for, and a host on another one should say so rather than fail with a
|
||||
loader error nobody can read.
|
||||
|
||||
---
|
||||
|
||||
Built by `scripts/deploy.py` from source revision `{COMMIT}`.
|
||||
Generated
+587
@@ -0,0 +1,587 @@
|
||||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
|
||||
[[package]]
|
||||
name = "aho-corasick"
|
||||
version = "1.1.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "autocfg"
|
||||
version = "1.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
|
||||
|
||||
[[package]]
|
||||
name = "bindgen"
|
||||
version = "0.70.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f49d8fed880d473ea71efb9bf597651e77201bdd4893efe54c9e5d65ae04ce6f"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"cexpr",
|
||||
"clang-sys",
|
||||
"itertools",
|
||||
"log",
|
||||
"prettyplease",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"regex",
|
||||
"rustc-hash",
|
||||
"shlex 1.3.0",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bitflags"
|
||||
version = "2.13.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
|
||||
|
||||
[[package]]
|
||||
name = "bumpalo"
|
||||
version = "3.20.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
|
||||
|
||||
[[package]]
|
||||
name = "cc"
|
||||
version = "1.4.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "005ec2760ca554fae18df7a11195552ec576cd665632a881bc011d5bb2fd4d80"
|
||||
dependencies = [
|
||||
"find-msvc-tools",
|
||||
"shlex 2.0.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cexpr"
|
||||
version = "0.6.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766"
|
||||
dependencies = [
|
||||
"nom",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cfg-if"
|
||||
version = "1.0.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
|
||||
|
||||
[[package]]
|
||||
name = "clang-sys"
|
||||
version = "1.9.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "157a8ba7b480713b56f4c09fd13fc3e0a22a5dfab8097ba61cbc5feef950788a"
|
||||
dependencies = [
|
||||
"glob",
|
||||
"libc",
|
||||
"libloading",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "either"
|
||||
version = "1.18.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
|
||||
|
||||
[[package]]
|
||||
name = "equivalent"
|
||||
version = "1.0.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
|
||||
|
||||
[[package]]
|
||||
name = "find-msvc-tools"
|
||||
version = "0.1.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
|
||||
|
||||
[[package]]
|
||||
name = "glob"
|
||||
version = "0.3.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
|
||||
|
||||
[[package]]
|
||||
name = "hashbrown"
|
||||
version = "0.17.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
|
||||
|
||||
[[package]]
|
||||
name = "heck"
|
||||
version = "0.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
|
||||
|
||||
[[package]]
|
||||
name = "indexmap"
|
||||
version = "2.14.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
|
||||
dependencies = [
|
||||
"equivalent",
|
||||
"hashbrown",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "indoc"
|
||||
version = "2.0.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
|
||||
dependencies = [
|
||||
"rustversion",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "itertools"
|
||||
version = "0.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
|
||||
dependencies = [
|
||||
"either",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "itoa"
|
||||
version = "1.0.18"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
|
||||
|
||||
[[package]]
|
||||
name = "libc"
|
||||
version = "0.2.189"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
|
||||
|
||||
[[package]]
|
||||
name = "libloading"
|
||||
version = "0.8.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "log"
|
||||
version = "0.4.34"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
|
||||
|
||||
[[package]]
|
||||
name = "memchr"
|
||||
version = "2.8.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
|
||||
|
||||
[[package]]
|
||||
name = "memoffset"
|
||||
version = "0.9.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
|
||||
dependencies = [
|
||||
"autocfg",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "minimal-lexical"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
|
||||
|
||||
[[package]]
|
||||
name = "mjrl-fsm"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"bindgen",
|
||||
"cc",
|
||||
"pyo3",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"toml",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nom"
|
||||
version = "7.1.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
"minimal-lexical",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
|
||||
|
||||
[[package]]
|
||||
name = "portable-atomic"
|
||||
version = "1.15.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
|
||||
|
||||
[[package]]
|
||||
name = "prettyplease"
|
||||
version = "0.2.37"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "proc-macro2"
|
||||
version = "1.0.107"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
|
||||
dependencies = [
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyo3"
|
||||
version = "0.22.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f402062616ab18202ae8319da13fa4279883a2b8a9d9f83f20dbade813ce1884"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"indoc",
|
||||
"libc",
|
||||
"memoffset",
|
||||
"once_cell",
|
||||
"portable-atomic",
|
||||
"pyo3-build-config",
|
||||
"pyo3-ffi",
|
||||
"pyo3-macros",
|
||||
"unindent",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyo3-build-config"
|
||||
version = "0.22.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b14b5775b5ff446dd1056212d778012cbe8a0fbffd368029fd9e25b514479c38"
|
||||
dependencies = [
|
||||
"once_cell",
|
||||
"python3-dll-a",
|
||||
"target-lexicon",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyo3-ffi"
|
||||
version = "0.22.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9ab5bcf04a2cdcbb50c7d6105de943f543f9ed92af55818fd17b660390fc8636"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"pyo3-build-config",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyo3-macros"
|
||||
version = "0.22.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0fd24d897903a9e6d80b968368a34e1525aeb719d568dba8b3d4bfa5dc67d453"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"pyo3-macros-backend",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyo3-macros-backend"
|
||||
version = "0.22.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "36c011a03ba1e50152b4b394b479826cad97e7a21eb52df179cd91ac411cbfbe"
|
||||
dependencies = [
|
||||
"heck",
|
||||
"proc-macro2",
|
||||
"pyo3-build-config",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "python3-dll-a"
|
||||
version = "0.2.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d80ba7540edb18890d444c5aa8e1f1f99b1bdf26fb26ae383135325f4a36042b"
|
||||
dependencies = [
|
||||
"cc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "quote"
|
||||
version = "1.0.47"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "regex"
|
||||
version = "1.13.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
|
||||
dependencies = [
|
||||
"aho-corasick",
|
||||
"memchr",
|
||||
"regex-automata",
|
||||
"regex-syntax",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "regex-automata"
|
||||
version = "0.4.18"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
|
||||
dependencies = [
|
||||
"aho-corasick",
|
||||
"memchr",
|
||||
"regex-syntax",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "regex-syntax"
|
||||
version = "0.8.11"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
|
||||
|
||||
[[package]]
|
||||
name = "rustc-hash"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
|
||||
|
||||
[[package]]
|
||||
name = "rustversion"
|
||||
version = "1.0.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
|
||||
|
||||
[[package]]
|
||||
name = "serde"
|
||||
version = "1.0.229"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
|
||||
dependencies = [
|
||||
"serde_core",
|
||||
"serde_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "serde_core"
|
||||
version = "1.0.229"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
|
||||
dependencies = [
|
||||
"serde_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "serde_derive"
|
||||
version = "1.0.229"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "serde_json"
|
||||
version = "1.0.151"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
|
||||
dependencies = [
|
||||
"itoa",
|
||||
"memchr",
|
||||
"serde",
|
||||
"serde_core",
|
||||
"zmij",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "serde_spanned"
|
||||
version = "0.6.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
|
||||
dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "shlex"
|
||||
version = "1.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
|
||||
|
||||
[[package]]
|
||||
name = "shlex"
|
||||
version = "2.0.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
|
||||
|
||||
[[package]]
|
||||
name = "syn"
|
||||
version = "2.0.119"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "syn"
|
||||
version = "3.0.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "target-lexicon"
|
||||
version = "0.12.16"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1"
|
||||
|
||||
[[package]]
|
||||
name = "toml"
|
||||
version = "0.8.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
|
||||
dependencies = [
|
||||
"serde",
|
||||
"serde_spanned",
|
||||
"toml_datetime",
|
||||
"toml_edit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "toml_datetime"
|
||||
version = "0.6.11"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
|
||||
dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "toml_edit"
|
||||
version = "0.22.27"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
|
||||
dependencies = [
|
||||
"indexmap",
|
||||
"serde",
|
||||
"serde_spanned",
|
||||
"toml_datetime",
|
||||
"toml_write",
|
||||
"winnow",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "toml_write"
|
||||
version = "0.1.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
|
||||
|
||||
[[package]]
|
||||
name = "unicode-ident"
|
||||
version = "1.0.24"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
|
||||
|
||||
[[package]]
|
||||
name = "unindent"
|
||||
version = "0.2.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen"
|
||||
version = "0.2.128"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"once_cell",
|
||||
"rustversion",
|
||||
"wasm-bindgen-macro",
|
||||
"wasm-bindgen-shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro"
|
||||
version = "0.2.128"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed"
|
||||
dependencies = [
|
||||
"quote",
|
||||
"wasm-bindgen-macro-support",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro-support"
|
||||
version = "0.2.128"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a"
|
||||
dependencies = [
|
||||
"bumpalo",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.5",
|
||||
"wasm-bindgen-shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-shared"
|
||||
version = "0.2.128"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e"
|
||||
dependencies = [
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-link"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "winnow"
|
||||
version = "0.7.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zmij"
|
||||
version = "1.0.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
|
||||
@@ -0,0 +1,85 @@
|
||||
[package]
|
||||
name = "mjrl-fsm"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "Run an exported policy on a Rockchip RK3576 NPU, over the robot's DDS bus"
|
||||
license = "Apache-2.0"
|
||||
|
||||
[lib]
|
||||
# `cdylib` is what `wasm32` needs to produce a module wasm-bindgen can process;
|
||||
# `rlib` is what everything else links against, including this crate's own
|
||||
# tests. Both, so one manifest serves the browser and the robot.
|
||||
crate-type = ["cdylib", "rlib"]
|
||||
|
||||
# The robot's host. `required-features` rather than a `#[cfg]` inside the file:
|
||||
# a machine without the device feature has no bus and no NPU, and `cargo build`
|
||||
# there should build the library rather than fail on a binary it cannot use.
|
||||
[[bin]]
|
||||
name = "controller"
|
||||
path = "src/bin/controller.rs"
|
||||
required-features = ["device"]
|
||||
|
||||
[dependencies]
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
toml = "0.8"
|
||||
# Only for the browser host. Optional so the device build and `cargo test` do
|
||||
# not pull a JS-facing dependency they have no use for.
|
||||
wasm-bindgen = { version = "0.2", optional = true }
|
||||
# Only for the `play` host. `abi3-py310` builds one extension that every Python
|
||||
# this repository supports (3.10-3.13) can import, so the artifact does not have
|
||||
# to be rebuilt per interpreter.
|
||||
pyo3 = { version = "0.22", features = ["abi3-py310", "extension-module"], optional = true }
|
||||
|
||||
[build-dependencies]
|
||||
cc = "1"
|
||||
# The IDL structs are nested and change with the robot; generating the Rust
|
||||
# mirrors from the headers idlc just emitted is the only way they cannot drift.
|
||||
bindgen = "0.70"
|
||||
|
||||
[features]
|
||||
# `device` is everything that talks to hardware: the DDS bus and, under `rknn`,
|
||||
# the NPU. It is on by default so the ordinary build and `cargo test` are
|
||||
# unchanged -- but it is a **feature** rather than the only way to build, because
|
||||
# the layers above it have three consumers and only one of them is a robot.
|
||||
#
|
||||
# The other two are why this line exists:
|
||||
#
|
||||
# cargo check --target wasm32-unknown-unknown --no-default-features
|
||||
#
|
||||
# passing is the proof that observation, action, contract and FSM contain no
|
||||
# syscall. A browser has no DDS and no NPU; if the core cannot be built without
|
||||
# them then it cannot be the core, and the web simulator would have to reimplement
|
||||
# it -- which is the second implementation this whole path exists to avoid.
|
||||
default = ["device"]
|
||||
device = []
|
||||
|
||||
# The RKNN runtime is an aarch64 .so; a host build has no NPU to talk to. Off by
|
||||
# default so `cargo test` works on a development machine, and turned on by the
|
||||
# aarch64 profile in .cargo/config.toml. Without it every mode falls back to the
|
||||
# stub engine, exactly as the C++ controller does when built without RKNN.
|
||||
rknn = ["device"]
|
||||
|
||||
# The browser host: `WebFsm`, the JS-facing wrapper around `Controller`. Off by
|
||||
# default, because it only makes sense for `--target wasm32-unknown-unknown`.
|
||||
web = ["dep:wasm-bindgen"]
|
||||
|
||||
# The `play` host: an importable extension module wrapping the same `Controller`
|
||||
# the browser and the robot run. Off by default; `extension-module` links
|
||||
# against no libpython, so a build with it on cannot be run as an ordinary
|
||||
# binary and `cargo test` would fail to link.
|
||||
#
|
||||
# `generate-import-lib` is for the Windows build, cross-compiled in
|
||||
# `docker-build.sh`: a Windows DLL has to link against `python3.dll` through an
|
||||
# import library, which a Linux machine has no Python to take it from, and
|
||||
# without this the link fails with `cannot find -lpython3`. It makes one from
|
||||
# the stable ABI's symbol list instead. On every other target it does nothing.
|
||||
py = ["dep:pyo3", "pyo3/generate-import-lib"]
|
||||
|
||||
[profile.release]
|
||||
# A missed control deadline is a real-time failure, so the loop is built for
|
||||
# latency rather than size: no unwinding tables to walk, one codegen unit so the
|
||||
# hot path can inline across modules.
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
panic = "abort"
|
||||
@@ -0,0 +1,603 @@
|
||||
# fsm — the shape it is being given
|
||||
|
||||
**Status: built.** The core is there -- the premise, the observation port, the
|
||||
FSM cascade and the two-phase tick -- and so are all three hosts. Each section
|
||||
below says which. Read [`README.md`](README.md) for the crate as a whole.
|
||||
|
||||
---
|
||||
|
||||
## The problem
|
||||
|
||||
The same three steps — **observation → inference → execution** — now have three
|
||||
places they have to run:
|
||||
|
||||
| | observation from | inference on | execution to |
|
||||
|---|---|---|---|
|
||||
| `kk-rl-mjlab`'s `play` | mjlab's simulation | ONNX, on the host | mjlab actuators |
|
||||
| BEUNLIMITED's web sim | MuJoCo WASM | ONNX, in the browser | the same actuators |
|
||||
| the robot | servos and IMU | RKNN, on the NPU | DDS to the motor controller |
|
||||
|
||||
That logic existed **three times**: this crate in Rust, `rl-wbc-fsm` in C++,
|
||||
`policy.ts` in TypeScript. They agreed, and they agreed the expensive way — by
|
||||
people reading each other's code. A sign corrected in one of them left the other
|
||||
two driving a robot that was wrong with every other number still correct.
|
||||
|
||||
The fix is not to pick one and delete the others: two of the three consumers
|
||||
cannot run the third's language. The fix is that **the logic is one crate and
|
||||
the three differences are ports**. The C++ is no longer one of them — this crate
|
||||
ships its own binary, `src/bin/controller.rs`, and `rl-wbc-fsm` is kept as
|
||||
reading: its priority cascade, its per-mode contracts and its safe-state rule
|
||||
are all here, and where a decision below says "the C++ does X", that is where X
|
||||
came from.
|
||||
|
||||
## The premise, verified
|
||||
|
||||
Everything below assumes the observation/action/FSM logic contains no syscall,
|
||||
because a browser cannot provide one. That is now checked rather than assumed:
|
||||
|
||||
```bash
|
||||
cargo check --target wasm32-unknown-unknown --no-default-features # passes
|
||||
cargo check # passes
|
||||
cargo test # passes
|
||||
```
|
||||
|
||||
`device` is a feature (default on, so the ordinary build is unchanged) covering
|
||||
`dds` and `rknn`; `build.rs` compiles in the task hooks and then returns without it,
|
||||
so `idlc`, `bindgen` and `cc` never run for a target that could not use them. `obs.rs`,
|
||||
`action.rs`, `layout.rs` and `types.rs` — 1282 lines when this was written, 2666 now —
|
||||
compiled to `wasm32` **unmodified**. They were already clean; the feature only
|
||||
stopped the two device modules from dragging them down.
|
||||
|
||||
If that check ever fails, the core has stopped being the core, and the web
|
||||
simulator is back to reimplementing it.
|
||||
|
||||
---
|
||||
|
||||
## Shape
|
||||
|
||||
```
|
||||
controller (pure: no I/O, no clock of its own, one build for every target)
|
||||
layout the policy's contract, parsed exists
|
||||
obs RawState -> observation vector exists
|
||||
action action vector -> joint targets exists
|
||||
types the values that cross a port exists
|
||||
source what a state source can supply, and how exists
|
||||
config modes and transition rules exists
|
||||
vocabulary the words a pad and a keyboard can say exists
|
||||
trajectory the recorded motion a reference-guided mode reads exists
|
||||
fsm the ordered cascade exists
|
||||
control the two-phase tick, the regimes, the per-mode runtimes exists
|
||||
operator a task's controls: pad and keys -> each mode's command exists
|
||||
guide what each control does in each mode, as data exists
|
||||
hook a task's own code at fixed points of a mode's loop exists
|
||||
reference a recorded run replayed, and where hosts differ exists
|
||||
|
||||
hosts (one per environment, outside the core, each in its own language)
|
||||
device Rust, this crate's `device` feature exists
|
||||
web JavaScript, via wasm-bindgen exists
|
||||
play Python, via pyo3 exists
|
||||
|
||||
the bundle format (std::fs, so the one module a browser does not compile)
|
||||
bundle open a directory, check it, build a host from it exists
|
||||
```
|
||||
|
||||
The core never calls out. A host calls in, is told what to do, and does it. That
|
||||
is forced by wasm and it is the right shape anyway: it makes the whole of the
|
||||
control logic a pure function of its inputs, which is the only way the three
|
||||
environments can be compared at all.
|
||||
|
||||
---
|
||||
|
||||
## The observation port
|
||||
|
||||
### Ports are not interchangeable, and pretending they are is the failure
|
||||
|
||||
The obvious design is one trait with three implementations. It is wrong, and the
|
||||
evidence is a term already in the shipped contract.
|
||||
|
||||
`joint_torque` carries one value per observed joint and history frame. The host
|
||||
determines where each value comes from.
|
||||
|
||||
| environment | source |
|
||||
|---|---|
|
||||
| mjlab | `data.actuator_force` — what the solver applied |
|
||||
| MuJoCo WASM | the same |
|
||||
| the robot | `MotorControl_State.motors[i].tau` — what the servo reports |
|
||||
|
||||
That last row used to read `tau = kp*(t − q) − kd*qd`, clamped to
|
||||
`effort_limit`, with the note that the servo does not report torque. **It does**,
|
||||
and `dds.rs::take_state` had been reading it into `state.tau` the whole time;
|
||||
the claim came from the C++ controller (`action.cpp:100-103`) and was never
|
||||
true of this robot. Declaring the term reconstructed made the controller
|
||||
re-derive a number it already had, and that is the one term the reference
|
||||
replay ever disagreed on — 2.146 N·m worst case.
|
||||
|
||||
The reconstruction is still there, in `applied_torque`, and is still what a
|
||||
host without servo torque would use. Which one a host takes is decided by what
|
||||
its `SourceCapability` declares, not by a branch in the loop.
|
||||
|
||||
Two consequences the naive design hides:
|
||||
|
||||
1. **"observation → inference → execution" is not a chain.** It has a feedback
|
||||
edge, and the edge closes in a different place per environment.
|
||||
2. **A term's name does not tell you what it is.** Two runs can agree on every
|
||||
dimension, scale and index and still differ in what the numbers mean.
|
||||
|
||||
### So a source declares what it can supply, and how
|
||||
|
||||
**Built — `src/source.rs`.**
|
||||
|
||||
```rust
|
||||
enum Provenance {
|
||||
Simulated, // the physics engine computed it: ground truth
|
||||
Measured, // a sensor read it
|
||||
Reconstructed, // this controller computed it from what it commanded
|
||||
Estimated, // a state estimator produced it
|
||||
}
|
||||
```
|
||||
|
||||
A source declares a `{term -> Provenance}` map. `types.rs` already has half of
|
||||
this, in the one place it was unavoidable:
|
||||
|
||||
```rust
|
||||
/// Body linear velocity. Almost never available: it needs a state estimator,
|
||||
/// not an IMU. `has_lin_vel` says whether it means anything.
|
||||
pub lin_vel: [f32; 3],
|
||||
pub has_lin_vel: bool,
|
||||
```
|
||||
|
||||
Generalising it is the change. A contract asking for a term no source supplies is
|
||||
**refused at load**, not discovered at tick 40 000.
|
||||
|
||||
### The check produces a list, not a verdict
|
||||
|
||||
The valuable output is not "ok". It is the terms whose provenance differs from
|
||||
the one they had in training:
|
||||
|
||||
```
|
||||
locomotion on `device`:
|
||||
joint_torque trained Simulated, here Measured
|
||||
base_lin_vel trained Simulated, here Estimated
|
||||
(6 terms unchanged)
|
||||
```
|
||||
|
||||
**That list is the sim2real gap, enumerated.** Nobody can see it today. It costs
|
||||
one field per term in the exported contract — every term kk-rl-mjlab exports is
|
||||
`Simulated`, so producing it is nearly free, and the value is entirely on the
|
||||
consuming side.
|
||||
|
||||
---
|
||||
|
||||
## The inference port
|
||||
|
||||
### Two phases, because the host owns the model
|
||||
|
||||
In the browser the model runs in onnxruntime-web, which is JavaScript. A wasm
|
||||
module cannot call it and return within one synchronous call. So the tick splits:
|
||||
|
||||
**Built — `src/control.rs`.** The cascade decides *which* mode; this decides what
|
||||
a tick does inside one.
|
||||
|
||||
```rust
|
||||
enum Step {
|
||||
/// No inference this tick — warm start, mode-switch ramp, or a hold state.
|
||||
/// The targets are final; read them from `command()`.
|
||||
Hold,
|
||||
/// The host must run `observation()` through this mode's model and call
|
||||
/// `resume`.
|
||||
Infer { mode: String },
|
||||
}
|
||||
|
||||
fn tick(&mut self, input: &TickInput<'_>, now: Micros) -> Result<Step, Error>;
|
||||
fn observation(&self) -> &[f32]; // borrowed: no allocation per tick
|
||||
fn resume(&mut self, action: &[f32]) -> Result<&MotorCommand, Error>;
|
||||
fn command(&self) -> &MotorCommand; // valid after either phase
|
||||
```
|
||||
|
||||
`Hold` carries nothing, because the command is readable either way and a payload on
|
||||
one arm would be a second place to read it from. `TickInput` is the bundle of things
|
||||
only the host knows — the state, the command, whether each is fresh, the latched
|
||||
buttons, each mode's operator and its task's controls — so everything derivable from
|
||||
them is derived here instead of subtly differently in three places.
|
||||
|
||||
This is **not** a concession to wasm. `rl-wbc-fsm` already works this way: the
|
||||
output loop runs at 1 kHz and inference at the active contract's `control_hz`
|
||||
(50 for locomotion, 200 for jump), with the last decoded target held and an EMA
|
||||
moving toward it in between. Splitting the tick only writes that fact into the
|
||||
type instead of leaving it in a timer comparison.
|
||||
|
||||
It also makes the decisive property testable: `tick` is a pure function of its
|
||||
`TickInput` and `now`, so a recorded episode replays bit-exactly. The C++ has
|
||||
this only for `jump_gate.hpp`, and says why — *"pure, time-injected, unit-tested
|
||||
in isolation from the DDS control loop"*.
|
||||
|
||||
### Rate lives in the core, not the host
|
||||
|
||||
`control_hz` comes from the active mode's contract, and the core decides which
|
||||
ticks infer. A host that decided for itself would be free to get it wrong, and
|
||||
running a policy at the wrong rate aliases it *and* re-scales
|
||||
`joint_pos_rate_limit`, which is per-tick.
|
||||
|
||||
`Bundle::output_rate_hz` returns the fastest mode's rate for the host loop and
|
||||
shared output filter. Each mode keeps its inference period. With modes at 50
|
||||
and 200 Hz, the loop runs at 200 Hz and the slower mode infers every fourth
|
||||
tick. The output filter derives its coefficient from the cutoff and loop rate.
|
||||
|
||||
### A non-inferring engine is a state, not an error
|
||||
|
||||
`rl-wbc-fsm` ships a `StubEngine` returning zeros, and `is_stub()` so the loop
|
||||
refuses to call that "running". The interlock that matters:
|
||||
|
||||
> **"Zero action == hold still" is not universal.** It holds only while the
|
||||
> decode baseline is the home pose. For a reference-residual policy a zero action
|
||||
> means "track the recording exactly" — a full open-loop jump.
|
||||
|
||||
So the stub must be visible to the FSM, not hidden behind the port. It is:
|
||||
`rknn::make_engine` falls back to a stub **with a reason**, `DeviceHost::stub_modes`
|
||||
reports every one, `controller` prints them at startup and again the first time
|
||||
such a mode runs, and `Outcome::Inferred` carries `stub` so the loop cannot call
|
||||
it running.
|
||||
|
||||
And a mode whose decode uses a reference **refuses to run at all** while its
|
||||
engine is a stub. `DeviceHost::note_stub` hands the fact it owns — this engine
|
||||
is a stub — to `Controller::interlock_stub_engine`, which owns the rule, because
|
||||
the contract is there: a `reference` block with `residual_action` bars the mode,
|
||||
anything else does not. A barred mode takes the path a stale-feedback tick
|
||||
already takes — keep the mode, hold steadily — rather than inventing a second
|
||||
way to refuse, and `stub_modes` says which mode and why.
|
||||
|
||||
Only a residual is barred. For an ordinary policy a stub's zeros **are** the home
|
||||
pose held, which is what `stub_modes` promises and is true; barring those too
|
||||
would make a bundle pending conversion useless for no safety gain. The test
|
||||
carries both directions.
|
||||
|
||||
---
|
||||
|
||||
## The execution port
|
||||
|
||||
The core produces joint targets in **wire order** and nothing else. What differs:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| mjlab | write actuator targets; mjlab's `<position>` actuators close the loop |
|
||||
| web | the same, through the WASM model |
|
||||
| device | a DDS `MotorCommand` with `pos`/`kp`/`kd` and `tau = 0` — the MCU closes the torque loop |
|
||||
|
||||
Two invariants the port may not round off:
|
||||
|
||||
- **Wire order is not policy order.** `layout.rs` pairs them by joint *name*;
|
||||
the two never share a type, deliberately. A wrong order is a robot that runs
|
||||
and is wrong.
|
||||
- **The clamps are per-tick.** `joint_pos_rate_limit` is rad *per tick*, so a
|
||||
host running at a different rate silently changes it. It belongs to the core.
|
||||
|
||||
---
|
||||
|
||||
## The FSM
|
||||
|
||||
### States are data; transitions are an ordered list
|
||||
|
||||
`rl-wbc-fsm` has no state enum and no transition table: states are strings in
|
||||
TOML, and transitions are a priority cascade in `fsm.cpp`, recomputed every tick.
|
||||
The cascade order is the design — safety before operator intent, recovery before
|
||||
hand-off — and a `condition -> state` map would lose exactly that.
|
||||
|
||||
**Built — `src/config.rs`, `src/fsm.rs`.**
|
||||
|
||||
So transitions become an **ordered rule list**, first match wins:
|
||||
|
||||
```toml
|
||||
[[fsm.rule]] when = "feedback_stale" enter = "safe"
|
||||
[[fsm.rule]] when = "tilted" enter = "safe"
|
||||
[[fsm.rule]] when = "in_state:safe" enter = "@initial"
|
||||
[[fsm.rule]] when = "warm_start_reached" enter = "@warm_start_ref"
|
||||
[[fsm.rule]] when = "in_state:warmstart" enter = "@stay"
|
||||
[[fsm.rule]] when = "button:jump" enter = "jump"
|
||||
[[fsm.rule]] when = "button:claw_left" enter = "claw_left"
|
||||
[[fsm.rule]] when = "always" enter = "locomotion"
|
||||
```
|
||||
|
||||
Two rules about the rules:
|
||||
|
||||
- **`when` is a closed vocabulary, not an expression language.** An unknown word
|
||||
fails the load. An expression language here cannot be validated, cannot be
|
||||
tested, and would need a parser inside the wasm module.
|
||||
- **The last rule must be `always`**, checked at load. "No rule matched" then
|
||||
cannot happen, and a test can assert the cascade is total.
|
||||
|
||||
Starting vocabulary — every entry is something the C++ already decides on:
|
||||
|
||||
| `when` | true when |
|
||||
|---|---|
|
||||
| `always` | always; required as the last rule |
|
||||
| `feedback_stale` | no state sample within `state_timeout_ms` |
|
||||
| `tilted` | tilt exceeds `tilt_limit`, unless the active state sets `skip_tilt_check` |
|
||||
| `command_stale` | no operator command within `command_timeout_ms` |
|
||||
| `in_state:<name>` | the active state is `<name>` |
|
||||
| `warm_start_reached` | the active state is a warm start and the measured pose reached its target |
|
||||
| `button:<id>` | the operator's latched mode equals `<id>` |
|
||||
| `gripper_active` | either gripper is engaged — see below |
|
||||
|
||||
`@initial` and `@warm_start_ref` are indirections into `[fsm]`. `@stay` matches
|
||||
and changes nothing — needed because a cascade without it quietly claims every
|
||||
tick ends somewhere new. A warm start ignores the operator until it has finished,
|
||||
which the C++ says with a bare `return` mid-function; here it is a rule, visible
|
||||
beside the order it depends on. The first draft had no `@stay` and walked a robot
|
||||
away mid-ramp, because the cascade fell through to `always`. The test that caught
|
||||
it is `the_ordinary_path`.
|
||||
|
||||
`gripper_active` is the one word that parses and can never be true. Its state came
|
||||
from `robot_control`, which this controller stopped reading when it started reading
|
||||
the pad directly, and the pad carries no gripper — every host passes `false`. So a
|
||||
config *using* it is refused at load, for the same reason a rule naming a button
|
||||
nothing latches is: a rule that can never fire looks exactly like one that merely
|
||||
has not. The word stays in the vocabulary, because the vocabulary is what a file
|
||||
may say and this will be true again when something supplies it.
|
||||
|
||||
### Entry conditions stay on the state
|
||||
|
||||
`skip_tilt_check` suspends the attitude fallback **only while the state is
|
||||
already active** — the tick that enters it is still checked, so the robot cannot
|
||||
launch from an already-tipped pose. That is a guard on the state, not an edge, and
|
||||
it stays where the C++ put it. A config whose safe state sets it fails to load,
|
||||
for the reason the C++ gives: *"exempting it would leave no attitude protection
|
||||
anywhere."*
|
||||
|
||||
### Switching modes ramps; it does not blend
|
||||
|
||||
Entering a mode with a model:
|
||||
|
||||
1. record the pose at entry;
|
||||
2. direct-PD ramp toward **that model's own default pose**, at the ramp gains,
|
||||
not the policy's;
|
||||
3. hand off only when the measured pose is within `pose_reach_tol` of it —
|
||||
**not when the ramp duration elapses**. Soft gains mean the hand-off waits
|
||||
indefinitely and the robot soft-holds. That is by design; raise the gains.
|
||||
|
||||
Set `mode_switch_ramp_s`, `pose_reach_tol`, `ramp_kp` and `ramp_kd` for the
|
||||
target's actuator and feedback units. The ramp limits the step in commanded
|
||||
pose when a mode changes.
|
||||
|
||||
### Per-mode, and per-mode only
|
||||
|
||||
The split the C++ found, kept verbatim:
|
||||
|
||||
> This file holds only what is GLOBAL across policies. Each FSM mode's I/O
|
||||
> contract (obs/action layout, home pose, action_scale, gains) is loaded at
|
||||
> runtime from that policy's layout, shipped next to its model.
|
||||
|
||||
One mode names one model file. Everything about how that policy is driven comes
|
||||
from its own contract. The TOML adds only bench knobs, composed rather than
|
||||
overriding:
|
||||
|
||||
```
|
||||
effective kp = contract.kp * state.kp_scale + state.kp_offset
|
||||
```
|
||||
|
||||
The trained gain stays in the contract; the bench adjustment stays in the config;
|
||||
neither can be mistaken for the other.
|
||||
|
||||
Modes are built **up front**, in `Controller::new`, and so are their engines in
|
||||
`Bundle::into_host`. This paragraph said "lazily on first entry"; building on
|
||||
first entry would put every way a contract can be refused — a term this host
|
||||
cannot supply, a missing recording, a model that will not load — behind an
|
||||
operator pressing a button, on a robot that is already standing. Start-up is
|
||||
where a refusal is cheap. A switch costs nothing either way.
|
||||
|
||||
### A mode may be driven by a recording
|
||||
|
||||
A velocity policy needs nothing but its contract: the command is three numbers
|
||||
and every observation is measurable now. A reference-guided one is not runnable
|
||||
without the motion it was trained against, so the motion travels with it — a
|
||||
`reference` block in the contract, and a companion `<name>.trajectory.json`
|
||||
beside it that the bundler copies in and `trajectory.rs` reads. The contract
|
||||
declares whether actions are residuals on the recording's commanded targets
|
||||
and whether the recording begins on mode entry or on an operator event.
|
||||
|
||||
Three things about it are load-time refusals rather than runtime surprises,
|
||||
because each is silent when wrong. A contract naming a recording that is not
|
||||
beside it is refused when the bundle opens — the alternative is a mode that loads
|
||||
and fails at its first tick, which is the right answer at the wrong moment. A
|
||||
table in a different joint order than the robot's is refused, because a permuted
|
||||
trajectory is a robot moving the wrong limbs and nothing downstream can see it.
|
||||
And a residual declared on any baseline other than `q_cmd` is refused rather than
|
||||
read off `q`. The measured state table `q` feeds the policy preview. The
|
||||
recorded controller targets in `q_cmd` supply the residual baseline. The two
|
||||
tables can have different sample rates and lengths.
|
||||
|
||||
**A recorded motion ends**, which is the one way it is unlike a gait.
|
||||
`Controller::take_completed` names the mode whose recording has played out, the
|
||||
host releases that mode's button on its own latch, and the cascade's final
|
||||
`always` rule catches the robot. The controller does not leave the mode itself:
|
||||
that would be a second thing deciding modes, which is the whole complaint against
|
||||
the C++'s jump latch.
|
||||
|
||||
### What a command is differs per mode
|
||||
|
||||
From `config.hpp`:
|
||||
|
||||
> Policies do not agree on what a command is: locomotion wants
|
||||
> (lin_vel_x, lin_vel_y, yaw_rate), jump wants (jump_x, jump_y, jump_go).
|
||||
|
||||
So a mode may declare its own `command_terms`, falling back to the global list.
|
||||
The controls half of that has since moved: a task's `controls.yaml` is a list of
|
||||
commands, one entry per command term an operator drives, and the contract
|
||||
carries it as `operator_controller/2`. A mode can bind velocity and posture
|
||||
commands to the same devices. `src/operator.rs` reads those bindings, and the
|
||||
FSM's `command_terms` still supplies the order of the `commands` observation.
|
||||
|
||||
---
|
||||
|
||||
## Deliberately not copied
|
||||
|
||||
- `walk_enter_speed`, `walk_exit_speed`, `walk_exit_hold_s` — parsed by the C++,
|
||||
read by nothing. Vestigial stand↔walk hysteresis.
|
||||
- The jump one-shot latch works by rewriting the operator's mode before the FSM
|
||||
sees it. It is correct and well-tested, but it means the FSM is not the only
|
||||
thing deciding modes. Here `button:jump` is an ordinary cascade rule, and the
|
||||
motion starts when the mode is entered, once the switch-in ramp is done. A
|
||||
recording that wants a person to pick the moment declares a `go_event` instead:
|
||||
a binding the mode reads by name rather than a transition, because a push-off
|
||||
40 ms wide is an event inside a mode and not a mode. Leaving is the cascade's too:
|
||||
when the recording plays out, `Controller::take_completed` names the mode, the
|
||||
host releases that button on its own latch, and the final `always` rule catches
|
||||
the robot. One way out, rather than a second one written beside it.
|
||||
- The controller reads raw pad input. The deployment manifest defines mode
|
||||
switches, and each task's `controls.yaml` defines inputs within a mode.
|
||||
|
||||
---
|
||||
|
||||
## What exists today
|
||||
|
||||
`README.md` describes the crate as it stands: a state machine whose states are a
|
||||
file, three hosts that call it, and `src/bin/controller.rs`, which on the robot is
|
||||
the whole program. This section used to read "no state machine, no `main.rs`" and
|
||||
to warn that the README's "two modes" were `rl-wbc-fsm`'s; neither is true now.
|
||||
|
||||
Built as of this document:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| the `device` feature split and the wasm check | `Cargo.toml`, `build.rs`, `lib.rs` |
|
||||
| provenance, capability, divergence list | `src/source.rs` |
|
||||
| modes and the ordered rule list, with its load-time checks | `src/config.rs` |
|
||||
| the words a control may be named, read from the dictionary | `src/vocabulary.rs` |
|
||||
| the cascade, and the guards that belong to a state | `src/fsm.rs` |
|
||||
| the two-phase tick, four regimes, per-mode runtimes | `src/control.rs` |
|
||||
| each mode's reading of the pad and keys, command shaping | `src/operator.rs` |
|
||||
| the pad's manual as data (`pad_guide`) | `src/guide.rs` |
|
||||
| a task's deploy hook, compiled in from `tasks/**/deploy/lib.rs` by `build.rs` | `src/hook.rs` |
|
||||
| the recorded motion a reference-guided mode is driven by | `src/trajectory.rs` |
|
||||
| a recorded run replayed, and where a host differs | `src/reference.rs` |
|
||||
| the bundle format, one reader for every host with a filesystem | `src/bundle.rs` |
|
||||
| the robot's host: freshness, the button latch, engine dispatch | `src/device.rs` |
|
||||
| the browser's host: the JS-facing `WebFsm` | `src/web.rs` |
|
||||
| `play`'s host: the pyo3 `Fsm` | `src/py.rs` |
|
||||
| the robot's program: the loop, the two clocks, the shutdown path | `src/bin/controller.rs` |
|
||||
|
||||
156 tests. Two of them are control groups, because both of these modules replace
|
||||
something that was previously written into code and would pass every obvious
|
||||
assertion while ignoring the file: `the_cascade_is_the_file_s_rather_than_agreeing_with_it`
|
||||
moves the tilt rule below the operator's button and requires a tilted robot to
|
||||
jump, and `the_check_reads_what_training_recorded_rather_than_assuming_it` trains
|
||||
a term as reconstructed and requires simulation to be reported as the divergence.
|
||||
|
||||
`control.rs` earned a third test the hard way. Its first `publish` used
|
||||
`cmd.pos` as both the standing target and the filtered output, so after one
|
||||
sample the filter converged on itself and stopped -- at 50 Hz on a 1 kHz loop,
|
||||
nineteen ticks in twenty did nothing. On a robot that reads as a controller
|
||||
that feels sluggish, with nothing logged.
|
||||
`the_output_filter_starts_where_the_robot_is` requires convergence to the
|
||||
decoded target, and measured 0.0295 against 0.25.
|
||||
|
||||
`device.rs`'s two latch tests were run against deliberately broken versions
|
||||
before being trusted. A latch acting on the trigger's *level* rather than its
|
||||
rising edge fails `a_held_trigger_is_one_request` -- but only because the press
|
||||
is four frames long: at three, level-acting toggles an odd number of times and
|
||||
ends latched, passing for exactly the wrong reason. Dropping `latch.forget()`
|
||||
on a stale command fails `a_latched_button_does_not_outlive_the_operator`.
|
||||
|
||||
## Hosts
|
||||
|
||||
`device.rs` is the first, and the one whose I/O already existed. It owns what
|
||||
the core may not: a clock (so `feedback_stale` and `command_stale` can be
|
||||
decided), the engines, and the latch that turns the operator's trigger pulses
|
||||
into a `button:<name>` the rules can match. `pump` is compiled against the real
|
||||
`DdsIo` -- four calls, and that compile is the check that the shapes still fit.
|
||||
|
||||
`[[fsm.button]]` names one thing the operator can ask for and which control asks
|
||||
it: a pad button or a keyboard key, one device per entry, and two entries with
|
||||
one name share its latch -- the pad's A and the keyboard's Space are both the
|
||||
jump. It also carries the gesture (`rise`, `fall`, `hold`, `toggle`, or a click
|
||||
count `single` … `quintuple` counted inside `[fsm] click_window_ms`), an optional
|
||||
`with` modifier (a pad button, or `ctrl` / `shift` / `alt` for a key), `from`
|
||||
(the states it may be pressed in) and `leaves` (latches it lets go of). Pad
|
||||
buttons and keyboard keys each have their own bindings.
|
||||
|
||||
Every one of its load-time checks is a binding that loads and then does nothing,
|
||||
or does two things: a rule naming a button no entry declares; an entry with
|
||||
neither a pad button nor a key, or with both; two entries on the same control
|
||||
with the same gesture; a `pad` or a `with` the robot's service does not publish;
|
||||
a `key` the dictionary does not list; a `with` that is not a modifier of its
|
||||
entry's device; a `from` naming no state; a `leaves` that is not a moment or
|
||||
names nothing that latches; a control used as a modifier that is also bound to
|
||||
`hold`; a binding no rule reacts to; and any key `[fsm]` does not read.
|
||||
`[fsm] exclusive` makes the latches one at a time, the last switch pressed
|
||||
being the mode.
|
||||
|
||||
**`event = true` is the exception to the last of those.** A control read by a
|
||||
*mode* rather than by a rule — a recorded motion's `go`, matched against the
|
||||
contract's `reference.go_event` — has no rule reacting to it, which is exactly
|
||||
the shape of a dead binding. So it is declared rather than inferred, and
|
||||
`Controller::new` checks it in both directions: an `event` no contract names, and
|
||||
a `go_event` no button declares, are each a control that loads and does nothing.
|
||||
|
||||
The words themselves come from `controller/vocabulary.json`, schema
|
||||
`kk-control-vocabulary/2`, `include_str!`-ed so a board carries no file and a
|
||||
browser fetches nothing. It was `kk-gamepad-vocabulary/1` while the pad was all
|
||||
it described; `keys` joined it because `key` was the one field nothing checked —
|
||||
`pad` and `with` were held against the dictionary from the start and `key` took
|
||||
any string at all, so a typo loaded, bound nothing, and looked exactly like a
|
||||
rule that had not fired.
|
||||
|
||||
In a bundle the halves come from two files on purpose. `deploy/manifests.json` says
|
||||
which button or key switches into which mode, and from which modes, and its
|
||||
top-level `buttons` list names bindings for other events, such as a recording's
|
||||
`go`. Each task's `controls.yaml` defines the bindings inside a mode. Neither is wrong on its own when both pick the same control, and
|
||||
neither can catch it, so `Bundle::open` is where the sum is checked
|
||||
(`operator::check_switches`, which every host's operators run again): opening the
|
||||
bundle is the first place both exist.
|
||||
|
||||
`web.rs` is the second, and the one the two-phase tick exists for: the model runs
|
||||
in onnxruntime-web, which is JavaScript, so a wasm module cannot call it and
|
||||
return inside one synchronous call. Everything crosses as a flat `Float32Array`
|
||||
-- no structs per tick -- and the configuration crosses as text once, because a
|
||||
browser has no filesystem.
|
||||
|
||||
```bash
|
||||
cargo build --release --target wasm32-unknown-unknown --no-default-features --features web
|
||||
wasm-bindgen --target web --no-typescript --out-dir <dir> \
|
||||
target/wasm32-unknown-unknown/release/mjrl_fsm.wasm
|
||||
```
|
||||
|
||||
776 KB of wasm and 19 KB of glue, against MuJoCo's 10.2 MB and onnxruntime's
|
||||
14 MB in the same page.
|
||||
|
||||
**It found a bug on first contact with a real file, which is the point.** Pointed
|
||||
at an exported `layout.json`, it refused: *observation term 'velocity_commands'
|
||||
is not implemented*. `rl-wbc-fsm` has mapped that name to the builder's
|
||||
`commands` since `contract.cpp:27` was written -- locomotion names its command
|
||||
block `velocity_commands`, the older jumps `jump_command` / `jump_go`, and all
|
||||
three feed one term. This crate never had the mapping, so it would have refused
|
||||
every policy in circulation, including `rl-wbc-fsm`'s own
|
||||
`locomotion.isaac_layout.json`. It went unnoticed because the tests build their
|
||||
own contracts and named the term `commands` -- what the builder wanted. Two
|
||||
implementations agreeing with themselves.
|
||||
|
||||
`py.rs` is the third, and the one that makes an FSM something you can look at before
|
||||
you ship it: until it existed, the only way to find out what a `controller.toml` did
|
||||
was to bundle it and upload it. `Fsm(config, contracts, robot, now_us=0,
|
||||
trajectories=None, operators=False)` — `trajectories` is `{mode: text}` for the
|
||||
recordings those contracts name, optional rather than a positional every caller
|
||||
would pass `{}` for; with `operators`, keys (`set_key`) and whole pad frames
|
||||
(`set_pad_frame`) are read through each mode's own controls, as on the robot, which
|
||||
is what `play --app` runs. State crosses as plain sequences of `f32`; errors are
|
||||
`ValueError` with the reason in them. Buttons are `set_key(key, down)` and
|
||||
`set_pad(button, down)` — **state, both edges**, because `rise` and `fall` are both
|
||||
in the vocabulary and a one-shot "was pressed" could only drive half of it — and
|
||||
`pad_guide()` is what a host draws the pad from; `bindings` lists the mode switches.
|
||||
The earlier `press_key` and `key_bindings` are gone for exactly that reason: a
|
||||
one-shot press cannot produce a `fall`.
|
||||
|
||||
What `device.rs` does not include is a binary. That is `src/bin/controller.rs`
|
||||
now: the 1 kHz loop, the two clocks it must not derive one from the other, the
|
||||
`--dry-run` and `--check-reference` paths, and a shutdown that leaves through the
|
||||
safe state by telling the FSM the bus has gone stale rather than by writing a
|
||||
second shutdown behaviour. None of it can be tested off the board.
|
||||
|
||||
## Open
|
||||
|
||||
- **Provenance in the exported contract.** Needs a field per observation term in
|
||||
kk-rl-mjlab's exporter. Cheap to produce, and the whole value of the divergence
|
||||
list depends on it.
|
||||
- **Per-mode command vocabulary**, above.
|
||||
@@ -0,0 +1,121 @@
|
||||
# Cross-build the FSM for the RK3576 board, from a host of any kind.
|
||||
#
|
||||
# Cross-compiling, not emulating. Running an arm64 container under qemu needs a
|
||||
# `binfmt` registration carrying the `F` flag, which is host-wide state a build
|
||||
# should not be quietly changing -- and it is far slower. This image is x86_64
|
||||
# and targets aarch64 through a normal cross toolchain.
|
||||
#
|
||||
# bash deploy/fsm/docker-build.sh
|
||||
#
|
||||
# ## Why ubuntu:22.04 and not something newer
|
||||
#
|
||||
# **The base image's glibc is a hard floor on the board.** The RK3576 image is
|
||||
# Ubuntu 22.04 (glibc 2.35); a binary linked against Debian bookworm's 2.36 dies
|
||||
# on the board with `version GLIBC_2.36 not found` -- at startup, on the robot,
|
||||
# nowhere near the build that caused it. 22.04 here matches the rootfs the board
|
||||
# actually runs.
|
||||
|
||||
FROM ubuntu:22.04
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential cmake git curl ca-certificates pkg-config \
|
||||
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-dev-arm64-cross \
|
||||
clang libclang-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# CycloneDDS is needed twice, for two different reasons:
|
||||
#
|
||||
# - **x86_64**, for `idlc`. It is a code generator that runs on the build
|
||||
# machine, so it must be a host binary whatever the target is.
|
||||
# - **aarch64**, for `libddsc.so`, which the binary links against.
|
||||
#
|
||||
# The ref is an argument because the shared-library soname has to match the
|
||||
# board's. The board carries `libddsc.so.11`; a CycloneDDS that produces a
|
||||
# different soname builds fine here and then cannot find its library at runtime.
|
||||
# `docker-build.sh` checks the built soname rather than trusting this default.
|
||||
ARG CYCLONEDDS_REF=master
|
||||
ARG CYCLONEDDS_URL=https://github.com/eclipse-cyclonedds/cyclonedds.git
|
||||
|
||||
RUN git clone --depth 1 --branch "${CYCLONEDDS_REF}" "${CYCLONEDDS_URL}" /src/cyclonedds
|
||||
|
||||
# 1) Host build -- only idlc is kept.
|
||||
RUN cmake -S /src/cyclonedds -B /build/host \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_INSTALL_PREFIX=/opt/cyclonedds-host \
|
||||
-DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=ON \
|
||||
&& cmake --build /build/host -j"$(nproc)" \
|
||||
&& cmake --install /build/host \
|
||||
&& rm -rf /build/host
|
||||
|
||||
# 2) Target build -- the library the FSM links against. `-DBUILD_IDLC=OFF`
|
||||
# because an aarch64 idlc could not run here anyway.
|
||||
RUN cmake -S /src/cyclonedds -B /build/target \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_INSTALL_PREFIX=/opt/cyclonedds-aarch64 \
|
||||
-DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
|
||||
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
|
||||
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ \
|
||||
-DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=OFF \
|
||||
&& cmake --build /build/target -j"$(nproc)" \
|
||||
&& cmake --install /build/target \
|
||||
&& rm -rf /build/target /src/cyclonedds
|
||||
|
||||
ENV PATH="/opt/cyclonedds-host/bin:/usr/local/cargo/bin:${PATH}"
|
||||
|
||||
ARG RUST_VERSION=1.98.0
|
||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
||||
| env CARGO_HOME=/usr/local/cargo RUSTUP_HOME=/usr/local/rustup \
|
||||
sh -s -- -y --profile minimal --default-toolchain "${RUST_VERSION}" \
|
||||
--target aarch64-unknown-linux-gnu --no-modify-path
|
||||
ENV CARGO_HOME=/usr/local/cargo RUSTUP_HOME=/usr/local/rustup
|
||||
|
||||
# `build.rs` reads CYCLONEDDS_HOME for the headers and the link path. It points
|
||||
# at the **aarch64** tree; idlc comes from PATH, which is the host tree above.
|
||||
ENV CYCLONEDDS_HOME=/opt/cyclonedds-aarch64 \
|
||||
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
|
||||
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
|
||||
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
|
||||
AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar
|
||||
# bindgen parses the generated headers with libclang, which defaults to the host
|
||||
# target. Without this it would compute x86_64 struct layouts for an aarch64
|
||||
# binary -- the two agree here, but silently, and would stop agreeing the moment
|
||||
# a message gained a pointer or a long.
|
||||
ENV BINDGEN_EXTRA_CLANG_ARGS_aarch64_unknown_linux_gnu="--target=aarch64-linux-gnu --sysroot=/usr/aarch64-linux-gnu"
|
||||
|
||||
# The second cross build: `play --app` on Windows, the controller as a CPython
|
||||
# extension for `win-amd64`. The GNU target rather than MSVC, because MinGW is
|
||||
# an apt package and MSVC's libraries come only with a licence to accept. The
|
||||
# DLL it makes imports `python3.dll` and system DLLs and nothing else: no
|
||||
# libgcc or winpthread to ship beside it.
|
||||
#
|
||||
# Its own layer, after the board's, so that adding it to an image built before
|
||||
# it reuses every layer above -- CycloneDDS is not compiled again. The macOS
|
||||
# layer below is added the same way.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gcc-mingw-w64-x86-64 \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& rustup target add x86_64-pc-windows-gnu
|
||||
|
||||
# The third: `play --app` on macOS, one universal2 file carrying arm64 and
|
||||
# x86_64, so every Mac and every Python build on it -- Homebrew's, conda's,
|
||||
# python.org's universal2 one, one under Rosetta -- loads the same file and
|
||||
# dyld picks the slice. Linked by zig through cargo-zigbuild, which carries
|
||||
# libSystem's stubs, so no Apple SDK is needed and none has a licence to accept;
|
||||
# rustc's warning that `xcrun` cannot find one is expected and harmless. zig
|
||||
# also signs the arm64 slice ad hoc, without which Apple Silicon kills the
|
||||
# process that loads it. Minimum macOS 11.7.1, zig 0.13's floor.
|
||||
#
|
||||
# Both pinned: which zig links a binary is not visible in it.
|
||||
ARG ZIG_VERSION=0.13.0
|
||||
ARG CARGO_ZIGBUILD_VERSION=0.23.4
|
||||
RUN curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-linux-x86_64-${ZIG_VERSION}.tar.xz" \
|
||||
| tar -xJ -C /opt \
|
||||
&& ln -s "/opt/zig-linux-x86_64-${ZIG_VERSION}/zig" /usr/local/bin/zig \
|
||||
&& cargo install --locked "cargo-zigbuild@${CARGO_ZIGBUILD_VERSION}" \
|
||||
&& rm -rf /usr/local/cargo/registry \
|
||||
&& rustup target add aarch64-apple-darwin x86_64-apple-darwin
|
||||
|
||||
WORKDIR /work
|
||||
ENTRYPOINT ["cargo"]
|
||||
CMD ["build", "--release", "--target", "aarch64-unknown-linux-gnu", "--features", "rknn"]
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user