Files
chenlin c32df70326
Checks / Layout, translation and Colab checks (push) Has been cancelled
Checks / Ruff (push) Has been cancelled
Checks / Framework tests (native backend, CPU) (push) Has been cancelled
Checks / Portable Rust controller (push) Has been cancelled
release: v0.1 generic robot training and application platform
2026-11-05 11:05:01 +08:00

197 lines
9.5 KiB
Markdown

# Robot application formats
`scripts/deploy.py` writes a directory and a sibling `.app`: an ordinary ZIP with
`bundle.json` at its root. Entry order, timestamps and permissions are normalized.
Readers select a format by its exact `schema` tag, not by guessing from files.
[`app.schema`](app.schema) describes both versions. Python readers validate it
before loading control code or models.
## Self-contained native application — `mjrl-robot-app/1`
This format includes a robot model and supports `rl_onnx`, `ik_mocap`,
`wheeled_velocity`, `cpg_keyframe` and `model_viewer`. The Python runtime runs on
Linux, Windows and macOS with compatible Python and the declared dependencies.
It is a native MuJoCo application; it does not claim a browser, NPU or hardware
binary. Host Python/MuJoCo libraries are dependencies, not embedded interpreters.
```
robot/
bundle.json
app.schema
LICENSE
README.md
model/robot.xml
model/meshes/<digest>.<extension>
model/textures/<digest>.<extension>
controller.toml
<mode>.json
controls/<mode>.yaml
<mode>.trajectory.json
models/<mode>.onnx
<mode>.layout.json
verification.json
verification/<mode>.json
reference.json
manual.en.json
manual.zh.json
runtime/controller.pyz
runtime/requirements.txt
```
Only applicable companions are present: a viewer needs neither a policy nor a
trajectory. A controlled mode requires both native physics and control acceptance.
A geometry viewer claims model compilation/display only. It cannot inherit a
physical or locomotion capability because it successfully opened a file.
Every file except `bundle.json` is inventoried with byte length and SHA-256. Paths
are local relative paths. The reader rejects missing/extra files, mismatched bytes,
links, archive traversal, duplicate entries and excessive archive expansion. The
normalized model has no XML includes or plugin declarations and its mesh/texture
references must remain inside the app. Controller dependencies must be inventoried.
Hashes detect changes; they are not signatures authenticating an outside publisher.
Validation uses the trusted schema shipped with the host package or portable
runtime. The application's `app.schema` is an informational copy and cannot
weaken the reader's rules.
The manifest declares `robot_id`, `model`, `default_mode`, `modes`, `controller`,
`manuals`, `reference`, `verification`, `runtimes`, `capabilities`, `source` and
`files`. Each mode names its `control_mode` and `contract`; it names `controls`,
`trajectory` companions through the contract, and `policy`/`layout` for ONNX.
The declared capability must agree with the controller kind.
`controller.toml` uses `mjrl-app-controller/1` and declares `default_mode`. Native
mode selection is explicit at launch (`--app-mode`) or through
`RobotApplication.select_mode`. Selecting a different mode resets the world to that
mode's verified initial state. It is not a hardware state-machine transition.
### Build and replay
First create a normalized model and controller with `scripts/onboard.py`. Then
verify the exact controller, model and explicit physical response probes:
```bash
python scripts/onboard.py verify --model out/robot/robot.xml \
--config out/robot/verify-input.json --report out/robot/verification.json
python scripts/deploy.py --robot robot --model out/robot/robot.xml \
--control out/robot/controller.json --controls out/robot/controls.yaml \
--verification out/robot/verification.json --bundle out/apps
python scripts/play.py --app out/apps/robot.app --headless --steps 100
python scripts/deploy.py --check-reference out/apps/robot.app
```
An ONNX mode also supplies `--policy <export-directory>` containing `actor.onnx`
and `layout.json`; `--control` identifies the same controller JSON used by verify.
A viewer omits `--control` and `--verification`.
Multiple modes use `--platform-manifest FILE` with `robot_id`, `model`, `modes`,
optional `default_mode`, and optional shared `verification`. Mode entries accept
`controller`, `controls`, `policy`, `layout` and a mode-specific `verification`.
Paths resolve relative to the build manifest. Each controlled mode must match its
own passing report; a report for another controller cannot authorize it.
Existing output directories/archives are refused rather than overwritten.
Concurrent builders targeting the same output are also refused.
Supply asset and policy license or attribution files with repeatable
`--license PATH` options, or the build manifest's `licenses` list. They are copied
under `licenses/`, listed in the application manifest, and covered by file hashes.
The platform's Apache license does not replace the imported asset's license.
A report records the source XML hash, compiled model hash (including dependency
effects), controller hash, controls hash and policy/trajectory dependency hashes.
The bundler checks these before relocation. It compiles the relocated model and
constructs each packaged controller and replays every reference before publishing.
Reports must contain the current initial-pose, finite-state, contact, joint-margin
and sustained-effort checks; a historical pass flag alone is insufficient.
No physical asset values
are guessed to repair a failed bundle.
The application records the verified initial state, control rate, probe inputs,
measured samples and coordinate conventions. `--check-reference` replays these
inputs and compares qpos, qvel, control, actuator force and time. Ordinary playback
is not an acceptance test unless its command and response measurements are checked.
For an independent host, unpack the archive, install `runtime/requirements.txt` in
an isolated Python environment, and run:
```bash
python runtime/controller.pyz --app . --headless --steps 100
```
The packaged Python code is run only by that explicit command. The workbench
loads its installed runtime implementation, not arbitrary code from an opened app.
Use `--reference-out FILE` to record actual playback and `--video FILE.mp4` with
finite `--steps` for a preview. On a headless Linux rendering host, select an
available MuJoCo renderer (for example `MUJOCO_GL=egl`).
On macOS, MuJoCo's interactive viewer requires the `mjpython` launcher instead
of `python`; headless replay uses ordinary Python. Native gamepad polling is
implemented for Linux and Windows; macOS clients can use keyboard or API input.
For a fixed velocity command, `--evaluation-out FILE` writes body-frame mean and
final velocities, MAE/RMSE, peak tilt, resets and warning counts. It does not infer
acceptance thresholds. The preview camera follows a model with one robot root.
## Legacy policy application — `kk-policy-bundle/1`
The existing Rust FSM path remains supported for policies whose contracts its
reader implements. This version contains controller runtimes and policies, but
its simulation world still comes from an installed task. It does not promise a
self-contained model or traditional-control modes.
| Host | Inference | Runtime |
|---|---|---|
| `board` | RKNN | `runtime/board/controller` |
| `web` | ONNX | `runtime/web/controller.wasm` and `controller.js` |
| `mjlab` | ONNX | `runtime/mjlab/<platform>/controller.so` or `.pyd` |
Shared files are `controller.toml`, `<mode>.json`, optional trajectory companions,
`reference.json`, manuals, `README.md`, and `models/<mode>.onnx`/`.rknn`. The
schema's `x-files` inventory describes this legacy format only.
A default legacy build prepares board conversion and cross-builds, checks every
host declared in `runtimes`, produces numerical reference vectors, validates the
schema and packs the archive. `--allow-incomplete` skips unavailable board work
and explicitly records omissions; it is not a board-ready release.
`deploy/manifests.json` starts empty in the generic platform. Populate it with
explicit exports for your hardware, or use repeated `--mode NAME=EXPORT` and a
provided `--fsm FILE`. One mode can use the generated single-mode FSM. No robot,
board address, motor order or hardware-specific mode is selected implicitly.
The Rust contracts map `wire_joint_order`, `obs_joint_order` and
`action_joint_order` by joint name. `default_joint_pos` covers the wire order.
The policy contract records observation terms/history, action transform, rates,
gains, limits and command ranges. Never replace these with positional alignment
or guessed dimensions. Unsupported versioned native action semantics must fail
closed in the legacy reader.
The implementation references are:
| Contract | Reader |
|---|---|
| Bundle and host selection | `fsm/src/bundle.rs` |
| FSM | `fsm/src/config.rs` |
| Named policy layout | `fsm/src/layout.rs` |
| Input mappings | `fsm/src/operator.rs` |
| Reference motion | `fsm/src/trajectory.rs` |
| Recorded reference vectors | `fsm/src/reference.rs` |
| Browser and Python entry points | `fsm/src/web.rs`, `fsm/src/py.rs` |
The generated legacy bundle README, from
[`fsm/BUNDLE_README.md`](fsm/BUNDLE_README.md), documents its actual host API.
## Manuals and translations
Both application formats carry `manual.en.json` (`kk-bundle-manual/1`). The English
manual comes from the controller's bindings. It lists each mode, entry/exit,
keyboard bindings, pad controls and mode switches where implemented.
The bundle-manual skill translates only `source`, `summary`, `enter`, `leave`,
`keyboard`, `pad` and `does`, preserving every identifier and list structure.
```bash
python scripts/deploy.py --translate out/apps/robot \
--language zh --manual /tmp/manual.zh.json
```
The command validates the translation, updates inventory digests and repacks the
archive. Never translate joint names, mode names, file paths or physical key IDs.