197 lines
9.5 KiB
Markdown
197 lines
9.5 KiB
Markdown
# Robot application formats
|
|
|
|
`scripts/deploy.py` writes a directory and a sibling `.app`: an ordinary ZIP with
|
|
`bundle.json` at its root. Entry order, timestamps and permissions are normalized.
|
|
Readers select a format by its exact `schema` tag, not by guessing from files.
|
|
[`app.schema`](app.schema) describes both versions. Python readers validate it
|
|
before loading control code or models.
|
|
|
|
## Self-contained native application — `mjrl-robot-app/1`
|
|
|
|
This format includes a robot model and supports `rl_onnx`, `ik_mocap`,
|
|
`wheeled_velocity`, `cpg_keyframe` and `model_viewer`. The Python runtime runs on
|
|
Linux, Windows and macOS with compatible Python and the declared dependencies.
|
|
It is a native MuJoCo application; it does not claim a browser, NPU or hardware
|
|
binary. Host Python/MuJoCo libraries are dependencies, not embedded interpreters.
|
|
|
|
```
|
|
robot/
|
|
bundle.json
|
|
app.schema
|
|
LICENSE
|
|
README.md
|
|
model/robot.xml
|
|
model/meshes/<digest>.<extension>
|
|
model/textures/<digest>.<extension>
|
|
controller.toml
|
|
<mode>.json
|
|
controls/<mode>.yaml
|
|
<mode>.trajectory.json
|
|
models/<mode>.onnx
|
|
<mode>.layout.json
|
|
verification.json
|
|
verification/<mode>.json
|
|
reference.json
|
|
manual.en.json
|
|
manual.zh.json
|
|
runtime/controller.pyz
|
|
runtime/requirements.txt
|
|
```
|
|
|
|
Only applicable companions are present: a viewer needs neither a policy nor a
|
|
trajectory. A controlled mode requires both native physics and control acceptance.
|
|
A geometry viewer claims model compilation/display only. It cannot inherit a
|
|
physical or locomotion capability because it successfully opened a file.
|
|
|
|
Every file except `bundle.json` is inventoried with byte length and SHA-256. Paths
|
|
are local relative paths. The reader rejects missing/extra files, mismatched bytes,
|
|
links, archive traversal, duplicate entries and excessive archive expansion. The
|
|
normalized model has no XML includes or plugin declarations and its mesh/texture
|
|
references must remain inside the app. Controller dependencies must be inventoried.
|
|
Hashes detect changes; they are not signatures authenticating an outside publisher.
|
|
Validation uses the trusted schema shipped with the host package or portable
|
|
runtime. The application's `app.schema` is an informational copy and cannot
|
|
weaken the reader's rules.
|
|
|
|
The manifest declares `robot_id`, `model`, `default_mode`, `modes`, `controller`,
|
|
`manuals`, `reference`, `verification`, `runtimes`, `capabilities`, `source` and
|
|
`files`. Each mode names its `control_mode` and `contract`; it names `controls`,
|
|
`trajectory` companions through the contract, and `policy`/`layout` for ONNX.
|
|
The declared capability must agree with the controller kind.
|
|
|
|
`controller.toml` uses `mjrl-app-controller/1` and declares `default_mode`. Native
|
|
mode selection is explicit at launch (`--app-mode`) or through
|
|
`RobotApplication.select_mode`. Selecting a different mode resets the world to that
|
|
mode's verified initial state. It is not a hardware state-machine transition.
|
|
|
|
### Build and replay
|
|
|
|
First create a normalized model and controller with `scripts/onboard.py`. Then
|
|
verify the exact controller, model and explicit physical response probes:
|
|
|
|
```bash
|
|
python scripts/onboard.py verify --model out/robot/robot.xml \
|
|
--config out/robot/verify-input.json --report out/robot/verification.json
|
|
python scripts/deploy.py --robot robot --model out/robot/robot.xml \
|
|
--control out/robot/controller.json --controls out/robot/controls.yaml \
|
|
--verification out/robot/verification.json --bundle out/apps
|
|
python scripts/play.py --app out/apps/robot.app --headless --steps 100
|
|
python scripts/deploy.py --check-reference out/apps/robot.app
|
|
```
|
|
|
|
An ONNX mode also supplies `--policy <export-directory>` containing `actor.onnx`
|
|
and `layout.json`; `--control` identifies the same controller JSON used by verify.
|
|
A viewer omits `--control` and `--verification`.
|
|
|
|
Multiple modes use `--platform-manifest FILE` with `robot_id`, `model`, `modes`,
|
|
optional `default_mode`, and optional shared `verification`. Mode entries accept
|
|
`controller`, `controls`, `policy`, `layout` and a mode-specific `verification`.
|
|
Paths resolve relative to the build manifest. Each controlled mode must match its
|
|
own passing report; a report for another controller cannot authorize it.
|
|
Existing output directories/archives are refused rather than overwritten.
|
|
Concurrent builders targeting the same output are also refused.
|
|
|
|
Supply asset and policy license or attribution files with repeatable
|
|
`--license PATH` options, or the build manifest's `licenses` list. They are copied
|
|
under `licenses/`, listed in the application manifest, and covered by file hashes.
|
|
The platform's Apache license does not replace the imported asset's license.
|
|
|
|
A report records the source XML hash, compiled model hash (including dependency
|
|
effects), controller hash, controls hash and policy/trajectory dependency hashes.
|
|
The bundler checks these before relocation. It compiles the relocated model and
|
|
constructs each packaged controller and replays every reference before publishing.
|
|
Reports must contain the current initial-pose, finite-state, contact, joint-margin
|
|
and sustained-effort checks; a historical pass flag alone is insufficient.
|
|
No physical asset values
|
|
are guessed to repair a failed bundle.
|
|
|
|
The application records the verified initial state, control rate, probe inputs,
|
|
measured samples and coordinate conventions. `--check-reference` replays these
|
|
inputs and compares qpos, qvel, control, actuator force and time. Ordinary playback
|
|
is not an acceptance test unless its command and response measurements are checked.
|
|
|
|
For an independent host, unpack the archive, install `runtime/requirements.txt` in
|
|
an isolated Python environment, and run:
|
|
|
|
```bash
|
|
python runtime/controller.pyz --app . --headless --steps 100
|
|
```
|
|
|
|
The packaged Python code is run only by that explicit command. The workbench
|
|
loads its installed runtime implementation, not arbitrary code from an opened app.
|
|
Use `--reference-out FILE` to record actual playback and `--video FILE.mp4` with
|
|
finite `--steps` for a preview. On a headless Linux rendering host, select an
|
|
available MuJoCo renderer (for example `MUJOCO_GL=egl`).
|
|
On macOS, MuJoCo's interactive viewer requires the `mjpython` launcher instead
|
|
of `python`; headless replay uses ordinary Python. Native gamepad polling is
|
|
implemented for Linux and Windows; macOS clients can use keyboard or API input.
|
|
For a fixed velocity command, `--evaluation-out FILE` writes body-frame mean and
|
|
final velocities, MAE/RMSE, peak tilt, resets and warning counts. It does not infer
|
|
acceptance thresholds. The preview camera follows a model with one robot root.
|
|
|
|
## Legacy policy application — `kk-policy-bundle/1`
|
|
|
|
The existing Rust FSM path remains supported for policies whose contracts its
|
|
reader implements. This version contains controller runtimes and policies, but
|
|
its simulation world still comes from an installed task. It does not promise a
|
|
self-contained model or traditional-control modes.
|
|
|
|
| Host | Inference | Runtime |
|
|
|---|---|---|
|
|
| `board` | RKNN | `runtime/board/controller` |
|
|
| `web` | ONNX | `runtime/web/controller.wasm` and `controller.js` |
|
|
| `mjlab` | ONNX | `runtime/mjlab/<platform>/controller.so` or `.pyd` |
|
|
|
|
Shared files are `controller.toml`, `<mode>.json`, optional trajectory companions,
|
|
`reference.json`, manuals, `README.md`, and `models/<mode>.onnx`/`.rknn`. The
|
|
schema's `x-files` inventory describes this legacy format only.
|
|
|
|
A default legacy build prepares board conversion and cross-builds, checks every
|
|
host declared in `runtimes`, produces numerical reference vectors, validates the
|
|
schema and packs the archive. `--allow-incomplete` skips unavailable board work
|
|
and explicitly records omissions; it is not a board-ready release.
|
|
|
|
`deploy/manifests.json` starts empty in the generic platform. Populate it with
|
|
explicit exports for your hardware, or use repeated `--mode NAME=EXPORT` and a
|
|
provided `--fsm FILE`. One mode can use the generated single-mode FSM. No robot,
|
|
board address, motor order or hardware-specific mode is selected implicitly.
|
|
|
|
The Rust contracts map `wire_joint_order`, `obs_joint_order` and
|
|
`action_joint_order` by joint name. `default_joint_pos` covers the wire order.
|
|
The policy contract records observation terms/history, action transform, rates,
|
|
gains, limits and command ranges. Never replace these with positional alignment
|
|
or guessed dimensions. Unsupported versioned native action semantics must fail
|
|
closed in the legacy reader.
|
|
|
|
The implementation references are:
|
|
|
|
| Contract | Reader |
|
|
|---|---|
|
|
| Bundle and host selection | `fsm/src/bundle.rs` |
|
|
| FSM | `fsm/src/config.rs` |
|
|
| Named policy layout | `fsm/src/layout.rs` |
|
|
| Input mappings | `fsm/src/operator.rs` |
|
|
| Reference motion | `fsm/src/trajectory.rs` |
|
|
| Recorded reference vectors | `fsm/src/reference.rs` |
|
|
| Browser and Python entry points | `fsm/src/web.rs`, `fsm/src/py.rs` |
|
|
|
|
The generated legacy bundle README, from
|
|
[`fsm/BUNDLE_README.md`](fsm/BUNDLE_README.md), documents its actual host API.
|
|
|
|
## Manuals and translations
|
|
|
|
Both application formats carry `manual.en.json` (`kk-bundle-manual/1`). The English
|
|
manual comes from the controller's bindings. It lists each mode, entry/exit,
|
|
keyboard bindings, pad controls and mode switches where implemented.
|
|
|
|
The bundle-manual skill translates only `source`, `summary`, `enter`, `leave`,
|
|
`keyboard`, `pad` and `does`, preserving every identifier and list structure.
|
|
|
|
```bash
|
|
python scripts/deploy.py --translate out/apps/robot \
|
|
--language zh --manual /tmp/manual.zh.json
|
|
```
|
|
|
|
The command validates the translation, updates inventory digests and repacks the
|
|
archive. Never translate joint names, mode names, file paths or physical key IDs.
|