103 lines
6.0 KiB
Markdown
103 lines
6.0 KiB
Markdown
# Architecture and extension seams
|
|
|
|
## Layers
|
|
|
|
```text
|
|
Coding agent / operator
|
|
AGENTS.md + rl-engineer playbook
|
|
|
|
|
Unified CLI (no robot-specific branches)
|
|
strict experiment JSON, bounded commands, durable run/pipeline status
|
|
|
|
|
Metadata registry → lazy task factory → config adapter
|
|
robot facts mjlab baseline reward overrides / evaluation protocol
|
|
|
|
|
mjlab ManagerBasedRlEnv
|
|
observation | action | command | reward | termination | event | curriculum
|
|
|
|
|
MuJoCo Warp (CUDA) → RSL-RL PPO
|
|
|
|
|
checkpoint + source/asset identity
|
|
|
|
|
fixed-command evaluation + video → ONNX parity → research bundle
|
|
```
|
|
|
|
## Relationship to Jumper
|
|
|
|
Reuse the architectural principles: independent task modules, centralized commands, explicit
|
|
agent playbooks, and a deployment contract resolved from the actual environment. Do not copy
|
|
its hexapod control assumptions, native backend patches, RKNN conversion or Rust FSM. This
|
|
project uses the existing mjlab `Simulation` and manager API without monkey-patching it. Native
|
|
MuJoCo is used only to inspect MJCF and by the optional viewer, not as a hidden RL fallback.
|
|
|
|
## Backend and adapter contract
|
|
|
|
Registry entries reference environment, training and asset factories through importable Python
|
|
paths. A compatible backend must provide those interfaces; the registry does not download or
|
|
convert missing adapters. Fingerprints include **all Python files**, not merely git HEAD.
|
|
|
|
The validated mjlab 1.6.0 source, its two inherited runtime patches, T1 extension and RSL-RL 5.5.1
|
|
are now vendored under `rl/`, installed as top-level packages with the project. Runtime guards
|
|
reject an external shadowing backend. `uv.lock` fixes transitive dependencies; `rl/VENDOR.json`
|
|
checks source and asset inventory. See [Vendor provenance](VENDOR.md) and [Setup](SETUP.md).
|
|
|
|
G1 and T1 use bundled adapted models with original asset licenses. `BOOSTER_ASSETS_ROOT` is an
|
|
optional explicit T1 override, not a hidden HOME default. Raw Unitree ROS XML can be inspected
|
|
but is not mechanically or semantically interchangeable with those models. `check-robot` checks
|
|
resolved interfaces against independent declarations in `robot_specs/`, then measures short
|
|
contact/PD response. See [Robot checks](NEW_ROBOT.md).
|
|
|
|
## Add a robot/task without changing the CLI
|
|
|
|
1. Provide a robot `EntityCfg` factory: MJCF `spec_fn`, PD/motor definitions, initial posture,
|
|
collision rules, foot sites and required sensors. Do not infer motor ratings from joint names.
|
|
2. Provide an environment factory `factory(play=False)` returning an independent config and an
|
|
RSL-RL config factory. The initial contract is a flat velocity task: entity `robot`, command
|
|
`twist`, `joint_pos` position action, actor/critic observation groups, reset events.
|
|
3. Add metadata in `registry.py`: ID, robot, expected action DoF, lazy factory paths, asset
|
|
reference (Path or callable returning Path), and number of parent levels covering its assets.
|
|
4. Add an experiment. No train/evaluate/export command needs robot-specific changes.
|
|
5. Validate all task seams with the playbook. The current exporter deliberately requires one
|
|
built-in position actuator per controlled joint. Tendons, coupled drives, recurrent networks,
|
|
multiple action groups and history stacks need an explicitly versioned contract extension.
|
|
|
|
Task registry extensibility is not automatic adaptation to arbitrary morphology. A future
|
|
tracking/manipulation task needs a separate task protocol rather than abusing the twist interface.
|
|
|
|
## Reproducibility and evaluation
|
|
|
|
Runs contain the input experiment, resolved environment/agent configuration, checkpoint hashes,
|
|
Python/config source archive, both backend source identities, dependency lock/versions and
|
|
source/asset fingerprints. Bundled and optional external model assets retain their original
|
|
licenses; meshes are not copied into training source archives. Replay/export refuse changed source/assets/runtime
|
|
versions. Archives are for manual trusted restoration, never automatic execution.
|
|
|
|
Comparison intentionally permits different reward/source hashes but requires the same task and
|
|
evaluation protocol. Inspect provenance to ensure differences really are reward-only; the CLI does
|
|
not prove scientific causal isolation. Use multiple seeds, more than one command and adequate
|
|
horizons. `status=completed` means the operation finished, not the policy passed a quality gate.
|
|
|
|
## Resource and trust model
|
|
|
|
No remote shell, multi-agent scheduler, cloud logger or motor transport is started. Training is
|
|
bounded by the experiment's iterations. `train --background` optionally launches a detached local
|
|
Linux worker with progress, logs and PID-guarded stop; foreground execution remains the default.
|
|
`pipeline` records the current stage and failure. Graceful interruption records `interrupted`;
|
|
SIGKILL/power loss can leave stale status, which `status` reports as lost when the recorded process
|
|
is dead. Never rewrite a failed/interrupted run to completed.
|
|
|
|
`training.py` atomically publishes intermediate checkpoints, records completed-update counts and
|
|
observes the backend logger without editing external packages. `jobs.py` / `worker.py` manage
|
|
local process lifecycle; `curves.py` reads scalar logs without a server. `evaluation.py` defines
|
|
versioned randomized/stress profiles; `benchmark.py` aggregates seed-level reports and selects
|
|
explicit candidates. `measurement.py` records terminal joint states before manual reset.
|
|
|
|
Evaluation/resume may explicitly load a committed checkpoint from an incomplete run after
|
|
checking its hash and provenance. Export still requires a completed run. Resume uses a new output
|
|
branch and restores optimizer/model/curriculum-counter state, not simulator or RNG trajectories.
|
|
See [Training operations](TRAINING_OPERATIONS.md) for the statistical and lifecycle contracts.
|
|
|
|
Hash manifests are integrity evidence, not cryptographic signatures. Checkpoints and Python task
|
|
factories are executable trusted inputs. Source paths and git status are recorded; `.env` and
|
|
credentials are not copied. Binary policies cannot establish physical safety.
|