6.0 KiB
Architecture and extension seams
Layers
Coding agent / operator
AGENTS.md + rl-engineer playbook
|
Unified CLI (no robot-specific branches)
strict experiment JSON, bounded commands, durable run/pipeline status
|
Metadata registry → lazy task factory → config adapter
robot facts mjlab baseline reward overrides / evaluation protocol
|
mjlab ManagerBasedRlEnv
observation | action | command | reward | termination | event | curriculum
|
MuJoCo Warp (CUDA) → RSL-RL PPO
|
checkpoint + source/asset identity
|
fixed-command evaluation + video → ONNX parity → research bundle
Relationship to Jumper
Reuse the architectural principles: independent task modules, centralized commands, explicit
agent playbooks, and a deployment contract resolved from the actual environment. Do not copy
its hexapod control assumptions, native backend patches, RKNN conversion or Rust FSM. This
project uses the existing mjlab Simulation and manager API without monkey-patching it. Native
MuJoCo is used only to inspect MJCF and by the optional viewer, not as a hidden RL fallback.
Backend and adapter contract
Registry entries reference environment, training and asset factories through importable Python paths. A compatible backend must provide those interfaces; the registry does not download or convert missing adapters. Fingerprints include all Python files, not merely git HEAD.
The validated mjlab 1.6.0 source, its two inherited runtime patches, T1 extension and RSL-RL 5.5.1
are now vendored under rl/, installed as top-level packages with the project. Runtime guards
reject an external shadowing backend. uv.lock fixes transitive dependencies; rl/VENDOR.json
checks source and asset inventory. See Vendor provenance and Setup.
G1 and T1 use bundled adapted models with original asset licenses. BOOSTER_ASSETS_ROOT is an
optional explicit T1 override, not a hidden HOME default. Raw Unitree ROS XML can be inspected
but is not mechanically or semantically interchangeable with those models. check-robot checks
resolved interfaces against independent declarations in robot_specs/, then measures short
contact/PD response. See Robot checks.
Add a robot/task without changing the CLI
- Provide a robot
EntityCfgfactory: MJCFspec_fn, PD/motor definitions, initial posture, collision rules, foot sites and required sensors. Do not infer motor ratings from joint names. - Provide an environment factory
factory(play=False)returning an independent config and an RSL-RL config factory. The initial contract is a flat velocity task: entityrobot, commandtwist,joint_posposition action, actor/critic observation groups, reset events. - Add metadata in
registry.py: ID, robot, expected action DoF, lazy factory paths, asset reference (Path or callable returning Path), and number of parent levels covering its assets. - Add an experiment. No train/evaluate/export command needs robot-specific changes.
- Validate all task seams with the playbook. The current exporter deliberately requires one built-in position actuator per controlled joint. Tendons, coupled drives, recurrent networks, multiple action groups and history stacks need an explicitly versioned contract extension.
Task registry extensibility is not automatic adaptation to arbitrary morphology. A future tracking/manipulation task needs a separate task protocol rather than abusing the twist interface.
Reproducibility and evaluation
Runs contain the input experiment, resolved environment/agent configuration, checkpoint hashes, Python/config source archive, both backend source identities, dependency lock/versions and source/asset fingerprints. Bundled and optional external model assets retain their original licenses; meshes are not copied into training source archives. Replay/export refuse changed source/assets/runtime versions. Archives are for manual trusted restoration, never automatic execution.
Comparison intentionally permits different reward/source hashes but requires the same task and
evaluation protocol. Inspect provenance to ensure differences really are reward-only; the CLI does
not prove scientific causal isolation. Use multiple seeds, more than one command and adequate
horizons. status=completed means the operation finished, not the policy passed a quality gate.
Resource and trust model
No remote shell, multi-agent scheduler, cloud logger or motor transport is started. Training is
bounded by the experiment's iterations. train --background optionally launches a detached local
Linux worker with progress, logs and PID-guarded stop; foreground execution remains the default.
pipeline records the current stage and failure. Graceful interruption records interrupted;
SIGKILL/power loss can leave stale status, which status reports as lost when the recorded process
is dead. Never rewrite a failed/interrupted run to completed.
training.py atomically publishes intermediate checkpoints, records completed-update counts and
observes the backend logger without editing external packages. jobs.py / worker.py manage
local process lifecycle; curves.py reads scalar logs without a server. evaluation.py defines
versioned randomized/stress profiles; benchmark.py aggregates seed-level reports and selects
explicit candidates. measurement.py records terminal joint states before manual reset.
Evaluation/resume may explicitly load a committed checkpoint from an incomplete run after checking its hash and provenance. Export still requires a completed run. Resume uses a new output branch and restores optimizer/model/curriculum-counter state, not simulator or RNG trajectories. See Training operations for the statistical and lifecycle contracts.
Hash manifests are integrity evidence, not cryptographic signatures. Checkpoints and Python task
factories are executable trusted inputs. Source paths and git status are recorded; .env and
credentials are not copied. Binary policies cannot establish physical safety.