194 lines
11 KiB
Markdown
194 lines
11 KiB
Markdown
# Universal Humanoid
|
||
|
||
**An agent-ready framework for modular humanoid reinforcement learning.**
|
||
|
||
Universal Humanoid connects AI coding assistants with a reproducible robotics workflow: define tasks, iterate on rewards, train policies, evaluate behavior, replay results, and export inference bundles. Built on **mjlab**, **MuJoCo Warp**, and **RSL-RL**, it separates robot-specific configuration from shared training and evaluation tools.
|
||
|
||
“Universal” describes an extensible engineering workflow—not a single policy that controls every robot without adaptation.
|
||
|
||
[中文](README.zh.md) · [Setup](docs/SETUP.md) · [Robot checks](docs/NEW_ROBOT.md) · [Training operations](docs/TRAINING_OPERATIONS.md) · [Architecture](docs/ARCHITECTURE.md) · [Agent manuals](AGENTS.md) · [CLI](docs/CLI.md) · [Validation](docs/VALIDATION.md)
|
||
|
||
## Overview
|
||
|
||
```text
|
||
Developer / AI coding assistant
|
||
│
|
||
Task & reward configuration
|
||
│
|
||
Robot-specific adapter
|
||
│
|
||
mjlab → MuJoCo Warp → RSL-RL PPO
|
||
│
|
||
Checkpoint & experiment record
|
||
│
|
||
Evaluation & visual replay
|
||
│
|
||
Verified ONNX research bundle
|
||
```
|
||
|
||
Use the CLI directly or let a coding assistant operate the same commands. The framework does not require a particular model provider or a separate hosted agent service; assistants use their own configured models and tools.
|
||
|
||
## Features
|
||
|
||
- **Vendored runtime:** mjlab and RSL-RL copies under `rl/`, bundled G1/T1 assets, editable installation, hashed dependency lock and source inventory.
|
||
- **Robot conformance:** one-command checks for foot sites, IMU frames, actual contact response, PD/limits, joint order and reset posture.
|
||
- **Modular tasks:** lazy-loaded robot/environment factories behind a shared CLI.
|
||
- **Reward iteration:** strict experiment JSON, explicit weight/parameter overrides, and registered Python reward functions.
|
||
- **GPU training:** batched MuJoCo Warp simulation with RSL-RL PPO, committed intermediate checkpoints and optimizer continuation.
|
||
- **Training operations:** optional Linux background jobs, queryable iteration progress, guarded stop, and TensorBoard scalar curves in JSON/CSV/SVG.
|
||
- **Experiment provenance:** resolved configurations, source snapshots, dependency/asset fingerprints, and checkpoint hashes.
|
||
- **Measured evaluation:** checkpoint × evaluation-seed × four-command matrices, per-world randomized resets, noisy/push stress tests, mean/sample variance and explicitly defined failure rates.
|
||
- **Checkpoint selection:** explicit-metric ranking and paired matrix comparison; no automatic deployment promotion.
|
||
- **Replay:** MP4, per-step joint/torque CSV before reset, and mjlab's native/Viser viewer integrations.
|
||
- **Policy export:** ONNX with observation/action contracts, resolved joint order, numerical parity checks, and integrity manifests.
|
||
- **Agent guidance:** setup-env, new-task, train, controls, deploy and cross-stage RL manuals, with environment scripts and layout/documentation synchronization tests.
|
||
|
||
## Project status
|
||
|
||
**Early-stage research framework.** Backends, robot adapters and example model resources are now included. The reference installation targets Linux x86-64 with NVIDIA CUDA; portability does not imply every platform/GPU or arbitrary robot is supported.
|
||
|
||
| Robot | Task | Integration status |
|
||
|---|---|---|
|
||
| Unitree G1, 29 DoF | Flat-ground velocity tracking | Adapter wired to mjlab's adapted G1 model; bounded pipeline validation recorded |
|
||
| Booster T1, 23 DoF | Flat-ground velocity tracking | Included adapter and BSD-licensed model/meshes; bounded validation recorded |
|
||
| Other humanoids | — | Add and validate a robot/task adapter; asset discovery alone does not imply support |
|
||
|
||
**Backend policy:** install this project, not separate `mjlab`/`rsl-rl-lib` distributions. The retained backend changes—including simulation warning handling and T1 velocity-command support—are recorded in [VENDOR.md](docs/VENDOR.md), not hidden in an external working tree. See [Setup](docs/SETUP.md) and the [release checklist](docs/RELEASING.md).
|
||
|
||
The vendored build has passed [relocated installation, robot-gate and wheel checks](docs/PORTABILITY_VALIDATION.md) on the reference GPU host.
|
||
|
||
The included experiments are starting recipes, not pretrained walking policies. Initial validation used short training runs whose policies still fell. G1 and T1 use separate policies. See [recorded evidence and limitations](docs/VALIDATION.md).
|
||
|
||
## Getting started
|
||
|
||
### Explore without simulation dependencies
|
||
|
||
From the repository root, with Python 3.11 or newer:
|
||
|
||
```bash
|
||
PYTHONPATH=src python -m universal_humanoid list
|
||
PYTHONPATH=src python -m universal_humanoid plan --config experiments/g1_smoke.json
|
||
```
|
||
|
||
### Configure a simulation environment
|
||
|
||
Training requires a supported NVIDIA GPU/driver. Install the locked environment; no external backend or asset path is needed for G1/T1:
|
||
|
||
```bash
|
||
python3 .agents/skills/setup-env/scripts/detect.py
|
||
uv sync --locked --extra dev --python 3.13
|
||
./scripts/uh doctor
|
||
./scripts/uh verify-vendor
|
||
./scripts/uh check-robot --config experiments/g1_smoke.json --output runs/g1-check.json
|
||
./scripts/uh smoke --config experiments/g1_smoke.json --steps 16
|
||
```
|
||
|
||
Standard `pip install -e '.[dev]'` is also supported; [Setup](docs/SETUP.md) explains the hashed pip lock alternative. `doctor` checks installation/CUDA, while `check-robot` checks the actual adapter and measured contacts. Use the setup-env scripts for a logged multi-gate report. Legacy external `MJLAB_ROOT` overrides are rejected.
|
||
|
||
### Run a bounded end-to-end experiment
|
||
|
||
```bash
|
||
# Two PPO iterations, evaluation, then a research-only ONNX bundle.
|
||
# Use a NEW output directory for every run.
|
||
./scripts/uh pipeline --config experiments/g1_smoke.json \
|
||
--output runs/g1-first --eval-steps 250 --eval-envs 4
|
||
./scripts/uh verify-bundle runs/g1-first/bundle --inference
|
||
```
|
||
|
||
This validates the pipeline, not locomotion quality. Select `experiments/t1_smoke.json` for the included T1 adapter and assets.
|
||
|
||
## Reward → train → evaluate → replay → export
|
||
|
||
```bash
|
||
./scripts/uh rewards --config experiments/g1_flat.json
|
||
./scripts/uh plan --config experiments/g1_reward_candidate.json
|
||
|
||
# Longer training: choose a resource budget before launching.
|
||
./scripts/uh train --config experiments/g1_flat.json --output runs/g1-baseline
|
||
|
||
./scripts/uh evaluate --checkpoint runs/g1-baseline/final.pt \
|
||
--output runs/g1-baseline/evaluation.json --steps 1500 --num-envs 8 \
|
||
--command 0.3 0 0 --video runs/g1-baseline/replay.mp4
|
||
|
||
./scripts/uh play --checkpoint runs/g1-baseline/final.pt --viewer viser
|
||
# Desktop viewer: --viewer native (requires a display).
|
||
|
||
./scripts/uh export --checkpoint runs/g1-baseline/final.pt \
|
||
--evaluation runs/g1-baseline/evaluation.json --output bundles/g1-baseline
|
||
./scripts/uh verify-bundle bundles/g1-baseline --inference
|
||
```
|
||
|
||
Change tuning in a new experiment JSON. Add reward mechanisms in `src/universal_humanoid/rewards.py` and register them in `FUNCTIONS`. Unknown term names, parameters, and function references are rejected.
|
||
|
||
Evaluate/export a baseline **before** editing source. Replay and export check source, asset, and dependency identity; historical sources can be restored from a trusted run archive into a separate environment. Never overwrite shared dependencies to replay an old experiment.
|
||
|
||
Automation should read `run.json`, `pipeline.json`, and `evaluation.json`. Console output also contains backend diagnostics and is not a single JSON document.
|
||
|
||
## Monitor learning and manage experiments
|
||
|
||
```bash
|
||
./scripts/uh train --config experiments/g1_flat.json --output runs/g1-long --background
|
||
./scripts/uh status runs/g1-long
|
||
./scripts/uh curves runs/g1-long --output analysis/g1-curves
|
||
./scripts/uh checkpoints runs/g1-long
|
||
# Resume an explicitly selected checkpoint into a NEW run:
|
||
./scripts/uh train --resume runs/g1-long/model_0.pt --iterations 200 \
|
||
--output runs/g1-continued --background
|
||
|
||
./scripts/uh benchmark --checkpoints runs/g1-long/final.pt \
|
||
--seeds 101 202 303 --profiles randomized stress \
|
||
--steps 1000 --num-envs 8 --output analysis/g1-matrix
|
||
./scripts/uh play --checkpoint runs/g1-long/final.pt \
|
||
--measure analysis/g1-joints.csv --steps 1000 --profile stress
|
||
```
|
||
|
||
These are separate lifecycle examples, not a script to launch all jobs simultaneously. Wait for
|
||
checkpoints to be committed and reserve GPU capacity before evaluating beside a running trainer.
|
||
Matrix seeds control evaluation randomness, not independent training replicas. Read tracking,
|
||
action-noise and episode-length curves—not just elapsed time. See [Training operations](docs/TRAINING_OPERATIONS.md)
|
||
for stopping, continuation semantics, statistics, checkpoint selection and CSV timing.
|
||
|
||
## Working with a coding assistant
|
||
|
||
Start with [AGENTS.md](AGENTS.md) and the [RL engineer playbook](.agents/skills/rl-engineer/SKILL.md). For example:
|
||
|
||
> Establish a G1 baseline, propose a reward revision, and test both within an agreed training budget. Compare tracking error and failure events under the same evaluation protocol, record a replay, and export a research bundle. Do not connect to physical hardware.
|
||
|
||
Reward totals alone are not a valid comparison across different reward definitions. Use task metrics, visual inspection, multiple seeds, and appropriate evaluation horizons.
|
||
|
||
## Assets and extensions
|
||
|
||
```bash
|
||
# Optional inventory of additional models, not needed for the bundled G1/T1 tasks:
|
||
export UNITREE_ROBOTS_ROOT=/path/to/unitree_ros/robots
|
||
./scripts/uh assets
|
||
./scripts/uh inspect-asset "$UNITREE_ROBOTS_ROOT/g1_description/g1_29dof.xml"
|
||
```
|
||
|
||
G1 training uses the bundled sensor/collision-adapted asset. T1 uses its bundled `T1_23dof.xml` and meshes; `BOOSTER_ASSETS_ROOT` is an optional explicit replacement, not a requirement. Raw robot descriptions are not silently substituted into a trained task. New morphologies need validated sensors, contacts, control gains, joint ordering, and observation/action definitions. See [extension seams](docs/ARCHITECTURE.md).
|
||
|
||
## Development
|
||
|
||
Git, license, dependency-lock and CI details are recorded in the [repository baseline](docs/REPOSITORY_BASELINE.md).
|
||
|
||
See [CONTRIBUTING.md](CONTRIBUTING.md) for change boundaries and required checks.
|
||
With the configured runtime and test dependencies installed:
|
||
|
||
```bash
|
||
./scripts/check
|
||
```
|
||
|
||
Core tests can run without a simulator; adapter tests require mjlab/torch. Unit tests and GPU validation are separate forms of evidence. Before publishing a release, follow [RELEASING.md](docs/RELEASING.md).
|
||
|
||
## Scope and safety
|
||
|
||
Not currently provided: physical robot transport, state estimation, watchdog/E-stop supervision, ROS/DDS mappings, automatic morphology conversion, a universal zero-shot policy, rough-terrain task coverage, motion imitation, multi-GPU training, or native-CPU RL.
|
||
|
||
Exported bundles are for **research/inference**, not robot executables. Every bundle has `hardware_approved: false`; no CLI command enables motors. Load only trusted PyTorch checkpoints—the upstream loader uses pickle. Hashes detect accidental changes; they are not signatures or a sandbox.
|
||
|
||
## Acknowledgments and licensing
|
||
|
||
The workflow design draws inspiration from Jumper's modular task and engineering conventions. Simulation and learning are provided by mjlab, MuJoCo Warp, and RSL-RL. See [NOTICE.md](NOTICE.md) for component and asset attribution.
|
||
|
||
Original project code and documentation are licensed under [Apache-2.0](LICENSE), following Jumper's licensing approach. Bundled dependencies and robot assets retain their original licenses; see [NOTICE.md](NOTICE.md).
|