2026-11-03 19:46:08 +08:00

Universal Humanoid

An agent-ready framework for modular humanoid reinforcement learning.

Universal Humanoid connects AI coding assistants with a reproducible robotics workflow: define tasks, iterate on rewards, train policies, evaluate behavior, replay results, and export inference bundles. Built on mjlab, MuJoCo Warp, and RSL-RL, it separates robot-specific configuration from shared training and evaluation tools.

“Universal” describes an extensible engineering workflow—not a single policy that controls every robot without adaptation.

中文 · Setup · Robot checks · Training operations · Architecture · Agent manuals · CLI · Validation

Overview

Developer / AI coding assistant
                │
        Task & reward configuration
                │
        Robot-specific adapter
                │
     mjlab → MuJoCo Warp → RSL-RL PPO
                │
      Checkpoint & experiment record
                │
       Evaluation & visual replay
                │
       Verified ONNX research bundle

Use the CLI directly or let a coding assistant operate the same commands. The framework does not require a particular model provider or a separate hosted agent service; assistants use their own configured models and tools.

Features

  • Vendored runtime: mjlab and RSL-RL copies under rl/, bundled G1/T1 assets, editable installation, hashed dependency lock and source inventory.
  • Robot conformance: one-command checks for foot sites, IMU frames, actual contact response, PD/limits, joint order and reset posture.
  • Modular tasks: lazy-loaded robot/environment factories behind a shared CLI.
  • Reward iteration: strict experiment JSON, explicit weight/parameter overrides, and registered Python reward functions.
  • GPU training: batched MuJoCo Warp simulation with RSL-RL PPO, committed intermediate checkpoints and optimizer continuation.
  • Training operations: optional Linux background jobs, queryable iteration progress, guarded stop, and TensorBoard scalar curves in JSON/CSV/SVG.
  • Experiment provenance: resolved configurations, source snapshots, dependency/asset fingerprints, and checkpoint hashes.
  • Measured evaluation: checkpoint × evaluation-seed × four-command matrices, per-world randomized resets, noisy/push stress tests, mean/sample variance and explicitly defined failure rates.
  • Checkpoint selection: explicit-metric ranking and paired matrix comparison; no automatic deployment promotion.
  • Replay: MP4, per-step joint/torque CSV before reset, and mjlab's native/Viser viewer integrations.
  • Policy export: ONNX with observation/action contracts, resolved joint order, numerical parity checks, and integrity manifests.
  • Agent guidance: setup-env, new-task, train, controls, deploy and cross-stage RL manuals, with environment scripts and layout/documentation synchronization tests.

Project status

Early-stage research framework. Backends, robot adapters and example model resources are now included. The reference installation targets Linux x86-64 with NVIDIA CUDA; portability does not imply every platform/GPU or arbitrary robot is supported.

Robot Task Integration status
Unitree G1, 29 DoF Flat-ground velocity tracking Adapter wired to mjlab's adapted G1 model; bounded pipeline validation recorded
Booster T1, 23 DoF Flat-ground velocity tracking Included adapter and BSD-licensed model/meshes; bounded validation recorded
Other humanoids — Add and validate a robot/task adapter; asset discovery alone does not imply support

Backend policy: install this project, not separate mjlab/rsl-rl-lib distributions. The retained backend changes—including simulation warning handling and T1 velocity-command support—are recorded in VENDOR.md, not hidden in an external working tree. See Setup and the release checklist.

The vendored build has passed relocated installation, robot-gate and wheel checks on the reference GPU host.

The included experiments are starting recipes, not pretrained walking policies. Initial validation used short training runs whose policies still fell. G1 and T1 use separate policies. See recorded evidence and limitations.

Getting started

Explore without simulation dependencies

From the repository root, with Python 3.11 or newer:

PYTHONPATH=src python -m universal_humanoid list
PYTHONPATH=src python -m universal_humanoid plan --config experiments/g1_smoke.json

Configure a simulation environment

Training requires a supported NVIDIA GPU/driver. Install the locked environment; no external backend or asset path is needed for G1/T1:

python3 .agents/skills/setup-env/scripts/detect.py
uv sync --locked --extra dev --python 3.13
./scripts/uh doctor
./scripts/uh verify-vendor
./scripts/uh check-robot --config experiments/g1_smoke.json --output runs/g1-check.json
./scripts/uh smoke --config experiments/g1_smoke.json --steps 16

Standard pip install -e '.[dev]' is also supported; Setup explains the hashed pip lock alternative. doctor checks installation/CUDA, while check-robot checks the actual adapter and measured contacts. Use the setup-env scripts for a logged multi-gate report. Legacy external MJLAB_ROOT overrides are rejected.

Run a bounded end-to-end experiment

# Two PPO iterations, evaluation, then a research-only ONNX bundle.
# Use a NEW output directory for every run.
./scripts/uh pipeline --config experiments/g1_smoke.json \
  --output runs/g1-first --eval-steps 250 --eval-envs 4
./scripts/uh verify-bundle runs/g1-first/bundle --inference

This validates the pipeline, not locomotion quality. Select experiments/t1_smoke.json for the included T1 adapter and assets.

Reward → train → evaluate → replay → export

./scripts/uh rewards --config experiments/g1_flat.json
./scripts/uh plan --config experiments/g1_reward_candidate.json

# Longer training: choose a resource budget before launching.
./scripts/uh train --config experiments/g1_flat.json --output runs/g1-baseline

./scripts/uh evaluate --checkpoint runs/g1-baseline/final.pt \
  --output runs/g1-baseline/evaluation.json --steps 1500 --num-envs 8 \
  --command 0.3 0 0 --video runs/g1-baseline/replay.mp4

./scripts/uh play --checkpoint runs/g1-baseline/final.pt --viewer viser
# Desktop viewer: --viewer native (requires a display).

./scripts/uh export --checkpoint runs/g1-baseline/final.pt \
  --evaluation runs/g1-baseline/evaluation.json --output bundles/g1-baseline
./scripts/uh verify-bundle bundles/g1-baseline --inference

Change tuning in a new experiment JSON. Add reward mechanisms in src/universal_humanoid/rewards.py and register them in FUNCTIONS. Unknown term names, parameters, and function references are rejected.

Evaluate/export a baseline before editing source. Replay and export check source, asset, and dependency identity; historical sources can be restored from a trusted run archive into a separate environment. Never overwrite shared dependencies to replay an old experiment.

Automation should read run.json, pipeline.json, and evaluation.json. Console output also contains backend diagnostics and is not a single JSON document.

Monitor learning and manage experiments

./scripts/uh train --config experiments/g1_flat.json --output runs/g1-long --background
./scripts/uh status runs/g1-long
./scripts/uh curves runs/g1-long --output analysis/g1-curves
./scripts/uh checkpoints runs/g1-long
# Resume an explicitly selected checkpoint into a NEW run:
./scripts/uh train --resume runs/g1-long/model_0.pt --iterations 200 \
  --output runs/g1-continued --background

./scripts/uh benchmark --checkpoints runs/g1-long/final.pt \
  --seeds 101 202 303 --profiles randomized stress \
  --steps 1000 --num-envs 8 --output analysis/g1-matrix
./scripts/uh play --checkpoint runs/g1-long/final.pt \
  --measure analysis/g1-joints.csv --steps 1000 --profile stress

These are separate lifecycle examples, not a script to launch all jobs simultaneously. Wait for checkpoints to be committed and reserve GPU capacity before evaluating beside a running trainer. Matrix seeds control evaluation randomness, not independent training replicas. Read tracking, action-noise and episode-length curves—not just elapsed time. See Training operations for stopping, continuation semantics, statistics, checkpoint selection and CSV timing.

Working with a coding assistant

Start with AGENTS.md and the RL engineer playbook. For example:

Establish a G1 baseline, propose a reward revision, and test both within an agreed training budget. Compare tracking error and failure events under the same evaluation protocol, record a replay, and export a research bundle. Do not connect to physical hardware.

Reward totals alone are not a valid comparison across different reward definitions. Use task metrics, visual inspection, multiple seeds, and appropriate evaluation horizons.

Assets and extensions

# Optional inventory of additional models, not needed for the bundled G1/T1 tasks:
export UNITREE_ROBOTS_ROOT=/path/to/unitree_ros/robots
./scripts/uh assets
./scripts/uh inspect-asset "$UNITREE_ROBOTS_ROOT/g1_description/g1_29dof.xml"

G1 training uses the bundled sensor/collision-adapted asset. T1 uses its bundled T1_23dof.xml and meshes; BOOSTER_ASSETS_ROOT is an optional explicit replacement, not a requirement. Raw robot descriptions are not silently substituted into a trained task. New morphologies need validated sensors, contacts, control gains, joint ordering, and observation/action definitions. See extension seams.

Development

Git, license, dependency-lock and CI details are recorded in the repository baseline.

See CONTRIBUTING.md for change boundaries and required checks. With the configured runtime and test dependencies installed:

./scripts/check

Core tests can run without a simulator; adapter tests require mjlab/torch. Unit tests and GPU validation are separate forms of evidence. Before publishing a release, follow RELEASING.md.

Scope and safety

Not currently provided: physical robot transport, state estimation, watchdog/E-stop supervision, ROS/DDS mappings, automatic morphology conversion, a universal zero-shot policy, rough-terrain task coverage, motion imitation, multi-GPU training, or native-CPU RL.

Exported bundles are for research/inference, not robot executables. Every bundle has hardware_approved: false; no CLI command enables motors. Load only trusted PyTorch checkpoints—the upstream loader uses pickle. Hashes detect accidental changes; they are not signatures or a sandbox.

Acknowledgments and licensing

The workflow design draws inspiration from Jumper's modular task and engineering conventions. Simulation and learning are provided by mjlab, MuJoCo Warp, and RSL-RL. See NOTICE.md for component and asset attribution.

Original project code and documentation are licensed under Apache-2.0, following Jumper's licensing approach. Bundled dependencies and robot assets retain their original licenses; see NOTICE.md.

S
Description
No description provided
Readme 17 MiB
Languages
Python 99.9%