Files
2026-11-03 19:46:08 +08:00

5.6 KiB
Raw Permalink Blame History

Reference-environment validation

For current vendored installation, relocated-checkout/wheel tests and automatic robot gates, see Portability validation.

For the subsequent background/resume, randomized matrix evaluation, TensorBoard and CSV work, see Training-operations validation. The sections below preserve the initial pipeline validation and its original source/protocol version.

This is a historical validation record for one development configuration, not a minimum hardware specification or a guarantee of compatibility with every installation. See Setup for environment requirements and Releasing for distribution limitations. Paths under runs/ below identify development artifacts; they are ignored by Git and are not bundled with a repository checkout or offered here as downloads.

Measured on 2026-11-03, NVIDIA GeForce RTX 5080, 16 GB, in a reference mjlab Python 3.13 environment. These results validate engineering plumbing, not learned locomotion.

Runtime

Component Observed version
mjlab 1.6.0, customized checkout
mjlab git HEAD 8bb2bc9e187797689ec82c78c35019afd2fd0376 (dirty/untracked changes retained)
MuJoCo / MuJoCo Warp 3.16.0 / 3.16.0
Warp 1.18.0
PyTorch 2.14.0+cu130
RSL-RL 5.5.1

The exact Python/asset hashes and dirty git status are in each train/run.json. The reference repositories were not modified. sources.zip preserves 230 Python/license entries per final run. Do not replace the customized dependency with a public version just because its version number matches.

Automated checks

./scripts/check: 35 tests passed, Ruff lint and format checks passed. One upstream torch.jit.script deprecation warning remains. Tests cover strict configs, lightweight discovery, independent factories, reward shape/sign, unknown overrides, retained evaluation terminations, source/experiment/checkpoint drift, partial-run rejection, bundle tampering/path traversal and protocol mismatch.

An initial direct pytest invocation inherited ROS's Python 3.12 plugin path and failed on a missing lark import. The project check script excludes inherited ROS paths and disables unrelated pytest plugin autoload; no external Python environment was modified.

Actual CUDA simulation and PPO

Zero-action checks ran for G1 and T1 with 4 environments and 8 steps; observations and rewards were finite. Shapes: G1 actor/critic/action = 99 / 111 / 29; T1 = 81 / 93 / 23.

Final end-to-end runs each used 16 environments, 2 PPO iterations, 24 rollout steps per iteration (768 collected environment transitions), then 4 evaluation worlds × 500 steps, dt 0.02 s, 5 s episode limit, fixed command (0.3, 0, 0), seed 42.

Run directory XY tracking RMSE (m/s) Yaw RMSE (rad/s) Failure events Mean completed episode (s) ONNX max abs error
runs/g1_validation 0.709230 0.322807 24 1.500 5.59e-8
runs/t1_validation 0.480955 0.047538 24 1.560 5.96e-8
runs/g1_reward_validation 0.675789 0.313275 24 1.480 5.22e-8

All three policies fall. None is a deployment candidate. Failure events count repeated episode failures across resets, not a probability. The current nominal evaluation deliberately uses identical resets in parallel worlds; those worlds are not independent statistical trials. The small candidate RMSE difference is not evidence of an improved learned policy after two updates.

Each run has pipeline.json, train/final.pt, evaluation.json, and bundle/. verify-bundle --inference passed for G1 and T1. ONNX parity was checked against the PyTorch actor at 16 actual rollout observations for every export. The reward candidate exercises local reward function injection, training, evaluation, export and protocol-matched comparison.

Replay evidence

runs/t1_validation/replay.mp4: H.264, 100 frames / 2 s / 50 FPS / 320×240; video_evaluation.json is the corresponding one-world report. Encoding and decoding succeeded, and a decoded frame was inspected: the T1 and ground are visible, not an empty render. This replay also records an episode failure. Desktop native and interactive Viser sessions were not manually exercised; their CLI paths use mjlab's existing viewer classes.

External Unitree asset checks

Native MuJoCo compilation succeeded for:

  • unitree_ros/robots/g1_description/g1_29dof.xml: nq=36, nv=35, nu=29.
  • unitree_ros/robots/h1_description/mjcf/h1.xml: nq=26, nv=25, nu=19.

These are asset compilation checks only. They are not registered RL task adapters; the validated G1 training run uses mjlab's adapted G1 asset. T1 training uses an external Booster T1 asset through the registered adapter.

Failed development runs retained

runs/g1_smoke_v1 trained but evaluation failed because the upstream command rejected heading_command=False with a non-null heading range. The adapter now clears the heading range and regression tests check evaluation configuration. runs/g1_smoke_v2 passed an earlier short pipeline before the final provenance/archive changes. Its old checkpoints intentionally fail current source identity checks; use the final *_validation runs above.

Remaining validation

Long-run convergence, gait quality, domain-randomized/stress evaluation, independent seeds/reset conditions, contact realism, raw Unitree asset adaptation, rough terrain, motion imitation, sim-to-sim dynamics parity and all physical deployment safety remain unverified/unimplemented. No long 3,000-iteration recipe, robot transport or motor-enable operation was executed.