Files
chenlin ceca51b0d8
Checks / Layout, translation and Colab checks (push) Has been cancelled
Checks / Ruff (push) Has been cancelled
Checks / Framework tests (native backend, CPU) (push) Has been cancelled
Checks / Portable Rust controller (push) Has been cancelled
Improve generic robot onboarding, walking controls and end-effector interaction
Default humanoid intake to command-following gait and support bounded Cartesian target dragging for arms. Carry model and actuator semantics through calibration, native verification and packaging. Document reusable intake decisions and cover them with regression tests.
2026-11-06 10:54:58 +08:00

21 KiB
Raw Permalink Blame History

Deterministic controllers and external policies

Use scripts/onboard.py setup-tier2 to turn a MuJoCo model into a native controller setup. Use adapt-tier1 when an existing actor and explicit training metadata are available. Both routes produce artifacts that can be verified and packaged independently of a training task.

Tier 2 controller setup

python scripts/onboard.py build-mjcf assets/planar_arm/robot.urdf \
  --output out/planar_arm.xml
python scripts/onboard.py setup-tier2 --model out/planar_arm.xml \
  --config assets/planar_arm/ik.json --output out/arm_ik
cp assets/planar_arm/verify_ik.json out/arm_ik/verify-input.json
python scripts/onboard.py verify --model out/arm_ik/robot.xml \
  --config out/arm_ik/verify-input.json --report out/arm_ik/verification.json
python scripts/deploy.py --robot planar_arm --model out/arm_ik/robot.xml \
  --control out/arm_ik/controller.json --controls out/arm_ik/controls.yaml \
  --verification out/arm_ik/verification.json

The setup writes robot.xml with controller-specific actuators, a resolved controller.json, and controls.yaml validated against the same dictionary used in training and deployment. A passive direct model has no bindings. Referenced meshes and textures remain at their original paths until the bundler copies them.

The config schema is mjrl-controller/1. A setup may omit that field and accept the current schema. All lengths are in metres, angles in radians and times in seconds. Hinges and slides use native MuJoCo units for their position, velocity and effort. control_hz may be supplied when one control period is a whole number of physics steps. By default it is the closest available rate to 50 Hz obtained by rounding the number of substeps. Set gains and effort limits for your model. The bench defaults are kp: 40, kd: 4, and effort_limit: 30 for position and CPG controllers, or kv: 5 and effort_limit: 30 for velocity actuators.

For ik, joint_sliders, trajectory, and cpg, each of kp, kd, and effort_limit accepts either one scalar or a map covering every name in joints exactly. Maps are resolved by name, independently of actuator order. Values must be finite; kp and effort_limit must be positive and kd nonnegative. For example, "kp": {"hip": 70, "shoulder": 30} gives those two controlled joints different stiffnesses. Missing or extra names and positional arrays are rejected.

Normalization intersects the requested effort with the source actuator force limits, pure motor input limits, and joint actuator-force limits. Source force and motor input bounds are converted through the authored gear before creating unit-gear actuators. Position and velocity target ranges are not effort limits. The resulting model retains the effective bounds even when the configuration requests a larger effort. Each joint must retain effort authority in both directions. CPG clips its computed effort to the smaller magnitude of the two effective bounds; position servos retain the asymmetric bounds when present.

kind Required inputs Output
ik joints. Provide site, or select end_effector_body and optional end_effector_offset in its local frame Position targets from damped least-squares IK
joint_sliders joints Named scalar position targets with joint limits
trajectory joints and trajectory as a keyframe object or a path beside the config Time-interpolated joint position targets
differential left_joint, right_joint, wheel_radius, track_width Left and right velocity targets
omni wheels, each with joint, position: [x, y], angle and radius One velocity target per wheel
cpg joints, frequency_hz, and one amplitude per joint Bounded effort from PD tracking of sinusoidal position and velocity targets
direct Optional actuators to limit which existing named actuators are driven Native actuator inputs

direct is an API-level actuator inspection mode. In an application it is restricted to model_viewer with an empty actuator list; controlled apps use one of the other verified kinds. A viewer never advances robot dynamics.

A new IK site defaults to mjrl_ee. If end_effector_body is omitted, setup follows the chain from the last named joint to its unique leaf. A branching chain needs an explicit endpoint. Setup places a collision-free target sphere on a mocap=true body named mjrl_target unless a different mocap_body is configured. A generated marker starts at the tool point in the verified reset pose. An existing mocap target keeps its authored origin unless start_target_at_site is set. Use the viewer drag controls, or send target_x, target_y, and target_z as world-space offsets from the starting position. Both paths use the configured axis bounds. IK solves for the site position. The solver leaves other joints and the simulated position untouched.

Joint sliders accept a partial map of joint names to targets and hold other targets. The MuJoCo viewer's Control panel can also set these targets directly. The generated controls bind the first four axes unless operator_axes selects another subset. command_ranges can narrow command travel per axis.

Set "target_rate_limits": {"elbow": 0.5} to limit the speed of selected slider targets. Give a positive finite maximum in rad/s for each hinge or m/s for each slide. Unlisted joints take the requested target immediately. At each controller tick a limited target moves toward the requested position by at most the configured rate times elapsed simulation time. A single named command, viewer slider edit, or release-button tap latches the destination until it is reached or replaced. Reset starts from the actual joint positions. Measure the resulting effort and settling time under the intended load when choosing rates.

For a calibrated slider pose, supply home as a finite map covering exactly joints, with each value inside both its physical limits and declared command range. Without home, the operator rest uses the model reference pose, clipped to the joint limits. home sets the operator rest and release targets; it does not move the simulated robot on reset. Reset holds the actual supplied joint positions. Supply the calibrated initial pose and base position to verification so the packaged application starts from that verified state. Escape or the pad's release button restores only operator-bound joints to home, preserving targets on unbound joints such as supporting legs.

Trajectory files use schema: "mjrl-trajectory/1", joint_order, strictly increasing time_s from zero, and one q row per timestamp. Positions interpolate linearly in time and hold at the last keyframe. Set loop: true for a loop whose first and last poses match. playback_rate changes progress on the simulation clock. Setup copies a trajectory into motion.trajectory.json.

Wheel commands use +X forward, +Y left, and +Z counterclockwise. Differential drive has zero lateral range. Omni angle is the angle from +X toward +Y of each wheel's driven direction, and position is its body-frame contact offset. The model needs wheel geometry and contacts that physically permit the intended motion. For wheel axes whose positive rotation is reversed, set a differential directions array in left-to-right order, or direction: -1 on an omni wheel. When wheel rates exceed max_wheel_speed, the mixer scales them together to preserve their ratios. W/S moves forward/backward in both modes. A/D turns a differential base and moves an omni base laterally. J/L turns the omni base. The pad uses its left stick for translation and its right stick for yaw.

CPG phase and centre vectors follow joints. Phase defaults to zero and centre to the model's home pose. A joint with a finite limit must accommodate its full excursion. The runtime uses the analytic sinusoidal velocity as the derivative target. Set amplitude_scale between zero and one. The motion's stability depends on the model, posture, contact geometry, gains and phase.

Measured examples

The source models and verification inputs are in assets/planar_arm, assets/differential_drive, and assets/cpg_stance. Use each verify*.json beside its generated controller.json and controls.yaml as in the setup example. Verification compares observed motion to an explicit expected response and repeats the probe with actuation disabled.

Wheel modes need a measured body response for each enabled command axis. Command one enabled axis at a time in the required probes. A differential base normally needs separate forward and yaw probes. An omni base also needs a lateral probe. A declared tolerance must distinguish the expected response from zero and from the measured response with actuation disabled. Packaging checks this evidence and replays the actuated response. The disabled-actuation measurements come from the verification report.

Native MuJoCo 3.11.0 probe on Intel Core i7-14700F Measured response
Planar arm IK, 1.5 s 0.00000514 m tip error after a 0.113 m target displacement
Planar arm sliders, 1.5 s Maximum final joint error 6.42 × 10⁻¹⁰ rad
Planar arm trajectory, 2.0 s Maximum final joint error 8.92 × 10⁻⁷ rad
Differential drive, separate 1.5 s forward and yaw probes 0.500 m forward at a 0.35 m/s command. 0.951 rad/s final yaw rate at a 1 rad/s command
Four-foot stationary CPG, 10.4 s from 2° initial roll 0.0109 rad maximum final joint error, 8.3 mm peak base translation, 2.34° maximum tilt change from the initial pose, and zero MuJoCo warnings

The CPG example demonstrates a bounded stationary cycle under gravity with foot contacts. The omni mixer is covered by joint-rate and saturation tests in test_controllers_generic.py.

Tier 1 policy adapter

Supply an ONNX actor or a trusted TorchScript module and the matching robot model. The adapter accepts unified mjrl-policy-adapter/1 metadata or extracts it from the actor's upstream training configs.

Extract metadata from upstream configs

Choose the config files associated with the actual actor. For example, use the saved environment and agent configs from the same training run.

python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
  --output out/metadata-draft --report out/metadata-extraction.json

Extraction writes metadata.json and manifest.json. Review the provenance of each value, conflicts, missing_fields and unresolved in the manifest. An incomplete extraction exits with status 1 and keeps both files for review. Python files are parsed as static syntax via AST. Imports, inheritance and expressions that the extractor cannot resolve appear in the diagnostics.

The model provides joint names, limits and reference pose. The source policy's observation and action definitions provide channel order. A joint_names list can identify joints used by a configuration, and a default-angle dictionary identifies values by name. Establish the actor's actual observation and action orders from its training definitions. Use graph widths to check shapes after recovering those semantics.

Write a JSON or YAML overrides file with the missing or corrected values you can trace to the source. Rerun extraction to a fresh directory and confirm ready: true before passing the metadata to the adapter.

python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
  --overrides reviewed.json --output out/metadata-ready
python scripts/onboard.py adapt-tier1 actor.onnx --metadata out/metadata-ready/metadata.json \
  --model robot.xml --output out/adapted_actor

After reviewing the draft, the adapter can also extract from the same inputs and overrides in one command. An incomplete extraction keeps the draft and stops before opening the actor. Use a fresh policy output directory.

python scripts/onboard.py adapt-tier1 actor.onnx --config env.yaml --config agent.yaml \
  --overrides reviewed.json --model robot.xml --output out/adapted_from_configs

Supply unified metadata

A complete JSON or YAML document can go straight to adapt-tier1 --metadata. For example, this actor consumes two relative joint positions and returns two position actions. The source and deployment use different joint orders.

{
  "schema": "mjrl-policy-adapter/1",
  "wire_joint_order": ["shoulder", "elbow"],
  "obs_joint_order": ["shoulder", "elbow"],
  "action_joint_order": ["shoulder", "elbow"],
  "source_action_joint_order": ["elbow", "shoulder"],
  "default_joint_pos": {"shoulder": 0, "elbow": 0},
  "action_scale": {"shoulder": 0.15, "elbow": 0.3},
  "control": {"kp": 20, "kd": 0.75, "effort_limit": 10, "control_hz": 50},
  "observation": {
    "terms": [{"name": "joint_pos", "source_joint_order": ["elbow", "shoulder"]}]
  }
}
python scripts/onboard.py adapt-tier1 actor.onnx --metadata policy.json \
  --model out/arm_sliders/robot.xml --output out/adapted_actor

The model check resolves each wire joint by name, verifies the home pose against the compiled stops and reads finite position limits from the model when possible. An explicit joint_limits map can supply narrower limits. The default targets are positions relative to default_joint_pos. Metadata names the source graph's actual quantities and units. Tensor dimensions establish only shape compatibility.

The adapter emits layout.json, a native controller.json, and an actor.onnx with one float32 obs[1, observation.dim] input and one actions[1, action.dim] output. Supported observation quantities are base_ang_vel, base_lin_vel, projected_gravity, commands, joint_pos, joint_vel, joint_torque and actions for the previous action. Unknown terms or incompatible metadata fail during adaptation. Base velocities and projected gravity are expressed in the base frame. For interactive policies, supply command_ranges.twist with a range containing zero for each observed command channel. The adapter writes a validated controls.yaml for axes with nonzero travel. A policy with no observed command term has an empty command order. A policy that observes commands can omit ranges to keep those inputs fixed at zero. Both fixed cases list controls: null in the adapt-tier1 output. Native replay rejects nonzero commands outside a declared range or on an axis without one.

For each axis with nonzero trained travel, verify a fixed command with the other enabled axes at zero. Use a body-frame mean velocity or tracking-error window with its target set to the commanded value. Choose a tolerance that distinguishes the response from zero. Combined-motion probes can supplement the separate axis checks. Native packaging checks this evidence and reproduces the actuated response.

  • observation.terms is the deployment term order. The canonical graph input groups each term's history together, oldest frame first. Describe the source with source_term_order, source_history_order, and source_order: "frame_major" when needed. Frame-major input requires matching history lengths and strides. Each term may carry history_length and history_stride.
  • source_joint_order on a per-joint term permutes that term into the source policy's order. For commands, use observation.command_order and a term's source_command_order. The default command channels are lin_vel_x, lin_vel_y and ang_vel_z.
  • A term may specify a clip pair and a scale scalar or per-frame channel vector. The adapter clips first and then scales. If the source requires external normalization, specify observation.normalization with mean, std, and epsilon. Vectors follow source order after the observation transform. The graph computes (x - mean) / (std + epsilon). Omit this object when normalization is already in the source graph or there is none.
  • For multiple source inputs, specify observation.inputs as ordered {"name": "...", "dim": width} objects. The canonical observation is transformed and split in that order. A TorchScript module receives tensors in the listed positional order.
  • A scalar action scale stays in the layout. A map by joint moves the action transform into the graph and sets the layout's scale to one. If the actor observes its previous actions, the graph inverts that transform for the history term. The adapter rejects irreversible transforms for this case.
  • For mixed joint commands, specify action.targets by action joint with kind: "position" or "velocity", a numeric offset, and optional clip limits in the target's units. Specify joint_order on each per-joint observation term when terms use distinct subsets. Supply scalar or full-wire gain and effort maps for the actuated joints. A velocity target needs zero stiffness and positive damping. List mechanically passive joints under passive_joints if they appear in the wire or observation order.

Adaptation validates graph dimensions and checks the converted graph against the source on ten numerical inputs. The report gives the maximum absolute difference. A passing graph check establishes numerical agreement over those inputs. Native physics verification measures the simulated robot's response separately.

Native hybrid policy export

python scripts/export.py --task my_robot --checkpoint model_N.pt \
  --native-layout --out out/native_policy

The native export carries robot.xml, its model assets, the actor, the policy layout and the controller configuration. It copies operator bindings when the robot spec supplies them. The model is exported from the live scene, including the compiled actuators, joint armatures and configured reset keyframe. Export uses the checkpoint's saved command curriculum range. Verify the exact exported model and controller before packaging.

An export automatically uses mjrl-native-layout/1 when the legacy scalar position layout cannot describe its action or observation semantics. For example, a wheeled biped may observe positions for its leg joints and velocities for both legs and wheels. Its native layout names both selections explicitly and identifies each actor output as a position or velocity target. The runtime applies the named affine transform and the training target clip, then evaluates PD per physics step. A wheel's control law tracks velocity with zero position stiffness. The legacy Rust bundle refuses the native layout. Passive joints can contribute position, velocity, and torque observations. The runtime lets their model dynamics determine their response.

Use a fixed body command to measure the app's tracking:

python scripts/play.py --app out/my_robot.app --headless --steps 100 \
  --command 0.08 0 0 --evaluation-out out/tracking.json

The report includes final and mean body velocity, axis-by-axis errors, peak tilt and simulated duration. Use --reference-out for raw simulation frames and the onboarding verifier for acceptance against declared tolerances.

The hybrid smoke asset and robot spec input exercise this path. A single native CPU PPO iteration on four environments produced a 19-input, four-output actor. Its Torch-to-ONNX maximum difference was 1.43 × 10⁻⁶. At a 0.08 m/s command over two simulated seconds, the checkpoint and portable replay differed by at most 0.00188 rad in final joint positions and 0.00304 rad/s in final joint velocities. The policy failed separate forward and yaw tracking probes. It finished at −0.00148 m/s for a 0.08 m/s request and 0.02854 rad/s for a 0.15 rad/s request. This smoke establishes execution and export parity on the fixture.

Explicit sensor frames and calibrated motors

The default base observations use the authored root body frame. An upstream policy may instead observe an IMU site's angular velocity while keeping gravity in the root frame. Declare only the overridden terms in observation.frames:

{"observation": {
  "terms": [{"name": "base_ang_vel"}, {"name": "projected_gravity"}],
  "frames": {"base_ang_vel": {"kind": "site", "name": "imu"}}
}}

Frames apply to base_ang_vel, base_lin_vel and projected_gravity; each must name an observed term and an existing body or site. Local velocities use that frame's origin. These contracts use mjrl-native-layout/1 to prevent a legacy reader from silently applying different sensor semantics.

For an actor trained with a calibrated BAM motor, install the optional bam extra under Python 3.12. In the top-level motor_model block, set kind to bam and provide the calibrated motor_name, model, firmware gain kp, supply voltage vin, current limit max_current, supply resistance vin_drop_resistance and minimum voltage vin_min from the training setup. This opt-in law requires position targets and replaces the ordinary PD effort calculation. motor_model.kp is a firmware gain, distinct from the export's control.kp. The declared effort limit still bounds output. BAM updates load-dependent friction and damping at every physics tick, resets independently for each probe, and updates passive friction with actuation disabled for the counterfactual. Native bundles pin better-actuator-models==1.0.2 and declare its Python >=3.12,<3.13 requirement. The original actor is not retrained.