Default humanoid intake to command-following gait and support bounded Cartesian target dragging for arms. Carry model and actuator semantics through calibration, native verification and packaging. Document reusable intake decisions and cover them with regression tests.
21 KiB
Deterministic controllers and external policies
Use scripts/onboard.py setup-tier2 to turn a MuJoCo model into a native controller
setup. Use adapt-tier1 when an existing actor and explicit training metadata are
available. Both routes produce artifacts that can be verified and packaged independently
of a training task.
Tier 2 controller setup
python scripts/onboard.py build-mjcf assets/planar_arm/robot.urdf \
--output out/planar_arm.xml
python scripts/onboard.py setup-tier2 --model out/planar_arm.xml \
--config assets/planar_arm/ik.json --output out/arm_ik
cp assets/planar_arm/verify_ik.json out/arm_ik/verify-input.json
python scripts/onboard.py verify --model out/arm_ik/robot.xml \
--config out/arm_ik/verify-input.json --report out/arm_ik/verification.json
python scripts/deploy.py --robot planar_arm --model out/arm_ik/robot.xml \
--control out/arm_ik/controller.json --controls out/arm_ik/controls.yaml \
--verification out/arm_ik/verification.json
The setup writes robot.xml with controller-specific actuators, a resolved
controller.json, and controls.yaml validated against the same dictionary used
in training and deployment. A passive direct model has no bindings. Referenced
meshes and textures remain at their original paths until the bundler copies them.
The config schema is mjrl-controller/1. A setup may omit that field and accept the
current schema. All lengths are in metres, angles in radians and times in seconds.
Hinges and slides use native MuJoCo units for their position, velocity and effort.
control_hz may be supplied when one control period is a whole number of physics
steps. By default it is the closest available rate to 50 Hz obtained by rounding
the number of substeps. Set gains and effort limits for your model. The bench
defaults are kp: 40, kd: 4, and effort_limit: 30 for position and CPG
controllers, or kv: 5 and effort_limit: 30 for velocity actuators.
For ik, joint_sliders, trajectory, and cpg, each of kp, kd, and
effort_limit accepts either one scalar or a map covering every name in joints
exactly. Maps are resolved by name, independently of actuator order. Values must
be finite; kp and effort_limit must be positive and kd nonnegative. For
example, "kp": {"hip": 70, "shoulder": 30} gives those two controlled joints
different stiffnesses. Missing or extra names and positional arrays are rejected.
Normalization intersects the requested effort with the source actuator force limits, pure motor input limits, and joint actuator-force limits. Source force and motor input bounds are converted through the authored gear before creating unit-gear actuators. Position and velocity target ranges are not effort limits. The resulting model retains the effective bounds even when the configuration requests a larger effort. Each joint must retain effort authority in both directions. CPG clips its computed effort to the smaller magnitude of the two effective bounds; position servos retain the asymmetric bounds when present.
kind |
Required inputs | Output |
|---|---|---|
ik |
joints. Provide site, or select end_effector_body and optional end_effector_offset in its local frame |
Position targets from damped least-squares IK |
joint_sliders |
joints |
Named scalar position targets with joint limits |
trajectory |
joints and trajectory as a keyframe object or a path beside the config |
Time-interpolated joint position targets |
differential |
left_joint, right_joint, wheel_radius, track_width |
Left and right velocity targets |
omni |
wheels, each with joint, position: [x, y], angle and radius |
One velocity target per wheel |
cpg |
joints, frequency_hz, and one amplitude per joint |
Bounded effort from PD tracking of sinusoidal position and velocity targets |
direct |
Optional actuators to limit which existing named actuators are driven |
Native actuator inputs |
direct is an API-level actuator inspection mode. In an application it is
restricted to model_viewer with an empty actuator list; controlled apps use one
of the other verified kinds. A viewer never advances robot dynamics.
A new IK site defaults to mjrl_ee. If end_effector_body is omitted, setup follows
the chain from the last named joint to its unique leaf. A branching chain needs an
explicit endpoint. Setup places a collision-free target sphere on a mocap=true
body named mjrl_target unless a different mocap_body is configured. A generated
marker starts at the tool point in the verified reset pose. An existing mocap
target keeps its authored origin unless start_target_at_site is set. Use
the viewer drag controls, or
send target_x, target_y, and target_z as world-space offsets from the
starting position. Both paths use the configured axis bounds. IK solves for the site position.
The solver leaves other joints and the simulated position untouched.
Joint sliders accept a partial map of joint names to targets and hold other
targets. The MuJoCo viewer's Control panel can also set these targets directly.
The generated controls bind the first four axes unless operator_axes selects
another subset. command_ranges can narrow command travel per axis.
Set "target_rate_limits": {"elbow": 0.5} to limit the speed of selected
slider targets. Give a positive finite maximum in rad/s for each hinge or m/s
for each slide. Unlisted joints take the requested target immediately. At each
controller tick a limited target moves toward the requested position by at most
the configured rate times elapsed simulation time. A single named command,
viewer slider edit, or release-button tap latches the destination until it is
reached or replaced. Reset starts from the actual joint positions. Measure the
resulting effort and settling time under the intended load when choosing rates.
For a calibrated slider pose, supply home as a finite map covering exactly
joints, with each value inside both its physical limits and declared command
range. Without home, the operator rest uses the model reference pose, clipped
to the joint limits. home sets the operator rest and release targets; it does
not move the simulated robot on reset. Reset holds the actual supplied joint
positions. Supply the calibrated initial pose and base position to verification
so the packaged application starts from that verified state. Escape or the pad's
release button restores only operator-bound joints to home, preserving targets
on unbound joints such as supporting legs.
Trajectory files use schema: "mjrl-trajectory/1", joint_order, strictly
increasing time_s from zero, and one q row per timestamp. Positions interpolate
linearly in time and hold at the last keyframe. Set loop: true for a loop whose
first and last poses match. playback_rate changes progress on the simulation
clock. Setup copies a trajectory into motion.trajectory.json.
Wheel commands use +X forward, +Y left, and +Z counterclockwise. Differential
drive has zero lateral range. Omni angle is the angle from +X toward +Y of
each wheel's driven direction, and position is its body-frame contact offset.
The model needs wheel geometry and contacts that physically permit the intended
motion. For wheel axes whose positive rotation is reversed, set a differential
directions array in left-to-right order, or direction: -1 on an omni wheel.
When wheel rates exceed max_wheel_speed, the mixer scales them together to
preserve their ratios. W/S moves forward/backward in both modes. A/D turns a
differential base and moves an omni base laterally. J/L turns the omni base.
The pad uses its left stick for translation and its right stick for yaw.
CPG phase and centre vectors follow joints. Phase defaults to zero and centre to
the model's home pose. A joint with a finite limit must accommodate its full
excursion. The runtime uses the analytic sinusoidal velocity as the derivative
target. Set amplitude_scale between zero and one. The motion's stability
depends on the model, posture, contact geometry, gains and phase.
Measured examples
The source models and verification inputs are in
assets/planar_arm,
assets/differential_drive, and
assets/cpg_stance.
Use each verify*.json beside its generated controller.json and controls.yaml
as in the setup example. Verification compares observed motion to an explicit
expected response and repeats the probe with actuation disabled.
Wheel modes need a measured body response for each enabled command axis. Command one enabled axis at a time in the required probes. A differential base normally needs separate forward and yaw probes. An omni base also needs a lateral probe. A declared tolerance must distinguish the expected response from zero and from the measured response with actuation disabled. Packaging checks this evidence and replays the actuated response. The disabled-actuation measurements come from the verification report.
| Native MuJoCo 3.11.0 probe on Intel Core i7-14700F | Measured response |
|---|---|
| Planar arm IK, 1.5 s | 0.00000514 m tip error after a 0.113 m target displacement |
| Planar arm sliders, 1.5 s | Maximum final joint error 6.42 × 10⁻¹⁰ rad |
| Planar arm trajectory, 2.0 s | Maximum final joint error 8.92 × 10⁻⁷ rad |
| Differential drive, separate 1.5 s forward and yaw probes | 0.500 m forward at a 0.35 m/s command. 0.951 rad/s final yaw rate at a 1 rad/s command |
| Four-foot stationary CPG, 10.4 s from 2° initial roll | 0.0109 rad maximum final joint error, 8.3 mm peak base translation, 2.34° maximum tilt change from the initial pose, and zero MuJoCo warnings |
The CPG example demonstrates a bounded stationary cycle under gravity with foot
contacts. The omni mixer is covered by joint-rate
and saturation tests in test_controllers_generic.py.
Tier 1 policy adapter
Supply an ONNX actor or a trusted TorchScript module and the matching robot
model. The adapter accepts unified mjrl-policy-adapter/1 metadata or extracts
it from the actor's upstream training configs.
Extract metadata from upstream configs
Choose the config files associated with the actual actor. For example, use the saved environment and agent configs from the same training run.
python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
--output out/metadata-draft --report out/metadata-extraction.json
Extraction writes metadata.json and manifest.json. Review the provenance
of each value, conflicts, missing_fields and unresolved in the manifest.
An incomplete extraction exits with status 1 and keeps both files for review.
Python files are parsed as static syntax via AST. Imports, inheritance and
expressions that the extractor cannot resolve appear in the diagnostics.
The model provides joint names, limits and reference pose. The source policy's
observation and action definitions provide channel order. A joint_names list
can identify joints used by a configuration, and a default-angle dictionary
identifies values by name. Establish the actor's actual observation and action
orders from its training definitions. Use graph widths to check shapes after
recovering those semantics.
Write a JSON or YAML overrides file with the missing or corrected values you
can trace to the source. Rerun extraction to a fresh directory and confirm
ready: true before passing the metadata to the adapter.
python scripts/onboard.py extract-tier1 env.yaml agent.yaml --model robot.xml \
--overrides reviewed.json --output out/metadata-ready
python scripts/onboard.py adapt-tier1 actor.onnx --metadata out/metadata-ready/metadata.json \
--model robot.xml --output out/adapted_actor
After reviewing the draft, the adapter can also extract from the same inputs and overrides in one command. An incomplete extraction keeps the draft and stops before opening the actor. Use a fresh policy output directory.
python scripts/onboard.py adapt-tier1 actor.onnx --config env.yaml --config agent.yaml \
--overrides reviewed.json --model robot.xml --output out/adapted_from_configs
Supply unified metadata
A complete JSON or YAML document can go straight to adapt-tier1 --metadata.
For example, this actor consumes two relative joint positions and returns two
position actions. The source and deployment use different joint orders.
{
"schema": "mjrl-policy-adapter/1",
"wire_joint_order": ["shoulder", "elbow"],
"obs_joint_order": ["shoulder", "elbow"],
"action_joint_order": ["shoulder", "elbow"],
"source_action_joint_order": ["elbow", "shoulder"],
"default_joint_pos": {"shoulder": 0, "elbow": 0},
"action_scale": {"shoulder": 0.15, "elbow": 0.3},
"control": {"kp": 20, "kd": 0.75, "effort_limit": 10, "control_hz": 50},
"observation": {
"terms": [{"name": "joint_pos", "source_joint_order": ["elbow", "shoulder"]}]
}
}
python scripts/onboard.py adapt-tier1 actor.onnx --metadata policy.json \
--model out/arm_sliders/robot.xml --output out/adapted_actor
The model check resolves each wire joint by name, verifies the home pose against
the compiled stops and reads finite position limits from the model when possible.
An explicit joint_limits map can supply narrower limits. The default targets
are positions relative to default_joint_pos. Metadata names the source graph's
actual quantities and units. Tensor dimensions establish only shape compatibility.
The adapter emits layout.json, a native controller.json, and an
actor.onnx with one float32
obs[1, observation.dim] input and one actions[1, action.dim] output. Supported
observation quantities are base_ang_vel, base_lin_vel, projected_gravity,
commands, joint_pos, joint_vel, joint_torque and actions for the
previous action. Unknown terms or incompatible metadata fail during adaptation.
Base velocities and projected gravity are expressed in the base frame.
For interactive policies, supply command_ranges.twist with a range containing
zero for each observed command channel. The adapter writes a validated
controls.yaml for axes with nonzero travel. A policy with no observed
command term has an empty command order. A policy that observes commands can
omit ranges to keep those inputs fixed at zero. Both fixed cases list
controls: null in the adapt-tier1 output. Native replay rejects nonzero commands
outside a declared range or on an axis without one.
For each axis with nonzero trained travel, verify a fixed command with the other enabled axes at zero. Use a body-frame mean velocity or tracking-error window with its target set to the commanded value. Choose a tolerance that distinguishes the response from zero. Combined-motion probes can supplement the separate axis checks. Native packaging checks this evidence and reproduces the actuated response.
observation.termsis the deployment term order. The canonical graph input groups each term's history together, oldest frame first. Describe the source withsource_term_order,source_history_order, andsource_order: "frame_major"when needed. Frame-major input requires matching history lengths and strides. Each term may carryhistory_lengthandhistory_stride.source_joint_orderon a per-joint term permutes that term into the source policy's order. For commands, useobservation.command_orderand a term'ssource_command_order. The default command channels arelin_vel_x,lin_vel_yandang_vel_z.- A term may specify a
clippair and ascalescalar or per-frame channel vector. The adapter clips first and then scales. If the source requires external normalization, specifyobservation.normalizationwithmean,std, andepsilon. Vectors follow source order after the observation transform. The graph computes(x - mean) / (std + epsilon). Omit this object when normalization is already in the source graph or there is none. - For multiple source inputs, specify
observation.inputsas ordered{"name": "...", "dim": width}objects. The canonical observation is transformed and split in that order. A TorchScript module receives tensors in the listed positional order. - A scalar action scale stays in the layout. A map by joint moves the action transform into the graph and sets the layout's scale to one. If the actor observes its previous actions, the graph inverts that transform for the history term. The adapter rejects irreversible transforms for this case.
- For mixed joint commands, specify
action.targetsby action joint withkind: "position"or"velocity", a numericoffset, and optionalcliplimits in the target's units. Specifyjoint_orderon each per-joint observation term when terms use distinct subsets. Supply scalar or full-wire gain and effort maps for the actuated joints. A velocity target needs zero stiffness and positive damping. List mechanically passive joints underpassive_jointsif they appear in the wire or observation order.
Adaptation validates graph dimensions and checks the converted graph against the source on ten numerical inputs. The report gives the maximum absolute difference. A passing graph check establishes numerical agreement over those inputs. Native physics verification measures the simulated robot's response separately.
Native hybrid policy export
python scripts/export.py --task my_robot --checkpoint model_N.pt \
--native-layout --out out/native_policy
The native export carries robot.xml, its model assets, the actor, the policy
layout and the controller configuration. It copies operator bindings when the
robot spec supplies them. The model is exported from the live scene, including
the compiled actuators, joint armatures and configured reset keyframe. Export
uses the checkpoint's saved command curriculum range. Verify the exact exported
model and controller before packaging.
An export automatically uses mjrl-native-layout/1 when the legacy scalar
position layout cannot describe its action or observation semantics. For
example, a wheeled biped may observe positions for its leg joints and velocities
for both legs and wheels. Its native layout names both selections explicitly and
identifies each actor output as a position or velocity target. The runtime
applies the named affine transform and the training target clip, then evaluates
PD per physics step. A wheel's control law tracks velocity with zero position
stiffness. The legacy Rust bundle refuses the native layout.
Passive joints can contribute position, velocity, and torque observations.
The runtime lets their model dynamics determine their response.
Use a fixed body command to measure the app's tracking:
python scripts/play.py --app out/my_robot.app --headless --steps 100 \
--command 0.08 0 0 --evaluation-out out/tracking.json
The report includes final and mean body velocity, axis-by-axis errors, peak
tilt and simulated duration. Use --reference-out for raw simulation frames
and the onboarding verifier for acceptance against declared tolerances.
The hybrid smoke asset and robot spec input exercise this path. A single native CPU PPO iteration on four environments produced a 19-input, four-output actor. Its Torch-to-ONNX maximum difference was 1.43 × 10⁻⁶. At a 0.08 m/s command over two simulated seconds, the checkpoint and portable replay differed by at most 0.00188 rad in final joint positions and 0.00304 rad/s in final joint velocities. The policy failed separate forward and yaw tracking probes. It finished at −0.00148 m/s for a 0.08 m/s request and 0.02854 rad/s for a 0.15 rad/s request. This smoke establishes execution and export parity on the fixture.
Explicit sensor frames and calibrated motors
The default base observations use the authored root body frame. An upstream
policy may instead observe an IMU site's angular velocity while keeping gravity
in the root frame. Declare only the overridden terms in observation.frames:
{"observation": {
"terms": [{"name": "base_ang_vel"}, {"name": "projected_gravity"}],
"frames": {"base_ang_vel": {"kind": "site", "name": "imu"}}
}}
Frames apply to base_ang_vel, base_lin_vel and projected_gravity; each must
name an observed term and an existing body or site. Local velocities use that
frame's origin. These contracts use mjrl-native-layout/1 to prevent a legacy
reader from silently applying different sensor semantics.
For an actor trained with a calibrated BAM motor, install the optional bam
extra under Python 3.12. In the top-level motor_model block, set kind to
bam and provide the calibrated motor_name, model, firmware gain kp,
supply voltage vin, current limit max_current, supply resistance
vin_drop_resistance and minimum voltage vin_min from the training setup.
This opt-in law requires position targets and replaces the ordinary PD effort
calculation. motor_model.kp is a firmware gain, distinct from the export's
control.kp. The declared effort limit still bounds output. BAM updates
load-dependent friction and damping at every physics tick, resets independently
for each probe, and updates passive friction with actuation disabled for the
counterfactual. Native bundles pin better-actuator-models==1.0.2 and declare its
Python >=3.12,<3.13 requirement. The original actor is not retrained.