feat: release v0.3 with RP1 trained skills and scaled arenas
This commit is contained in:
@@ -29,11 +29,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.f32 filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.stl filter=lfs diff=lfs merge=lfs -text
|
||||
*.STL filter=lfs diff=lfs merge=lfs -text
|
||||
*.wav filter=lfs diff=lfs merge=lfs -text
|
||||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
*.webp filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
@@ -11,7 +11,7 @@ license: apache-2.0
|
||||
|
||||
# Browser robot simulation platform
|
||||
|
||||
MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers) and **Microban** Robot Packages. Physics and ONNX inference run locally in the browser; no simulation backend is required.
|
||||
MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), **Microban**, and **RP1** Robot Packages. Physics and ONNX inference run locally in the browser; no simulation backend is required.
|
||||
|
||||
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
|
||||
|
||||
@@ -20,7 +20,8 @@ The application opens directly into simulation, defaults to Microduck, and keeps
|
||||
| Robot | Default scene / presentation | Controls |
|
||||
|---|---|---|
|
||||
| Microduck | Shared 4.5 × 4.5 m bounded arena, ball, original GLB/palette/comic effects | Existing walking, skating, skills and recovery |
|
||||
| Microban | Shared 4.5 × 4.5 m bounded arena, dark/orange grid and comic UI, MJCF robot visuals | Walking policy + independent head rotation |
|
||||
| Microban | Robot-scaled 5.5 × 5.5 m arena, dark/orange grid and comic UI, MJCF robot visuals | Walking policy + independent head rotation |
|
||||
| RP1 | Robot-scaled 22 × 22 m arena/comic UI, original 24-DOF MJCF visuals | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
|
||||
|
||||
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
|
||||
|
||||
@@ -40,12 +41,13 @@ Microduck policies / assets originate from [pollen-robotics/microduck](https://g
|
||||
| Microduck ground pick | G |
|
||||
| Microduck forward roll; roller crouch | T |
|
||||
| Microduck feet / rollers | M or quickbar |
|
||||
| Restore rear chase camera (both robots) | C |
|
||||
| Restore rear chase camera (all robots) | C |
|
||||
| Microban independent head rotation | H |
|
||||
| RP1 trained dance (10 s segment) / wave (4.36 s) | J / H |
|
||||
|
||||
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
|
||||
|
||||
All bundled robots, and future Packages omitting `presentation.world`, use the shared **4.5 × 4.5 m** arena (length and width increased 50% from 3 m). Its four real collision walls are 0.25 m high and 0.05 m thick, with Microduck's dark glowing grid. Microban's interior stays flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. Both robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Microduck mouse dragging still applies physical forces to the robot / ball. Microban mouse dragging orbits the camera. Gamepad / touch are not part of this release's acceptance scope.
|
||||
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Microduck mouse dragging still applies physical forces to the robot / ball. Microban / RP1 mouse dragging orbits the camera. Gamepad / touch are not part of this release's acceptance scope.
|
||||
|
||||
## Microban fidelity and limits
|
||||
|
||||
@@ -60,9 +62,17 @@ All bundled robots, and future Packages omitting `presentation.world`, use the s
|
||||
|
||||
BAM battery / friction / delay modeling, Placo squat, training, hardware control, ROS, camera observations, external Packages and online sharing are **not implemented** in this release. Ghost networking, stickers and celebration / entrance sequences are deferred. Audio retains the existing `SOUND_DISABLED=true` master setting; simulation never waits for an audio-unlock gesture.
|
||||
|
||||
## RP1 trained skills
|
||||
|
||||
Walking uses final `model_9000.pt` from `rp1_24dof_flat/2026-10-28_15-47-49`, float32 `[1,243]` (three term-major history frames) → `[1,24]`. Dance uses resumed `2026-10-29_11-05-14_dance_resume/model_23998.pt`; wave retains its separate final `model_7999.pt`. Their actual actor branch is **student**, float32 `[1,126]` → `[1,24]`, with joint-reference position/velocity commands. All exports include the empirical normalizer and use their own reference defaults, named joint order and action history.
|
||||
|
||||
Independent ONNX skills share `load/reset/run/dispose` and an exclusive full-body FSM: walk → settle → blend in → play → blend back to walk. Release movement keys before requesting a skill; movement interrupts it. Reference playback is simulation-clocked, freezes on pause, restarts on each activation, and never writes kinematic poses. **Wave plays its complete 4.36 s clip. Dance exposes only the validated first 10 s**: the previous dance checkpoint fell near 20 s during full 131.46 s playback; full-length playback of the updated checkpoint has not been qualified. These are trained policies, not scripted motion substitutes. Completed training does not imply exact training-simulator or hardware equivalence. See [RP1 integration and weight updates](docs/robots/rp1.md) and [completed-skills verification](docs/superpowers/verification/2026-10-29-rp1-completed-skills.md), plus the [23998 dance update](docs/superpowers/verification/2026-10-29-rp1-dance-23998.md).
|
||||
|
||||
**RP1 description asset licensing is not supplied in the upstream checkout. Local evaluation only; confirm rights before public deployment/redistribution.** Provenance and caveats are recorded in [the package SOURCE.md](app/public/robots/rp1/SOURCE.md).
|
||||
|
||||
## Development
|
||||
|
||||
Binary robot meshes and ONNX files use Git LFS:
|
||||
Binary robot meshes, ONNX policies and `.f32` motion samples use Git LFS:
|
||||
|
||||
```bash
|
||||
git lfs install
|
||||
@@ -106,9 +116,9 @@ For prefixed hosting, build and preview with `VITE_BASE=/play/`; `DEPLOY_PATH=/p
|
||||
- `app/src/robots/microduck/runtime.js`: compatibility adapter retaining original skill / recovery quirks and GLB presentation while using shared runtimes, scheduler, input, observation / action mapping and actuation.
|
||||
- `app/src/game/worlds/arena/`: arena physics composition and rendering. `game/game.js` and `game/arena.js` are compatibility entry points, not generic engines.
|
||||
|
||||
A robot expressible with existing capabilities can be added by a Package and index entry, without a robot-id branch in the platform. Current generic scope is floating-base robots with scalar joint groups, unit-gear actuators, supported proprioceptive terms, ONNX primary control, position targets and optional sinusoidal overlays. Unsupported capabilities are rejected, not silently simulated incorrectly. New capabilities require explicit registry / schema work.
|
||||
A robot expressible with existing capabilities can be added by a Package and index entry, without a robot-id branch in the platform. Current generic scope is floating-base robots with scalar joint groups, unit-gear actuators, supported proprioceptive / joint-reference terms, ONNX primary control, independent episodic policies, simulation-clocked motion references, position targets and optional sinusoidal overlays. Unsupported capabilities are rejected, not silently simulated incorrectly. New capabilities require explicit registry / schema work.
|
||||
|
||||
See [Package authoring](docs/robots/package-authoring.md), [Microban details](docs/robots/microban.md), [approved design](docs/superpowers/specs/2026-10-27-multi-robot-platform-design.md), [implementation plan](docs/superpowers/plans/2026-10-27-multi-robot-platform-implementation.md) and [initial verification](docs/superpowers/verification/2026-10-27-multi-robot-platform.md) and [style/camera verification](docs/superpowers/verification/2026-10-28-style-and-chase-camera.md), plus [arena-style verification](docs/superpowers/verification/2026-10-28-arena-presentation.md) and [current bounded-default verification](docs/superpowers/verification/2026-10-28-default-bounded-arena.md).
|
||||
See [Package authoring](docs/robots/package-authoring.md), [Microban details](docs/robots/microban.md), [approved design](docs/superpowers/specs/2026-10-27-multi-robot-platform-design.md), [implementation plan](docs/superpowers/plans/2026-10-27-multi-robot-platform-implementation.md) and [initial verification](docs/superpowers/verification/2026-10-27-multi-robot-platform.md) and [style/camera verification](docs/superpowers/verification/2026-10-28-style-and-chase-camera.md), plus [arena-style verification](docs/superpowers/verification/2026-10-28-arena-presentation.md) and [initial bounded-default verification](docs/superpowers/verification/2026-10-28-default-bounded-arena.md). Current size-adaptive behavior is documented in [robot-scaled arena verification](docs/superpowers/verification/2026-10-28-robot-scaled-arena.md).
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -57,8 +57,8 @@ test("arena appearance matches Microduck while Microban physics stays flat and W
|
||||
};
|
||||
});
|
||||
expect(state).toEqual({
|
||||
cell: 0.1,
|
||||
section: 0.6,
|
||||
cell: 0.1 * (0.33 / 0.27),
|
||||
section: 0.6 * (0.33 / 0.27),
|
||||
color: 0xffb366,
|
||||
reveal: 1,
|
||||
topo: 0,
|
||||
|
||||
+101
-67
@@ -2,21 +2,37 @@ import { test, expect } from "@playwright/test";
|
||||
import { bootDuck } from "./support/simulation.js";
|
||||
import { arenaWalls } from "../src/platform/worlds/arena.js";
|
||||
const boundary = { half_extent: 2.25, wall_height: 0.25, wall_thickness: 0.05 };
|
||||
test("Microduck and Microban share the 4.5m collision arena; visual inner faces match and C remains attached", async ({
|
||||
test("all three robots keep Microduck's arena-to-body proportions; visual and physical walls agree", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
await bootDuck(page);
|
||||
for (const id of ["microduck-legs", "microban"]) {
|
||||
if (id === "microban")
|
||||
for (const [id, scale] of [
|
||||
["microduck-legs", 1],
|
||||
["microban", 0.33 / 0.27],
|
||||
["rp1", 1.32 / 0.27],
|
||||
]) {
|
||||
if (id !== "microduck-legs")
|
||||
expect(
|
||||
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
|
||||
await page.evaluate((id) => window.__gameApi.switchRobot(id), id),
|
||||
).toBe(true);
|
||||
const fitted = Object.fromEntries(
|
||||
Object.entries(boundary).map(([k, v]) => [k, v * scale]),
|
||||
);
|
||||
const state = await page.evaluate(() => {
|
||||
const r = window.rl;
|
||||
return {
|
||||
id: r.world.id,
|
||||
boundary: r.world.boundary,
|
||||
scale: r.world.scale,
|
||||
following: r.chaseCam,
|
||||
grid: r.scene.getObjectByName("arena-grid")
|
||||
? {
|
||||
cell: r.scene.getObjectByName("arena-grid").material.uniforms
|
||||
.uCell.value,
|
||||
fade: r.scene.getObjectByName("arena-grid").material.uniforms
|
||||
.uFadeDist.value,
|
||||
}
|
||||
: null,
|
||||
walls: ["wall_px", "wall_nx", "wall_py", "wall_ny"].map((name) => {
|
||||
const g = r.model.geom(name).id;
|
||||
return {
|
||||
@@ -33,24 +49,30 @@ test("Microduck and Microban share the 4.5m collision arena; visual inner faces
|
||||
};
|
||||
});
|
||||
expect(state.id).toBe("arena");
|
||||
expect(state.boundary).toEqual(boundary);
|
||||
expect(state.boundary).toEqual(fitted);
|
||||
expect(state.scale).toBe(scale);
|
||||
expect(state.following).toBe(true);
|
||||
expect(state.walls).toEqual(
|
||||
arenaWalls(boundary).map((w) => ({ ...w, type: 6, body: 0 })),
|
||||
arenaWalls(fitted).map((w) => ({ ...w, type: 6, body: 0 })),
|
||||
);
|
||||
expect(state.visuals).toEqual([
|
||||
[2.25, 0.125, 0],
|
||||
[-2.25, 0.125, 0],
|
||||
[0, 0.125, 2.25],
|
||||
[0, 0.125, -2.25],
|
||||
[fitted.half_extent, fitted.wall_height / 2, 0],
|
||||
[-fitted.half_extent, fitted.wall_height / 2, 0],
|
||||
[0, fitted.wall_height / 2, fitted.half_extent],
|
||||
[0, fitted.wall_height / 2, -fitted.half_extent],
|
||||
]);
|
||||
if (state.grid) {
|
||||
expect(state.grid.cell).toBeCloseTo(0.1 * scale, 8);
|
||||
expect(state.grid.fade).toBeCloseTo(3 * scale, 8);
|
||||
}
|
||||
}
|
||||
// QA overview only; normal entry remains the rear chase framing above.
|
||||
await page.locator("canvas").click({ position: { x: 1000, y: 700 } });
|
||||
await page.evaluate(() => {
|
||||
const r = window.rl;
|
||||
r.controls.maxDistance = 8;
|
||||
r.camera.position.set(-3, 3, 3);
|
||||
const scale = r.world.scale;
|
||||
r.controls.maxDistance = 8 * scale;
|
||||
r.camera.position.set(-3 * scale, 3 * scale, 3 * scale);
|
||||
r.controls.target.set(0, 0, 0);
|
||||
r.controls.update();
|
||||
});
|
||||
@@ -87,63 +109,75 @@ test("a future Package with no world field automatically loads the bounded defau
|
||||
world: window.rl.world.id,
|
||||
bounds: window.rl.world.boundary,
|
||||
})),
|
||||
).toEqual({ id: "future-bot", world: "arena", bounds: boundary });
|
||||
).toEqual({
|
||||
id: "future-bot",
|
||||
world: "arena",
|
||||
bounds: Object.fromEntries(
|
||||
Object.entries(boundary).map(([k, v]) => [k, v * (0.33 / 0.27)]),
|
||||
),
|
||||
});
|
||||
});
|
||||
test("real MuJoCo wall contacts block commanded motion on all four sides without coordinate clamping", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
await bootDuck(page);
|
||||
expect(
|
||||
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
|
||||
).toBe(true);
|
||||
const results = await page.evaluate(async () => {
|
||||
const r = window.rl,
|
||||
report = [];
|
||||
for (const [name, axis, sign, yaw] of [
|
||||
["wall_px", 0, 1, 0],
|
||||
["wall_nx", 0, -1, Math.PI],
|
||||
["wall_py", 1, 1, Math.PI / 2],
|
||||
["wall_ny", 1, -1, -Math.PI / 2],
|
||||
]) {
|
||||
await r.resetSim();
|
||||
const id = r.model.geom(name).id;
|
||||
r.data.qpos[axis] = sign * 2.05;
|
||||
r.data.qpos[3] = Math.cos(yaw / 2);
|
||||
r.data.qpos[6] = Math.sin(yaw / 2);
|
||||
r.mujoco.mj_forward(r.model, r.data);
|
||||
r.command.set([0.15, 0, 0]);
|
||||
let contacts = 0,
|
||||
max = sign * r.data.qpos[axis];
|
||||
for (let tick = 0; tick < 30; tick++) {
|
||||
await r.step(5);
|
||||
max = Math.max(max, sign * r.data.qpos[axis]);
|
||||
const list = r.data.contact;
|
||||
for (let i = 0; i < r.data.ncon; i++) {
|
||||
const c = list.get(i);
|
||||
if (c.geom1 === id || c.geom2 === id) contacts++;
|
||||
c.delete();
|
||||
for (const robot of [
|
||||
{ id: "microban", forward: 0.15, pad: 0.2 },
|
||||
{ id: "rp1", forward: 0.4, pad: 0.35 },
|
||||
])
|
||||
test(`${robot.id}: real scaled wall contacts block motion on all four sides without coordinate clamping`, async ({
|
||||
page,
|
||||
}, info) => {
|
||||
await bootDuck(page);
|
||||
expect(
|
||||
await page.evaluate((id) => window.__gameApi.switchRobot(id), robot.id),
|
||||
).toBe(true);
|
||||
const results = await page.evaluate(async (robot) => {
|
||||
const r = window.rl,
|
||||
report = [],
|
||||
half = r.world.boundary.half_extent;
|
||||
for (const [name, axis, sign, yaw] of [
|
||||
["wall_px", 0, 1, 0],
|
||||
["wall_nx", 0, -1, Math.PI],
|
||||
["wall_py", 1, 1, Math.PI / 2],
|
||||
["wall_ny", 1, -1, -Math.PI / 2],
|
||||
]) {
|
||||
await r.resetSim();
|
||||
const id = r.model.geom(name).id;
|
||||
r.data.qpos[axis] = sign * (half - robot.pad);
|
||||
r.data.qpos[3] = Math.cos(yaw / 2);
|
||||
r.data.qpos[6] = Math.sin(yaw / 2);
|
||||
r.mujoco.mj_forward(r.model, r.data);
|
||||
r.command.set([robot.forward, 0, 0]);
|
||||
let contacts = 0,
|
||||
max = sign * r.data.qpos[axis];
|
||||
for (let tick = 0; tick < 30; tick++) {
|
||||
await r.step(5);
|
||||
max = Math.max(max, sign * r.data.qpos[axis]);
|
||||
const list = r.data.contact;
|
||||
for (let i = 0; i < r.data.ncon; i++) {
|
||||
const c = list.get(i);
|
||||
if (c.geom1 === id || c.geom2 === id) contacts++;
|
||||
c.delete();
|
||||
}
|
||||
list.delete();
|
||||
}
|
||||
list.delete();
|
||||
report.push({
|
||||
name,
|
||||
contacts,
|
||||
max,
|
||||
half,
|
||||
time: r.data.time,
|
||||
fallen: r.fallen,
|
||||
finite: Array.from(r.data.qpos).every(Number.isFinite),
|
||||
});
|
||||
}
|
||||
report.push({
|
||||
name,
|
||||
contacts,
|
||||
max,
|
||||
time: r.data.time,
|
||||
fallen: r.fallen,
|
||||
finite: Array.from(r.data.qpos).every(Number.isFinite),
|
||||
});
|
||||
return report;
|
||||
}, robot);
|
||||
for (const r of results) {
|
||||
expect(r.contacts, JSON.stringify(r)).toBeGreaterThan(0);
|
||||
expect(r.max, JSON.stringify(r)).toBeLessThan(r.half);
|
||||
expect(r.time).toBeCloseTo(3, 6);
|
||||
expect(r.finite).toBe(true);
|
||||
}
|
||||
return report;
|
||||
await info.attach("wall-contacts.json", {
|
||||
body: JSON.stringify(results, null, 2),
|
||||
contentType: "application/json",
|
||||
});
|
||||
});
|
||||
for (const r of results) {
|
||||
expect(r.contacts, JSON.stringify(r)).toBeGreaterThan(0);
|
||||
expect(r.max, JSON.stringify(r)).toBeLessThan(2.25);
|
||||
expect(r.time).toBeCloseTo(3, 6);
|
||||
expect(r.finite).toBe(true);
|
||||
}
|
||||
await info.attach("wall-contacts.json", {
|
||||
body: JSON.stringify(results, null, 2),
|
||||
contentType: "application/json",
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { test, expect } from "@playwright/test";
|
||||
test.use({ video: { mode: "on", size: { width: 1280, height: 800 } } });
|
||||
test("deployed assets respect the base path and both Robot Packages boot", async ({
|
||||
test("deployed assets respect the base path and all three Robot Packages boot", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
const failures = [],
|
||||
@@ -29,6 +29,12 @@ test("deployed assets respect the base path and both Robot Packages boot", async
|
||||
await expect(page.getByRole("status")).toContainText("Simplified actuation");
|
||||
await page.waitForTimeout(500);
|
||||
await page.screenshot({ path: info.outputPath("microban.png") });
|
||||
await page.getByLabel("Robot", { exact: true }).selectOption("rp1");
|
||||
await expect
|
||||
.poll(() => page.evaluate(() => window.__store.getState().robotId))
|
||||
.toBe("rp1");
|
||||
await expect(page.getByRole("status")).toContainText("Skill: walk");
|
||||
await page.screenshot({ path: info.outputPath("rp1.png") });
|
||||
expect(errors).toEqual([]);
|
||||
expect(failures).toEqual([]);
|
||||
const origin = new URL(page.url()).origin;
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
import { test, expect } from "@playwright/test";
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
async function allowStaged(page) {
|
||||
// Qualification can exercise disabled candidates; normal CI uses the actual manifest.
|
||||
if (process.env.RP1_TEST_STAGED) {
|
||||
await page.route("**/robots/rp1/robot.json*", async (route) => {
|
||||
const response = await route.fetch(),
|
||||
config = await response.json();
|
||||
for (const skill of config.skills.filter((s) => s.kind === "episodic")) {
|
||||
skill.available = true;
|
||||
config.input.bindings.find(
|
||||
(b) => b.action === `skill:${skill.id}`,
|
||||
).disabled = false;
|
||||
}
|
||||
await route.fulfill({ response, json: config });
|
||||
});
|
||||
}
|
||||
}
|
||||
async function boot(page) {
|
||||
await allowStaged(page);
|
||||
await page.goto("/");
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Pause", exact: true }),
|
||||
).toBeEnabled();
|
||||
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
|
||||
true,
|
||||
);
|
||||
await page.evaluate(() => window.rl.pause());
|
||||
}
|
||||
|
||||
for (const skill of ["wave", "dance"]) {
|
||||
test(`trained ${skill}: three headings, configured segment, physical walk→skill→walk`, async ({
|
||||
page,
|
||||
}, info) => {
|
||||
test.setTimeout(240000);
|
||||
await boot(page);
|
||||
const reports = [];
|
||||
for (const turn of [0, 1.8, -1.8]) {
|
||||
const report = await page.evaluate(
|
||||
async ({ skill, turn }) => {
|
||||
const r = window.rl;
|
||||
await r.resetSim();
|
||||
if (turn) {
|
||||
r.command.set([0, 0, Math.sign(turn) * 0.6]);
|
||||
await r.step(Math.round(Math.abs(turn) * 50));
|
||||
}
|
||||
r.command.set([0.4, 0, 0]);
|
||||
await r.step(50);
|
||||
r.command.fill(0);
|
||||
await r.step(100);
|
||||
const before = Array.from(r.data.qpos.slice(0, 3));
|
||||
if (!r.policyState.request(skill, r.data.time))
|
||||
throw new Error("request refused");
|
||||
const states = [],
|
||||
samples = [];
|
||||
let entered = false,
|
||||
fallen = null,
|
||||
completed = false,
|
||||
minHeight = Infinity;
|
||||
let previousState = "",
|
||||
previousPosition = before,
|
||||
maxStep = 0,
|
||||
maxAction = 0,
|
||||
maxFrame = 0;
|
||||
const duration = r.policyState.episodes.get(skill).duration;
|
||||
for (let i = 0; i < (duration + 10) * 50; i++) {
|
||||
await r.step(1);
|
||||
const state = r.policyState.state;
|
||||
if (state !== previousState) {
|
||||
states.push({
|
||||
state,
|
||||
time: r.data.time,
|
||||
reference: r.referenceFrame,
|
||||
});
|
||||
previousState = state;
|
||||
}
|
||||
const p = Array.from(r.data.qpos.slice(0, 3));
|
||||
maxStep = Math.max(
|
||||
maxStep,
|
||||
Math.hypot(...p.map((v, j) => v - previousPosition[j])),
|
||||
);
|
||||
previousPosition = p;
|
||||
minHeight = Math.min(minHeight, p[2]);
|
||||
maxAction = Math.max(
|
||||
maxAction,
|
||||
...Array.from(r.lastAction, Math.abs),
|
||||
);
|
||||
maxFrame = Math.max(maxFrame, r.referenceFrame ?? 0);
|
||||
if (
|
||||
![
|
||||
...r.data.qpos,
|
||||
...r.data.qvel,
|
||||
...r.data.ctrl,
|
||||
...r.lastAction,
|
||||
].every(Number.isFinite)
|
||||
)
|
||||
throw new Error("non-finite physics/action state");
|
||||
if (r.activePolicy === skill) entered = true;
|
||||
if (i % 50 === 0 && r.activePolicy === skill) {
|
||||
const obs = Array.from(r.buildObs());
|
||||
if (obs.length !== 126)
|
||||
throw new Error(`wrong observation length ${obs.length}`);
|
||||
samples.push({
|
||||
time: r.data.time,
|
||||
frame: r.referenceFrame,
|
||||
pos: p,
|
||||
obs,
|
||||
});
|
||||
}
|
||||
if (r.fallen) {
|
||||
fallen = { time: r.data.time, frame: r.referenceFrame, p };
|
||||
break;
|
||||
}
|
||||
if (entered && state === "walk") {
|
||||
completed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const after = Array.from(r.data.qpos.slice(0, 3));
|
||||
const [w, x, y, z] = r.data.qpos.slice(3, 7);
|
||||
const yaw = Math.atan2(2 * (w * z + x * y), 1 - 2 * (y * y + z * z));
|
||||
if (completed) {
|
||||
r.command.set([0.4, 0, 0]);
|
||||
await r.step(150);
|
||||
}
|
||||
const walkAfter = Array.from(r.data.qpos.slice(0, 3));
|
||||
return {
|
||||
skill,
|
||||
turn,
|
||||
duration,
|
||||
before,
|
||||
after,
|
||||
states,
|
||||
samples,
|
||||
entered,
|
||||
completed,
|
||||
fallen,
|
||||
finalFallen: r.fallen,
|
||||
minHeight,
|
||||
maxStep,
|
||||
maxAction,
|
||||
maxFrame,
|
||||
stats: r.actuationStats,
|
||||
walkAfter,
|
||||
forwardAfter:
|
||||
(walkAfter[0] - after[0]) * Math.cos(yaw) +
|
||||
(walkAfter[1] - after[1]) * Math.sin(yaw),
|
||||
};
|
||||
},
|
||||
{ skill, turn },
|
||||
);
|
||||
reports.push(report);
|
||||
const base = new URL("../public/robots/rp1/", import.meta.url);
|
||||
const descriptor = JSON.parse(
|
||||
readFileSync(new URL(`motions/${skill}.json`, base)),
|
||||
);
|
||||
const binary = readFileSync(new URL(descriptor.data, base));
|
||||
for (const sample of report.samples) {
|
||||
for (let j = 0; j < 24; j++) {
|
||||
expect(sample.obs[j]).toBe(
|
||||
binary.readFloatLE((sample.frame * 24 + j) * 4),
|
||||
);
|
||||
expect(sample.obs[j + 24]).toBe(
|
||||
binary.readFloatLE(
|
||||
(descriptor.frames * 24 + sample.frame * 24 + j) * 4,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
await info.attach(`${skill}-${turn}-playback.json`, {
|
||||
body: JSON.stringify(report),
|
||||
contentType: "application/json",
|
||||
});
|
||||
console.log(
|
||||
`RP1 ${skill} playback`,
|
||||
JSON.stringify({ ...report, samples: undefined }),
|
||||
);
|
||||
expect(report.entered).toBe(true);
|
||||
expect(report.fallen).toBeNull();
|
||||
expect(report.completed).toBe(true);
|
||||
expect(report.finalFallen).toBe(false);
|
||||
expect(report.maxStep).toBeLessThan(0.15);
|
||||
expect(report.minHeight).toBeGreaterThan(0.4);
|
||||
expect(report.forwardAfter).toBeGreaterThan(0.4);
|
||||
expect(report.maxFrame).toBeGreaterThanOrEqual(report.duration * 50 - 2);
|
||||
expect(report.stats.longestSaturatedSeconds).toBeLessThan(0.5);
|
||||
}
|
||||
await page.screenshot({ path: info.outputPath(`${skill}.png`) });
|
||||
});
|
||||
}
|
||||
|
||||
test("trained skill keyboard activation, pause freezes reference, movement interrupts, reset clears", async ({
|
||||
page,
|
||||
}) => {
|
||||
await boot(page);
|
||||
for (const [skill, key] of [
|
||||
["wave", "KeyH"],
|
||||
["dance", "KeyJ"],
|
||||
]) {
|
||||
await page.evaluate(async () => {
|
||||
const r = window.rl;
|
||||
await r.resetSim();
|
||||
await r.step(100);
|
||||
r.resume();
|
||||
});
|
||||
await page
|
||||
.locator("canvas")
|
||||
.first()
|
||||
.click({ position: { x: 700, y: 600 } });
|
||||
await page.keyboard.press(key);
|
||||
await expect
|
||||
.poll(() => page.evaluate(() => window.rl.policyState.state))
|
||||
.not.toBe("walk");
|
||||
await page.evaluate(async () => {
|
||||
await window.rl.pause();
|
||||
await window.rl.step(45);
|
||||
});
|
||||
expect(await page.evaluate(() => window.rl.activePolicy)).toBe(skill);
|
||||
const paused = await page.evaluate(() => ({
|
||||
time: window.rl.data.time,
|
||||
frame: window.rl.referenceFrame,
|
||||
}));
|
||||
await page.waitForTimeout(250);
|
||||
expect(
|
||||
await page.evaluate(() => ({
|
||||
time: window.rl.data.time,
|
||||
frame: window.rl.referenceFrame,
|
||||
})),
|
||||
).toEqual(paused);
|
||||
const result = await page.evaluate(async () => {
|
||||
const r = window.rl;
|
||||
r.command.set([0.4, 0, 0]);
|
||||
await r.step(1);
|
||||
const interrupted = {
|
||||
active: r.activePolicy,
|
||||
state: r.policyState.state,
|
||||
};
|
||||
await r.step(100);
|
||||
return { interrupted, state: r.policyState.state, fallen: r.fallen };
|
||||
});
|
||||
expect(result.interrupted).toEqual({ active: "walk", state: "returning" });
|
||||
expect(result.state).toBe("walk");
|
||||
expect(result.fallen).toBe(false);
|
||||
await page.keyboard.press("Space");
|
||||
await expect.poll(() => page.evaluate(() => window.rl.data.time)).toBe(0);
|
||||
expect(await page.evaluate(() => window.rl.referenceFrame)).toBeNull();
|
||||
expect(
|
||||
await page.evaluate(() =>
|
||||
Array.from(window.rl.lastAction).every((v) => v === 0),
|
||||
),
|
||||
).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test("repeated real skills restart reference at zero without resetting the robot", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
await boot(page);
|
||||
const report = await page.evaluate(async () => {
|
||||
const r = window.rl,
|
||||
episodes = [];
|
||||
await r.step(100);
|
||||
for (const skill of ["wave", "dance", "wave", "dance"]) {
|
||||
r.command.fill(0);
|
||||
await r.step(100);
|
||||
if (!r.policyState.request(skill, r.data.time))
|
||||
throw new Error("request refused");
|
||||
let firstFrame = null,
|
||||
lastFrame = null,
|
||||
entered = false,
|
||||
complete = false;
|
||||
const duration = r.policyState.episodes.get(skill).duration;
|
||||
for (let i = 0; i < (duration + 7) * 50; i++) {
|
||||
await r.step(1);
|
||||
if (r.fallen) throw new Error(`${skill}: fallen at ${r.data.time}`);
|
||||
if (r.activePolicy === skill) {
|
||||
firstFrame ??= r.referenceFrame;
|
||||
lastFrame = r.referenceFrame;
|
||||
entered = true;
|
||||
}
|
||||
if (entered && r.policyState.state === "walk") {
|
||||
complete = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
episodes.push({
|
||||
skill,
|
||||
firstFrame,
|
||||
lastFrame,
|
||||
complete,
|
||||
time: r.data.time,
|
||||
});
|
||||
r.command.set([0.4, 0, 0]);
|
||||
await r.step(50);
|
||||
}
|
||||
return episodes;
|
||||
});
|
||||
await info.attach("repeated-skills.json", {
|
||||
body: JSON.stringify(report),
|
||||
contentType: "application/json",
|
||||
});
|
||||
for (const episode of report) {
|
||||
expect(episode.firstFrame).toBe(0);
|
||||
expect(episode.lastFrame).toBe(episode.skill === "wave" ? 217 : 499);
|
||||
expect(episode.complete).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test("truncated motion asset rolls back the candidate and permits a clean retry", async ({
|
||||
page,
|
||||
}) => {
|
||||
await allowStaged(page);
|
||||
await page.goto("/");
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Pause", exact: true }),
|
||||
).toBeEnabled();
|
||||
await page.route("**/robots/rp1/motions/wave.f32*", (route) =>
|
||||
route.fulfill({
|
||||
body: Buffer.alloc(8),
|
||||
contentType: "application/octet-stream",
|
||||
}),
|
||||
);
|
||||
const old = await page.evaluate(() => {
|
||||
window.__oldRobotModel = window.rl.model;
|
||||
return window.__store.getState().robotId;
|
||||
});
|
||||
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
|
||||
false,
|
||||
);
|
||||
expect(await page.evaluate(() => window.__store.getState().robotId)).toBe(
|
||||
old,
|
||||
);
|
||||
expect(
|
||||
await page.evaluate(() => window.__store.getState().runtimeError),
|
||||
).toContain("sample count");
|
||||
expect(
|
||||
await page.evaluate(() => window.rl.model === window.__oldRobotModel),
|
||||
).toBe(true);
|
||||
await page.unroute("**/robots/rp1/motions/wave.f32*");
|
||||
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
|
||||
true,
|
||||
);
|
||||
expect(
|
||||
await page.evaluate(() => window.__store.getState().runtimeError),
|
||||
).toBeNull();
|
||||
});
|
||||
@@ -0,0 +1,291 @@
|
||||
import { test, expect } from "@playwright/test";
|
||||
async function boot(page) {
|
||||
await page.goto("/");
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Pause", exact: true }),
|
||||
).toBeEnabled();
|
||||
await page.getByLabel("Robot", { exact: true }).selectOption("rp1");
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__store.getState().robotId === "rp1" ||
|
||||
window.__store.getState().runtimeError,
|
||||
);
|
||||
expect(
|
||||
await page.evaluate(() => window.__store.getState().runtimeError),
|
||||
).toBeNull();
|
||||
await page.evaluate(() => window.rl.pause());
|
||||
}
|
||||
test("third robot: real WASM/ONNX, keyboard command contract, trained skill bindings, fall/reset, switching", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
const errors = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await boot(page);
|
||||
expect(
|
||||
await page.getByLabel("Robot", { exact: true }).locator("option").count(),
|
||||
).toBe(3);
|
||||
expect(await page.evaluate(() => window.rl.buildObs().length)).toBe(243);
|
||||
expect(await page.evaluate(() => window.rl.lastAction.length)).toBe(24);
|
||||
await page
|
||||
.locator("canvas")
|
||||
.first()
|
||||
.click({ position: { x: 700, y: 600 } });
|
||||
await page.evaluate(() => window.rl.resume());
|
||||
await expect(page.getByRole("button", { name: /^J\s*Dance$/ })).toBeEnabled();
|
||||
await expect(page.getByRole("button", { name: /^H\s*Wave$/ })).toBeEnabled();
|
||||
expect(
|
||||
await page
|
||||
.getByRole("complementary", { name: "Keyboard controls" })
|
||||
.innerText(),
|
||||
).not.toMatch(/\p{Script=Han}/u);
|
||||
for (const [key, expected] of [
|
||||
["KeyW", [0.4, 0, 0]],
|
||||
["KeyS", [-0.3, 0, 0]],
|
||||
["KeyA", [0, 0, 0.6]],
|
||||
["KeyD", [0, 0, -0.6]],
|
||||
]) {
|
||||
await page.evaluate(async () => {
|
||||
await window.rl.resetSim();
|
||||
window.rl.resume();
|
||||
});
|
||||
await page.keyboard.down(key);
|
||||
const command = await page.evaluate(() => Array.from(window.rl.command));
|
||||
command.forEach((v, i) => expect(v).toBeCloseTo(expected[i], 5));
|
||||
await page.keyboard.up(key);
|
||||
expect(await page.evaluate(() => Array.from(window.rl.command))).toEqual([
|
||||
0, 0, 0,
|
||||
]);
|
||||
await page.evaluate(() => window.rl.pause());
|
||||
}
|
||||
const report = await page.evaluate(async () => {
|
||||
const r = window.rl,
|
||||
trials = [];
|
||||
for (const [name, command] of [
|
||||
["stand", [0, 0, 0]],
|
||||
["forward", [0.4, 0, 0]],
|
||||
["backward", [-0.3, 0, 0]],
|
||||
["left", [0, 0, 0.6]],
|
||||
["right", [0, 0, -0.6]],
|
||||
]) {
|
||||
await r.resetSim();
|
||||
r.command.set(command);
|
||||
const before = Array.from(r.data.qpos.slice(0, 3));
|
||||
let firstFall = null;
|
||||
for (let i = 0; i < 150; i++) {
|
||||
await r.step(1);
|
||||
if (
|
||||
![
|
||||
...r.data.qpos,
|
||||
...r.data.qvel,
|
||||
...r.data.ctrl,
|
||||
...r.lastAction,
|
||||
].every(Number.isFinite)
|
||||
)
|
||||
throw new Error("non-finite RP1 state");
|
||||
if (r.fallen) {
|
||||
firstFall = r.data.time;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const [w, x, y, z] = r.data.qpos.slice(3, 7);
|
||||
trials.push({
|
||||
name,
|
||||
before,
|
||||
after: Array.from(r.data.qpos.slice(0, 3)),
|
||||
time: r.data.time,
|
||||
yaw: Math.atan2(2 * (w * z + x * y), 1 - 2 * (y * y + z * z)),
|
||||
firstFall,
|
||||
actuation: r.actuationStats,
|
||||
});
|
||||
}
|
||||
return trials;
|
||||
});
|
||||
for (const trial of report) expect(trial.firstFall).toBeNull();
|
||||
expect(report.find((t) => t.name === "forward").after[0]).toBeGreaterThan(
|
||||
0.4,
|
||||
);
|
||||
expect(report.find((t) => t.name === "backward").after[0]).toBeLessThan(-0.3);
|
||||
expect(report.find((t) => t.name === "left").yaw).toBeGreaterThan(0.5);
|
||||
expect(report.find((t) => t.name === "right").yaw).toBeLessThan(-0.5);
|
||||
console.log("RP1 completed-checkpoint trials", JSON.stringify(report));
|
||||
await info.attach("rp1-preview-trials.json", {
|
||||
body: JSON.stringify(report, null, 2),
|
||||
contentType: "application/json",
|
||||
});
|
||||
await page.evaluate(async () => {
|
||||
await window.rl.resetSim();
|
||||
window.rl.data.qpos[2] = 1.2;
|
||||
window.rl.data.qpos.set([Math.SQRT1_2, Math.SQRT1_2, 0, 0], 3);
|
||||
window.rl.mujoco.mj_forward(window.rl.model, window.rl.data);
|
||||
await window.rl.step(1);
|
||||
});
|
||||
expect(await page.evaluate(() => window.rl.fallen)).toBe(true);
|
||||
await page.keyboard.press("Space");
|
||||
await expect.poll(() => page.evaluate(() => window.rl.fallen)).toBe(false);
|
||||
expect(await page.evaluate(() => window.rl.policyState.state)).toBe("walk");
|
||||
await page.screenshot({ path: info.outputPath("rp1.png") });
|
||||
for (const robot of ["microban", "microduck", "rp1"]) {
|
||||
await page.getByLabel("Robot", { exact: true }).selectOption(robot);
|
||||
await expect
|
||||
.poll(() => page.evaluate(() => window.__store.getState().robotId))
|
||||
.toBe(robot === "microduck" ? "microduck-legs" : robot);
|
||||
}
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
test("three 25-second bounded RP1 sequences preserve direction and balance; realtime controller stays near 50Hz", async ({
|
||||
page,
|
||||
}, info) => {
|
||||
await boot(page);
|
||||
const report = await page.evaluate(async () => {
|
||||
const r = window.rl,
|
||||
trials = [];
|
||||
const pose = () => {
|
||||
const [w, x, y, z] = r.data.qpos.slice(3, 7);
|
||||
return {
|
||||
pos: Array.from(r.data.qpos.slice(0, 3)),
|
||||
yaw: Math.atan2(2 * (w * z + x * y), 1 - 2 * (y * y + z * z)),
|
||||
};
|
||||
};
|
||||
for (let trial = 0; trial < 3; trial++) {
|
||||
await r.resetSim();
|
||||
const segments = [];
|
||||
for (const [name, seconds, vx, wz] of [
|
||||
["stand", 5, 0, 0],
|
||||
["forward", 3, 0.4, 0],
|
||||
["rest", 2, 0, 0],
|
||||
["backward", 3, -0.3, 0],
|
||||
["rest", 2, 0, 0],
|
||||
["left", 3, 0, 0.6],
|
||||
["rest", 2, 0, 0],
|
||||
["right", 3, 0, -0.6],
|
||||
["rest", 2, 0, 0],
|
||||
]) {
|
||||
const before = pose();
|
||||
r.command.set([vx, 0, wz]);
|
||||
let yaw = 0,
|
||||
previousYaw = before.yaw,
|
||||
minHeight = Infinity;
|
||||
for (let i = 0; i < seconds * 10; i++) {
|
||||
await r.step(5);
|
||||
const p = pose();
|
||||
if (r.fallen)
|
||||
throw new Error(`trial ${trial} ${name}: fallen at ${r.data.time}`);
|
||||
if (Math.max(Math.abs(p.pos[0]), Math.abs(p.pos[1])) > 1.9)
|
||||
throw new Error("unexpected wall approach");
|
||||
yaw += Math.atan2(
|
||||
Math.sin(p.yaw - previousYaw),
|
||||
Math.cos(p.yaw - previousYaw),
|
||||
);
|
||||
previousYaw = p.yaw;
|
||||
minHeight = Math.min(minHeight, p.pos[2]);
|
||||
}
|
||||
const after = pose();
|
||||
segments.push({
|
||||
name,
|
||||
minHeight,
|
||||
yaw,
|
||||
displacement:
|
||||
(after.pos[0] - before.pos[0]) * Math.cos(before.yaw) +
|
||||
(after.pos[1] - before.pos[1]) * Math.sin(before.yaw),
|
||||
});
|
||||
}
|
||||
trials.push({
|
||||
trial,
|
||||
time: r.data.time,
|
||||
segments,
|
||||
actuation: r.actuationStats,
|
||||
});
|
||||
}
|
||||
return trials;
|
||||
});
|
||||
for (const trial of report) {
|
||||
const find = (name) => trial.segments.find((s) => s.name === name);
|
||||
expect(trial.time).toBeCloseTo(25, 5);
|
||||
expect(find("forward").displacement).toBeGreaterThan(0.4);
|
||||
expect(find("backward").displacement).toBeLessThan(-0.3);
|
||||
expect(find("left").yaw).toBeGreaterThan(0.5);
|
||||
expect(find("right").yaw).toBeLessThan(-0.5);
|
||||
expect(trial.actuation.longestSaturatedSeconds).toBeLessThan(0.5);
|
||||
}
|
||||
await info.attach("rp1-three-sequences.json", {
|
||||
body: JSON.stringify(report, null, 2),
|
||||
contentType: "application/json",
|
||||
});
|
||||
await page.evaluate(async () => {
|
||||
await window.rl.resetSim();
|
||||
window.rl.resume();
|
||||
});
|
||||
const first = await page.evaluate(() => ({
|
||||
sim: window.rl.data.time,
|
||||
wall: performance.now(),
|
||||
}));
|
||||
await page.waitForTimeout(4000);
|
||||
const second = await page.evaluate(() => ({
|
||||
sim: window.rl.data.time,
|
||||
wall: performance.now(),
|
||||
fallen: window.rl.fallen,
|
||||
}));
|
||||
const ratio = (second.sim - first.sim) / ((second.wall - first.wall) / 1000);
|
||||
expect(second.fallen).toBe(false);
|
||||
expect(ratio).toBeGreaterThan(0.8);
|
||||
expect(ratio).toBeLessThan(1.15);
|
||||
await info.attach("rp1-realtime.json", {
|
||||
body: JSON.stringify({ first, second, ratio }),
|
||||
contentType: "application/json",
|
||||
});
|
||||
});
|
||||
|
||||
test("episodic ONNX lifecycle is exclusive and resets on return (walk reused as interface fixture, NOT a dance policy)", async ({
|
||||
page,
|
||||
}) => {
|
||||
await page.route("**/robots/rp1/robot.json*", async (route) => {
|
||||
const response = await route.fetch(),
|
||||
config = await response.json();
|
||||
const skill = config.skills.find((s) => s.id === "dance");
|
||||
Object.assign(skill, {
|
||||
available: true,
|
||||
duration: 0.8,
|
||||
blend: 0.2,
|
||||
observations: config.observations,
|
||||
actions: config.actions,
|
||||
defaults: config.joints.defaults,
|
||||
});
|
||||
delete skill.reference;
|
||||
skill.controller.policy = "policies/walk.onnx";
|
||||
config.input.bindings.find((b) => b.action === "skill:dance").disabled =
|
||||
false;
|
||||
await route.fulfill({ response, json: config });
|
||||
});
|
||||
await boot(page);
|
||||
const report = await page.evaluate(async () => {
|
||||
const r = window.rl;
|
||||
await r.resetSim();
|
||||
if (!r.policyState.request("dance", r.data.time))
|
||||
throw new Error("request refused");
|
||||
await r.step(23); // Real zero-command locomotion must satisfy the stop gate.
|
||||
const first = { active: r.activePolicy, state: r.policyState.state };
|
||||
await r.step(65);
|
||||
const returned = { active: r.activePolicy, state: r.policyState.state };
|
||||
r.policyState.request("dance", r.data.time);
|
||||
await r.step(23);
|
||||
r.command.set([0.4, 0, 0]);
|
||||
await r.step(1);
|
||||
const interrupted = { active: r.activePolicy, state: r.policyState.state };
|
||||
await r.step(12);
|
||||
return {
|
||||
first,
|
||||
returned,
|
||||
interrupted,
|
||||
active: r.activePolicy,
|
||||
state: r.policyState.state,
|
||||
fallen: r.fallen,
|
||||
};
|
||||
});
|
||||
expect(report.first).toEqual({ active: "dance", state: "entering" });
|
||||
expect(report.returned).toEqual({ active: "walk", state: "walk" });
|
||||
expect(report.interrupted).toEqual({ active: "walk", state: "returning" });
|
||||
expect(report.active).toBe("walk");
|
||||
expect(report.state).toBe("walk");
|
||||
expect(report.fallen).toBe(false);
|
||||
});
|
||||
@@ -41,7 +41,7 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
|
||||
await page.evaluate(() => window.__gameApi.pause());
|
||||
const measurements = [];
|
||||
for (let i = 0; i < 20; i++) {
|
||||
const id = i % 2 === 0 ? "microban" : "microduck-legs";
|
||||
const id = ["microban", "microduck-legs", "rp1"][i % 3];
|
||||
console.log(`Ownership transition ${i + 1}/20: ${id}`);
|
||||
const sample = await page.evaluate(async (id) => {
|
||||
const model = window.rl.model,
|
||||
@@ -65,8 +65,9 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
|
||||
measurements.push(sample);
|
||||
}
|
||||
const duck = measurements.filter((s) => s.id === "microduck-legs"),
|
||||
ban = measurements.filter((s) => s.id === "microban");
|
||||
for (const samples of [duck, ban]) {
|
||||
ban = measurements.filter((s) => s.id === "microban"),
|
||||
rp1 = measurements.filter((s) => s.id === "rp1");
|
||||
for (const samples of [duck, ban, rp1]) {
|
||||
expect(samples.at(-1).listeners).toBe(samples[0].listeners);
|
||||
expect(samples.at(-1).sceneChildren).toBe(samples[0].sceneChildren);
|
||||
expect(samples.at(-1).memory.geometries).toBeLessThanOrEqual(
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "microduck-sandbox",
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "microduck-sandbox",
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"dependencies": {
|
||||
"@emotion/react": "^11.14.0",
|
||||
"@emotion/styled": "^11.14.1",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "microduck-sandbox",
|
||||
"private": true,
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -16,5 +16,11 @@
|
||||
"family": "microban",
|
||||
"name": "Microban",
|
||||
"manifest": "microban/robot.json"
|
||||
},
|
||||
{
|
||||
"id": "rp1",
|
||||
"family": "rp1",
|
||||
"name": "RP1",
|
||||
"manifest": "rp1/robot.json"
|
||||
}
|
||||
]
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
"name": "Microban",
|
||||
"family": "microban",
|
||||
"adapter": "generic",
|
||||
"dimensions": { "height": 0.33 },
|
||||
"model": "model.xml",
|
||||
"assets": [
|
||||
{
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
"name": "Microduck",
|
||||
"family": "microduck",
|
||||
"adapter": "microduck",
|
||||
"dimensions": { "height": 0.27 },
|
||||
"model": "../../robot/mjlab/robot_allcollisions.xml",
|
||||
"assets": [],
|
||||
"base": {
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
"name": "Microduck",
|
||||
"family": "microduck",
|
||||
"adapter": "microduck",
|
||||
"dimensions": { "height": 0.27 },
|
||||
"model": "../../robot/mjlab/robot_allcollisions_rollers.xml",
|
||||
"assets": [],
|
||||
"base": {
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
Copyright (c) 2022-2025, The Isaac Lab Project Developers.
|
||||
Copyright (c) 2025-2026, The RoboLab Project Developers.
|
||||
All rights reserved.
|
||||
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
3. Neither the name of the copyright holder nor the names of its
|
||||
contributors may be used to endorse or promote products derived from
|
||||
this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
||||
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
||||
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
||||
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
||||
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
@@ -0,0 +1,37 @@
|
||||
# RP1 local-evaluation package
|
||||
|
||||
## Inputs and provenance
|
||||
|
||||
- Training repository: `/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train`.
|
||||
- Model/meshes: `robolab/data/robots/roboparty/rp1`, upstream `https://github.com/Roboparty/rp1_description`, commit `d8c6dd03aeecc7f7ead3d962e09fedba4c8460f0` (clean asset worktree at original import).
|
||||
- Input MJCF: `mjcf/robot_24dof.xml`; `compiler.meshdir` changes from `../` to `.` for package-relative VFS loading. XML is reserialized. Bodies, inertias, primitive collision geometry, motors, sensors and mesh bytes are retained.
|
||||
- Walk: final `rp1_24dof_flat/2026-10-28_15-47-49/model_9000.pt` (9001 configured iterations).
|
||||
- Dance: resumed final `rp1_24dof_beyondmimic_dance/2026-10-29_11-05-14_dance_resume/model_23998.pt` (starts from iteration 7999, adds 16000 iterations; final 23998).
|
||||
- Wave: final `rp1_24dof_beyondmimic_wave/2026-10-28_19-00-23_wave/model_7999.pt` (8000 iterations).
|
||||
- Importers do not launch/stop training or modify the training repository. Completed iterations do not establish convergence, arbitrary-duration stability or hardware safety.
|
||||
- Logged `params/env.yaml` and `params/agent.yaml` determine actual actor branch, terms, joint order, defaults, actuator contract and network. `source-manifest.json` records checkpoint/config/motion/resource SHA-256 and export error.
|
||||
|
||||
## Deterministic exports
|
||||
|
||||
`app/scripts/import-rp1.py` exports walking, ELU MLP `243→512→256→128→24`, including empirical normalization `(obs - mean) / (std + 0.01)` and runner ±100 action clip. Three term-major history frames follow training joint order. First sample backfills history; reset/skill entry clears it. PyTorch/ONNX max absolute error: approximately **1.55e-6**.
|
||||
|
||||
`app/scripts/import-rp1-motion.py` exports dance/wave, ELU MLP `126→512→256→128→24`, including empirical normalization and no additional action clip (runner clip is null). Logged actor branch is **student**, not the privileged policy observation group. Inputs are reference joint positions/velocities (48), gyro (3), projected gravity (3), relative joint positions (24), joint velocities (24), last actor action (24). Each skill uses its own trained defaults and last action. The distribution and critic are not deployed. PyTorch/ONNX max errors: dance **1.31e-6**, wave **1.91e-6**.
|
||||
|
||||
Actions are reference-pose-relative joint targets, scale `0.25`. Named gains, torque caps and four-quadrant motor torque-speed limits are preserved. Browser intentionally omits training-time random actuator delays (0–2 physics steps), randomized physics/default positions, external pushes and observation noise. It is not an exact reproduction of Newton/MJWarp training or hardware motor control.
|
||||
|
||||
## Motion references and deployed limits
|
||||
|
||||
Source archives under `robolab/data/motions/rp1_bm/`:
|
||||
|
||||
- `dance1_subject2_smooth6hz.npz`: 6573 frames at 50 fps, 131.46 s total. **Only the validated first 10 s are enabled.** The previous 7999 checkpoint fell at frame 994 (~19.9 s) in a full-clip rollout. The updated 23998 checkpoint has been qualified for the 10 s prefix only; full continuous playback has not been requalified.
|
||||
- `wave_R_001__A428.npz`: 218 frames at 50 fps, full 4.36 s enabled.
|
||||
|
||||
`.json` reference descriptors plus little-endian `.f32` files contain joint position/velocity commands in logged joint order. Motion command time advances one frame per 50 Hz control tick, freezes on pause and resets at activation. Reference end clamps; there is no random resampling or wrapping. References only supply policy observations: they do not teleport the root, set kinematic joint poses, or replace trained actions with scripted animation.
|
||||
|
||||
Both skills use exclusive full-body policies, a physical stop gate, entry/return blends and locomotion interruption. Validation includes three initial headings, walking before/after, repeated episodes without robot reset, keyboard/pause/reset, numerical reference checks and malformed-asset rollback. Initial integration: `docs/superpowers/verification/2026-10-29-rp1-completed-skills.md`; current dance weights: `docs/superpowers/verification/2026-10-29-rp1-dance-23998.md`. UI labels are simply `Dance` and `Wave`; durations and transition behavior remain unchanged.
|
||||
|
||||
## License limitation — local evaluation only
|
||||
|
||||
The RP1 description submodule does not supply a license file in this checkout. Do **not** infer an Apache/BSD license for its meshes/model or motion data from the platform or RoboLab. Confirm upstream rights before redistributing this package or deploying publicly.
|
||||
|
||||
`LICENSES/RoboLab-BSD-3-Clause.txt` preserves RoboLab's code notice; it does not grant rights to separately sourced RP1 description assets, motion recordings/retargeting data or the user's checkpoints. Training-code, model, checkpoint and motion provenance must be reviewed independently for redistribution. No public deployment was performed.
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,282 @@
|
||||
<?xml version='1.0' encoding='utf-8'?>
|
||||
<mujoco model="rp1">
|
||||
<compiler angle="radian" meshdir="." autolimits="true" eulerseq="zyx" balanceinertia="true" />
|
||||
<option timestep="0.005" integrator="implicitfast" iterations="100" solver="Newton" gravity="0 0 -9.81">
|
||||
<flag frictionloss="enable" />
|
||||
</option>
|
||||
<size memory="32M" />
|
||||
<default>
|
||||
<default class="RP1">
|
||||
<joint damping="0.01" frictionloss="0.1" armature="0.01" />
|
||||
<default class="visual">
|
||||
<geom type="mesh" contype="0" conaffinity="0" group="2" />
|
||||
</default>
|
||||
<default class="collision">
|
||||
<geom group="3" condim="4" contype="1" conaffinity="1" solref="0.01 1" friction="0.9 0.2 0.2" />
|
||||
</default>
|
||||
</default>
|
||||
</default>
|
||||
<worldbody>
|
||||
<body name="base_link" pos="0 0 0.78" quat="1 0 0 0" childclass="RP1">
|
||||
<site name="imu" size="0.01" pos="0.0 0 0.0" />
|
||||
<freejoint name="base_link_freejoint" />
|
||||
<inertial pos="-0.03786057 -0.00301243 -0.04479345" mass="3.295" fullinertia="0.00960796 0.00699457 0.00815508 0.00000112 -0.00015116 -0.00000270" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="base_link" material="base_link_material" />
|
||||
|
||||
<body name="left_hip_pitch_link" pos="-0.03843 0.062987 -0.051639" quat="1 0 0 0">
|
||||
<joint axis="0 1 0" name="left_hip_pitch_joint" type="hinge" range="-2.86 2.86" armature="0.0290856" />
|
||||
<inertial pos="-0.00326988 0.05886633 -0.05765170" mass="1.83930168" fullinertia="0.00338343 0.00286182 0.00254132 0.00000610 -0.00000685 0.00030016" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_hip_pitch_link" material="left_hip_pitch_material" />
|
||||
<geom name="left_hip_pitch_collision" type="capsule" class="collision" size="0.04970" fromto="-0.01122 0.06756 -0.06890 -0.01122 0.05070 -0.01118" />
|
||||
<body name="left_hip_roll_link" pos="0.031915 0.061254 -0.060672" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="left_hip_roll_joint" type="hinge" range="-0.74 2.6" armature="0.0290856" />
|
||||
<inertial pos="-0.00518991 -0.00003499 -0.08084106" mass="1.473" fullinertia="0.00376408 0.00382823 0.00192275 0.00000313 -0.00065554 -0.00000256" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_hip_roll_link" material="left_hip_roll_material" />
|
||||
<geom name="left_hip_roll_collision" type="capsule" class="collision" size="0.05622" fromto="-0.01270 0 -0.07578 -0.01270 0 -0.00658" />
|
||||
<body name="left_hip_yaw_link" pos="-0.012698 0 -0.129" quat="1 0 0 0">
|
||||
<joint axis="0 0 -1" name="left_hip_yaw_joint" type="hinge" range="-2.76 2.76" armature="0.00884" />
|
||||
<inertial pos="-0.01959270 -0.00023200 -0.12376414" mass="2.483" fullinertia="0.01126838 0.01225761 0.00388513 0.00001684 -0.00196020 0.00002442" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_hip_yaw_link" material="left_hip_yaw_material" />
|
||||
<geom name="left_hip_yaw_collision" type="capsule" class="collision" size="0.05622" fromto="-0.03046 0 -0.16235 -0.00001 0 -0.05622" />
|
||||
<body name="left_knee_link" pos="-0.033201 -0.030809 -0.16509" quat="1 0 0 0">
|
||||
<joint axis="0 -1 0" name="left_knee_joint" type="hinge" range="-2.83 0.06" armature="0.0290856" />
|
||||
<inertial pos="0.03105182 0.03046366 -0.13362250" mass="2.557" fullinertia="0.01819333 0.01828843 0.00291276 -0.00006969 -0.00019845 0.00002464" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_knee_link" material="left_knee_material" />
|
||||
<geom name="left_knee_upper_collision" type="capsule" class="collision" size="0.0520" fromto="0.0330 0.03081 -0.1450 0.0030 0.03081 -0.0040" />
|
||||
<geom name="left_knee_lower_collision" type="capsule" class="collision" size="0.0405" fromto="0.0270 0.03081 -0.2780 0.0330 0.03081 -0.1850" />
|
||||
<body name="left_ankle_pitch_link" pos="0.03131 0.030811 -0.30533" quat="1 0 0 0">
|
||||
<joint axis="0 1 0" name="left_ankle_pitch_joint" type="hinge" range="-0.78 0.61" armature="0.006266268" frictionloss="0.1" />
|
||||
<inertial pos="0.00126332 0.00000000 -0.01319163" mass="0.16605731" fullinertia="0.00003624 0.00004292 0.00002407 0.00000000 0.00000148 0.00000000" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_ankle_pitch_link" material="left_ankle_pitch_material" />
|
||||
<geom name="left_ankle_pitch_collision" type="capsule" class="collision" size="0.02150" fromto="-0.00145 0 -0.00614 0.00959 0 -0.01051" />
|
||||
<body name="left_ankle_roll_link" pos="0.013067 0 -0.018922" quat="1 0 0 0">
|
||||
<joint axis="0.999993677858222 0 0.00355587451777954" name="left_ankle_roll_joint" type="hinge" range="-0.26 0.26" armature="0.006266268" frictionloss="0.1" />
|
||||
<inertial pos="0.008711 -0.000002 -0.021633" mass="0.6430" fullinertia="0.000361 0.001920 0.002002 0.000000 0.000123 0.000000" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_ankle_roll_link" material="left_ankle_roll_material" />
|
||||
<geom name="left_foot_toe_neg_collision" type="capsule" class="collision" size="0.01" fromto="0.03 -0.03 -0.035 0.09 -0.03 -0.035" />
|
||||
<geom name="left_foot_outer_neg_collision" type="capsule" class="collision" size="0.01" fromto="-0.065 -0.02 -0.035 0.11 -0.02 -0.035" />
|
||||
<geom name="left_foot_inner_neg_collision" type="capsule" class="collision" size="0.01" fromto="-0.0775 -0.01 -0.035 0.1225 -0.01 -0.035" />
|
||||
<geom name="left_foot_center_collision" type="capsule" class="collision" size="0.01" fromto="-0.0825 0 -0.035 0.1275 0 -0.035" />
|
||||
<geom name="left_foot_inner_pos_collision" type="capsule" class="collision" size="0.01" fromto="-0.0775 0.01 -0.035 0.1225 0.01 -0.035" />
|
||||
<geom name="left_foot_outer_pos_collision" type="capsule" class="collision" size="0.01" fromto="-0.065 0.02 -0.035 0.11 0.02 -0.035" />
|
||||
<geom name="left_foot_toe_pos_collision" type="capsule" class="collision" size="0.01" fromto="0.03 0.03 -0.035 0.09 0.03 -0.035" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
|
||||
<body name="right_hip_pitch_link" pos="-0.03843 -0.069013 -0.051639" quat="1 0 0 0">
|
||||
<joint axis="0 -1 0" name="right_hip_pitch_joint" type="hinge" range="-2.86 2.86" armature="0.0290856" />
|
||||
<inertial pos="-0.00326831 -0.05867998 -0.05764945" mass="1.84174869" fullinertia="0.00339042 0.00287691 0.00253348 0.00000130 -0.00000978 -0.00029218" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_hip_pitch_link" material="right_hip_pitch_material" />
|
||||
<geom name="right_hip_pitch_collision" type="capsule" class="collision" size="0.04970" fromto="-0.01122 -0.06756 -0.06890 -0.01122 -0.05070 -0.01118" />
|
||||
<body name="right_hip_roll_link" pos="0.031915 -0.061254 -0.060672" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="right_hip_roll_joint" type="hinge" range="-2.6 0.74" armature="0.0290856" />
|
||||
<inertial pos="-0.00518991 -0.00003499 -0.08084106" mass="1.473" fullinertia="0.00376408 0.00382823 0.00192275 0.00000313 -0.00065554 -0.00000256" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_hip_roll_link" material="right_hip_roll_material" />
|
||||
<geom name="right_hip_roll_collision" type="capsule" class="collision" size="0.05622" fromto="-0.01270 0 -0.07578 -0.01270 0 -0.00658" />
|
||||
<body name="right_hip_yaw_link" pos="-0.012698 0 -0.129" quat="1 0 0 0">
|
||||
<joint axis="0 0 -1" name="right_hip_yaw_joint" type="hinge" range="-2.76 2.76" armature="0.00884" />
|
||||
<inertial pos="-0.01958069 0.00023065 -0.12376565" mass="2.483" fullinertia="0.01132547 0.01225907 0.00383024 -0.00001670 -0.00196009 -0.00002438" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_hip_yaw_link" material="right_hip_yaw_material" />
|
||||
<geom name="right_hip_yaw_collision" type="capsule" class="collision" size="0.05622" fromto="-0.03046 0 -0.16235 -0.00001 0 -0.05622" />
|
||||
<body name="right_knee_link" pos="-0.0332010004232152 0.0308089665012395 -0.165090022909223" quat="1 0 0 0">
|
||||
<joint axis="0 1 0" name="right_knee_joint" type="hinge" range="-0.06 2.83" armature="0.0290856" />
|
||||
<inertial pos="0.03115290 -0.03066848 -0.13404559" mass="2.557" fullinertia="0.01802753 0.01813266 0.00288242 0.00006186 -0.00023263 0.00002714" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_knee_link" material="right_knee_material" />
|
||||
<geom name="right_knee_upper_collision" type="capsule" class="collision" size="0.0520" fromto="0.0330 -0.03081 -0.1450 0.0030 -0.03081 -0.0040" />
|
||||
<geom name="right_knee_lower_collision" type="capsule" class="collision" size="0.0405" fromto="0.0270 -0.03081 -0.2780 0.0330 -0.03081 -0.1850" />
|
||||
<body name="right_ankle_pitch_link" pos="0.03131 -0.030809 -0.30533" quat="1 0 0 0">
|
||||
<joint axis="0 1 0" name="right_ankle_pitch_joint" type="hinge" range="-0.78 0.61" armature="0.006266268" frictionloss="0.1" />
|
||||
<inertial pos="0.00126328 0.00000000 -0.01319163" mass="0.16605731" fullinertia="0.00003624 0.00004292 0.00002407 0.00000000 0.00000148 0.00000000" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_ankle_pitch_link" material="right_ankle_pitch_material" />
|
||||
<geom name="right_ankle_pitch_collision" type="capsule" class="collision" size="0.02150" fromto="-0.00145 0 -0.00614 0.00959 0 -0.01051" />
|
||||
<body name="right_ankle_roll_link" pos="0.013067 0 -0.018922" quat="1 0 0 0">
|
||||
<joint axis="0.999993677858222 0 0.00355587451777984" name="right_ankle_roll_joint" type="hinge" range="-0.26 0.26" armature="0.006266268" frictionloss="0.1" />
|
||||
<inertial pos="0.008711 -0.000002 -0.021633" mass="0.6430" fullinertia="0.000361 0.001920 0.002002 0.000000 0.000123 0.000000" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_ankle_roll_link" material="right_ankle_roll_material" />
|
||||
<geom name="right_foot_toe_neg_collision" type="capsule" class="collision" size="0.01" fromto="0.03 -0.03 -0.035 0.09 -0.03 -0.035" />
|
||||
<geom name="right_foot_outer_neg_collision" type="capsule" class="collision" size="0.01" fromto="-0.065 -0.02 -0.035 0.11 -0.02 -0.035" />
|
||||
<geom name="right_foot_inner_neg_collision" type="capsule" class="collision" size="0.01" fromto="-0.0775 -0.01 -0.035 0.1225 -0.01 -0.035" />
|
||||
<geom name="right_foot_center_collision" type="capsule" class="collision" size="0.01" fromto="-0.0825 0 -0.035 0.1275 0 -0.035" />
|
||||
<geom name="right_foot_inner_pos_collision" type="capsule" class="collision" size="0.01" fromto="-0.0775 0.01 -0.035 0.1225 0.01 -0.035" />
|
||||
<geom name="right_foot_outer_pos_collision" type="capsule" class="collision" size="0.01" fromto="-0.065 0.02 -0.035 0.11 0.02 -0.035" />
|
||||
<geom name="right_foot_toe_pos_collision" type="capsule" class="collision" size="0.01" fromto="0.03 0.03 -0.035 0.09 0.03 -0.035" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
|
||||
<body name="waist_roll_link" pos="-0.00757994881845658 -0.00301269924435144 0.0529838303646947" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="waist_roll_joint" type="hinge" range="-0.7 0.7" armature="0.0290856" />
|
||||
<inertial pos="-0.03058205 0.00004115 -0.00029517" mass="1.53201882" fullinertia="0.00221249 0.00166198 0.00164327 -0.00000117 -0.00000777 -0.00000329" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="waist_roll_link" material="waist_roll_material" />
|
||||
<geom name="waist_roll_neg_y_collision" type="capsule" class="collision" size="0.04164" fromto="-0.03686 -0.01336 -0.01336 -0.03686 -0.01336 0.02536" />
|
||||
<geom name="waist_roll_pos_y_collision" type="capsule" class="collision" size="0.04164" fromto="-0.03686 0.01336 -0.01336 -0.03686 0.01336 0.02536" />
|
||||
<body name="waist_yaw_link" pos="-0.0308449999997339 0 0.0679999999997294" quat="1 0 0 0">
|
||||
<joint axis="0 0 -1" name="waist_yaw_joint" type="hinge" range="-2.91 2.91" armature="0.0290856" />
|
||||
<inertial pos="-0.001424 0.000080 0.128866" mass="8.3430" fullinertia="0.109179 0.092853 0.041145 -0.000027 0.002082 -0.000013" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="waist_yaw_link" material="waist_yaw_material" />
|
||||
<geom name="torso_lower_collision" type="capsule" class="collision" size="0.090" fromto="0.0100 0 0.0394 0.0100 0 0.0800" />
|
||||
<geom name="torso_middle_collision" type="capsule" class="collision" size="0.120" fromto="-0.0244 0 0.1450 -0.0244 0 0.2100" />
|
||||
<geom name="torso_upper_collision" type="capsule" class="collision" size="0.088" fromto="-0.0185 0 0.3350 -0.0185 0 0.33668" />
|
||||
|
||||
<body name="left_shoulder_pitch_link" pos="0 0.116496194698121 0.19848715574272" quat="1 0 0 0">
|
||||
<joint axis="0 0.99619 0.087156" name="left_shoulder_pitch_joint" type="hinge" range="-3.4 2.35" armature="0.00884" />
|
||||
<inertial pos="-0.00227488 0.05877203 -0.01145894" mass="0.82850012" fullinertia="0.00077624 0.00060519 0.00074547 0.00001858 -0.00000678 0.00008598" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_shoulder_pitch_link" material="left_shoulder_pitch_material" />
|
||||
<geom name="left_shoulder_pitch_collision" type="capsule" class="collision" size="0.03808" fromto="-0.00326 0.03582 -0.00238 -0.00326 0.06284 -0.01210" />
|
||||
<body name="left_shoulder_roll_link" pos="0.0287432951006891 0.0653494445514435 -0.014608906526987" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="left_shoulder_roll_joint" type="hinge" range="-0.47 2.9" armature="0.00313313" />
|
||||
<inertial pos="-0.01613398 0.00000564 -0.06609383" mass="0.76916758" fullinertia="0.00079857 0.00081142 0.00051033 0.00000012 -0.00012702 -0.00000004" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_shoulder_roll_link" material="left_shoulder_roll_material" />
|
||||
<geom name="left_shoulder_roll_collision" type="capsule" class="collision" size="0.04000" fromto="-0.01906 0 -0.06800 -0.01906 0 -0.00656" />
|
||||
<body name="left_shoulder_yaw_link" pos="-0.0190574930004302 0 -0.105000039994097" quat="1 0 0 0">
|
||||
<joint axis="0 0 -1" name="left_shoulder_yaw_joint" type="hinge" range="-2.87 2.87" armature="0.00313313" />
|
||||
<inertial pos="0.01447570 -0.00112029 -0.08587480" mass="0.95227320" fullinertia="0.00179605 0.00188468 0.00081430 -0.00000292 0.00031386 -0.00001579" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_shoulder_yaw_link" material="left_shoulder_yaw_material" />
|
||||
<geom name="left_shoulder_yaw_collision" type="capsule" class="collision" size="0.04008" fromto="0.01816 0.00008 -0.10150 0.00008 0.00008 -0.01708" />
|
||||
<body name="left_elbow_link" pos="0.020796 -0.031888 -0.10414" quat="1 0 0 0">
|
||||
<joint axis="0 -1 0" name="left_elbow_joint" type="hinge" range="-1.6 1.24" armature="0.00313313" />
|
||||
<inertial pos="0.07222795 0.03107894 -0.01809537" mass="0.77922872" fullinertia="0.00060310 0.00112121 0.00118539 -0.00005057 0.00006993 0.00001181" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_elbow_link" material="left_elbow_material" />
|
||||
<geom name="left_elbow_collision" type="capsule" class="collision" size="0.04220" fromto="0.00487 0.03186 -0.00484 0.07959 0.03186 -0.01404" />
|
||||
<body name="left_wrist_link" pos="0.116093381994033 0.031859446289993 -0.0187927584979322" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="left_wrist_joint" type="hinge" range="-2.56 2.56" armature="0.00313313" />
|
||||
<inertial pos="0.10279104 -0.00368067 0.00149822" mass="0.46025356" fullinertia="0.00032495 0.00202426 0.00199776 0.00013222 -0.00009324 0.00001796" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="left_wrist_link" material="left_wrist_material" />
|
||||
<geom name="left_wrist_collision" type="capsule" class="collision" size="0.03948" fromto="0.01646 -0.00206 0.00203 0.16132 -0.01175 0.00203" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
|
||||
<body name="right_shoulder_pitch_link" pos="0 -0.116496194698064 0.198487155742749" quat="1 0 0 0">
|
||||
<joint axis="0 -0.99619 0.087156" name="right_shoulder_pitch_joint" type="hinge" range="-2.35 3.4" armature="0.00884" />
|
||||
<inertial pos="-0.00227477 -0.05856201 -0.01145858" mass="0.82850796" fullinertia="0.00077392 0.00060520 0.00074316 -0.00001370 -0.00000678 -0.00008562" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_shoulder_pitch_link" material="right_shoulder_pitch_material" />
|
||||
<geom name="right_shoulder_pitch_collision" type="capsule" class="collision" size="0.03808" fromto="-0.00326 -0.06284 -0.01210 -0.00326 -0.03582 -0.00238" />
|
||||
<body name="right_shoulder_roll_link" pos="0.0287432951007422 -0.0653494445235536 -0.0146089065305009" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="right_shoulder_roll_joint" type="hinge" range="-2.9 0.47" armature="0.00313313" />
|
||||
<inertial pos="-0.01626203 0.00000539 -0.06810375" mass="0.80442734" fullinertia="0.00087815 0.00088810 0.00052415 0.00000011 -0.00013154 -0.00000005" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_shoulder_roll_link" material="right_shoulder_roll_material" />
|
||||
<geom name="right_shoulder_roll_collision" type="capsule" class="collision" size="0.04000" fromto="-0.01906 0 -0.07700 -0.01906 0 -0.00656" />
|
||||
<body name="right_shoulder_yaw_link" pos="-0.0190573923593691 0 -0.105000126276637" quat="1 0 0 0">
|
||||
<joint axis="0 0 -1" name="right_shoulder_yaw_joint" type="hinge" range="-2.87 2.87" armature="0.00313313" />
|
||||
<inertial pos="0.01430772 0.00112150 -0.08592529" mass="0.95225273" fullinertia="0.00179792 0.00188417 0.00081194 -0.00000160 0.00031064 0.00001448" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_shoulder_yaw_link" material="right_shoulder_yaw_material" />
|
||||
<geom name="right_shoulder_yaw_collision" type="capsule" class="collision" size="0.04008" fromto="0.01816 -0.00008 -0.10150 0.00008 -0.00008 -0.01708" />
|
||||
<body name="right_elbow_link" pos="0.0207956801502895 0.031887525882762 -0.104135222723484" quat="1 0 0 0">
|
||||
<joint axis="0 1 0" name="right_elbow_joint" type="hinge" range="-1.24 1.6" armature="0.00313313" />
|
||||
<inertial pos="0.07222795 -0.03090613 -0.01799316" mass="0.77922499" fullinertia="0.00060275 0.00112238 0.00118388 0.00005257 0.00007112 -0.00001226" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_elbow_link" material="right_elbow_material" />
|
||||
<geom name="right_elbow_collision" type="capsule" class="collision" size="0.04220" fromto="0.00487 -0.03186 -0.00484 0.07959 -0.03186 -0.01404" />
|
||||
<body name="right_wrist_link" pos="0.11609334409225 -0.031859369715491 -0.0187927299688337" quat="1 0 0 0">
|
||||
<joint axis="1 0 0" name="right_wrist_joint" type="hinge" range="-2.56 2.56" armature="0.00313313" />
|
||||
<inertial pos="0.10279542 0.00367933 0.00149885" mass="0.46029783" fullinertia="0.00032496 0.00202412 0.00199758 -0.00013231 -0.00009325 -0.00001798" />
|
||||
<geom type="mesh" class="visual" pos="0 0 0" quat="1 0 0 0" mesh="right_wrist_link" material="right_wrist_material" />
|
||||
<geom name="right_wrist_collision" type="capsule" class="collision" size="0.03948" fromto="0.01646 0.00206 0.00203 0.16132 0.01175 0.00203" />
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</body>
|
||||
</worldbody>
|
||||
<asset>
|
||||
<mesh file="meshes/base_link.STL" />
|
||||
<mesh file="meshes/left_hip_pitch_link.STL" />
|
||||
<mesh file="meshes/left_hip_roll_link.STL" />
|
||||
<mesh file="meshes/left_hip_yaw_link.STL" />
|
||||
<mesh file="meshes/left_knee_link.STL" />
|
||||
<mesh file="meshes/left_ankle_pitch_link.STL" />
|
||||
<mesh file="meshes/left_ankle_roll_link.STL" />
|
||||
<mesh file="meshes/right_hip_pitch_link.STL" />
|
||||
<mesh file="meshes/right_hip_roll_link.STL" />
|
||||
<mesh file="meshes/right_hip_yaw_link.STL" />
|
||||
<mesh file="meshes/right_knee_link.STL" />
|
||||
<mesh file="meshes/right_ankle_pitch_link.STL" />
|
||||
<mesh file="meshes/right_ankle_roll_link.STL" />
|
||||
<mesh file="meshes/waist_roll_link.STL" />
|
||||
<mesh file="meshes/waist_yaw_link.STL" />
|
||||
<mesh file="meshes/left_shoulder_pitch_link.STL" />
|
||||
<mesh file="meshes/left_shoulder_roll_link.STL" />
|
||||
<mesh file="meshes/left_shoulder_yaw_link.STL" />
|
||||
<mesh file="meshes/left_elbow_link.STL" />
|
||||
<mesh file="meshes/left_wrist_link.STL" />
|
||||
<mesh file="meshes/right_shoulder_pitch_link.STL" />
|
||||
<mesh file="meshes/right_shoulder_roll_link.STL" />
|
||||
<mesh file="meshes/right_shoulder_yaw_link.STL" />
|
||||
<mesh file="meshes/right_elbow_link.STL" />
|
||||
<mesh file="meshes/right_wrist_link.STL" />
|
||||
<material name="base_link_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_hip_pitch_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_hip_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_hip_yaw_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_knee_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_ankle_pitch_material" rgba="0.650980 0.619608 0.588235 1" />
|
||||
<material name="left_ankle_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_hip_pitch_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_hip_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_hip_yaw_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_knee_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_ankle_pitch_material" rgba="0.650980 0.619608 0.588235 1" />
|
||||
<material name="right_ankle_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="waist_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="waist_yaw_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_shoulder_pitch_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_shoulder_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_shoulder_yaw_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_elbow_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="left_wrist_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_shoulder_pitch_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_shoulder_roll_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_shoulder_yaw_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_elbow_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
<material name="right_wrist_material" rgba="0.752941 0.752941 0.752941 1" />
|
||||
</asset>
|
||||
<actuator>
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="left_hip_pitch_joint" joint="left_hip_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="left_hip_roll_joint" joint="left_hip_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-55 55" name="left_hip_yaw_joint" joint="left_hip_yaw_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="left_knee_joint" joint="left_knee_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-64 64" name="left_ankle_pitch_joint" joint="left_ankle_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-64 64" name="left_ankle_roll_joint" joint="left_ankle_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="right_hip_pitch_joint" joint="right_hip_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="right_hip_roll_joint" joint="right_hip_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-55 55" name="right_hip_yaw_joint" joint="right_hip_yaw_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="right_knee_joint" joint="right_knee_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-64 64" name="right_ankle_pitch_joint" joint="right_ankle_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-64 64" name="right_ankle_roll_joint" joint="right_ankle_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="waist_roll_joint" joint="waist_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-140 140" name="waist_yaw_joint" joint="waist_yaw_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-55 55" name="left_shoulder_pitch_joint" joint="left_shoulder_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="left_shoulder_roll_joint" joint="left_shoulder_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="left_shoulder_yaw_joint" joint="left_shoulder_yaw_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="left_elbow_joint" joint="left_elbow_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="left_wrist_joint" joint="left_wrist_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-55 55" name="right_shoulder_pitch_joint" joint="right_shoulder_pitch_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="right_shoulder_roll_joint" joint="right_shoulder_roll_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="right_shoulder_yaw_joint" joint="right_shoulder_yaw_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="right_elbow_joint" joint="right_elbow_joint" />
|
||||
<motor class="RP1" gear="1" ctrlrange="-32 32" name="right_wrist_joint" joint="right_wrist_joint" />
|
||||
</actuator>
|
||||
<sensor>
|
||||
<framequat name="orientation" objtype="site" noise="0.001" objname="imu" />
|
||||
<framepos name="position" objtype="site" noise="0.001" objname="imu" />
|
||||
<gyro name="angular-velocity" site="imu" noise="0.005" cutoff="34.9" />
|
||||
<velocimeter name="linear-velocity" site="imu" noise="0.001" cutoff="30" />
|
||||
<accelerometer name="linear-acceleration" site="imu" noise="0.005" cutoff="157" />
|
||||
<magnetometer name="magnetometer" site="imu" />
|
||||
</sensor>
|
||||
<visual>
|
||||
<global offwidth="1920" offheight="1080" />
|
||||
</visual>
|
||||
</mujoco>
|
||||
Binary file not shown.
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"layout": "positions_then_velocities",
|
||||
"dtype": "float32-le",
|
||||
"step_dt": 0.02,
|
||||
"frames": 6573,
|
||||
"joint_names": [
|
||||
"left_hip_pitch_joint",
|
||||
"left_hip_roll_joint",
|
||||
"left_hip_yaw_joint",
|
||||
"left_knee_joint",
|
||||
"left_ankle_pitch_joint",
|
||||
"left_ankle_roll_joint",
|
||||
"right_hip_pitch_joint",
|
||||
"right_hip_roll_joint",
|
||||
"right_hip_yaw_joint",
|
||||
"right_knee_joint",
|
||||
"right_ankle_pitch_joint",
|
||||
"right_ankle_roll_joint",
|
||||
"waist_roll_joint",
|
||||
"waist_yaw_joint",
|
||||
"left_shoulder_pitch_joint",
|
||||
"left_shoulder_roll_joint",
|
||||
"left_shoulder_yaw_joint",
|
||||
"left_elbow_joint",
|
||||
"left_wrist_joint",
|
||||
"right_shoulder_pitch_joint",
|
||||
"right_shoulder_roll_joint",
|
||||
"right_shoulder_yaw_joint",
|
||||
"right_elbow_joint",
|
||||
"right_wrist_joint"
|
||||
],
|
||||
"data": "motions/dance.f32"
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"layout": "positions_then_velocities",
|
||||
"dtype": "float32-le",
|
||||
"step_dt": 0.02,
|
||||
"frames": 218,
|
||||
"joint_names": [
|
||||
"left_hip_pitch_joint",
|
||||
"left_hip_roll_joint",
|
||||
"left_hip_yaw_joint",
|
||||
"left_knee_joint",
|
||||
"left_ankle_pitch_joint",
|
||||
"left_ankle_roll_joint",
|
||||
"right_hip_pitch_joint",
|
||||
"right_hip_roll_joint",
|
||||
"right_hip_yaw_joint",
|
||||
"right_knee_joint",
|
||||
"right_ankle_pitch_joint",
|
||||
"right_ankle_roll_joint",
|
||||
"waist_roll_joint",
|
||||
"waist_yaw_joint",
|
||||
"left_shoulder_pitch_joint",
|
||||
"left_shoulder_roll_joint",
|
||||
"left_shoulder_yaw_joint",
|
||||
"left_elbow_joint",
|
||||
"left_wrist_joint",
|
||||
"right_shoulder_pitch_joint",
|
||||
"right_shoulder_roll_joint",
|
||||
"right_shoulder_yaw_joint",
|
||||
"right_elbow_joint",
|
||||
"right_wrist_joint"
|
||||
],
|
||||
"data": "motions/wave.f32"
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,692 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"id": "rp1",
|
||||
"name": "RP1",
|
||||
"family": "rp1",
|
||||
"adapter": "generic",
|
||||
"dimensions": {
|
||||
"height": 1.32
|
||||
},
|
||||
"model": "model.xml",
|
||||
"assets": [
|
||||
{
|
||||
"path": "meshes/base_link.STL",
|
||||
"vfs": "meshes/base_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_hip_pitch_link.STL",
|
||||
"vfs": "meshes/left_hip_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_hip_roll_link.STL",
|
||||
"vfs": "meshes/left_hip_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_hip_yaw_link.STL",
|
||||
"vfs": "meshes/left_hip_yaw_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_knee_link.STL",
|
||||
"vfs": "meshes/left_knee_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_ankle_pitch_link.STL",
|
||||
"vfs": "meshes/left_ankle_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_ankle_roll_link.STL",
|
||||
"vfs": "meshes/left_ankle_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_hip_pitch_link.STL",
|
||||
"vfs": "meshes/right_hip_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_hip_roll_link.STL",
|
||||
"vfs": "meshes/right_hip_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_hip_yaw_link.STL",
|
||||
"vfs": "meshes/right_hip_yaw_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_knee_link.STL",
|
||||
"vfs": "meshes/right_knee_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_ankle_pitch_link.STL",
|
||||
"vfs": "meshes/right_ankle_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_ankle_roll_link.STL",
|
||||
"vfs": "meshes/right_ankle_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/waist_roll_link.STL",
|
||||
"vfs": "meshes/waist_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/waist_yaw_link.STL",
|
||||
"vfs": "meshes/waist_yaw_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_shoulder_pitch_link.STL",
|
||||
"vfs": "meshes/left_shoulder_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_shoulder_roll_link.STL",
|
||||
"vfs": "meshes/left_shoulder_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_shoulder_yaw_link.STL",
|
||||
"vfs": "meshes/left_shoulder_yaw_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_elbow_link.STL",
|
||||
"vfs": "meshes/left_elbow_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/left_wrist_link.STL",
|
||||
"vfs": "meshes/left_wrist_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_shoulder_pitch_link.STL",
|
||||
"vfs": "meshes/right_shoulder_pitch_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_shoulder_roll_link.STL",
|
||||
"vfs": "meshes/right_shoulder_roll_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_shoulder_yaw_link.STL",
|
||||
"vfs": "meshes/right_shoulder_yaw_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_elbow_link.STL",
|
||||
"vfs": "meshes/right_elbow_link.STL"
|
||||
},
|
||||
{
|
||||
"path": "meshes/right_wrist_link.STL",
|
||||
"vfs": "meshes/right_wrist_link.STL"
|
||||
}
|
||||
],
|
||||
"base": {
|
||||
"type": "floating",
|
||||
"root_body": "base_link",
|
||||
"freejoint": "base_link_freejoint",
|
||||
"spawn": {
|
||||
"pos": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.78
|
||||
],
|
||||
"yaw": 0
|
||||
},
|
||||
"initial": {
|
||||
"left_hip_pitch_joint": -0.1,
|
||||
"left_hip_roll_joint": 0.0,
|
||||
"left_hip_yaw_joint": 0.0,
|
||||
"left_knee_joint": -0.3,
|
||||
"left_ankle_pitch_joint": -0.2,
|
||||
"left_ankle_roll_joint": 0.0,
|
||||
"right_hip_pitch_joint": 0.1,
|
||||
"right_hip_roll_joint": 0.0,
|
||||
"right_hip_yaw_joint": 0.0,
|
||||
"right_knee_joint": 0.3,
|
||||
"right_ankle_pitch_joint": -0.2,
|
||||
"right_ankle_roll_joint": 0.0,
|
||||
"waist_roll_joint": 0.0,
|
||||
"waist_yaw_joint": 0.0,
|
||||
"left_shoulder_pitch_joint": 0.2,
|
||||
"left_shoulder_roll_joint": 0.2,
|
||||
"left_shoulder_yaw_joint": 0.0,
|
||||
"left_elbow_joint": -1.2,
|
||||
"left_wrist_joint": 0.0,
|
||||
"right_shoulder_pitch_joint": -0.2,
|
||||
"right_shoulder_roll_joint": -0.2,
|
||||
"right_shoulder_yaw_joint": 0.0,
|
||||
"right_elbow_joint": 1.2,
|
||||
"right_wrist_joint": 0.0
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"timestep": 0.005,
|
||||
"decimation": 4
|
||||
},
|
||||
"joints": {
|
||||
"groups": {
|
||||
"policy": [
|
||||
"left_hip_pitch_joint",
|
||||
"left_hip_roll_joint",
|
||||
"left_hip_yaw_joint",
|
||||
"left_knee_joint",
|
||||
"left_ankle_pitch_joint",
|
||||
"left_ankle_roll_joint",
|
||||
"right_hip_pitch_joint",
|
||||
"right_hip_roll_joint",
|
||||
"right_hip_yaw_joint",
|
||||
"right_knee_joint",
|
||||
"right_ankle_pitch_joint",
|
||||
"right_ankle_roll_joint",
|
||||
"waist_roll_joint",
|
||||
"waist_yaw_joint",
|
||||
"left_shoulder_pitch_joint",
|
||||
"left_shoulder_roll_joint",
|
||||
"left_shoulder_yaw_joint",
|
||||
"left_elbow_joint",
|
||||
"left_wrist_joint",
|
||||
"right_shoulder_pitch_joint",
|
||||
"right_shoulder_roll_joint",
|
||||
"right_shoulder_yaw_joint",
|
||||
"right_elbow_joint",
|
||||
"right_wrist_joint"
|
||||
]
|
||||
},
|
||||
"defaults": {
|
||||
"left_hip_pitch_joint": -0.1,
|
||||
"left_hip_roll_joint": 0.0,
|
||||
"left_hip_yaw_joint": 0.0,
|
||||
"left_knee_joint": -0.3,
|
||||
"left_ankle_pitch_joint": -0.2,
|
||||
"left_ankle_roll_joint": 0.0,
|
||||
"right_hip_pitch_joint": 0.1,
|
||||
"right_hip_roll_joint": 0.0,
|
||||
"right_hip_yaw_joint": 0.0,
|
||||
"right_knee_joint": 0.3,
|
||||
"right_ankle_pitch_joint": -0.2,
|
||||
"right_ankle_roll_joint": 0.0,
|
||||
"waist_roll_joint": 0.0,
|
||||
"waist_yaw_joint": 0.0,
|
||||
"left_shoulder_pitch_joint": 0.2,
|
||||
"left_shoulder_roll_joint": 0.2,
|
||||
"left_shoulder_yaw_joint": 0.0,
|
||||
"left_elbow_joint": -1.2,
|
||||
"left_wrist_joint": 0.0,
|
||||
"right_shoulder_pitch_joint": -0.2,
|
||||
"right_shoulder_roll_joint": -0.2,
|
||||
"right_shoulder_yaw_joint": 0.0,
|
||||
"right_elbow_joint": 1.2,
|
||||
"right_wrist_joint": 0.0
|
||||
}
|
||||
},
|
||||
"observations": {
|
||||
"obs": {
|
||||
"history": 3,
|
||||
"layout": "term_major",
|
||||
"terms": [
|
||||
{
|
||||
"type": "gyro",
|
||||
"sensor": "angular-velocity"
|
||||
},
|
||||
{
|
||||
"type": "projected_gravity",
|
||||
"body": "base_link"
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"size": 3
|
||||
},
|
||||
{
|
||||
"type": "joint_pos_rel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "joint_vel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "last_action",
|
||||
"group": "policy"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"type": "joint_position",
|
||||
"group": "policy",
|
||||
"scale": 0.25,
|
||||
"offset": "default"
|
||||
}
|
||||
],
|
||||
"actuation": {
|
||||
"type": "joint_pd",
|
||||
"kp": 120,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140,
|
||||
"joints": {
|
||||
"left_hip_pitch_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"left_hip_roll_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"left_hip_yaw_joint": {
|
||||
"kp": 35.0,
|
||||
"kd": 2.5,
|
||||
"torque_limit": 55.0,
|
||||
"velocity_limit": 16.23156,
|
||||
"keep_velocity": 6.28319
|
||||
},
|
||||
"left_knee_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"left_ankle_pitch_joint": {
|
||||
"kp": 25.0,
|
||||
"kd": 1.5,
|
||||
"torque_limit": 64.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"left_ankle_roll_joint": {
|
||||
"kp": 25.0,
|
||||
"kd": 1.5,
|
||||
"torque_limit": 64.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_hip_pitch_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"right_hip_roll_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"right_hip_yaw_joint": {
|
||||
"kp": 35.0,
|
||||
"kd": 2.5,
|
||||
"torque_limit": 55.0,
|
||||
"velocity_limit": 16.23156,
|
||||
"keep_velocity": 6.28319
|
||||
},
|
||||
"right_knee_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"right_ankle_pitch_joint": {
|
||||
"kp": 25.0,
|
||||
"kd": 1.5,
|
||||
"torque_limit": 64.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_ankle_roll_joint": {
|
||||
"kp": 25.0,
|
||||
"kd": 1.5,
|
||||
"torque_limit": 64.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"waist_roll_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"waist_yaw_joint": {
|
||||
"kp": 120.0,
|
||||
"kd": 7.5,
|
||||
"torque_limit": 140.0,
|
||||
"velocity_limit": 13.61357,
|
||||
"keep_velocity": 8.37758
|
||||
},
|
||||
"left_shoulder_pitch_joint": {
|
||||
"kp": 35.0,
|
||||
"kd": 2.5,
|
||||
"torque_limit": 55.0,
|
||||
"velocity_limit": 16.23156,
|
||||
"keep_velocity": 6.28319
|
||||
},
|
||||
"left_shoulder_roll_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"left_shoulder_yaw_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"left_elbow_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"left_wrist_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_shoulder_pitch_joint": {
|
||||
"kp": 35.0,
|
||||
"kd": 2.5,
|
||||
"torque_limit": 55.0,
|
||||
"velocity_limit": 16.23156,
|
||||
"keep_velocity": 6.28319
|
||||
},
|
||||
"right_shoulder_roll_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_shoulder_yaw_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_elbow_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
},
|
||||
"right_wrist_joint": {
|
||||
"kp": 12.5,
|
||||
"kd": 0.75,
|
||||
"torque_limit": 32.0,
|
||||
"velocity_limit": 15.70796,
|
||||
"keep_velocity": 14.66077
|
||||
}
|
||||
}
|
||||
},
|
||||
"commands": {
|
||||
"size": 3,
|
||||
"limits": [
|
||||
0.4,
|
||||
-0.3,
|
||||
0.6
|
||||
]
|
||||
},
|
||||
"skills": [
|
||||
{
|
||||
"id": "walk",
|
||||
"kind": "primary",
|
||||
"owns": "policy",
|
||||
"controller": {
|
||||
"type": "onnx",
|
||||
"policy": "policies/walk.onnx"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "dance",
|
||||
"kind": "episodic",
|
||||
"owns": "policy",
|
||||
"available": true,
|
||||
"duration": 10.0,
|
||||
"blend": 0.5,
|
||||
"controller": {
|
||||
"type": "onnx",
|
||||
"policy": "policies/dance.onnx"
|
||||
},
|
||||
"defaults": {
|
||||
"left_hip_pitch_joint": -0.1,
|
||||
"left_hip_roll_joint": 0.0,
|
||||
"left_hip_yaw_joint": 0.0,
|
||||
"left_knee_joint": -0.3,
|
||||
"left_ankle_pitch_joint": -0.2,
|
||||
"left_ankle_roll_joint": 0.0,
|
||||
"right_hip_pitch_joint": 0.1,
|
||||
"right_hip_roll_joint": 0.0,
|
||||
"right_hip_yaw_joint": 0.0,
|
||||
"right_knee_joint": 0.3,
|
||||
"right_ankle_pitch_joint": -0.2,
|
||||
"right_ankle_roll_joint": 0.0,
|
||||
"waist_roll_joint": 0.0,
|
||||
"waist_yaw_joint": 0.0,
|
||||
"left_shoulder_pitch_joint": 0.2,
|
||||
"left_shoulder_roll_joint": 0.2,
|
||||
"left_shoulder_yaw_joint": 0.0,
|
||||
"left_elbow_joint": -1.2,
|
||||
"left_wrist_joint": 0.0,
|
||||
"right_shoulder_pitch_joint": -0.2,
|
||||
"right_shoulder_roll_joint": -0.2,
|
||||
"right_shoulder_yaw_joint": 0.0,
|
||||
"right_elbow_joint": 1.2,
|
||||
"right_wrist_joint": 0.0
|
||||
},
|
||||
"reference": "motions/dance.json",
|
||||
"observations": {
|
||||
"obs": {
|
||||
"terms": [
|
||||
{
|
||||
"type": "motion_command",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "gyro",
|
||||
"sensor": "angular-velocity"
|
||||
},
|
||||
{
|
||||
"type": "projected_gravity",
|
||||
"body": "base_link"
|
||||
},
|
||||
{
|
||||
"type": "joint_pos_rel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "joint_vel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "last_action",
|
||||
"group": "policy"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"type": "joint_position",
|
||||
"group": "policy",
|
||||
"scale": 0.25,
|
||||
"offset": "default"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "wave",
|
||||
"kind": "episodic",
|
||||
"owns": "policy",
|
||||
"available": true,
|
||||
"duration": 4.36,
|
||||
"blend": 0.5,
|
||||
"controller": {
|
||||
"type": "onnx",
|
||||
"policy": "policies/wave.onnx"
|
||||
},
|
||||
"defaults": {
|
||||
"left_hip_pitch_joint": -0.1,
|
||||
"left_hip_roll_joint": 0.0,
|
||||
"left_hip_yaw_joint": 0.0,
|
||||
"left_knee_joint": -0.3,
|
||||
"left_ankle_pitch_joint": -0.2,
|
||||
"left_ankle_roll_joint": 0.0,
|
||||
"right_hip_pitch_joint": 0.1,
|
||||
"right_hip_roll_joint": 0.0,
|
||||
"right_hip_yaw_joint": 0.0,
|
||||
"right_knee_joint": 0.3,
|
||||
"right_ankle_pitch_joint": -0.2,
|
||||
"right_ankle_roll_joint": 0.0,
|
||||
"waist_roll_joint": 0.0,
|
||||
"waist_yaw_joint": 0.0,
|
||||
"left_shoulder_pitch_joint": 0.2,
|
||||
"left_shoulder_roll_joint": 0.2,
|
||||
"left_shoulder_yaw_joint": 0.0,
|
||||
"left_elbow_joint": -1.2,
|
||||
"left_wrist_joint": 0.0,
|
||||
"right_shoulder_pitch_joint": -0.2,
|
||||
"right_shoulder_roll_joint": -0.2,
|
||||
"right_shoulder_yaw_joint": 0.0,
|
||||
"right_elbow_joint": 1.2,
|
||||
"right_wrist_joint": 0.0
|
||||
},
|
||||
"reference": "motions/wave.json",
|
||||
"observations": {
|
||||
"obs": {
|
||||
"terms": [
|
||||
{
|
||||
"type": "motion_command",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "gyro",
|
||||
"sensor": "angular-velocity"
|
||||
},
|
||||
{
|
||||
"type": "projected_gravity",
|
||||
"body": "base_link"
|
||||
},
|
||||
{
|
||||
"type": "joint_pos_rel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "joint_vel",
|
||||
"group": "policy"
|
||||
},
|
||||
{
|
||||
"type": "last_action",
|
||||
"group": "policy"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"type": "joint_position",
|
||||
"group": "policy",
|
||||
"scale": 0.25,
|
||||
"offset": "default"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"input": {
|
||||
"bindings": [
|
||||
{
|
||||
"action": "move_forward",
|
||||
"codes": [
|
||||
"ArrowUp",
|
||||
"KeyW"
|
||||
],
|
||||
"label": "Forward"
|
||||
},
|
||||
{
|
||||
"action": "move_backward",
|
||||
"codes": [
|
||||
"ArrowDown",
|
||||
"KeyS"
|
||||
],
|
||||
"label": "Backward"
|
||||
},
|
||||
{
|
||||
"action": "turn_left",
|
||||
"codes": [
|
||||
"ArrowLeft",
|
||||
"KeyA"
|
||||
],
|
||||
"label": "Turn left"
|
||||
},
|
||||
{
|
||||
"action": "turn_right",
|
||||
"codes": [
|
||||
"ArrowRight",
|
||||
"KeyD"
|
||||
],
|
||||
"label": "Turn right"
|
||||
},
|
||||
{
|
||||
"action": "reset",
|
||||
"codes": [
|
||||
"Space"
|
||||
],
|
||||
"label": "Reset"
|
||||
},
|
||||
{
|
||||
"action": "pause",
|
||||
"codes": [
|
||||
"Escape"
|
||||
],
|
||||
"label": "Pause / continue"
|
||||
},
|
||||
{
|
||||
"action": "chaseToggle",
|
||||
"codes": [
|
||||
"KeyC"
|
||||
],
|
||||
"label": "Restore chase camera"
|
||||
},
|
||||
{
|
||||
"action": "skill:dance",
|
||||
"codes": [
|
||||
"KeyJ"
|
||||
],
|
||||
"label": "Dance",
|
||||
"disabled": false
|
||||
},
|
||||
{
|
||||
"action": "skill:wave",
|
||||
"codes": [
|
||||
"KeyH"
|
||||
],
|
||||
"label": "Wave",
|
||||
"disabled": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"render": {
|
||||
"source": "mjcf_visual",
|
||||
"groups": [
|
||||
2
|
||||
],
|
||||
"camera": {
|
||||
"distance": 2.4
|
||||
}
|
||||
},
|
||||
"presentation": {
|
||||
"theme": "comic"
|
||||
},
|
||||
"termination": {
|
||||
"tilt_z": -0.5,
|
||||
"min_height": 0.4,
|
||||
"on_fall": "hold"
|
||||
},
|
||||
"license": "Local evaluation; RP1 mesh license not supplied upstream. Local evaluation only; see SOURCE.md. Actuation omits randomized delay/domain randomization."
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
{
|
||||
"source": "https://github.com/Roboparty/rp1_description",
|
||||
"commit": "d8c6dd03aeecc7f7ead3d962e09fedba4c8460f0",
|
||||
"checkpoint": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl/rp1_24dof_flat/2026-10-28_15-47-49/model_9000.pt",
|
||||
"checkpoint_sha256": "5ecf0891f65cff08b2f93fef4326bb097021fb2396598de2d80be38e9247c20e",
|
||||
"iteration": 9000,
|
||||
"training_params_sha256": {
|
||||
"env.yaml": "1b899ff3a774fcba93a64d79298c7e2cc53151fc9ca80461e3c230302fb3fc82",
|
||||
"agent.yaml": "05c11617af2dc7a8efdd0b7ba6deb768e8e59f43dab451b1b6305cacaf9c7ceb"
|
||||
},
|
||||
"source_model_sha256": "4cdb024f9c2a721fa0ef2a7e987ab69ebaf66218bfdd2c20aa91c2f2d5e3495e",
|
||||
"export_max_abs_error": 1.5497207641601562e-06,
|
||||
"files": [
|
||||
{
|
||||
"file": "model.xml",
|
||||
"sha256": "8c663d2cf2382d9ed0c705210c33a11dac12d3fb83eed573e4796b10f9cd5473"
|
||||
},
|
||||
{
|
||||
"file": "policies/walk.onnx",
|
||||
"sha256": "74593afcf52e7cca56e521ad93900d62fc5f9ceefad6114d590bf77f05413be8"
|
||||
},
|
||||
{
|
||||
"file": "LICENSES/RoboLab-BSD-3-Clause.txt",
|
||||
"sha256": "e8dec07c6537300f8dcbf9801c68137c23a0b745b3d7f0278a6d30bc5ed50297"
|
||||
},
|
||||
{
|
||||
"file": "meshes/base_link.STL",
|
||||
"sha256": "8fa93e11010c1d0a35ae79f03638fadfc26335320dbfbbd6ff56b5c6cb3c0d2e"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_hip_pitch_link.STL",
|
||||
"sha256": "cd12a399efb65721f3c75b73619971eafb26958367c30222ef3110bd4090a19d"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_hip_roll_link.STL",
|
||||
"sha256": "4ccc27044b9c274cd81dae6775a59036c6449bb3d64fb6ed70294f17046edcb8"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_hip_yaw_link.STL",
|
||||
"sha256": "e6361ed133e72f2040c88529746d4c17a057d60b0e8d253318fc94bea514e129"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_knee_link.STL",
|
||||
"sha256": "878aa675f24ad2154265e72c4a3fbed65de041337dfe5c80c74f3333e6f6602a"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_ankle_pitch_link.STL",
|
||||
"sha256": "c4f5d66f6d6f0eaa0ef232b841d85907fca99ec81013b0682f35277ae16cc9e1"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_ankle_roll_link.STL",
|
||||
"sha256": "4ac951e3a1adef25aff5d5a9557bab03e293340bb5713cb2b43d3888947560c0"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_hip_pitch_link.STL",
|
||||
"sha256": "a427e4c982b181623b8f322f45be08ebfc2f73abdf5492afb8728e0fefa80d83"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_hip_roll_link.STL",
|
||||
"sha256": "52c5c2edb3e83aafdbc633ee10ff4c368993e6c889d6a560d75982da5cc9a799"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_hip_yaw_link.STL",
|
||||
"sha256": "7e792ca8795b4f44b7de1e2e8747a6c188c10fb8810b17f3269965dde27292bc"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_knee_link.STL",
|
||||
"sha256": "ffe17fc9ba1614ae86610dc1b657461a3d1d70350729296c9210f68f3934d01d"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_ankle_pitch_link.STL",
|
||||
"sha256": "f19f73d04f8b0252e4001de541657864c44c5e7da204b7487d24eff162644b6c"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_ankle_roll_link.STL",
|
||||
"sha256": "057843359003abc4e5ec0a9763833440c42b881f853c1ddbd3945bd1168ff947"
|
||||
},
|
||||
{
|
||||
"file": "meshes/waist_roll_link.STL",
|
||||
"sha256": "500f00489349bf018bbf325d9ed67dd272629b5f37e734a82b141329a9d656df"
|
||||
},
|
||||
{
|
||||
"file": "meshes/waist_yaw_link.STL",
|
||||
"sha256": "36de51037852da450e75b86ef000d145e9e1b37e30925edd3e2307d5c2f449d3"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_shoulder_pitch_link.STL",
|
||||
"sha256": "2c0eb4ed8ccd410dc2d7f6949829a59d05889ab27ea4944120e8689720e6c40b"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_shoulder_roll_link.STL",
|
||||
"sha256": "41f6738917180d3aa62798f279e99f4d64ed57950d6138f8e249c7ca629444a9"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_shoulder_yaw_link.STL",
|
||||
"sha256": "4cfe28c2a31c5b7602c8546cac8fa491a97f1b88cbe579d48034bf857702adbb"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_elbow_link.STL",
|
||||
"sha256": "45e761a34c5570ec7b200a6b15cf6e5d7dfe61fea1497e1954e17140c3c400b1"
|
||||
},
|
||||
{
|
||||
"file": "meshes/left_wrist_link.STL",
|
||||
"sha256": "1184e55a4a98a75f2d9143e65cd5b1b1f6f42f35c49b4565c94d22f542df7c73"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_shoulder_pitch_link.STL",
|
||||
"sha256": "21ac44a4e038f09a7f0472490ee06c3ecb5c87cac51fdb645ad4382875235be0"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_shoulder_roll_link.STL",
|
||||
"sha256": "4435df453ffe4325b6783d217081ec6360d52252ddb87b7f894434eb9e60c39b"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_shoulder_yaw_link.STL",
|
||||
"sha256": "a88c4bce9e69979ae082de8420a031ca6772a4e0d08ab9f10af4b9617f4ec723"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_elbow_link.STL",
|
||||
"sha256": "34c9eeeb41588fc9970940d4d82db73ee0c3881613d1a76f0c90421d3b7c8e81"
|
||||
},
|
||||
{
|
||||
"file": "meshes/right_wrist_link.STL",
|
||||
"sha256": "09064a38d9f2bddcc8c6b705811426bc88473c9587221012d08bdbddce94eda2"
|
||||
},
|
||||
{
|
||||
"file": "policies/wave.onnx",
|
||||
"sha256": "1b20142e8ebb41e6aa0424a79fe2fe9abbc015c598acc0444d473f975e7f4ae2"
|
||||
},
|
||||
{
|
||||
"file": "motions/wave.json",
|
||||
"sha256": "54cdcd71d21e8a7e339a3beaf12e4810b753b7eb21b27bf9cd4f66b348823b8e"
|
||||
},
|
||||
{
|
||||
"file": "motions/wave.f32",
|
||||
"sha256": "6e3187978c53410a4faf4e2adad72149c596b274cbab378520d9490d60fbc04a"
|
||||
},
|
||||
{
|
||||
"file": "policies/dance.onnx",
|
||||
"sha256": "ea2b58b4004e62a789bfa8c231c25238c5a7a1347405b804c4f74c7464de9246"
|
||||
},
|
||||
{
|
||||
"file": "motions/dance.json",
|
||||
"sha256": "96df97dcdc8e6ee7c4dd2f0e421eb134877dc3061df72137365d9ce8b91d7a14"
|
||||
},
|
||||
{
|
||||
"file": "motions/dance.f32",
|
||||
"sha256": "4af5c93c4881019f9d5bb03d58a17a6368eecc6c9fe49eef0a05cf814b164522"
|
||||
}
|
||||
],
|
||||
"skills": {
|
||||
"dance": {
|
||||
"checkpoint": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl/rp1_24dof_beyondmimic_dance/2026-10-29_11-05-14_dance_resume/model_23998.pt",
|
||||
"checkpoint_sha256": "45f537be2678c4bd956de25719477523900e60d33fda49b6e8b7b2105f16cd84",
|
||||
"iteration": 23998,
|
||||
"configured_iterations": 16000,
|
||||
"initial_iteration": 7999,
|
||||
"expected_final_iteration": 23998,
|
||||
"resumed_from": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl/rp1_24dof_beyondmimic_dance/2026-10-28_17-40-59_dance/model_7999.pt",
|
||||
"motion": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/robolab/data/motions/rp1_bm/dance1_subject2_smooth6hz.npz",
|
||||
"motion_sha256": "3e56bdff732df480994167462fe7f0fb64a71c398dc09b72c57fb9e43254cf69",
|
||||
"frames": 6573,
|
||||
"fps": 50.0,
|
||||
"actor_observation_group": "student",
|
||||
"export_max_abs_error": 1.3113021850585938e-06,
|
||||
"training_params_sha256": {
|
||||
"env.yaml": "841e1d9266a1dbba371cb7a68b04a778395de04aa6fd2c6b6bbf96897795ce0e",
|
||||
"agent.yaml": "6b92c5f47b25026ff972fe2a833b95b40bfbec187361611e90cd1d18fffcc8bc"
|
||||
},
|
||||
"deployed_duration": 10.0,
|
||||
"validation": "browser-wasm-three-headings-and-transitions-passed",
|
||||
"limitation": "Only the first 10s are qualified for this checkpoint; full 131.46s playback has not been validated."
|
||||
},
|
||||
"wave": {
|
||||
"checkpoint": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl/rp1_24dof_beyondmimic_wave/2026-10-28_19-00-23_wave/model_7999.pt",
|
||||
"checkpoint_sha256": "14f33f4b20b7c64b371d4f15d2471d2c72e2fddf5d4a277efaa8f11b9abd8316",
|
||||
"iteration": 7999,
|
||||
"configured_iterations": 8000,
|
||||
"motion": "/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/robolab/data/motions/rp1_bm/wave_R_001__A428.npz",
|
||||
"motion_sha256": "6073c5db30e4dc5c66bb8dfe5b3debd5b7155cee4ad029f89ac03faf05756452",
|
||||
"frames": 218,
|
||||
"fps": 50.0,
|
||||
"actor_observation_group": "student",
|
||||
"export_max_abs_error": 1.9073486328125e-06,
|
||||
"training_params_sha256": {
|
||||
"env.yaml": "b33100d6fd34e79ac9b3e107d9a3eb6302c72c3f8d25e13fb09eaa5a389c4400",
|
||||
"agent.yaml": "1f87d8f271a74ce3872221243021ea71ad7ca28a4c8d95b48bf092f21d0b2947"
|
||||
},
|
||||
"deployed_duration": 4.36,
|
||||
"validation": "browser-wasm-three-headings-and-transitions-passed"
|
||||
}
|
||||
},
|
||||
"training_completed": true
|
||||
}
|
||||
@@ -3,6 +3,7 @@
|
||||
"id": "arena",
|
||||
"type": "flat",
|
||||
"presentation": "arena-grid",
|
||||
"scale_to_robot": { "reference_height": 0.27 },
|
||||
"boundary": {
|
||||
"half_extent": 2.25,
|
||||
"wall_height": 0.25,
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Stage a completed RP1 BeyondMimic STUDENT actor + reference. Never starts training.
|
||||
|
||||
Use roboparty_train/.venv/bin/python. Stages skills disabled; enable only after
|
||||
real physics playback/transition validation. Trusted local checkpoints only.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
import numpy as np
|
||||
import onnx
|
||||
from onnx.reference import ReferenceEvaluator
|
||||
import torch
|
||||
import yaml
|
||||
|
||||
|
||||
def sha(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def dump(path, value):
|
||||
temp = path.with_suffix(path.suffix + ".tmp")
|
||||
temp.write_text(json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n")
|
||||
temp.replace(path)
|
||||
|
||||
|
||||
def parameter(value, name):
|
||||
if not isinstance(value, dict):
|
||||
return float(value)
|
||||
matches = [float(v) for p, v in value.items() if re.fullmatch(p, name)]
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"Actuator parameter is ambiguous/missing: {name}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--checkpoint", type=Path, required=True)
|
||||
p.add_argument("--skill", choices=["dance", "wave"], required=True)
|
||||
p.add_argument("--duration", type=float, help="Episode seconds; dance defaults to the trained 10s horizon, wave to its full clip")
|
||||
p.add_argument("--package", type=Path, default=Path(__file__).resolve().parents[1] / "public/robots/rp1")
|
||||
args = p.parse_args()
|
||||
cp, target, skill_id = args.checkpoint.resolve(), args.package.resolve(), args.skill
|
||||
params = cp.parent / "params"
|
||||
env = yaml.load((params / "env.yaml").read_text(), Loader=yaml.BaseLoader)
|
||||
agent = yaml.load((params / "agent.yaml").read_text(), Loader=yaml.BaseLoader)
|
||||
config = json.loads((target / "robot.json").read_text())
|
||||
robot, group = env["scene"]["robot"], env["observations"]["student"]
|
||||
names = config["joints"]["groups"]["policy"]
|
||||
action = env["actions"]["joint_pos"]
|
||||
expected_terms = ["command", "base_ang_vel", "projected_gravity", "joint_pos", "joint_vel", "actions"]
|
||||
expected_funcs = ["generated_commands", "base_ang_vel", "projected_gravity", "joint_pos_rel", "joint_vel_rel", "last_action"]
|
||||
terms = [(k, v) for k, v in group.items() if isinstance(v, dict) and "func" in v]
|
||||
if not (
|
||||
agent["obs_groups"]["actor"] == ["student"]
|
||||
and group["history_length"] == "1"
|
||||
and group["concatenate_terms"] == "true"
|
||||
and [k for k, _ in terms] == expected_terms
|
||||
and [v["func"].split(":")[-1] for _, v in terms] == expected_funcs
|
||||
and all(v.get("scale") == "null" and v.get("clip") == "null"
|
||||
and v.get("modifiers") == "null" and v.get("delay_max_lag") == "0" for _, v in terms)
|
||||
and agent["actor"]["activation"] == "elu"
|
||||
and agent["actor"]["obs_normalization"] == "true"
|
||||
and agent["clip_actions"] == "null"
|
||||
and robot["joint_ordering"] == action["joint_names"] == names
|
||||
and action["preserve_order"] == action["use_default_offset"] == "true"
|
||||
and action["clip"] == "null" and float(action["offset"]) == 0
|
||||
and len(names) == 24
|
||||
and all(float(v) == 0 for v in robot["init_state"]["joint_vel"].values())
|
||||
and config["sim"] == {"timestep": float(env["sim"]["dt"]), "decimation": int(env["decimation"])}
|
||||
):
|
||||
raise ValueError("Unsupported logged actor/observation/action contract; update explicitly")
|
||||
actuator = robot["actuators"]["all_joints"]
|
||||
for n in names:
|
||||
for out, key in {"kp": "stiffness", "kd": "damping", "torque_limit": "actuator_effort_limit",
|
||||
"velocity_limit": "actuator_velocity_limit", "keep_velocity": "keep_velocity"}.items():
|
||||
if abs(config["actuation"]["joints"][n][out] - parameter(actuator[key], n)) > 1e-8:
|
||||
raise ValueError(f"Per-skill actuator contract differs: {n}/{out}")
|
||||
before = sha(cp)
|
||||
saved = torch.load(cp, map_location="cpu", weights_only=True)
|
||||
if int(saved["iter"]) < int(agent["max_iterations"]) - 1 or cp.stem != f"model_{saved['iter']}":
|
||||
raise ValueError("Only the final configured checkpoint may be imported")
|
||||
state = saved["actor_state_dict"]
|
||||
shapes = [(512, 126), (256, 512), (128, 256), (24, 128)]
|
||||
if any(tuple(state[f"mlp.{2*i}.weight"].shape) != shape for i, shape in enumerate(shapes)):
|
||||
raise ValueError("Unsupported actor architecture")
|
||||
if any(not torch.isfinite(v).all() for v in state.values()) or (state["obs_normalizer._std"] < 0).any():
|
||||
raise ValueError("Invalid checkpoint tensors")
|
||||
|
||||
class Actor(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.register_buffer("mean", state["obs_normalizer._mean"])
|
||||
self.register_buffer("std", state["obs_normalizer._std"])
|
||||
layers = []
|
||||
for i, (out_size, in_size) in enumerate(shapes):
|
||||
layer = torch.nn.Linear(in_size, out_size)
|
||||
layer.load_state_dict({"weight": state[f"mlp.{2*i}.weight"], "bias": state[f"mlp.{2*i}.bias"]})
|
||||
layers.append(layer)
|
||||
if i < len(shapes) - 1:
|
||||
layers.append(torch.nn.ELU())
|
||||
self.mlp = torch.nn.Sequential(*layers)
|
||||
|
||||
def forward(self, obs):
|
||||
return self.mlp((obs - self.mean) / (self.std + 1e-2))
|
||||
|
||||
motion_file = Path(env["commands"]["motion"]["motion_file"])
|
||||
with np.load(motion_file, allow_pickle=False) as motion:
|
||||
source_names = list(motion["joint_names"].astype(str))
|
||||
order = [source_names.index(n) for n in names]
|
||||
pos, vel = motion["joint_pos"][:, order], motion["joint_vel"][:, order]
|
||||
fps = float(motion["fps"].item())
|
||||
dt = config["sim"]["timestep"] * config["sim"]["decimation"]
|
||||
if abs(fps * dt - 1) > 1e-9 or pos.shape != vel.shape or pos.shape[1] != 24 or len(pos) < 2:
|
||||
raise ValueError("Motion/control clock or shape mismatch (no silent resampling)")
|
||||
if not (np.isfinite(pos).all() and np.isfinite(vel).all()):
|
||||
raise ValueError("Non-finite motion")
|
||||
duration = args.duration if args.duration is not None else min(10.0, len(pos) * dt)
|
||||
if not np.isfinite(duration) or not 0 < duration <= len(pos) * dt:
|
||||
raise ValueError("Duration must be within the reference clip")
|
||||
defaults = {n: float(robot["init_state"]["joint_pos"][n]) for n in names}
|
||||
for folder in ["policies", "motions"]:
|
||||
(target / folder).mkdir(exist_ok=True)
|
||||
policy = target / f"policies/{skill_id}.export.onnx"
|
||||
actor = Actor().eval()
|
||||
torch.onnx.export(actor, torch.zeros(1, 126), str(policy), input_names=["obs"],
|
||||
output_names=["actions"], opset_version=17, dynamo=False)
|
||||
model = onnx.load(policy)
|
||||
onnx.helper.set_model_props(model, {"checkpoint": cp.name, "checkpoint_sha256": before,
|
||||
"iteration": str(saved["iter"]), "actor_observation_group": "student",
|
||||
"joint_names": ",".join(names), "default_joint_pos": ",".join(map(str, defaults.values())),
|
||||
"motion_sha256": sha(motion_file), "status": "completed-training"})
|
||||
onnx.checker.check_model(model)
|
||||
onnx.save(model, policy)
|
||||
evaluator = ReferenceEvaluator(model)
|
||||
rng = np.random.default_rng(42)
|
||||
samples = [np.zeros((1, 126), np.float32), *[rng.normal(size=(1, 126)).astype(np.float32) for _ in range(16)]]
|
||||
standing = np.r_[pos[0], vel[0], [0, 0, 0], [0, 0, -1], np.zeros(72)].astype(np.float32)[None]
|
||||
samples.append(standing)
|
||||
error = 0.0
|
||||
for obs in samples:
|
||||
with torch.no_grad():
|
||||
expected = actor(torch.from_numpy(obs)).numpy()
|
||||
actual = evaluator.run(None, {"obs": obs})[0]
|
||||
np.testing.assert_allclose(actual, expected, rtol=2e-4, atol=2e-5)
|
||||
error = max(error, float(np.max(np.abs(actual - expected))))
|
||||
if before != sha(cp):
|
||||
raise ValueError("Checkpoint changed during export")
|
||||
policy.replace(target / f"policies/{skill_id}.onnx")
|
||||
binary = target / f"motions/{skill_id}.f32"
|
||||
temp = binary.with_suffix(".tmp")
|
||||
temp.write_bytes(np.concatenate([pos.ravel(), vel.ravel()]).astype("<f4").tobytes())
|
||||
temp.replace(binary)
|
||||
dump(target / f"motions/{skill_id}.json", {"schema": 1, "layout": "positions_then_velocities",
|
||||
"dtype": "float32-le", "step_dt": dt, "frames": len(pos), "joint_names": names,
|
||||
"data": f"motions/{skill_id}.f32"})
|
||||
skill = next(s for s in config["skills"] if s["id"] == skill_id)
|
||||
skill.update({"available": False, "duration": duration, "blend": 0.5,
|
||||
"defaults": defaults, "reference": f"motions/{skill_id}.json",
|
||||
"observations": {"obs": {"terms": [{"type": "motion_command", "group": "policy"},
|
||||
{"type": "gyro", "sensor": config["observations"]["obs"]["terms"][0]["sensor"]},
|
||||
{"type": "projected_gravity", "body": "base_link"},
|
||||
{"type": "joint_pos_rel", "group": "policy"}, {"type": "joint_vel", "group": "policy"},
|
||||
{"type": "last_action", "group": "policy"}]}},
|
||||
"actions": [{"type": "joint_position", "group": "policy", "scale": float(action["scale"]), "offset": "default"}],
|
||||
"controller": {"type": "onnx", "policy": f"policies/{skill_id}.onnx"}})
|
||||
next(b for b in config["input"]["bindings"] if b["action"] == f"skill:{skill_id}")["disabled"] = True
|
||||
provenance_path = target / "source-manifest.json"
|
||||
provenance = json.loads(provenance_path.read_text())
|
||||
provenance.setdefault("skills", {})[skill_id] = {"checkpoint": str(cp), "checkpoint_sha256": before,
|
||||
"iteration": int(saved["iter"]), "configured_iterations": int(agent["max_iterations"]),
|
||||
"motion": str(motion_file), "motion_sha256": sha(motion_file), "frames": len(pos), "fps": fps,
|
||||
"actor_observation_group": "student", "export_max_abs_error": error,
|
||||
"training_params_sha256": {n: sha(params / n) for n in ["env.yaml", "agent.yaml"]},
|
||||
"deployed_duration": duration, "validation": "staged-disabled"}
|
||||
for f in [f"policies/{skill_id}.onnx", f"motions/{skill_id}.json", f"motions/{skill_id}.f32"]:
|
||||
provenance["files"] = [r for r in provenance["files"] if r["file"] != f]
|
||||
provenance["files"].append({"file": f, "sha256": sha(target / f)})
|
||||
dump(provenance_path, provenance)
|
||||
dump(target / "robot.json", config)
|
||||
print(f"Staged {skill_id}: final {saved['iter']}, 126→24, {len(pos)} frames / {len(pos)*dt:.2f}s, max error {error:.3g}. DISABLED pending playback validation.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,211 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Import a trusted RP1-Flat checkpoint without starting/stopping Isaac Lab.
|
||||
|
||||
Run with roboparty_train/.venv/bin/python. Never run on an untrusted .pt file.
|
||||
Only the logged 24-DOF Direct/term-major/ELU MLP contract is supported.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
import numpy as np
|
||||
import onnx
|
||||
from onnx.reference import ReferenceEvaluator
|
||||
import torch
|
||||
import yaml
|
||||
|
||||
|
||||
def dump(path, value):
|
||||
path.write_text(json.dumps(value, indent=2, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def sha(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def resolve_parameter(value, name):
|
||||
if not isinstance(value, dict):
|
||||
return float(value)
|
||||
matches = [float(v) for pattern, v in value.items() if re.fullmatch(pattern, name)]
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"Ambiguous/missing actuator parameter: {name}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--source", type=Path, required=True)
|
||||
parser.add_argument("--checkpoint", type=Path, required=True)
|
||||
parser.add_argument("--policy-only", action="store_true", help="Preserve assets/other skills; replace walk ONNX and provenance only")
|
||||
parser.add_argument("--completed", action="store_true", help="Require final configured iteration and mark walking training complete")
|
||||
args = parser.parse_args()
|
||||
source, checkpoint = args.source.resolve(), args.checkpoint.resolve()
|
||||
public = Path(__file__).resolve().parents[1] / "public" / "robots"
|
||||
target = public / "rp1"
|
||||
params = checkpoint.parent / "params"
|
||||
# BaseLoader reads YAML data only, ignoring Python object construction tags.
|
||||
env = yaml.load((params / "env.yaml").read_text(), Loader=yaml.BaseLoader)
|
||||
agent = yaml.load((params / "agent.yaml").read_text(), Loader=yaml.BaseLoader)
|
||||
robot = env["scene_context"]["robot"]
|
||||
names = robot["joint_ordering"]
|
||||
if not (len(names) == 24 and env["robot"]["observation_history_layout"] == "term_major"
|
||||
and env["robot"]["actor_obs_history_length"] == "3"
|
||||
and agent["actor"]["activation"] == "elu"
|
||||
and agent["actor"]["obs_normalization"] == "true"
|
||||
and all(float(v) == 1 for v in env["normalization"]["obs_scales"].values())):
|
||||
raise ValueError("Unsupported training contract; update importer explicitly")
|
||||
defaults = {n: float(robot["init_state"]["joint_pos"][n]) for n in names}
|
||||
if args.policy_only:
|
||||
current = json.loads((target / "robot.json").read_text())
|
||||
if not (current["joints"]["groups"]["policy"] == names
|
||||
and current["joints"]["defaults"] == defaults
|
||||
and current["actions"][0]["scale"] == float(env["robot"]["action_scale"])
|
||||
and current["sim"] == {"timestep": float(env["sim"]["dt"]), "decimation": int(env["decimation"])}):
|
||||
raise ValueError("Training contract changed; full re-import/revalidation required")
|
||||
saved = torch.load(checkpoint, map_location="cpu", weights_only=True)
|
||||
if args.completed and int(saved["iter"]) < int(agent["max_iterations"]) - 1:
|
||||
raise ValueError("--completed requires the final configured training iteration")
|
||||
state = saved["actor_state_dict"]
|
||||
expected = [(512, 243), (256, 512), (128, 256), (24, 128)]
|
||||
if any(tuple(state[f"mlp.{2*i}.weight"].shape) != shape for i, shape in enumerate(expected)):
|
||||
raise ValueError("Unsupported actor architecture")
|
||||
if any(not torch.isfinite(v).all() for v in state.values()):
|
||||
raise ValueError("Non-finite checkpoint")
|
||||
|
||||
class Actor(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.register_buffer("mean", state["obs_normalizer._mean"])
|
||||
self.register_buffer("std", state["obs_normalizer._std"])
|
||||
layers = []
|
||||
for i, (out_size, in_size) in enumerate(expected):
|
||||
layer = torch.nn.Linear(in_size, out_size)
|
||||
layer.load_state_dict({"weight": state[f"mlp.{2*i}.weight"], "bias": state[f"mlp.{2*i}.bias"]})
|
||||
layers.append(layer)
|
||||
if i < len(expected) - 1:
|
||||
layers.append(torch.nn.ELU())
|
||||
self.mlp = torch.nn.Sequential(*layers)
|
||||
|
||||
def forward(self, obs):
|
||||
# RSL-RL EmpiricalNormalization epsilon is 1e-2, not sqrt(var+eps).
|
||||
action = self.mlp((obs - self.mean) / (self.std + 1e-2))
|
||||
return action.clamp(-float(agent["clip_actions"]), float(agent["clip_actions"]))
|
||||
|
||||
for folder in ["policies", "meshes", "LICENSES"]:
|
||||
(target / folder).mkdir(parents=True, exist_ok=True)
|
||||
actor = Actor().eval()
|
||||
policy = target / "policies" / "walk.export.onnx"
|
||||
torch.onnx.export(actor, torch.zeros(1, 243), str(policy), input_names=["obs"],
|
||||
output_names=["actions"], opset_version=17, dynamo=False)
|
||||
model = onnx.load(policy)
|
||||
onnx.helper.set_model_props(model, {
|
||||
"checkpoint": checkpoint.name, "checkpoint_sha256": sha(checkpoint),
|
||||
"iteration": str(saved["iter"]), "joint_names": ",".join(names),
|
||||
"default_joint_pos": ",".join(str(robot["init_state"]["joint_pos"][n]) for n in names),
|
||||
"history_layout": "term_major", "history_length": "3", "status": "completed-training" if args.completed else "training-preview",
|
||||
})
|
||||
onnx.checker.check_model(model)
|
||||
onnx.save(model, policy)
|
||||
evaluator = ReferenceEvaluator(model)
|
||||
rng = np.random.default_rng(42)
|
||||
max_error = 0.0
|
||||
for obs in [np.zeros((1, 243), dtype=np.float32), *[rng.normal(size=(1, 243)).astype(np.float32) for _ in range(8)]]:
|
||||
with torch.no_grad():
|
||||
expected_action = actor(torch.from_numpy(obs)).numpy()
|
||||
actual = evaluator.run(None, {"obs": obs})[0]
|
||||
np.testing.assert_allclose(actual, expected_action, rtol=2e-4, atol=2e-5)
|
||||
max_error = max(max_error, float(np.abs(actual - expected_action).max()))
|
||||
|
||||
policy.replace(target / "policies" / "walk.onnx")
|
||||
if args.policy_only:
|
||||
provenance_file = target / "source-manifest.json"
|
||||
provenance = json.loads(provenance_file.read_text())
|
||||
provenance.update({"training_completed": args.completed, "checkpoint": str(checkpoint), "checkpoint_sha256": sha(checkpoint),
|
||||
"iteration": int(saved["iter"]), "export_max_abs_error": max_error,
|
||||
"training_params_sha256": {name: sha(params / name) for name in ["env.yaml", "agent.yaml"]}})
|
||||
for record in provenance["files"]:
|
||||
if record["file"] == "policies/walk.onnx":
|
||||
record["sha256"] = sha(target / record["file"])
|
||||
dump(provenance_file, provenance)
|
||||
if args.completed:
|
||||
current["name"] = "RP1"
|
||||
dump(target / "robot.json", current)
|
||||
print(f"Updated walk policy only: iteration {saved['iter']}, max error {max_error:.3g}")
|
||||
return
|
||||
|
||||
asset_root = source / "robolab" / "data" / "robots" / "roboparty" / "rp1"
|
||||
tree = ET.parse(asset_root / "mjcf" / "robot_24dof.xml")
|
||||
root = tree.getroot()
|
||||
root.find("compiler").set("meshdir", ".")
|
||||
assets = []
|
||||
for mesh in root.findall("./asset/mesh"):
|
||||
rel = mesh.attrib["file"]
|
||||
shutil.copyfile(asset_root / rel, target / rel)
|
||||
assets.append({"path": rel, "vfs": rel})
|
||||
tree.write(target / "model.xml", encoding="utf-8", xml_declaration=True)
|
||||
actuator = robot["actuators"]["all_joints"]
|
||||
parameters = {}
|
||||
for name in names:
|
||||
parameters[name] = {out: resolve_parameter(actuator[key], name) for out, key in {
|
||||
"kp": "stiffness", "kd": "damping", "torque_limit": "actuator_effort_limit",
|
||||
"velocity_limit": "actuator_velocity_limit", "keep_velocity": "keep_velocity",
|
||||
}.items()}
|
||||
config = json.loads((public / "microban" / "robot.json").read_text())
|
||||
config.update({
|
||||
"id": "rp1", "family": "rp1", "name": "RP1" if args.completed else "RP1 (training preview)",
|
||||
"dimensions": {"height": 1.32},
|
||||
"assets": assets,
|
||||
"base": {"type": "floating", "root_body": "base_link", "freejoint": "base_link_freejoint",
|
||||
"spawn": {"pos": [float(v) for v in robot["init_state"]["pos"]], "yaw": 0}, "initial": defaults},
|
||||
"sim": {"timestep": float(env["sim"]["dt"]), "decimation": int(env["decimation"])},
|
||||
"joints": {"groups": {"policy": names}, "defaults": defaults},
|
||||
"observations": {"obs": {"history": 3, "layout": "term_major", "terms": [
|
||||
{"type": "gyro", "sensor": "base_ang_vel"},
|
||||
{"type": "projected_gravity", "body": "base_link"},
|
||||
{"type": "command", "size": 3},
|
||||
{"type": "joint_pos_rel", "group": "policy"},
|
||||
{"type": "joint_vel", "group": "policy"},
|
||||
{"type": "last_action", "group": "policy"},
|
||||
]}},
|
||||
"actions": [{"type": "joint_position", "group": "policy", "scale": float(env["robot"]["action_scale"]), "offset": "default"}],
|
||||
"actuation": {"type": "joint_pd", "kp": 120, "kd": 7.5, "torque_limit": 140, "joints": parameters},
|
||||
"commands": {"size": 3, "limits": [0.4, -0.3, 0.6]},
|
||||
"skills": [{"id": "walk", "kind": "primary", "owns": "policy", "controller": {"type": "onnx", "policy": "policies/walk.onnx"}},
|
||||
*[{"id": s, "kind": "episodic", "owns": "policy", "available": False,
|
||||
"duration": d, "blend": 0.5, "controller": {"type": "onnx"}} for s, d in [("dance", 10), ("wave", 4)]]],
|
||||
"render": {"source": "mjcf_visual", "groups": [2], "camera": {"distance": 2.4}},
|
||||
"termination": {"tilt_z": -0.5, "min_height": 0.4, "on_fall": "hold"},
|
||||
"license": "Training preview; RP1 mesh license not supplied upstream. Local evaluation only; see SOURCE.md. Actuation omits randomized delay/domain randomization.",
|
||||
})
|
||||
config["input"]["bindings"] = [b for b in config["input"]["bindings"] if not b["action"].startswith("toggle:")]
|
||||
config["input"]["bindings"].extend([
|
||||
{"action": "skill:dance", "codes": ["KeyJ"], "label": "Dance", "disabled": True},
|
||||
{"action": "skill:wave", "codes": ["KeyH"], "label": "Wave", "disabled": True},
|
||||
])
|
||||
# Sensor name is read from the actual source model, not guessed from other robots.
|
||||
gyros = root.findall("./sensor/gyro")
|
||||
if len(gyros) != 1 or gyros[0].get("site") != "imu":
|
||||
raise ValueError("Expected root-frame IMU gyro")
|
||||
config["observations"]["obs"]["terms"][0]["sensor"] = gyros[0].get("name")
|
||||
dump(target / "robot.json", config)
|
||||
shutil.copyfile(source / "robolab" / "LICENSE", target / "LICENSES" / "RoboLab-BSD-3-Clause.txt")
|
||||
commit = subprocess.check_output(["git", "-C", str(asset_root), "rev-parse", "HEAD"], text=True).strip()
|
||||
files = ["model.xml", "policies/walk.onnx", "LICENSES/RoboLab-BSD-3-Clause.txt", *[a["path"] for a in assets]]
|
||||
dump(target / "source-manifest.json", {
|
||||
"source": "https://github.com/Roboparty/rp1_description", "commit": commit,
|
||||
"training_completed": args.completed, "checkpoint": str(checkpoint), "checkpoint_sha256": sha(checkpoint), "iteration": int(saved["iter"]),
|
||||
"training_params_sha256": {name: sha(params / name) for name in ["env.yaml", "agent.yaml"]},
|
||||
"source_model_sha256": sha(asset_root / "mjcf" / "robot_24dof.xml"),
|
||||
"export_max_abs_error": max_error,
|
||||
"files": [{"file": f, "sha256": sha(target / f)} for f in files],
|
||||
})
|
||||
print(f"Imported {target}: iteration {saved['iter']}, ONNX/PyTorch max error {max_error:.3g}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
validateWorld,
|
||||
} from "../src/platform/loader/SchemaValidator.js";
|
||||
import { resolveWorldId } from "../src/platform/worlds/arena.js";
|
||||
import { JointMotionReference } from "../src/platform/policy/JointMotionReference.js";
|
||||
const root = fileURLToPath(new URL("../public/robots/", import.meta.url));
|
||||
const entries = JSON.parse(await readFile(resolve(root, "index.json"), "utf8"));
|
||||
for (const entry of entries) {
|
||||
@@ -18,9 +19,25 @@ for (const entry of entries) {
|
||||
config.model,
|
||||
...config.assets.map((a) => a.path),
|
||||
...config.skills
|
||||
.filter((s) => s.controller.type === "onnx")
|
||||
.filter((s) => s.controller.type === "onnx" && s.available !== false)
|
||||
.map((s) => s.controller.policy),
|
||||
];
|
||||
for (const skill of config.skills.filter((s) => s.reference)) {
|
||||
const spec = JSON.parse(
|
||||
await readFile(resolve(dirname(file), skill.reference), "utf8"),
|
||||
);
|
||||
const buffer = await readFile(resolve(dirname(file), spec.data));
|
||||
new JointMotionReference(
|
||||
spec,
|
||||
buffer.buffer.slice(
|
||||
buffer.byteOffset,
|
||||
buffer.byteOffset + buffer.byteLength,
|
||||
),
|
||||
config.joints.groups[skill.owns],
|
||||
config.sim.timestep * config.sim.decimation,
|
||||
);
|
||||
paths.push(skill.reference, spec.data);
|
||||
}
|
||||
for (const path of paths) await access(resolve(dirname(file), path));
|
||||
validateWorld(
|
||||
JSON.parse(
|
||||
|
||||
@@ -19,13 +19,14 @@ export function createArenaFloorPresentation({ sim, rig, world }) {
|
||||
}
|
||||
const root = new THREE.Group();
|
||||
root.name = "arena-floor-presentation";
|
||||
const grid = makeInfiniteGrid({ subdivisions: 1 });
|
||||
const scale = world?.scale ?? 1;
|
||||
const grid = makeInfiniteGrid({ subdivisions: 1, scale });
|
||||
grid.name = "arena-grid";
|
||||
grid.material.uniforms.uReveal.value = 1;
|
||||
grid.material.uniforms.uTopoScale.value = 0;
|
||||
root.add(grid);
|
||||
const { wallMeshes, wallMats } = world?.boundary
|
||||
? makeArenaWalls(world.boundary)
|
||||
? makeArenaWalls(world.boundary, { scale })
|
||||
: { wallMeshes: [], wallMats: [] };
|
||||
wallMeshes.forEach((mesh, i) => {
|
||||
mesh.name = `arena-boundary-${i}`;
|
||||
|
||||
@@ -15,16 +15,16 @@ import {
|
||||
// Lines derive from world coordinates, so re-centering the mesh under the
|
||||
// camera target every frame makes the grid effectively infinite without any
|
||||
// visible swimming.
|
||||
export function makeInfiniteGrid({ subdivisions = 256 } = {}) {
|
||||
export function makeInfiniteGrid({ subdivisions = 256, scale = 1 } = {}) {
|
||||
const material = new THREE.ShaderMaterial({
|
||||
transparent: true,
|
||||
depthWrite: false,
|
||||
uniforms: {
|
||||
uCell: { value: 0.1 },
|
||||
uSection: { value: GRID_SECTION },
|
||||
uCell: { value: 0.1 * scale },
|
||||
uSection: { value: GRID_SECTION * scale },
|
||||
uCellColor: { value: new THREE.Color(0x8e8371) },
|
||||
uSectionColor: { value: new THREE.Color(0xffb366) },
|
||||
uFadeDist: { value: 3.0 },
|
||||
uFadeDist: { value: 3.0 * scale },
|
||||
uFocus: { value: new THREE.Vector3() },
|
||||
// Entrance draw-in progress; 1 = steady state (branch skipped).
|
||||
// Starts at 0: the world stays hidden behind the welcome modal and
|
||||
@@ -143,7 +143,7 @@ export function makeInfiniteGrid({ subdivisions = 256 } = {}) {
|
||||
// is computed in world space, so the per-frame recentering under the
|
||||
// camera target doesn't move the bumps.
|
||||
const mesh = new THREE.Mesh(
|
||||
new THREE.PlaneGeometry(30, 30, subdivisions, subdivisions),
|
||||
new THREE.PlaneGeometry(30 * scale, 30 * scale, subdivisions, subdivisions),
|
||||
material,
|
||||
);
|
||||
mesh.rotation.x = -Math.PI / 2;
|
||||
@@ -154,19 +154,19 @@ export function makeInfiniteGrid({ subdivisions = 256 } = {}) {
|
||||
// cell/section lines from world coordinates, same radial fade around the
|
||||
// duck, plus a vertical fade toward the top edge so the walls read as a
|
||||
// light enclosure instead of solid slabs.
|
||||
function makeWallGridMaterial(alongX, wallHeight = ARENA_WALL_H) {
|
||||
function makeWallGridMaterial(alongX, wallHeight = ARENA_WALL_H, scale = 1) {
|
||||
return new THREE.ShaderMaterial({
|
||||
transparent: true,
|
||||
depthWrite: false,
|
||||
side: THREE.DoubleSide,
|
||||
uniforms: {
|
||||
uCell: { value: 0.1 },
|
||||
uSection: { value: GRID_SECTION },
|
||||
uCell: { value: 0.1 * scale },
|
||||
uSection: { value: GRID_SECTION * scale },
|
||||
uCellColor: { value: new THREE.Color(0x8e8371) },
|
||||
uSectionColor: { value: new THREE.Color(0xffb366) },
|
||||
// Gentler radial fade than the floor: the walls sit 1.5+ m from the
|
||||
// duck by construction and would vanish with the floor's 3 m fade.
|
||||
uFadeDist: { value: 5.0 },
|
||||
uFadeDist: { value: 5.0 * scale },
|
||||
uFocus: { value: new THREE.Vector3() },
|
||||
uWallH: { value: wallHeight },
|
||||
uAlongX: { value: alongX ? 1.0 : 0.0 },
|
||||
@@ -286,11 +286,14 @@ export function entranceLineCues() {
|
||||
// The four wall planes at their inner faces (three coords: MJCF x -> x,
|
||||
// MJCF y -> -z). Returns the meshes plus their materials (the ceremony and
|
||||
// the per-frame focus update both need the material list).
|
||||
export function makeArenaWalls({
|
||||
half_extent = ARENA_HALF,
|
||||
wall_height = ARENA_WALL_H,
|
||||
wall_thickness = ARENA_WALL_T,
|
||||
} = {}) {
|
||||
export function makeArenaWalls(
|
||||
{
|
||||
half_extent = ARENA_HALF,
|
||||
wall_height = ARENA_WALL_H,
|
||||
wall_thickness = ARENA_WALL_T,
|
||||
} = {},
|
||||
{ scale = 1 } = {},
|
||||
) {
|
||||
const wallMats = [];
|
||||
const wallMeshes = [];
|
||||
const wallLen = 2 * (half_extent + wall_thickness);
|
||||
@@ -301,7 +304,7 @@ export function makeArenaWalls({
|
||||
{ x: 0, z: -half_extent, rotY: 0, alongX: true },
|
||||
];
|
||||
for (const [index, w] of wallDefs.entries()) {
|
||||
const mat = makeWallGridMaterial(w.alongX, wall_height);
|
||||
const mat = makeWallGridMaterial(w.alongX, wall_height, scale);
|
||||
wallMats.push(mat);
|
||||
const mesh = new THREE.Mesh(
|
||||
new THREE.PlaneGeometry(wallLen, wall_height),
|
||||
|
||||
@@ -28,7 +28,9 @@ export class InputRouter {
|
||||
constructor({ bindings, getVelocityLimits, isEnabled = () => true }) {
|
||||
Object.assign(this, { bindings, getVelocityLimits, isEnabled });
|
||||
this.byCode = new Map(
|
||||
bindings.flatMap((b) => b.codes.map((code) => [code, b.action])),
|
||||
bindings
|
||||
.filter((b) => !b.disabled)
|
||||
.flatMap((b) => b.codes.map((code) => [code, b.action])),
|
||||
);
|
||||
}
|
||||
init(target = window) {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { validatePackage, validateWorld } from "./SchemaValidator.js";
|
||||
import { DEFAULT_WORLD } from "../worlds/arena.js";
|
||||
import { DEFAULT_WORLD, fitWorldToRobot } from "../worlds/arena.js";
|
||||
export function assetUrl(path, base, location = globalThis.location) {
|
||||
const url = new URL(path, base);
|
||||
if (location && url.origin !== location.origin)
|
||||
@@ -43,7 +43,7 @@ export async function loadIndex() {
|
||||
url: assetUrl(entry.manifest, base),
|
||||
}));
|
||||
}
|
||||
export async function loadWorld(id = DEFAULT_WORLD) {
|
||||
export async function loadWorld(id = DEFAULT_WORLD, config) {
|
||||
const base = new URL(
|
||||
`${import.meta.env.BASE_URL}worlds/${id}/`,
|
||||
location.origin,
|
||||
@@ -52,5 +52,5 @@ export async function loadWorld(id = DEFAULT_WORLD) {
|
||||
validateWorld(world);
|
||||
if (world.id !== id)
|
||||
throw new Error(`worlds/${id}: unsupported World schema`);
|
||||
return world;
|
||||
return fitWorldToRobot(world, config);
|
||||
}
|
||||
|
||||
@@ -12,6 +12,10 @@ function assertSchema(check, value, label) {
|
||||
}
|
||||
export function validateWorld(world) {
|
||||
assertSchema(checkWorld, world, "world.json");
|
||||
if (world.scale_to_robot && (world.type !== "flat" || !world.boundary))
|
||||
throw new Error(
|
||||
"world.json /scale_to_robot: requires a flat World with boundary",
|
||||
);
|
||||
if (world.boundary && world.type !== "flat")
|
||||
throw new Error("world.json /boundary: requires a flat World");
|
||||
if (world.type === "flat" && !world.floor)
|
||||
@@ -37,7 +41,11 @@ export function validatePackage(config) {
|
||||
ids.add(skill.id);
|
||||
if (!groups[skill.owns])
|
||||
throw new Error(`skills.${skill.id}.owns: unknown group ${skill.owns}`);
|
||||
if (skill.controller.type === "onnx" && !skill.controller.policy)
|
||||
if (
|
||||
skill.controller.type === "onnx" &&
|
||||
skill.available !== false &&
|
||||
!skill.controller.policy
|
||||
)
|
||||
throw new Error(`skills.${skill.id}.controller.policy: required`);
|
||||
if (skill.kind === "primary") primaries++;
|
||||
if (skill.kind === "overlay")
|
||||
@@ -50,17 +58,92 @@ export function validatePackage(config) {
|
||||
if (
|
||||
config.adapter === "generic" &&
|
||||
(primaries !== 1 ||
|
||||
config.skills.some((s) => !["primary", "overlay"].includes(s.kind)))
|
||||
config.skills.some(
|
||||
(s) => !["primary", "overlay", "episodic"].includes(s.kind),
|
||||
))
|
||||
)
|
||||
throw new Error(
|
||||
"generic runtime requires one primary and disjoint overlays",
|
||||
"generic runtime requires one primary, disjoint overlays and optional episodic skills",
|
||||
);
|
||||
for (const obs of Object.values(config.observations))
|
||||
const primary = config.skills.find((s) => s.kind === "primary");
|
||||
if (config.adapter === "generic") {
|
||||
if (primary.available === false || primary.controller.type !== "onnx")
|
||||
throw new Error("primary: an available ONNX controller is required");
|
||||
for (const skill of config.skills) {
|
||||
if (skill.available === false && skill.kind !== "episodic")
|
||||
throw new Error(
|
||||
"available:false is only supported for episodic skills",
|
||||
);
|
||||
if (skill.kind === "overlay" && skill.controller.type !== "oscillator")
|
||||
throw new Error("overlay: oscillator required");
|
||||
if (
|
||||
skill.kind === "episodic" &&
|
||||
(skill.owns !== primary.owns ||
|
||||
skill.controller.type !== "onnx" ||
|
||||
!skill.duration ||
|
||||
!skill.blend)
|
||||
)
|
||||
throw new Error(
|
||||
"episodic: requires primary ownership, ONNX, duration and blend",
|
||||
);
|
||||
if (
|
||||
skill.kind !== "episodic" &&
|
||||
(skill.observations ||
|
||||
skill.actions ||
|
||||
skill.defaults ||
|
||||
skill.reference)
|
||||
)
|
||||
throw new Error(
|
||||
"per-skill observations/actions/defaults require an episodic skill",
|
||||
);
|
||||
const motionTerms = Object.values(
|
||||
skill.observations ?? config.observations,
|
||||
)
|
||||
.flatMap((spec) => spec.terms)
|
||||
.filter((term) => term.type === "motion_command");
|
||||
if (motionTerms.length && (!skill.reference || skill.kind !== "episodic"))
|
||||
throw new Error(
|
||||
`skills.${skill.id}: motion_command requires an episodic reference`,
|
||||
);
|
||||
if (motionTerms.some((term) => term.group !== skill.owns))
|
||||
throw new Error(
|
||||
`skills.${skill.id}: motion reference must use owned joint order`,
|
||||
);
|
||||
for (const name of Object.keys(skill.defaults ?? {}))
|
||||
if (!groups[skill.owns].includes(name))
|
||||
throw new Error(
|
||||
`skills.${skill.id}.defaults: unknown owned joint ${name}`,
|
||||
);
|
||||
}
|
||||
for (const skill of config.skills.filter((s) =>
|
||||
["primary", "episodic"].includes(s.kind),
|
||||
)) {
|
||||
for (const action of skill.actions ?? config.actions)
|
||||
if (!groups[action.group])
|
||||
throw new Error(`actions: unknown group ${action.group}`);
|
||||
const names = (skill.actions ?? config.actions).flatMap(
|
||||
(a) => groups[a.group],
|
||||
);
|
||||
if (JSON.stringify(names) !== JSON.stringify(groups[skill.owns]))
|
||||
throw new Error(
|
||||
`skills.${skill.id}: action order must match owned joints`,
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const obs of [
|
||||
config.observations,
|
||||
...config.skills.map((s) => s.observations).filter(Boolean),
|
||||
].flatMap(Object.values))
|
||||
for (const term of obs.terms) {
|
||||
if (term.group && !groups[term.group])
|
||||
throw new Error(`observations: unknown group ${term.group}`);
|
||||
if (
|
||||
["joint_pos_rel", "joint_vel", "last_action"].includes(term.type) &&
|
||||
[
|
||||
"joint_pos_rel",
|
||||
"joint_vel",
|
||||
"last_action",
|
||||
"motion_command",
|
||||
].includes(term.type) &&
|
||||
!term.group
|
||||
)
|
||||
throw new Error(`${term.type}: group required`);
|
||||
@@ -76,7 +159,10 @@ export function validatePackage(config) {
|
||||
"observations.command: size missing or exceeds command buffer",
|
||||
);
|
||||
}
|
||||
for (const action of config.actions)
|
||||
for (const action of [
|
||||
config.actions,
|
||||
...config.skills.map((s) => s.actions).filter(Boolean),
|
||||
].flat())
|
||||
if (!groups[action.group])
|
||||
throw new Error(`actions: unknown group ${action.group}`);
|
||||
if (
|
||||
@@ -86,6 +172,17 @@ export function validatePackage(config) {
|
||||
)
|
||||
)
|
||||
throw new Error("actuation: PD gains and torque_limit required");
|
||||
for (const [name, p] of Object.entries(config.actuation.joints ?? {})) {
|
||||
if (!Object.values(groups).flat().includes(name))
|
||||
throw new Error(`actuation.joints: unknown joint ${name}`);
|
||||
if (
|
||||
(p.velocity_limit !== undefined || p.keep_velocity !== undefined) &&
|
||||
!(p.keep_velocity > 0 && p.keep_velocity < p.velocity_limit)
|
||||
)
|
||||
throw new Error(
|
||||
`actuation.${name}: keep_velocity must lie within velocity_limit`,
|
||||
);
|
||||
}
|
||||
const codes = new Set();
|
||||
for (const binding of config.input.bindings) {
|
||||
if (
|
||||
@@ -101,9 +198,19 @@ export function validatePackage(config) {
|
||||
...config.skills
|
||||
.filter((s) => s.kind === "overlay")
|
||||
.map((s) => `toggle:${s.id}`),
|
||||
...config.skills
|
||||
.filter((s) => s.kind === "episodic")
|
||||
.map((s) => `skill:${s.id}`),
|
||||
].includes(binding.action)
|
||||
)
|
||||
throw new Error(`input: unsupported action ${binding.action}`);
|
||||
if (binding.action.startsWith("skill:")) {
|
||||
const skill = config.skills.find((s) => s.id === binding.action.slice(6));
|
||||
if (skill && (skill.available === false) !== (binding.disabled === true))
|
||||
throw new Error(
|
||||
`input.${binding.action}: disabled must match skill availability`,
|
||||
);
|
||||
}
|
||||
for (const code of binding.codes) {
|
||||
if (codes.has(code)) throw new Error(`input: duplicate ${code}`);
|
||||
codes.add(code);
|
||||
|
||||
@@ -60,6 +60,14 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"dimensions": {
|
||||
"type": "object",
|
||||
"required": ["height"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"height": { "type": "number", "exclusiveMinimum": 0 }
|
||||
}
|
||||
},
|
||||
"base": {
|
||||
"type": "object",
|
||||
"required": ["type", "root_body", "freejoint", "spawn"],
|
||||
@@ -161,6 +169,8 @@
|
||||
"required": ["terms"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"history": { "type": "integer", "minimum": 1, "maximum": 64 },
|
||||
"layout": { "enum": ["frame_major", "term_major"] },
|
||||
"terms": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
@@ -176,6 +186,7 @@
|
||||
"joint_pos_rel",
|
||||
"joint_vel",
|
||||
"last_action",
|
||||
"motion_command",
|
||||
"command"
|
||||
]
|
||||
},
|
||||
@@ -240,6 +251,20 @@
|
||||
"torque_limit": {
|
||||
"type": "number",
|
||||
"exclusiveMinimum": 0
|
||||
},
|
||||
"joints": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"kp": { "type": "number", "minimum": 0 },
|
||||
"kd": { "type": "number", "minimum": 0 },
|
||||
"torque_limit": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"velocity_limit": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"keep_velocity": { "type": "number", "exclusiveMinimum": 0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -279,6 +304,13 @@
|
||||
"owns": {
|
||||
"type": "string"
|
||||
},
|
||||
"available": { "type": "boolean" },
|
||||
"duration": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"blend": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"observations": { "$ref": "#/properties/observations" },
|
||||
"actions": { "$ref": "#/properties/actions" },
|
||||
"defaults": { "$ref": "#/properties/joints/properties/defaults" },
|
||||
"reference": { "type": "string", "minLength": 1 },
|
||||
"controller": {
|
||||
"type": "object",
|
||||
"required": ["type"],
|
||||
@@ -332,7 +364,8 @@
|
||||
},
|
||||
"label": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"disabled": { "type": "boolean" }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,14 +8,22 @@
|
||||
"type": { "enum": ["flat", "adapter"] },
|
||||
"adapter": { "type": "string", "minLength": 1 },
|
||||
"presentation": { "enum": ["arena-grid"] },
|
||||
"scale_to_robot": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["reference_height"],
|
||||
"properties": {
|
||||
"reference_height": { "type": "number", "exclusiveMinimum": 0 }
|
||||
}
|
||||
},
|
||||
"boundary": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["half_extent", "wall_height", "wall_thickness"],
|
||||
"properties": {
|
||||
"half_extent": {"type":"number","exclusiveMinimum":0},
|
||||
"wall_height": {"type":"number","exclusiveMinimum":0},
|
||||
"wall_thickness": {"type":"number","exclusiveMinimum":0}
|
||||
"half_extent": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"wall_height": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"wall_thickness": { "type": "number", "exclusiveMinimum": 0 }
|
||||
}
|
||||
},
|
||||
"floor": {
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
import { fetchChecked } from "../loader/PackageLoader.js";
|
||||
|
||||
// A trained command stream, not a kinematic animation: only observations read
|
||||
// these samples. Neither qpos nor joint targets are written by this provider.
|
||||
export class JointMotionReference {
|
||||
static async load(pkg, path, names, dt) {
|
||||
const spec = await fetchChecked(pkg.asset(path), "json");
|
||||
const bytes = await fetchChecked(pkg.asset(spec.data), "bytes");
|
||||
return new JointMotionReference(spec, bytes, names, dt);
|
||||
}
|
||||
constructor(spec, bytes, names, dt) {
|
||||
if (
|
||||
spec.schema !== 1 ||
|
||||
spec.layout !== "positions_then_velocities" ||
|
||||
spec.dtype !== "float32-le" ||
|
||||
!Number.isInteger(spec.frames) ||
|
||||
spec.frames < 2 ||
|
||||
!Number.isFinite(spec.step_dt) ||
|
||||
spec.step_dt <= 0 ||
|
||||
Math.abs(spec.step_dt - dt) > 1e-9 ||
|
||||
JSON.stringify(spec.joint_names) !== JSON.stringify(names)
|
||||
)
|
||||
throw new Error(
|
||||
"Motion reference: unsupported layout, joint order or control clock",
|
||||
);
|
||||
const count = spec.frames * names.length;
|
||||
if (bytes.byteLength !== count * 2 * 4)
|
||||
throw new Error(
|
||||
"Motion reference: truncated/incorrect sample count (check git lfs pull)",
|
||||
);
|
||||
const view = new DataView(bytes);
|
||||
this.values = new Float32Array(count * 2);
|
||||
for (let i = 0; i < this.values.length; i++) {
|
||||
const v = view.getFloat32(i * 4, true);
|
||||
if (!Number.isFinite(v))
|
||||
throw new Error("Motion reference: non-finite sample");
|
||||
this.values[i] = v;
|
||||
}
|
||||
Object.assign(this, {
|
||||
frames: spec.frames,
|
||||
dt: spec.step_dt,
|
||||
width: names.length,
|
||||
count,
|
||||
});
|
||||
this.reset();
|
||||
}
|
||||
reset() {
|
||||
this.start = null;
|
||||
this.frame = 0;
|
||||
}
|
||||
sample(time) {
|
||||
if (!Number.isFinite(time))
|
||||
throw new Error("Motion reference: invalid simulation time");
|
||||
this.start ??= time;
|
||||
this.frame = Math.min(
|
||||
this.frames - 1,
|
||||
Math.max(0, Math.floor((time - this.start + 1e-9) / this.dt)),
|
||||
);
|
||||
}
|
||||
write(out, offset) {
|
||||
const a = this.frame * this.width;
|
||||
out.set(this.values.subarray(a, a + this.width), offset);
|
||||
out.set(
|
||||
this.values.subarray(this.count + a, this.count + a + this.width),
|
||||
offset + this.width,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ export class ObservationPipeline {
|
||||
this.inputs = {};
|
||||
this.ops = [];
|
||||
this.details = {};
|
||||
this.history = new Map();
|
||||
for (const [name, spec] of Object.entries(specs)) {
|
||||
let offset = 0;
|
||||
const terms = [];
|
||||
@@ -64,15 +65,64 @@ export class ObservationPipeline {
|
||||
});
|
||||
offset += size;
|
||||
}
|
||||
if (buffers[name] && buffers[name].length !== offset)
|
||||
const length = spec.history ?? 1;
|
||||
const size = offset * length;
|
||||
if (buffers[name] && buffers[name].length !== size)
|
||||
throw new Error(`Observation buffer ${name}: incorrect size`);
|
||||
this.inputs[name] = buffers[name] ?? new Float32Array(offset);
|
||||
this.inputs[name] = buffers[name] ?? new Float32Array(size);
|
||||
this.history.set(name, {
|
||||
length,
|
||||
layout: spec.layout ?? "frame_major",
|
||||
terms,
|
||||
frame: new Float32Array(offset),
|
||||
frames: new Float32Array(size),
|
||||
initialized: false,
|
||||
});
|
||||
this.details[name] = terms;
|
||||
}
|
||||
}
|
||||
write() {
|
||||
reset() {
|
||||
for (const h of this.history.values()) h.initialized = false;
|
||||
}
|
||||
write({ advance = true } = {}) {
|
||||
for (const op of this.ops)
|
||||
op.implementation.write(this.inputs[op.name], op.offset, this.ctx, op);
|
||||
op.implementation.write(
|
||||
this.history.get(op.name).frame,
|
||||
op.offset,
|
||||
this.ctx,
|
||||
op,
|
||||
);
|
||||
for (const [name, h] of this.history) {
|
||||
const size = h.frame.length;
|
||||
// Diagnostic reads sample live physics without pushing the history clock.
|
||||
const frames = advance ? h.frames : h.frames.slice();
|
||||
if (!h.initialized) {
|
||||
// Isaac Lab CircularBuffer backfills every slot on the first sample.
|
||||
for (let i = 0; i < h.length; i++) frames.set(h.frame, i * size);
|
||||
h.initialized = advance;
|
||||
} else {
|
||||
if (advance) frames.copyWithin(0, size);
|
||||
frames.set(h.frame, (h.length - 1) * size);
|
||||
}
|
||||
const out = this.inputs[name];
|
||||
if (h.layout === "frame_major") out.set(frames);
|
||||
else {
|
||||
let offset = 0;
|
||||
for (const term of h.terms)
|
||||
for (let i = 0; i < h.length; i++) {
|
||||
out.set(
|
||||
frames.subarray(
|
||||
i * size + term.offset,
|
||||
i * size + term.offset + term.size,
|
||||
),
|
||||
offset,
|
||||
);
|
||||
offset += term.size;
|
||||
}
|
||||
}
|
||||
if (!out.every(Number.isFinite))
|
||||
throw new Error(`observations.${name}: non-finite tensor`);
|
||||
}
|
||||
return this.inputs;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,6 +57,14 @@ observationTerms.register("command", {
|
||||
for (let i = 0; i < op.term.size; i++) out[offset + i] = c.command[i];
|
||||
},
|
||||
});
|
||||
observationTerms.register("motion_command", {
|
||||
size: (_term, _ctx, names) => names.length * 2,
|
||||
init: (_term, ctx) => {
|
||||
if (!ctx.reference)
|
||||
throw new Error("motion_command: missing per-skill reference");
|
||||
},
|
||||
write: (out, offset, ctx) => ctx.reference.write(out, offset),
|
||||
});
|
||||
for (const type of ["joint_pos_rel", "joint_vel", "last_action"])
|
||||
observationTerms.register(type, {
|
||||
size: (_term, _ctx, names) => names.length,
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { ObservationPipeline } from "../ObservationPipeline.js";
|
||||
import { ActionPipeline } from "../ActionPipeline.js";
|
||||
import { OnnxController } from "./OnnxController.js";
|
||||
|
||||
// Common lifecycle for independently trained, feed-forward joint-target skills.
|
||||
export class OnnxPolicy {
|
||||
static async load(ort, bytes, config, shared, skill, reference = null) {
|
||||
let session;
|
||||
try {
|
||||
const ctx = {
|
||||
...shared,
|
||||
reference,
|
||||
defaults: new Map([
|
||||
...shared.defaults,
|
||||
...Object.entries(skill.defaults ?? {}),
|
||||
]),
|
||||
lastAction: new Float32Array(shared.policyNames.length),
|
||||
};
|
||||
const obs = new ObservationPipeline(
|
||||
skill.observations ?? config.observations,
|
||||
ctx,
|
||||
);
|
||||
const actions = new ActionPipeline(skill.actions ?? config.actions, ctx);
|
||||
session = await ort.InferenceSession.create(bytes, {
|
||||
executionProviders: ["wasm"],
|
||||
});
|
||||
const controller = new OnnxController(ort, session, obs, actions.size);
|
||||
obs.write();
|
||||
const smoke = await controller.run();
|
||||
if (smoke.length !== actions.size || !smoke.every(Number.isFinite))
|
||||
throw new Error(`ONNX ${skill.id}: standing-pose smoke test failed`);
|
||||
const policy = new OnnxPolicy();
|
||||
Object.assign(policy, { ctx, obs, actions, controller });
|
||||
policy.reset();
|
||||
return policy;
|
||||
} catch (error) {
|
||||
await session?.release();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
reset() {
|
||||
this.ctx.lastAction.fill(0);
|
||||
this.ctx.reference?.reset();
|
||||
this.obs.reset();
|
||||
}
|
||||
async run() {
|
||||
this.ctx.reference?.sample(this.ctx.sim.data.time);
|
||||
this.obs.write();
|
||||
this.actions.apply(await this.controller.run());
|
||||
}
|
||||
dispose() {
|
||||
return this.controller.dispose();
|
||||
}
|
||||
}
|
||||
@@ -9,9 +9,9 @@ import {
|
||||
JointPdActuation,
|
||||
NativeActuation,
|
||||
} from "../sim/actuation/JointPdActuation.js";
|
||||
import { ObservationPipeline } from "../policy/ObservationPipeline.js";
|
||||
import { ActionPipeline } from "../policy/ActionPipeline.js";
|
||||
import { OnnxController } from "../policy/controllers/OnnxController.js";
|
||||
import { OnnxPolicy } from "../policy/controllers/OnnxPolicy.js";
|
||||
import { JointMotionReference } from "../policy/JointMotionReference.js";
|
||||
import { PolicyStateMachine } from "../skills/PolicyStateMachine.js";
|
||||
import { SkillManager } from "../skills/SkillManager.js";
|
||||
import { InputRouter } from "../input/InputRouter.js";
|
||||
import { RigRenderer } from "../render/RigRenderer.js";
|
||||
@@ -22,7 +22,8 @@ export class RobotRuntime {
|
||||
const { pkg, world } = options,
|
||||
config = pkg.config,
|
||||
{ mujoco, ort } = await loadRuntimes();
|
||||
let sim, session, rig, worldPresentation;
|
||||
let sim, rig, worldPresentation;
|
||||
const policies = new Map();
|
||||
try {
|
||||
sim = await SimCore.load(mujoco, pkg, world);
|
||||
const primary = config.skills.find((s) => s.kind === "primary");
|
||||
@@ -47,10 +48,7 @@ export class RobotRuntime {
|
||||
if ([...defaults.values()].some((v) => !Number.isFinite(v)))
|
||||
throw new Error("joints.defaults: missing/invalid reference pose");
|
||||
sim.reset(defaults);
|
||||
session = await ort.InferenceSession.create(bytes, {
|
||||
executionProviders: ["wasm"],
|
||||
});
|
||||
const ctx = {
|
||||
const shared = {
|
||||
sim,
|
||||
config,
|
||||
defaults,
|
||||
@@ -61,13 +59,35 @@ export class RobotRuntime {
|
||||
names.map((n) => [n, sim.data.qpos[sim.joints.get(n).qpos]]),
|
||||
),
|
||||
};
|
||||
const obs = new ObservationPipeline(config.observations, ctx),
|
||||
actions = new ActionPipeline(config.actions, ctx);
|
||||
const controller = new OnnxController(ort, session, obs, actions.size);
|
||||
obs.write();
|
||||
const smoke = await controller.run();
|
||||
if (smoke.length !== actions.size || !smoke.every(Number.isFinite))
|
||||
throw new Error("ONNX: standing-pose smoke test failed");
|
||||
for (const skill of config.skills.filter(
|
||||
(s) =>
|
||||
["primary", "episodic"].includes(s.kind) && s.available !== false,
|
||||
)) {
|
||||
const policyBytes =
|
||||
skill === primary
|
||||
? bytes
|
||||
: await fetchChecked(pkg.asset(skill.controller.policy), "bytes");
|
||||
const reference = skill.reference
|
||||
? await JointMotionReference.load(
|
||||
pkg,
|
||||
skill.reference,
|
||||
shared.policyNames,
|
||||
config.sim.timestep * config.sim.decimation,
|
||||
)
|
||||
: null;
|
||||
policies.set(
|
||||
skill.id,
|
||||
await OnnxPolicy.load(
|
||||
ort,
|
||||
policyBytes,
|
||||
config,
|
||||
shared,
|
||||
skill,
|
||||
reference,
|
||||
),
|
||||
);
|
||||
}
|
||||
const { ctx, obs, actions, controller } = policies.get(primary.id);
|
||||
rig = new RigRenderer(sim.model, sim.data, { ...config.render, world });
|
||||
worldPresentation = await options.createWorldPresentation?.({
|
||||
sim,
|
||||
@@ -82,13 +102,14 @@ export class RobotRuntime {
|
||||
obs,
|
||||
actions,
|
||||
controller,
|
||||
policies,
|
||||
rig,
|
||||
worldPresentation,
|
||||
});
|
||||
} catch (error) {
|
||||
worldPresentation?.dispose();
|
||||
rig?.dispose();
|
||||
await session?.release();
|
||||
await Promise.allSettled([...policies.values()].map((p) => p.dispose()));
|
||||
sim?.dispose();
|
||||
throw error;
|
||||
}
|
||||
@@ -106,6 +127,8 @@ export class RobotRuntime {
|
||||
this.active = false;
|
||||
this.positions = new Map();
|
||||
this.skills = new SkillManager(this.config);
|
||||
this.policyState = new PolicyStateMachine(this.config);
|
||||
this.activePolicy = this.policyState.primary;
|
||||
this.actuation =
|
||||
this.config.actuation.type === "joint_pd"
|
||||
? new JointPdActuation(
|
||||
@@ -130,6 +153,8 @@ export class RobotRuntime {
|
||||
);
|
||||
else if (action === "chaseToggle") this.restoreCamera();
|
||||
else if (action.startsWith("toggle:")) this.toggle(action.slice(7));
|
||||
else if (action.startsWith("skill:"))
|
||||
this.policyState.request(action.slice(6), this.sim.data.time);
|
||||
};
|
||||
this.tickCount = 0;
|
||||
this.hzAt = performance.now();
|
||||
@@ -187,12 +212,25 @@ export class RobotRuntime {
|
||||
}
|
||||
status() {
|
||||
return {
|
||||
policyStatus: this.policyStatus(),
|
||||
actuationLabel:
|
||||
this.config.actuation.type === "joint_pd"
|
||||
? "Simplified actuation · joint PD"
|
||||
: "",
|
||||
};
|
||||
}
|
||||
policyStatus() {
|
||||
return this.policyState.episodes.size
|
||||
? `Skill: ${this.policyState.active} · ${this.policyState.state}`
|
||||
: "";
|
||||
}
|
||||
publishPolicyStatus() {
|
||||
const policyStatus = this.policyStatus();
|
||||
if (policyStatus !== this.lastPolicyStatus) {
|
||||
this.lastPolicyStatus = policyStatus;
|
||||
this.onStatus({ policyStatus });
|
||||
}
|
||||
}
|
||||
async deactivate() {
|
||||
this.active = false;
|
||||
await this.pause();
|
||||
@@ -223,7 +261,9 @@ export class RobotRuntime {
|
||||
await this.scheduler.pause();
|
||||
this.input.clear();
|
||||
this.ctx.command.fill(0);
|
||||
this.ctx.lastAction.fill(0);
|
||||
for (const policy of this.policies.values()) policy.reset();
|
||||
this.policyState.reset();
|
||||
this.activePolicy = this.policyState.primary;
|
||||
this.skills.reset();
|
||||
this.sim.reset(this.ctx.defaults);
|
||||
this.odo = 0;
|
||||
@@ -235,6 +275,7 @@ export class RobotRuntime {
|
||||
this.ctx.targets.set(name, this.sim.data.qpos[joint.qpos]);
|
||||
this.fallen = false;
|
||||
this.onStatus({ fallen: false });
|
||||
this.publishPolicyStatus();
|
||||
this.rig.sync();
|
||||
if (running) this.scheduler.start();
|
||||
}
|
||||
@@ -248,10 +289,26 @@ export class RobotRuntime {
|
||||
}
|
||||
async controlStep() {
|
||||
if (!this.fallen) {
|
||||
const v = this.sim.data.qvel,
|
||||
a = this.sim.rootDof;
|
||||
const id = this.policyState.step(
|
||||
this.sim.data.time,
|
||||
this.input.command,
|
||||
Math.hypot(v[a], v[a + 1]),
|
||||
Math.abs(v[a + 5]),
|
||||
this.ctx.targets,
|
||||
);
|
||||
this.publishPolicyStatus();
|
||||
const policy = this.policies.get(id);
|
||||
if (id !== this.activePolicy) {
|
||||
policy.reset();
|
||||
this.activePolicy = id;
|
||||
}
|
||||
this.ctx.command.set(this.input.command);
|
||||
this.obs.write();
|
||||
this.actions.apply(await this.controller.run());
|
||||
if (this.policyState.state !== "walk") this.ctx.command.fill(0);
|
||||
await policy.run();
|
||||
this.skills.update(this.sim.data.time, this.ctx.targets);
|
||||
this.policyState.blend(this.sim.data.time, this.ctx.targets);
|
||||
}
|
||||
for (let i = 0; i < this.config.sim.decimation; i++) {
|
||||
this.actuation.apply();
|
||||
@@ -338,11 +395,24 @@ export class RobotRuntime {
|
||||
scene: this.scene,
|
||||
world: this.world,
|
||||
command: this.input.command,
|
||||
lastAction: this.ctx.lastAction,
|
||||
get lastAction() {
|
||||
return self.policies.get(self.activePolicy).ctx.lastAction;
|
||||
},
|
||||
get referenceFrame() {
|
||||
return (
|
||||
self.policies.get(self.activePolicy).ctx.reference?.frame ?? null
|
||||
);
|
||||
},
|
||||
skills: this.skills,
|
||||
policyState: this.policyState,
|
||||
get activePolicy() {
|
||||
return self.activePolicy;
|
||||
},
|
||||
buildObs: () => {
|
||||
this.ctx.command.set(this.input.command);
|
||||
return Object.values(this.obs.write())[0];
|
||||
return Object.values(
|
||||
this.policies.get(this.activePolicy).obs.write({ advance: false }),
|
||||
)[0];
|
||||
},
|
||||
pause: () => this.pause(),
|
||||
resume: () => this.resume(),
|
||||
@@ -377,7 +447,9 @@ export class RobotRuntime {
|
||||
this.controls?.dispose();
|
||||
this.worldPresentation?.dispose();
|
||||
this.rig.dispose();
|
||||
await this.controller.dispose();
|
||||
await Promise.allSettled(
|
||||
[...this.policies.values()].map((p) => p.dispose()),
|
||||
);
|
||||
this.sim.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -65,7 +65,7 @@ export class RuntimeManager {
|
||||
const entry = this.entries.find((e) => e.id === id);
|
||||
if (!entry) throw new Error(`Unknown Package ${id}`);
|
||||
const pkg = await loadPackage(entry.url),
|
||||
world = await loadWorld(resolveWorldId(pkg.config));
|
||||
world = await loadWorld(resolveWorldId(pkg.config), pkg.config);
|
||||
const factory = this.factories[pkg.config.adapter];
|
||||
if (!factory)
|
||||
throw new Error(`Unregistered adapter ${pkg.config.adapter}`);
|
||||
@@ -111,6 +111,7 @@ export class RuntimeManager {
|
||||
inputBindings: pkg.config.input.bindings,
|
||||
runtimeCapabilities: {},
|
||||
actuationLabel: "",
|
||||
policyStatus: "",
|
||||
fallen: false,
|
||||
...candidate.status?.(),
|
||||
});
|
||||
|
||||
@@ -1,6 +1,17 @@
|
||||
export class JointPdActuation {
|
||||
constructor(sim, targets, { kp, kd, torque_limit }) {
|
||||
constructor(sim, targets, { kp, kd, torque_limit, joints = {} }) {
|
||||
Object.assign(this, { sim, targets, kp, kd, limit: torque_limit });
|
||||
this.parameters = new Map(
|
||||
[...sim.joints.keys()].map((name) => [
|
||||
name,
|
||||
{
|
||||
kp,
|
||||
kd,
|
||||
torque_limit,
|
||||
...joints[name],
|
||||
},
|
||||
]),
|
||||
);
|
||||
this.pending = new Float64Array(sim.joints.size);
|
||||
this.clipped = new Uint8Array(sim.joints.size);
|
||||
this.streaks = new Uint32Array(sim.joints.size);
|
||||
@@ -22,11 +33,25 @@ export class JointPdActuation {
|
||||
const target = this.targets.get(name);
|
||||
if (!Number.isFinite(target))
|
||||
throw new Error(`actuation.${name}: non-finite target`);
|
||||
const torque = this.kp * (target - q[j.qpos]) - this.kd * v[j.dof];
|
||||
const p = this.parameters.get(name);
|
||||
const torque = p.kp * (target - q[j.qpos]) - p.kd * v[j.dof];
|
||||
if (!Number.isFinite(torque))
|
||||
throw new Error(`actuation.${name}: non-finite torque`);
|
||||
this.clipped[i] = Math.abs(torque) > this.limit ? 1 : 0;
|
||||
this.pending[i++] = Math.max(-this.limit, Math.min(this.limit, torque));
|
||||
let lower = -p.torque_limit,
|
||||
upper = p.torque_limit;
|
||||
if (p.velocity_limit) {
|
||||
const saturation =
|
||||
p.torque_limit / (1 - p.keep_velocity / p.velocity_limit);
|
||||
const corner = p.velocity_limit * (1 + p.torque_limit / saturation);
|
||||
const velocity = Math.max(-corner, Math.min(corner, v[j.dof]));
|
||||
lower = Math.max(
|
||||
saturation * (-1 - velocity / p.velocity_limit),
|
||||
lower,
|
||||
);
|
||||
upper = Math.min(saturation * (1 - velocity / p.velocity_limit), upper);
|
||||
}
|
||||
this.clipped[i] = torque < lower || torque > upper ? 1 : 0;
|
||||
this.pending[i++] = Math.max(lower, Math.min(upper, torque));
|
||||
}
|
||||
i = 0;
|
||||
for (const j of this.sim.joints.values()) {
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
// Full-body policies are exclusive, never overlays. All timing is simulation time.
|
||||
export class PolicyStateMachine {
|
||||
constructor(config) {
|
||||
const primary = config.skills.find((s) => s.kind === "primary");
|
||||
this.primary = primary.id;
|
||||
this.owned = new Set(config.joints.groups[primary.owns]);
|
||||
this.episodes = new Map(
|
||||
config.skills.filter((s) => s.kind === "episodic").map((s) => [s.id, s]),
|
||||
);
|
||||
this.reset();
|
||||
}
|
||||
reset() {
|
||||
this.state = "walk";
|
||||
this.active = this.primary;
|
||||
this.pending = null;
|
||||
this.stillSince = null;
|
||||
this.from = new Map();
|
||||
this.episode = null;
|
||||
this.episodeStart = 0;
|
||||
this.start = 0;
|
||||
}
|
||||
request(id, time) {
|
||||
if (
|
||||
!this.episodes.has(id) ||
|
||||
this.episodes.get(id).available === false ||
|
||||
this.state !== "walk"
|
||||
)
|
||||
return false;
|
||||
this.pending = id;
|
||||
this.state = "settling";
|
||||
this.start = time;
|
||||
this.stillSince = null;
|
||||
return true;
|
||||
}
|
||||
transition(state, active, time, targets) {
|
||||
this.state = state;
|
||||
this.active = active;
|
||||
this.start = time;
|
||||
this.from = new Map([...targets].filter(([name]) => this.owned.has(name)));
|
||||
}
|
||||
step(time, command, speed, angularSpeed, targets) {
|
||||
const moving = command.some((v) => Math.abs(v) > 0.01);
|
||||
if (this.state === "settling") {
|
||||
if (moving || time - this.start > 5) this.reset();
|
||||
else if (speed < 0.15 && angularSpeed < 0.2) {
|
||||
this.stillSince ??= time;
|
||||
if (time - this.stillSince >= 0.4) {
|
||||
this.episode = this.episodes.get(this.pending);
|
||||
this.episodeStart = time;
|
||||
this.transition("entering", this.pending, time, targets);
|
||||
}
|
||||
} else this.stillSince = null;
|
||||
} else if (["entering", "playing"].includes(this.state)) {
|
||||
if (moving || time - this.episodeStart >= this.episode.duration - 1e-9)
|
||||
this.transition("returning", this.primary, time, targets);
|
||||
else if (
|
||||
this.state === "entering" &&
|
||||
time - this.start >= this.episode.blend
|
||||
)
|
||||
this.state = "playing";
|
||||
} else if (
|
||||
this.state === "returning" &&
|
||||
time - this.start >= this.episode.blend
|
||||
)
|
||||
this.reset();
|
||||
return this.active;
|
||||
}
|
||||
blend(time, targets) {
|
||||
if (!["entering", "returning"].includes(this.state)) return;
|
||||
const alpha = Math.min(
|
||||
1,
|
||||
Math.max(0, (time - this.start) / this.episode.blend),
|
||||
);
|
||||
for (const [name, value] of this.from)
|
||||
if (targets.has(name))
|
||||
targets.set(name, value + (targets.get(name) - value) * alpha);
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,27 @@
|
||||
export const DEFAULT_WORLD = "arena";
|
||||
export const resolveWorldId = (config) =>
|
||||
config.presentation?.world ?? DEFAULT_WORLD;
|
||||
// Authored World dimensions are Microduck-sized. Resolve once before physics
|
||||
// compilation so collision boxes and presentation share the same world units.
|
||||
export function fitWorldToRobot(world, config) {
|
||||
if (!world.scale_to_robot) return world;
|
||||
const reference = world.scale_to_robot.reference_height;
|
||||
const height = config?.dimensions?.height ?? reference;
|
||||
const scale = height / reference;
|
||||
if (!Number.isFinite(scale) || scale <= 0)
|
||||
throw new Error("World scale: invalid robot/reference height");
|
||||
const result = structuredClone(world);
|
||||
const factor = scale / (world.scale ?? 1);
|
||||
for (const key of ["half_extent", "wall_height", "wall_thickness"]) {
|
||||
result.boundary[key] *= factor;
|
||||
if (!Number.isFinite(result.boundary[key]) || result.boundary[key] <= 0)
|
||||
throw new Error(`World scale: invalid boundary.${key}`);
|
||||
}
|
||||
if (result.floor.grid) result.floor.grid.size *= factor;
|
||||
// Runtime-only metadata; robot.json/world.json cannot override this value.
|
||||
result.scale = scale;
|
||||
return result;
|
||||
}
|
||||
export function arenaWalls({ half_extent, wall_height, wall_thickness }) {
|
||||
const ht = wall_thickness / 2,
|
||||
hh = wall_height / 2,
|
||||
|
||||
@@ -139,7 +139,7 @@ export async function bootGame({
|
||||
pkg ??= await loadPackage(
|
||||
`${import.meta.env.BASE_URL}robots/microduck-legs/robot.json`,
|
||||
);
|
||||
world ??= await loadWorld(resolveWorldId(pkg.config));
|
||||
world ??= await loadWorld(resolveWorldId(pkg.config), pkg.config);
|
||||
return await boot({
|
||||
scene,
|
||||
camera,
|
||||
|
||||
@@ -24,6 +24,7 @@ export const useGame = create(
|
||||
presentationTheme: "comic",
|
||||
runtimeCapabilities: {},
|
||||
actuationLabel: '',
|
||||
policyStatus: '',
|
||||
robotLicense: "",
|
||||
fallen: false,
|
||||
// Adapter-compatible boot diagnostics (no alternate boot screen).
|
||||
|
||||
@@ -91,9 +91,8 @@ export default function ControlsHelp() {
|
||||
type={clickable ? "button" : undefined}
|
||||
disabled={
|
||||
clickable
|
||||
? row.action === "chaseToggle"
|
||||
? blocked
|
||||
: disabled
|
||||
? row.disabled ||
|
||||
(row.action === "chaseToggle" ? blocked : disabled)
|
||||
: undefined
|
||||
}
|
||||
onClick={
|
||||
|
||||
@@ -5,8 +5,16 @@ export default function RuntimeStatus() {
|
||||
paused = useGame((s) => s.paused),
|
||||
error = useGame((s) => s.runtimeError),
|
||||
fallen = useGame((s) => s.fallen),
|
||||
actuationLabel = useGame((s) => s.actuationLabel);
|
||||
if (phase === "ready" && !paused && !error && !fallen && !actuationLabel)
|
||||
actuationLabel = useGame((s) => s.actuationLabel),
|
||||
policyStatus = useGame((s) => s.policyStatus);
|
||||
if (
|
||||
phase === "ready" &&
|
||||
!paused &&
|
||||
!error &&
|
||||
!fallen &&
|
||||
!actuationLabel &&
|
||||
!policyStatus
|
||||
)
|
||||
return null;
|
||||
return (
|
||||
<Box
|
||||
@@ -47,7 +55,10 @@ export default function RuntimeStatus() {
|
||||
paused ? (
|
||||
"Paused · Esc to continue"
|
||||
) : (
|
||||
actuationLabel
|
||||
<>
|
||||
{actuationLabel}
|
||||
{policyStatus && <div>{policyStatus}</div>}
|
||||
</>
|
||||
)
|
||||
) : (
|
||||
"Loading simulation…"
|
||||
|
||||
@@ -5,6 +5,7 @@ import * as THREE from "three";
|
||||
import {
|
||||
DEFAULT_WORLD,
|
||||
resolveWorldId,
|
||||
fitWorldToRobot,
|
||||
arenaWalls,
|
||||
} from "../src/platform/worlds/arena.js";
|
||||
import {
|
||||
@@ -17,7 +18,7 @@ const world = JSON.parse(
|
||||
new URL("../public/worlds/arena/world.json", import.meta.url),
|
||||
),
|
||||
);
|
||||
test("all bundled and future Packages default to the same 50%-larger arena; explicit diagnostic overrides remain possible", async () => {
|
||||
test("all bundled and future Packages share the arena template and declare nominal height; flat diagnostics stay explicit", async () => {
|
||||
assert.equal(DEFAULT_WORLD, "arena");
|
||||
validateWorld(world);
|
||||
assert.equal(world.type, "flat");
|
||||
@@ -27,7 +28,7 @@ test("all bundled and future Packages default to the same 50%-larger arena; expl
|
||||
wall_thickness: 0.05,
|
||||
});
|
||||
assert.equal(world.boundary.half_extent / 1.5, 1.5);
|
||||
for (const id of ["microduck-legs", "microduck-rollers", "microban"]) {
|
||||
for (const id of ["microduck-legs", "microduck-rollers", "microban", "rp1"]) {
|
||||
const config = JSON.parse(
|
||||
await fs.readFile(
|
||||
new URL(`../public/robots/${id}/robot.json`, import.meta.url),
|
||||
@@ -35,6 +36,12 @@ test("all bundled and future Packages default to the same 50%-larger arena; expl
|
||||
);
|
||||
validatePackage(config);
|
||||
assert.equal(resolveWorldId(config), "arena");
|
||||
const fitted = fitWorldToRobot(world, config);
|
||||
assert.equal(fitted.scale, config.dimensions.height / 0.27);
|
||||
assert.equal(fitted.boundary.half_extent, 2.25 * fitted.scale);
|
||||
assert.equal(fitted.boundary.wall_height, 0.25 * fitted.scale);
|
||||
assert.equal(fitted.boundary.wall_thickness, 0.05 * fitted.scale);
|
||||
assert.equal(world.boundary.half_extent, 2.25); // Never mutate the template.
|
||||
if (id === "microban") {
|
||||
config.id = "future-bot";
|
||||
config.family = "future-bot";
|
||||
@@ -46,6 +53,28 @@ test("all bundled and future Packages default to the same 50%-larger arena; expl
|
||||
}
|
||||
}
|
||||
});
|
||||
test("fitting is idempotent, missing height stays at reference scale, and fixed diagnostic Worlds do not scale", () => {
|
||||
const config = { dimensions: { height: 1.32 } };
|
||||
const fitted = fitWorldToRobot(world, config);
|
||||
assert.ok(Math.abs(fitted.boundary.half_extent * 2 - 22) < 1e-10);
|
||||
assert.deepEqual(fitWorldToRobot(fitted, config), fitted);
|
||||
assert.deepEqual(fitWorldToRobot(world, {}).boundary, world.boundary);
|
||||
const fixed = structuredClone(world);
|
||||
delete fixed.scale_to_robot;
|
||||
assert.equal(fitWorldToRobot(fixed, config), fixed);
|
||||
const gridded = structuredClone(world);
|
||||
gridded.floor.grid = { size: 3, divisions: 10, color: "#ffffff" };
|
||||
assert.equal(
|
||||
fitWorldToRobot(gridded, config).floor.grid.size,
|
||||
3 * fitted.scale,
|
||||
);
|
||||
assert.equal(gridded.floor.grid.size, 3);
|
||||
for (const height of [0, -1, Infinity, NaN])
|
||||
assert.throws(
|
||||
() => fitWorldToRobot(world, { dimensions: { height } }),
|
||||
/scale/,
|
||||
);
|
||||
});
|
||||
test("physical wall inner faces match 4.5m bounds and corners overlap", () => {
|
||||
const walls = arenaWalls(world.boundary);
|
||||
assert.equal(walls.length, 4);
|
||||
@@ -93,7 +122,53 @@ test("visual walls stay at physical inner faces while the floor focus follows; a
|
||||
assert.equal(releases, 10);
|
||||
assert.ok(meshes.every((m) => m.visible));
|
||||
});
|
||||
test("invalid physical boundaries are rejected", () => {
|
||||
test("RP1 visual walls, grid spacing and fade distances scale with the same collision dimensions", () => {
|
||||
const fitted = fitWorldToRobot(world, { dimensions: { height: 1.32 } });
|
||||
const names = [
|
||||
"platform_floor",
|
||||
...arenaWalls(fitted.boundary).map((w) => w.name),
|
||||
];
|
||||
const sim = { model: { geom: (name) => ({ id: names.indexOf(name) }) } };
|
||||
const rig = { items: names.map((_, id) => ({ id, mesh: new THREE.Mesh() })) };
|
||||
const view = createArenaFloorPresentation({ sim, rig, world: fitted });
|
||||
assert.equal(view.grid.material.uniforms.uCell.value, 0.1 * fitted.scale);
|
||||
assert.equal(view.grid.material.uniforms.uSection.value, 0.6 * fitted.scale);
|
||||
assert.equal(view.grid.material.uniforms.uFadeDist.value, 3 * fitted.scale);
|
||||
assert.equal(view.grid.geometry.parameters.width, 30 * fitted.scale);
|
||||
for (const mesh of view.walls) {
|
||||
assert.equal(mesh.geometry.parameters.height, fitted.boundary.wall_height);
|
||||
assert.equal(
|
||||
mesh.geometry.parameters.width,
|
||||
2 * (fitted.boundary.half_extent + fitted.boundary.wall_thickness),
|
||||
);
|
||||
assert.equal(
|
||||
mesh.material.uniforms.uWallH.value,
|
||||
fitted.boundary.wall_height,
|
||||
);
|
||||
assert.equal(mesh.material.uniforms.uFadeDist.value, 5 * fitted.scale);
|
||||
assert.equal(mesh.material.uniforms.uCell.value, 0.1 * fitted.scale);
|
||||
}
|
||||
assert.deepEqual(view.walls[0].position.toArray(), [
|
||||
fitted.boundary.half_extent,
|
||||
fitted.boundary.wall_height / 2,
|
||||
0,
|
||||
]);
|
||||
view.dispose();
|
||||
});
|
||||
test("invalid physical boundaries and scaling declarations are rejected", () => {
|
||||
for (const scale_to_robot of [
|
||||
{ reference_height: 0 },
|
||||
{ reference_height: -1 },
|
||||
{},
|
||||
])
|
||||
assert.throws(
|
||||
() => validateWorld({ ...world, scale_to_robot }),
|
||||
/scale_to_robot/,
|
||||
);
|
||||
assert.throws(
|
||||
() => validateWorld({ ...world, boundary: undefined }),
|
||||
/boundary/,
|
||||
);
|
||||
for (const bad of [
|
||||
{ half_extent: 0 },
|
||||
{ wall_height: -1 },
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { JointMotionReference } from "../src/platform/policy/JointMotionReference.js";
|
||||
import { ObservationPipeline } from "../src/platform/policy/ObservationPipeline.js";
|
||||
import { validatePackage } from "../src/platform/loader/SchemaValidator.js";
|
||||
|
||||
const names = ["b", "a"];
|
||||
const header = {
|
||||
schema: 1,
|
||||
dtype: "float32-le",
|
||||
layout: "positions_then_velocities",
|
||||
frames: 3,
|
||||
step_dt: 0.02,
|
||||
joint_names: names,
|
||||
data: "motion.f32",
|
||||
};
|
||||
const samples = [1, 2, 3, 4, 5, 6, 11, 12, 13, 14, 15, 16];
|
||||
function bytes(values = samples) {
|
||||
const out = new ArrayBuffer(values.length * 4),
|
||||
view = new DataView(out);
|
||||
values.forEach((v, i) => view.setFloat32(i * 4, v, true));
|
||||
return out;
|
||||
}
|
||||
function reference() {
|
||||
return new JointMotionReference(header, bytes(), names, 0.02);
|
||||
}
|
||||
function read(ref) {
|
||||
const out = new Float32Array(4);
|
||||
ref.write(out, 0);
|
||||
return [...out];
|
||||
}
|
||||
|
||||
test("joint reference preserves command order, is simulation-clocked, clamps without wrapping and resets", () => {
|
||||
const ref = reference();
|
||||
ref.sample(20);
|
||||
assert.deepEqual(read(ref), [1, 2, 11, 12]);
|
||||
ref.sample(20.02);
|
||||
assert.deepEqual(read(ref), [3, 4, 13, 14]);
|
||||
ref.sample(20.02);
|
||||
assert.equal(ref.frame, 1);
|
||||
ref.sample(99);
|
||||
assert.deepEqual(read(ref), [5, 6, 15, 16]);
|
||||
ref.reset();
|
||||
ref.sample(200);
|
||||
assert.deepEqual(read(ref), [1, 2, 11, 12]);
|
||||
assert.throws(() => ref.sample(NaN), /time/);
|
||||
});
|
||||
|
||||
test("reference layout, clock, joint order, finiteness and exact binary length fail closed", () => {
|
||||
for (const spec of [
|
||||
{ ...header, dtype: "float64" },
|
||||
{ ...header, frames: 0 },
|
||||
{ ...header, step_dt: 0.01 },
|
||||
{ ...header, joint_names: ["a", "b"] },
|
||||
{ ...header, layout: "unknown" },
|
||||
])
|
||||
assert.throws(
|
||||
() => new JointMotionReference(spec, bytes(), names, 0.02),
|
||||
/Motion reference/,
|
||||
);
|
||||
assert.throws(
|
||||
() => new JointMotionReference(header, bytes().slice(0, 8), names, 0.02),
|
||||
/sample count/,
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
new JointMotionReference(
|
||||
header,
|
||||
bytes(samples.map((v, i) => (i ? v : NaN))),
|
||||
names,
|
||||
0.02,
|
||||
),
|
||||
/non-finite/,
|
||||
);
|
||||
});
|
||||
|
||||
test("motion observations are reference inputs only; diagnostic reads do not advance or write physics/targets", () => {
|
||||
const ref = reference(),
|
||||
targets = new Map([["b", 100]]);
|
||||
const ctx = {
|
||||
reference: ref,
|
||||
targets,
|
||||
config: { joints: { groups: { policy: names } } },
|
||||
};
|
||||
const obs = new ObservationPipeline(
|
||||
{ obs: { terms: [{ type: "motion_command", group: "policy" }] } },
|
||||
ctx,
|
||||
);
|
||||
ref.sample(5);
|
||||
ref.sample(5.02);
|
||||
assert.deepEqual([...obs.write().obs], [3, 4, 13, 14]);
|
||||
assert.deepEqual([...obs.write({ advance: false }).obs], [3, 4, 13, 14]);
|
||||
assert.equal(ref.frame, 1);
|
||||
assert.equal(targets.get("b"), 100);
|
||||
assert.throws(
|
||||
() =>
|
||||
new ObservationPipeline(
|
||||
{ obs: { terms: [{ type: "motion_command", group: "policy" }] } },
|
||||
{ ...ctx, reference: null },
|
||||
),
|
||||
/reference/,
|
||||
);
|
||||
});
|
||||
|
||||
test("shipped RP1 motion contracts use actual 126D student terms and per-skill pose/clock", () => {
|
||||
const config = JSON.parse(
|
||||
readFileSync(new URL("../public/robots/rp1/robot.json", import.meta.url)),
|
||||
);
|
||||
validatePackage(config);
|
||||
for (const skill of config.skills.filter((s) => s.kind === "episodic")) {
|
||||
assert.equal(
|
||||
config.input.bindings.find((b) => b.action === `skill:${skill.id}`).label,
|
||||
skill.id === "dance" ? "Dance" : "Wave",
|
||||
);
|
||||
assert.deepEqual(
|
||||
skill.observations.obs.terms.map((t) => t.type),
|
||||
[
|
||||
"motion_command",
|
||||
"gyro",
|
||||
"projected_gravity",
|
||||
"joint_pos_rel",
|
||||
"joint_vel",
|
||||
"last_action",
|
||||
],
|
||||
);
|
||||
assert.equal(skill.actions[0].scale, 0.25);
|
||||
assert.equal(Object.keys(skill.defaults).length, 24);
|
||||
const base = new URL("../public/robots/rp1/", import.meta.url);
|
||||
const spec = JSON.parse(readFileSync(new URL(skill.reference, base)));
|
||||
const buffer = readFileSync(new URL(spec.data, base));
|
||||
const ref = new JointMotionReference(
|
||||
spec,
|
||||
buffer.buffer.slice(
|
||||
buffer.byteOffset,
|
||||
buffer.byteOffset + buffer.byteLength,
|
||||
),
|
||||
config.joints.groups.policy,
|
||||
0.02,
|
||||
);
|
||||
assert.ok(skill.duration <= ref.frames * ref.dt);
|
||||
const invalid = structuredClone(config);
|
||||
delete invalid.skills.find((s) => s.id === skill.id).reference;
|
||||
assert.throws(() => validatePackage(invalid), /reference/);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,275 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { validatePackage } from "../src/platform/loader/SchemaValidator.js";
|
||||
import { ObservationPipeline } from "../src/platform/policy/ObservationPipeline.js";
|
||||
import { TermRegistry } from "../src/platform/policy/TermRegistry.js";
|
||||
import { PolicyStateMachine } from "../src/platform/skills/PolicyStateMachine.js";
|
||||
import { JointPdActuation } from "../src/platform/sim/actuation/JointPdActuation.js";
|
||||
import { InputRouter } from "../src/platform/input/InputRouter.js";
|
||||
import { OnnxPolicy } from "../src/platform/policy/controllers/OnnxPolicy.js";
|
||||
const config = JSON.parse(
|
||||
readFileSync(new URL("../public/robots/rp1/robot.json", import.meta.url)),
|
||||
);
|
||||
|
||||
test("RP1 contract has 24 named actions, 243 term-major observations and fail-closed availability gates", () => {
|
||||
validatePackage(config);
|
||||
const names = config.joints.groups.policy;
|
||||
assert.equal(names.length, 24);
|
||||
assert.equal(config.actions[0].scale, 0.25);
|
||||
assert.equal(config.sim.timestep * config.sim.decimation, 0.02);
|
||||
assert.equal(config.observations.obs.history, 3);
|
||||
assert.equal(config.observations.obs.layout, "term_major");
|
||||
assert.deepEqual(
|
||||
config.observations.obs.terms.map((t) => t.type),
|
||||
[
|
||||
"gyro",
|
||||
"projected_gravity",
|
||||
"command",
|
||||
"joint_pos_rel",
|
||||
"joint_vel",
|
||||
"last_action",
|
||||
],
|
||||
);
|
||||
assert.equal(
|
||||
names.filter((n) => config.actuation.joints[n].kp === 120).length,
|
||||
8,
|
||||
);
|
||||
const invalid = structuredClone(config);
|
||||
invalid.skills[1].available = true;
|
||||
delete invalid.skills[1].controller.policy;
|
||||
invalid.input.bindings.find((b) => b.action === "skill:dance").disabled =
|
||||
true;
|
||||
assert.throws(() => validatePackage(invalid), /policy/);
|
||||
invalid.skills[1].controller.policy = "policies/dance.onnx";
|
||||
assert.throws(() => validatePackage(invalid), /disabled/);
|
||||
invalid.input.bindings.find((b) => b.action === "skill:dance").disabled =
|
||||
false;
|
||||
validatePackage(invalid);
|
||||
invalid.skills[1].owns = "missing";
|
||||
assert.throws(() => validatePackage(invalid), /group/);
|
||||
});
|
||||
|
||||
test("history is oldest→newest within each term, backfills first sample, debug doesn't advance, reset clears", () => {
|
||||
const registry = new TermRegistry([
|
||||
[
|
||||
"value",
|
||||
{
|
||||
size: (t) => t.size,
|
||||
write: (out, offset, ctx, op) =>
|
||||
out.set(ctx.value.slice(0, op.term.size), offset),
|
||||
},
|
||||
],
|
||||
]);
|
||||
const ctx = { value: [1, 2] };
|
||||
const specs = {
|
||||
obs: {
|
||||
history: 3,
|
||||
layout: "term_major",
|
||||
terms: [
|
||||
{ type: "value", size: 2 },
|
||||
{ type: "value", size: 1 },
|
||||
],
|
||||
},
|
||||
};
|
||||
const pipeline = new ObservationPipeline(specs, ctx, { registry });
|
||||
assert.deepEqual([...pipeline.write().obs], [1, 2, 1, 2, 1, 2, 1, 1, 1]);
|
||||
ctx.value = [3, 4];
|
||||
assert.deepEqual([...pipeline.write().obs], [1, 2, 1, 2, 3, 4, 1, 1, 3]);
|
||||
ctx.value = [5, 6];
|
||||
assert.deepEqual(
|
||||
[...pipeline.write({ advance: false }).obs],
|
||||
[1, 2, 1, 2, 5, 6, 1, 1, 5],
|
||||
);
|
||||
assert.deepEqual([...pipeline.write().obs], [1, 2, 3, 4, 5, 6, 1, 3, 5]);
|
||||
pipeline.reset();
|
||||
pipeline.write({ advance: false });
|
||||
ctx.value = [9, 10];
|
||||
assert.deepEqual([...pipeline.write().obs], [9, 10, 9, 10, 9, 10, 9, 9, 9]);
|
||||
ctx.value = [5, 6];
|
||||
specs.obs.layout = "frame_major";
|
||||
const frame = new ObservationPipeline(specs, ctx, { registry });
|
||||
frame.write();
|
||||
ctx.value = [7, 8];
|
||||
assert.deepEqual([...frame.write().obs], [5, 6, 5, 5, 6, 5, 7, 8, 7]);
|
||||
});
|
||||
|
||||
function stateMachine() {
|
||||
const c = structuredClone(config);
|
||||
c.skills[1].available = true;
|
||||
c.skills[1].duration = 10;
|
||||
c.skills[1].blend = 0.5;
|
||||
c.skills[2].available = false;
|
||||
c.joints.groups.policy = ["joint"];
|
||||
return new PolicyStateMachine(c);
|
||||
}
|
||||
test("exclusive full-body FSM waits for actual stop, blends, times out and returns to walk", () => {
|
||||
const fsm = stateMachine(),
|
||||
targets = new Map([["joint", 1]]);
|
||||
assert.equal(fsm.request("wave", 0), false);
|
||||
assert.equal(fsm.request("dance", 0), true);
|
||||
assert.equal(fsm.request("dance", 0), false);
|
||||
fsm.step(0.5, [0, 0, 0], 0.8, 0, targets);
|
||||
assert.equal(fsm.state, "settling");
|
||||
fsm.step(1, [0, 0, 0], 0, 0, targets);
|
||||
assert.equal(fsm.step(1.5, [0, 0, 0], 0, 0, targets), "dance");
|
||||
targets.set("joint", 3);
|
||||
fsm.blend(1.75, targets);
|
||||
assert.equal(targets.get("joint"), 2);
|
||||
fsm.step(2.1, [0, 0, 0], 0, 0, targets);
|
||||
assert.equal(fsm.state, "playing");
|
||||
assert.equal(fsm.step(11.6, [0, 0, 0], 0, 0, targets), "walk");
|
||||
assert.equal(fsm.state, "returning");
|
||||
fsm.step(12.2, [0, 0, 0], 0, 0, targets);
|
||||
assert.equal(fsm.state, "walk");
|
||||
fsm.request("dance", 13);
|
||||
fsm.step(19, [0, 0, 0], 1, 0, targets);
|
||||
assert.equal(fsm.state, "walk");
|
||||
});
|
||||
test("episode blending never overwrites independent overlay joint targets", () => {
|
||||
const fsm = stateMachine(),
|
||||
targets = new Map([
|
||||
["joint", 1],
|
||||
["head", 2],
|
||||
]);
|
||||
fsm.request("dance", 0);
|
||||
fsm.step(0.1, [0, 0, 0], 0, 0, targets);
|
||||
fsm.step(0.6, [0, 0, 0], 0, 0, targets);
|
||||
targets.set("joint", 3);
|
||||
targets.set("head", 7);
|
||||
fsm.blend(0.85, targets);
|
||||
assert.equal(targets.get("joint"), 2);
|
||||
assert.equal(targets.get("head"), 7);
|
||||
});
|
||||
test("movement cancels pending skills and interrupts active skills with a return blend; reset clears", () => {
|
||||
const fsm = stateMachine(),
|
||||
targets = new Map();
|
||||
fsm.request("dance", 0);
|
||||
fsm.step(0.1, [0.4, 0, 0], 0, 0, targets);
|
||||
assert.equal(fsm.state, "walk");
|
||||
fsm.request("dance", 1);
|
||||
fsm.step(1.1, [0, 0, 0], 0, 0, targets);
|
||||
fsm.step(1.6, [0, 0, 0], 0, 0, targets);
|
||||
assert.equal(fsm.active, "dance");
|
||||
fsm.step(1.7, [0, 0, 0.6], 0, 0, targets);
|
||||
assert.equal(fsm.state, "returning");
|
||||
assert.equal(fsm.active, "walk");
|
||||
fsm.reset();
|
||||
assert.equal(fsm.state, "walk");
|
||||
assert.equal(fsm.pending, null);
|
||||
});
|
||||
test("named per-joint gains and four-quadrant torque-speed envelope apply atomically", () => {
|
||||
const sim = {
|
||||
joints: new Map([
|
||||
["a", { qpos: 0, dof: 0, actuator: 1 }],
|
||||
["b", { qpos: 1, dof: 1, actuator: 0 }],
|
||||
]),
|
||||
data: { qpos: [0, 0], qvel: [10, -10], ctrl: new Float64Array(2) },
|
||||
};
|
||||
const targets = new Map([
|
||||
["a", 2],
|
||||
["b", -2],
|
||||
]);
|
||||
const pd = new JointPdActuation(sim, targets, {
|
||||
kp: 1,
|
||||
kd: 0,
|
||||
torque_limit: 10,
|
||||
joints: {
|
||||
a: { kp: 20, velocity_limit: 10, keep_velocity: 5 },
|
||||
b: { kp: 20, velocity_limit: 10, keep_velocity: 5 },
|
||||
},
|
||||
});
|
||||
pd.apply();
|
||||
assert.deepEqual([...sim.data.ctrl], [0, 0]);
|
||||
sim.data.qvel = [-10, 10];
|
||||
pd.apply();
|
||||
assert.deepEqual([...sim.data.ctrl], [-10, 10]);
|
||||
targets.set("b", NaN);
|
||||
assert.throws(() => pd.apply(), /non-finite/);
|
||||
assert.deepEqual([...sim.data.ctrl], [-10, 10]);
|
||||
});
|
||||
test("independent skills own histories/defaults and share only targets; failed smoke releases session", async () => {
|
||||
let released = 0;
|
||||
const session = {
|
||||
inputNames: ["obs"],
|
||||
outputNames: ["actions"],
|
||||
inputMetadata: [
|
||||
{ name: "obs", isTensor: true, type: "float32", shape: [1, 24] },
|
||||
],
|
||||
outputMetadata: [
|
||||
{ name: "actions", isTensor: true, type: "float32", shape: [1, 24] },
|
||||
],
|
||||
run: async () => ({ actions: { data: new Float32Array(24).fill(1) } }),
|
||||
release: async () => released++,
|
||||
};
|
||||
const ort = {
|
||||
Tensor: class {
|
||||
constructor(type, data) {
|
||||
this.data = data;
|
||||
}
|
||||
},
|
||||
InferenceSession: { create: async () => session },
|
||||
};
|
||||
const names = config.joints.groups.policy;
|
||||
const shared = {
|
||||
config,
|
||||
policyNames: names,
|
||||
defaults: new Map(names.map((n) => [n, 0])),
|
||||
command: new Float32Array(3),
|
||||
targets: new Map(),
|
||||
sim: {
|
||||
joints: new Map(names.map((n, i) => [n, { qpos: i }])),
|
||||
data: { qpos: new Float64Array(24) },
|
||||
},
|
||||
};
|
||||
const skill = {
|
||||
id: "fixture",
|
||||
defaults: Object.fromEntries(names.map((n) => [n, 0.5])),
|
||||
observations: {
|
||||
obs: { terms: [{ type: "joint_pos_rel", group: "policy" }] },
|
||||
},
|
||||
};
|
||||
const policy = await OnnxPolicy.load(
|
||||
ort,
|
||||
new Uint8Array(),
|
||||
config,
|
||||
shared,
|
||||
skill,
|
||||
);
|
||||
assert.equal(policy.ctx.targets, shared.targets);
|
||||
assert.equal(shared.defaults.get(names[0]), 0);
|
||||
assert.equal(policy.obs.write().obs[0], -0.5);
|
||||
await policy.run();
|
||||
assert.equal(shared.targets.get(names[0]), 0.75);
|
||||
policy.reset();
|
||||
assert.equal(policy.ctx.lastAction[0], 0);
|
||||
await policy.dispose();
|
||||
session.run = async () => ({
|
||||
actions: { data: new Float32Array(24).fill(NaN) },
|
||||
});
|
||||
await assert.rejects(
|
||||
() => OnnxPolicy.load(ort, new Uint8Array(), config, shared, skill),
|
||||
/smoke/,
|
||||
);
|
||||
assert.equal(released, 2);
|
||||
});
|
||||
test("unavailable-skill keyboard bindings never dispatch", () => {
|
||||
const router = new InputRouter({
|
||||
bindings: config.input.bindings.map((b) => ({
|
||||
...b,
|
||||
disabled: b.action.startsWith("skill:") ? true : b.disabled,
|
||||
})),
|
||||
getVelocityLimits: () => config.commands.limits,
|
||||
});
|
||||
const target = new EventTarget(),
|
||||
calls = [];
|
||||
router.onAction = (action) => calls.push(action);
|
||||
router.init(target);
|
||||
for (const code of ["KeyJ", "KeyH", "Space"]) {
|
||||
const e = new Event("keydown");
|
||||
Object.assign(e, { code });
|
||||
target.dispatchEvent(e);
|
||||
}
|
||||
assert.deepEqual(calls, ["reset"]);
|
||||
router.dispose();
|
||||
});
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## 首期功能
|
||||
|
||||
- 在工具栏 Robot 中选择 Microban;默认使用平台统一 `arena`:内部 4.5 × 4.5 m(原场地长宽各扩大 50%),墙高 .25 m、厚 .05 m,四边为真实碰撞盒。
|
||||
- 在工具栏 Robot 中选择 Microban;默认使用平台统一 `arena` 模板,按名义站立高度 0.33 m 相对 Microduck 0.27 m 等比适配:内部 **5.5 × 5.5 m**,墙高约 **0.306 m**、厚约 **0.061 m**,四边为真实碰撞盒。
|
||||
- 内部仍是平地,无高度场;复用 Microduck 的暗色橙色网格、围墙视觉、comic/CRT 和背后相机。重力、驱动、policy 参数不变;不是用坐标夹紧伪造边界。
|
||||
- 有界默认场地的运动验收为三轮各 70 s;另用显式 `flat` 诊断场地保留三轮各 84 s 原长距离协议,二者不能混称。
|
||||
- 导入/切换后默认在机器人背后跟随;鼠标拖拽切换到自由环绕,按 **C** 恢复背后跟随和默认距离。C 在暂停时也有效,重复按不会关闭跟随。
|
||||
@@ -62,4 +62,4 @@ tau = clamp(0.3 * (target - q) - 0.02 * dq, -0.65, 0.65)
|
||||
|
||||
`microban-walking.spec.js` 从重置状态完成三轮,每轮 84 s 仿真:站立、前后走、左右转、停顿、行走中转头、关闭转头。检查实际位移方向、展开的 yaw、head 幅度 / 回正、无跌倒 / 重置,并记录 PD 饱和统计。另有真实墙钟下近 50 Hz 测试、H 按键、跌倒手动 reset 与 IMU 坐标系测试。
|
||||
|
||||
详见 [验收记录](../superpowers/verification/2026-10-27-multi-robot-platform.md)。只对记录的桌面 Chromium / WASM 组合做过验证;没有实机、移动端、手柄 / 触屏、15 小时耐久或不同浏览器的保证。模型和策略许可按原文件与来源说明分别核对。
|
||||
初始策略接入详见 [验收记录](../superpowers/verification/2026-10-27-multi-robot-platform.md);当前场地尺寸与碰撞验证见 [按机器人尺寸缩放场地](../superpowers/verification/2026-10-28-robot-scaled-arena.md)。只对记录的桌面 Chromium / WASM 组合做过验证;没有实机、移动端、手柄 / 触屏、15 小时耐久或不同浏览器的保证。模型和策略许可按原文件与来源说明分别核对。
|
||||
|
||||
@@ -8,14 +8,15 @@ Robot 是数据包,平台核心不根据 robot id 分支。对于现有能力
|
||||
|
||||
- floating base、具名 freejoint / root body;受控关节为标量 hinge / slide。
|
||||
- 每个受控关节一个 unit-gear actuator,独立解析 joint / qpos / dof / actuator 地址,不依赖其数组顺序。
|
||||
- ONNX primary;不相交关节组的正弦 overlay。
|
||||
- gyro、body projected gravity、joint position relative、joint velocity、lastAction、command。
|
||||
- 关节位置目标;unit-gain torque motor 的子步 PD,或原生 position servo。
|
||||
- MJCF 编译后 visual geom / mesh / material 渲染;默认 `arena` 为 4.5 × 4.5 m 有碰撞围墙的平地,使用 Microduck 暗色/橙色地面和墙网格;同款 comic UI 与背后相机,C 恢复。
|
||||
- ONNX primary;不相交关节组的正弦 overlay;与 primary 独占相同关节组的 ONNX episodic 技能。
|
||||
- gyro、body projected gravity、joint position relative、joint velocity、lastAction、command;可选 1–64 帧 frame-major / term-major 历史,首次采样回填,reset 清空。
|
||||
- episodic 可用 `motion_command` 读取仿真时钟驱动的 joint position/velocity 参考。每技能独立进度,pause 冻结、激活从帧 0 重启;参考只进入观测,不运动学改写机器人。
|
||||
- 关节位置目标;unit-gain torque motor 的子步 PD,或原生 position servo。PD 可配置每关节增益/力矩上限与可选四象限 torque-speed envelope。
|
||||
- MJCF 编译后 visual geom / mesh / material 渲染;默认 `arena` 是按机器人站立高度缩放的有碰撞围墙平地。Microduck 基准为 4.5 × 4.5 m,使用暗色/橙色地面和墙网格;同款 comic UI 与背后相机,C 恢复。
|
||||
- `presentation.world` 可省略,统一由平台 `DEFAULT_WORLD` 解析,不要在每个新包里复制默认场地名。需要诊断时,通用包可显式设为 `"flat"`(无墙)。机器人出生点和尺寸应适合默认边界。
|
||||
- Package 绑定驱动的键盘输入与同源提示 / 技能按钮。
|
||||
|
||||
固定基座、ball joint 的多维 action、非单位 gear、自定义外部 JS、RNN、历史 / 图像张量、IK、velocity / effort action 等尚不支持。JSON Schema 与运行时都会拒绝不支持的能力,不能仅添加未实现的配置名来声称已经兼容。
|
||||
固定基座、ball joint 的多维 action、非单位 gear、自定义外部 JS、RNN、图像张量、BeyondMimic 的 privileged motion_preview/anchor/base-linear-velocity terms、IK、velocity / effort action 等尚不支持。JSON Schema 与运行时都会拒绝不支持的能力,不能仅添加未实现的配置名来声称已经兼容。
|
||||
|
||||
## 文件布局
|
||||
|
||||
@@ -25,6 +26,9 @@ app/public/robots/example/
|
||||
model.xml
|
||||
assets/...
|
||||
policies/walk.onnx
|
||||
policies/wave.onnx # 可选独立 episodic
|
||||
motions/wave.json # 可选 reference 描述
|
||||
motions/wave.f32 # positions_then_velocities / float32-le
|
||||
SOURCE.md
|
||||
LICENSES/...
|
||||
```
|
||||
@@ -37,6 +41,18 @@ app/public/robots/example/
|
||||
- `app/src/platform/loader/world.schema.json`
|
||||
- `SchemaValidator.js` 的跨字段检查(所有权、输入绑定、命令尺寸等)。
|
||||
|
||||
## 场地与机器人尺寸
|
||||
|
||||
在 Robot Package 顶层声明参考站姿的全身高度(米),例如:
|
||||
|
||||
```json
|
||||
"dimensions": { "height": 1.32 }
|
||||
```
|
||||
|
||||
默认 arena 的 `scale_to_robot.reference_height` 为 Microduck 的 `0.27 m`,场地长宽、墙高、墙厚、网格间距与淡出距离均按 `height / 0.27` 同比缩放。Microduck 保持 4.5 m 方形场地 / 0.25 m 墙高;Microban(0.33 m)为 5.5 m / 0.306 m;RP1(1.32 m)为 22 m / 1.222 m。物理碰撞和视觉墙使用同一份解析后 World 尺寸。
|
||||
|
||||
高度是稳定的名义尺寸,不是 base.spawn.z,不会随蹲下、跌倒或挥手动态改变;Microduck 腿/轮共用同一参考值。省略 dimensions 的旧包按基准倍率 1 兼容,新增机器人应明确填写。自定义 World 不声明 `scale_to_robot` 就保持固定米制尺寸;`flat` 诊断场地不受影响。解析后的 `world.scale` 是运行时派生值,不应写进 world.json。
|
||||
|
||||
## 必须逐项核对
|
||||
|
||||
1. **joint group**:`joints.groups.policy` 是 actor 的真实顺序,不是 MJCF 或 metadata 全体顺序;overlay 另列 group。
|
||||
@@ -46,7 +62,7 @@ app/public/robots/example/
|
||||
5. **actuator**:不能把关节角写进 torque motor。PD 增益 / 力矩上限由配置显式指定,不能自动等价于任意训练 metadata。
|
||||
6. **资源路径**:以 manifest 为基准解析;不得依赖源项目绝对路径。内置 asset URL 必须同源,保留 Vite base 与已有同源签名参数。
|
||||
7. **终止**:配置合理的倾角 / 高度;没有恢复策略时使用手动 reset,不用自动重置掩盖跌倒。
|
||||
8. **输入**:语义动作支持 `move_forward` / `move_backward` / `turn_left` / `turn_right`、reset、pause、`toggle:<overlay>`;提示由绑定生成。
|
||||
8. **输入**:语义动作支持 `move_forward` / `move_backward` / `turn_left` / `turn_right`、reset、pause、`toggle:<overlay>`、`skill:<episodic>`;提示由绑定生成。待权重的 episodic 必须设置 `available:false`,对应绑定 `disabled:true`,不加载虚构的权重路径。
|
||||
9. **许可**:填写 license,保存来源、修改与逐文件声明。复制权限不等于所有文件许可一致。
|
||||
|
||||
## 注册与测试
|
||||
@@ -73,4 +89,6 @@ npm --prefix app run test:e2e
|
||||
|
||||
确需新能力时,在独立阶段明确数值约定、schema、注册项、单元与浏览器测试。观测 / 动作注册表在 `platform/policy/TermRegistry.js`;可信特殊生命周期放在 `src/robots/` 的编译期注册工厂。首期没有 Worker 沙箱或任意 Package URL 的 JS 执行入口。
|
||||
|
||||
World 与 Robot 配置解耦:现有三个包与不指定 world 的新通用包均默认 `arena`。Microduck 专用适配器保留其球/可选起伏等特性,通用机器人只共享平地、边界和场地视觉;这不等于任意新机器人都已获得策略兼容或所有 Microduck 玩法。
|
||||
新增完整的历史观测、独立策略默认姿态与 episodic 状态机约定见 [RP1 接入说明](rp1.md)。每个技能可覆盖 observations/actions/defaults,并可声明 reference;`motion_command` 的 group 必须与技能 owns 一致,reference 的关节顺序/控制时钟/精确数据长度均验证。未知 observation term、RNN 或任意新 ONNX 输入结构仍需显式实现与验证。
|
||||
|
||||
World 与 Robot 配置解耦:所有现有包与不指定 world 的新通用包均默认 `arena`。Microduck 专用适配器保留其球/可选起伏等特性,通用机器人只共享平地、边界和场地视觉;这不等于任意新机器人都已获得策略兼容或所有 Microduck 玩法。
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# RP1:第三个机器人与独立多技能策略
|
||||
|
||||
## 当前可用
|
||||
|
||||
```bash
|
||||
npm --prefix app run dev -- --host 127.0.0.1
|
||||
```
|
||||
|
||||
在 Robot 下拉框选择 **RP1**,所有物理与推理都在浏览器,不需要训练后端:
|
||||
|
||||
- W/S 或 ↑/↓:前进/后退,命令 `+0.4 / -0.3 m/s`。
|
||||
- A/D 或 ←/→:左右转,命令角速度 `±0.6 rad/s`。
|
||||
- **J:舞蹈前 10 秒片段;H:完整 4.36 秒挥手**。也可点击按键面板的对应按钮。先释放移动键,再请求动作。
|
||||
- 动作完成自动混回走路;方向键可提前中断。Space 重置;Esc 暂停/继续;C 恢复跟随相机。
|
||||
- 暂停时动作按钮禁用;参考帧和技能计时也一起冻结。
|
||||
- 场地按名义站立高度 1.32 m 相对 Microduck 0.27 m 缩放:22 × 22 m,墙高约 1.222 m、厚约 0.244 m。网格/淡出距离同步缩放,真实碰撞墙与视觉一致。
|
||||
|
||||
右侧动作按钮仅显示英文 **Dance**、**Wave**,不显示括号补充文字。
|
||||
|
||||
**限制:完整舞蹈参考长 131.46 秒;旧版 model_7999 的长程测试在参考帧 994(约 19.9 秒)失稳。新版 model_23998 目前只验证并开放前 10 秒,完整时长尚未重新验收,不自动延长 duration。** 训练完成不等于任意长度播放已收敛;不能仅延长 duration 就宣称稳定。
|
||||
|
||||
## 已完成权重
|
||||
|
||||
均来自 `/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl`,显式选择最终、完整且稳定的文件,不动态追踪目录中的最新文件:
|
||||
|
||||
| 技能 | Run / checkpoint | 训练迭代 | 部署时长 |
|
||||
|---|---|---:|---:|
|
||||
| walk | `rp1_24dof_flat/2026-10-28_15-47-49/model_9000.pt` | 9000 / 配置 9001 次 | 连续 |
|
||||
| dance | `rp1_24dof_beyondmimic_dance/2026-10-29_11-05-14_dance_resume/model_23998.pt` | 从 7999 续训 16000 次,最终 23998 | 前 10 s |
|
||||
| wave | `rp1_24dof_beyondmimic_wave/2026-10-28_19-00-23_wave/model_7999.pt` | 7999 / 配置 8000 次 | 全 4.36 s |
|
||||
|
||||
包在 `app/public/robots/rp1/`;`source-manifest.json` 保存 checkpoint、配置、motion、ONNX 与资源哈希。导入不会启动、停止或修改训练。
|
||||
|
||||
## 数值合同
|
||||
|
||||
共同约定:floating base `base_link`,24 个具名关节;物理步长 0.005 s、decimation 4、控制 50 Hz;`default_joint_pos + 0.25 * action`;每物理子步应用具名 PD、力矩上限和四象限 torque-speed envelope。
|
||||
|
||||
### 行走
|
||||
|
||||
- float32 `[1,243] → [1,24]`,ELU MLP `243→512→256→128→24`。
|
||||
- 单帧:gyro 3、projected gravity 3、velocity command 3、relative joint position 24、joint velocity 24、last action 24。
|
||||
- 3 帧 term-major,每个 term 内 oldest→newest;首次采样回填,reset 清空历史。
|
||||
- 包含经验归一化 `(obs - mean) / (std + 0.01)` 和 runner 的 ±100 动作裁剪。
|
||||
|
||||
### BeyondMimic 舞蹈/挥手:实际是 student 分支
|
||||
|
||||
必须读取日志的 `agent.yaml: obs_groups.actor`,本次是 **`[student]`**,不是带 motion_preview/anchor 的 `policy` 观测组。
|
||||
|
||||
- float32 `[1,126] → [1,24]`,ELU MLP `126→512→256→128→24`,内含经验归一化;runner 未裁剪动作,导出器也不增加裁剪。
|
||||
- 顺序:`motion_command` 48(参考 joint_pos 24,然后 joint_vel 24)、root-frame gyro 3、projected gravity 3、相对该技能 defaults 的 joint_pos 24、joint_vel 24、last action 24。无历史。
|
||||
- actor、动作和 motion 都按日志 `joint_ordering` / action 的 `preserve_order` 具名顺序解析,不按 MJCF actuator 索引猜测。
|
||||
- `skills[].defaults` 保留各自训练的参考姿态;不会用行走策略的参考姿态代替。
|
||||
- 这两个 student actor **不需要** anchor/reference quaternion 或 motion_preview 输入。带这些输入的其他 BeyondMimic 导出仍不支持;未知 terms 和不同 actor branch 会明确拒绝,而不是伪造兼容。
|
||||
|
||||
浏览器省略训练时的随机延迟(0–2 physics steps)、域随机化、随机 default positions、外推力及观测噪声。它不是 Newton/MJWarp 或实机电机的精确等价物。跌倒停止推理、请求手动 reset,不自动起身。
|
||||
|
||||
## 统一生命周期与参考数据
|
||||
|
||||
`platform/policy/controllers/OnnxPolicy.js` 对每个技能独立拥有 session、观测历史、lastAction、defaults 和 motion reference,仅共享物理状态与最终 target map:
|
||||
|
||||
- `load()`:编译自己的观测/动作管线、检查真实 dtype/shape、执行有限值 smoke test。
|
||||
- `reset()`:清空该策略自己的历史/lastAction,参考时钟回到未启动状态。
|
||||
- `run()`:按仿真时间采样参考,构建观测、推理、原子提交关节目标。
|
||||
- `dispose()`:释放 session;部分加载失败释放已创建资源,切换失败回滚到旧机器人。
|
||||
|
||||
`JointMotionReference` **只写 observation,不写 qpos、速度或关节 targets**。不是运动学动画,也不在动作开始/结束传送 root。每次真正激活策略时从帧 0 开始,50 Hz 一帧,暂停不推进;末帧 clamp,不 wrap、不随机重采样、不复用上次进度。
|
||||
|
||||
`skills[].reference` 是 Package 相对路径,例如 `motions/wave.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"layout": "positions_then_velocities",
|
||||
"dtype": "float32-le",
|
||||
"step_dt": 0.02,
|
||||
"frames": 218,
|
||||
"joint_names": ["实际 24 个具名关节,顺序须与 owns 相同"],
|
||||
"data": "motions/wave.f32"
|
||||
}
|
||||
```
|
||||
|
||||
`.f32` 是 little-endian float32:所有 `[frames, joints]` positions,随后所有 `[frames, joints]` velocities。该 JSON 的 data 同样相对 Package 根目录解析,支持部署 base path / 同源签名。尺寸、顺序、控制时钟、长度和有限值均检查,不会悄悄插值或重排。`validate:packages` 也验证参考样本,不只检查文件存在。
|
||||
|
||||
## 状态机
|
||||
|
||||
```text
|
||||
walk → settling → entering → playing → returning → walk
|
||||
↘ 移动/取消/超时 → walk
|
||||
```
|
||||
|
||||
- 停稳门限:实际平移速度 <0.15 m/s、角速度 <0.2 rad/s,持续 0.4 s;5 s 超时取消。继续移动时拒绝 pending 动作。
|
||||
- entry/return 各 0.5 s 混合当前 target 与策略 target,只覆盖 primary 拥有的关节,不破坏独立 overlay。
|
||||
- duration 从进入新策略开始计时,**包含进入 blend**。Dance 使用参考帧 0–499,wave 使用 0–217;返回 walk 重置行走历史。
|
||||
- 方向键中断动作,以零命令行走混回,随后恢复移动输入;不支持 episode→episode 直接抢占。
|
||||
- pause 不推进计时/参考;reset 清除 pending/active episode 和所有策略状态;跌倒优先停止推理。状态栏显示当前 skill / phase。
|
||||
|
||||
## 更新权重
|
||||
|
||||
### 走路
|
||||
|
||||
```bash
|
||||
TRAIN=/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train
|
||||
"$TRAIN/.venv/bin/python" app/scripts/import-rp1.py \
|
||||
--source "$TRAIN" \
|
||||
--checkpoint "$TRAIN/logs/rsl_rl/rp1_24dof_flat/2026-10-28_15-47-49/model_9000.pt" \
|
||||
--policy-only --completed
|
||||
```
|
||||
|
||||
`--policy-only` 保留资产和其他技能;`--completed` 要求达到日志配置的最终迭代并更新训练状态/显示名。省略 `--policy-only` 会完整重建 manifest,把舞蹈/挥手恢复为不可用;不要用它无意覆盖已验证的配置。
|
||||
|
||||
### 舞蹈/挥手
|
||||
|
||||
```bash
|
||||
"$TRAIN/.venv/bin/python" app/scripts/import-rp1-motion.py --skill dance \
|
||||
--checkpoint "$TRAIN/logs/rsl_rl/rp1_24dof_beyondmimic_dance/2026-10-29_11-05-14_dance_resume/model_23998.pt"
|
||||
"$TRAIN/.venv/bin/python" app/scripts/import-rp1-motion.py --skill wave \
|
||||
--checkpoint "$TRAIN/logs/rsl_rl/rp1_24dof_beyondmimic_wave/2026-10-28_19-00-23_wave/model_7999.pt"
|
||||
```
|
||||
|
||||
导出器读取日志中真实 actor/观测/关节/default/action/actuator 合同和 motion 路径,拒绝中间 checkpoint、时钟不符、不支持的分支/结构或不同驱动参数。导出含归一化,并比较 PyTorch 与 ONNX 输出。Dance 默认部署 10 s,wave 默认全片;`--duration` 可显式缩短/选择验证时长,不能超出参考总长。
|
||||
|
||||
**导入始终把该动作设为 `available:false`、绑定 `disabled:true`,避免新权重未经物理验证就上线。** 先运行 staged 资格测试,再检查测试报告;通过后显式启用 skill / binding,并更新 label 与 `source-manifest.json` 的 validation。源文件与导出内容的哈希不能凭空改成“已验证”。
|
||||
|
||||
```bash
|
||||
npm --prefix app run validate:packages
|
||||
RP1_TEST_STAGED=1 npm --prefix app run test:e2e -- rp1-motion.spec.js
|
||||
npm --prefix app test
|
||||
npm --prefix app run build
|
||||
npm --prefix app run test:e2e
|
||||
```
|
||||
|
||||
如果现有 Vite 进程已经运行很久,添加新 public 文件/目录后请重启 dev server;本机发现过旧进程缓存文件集合/变换的问题。资格测试与最终回归使用单独的新进程,不能仅依赖旧服务热更新结果。
|
||||
|
||||
## 验证与许可
|
||||
|
||||
`rp1.spec.js` 验证最终行走权重、3×25 s 序列、频率、跌倒/reset/切换和通用 episode 接口 fixture;它复用 walk 的那个测试不是动作质量证明。
|
||||
|
||||
`rp1-motion.spec.js` 使用真实 dance/wave ONNX 和参考:三个朝向的实际行走→动作→行走,核对 observation 内参考数值、完整部署区间、无跌倒/传送,键盘、pause/reset/中断、连续四次无 reset 触发和损坏 reference 的加载回滚。
|
||||
|
||||
最新舞蹈权重与标签更新见 [23998 更新验收](../superpowers/verification/2026-10-29-rp1-dance-23998.md);原多技能集成见 [最终技能验收](../superpowers/verification/2026-10-29-rp1-completed-skills.md)。[原接入记录](../superpowers/verification/2026-10-28-rp1-platform.md) 描述先前中途权重;[场地尺寸验证](../superpowers/verification/2026-10-28-robot-scaled-arena.md) 描述按机器人缩放的边界。
|
||||
|
||||
RP1 description 和 motion 来源未明确提供可统一再分发的许可;仅供本机评估。请在公开部署/再分发前确认模型、mesh、用户权重及动作数据各自的权限,见 `app/public/robots/rp1/SOURCE.md`。没有自动提交或公开发布。
|
||||
@@ -0,0 +1,50 @@
|
||||
# Robot-relative arena sizing — 2026-10-28
|
||||
|
||||
## Requested behavior
|
||||
|
||||
Use Microduck's arena/body proportions for each robot instead of assigning all robots the same absolute 4.5 m arena and 0.25 m walls.
|
||||
|
||||
## Size convention
|
||||
|
||||
`robot.json` now declares `dimensions.height`: nominal full-body standing height in metres. Source MJCF visual-geometry bounds were measured at each package's reference/reset pose, excluding non-robot bodies. Approximate measured heights: Microduck legs 0.272 m, Microban 0.332 m, RP1 1.321 m. Declared rounded values are 0.27 / 0.33 / 1.32 m. Both Microduck locomotion variants use the same family reference height, avoiding scene resizing when switching feet/rollers.
|
||||
|
||||
This is deliberately not root spawn height, camera distance, or live pose height. Walking, crouching, falling or waving must not move the walls.
|
||||
|
||||
The arena template declares `scale_to_robot.reference_height = 0.27`. All lengths scale by `dimensions.height / reference_height`:
|
||||
|
||||
| Robot | Scale | Inner arena width × length | Wall height | Wall thickness |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Microduck | 1 | 4.5 × 4.5 m | 0.25 m | 0.05 m |
|
||||
| Microban | 1.222222 | 5.5 × 5.5 m | 0.305556 m | 0.061111 m |
|
||||
| RP1 | 4.888889 | 22 × 22 m | 1.222222 m | 0.244444 m |
|
||||
|
||||
Arena width / robot height remains 16.6667; wall height / robot height remains 0.925926. Existing Microduck baseline behavior is unchanged.
|
||||
|
||||
## Implementation
|
||||
|
||||
- `platform/worlds/arena.js::fitWorldToRobot` clones/resolves the authored World before MuJoCo compilation. It does not mutate shared configuration and repeated resolution does not double-scale.
|
||||
- `PackageLoader.loadWorld(id, config)` is used by RuntimeManager and the standalone Microduck entry. There are no robot-id branches in scaling logic.
|
||||
- Physical walls use the resolved `boundary`. Visual wall positions, heights and spans use those same dimensions.
|
||||
- Arena grid spacing, geometry extent and radial fade distances scale together; colours, reveal behavior and flat floor physics are unchanged. Without scaled fade, RP1's distant walls would disappear.
|
||||
- `world.scale` is runtime-only metadata; it cannot be injected as an authored world.json property.
|
||||
- Missing dimensions retain reference scale 1 for old packages. Future packages should explicitly declare nominal height. Worlds without `scale_to_robot` (including `flat`) retain fixed metre dimensions.
|
||||
- The RP1 full importer preserves its 1.32 m nominal height; policy-only export leaves package dimensions intact.
|
||||
|
||||
No robot meshes, masses, gains, gravity, command speeds or physics timestep are rescaled. No position clamp or automatic reset is used to create the boundary effect.
|
||||
|
||||
## Verification
|
||||
|
||||
- Unit tests: 49/49 pass, including per-package scale, unchanged template, idempotence, fallback, explicit fixed Worlds, invalid dimensions/reference scale, visual/physical dimensions, grid/fade scaling and disposal.
|
||||
- Targeted browser checks: 5/5 pass using actual MuJoCo WASM: all three robots' compiled collision walls and visual faces agree, future-package sizing, presentation cleanup, plus actual four-sided wall contacts for **both Microban and RP1**. Robots are placed near each wall and commanded forward; each wall reports contacts, finite state and root positions remaining inside the boundary, without coordinate clamping.
|
||||
- Full browser regression: **37/37 pass**, including unchanged Microduck deterministic baselines, Microban long walking trials, RP1 locomotion/skills/reset, 20 three-family ownership transitions, and deployment. Runtime model/data disposal and scene/GPU/listener plateaus remain valid.
|
||||
- Production build and format check pass; the existing non-fatal MuJoCo browser `module` warning remains.
|
||||
- Screenshot: `app/test-results/bounded-world-all-three-*/bounded-arena.png` shows RP1 at arena overview scale (generated and ignored by git).
|
||||
|
||||
```bash
|
||||
npm --prefix app test
|
||||
npm --prefix app run validate:packages
|
||||
npm --prefix app run build
|
||||
npm --prefix app run test:e2e
|
||||
```
|
||||
|
||||
This record supersedes the fixed-size field dimensions in earlier default-arena/RP1-integration verification records; those records describe the earlier implementation and remain historical evidence.
|
||||
@@ -0,0 +1,59 @@
|
||||
# RP1 platform integration verification — 2026-10-28
|
||||
|
||||
## Artifact
|
||||
|
||||
- Package: `app/public/robots/rp1/`, third family in the existing selector; generic runtime, no RP1-id branches in physics/rendering/runtime management.
|
||||
- Source description: `rp1_description@d8c6dd03aeecc7f7ead3d962e09fedba4c8460f0`.
|
||||
- Checkpoint: `rp1_24dof_flat/2026-10-28_15-47-49/model_3000.pt`, SHA-256 `ac57cfa7257b0cee309536af695f4f793be4b82bf106daa3372a0ae6da477931`.
|
||||
- ONNX SHA-256: `0b89b7b4c1c35dfb1912a0a39950e283f31c760d8e386ad8e2da20851cffc34a`.
|
||||
- Export includes empirical normalization and deterministic ELU actor. ONNX checker + zero/eight seeded random comparisons against PyTorch: max absolute error `9.5367431640625e-7`.
|
||||
- Both full import and `--policy-only` were run. `robot.json` hash remained identical after policy-only update (`6fdb2b039a5f6df83418d46de9129dd2bfa742c1e7623349614189697d2b2b5c`). No training process or source training code was changed.
|
||||
|
||||
## Numerical/browser checks
|
||||
|
||||
Real Chromium + MuJoCo WASM + ONNX Runtime WASM; not a mocked controller. The initial 3-second independent trials were reset separately:
|
||||
|
||||
| Trial | World X displacement | Yaw change | Fall |
|
||||
|---|---:|---:|---|
|
||||
| stand | −0.0021 m | 0.0002 rad | none |
|
||||
| forward +0.4 | +0.6953 m | −0.0150 rad | none |
|
||||
| backward −0.3 | −0.5985 m | approximately 0 rad | none |
|
||||
| left +0.6 | −0.0361 m | +1.6520 rad | none |
|
||||
| right −0.6 | −0.0688 m | −1.7249 rad | none |
|
||||
|
||||
Three **deterministic** 25-second sequences also pass: stand 5 s → forward 3 s → rest 2 s → backward 3 s → rest 2 s → left 3 s → rest 2 s → right 3 s → rest 2 s. No falls or in-sequence resets; root positions remain within `|x|, |y| < 1.9 m` (arena inner half-extent 2.25 m). Representative local forward/backward displacement: `+0.7551 / −0.5143 m`; turns `+1.5705 / −1.5989 rad`; minimum root height `0.7622 m`; PD saturation count 0/120000 per sequence. Identical trials are repeatability evidence, not domain-randomized robustness evidence.
|
||||
|
||||
A four-second realtime trial advances at approximately 1.02× wall time (acceptance range 0.8–1.15), consistent with the 50 Hz scheduler. The bound is measured for simulation/control, not a GPU rendering FPS promise.
|
||||
|
||||
Keyboard W/S/A/D commands match manifest limits; key-up clears them. J/H are shown but disabled while no weights exist. Artificial side tilt triggers fall detection; Space clears fall, history and FSM. Microduck → Microban → RP1 transitions are exercised. Screenshot: Playwright `test-results/rp1-third-robot-*/rp1.png` (generated, ignored by git).
|
||||
|
||||
## Interface checks
|
||||
|
||||
- Unit tests cover term-major/frame-major history, first-sample backfill, live debug reads without history pushes, reset, policy order, named gains, torque-speed limits, atomic invalid-action handling, independent skill defaults/history, load-failure session release, stop gating, timeout, blend, movement interruption and unavailable input suppression.
|
||||
- The episode browser fixture explicitly reuses the walk ONNX to exercise real stop gating, two sessions/reset/return ownership and movement interruption; it is **not a dance/wave implementation or a quality test of those future motions**.
|
||||
- Original BeyondMimic policy observations include motion-preview/anchor/reference terms absent from this runtime. Those weights require a real reference adapter or compatible export before enabling the reserved skills.
|
||||
|
||||
## Final results
|
||||
|
||||
- `validate:packages`: all four manifests (three families), resource paths and provenance hashes pass.
|
||||
- `npm test`: **47/47** unit tests pass.
|
||||
- `format:check` and production build pass (the existing MuJoCo browser-external `module` warning remains non-fatal).
|
||||
- Full browser regression: **36/36** pass, including all nine unchanged Microduck deterministic baselines and Microban long walking trials.
|
||||
- After restricting episode blending to primary-owned joints and enhancing tests, the final **6/6** affected browser tests were rerun successfully: three-family deployment, original Microban mounted-IMU, RP1 keyboard/fall/switching, RP1 3×25 s/realtime, real stop-gated episode/interruption, and 20 ownership transitions now cycling through **all three robot families**.
|
||||
- In that final ownership run, every retired model/data is deleted. RP1 listener count remains 15, scene children 5, renderer geometry/texture counts plateau at 69/2; no monotonic leak is observed.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
npm --prefix app run validate:packages
|
||||
npm --prefix app test
|
||||
npm --prefix app run build
|
||||
npm --prefix app run format:check
|
||||
npm --prefix app run test:e2e
|
||||
```
|
||||
|
||||
Playwright JSON output contains the `rp1-preview-trials.json`, `rp1-three-sequences.json` and `rp1-realtime.json` attachments. A full-suite run caught stale single-frame diagnostic reads after introducing history; `write({advance:false})` was corrected to sample live state without committing a history push, and the original Microban mounted-IMU regression then passed.
|
||||
|
||||
## Limits
|
||||
|
||||
Intermediate weights only; no statement of training convergence, real-robot readiness, recovery-policy availability or arbitrary BeyondMimic compatibility. Random delays, domain randomization and training disturbances are omitted. Motion-transition blends enforce target continuity, not universal dynamic safety. RP1 description licensing remains unprovided in this checkout; local evaluation only, not clearance for public asset redistribution.
|
||||
@@ -0,0 +1,32 @@
|
||||
# RP1 unattended integration window
|
||||
|
||||
- Authorized by user: 2026-10-28 18:15 Asia/Shanghai, approximately 15 hours (until 2026-10-29 09:15).
|
||||
- Scope: integrate completed locomotion, dance and wave checkpoints from `roboparty_train/logs/rsl_rl`; prepare/test interfaces while training runs; do not interrupt or modify training processes. Do not enable absent/unvalidated skills.
|
||||
- Working tree contains earlier uncommitted RP1 + arena work; preserve it. No automatic commits or public deployment.
|
||||
|
||||
## Progress
|
||||
|
||||
### 18:21 — discovery and final walking export
|
||||
|
||||
- Completed flat run: `rp1_24dof_flat/2026-10-28_15-47-49`, configured `max_iterations=9001`, final `model_9000.pt`, no corresponding active training process. Exported using `--policy-only`, preserving assets and reserved skills. PyTorch/ONNX maximum absolute error 1.55e-6. Browser validation was pending at this observation.
|
||||
- Dance actively training: PID 3972369, `rp1_24dof_beyondmimic_dance/2026-10-28_17-40-59_dance`, configured 8000 iterations; intermediate checkpoints are NOT completion evidence.
|
||||
- No wave run present yet; may be queued after dance in the operator's shell. Do not launch a duplicate GPU training job.
|
||||
- **Important actual actor contract:** the logged runner uses `obs_groups.actor: [student]`, not `[policy]`. The actor input is **126**, not the privileged policy group's motion-preview/anchor observation shape. Student observations are motion command (reference joint positions + velocities, 48), body angular velocity (3), projected gravity (3), relative joint positions (24), relative joint velocities (24), previous actor action (24). No observation history. Importer must inspect logged runner mapping and reject unsupported contracts rather than assume privileged observations.
|
||||
- Both source motions are 50 fps, matching the trained 50 Hz command clock. Dance: 6573 frames / 131.46 s. Wave: 218 frames / 4.36 s. Training advances one reference frame per control step; no wrapping/teleporting is allowed in the browser.
|
||||
|
||||
### 2026-10-29 09:19 onward — continuation and actual integration
|
||||
|
||||
- On the user's subsequent `继续`, confirmed both dance and wave runs had finished: final `model_7999.pt`, configured 8000 iterations, stable files and no matching live training jobs. This log records observed execution points, not an uninterrupted polling service.
|
||||
- Implemented actual student motion-reference observations and an exporter that reads logged contracts, preserves normalization/defaults and records hashes; no privileged-input approximation.
|
||||
- Exported dance/wave with ONNX/PyTorch maximum errors 9.54e-7 / 1.91e-6. Staged disabled, then ran physical qualification before enabling.
|
||||
- Full 131.46 s dance failed at frame 994 (~19.9 s), so it is not advertised as stable. The validated 10 s prefix is enabled and explicitly labelled; full 4.36 s wave is enabled.
|
||||
- Final walking policy and both deployed motion intervals passed three initial heading cases, movement before/after, pause/reset/interruption, continuous repeat without robot reset and malformed-reference rollback. Targeted browser tests 8/8 and unit tests 53/53 pass.
|
||||
- Current detailed results and remaining scope: [completed-skills verification](2026-10-29-rp1-completed-skills.md). Source training was not modified or stopped; no public deployment/automatic commit.
|
||||
|
||||
## Acceptance gates
|
||||
|
||||
1. Completed, stable, fully readable final checkpoints; checked logged actor/actuation/joint/observation contracts.
|
||||
2. Deterministic ONNX export including normalization; numerical parity with PyTorch; hashed checkpoint and motion provenance.
|
||||
3. Independent per-skill reference/history/action lifecycle, pause/reset/interrupt semantics and exclusive ownership.
|
||||
4. Real MuJoCo browser playback and transitions without falls, non-finite state, teleportation or scripted substitution for trained actions.
|
||||
5. Regression checks, source-hash/package validation and production build. Keep any failing skill disabled and record precise evidence.
|
||||
@@ -0,0 +1,87 @@
|
||||
# RP1 completed walking + independent dance/wave policies — 2026-10-29
|
||||
|
||||
Historical initial-integration record (dance checkpoint 7999). Current dance weights are checkpoint 23998; current UI labels are plain `Dance` / `Wave`. See the [subsequent weight/label update](2026-10-29-rp1-dance-23998.md) for current qualification and limits.
|
||||
|
||||
## Scope and completion evidence
|
||||
|
||||
User authorized an unattended integration window and subsequently requested continuation. After resuming on October 29, both BeyondMimic final checkpoints were present, stable and fully readable, with no corresponding active training process. We did not start, stop, resume or modify training jobs. Checkpoints were explicitly selected, not inferred from an arbitrary latest file.
|
||||
|
||||
| Skill | Checkpoint under roboparty_train/logs/rsl_rl | Logged final iteration | Export max abs error |
|
||||
|---|---|---:|---:|
|
||||
| walk | `rp1_24dof_flat/2026-10-28_15-47-49/model_9000.pt` | 9000 / 9001 configured | 1.55e-6 |
|
||||
| dance | `rp1_24dof_beyondmimic_dance/2026-10-28_17-40-59_dance/model_7999.pt` | 7999 / 8000 configured | 9.54e-7 |
|
||||
| wave | `rp1_24dof_beyondmimic_wave/2026-10-28_19-00-23_wave/model_7999.pt` | 7999 / 8000 configured | 1.91e-6 |
|
||||
|
||||
The initial intermediate walking policy was replaced with the completed checkpoint. The selector now displays `RP1`, not `training preview`. This describes completed training iterations, **not** a proof of convergence or real-hardware readiness.
|
||||
|
||||
## Actual actor contract — student, not privileged policy
|
||||
|
||||
The logged runner maps `obs_groups.actor` to `[student]`. Actor weights confirm `[512,126]` first-layer shape. The privileged observation group in the source config is **not** the actor input and must not be used to invent unnecessary preview/anchor inputs.
|
||||
|
||||
Dance/wave use float32 `[1,126] → [1,24]`: reference joint positions 24, reference velocities 24, body gyro 3, projected gravity 3, relative joint position 24, joint velocity 24, previous action 24. No observation history. Independent ELU MLPs include their empirical normalizers `(obs - mean)/(std + .01)`; runner action clipping is null, and no clip is added. Each policy has its own trained default-pose map, action history and reference clock. Named joint ordering and 0.25 position-target scale are checked against logged robot/action config. PD/torque-speed parameters are checked against the existing robot contract.
|
||||
|
||||
Walking retains 243D three-frame term-major observations and ±100 runner clipping. All policies use 0.005 s physics / 4 decimation / 50 Hz control.
|
||||
|
||||
## Integration and safety semantics
|
||||
|
||||
- New reusable `JointMotionReference` reads positions/velocities in explicit float32-LE layout. It checks exact byte count, frame count, joint order, clock and finiteness. Package validation also reads/checks reference bytes and source hashes.
|
||||
- Reference data only writes observation inputs. It never writes qpos, root transforms, velocities or actuator targets; the actual trained actor generates actions.
|
||||
- Each activation resets its own reference to frame 0. Simulation time drives advancement; pause freezes it. Reference end clamps without wrapping/random resampling. Reset clears all policy state; movement interrupts to locomotion.
|
||||
- Existing exclusive FSM / stop gate / 0.5 s target blends remain shared platform behavior. Duration endpoint comparison includes a small numerical tolerance so floating simulation time cannot execute one unintended extra reference frame.
|
||||
- Missing/truncated assets fail the candidate transaction, release partial policy resources and preserve the previous robot. Retrying with good data succeeds.
|
||||
- Motion importer stages skills **disabled** after every update; qualification tests can explicitly enable candidates with `RP1_TEST_STAGED=1`. Both deployed skills were enabled only after actual physics tests passed. There are no stand-in walk policies or procedural animations in the deployed dance/wave entries.
|
||||
|
||||
## Important negative result and deliberately bounded dance
|
||||
|
||||
The dance source archive is 6573 frames at 50 fps: **131.46 s**. An initial full-clip browser rollout failed:
|
||||
|
||||
- entered dance at simulation ~2.44 s;
|
||||
- fell at simulation **22.32 s**, reference **frame 994**, root height **0.38572 m**;
|
||||
- no non-finite values, resets or torque saturation hid the failure.
|
||||
|
||||
Therefore full continuous dance is **not qualified or advertised**. The deployed dance is explicitly the **first 10 seconds**, matching the training's 10 s episode horizon and original episodic skill duration. UI label is `Dance / 舞蹈(10 秒片段)`. Its source reference remains available for future qualified extensions, but duration is 10 and deployed reference frames are 0–499. Raising duration is not a validated solution.
|
||||
|
||||
Wave source is 218 frames at 50 fps: the **entire 4.36 s**, frames 0–217, is enabled. UI label is `Wave / 挥手(4.36 秒)`.
|
||||
|
||||
## Positive physics qualification
|
||||
|
||||
`app/e2e/rp1-motion.spec.js`, actual MuJoCo WASM and actual exported ONNX:
|
||||
|
||||
1. Each skill at three orientations reached through physical locomotion (straight, left turn, right turn), with forward walking before the request. Both complete their configured intervals, return to walking, and then make >0.4 m forward progress in 3 s without falling.
|
||||
2. Reference position/velocity observations are compared element-for-element against packaged float32 samples at the reported reference frame. Observation shape is 126, not silently padded/truncated.
|
||||
3. No non-finite state, no root teleportation: maximum per-control-step root displacement <0.014 m in the qualified trials. No sustained torque saturation (recorded saturation count zero).
|
||||
4. Minimum root height across the three wave trials approximately **0.758 m**; dance approximately **0.725 m**, safely above the unchanged 0.4 m fall threshold. The fall detector was not loosened.
|
||||
5. Real H/J keyboard activation; pause freezes both physics time/reference; movement interrupts and blends back; Space clears active/pending/reference/action state.
|
||||
6. **wave → dance → wave → dance without resetting the robot**: every episode starts at reference frame 0 and ends at its configured final frame, preserving balance.
|
||||
7. Truncated wave binary causes switch rollback, old model retention and clean retry.
|
||||
8. Recorded once-per-second upper-limb observations confirm meaningful trained motion rather than standing still: wave arm-joint tracking RMSE **0.084–0.092 rad**, maximum sampled joint excursion **1.97–2.01 rad**; dance RMSE **0.274–0.283 rad**, excursion **1.33–1.40 rad** across the three headings. These are sampled tracking metrics, not full-frame precision claims. Mid-motion production-bundle screenshots were manually inspected at reference frame 78 (`app/test-results/rp1-visual-check/{wave,dance}-mid.png`); wave's right wrist reached ~1.242 m while root was ~0.773 m.
|
||||
|
||||
`app/e2e/rp1.spec.js` also passes on the final walking policy: keyboard directions, falls/reset, all-family switching, three 25 s deterministic movement sequences and realtime ~50 Hz control. The walk-reused-as-episode test is still only an interface fixture, clearly separated from the real trained-motion tests.
|
||||
|
||||
## Validation status
|
||||
|
||||
- Unit tests: **53/53 passed**.
|
||||
- Package validation: all four package manifests pass; RP1's 33 resource paths, both reference streams and source SHA-256 records pass.
|
||||
- Targeted RP1 browser tests: **8/8 passed** on the final enabled manifest and fresh server.
|
||||
- Production build and format check passed. Existing non-fatal MuJoCo `module` externalization warning remains.
|
||||
- **Full browser regression: 42/42 passed** (10.7 minutes), including all Microduck deterministic baselines, Microban walking, all-family switching, 20 ownership/disposal transitions, race handling and deployment.
|
||||
- **Production-bundle RP1 checks: 8/8 passed** against a fresh Vite preview on port 5184. Reference samples, transitions, repeated activations, keyboard/reset and rollback pass outside the dev server as well. The local-only preview is left available at `http://127.0.0.1:5184/`; the temporary test dev server on 5183 was stopped. Existing user services were not restarted. Source/dist manifests were checked identical after the final build.
|
||||
- Final full-suite log: `/tmp/rp1-completed-skills-e2e.log`, exit code 0. Browser reports/screenshots are generated under ignored `app/test-results/`; later targeted/production runs replace the previous JSON report, while this verification record preserves the aggregate result.
|
||||
|
||||
An old long-running Vite server was observed returning stale transformed JS / HTML for newly added public files. Final qualification uses a separately started fresh server rather than trusting hot reload. No unrelated user server was stopped.
|
||||
|
||||
## Reproduction and limits
|
||||
|
||||
```bash
|
||||
npm --prefix app test
|
||||
npm --prefix app run validate:packages
|
||||
npm --prefix app run build
|
||||
npm --prefix app run format:check
|
||||
npm --prefix app run test:e2e
|
||||
```
|
||||
|
||||
Use a fresh dev server; `TEST_BASE_URL` can select an isolated instance. Production-facing RP1 tests also run against the built bundle; development-only fixture pages are not shipped in dist.
|
||||
|
||||
Training-time randomized actuator delays, random physics/default poses, pushes and observation corruption remain omitted. Long dance playback, arbitrary starts, uneven terrain, severe pushes, hardware, mobile browsers and public redistribution are not accepted by these tests. RP1 description / motion redistribution rights remain unconfirmed; local evaluation only. No commit or public deployment was performed.
|
||||
|
||||
See [RP1 operator/update documentation](../../robots/rp1.md), [source provenance](../../../app/public/robots/rp1/SOURCE.md) and the [initial unattended-window log](2026-10-28-rp1-unattended.md).
|
||||
@@ -0,0 +1,35 @@
|
||||
# RP1 dance checkpoint 23998 + plain English skill labels
|
||||
|
||||
## Requested change
|
||||
|
||||
- Replace dance weights with `rp1_24dof_beyondmimic_dance/2026-10-29_11-05-14_dance_resume/model_23998.pt` under the user's `roboparty_train/logs/rsl_rl`.
|
||||
- Remove parenthesized UI details: right-side labels are exactly **Dance** and **Wave**.
|
||||
|
||||
## Export and preserved contract
|
||||
|
||||
- Specified checkpoint is complete/readable, actor tensors finite, iteration 23998. This run starts from iteration 7999 and adds 16000 iterations: final `7999 + 16000 - 1 = 23998`. Logged `max_iterations=16000` is the additional training count, not the absolute final iteration.
|
||||
- Logged robot, student observations, actions, commands and decimation match the previous run exactly. Actor remains normalized ELU MLP, float32 `[1,126] → [1,24]`; no runtime, physics, actuator, observation or transition changes were needed.
|
||||
- ONNX export includes normalization; max absolute PyTorch/ONNX difference **1.3113021850585938e-6**. Checkpoint hash and per-file export hashes updated in `app/public/robots/rp1/source-manifest.json`.
|
||||
- Export staged dance disabled. It was re-enabled only after all real-physics qualification tests passed.
|
||||
- Dance remains the previously configured **10-second prefix**, frames 0–499; blend/stop gates unchanged. **Full 131.46-second playback of checkpoint 23998 has not been qualified.** The earlier long-rollout failure belongs to checkpoint 7999 and is not presented as a result for this new checkpoint.
|
||||
- Byte comparisons with pre-update backups prove walk/wave ONNX and dance reference JSON/binary unchanged. Robot manifest is otherwise identical after accounting for the two requested label changes. Walk/wave checkpoint provenance is unchanged.
|
||||
- Full importer labels also use plain `Dance` / `Wave`, so rebuilding a package does not reintroduce bracketed text. Browser selectors and a package-contract unit assertion protect the new labels.
|
||||
|
||||
## Verification
|
||||
|
||||
- **53/53 unit tests passed**.
|
||||
- Package schemas, both motion references, RP1's 33 enabled resource paths and source hashes pass validation.
|
||||
- **5/5 staged real-browser qualification tests passed** on a fresh isolated dev server.
|
||||
- **8/8 RP1 production-bundle browser tests passed**, including:
|
||||
- three physical initial-heading cases for dance and unchanged full wave;
|
||||
- complete configured playback, exact reference samples, return to walking and forward progress;
|
||||
- H/J keys, no Chinese or bracketed labels, pause/reference freeze, movement interruption and reset;
|
||||
- wave→dance→wave→dance without robot reset; references restart at frame 0;
|
||||
- malformed-reference rollback and retry;
|
||||
- final walking policy's 3×25 s sequences, fall/reset, all-family switching and realtime control.
|
||||
- Dance's three qualification trials: no falls/non-finite state/torque saturation; minimum root height **0.75032 m**; largest per-control-step root displacement **0.01157 m**; forward displacement after returning to walk **0.801–0.843 m** in 3 s.
|
||||
- Production build and formatting pass; only the pre-existing non-fatal MuJoCo `module` browser warning remains.
|
||||
|
||||
The existing local production preview at `http://127.0.0.1:5184/` serves the updated build. Refresh the page to load the new policy and labels. Temporary candidate server 5185 is not needed after qualification. Source training files/processes were not modified. No commit or public deployment was performed.
|
||||
|
||||
This is a targeted weight-update verification; the earlier 42-test full-suite result is recorded in [initial completed-skills verification](2026-10-29-rp1-completed-skills.md), not claimed to have been rerun for this checkpoint.
|
||||
Reference in New Issue
Block a user