2 Commits

Author SHA1 Message Date
chenlin a8be5e0134 feat: release v0.5 with shared interactive arenas and textured RP1
Unify lighting and line-colour controls across robots, replace the football with a shared layout of draggable physical props, and import the supplied RP1 GLBs without changing trained physics or policies.

Include reproducible visual import/provenance, regression coverage and verification records. Verified 66 unit tests, 55 browser tests and production-build checks.
2026-10-30 14:14:03 +08:00
chenlin 57b3108ab3 feat: release v0.4 with textured Microban visuals 2026-10-29 15:55:17 +08:00
101 changed files with 6032 additions and 328 deletions
+19 -6
View File
@@ -15,17 +15,30 @@ MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), *
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
## v0.5
- Brighter shared lighting and the original four-swatch **Line color** control for every robot.
- One session-wide layout of 5–7 physical, draggable geometries replaces the football and scales consistently across robots.
- RP1 uses the supplied textured GLBs, including fixed-hand detail, without changing its 24-DOF physics or trained skills.
- Verified with 66 unit tests, 55 browser tests and production-build checks. See [shared environment](docs/superpowers/verification/shared-robot-environment.md) and [RP1 materials](docs/superpowers/verification/rp1-glb-materials.md) verification records.
## Robots
| Robot | Default scene / presentation | Controls |
|---|---|---|
| Microduck | Shared 4.5 × 4.5 m bounded arena, ball, original GLB/palette/comic effects | Existing walking, skating, skills and recovery |
| Microban | Robot-scaled 5.5 × 5.5 m arena, dark/orange grid and comic UI, MJCF robot visuals | Walking policy + independent head rotation |
| RP1 | Robot-scaled 22 × 22 m arena/comic UI, original 24-DOF MJCF visuals | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
| Microduck | Shared 4.5 × 4.5 m bounded arena, 5–7 random interactive geometries, original GLB/comic effects | Existing walking, skating, skills and recovery |
| Microban | Shared robot-scaled 5.5 × 5.5 m arena, interactive geometries, textured per-body GLB visuals | Walking policy + independent head rotation |
| RP1 | Shared robot-scaled 22 × 22 m arena, interactive geometries, user-supplied textured GLB visuals on original 24-DOF physics | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's scene-line colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); exact source hashes, dirty-worktree provenance and license notices are included in its Package.
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban physics / policy assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); its textured visual replacement comes from the user-supplied `microban2.zip`. Exact hashes, dirty-worktree provenance and license caveats are included in its [Package source notes](app/public/robots/microban/SOURCE.md).
### Shared scene controls (all robots)
The scene has brighter ambient / environment illumination plus a hemisphere fill light. The lower-left **Line color** palette retains the original Cream / Graphite / Lavender / Sky swatches and square comic buttons, but now changes **floor and wall grid lines**, not robot materials. The same palette is available for every robot, and its selection is global across robot switches, reset and feet / rollers changes.
The football is removed. One browser session randomly generates **5–7 lightweight boxes, spheres, cylinders and capsules** shared by **all robots**, clear of both bundled spawn anchors and each other. Their types, colours and relative layout stay identical when switching robots; geometry and positions scale with the existing arena-to-robot ratio. They have real MuJoCo free joints: every robot can push them, they collide with one another, the floor and walls, and the mouse can grab / lift them using spring forces. Reset, locomotion changes and robot switches restore the same session layout; refreshing generates a new layout. Escaped toys are returned individually. This shares environment configuration/layout, not live object momentum between different-scale robot simulations.
## Keyboard and mouse
@@ -47,7 +60,7 @@ Microduck policies / assets originate from [pollen-robotics/microduck](https://g
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Microduck mouse dragging still applies physical forces to the robot / ball. Microban / RP1 mouse dragging orbits the camera. Gamepad / touch are not part of this release's acceptance scope.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Dragging any robot or interactive geometry applies physical forces; dragging empty space orbits the camera, consistently for all three robots. Gamepad / touch are not part of this release's acceptance scope.
## Microban fidelity and limits
+1 -1
View File
@@ -59,7 +59,7 @@ test("arena appearance matches Microduck while Microban physics stays flat and W
expect(state).toEqual({
cell: 0.1 * (0.33 / 0.27),
section: 0.6 * (0.33 / 0.27),
color: 0xffb366,
color: 0xf7e6cb,
reveal: 1,
topo: 0,
hiddenFloor: true,
+9 -5
View File
@@ -60,13 +60,15 @@ test("both robots default to the same rear camera and comic UI; Microban keeps f
await expect(
page.getByRole("navigation", { name: "Simulation controls" }),
).toHaveCSS("border-top-width", "2px");
await expect(page.getByRole("group", { name: "Duck colours" })).toHaveCount(
0,
);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
await expect(
page.getByRole("button", { name: "Rollers", exact: true }),
).toHaveCount(0);
await expect(page.getByText("Drag robot / ball: apply force")).toHaveCount(0);
await expect(
page.getByText("Drag robot / geometry: apply force"),
).toBeVisible();
await expect(page.getByRole("status")).toContainText("Simplified actuation");
expect(
await page.evaluate(() => window.rl.model.geom("platform_floor").id),
@@ -74,7 +76,9 @@ test("both robots default to the same rear camera and comic UI; Microban keeps f
await page.screenshot({ path: info.outputPath("microban-chase.png") });
await page.evaluate(() => window.__gameApi.switchRobot("microduck-legs"));
await rear(page);
await expect(page.getByRole("group", { name: "Duck colours" })).toBeVisible();
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
expect(errors).toEqual([]);
});
for (const id of ["microban", "microduck-legs"])
+18 -1
View File
@@ -20,7 +20,24 @@ test("deployed assets respect the base path and all three Robot Packages boot",
await expect(
page.getByRole("button", { name: "Pause", exact: true }),
).toBeEnabled();
await page.waitForTimeout(500); // capture after the ball's existing appear FX
await page.waitForTimeout(500); // let the physical toys settle
const toys = await page.evaluate(() => {
const r = window.rl;
return {
count: r.interactiveProps.length,
visible: r.scene.getObjectByName("interactive-props").visible,
hasBall:
r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
) >= 0,
};
});
expect(toys.count).toBeGreaterThanOrEqual(5);
expect(toys.count).toBeLessThanOrEqual(7);
expect(toys.visible).toBe(true);
expect(toys.hasBall).toBe(false);
await page.screenshot({ path: info.outputPath("microduck.png") });
await page.getByLabel("Robot", { exact: true }).selectOption("microban");
await expect
+212
View File
@@ -0,0 +1,212 @@
import { test, expect } from "@playwright/test";
async function boot(page) {
await page.goto("/");
await expect(
page.getByRole("button", { name: "Pause", exact: true }),
).toBeEnabled();
await page.evaluate(() => window.__gameApi.pause());
}
test("20 textured GLBs replace only Microban visuals, align to original geometry and follow all bodies", async ({
page,
}, info) => {
const errors = [];
page.on("pageerror", (error) => errors.push(error.message));
await boot(page);
expect(
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
).toBe(true);
const inspect = () =>
page.evaluate(() => {
const r = window.rl,
layer = r.scene.getObjectByName("body-visuals"),
rig = layer.parent;
rig.updateMatrixWorld(true);
let maxFrameError = 0,
maxBoundsError = 0,
maps = 0,
colorMaps = 0,
meshes = 0,
hidden = 0;
const bounds = (objects, frame) => {
const lo = [Infinity, Infinity, Infinity],
hi = [-Infinity, -Infinity, -Infinity];
const inverse = frame.matrixWorld.clone().invert(),
vector = frame.position.clone();
for (const mesh of objects) {
const transform = inverse.clone().multiply(mesh.matrixWorld),
attr = mesh.geometry.attributes.position;
for (let i = 0; i < attr.count; i++) {
vector.fromBufferAttribute(attr, i).applyMatrix4(transform);
const v = vector.toArray();
for (let axis = 0; axis < 3; axis++) {
lo[axis] = Math.min(lo[axis], v[axis]);
hi[axis] = Math.max(hi[axis], v[axis]);
}
}
}
return [...lo, ...hi];
};
for (const frame of layer.children) {
const body = frame.name.slice("body-visual:".length),
id = r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
body,
);
const p = Array.from(r.data.xpos.slice(id * 3, id * 3 + 3));
const q = Array.from(r.data.xquat.slice(id * 4, id * 4 + 4));
maxFrameError = Math.max(
maxFrameError,
...p.map((v, i) => Math.abs(v - frame.position.toArray()[i])),
...[q[1], q[2], q[3], q[0]].map((v, i) =>
Math.abs(v - frame.quaternion.toArray()[i]),
),
);
const originals = rig.children.filter(
(mesh) =>
mesh.userData.geomId !== undefined &&
r.model.geom_bodyid[mesh.userData.geomId] === id,
);
const replacements = [];
frame.traverse((object) => {
if (!object.isMesh) return;
meshes++;
replacements.push(object);
const materials = [object.material].flat();
maps += materials.filter(
(material) => material.normalMap?.image?.width > 0,
).length;
colorMaps += materials.filter(
(material) => material.map?.image?.width > 0,
).length;
});
hidden += originals.filter((mesh) => !mesh.visible).length;
const oldBounds = bounds(originals, frame),
newBounds = bounds(replacements, frame);
maxBoundsError = Math.max(
maxBoundsError,
...oldBounds.map((v, i) => Math.abs(v - newBounds[i])),
);
}
return {
count: layer.children.length,
maps,
colorMaps,
meshes,
hidden,
maxFrameError,
maxBoundsError,
obs: r.buildObs().length,
};
});
const initial = await inspect();
console.log("Microban visual alignment", initial);
expect(initial.count).toBe(20);
expect(initial.maps).toBeGreaterThanOrEqual(20);
expect(initial.colorMaps).toBeGreaterThanOrEqual(1);
expect(initial.hidden).toBeGreaterThan(20);
expect(initial.maxFrameError).toBeLessThan(1e-12);
expect(initial.maxBoundsError).toBeLessThan(0.0031);
expect(initial.obs).toBe(63);
// Change root yaw and every hinge, including head: no bind-pose-only rig.
await page.evaluate(() => {
const r = window.rl;
r.data.qpos[3] = Math.cos(0.4);
r.data.qpos[6] = Math.sin(0.4);
for (let i = 7; i < r.model.nq; i++) r.data.qpos[i] += 0.12;
r.mujoco.mj_forward(r.model, r.data);
});
await page.waitForTimeout(150);
const articulated = await inspect();
expect(articulated.maxFrameError).toBeLessThan(1e-12);
expect(articulated.maxBoundsError).toBeLessThan(0.0031);
await page.evaluate(() => window.rl.resetSim());
await page.waitForTimeout(150);
// Front 3/4 view for a useful visual acceptance artifact.
await page.evaluate(() => {
window.rl.camera.position.set(0.58, 0.38, 0.58);
window.rl.controls.target.set(0, 0.17, 0);
window.rl.controls.update();
});
await page.waitForTimeout(200);
const screenshot = info.outputPath("textured-microban.png");
await page.screenshot({ path: screenshot });
await info.attach("textured-microban.png", {
path: screenshot,
contentType: "image/png",
});
expect(errors).toEqual([]);
});
test("failed visual candidate rolls back without hiding the old robot; retry loads textures", async ({
page,
}) => {
await boot(page);
const before = await page.evaluate(
() => window.__gameApi.diagnostics().sceneChildren,
);
let fail = true;
await page.route(
"**/robots/microban/visuals/LINK10__visual_0__LINK10.glb",
(route) =>
fail
? route.fulfill({ status: 404, body: "missing visual" })
: route.continue(),
);
expect(
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
).toBe(false);
expect(await page.evaluate(() => window.__store.getState().robotId)).toBe(
"microduck-legs",
);
expect(
await page.evaluate(() => window.__gameApi.diagnostics().sceneChildren),
).toBe(before);
expect(
await page.evaluate(
() => window.rl.scene.getObjectByName("body-visuals") === undefined,
),
).toBe(true);
fail = false;
expect(
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
).toBe(true);
expect(
await page.evaluate(
() => window.rl.scene.getObjectByName("body-visuals").children.length,
),
).toBe(20);
});
test("visual replacement leaves deterministic physics and ONNX actions unchanged", async ({
page,
}) => {
await boot(page);
let textured = false;
await page.route("**/robots/microban/robot.json", async (route) => {
const response = await route.fetch(),
config = await response.json();
if (!textured) delete config.render.body_visuals;
await route.fulfill({ json: config });
});
async function run() {
expect(
await page.evaluate(() => window.__gameApi.switchRobot("microban")),
).toBe(true);
return page.evaluate(async () => {
const r = window.rl;
await r.resetSim();
await r.step(80);
return {
qpos: Array.from(r.data.qpos),
qvel: Array.from(r.data.qvel),
action: Array.from(r.lastAction),
};
});
}
const white = await run();
expect(
await page.evaluate(() => window.__gameApi.switchRobot("microduck-legs")),
).toBe(true);
textured = true;
const visual = await run();
expect(visual).toEqual(white);
});
+1 -1
View File
@@ -26,7 +26,7 @@ for (const scenario of [
world.boundary.half_extent = 1.5;
await route.fulfill({ response, json: world });
});
const errors = await bootDuck(page);
const errors = await bootDuck(page, { legacyPhysics: true });
await page.evaluate(async (scenario) => {
const r = window.rl;
if (scenario === "rollers" || scenario === "crouch") {
+5 -5
View File
@@ -11,7 +11,7 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await ready(page);
await page.getByRole("button", { name: "Lavender colours" }).click();
await page.getByRole("button", { name: "Lavender scene lines" }).click();
await page.getByRole("button", { name: "Rollers", exact: true }).click();
await page.waitForFunction(() => window.rl?.loco === "rollers");
await page.getByLabel("Robot", { exact: true }).selectOption("microban");
@@ -26,9 +26,9 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
await expect(
page.getByText("Toggle head rotation", { exact: false }),
).toBeVisible();
await expect(page.getByRole("group", { name: "Duck colours" })).toHaveCount(
0,
);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
expect(await page.evaluate(() => window.rl.buildObs().length)).toBe(63);
await page.getByLabel("Robot", { exact: true }).selectOption("microduck");
await page.waitForFunction(
@@ -37,7 +37,7 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
window.rl?.loco === "rollers",
);
await expect(
page.getByRole("button", { name: "Lavender colours" }),
page.getByRole("button", { name: "Lavender scene lines" }),
).toHaveAttribute("aria-pressed", "true");
expect(await page.evaluate(() => window.rl.data.time)).toBeLessThan(3);
expect(errors).toEqual([]);
+274
View File
@@ -0,0 +1,274 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
async function boot(page) {
await bootDuck(page);
await page.evaluate(() => window.__gameApi.pause());
}
test("all 63 RP1 material parts follow 25 physical bodies, including fixed fingers, with original shapes aligned", async ({
page,
}, info) => {
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await boot(page);
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
true,
);
const inspect = () =>
page.evaluate(async () => {
const r = window.rl,
layer = r.scene.getObjectByName("body-visuals"),
rig = layer.parent;
rig.updateMatrixWorld(true);
const xml = new DOMParser().parseFromString(
await (await fetch("robots/rp1/visuals/rp1_hand_fixed.urdf")).text(),
"text/xml",
);
const joints = new Map(
[...xml.querySelectorAll("joint")].map((j) => [
j.querySelector("child").getAttribute("link"),
j,
]),
);
const Matrix = rig.matrix.constructor,
Euler = r.controls.object.rotation.constructor;
const correction = new Matrix().makeRotationX(Math.PI / 2);
const urdfRelative = (name, anchor) => {
const result = new Matrix();
while (name !== anchor) {
const joint = joints.get(name),
origin = joint.querySelector("origin");
const xyz = (origin?.getAttribute("xyz") ?? "0 0 0")
.split(/\s+/)
.map(Number);
const angles = (origin?.getAttribute("rpy") ?? "0 0 0")
.split(/\s+/)
.map(Number);
const matrix = new Matrix().makeRotationFromEuler(
new Euler(...angles, "ZYX"),
);
matrix.setPosition(...xyz);
result.premultiply(matrix);
name = joint.querySelector("parent").getAttribute("link");
}
return result.multiply(correction);
};
const bounds = (objects, frame) => {
const lo = [Infinity, Infinity, Infinity],
hi = [-Infinity, -Infinity, -Infinity];
const inverse = frame.matrixWorld.clone().invert(),
vector = frame.position.clone();
for (const mesh of objects) {
const transform = inverse.clone().multiply(mesh.matrixWorld),
attr = mesh.geometry.attributes.position;
for (let i = 0; i < attr.count; i++) {
vector.fromBufferAttribute(attr, i).applyMatrix4(transform);
vector.toArray().forEach((v, a) => {
lo[a] = Math.min(lo[a], v);
hi[a] = Math.max(hi[a], v);
});
}
}
return [...lo, ...hi];
};
let frameError = 0,
boundsError = 0,
handError = 0,
handParts = 0,
normals = 0,
screens = 0,
emissive = 0,
hidden = 0,
meshes = 0;
for (const frame of layer.children) {
const body = frame.name.slice("body-visual:".length),
id = r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
body,
);
const p = Array.from(r.data.xpos.slice(id * 3, id * 3 + 3)),
q = Array.from(r.data.xquat.slice(id * 4, id * 4 + 4));
frameError = Math.max(
frameError,
...p.map((v, i) => Math.abs(v - frame.position.toArray()[i])),
...[q[1], q[2], q[3], q[0]].map((v, i) =>
Math.abs(v - frame.quaternion.toArray()[i]),
),
);
const originals = rig.children.filter(
(m) =>
m.userData.geomId !== undefined &&
r.model.geom_bodyid[m.userData.geomId] === id,
);
hidden += originals.filter((m) => !m.visible).length;
const replacements = [];
frame.traverse((node) => {
if (node.userData.sourcePart) {
handParts++;
const actual = frame.matrixWorld
.clone()
.invert()
.multiply(node.matrixWorld);
const expected = urdfRelative(
node.userData.sourcePart,
`${body.split("_")[0]}_wrist_roll_link`,
);
handError = Math.max(
handError,
...actual.elements.map((v, i) =>
Math.abs(v - expected.elements[i]),
),
);
}
if (!node.isMesh) return;
meshes++;
replacements.push(node);
for (const m of [node.material].flat()) {
if (m.normalMap?.image?.width > 0) normals++;
if (m.map?.image?.width > 0) screens++;
if (m.emissiveIntensity > 1) emissive++;
}
});
if (!body.endsWith("_wrist_link")) {
const before = bounds(originals, frame),
after = bounds(replacements, frame);
boundsError = Math.max(
boundsError,
...before.map((v, i) => Math.abs(v - after[i])),
);
}
}
return {
bodies: layer.children.length,
frameError,
boundsError,
handError,
handParts,
normals,
screens,
emissive,
hidden,
meshes,
obs: r.buildObs().length,
actions: r.lastAction.length,
actuators: r.model.nu,
};
});
const initial = await inspect();
console.log("RP1 GLB alignment", initial);
expect(initial.bodies).toBe(25);
expect(initial.handParts).toBe(40);
expect(initial.frameError).toBeLessThan(1e-12);
expect(initial.boundsError).toBeLessThan(1e-7);
expect(initial.handError).toBeLessThan(1e-10);
expect(initial.normals).toBeGreaterThanOrEqual(63);
expect(initial.screens).toBeGreaterThanOrEqual(1);
expect(initial.emissive).toBeGreaterThanOrEqual(1);
expect(initial.hidden).toBe(25);
expect(initial.obs).toBe(243);
expect(initial.actions).toBe(24);
expect(initial.actuators).toBe(24);
await page.evaluate(() => {
const r = window.rl;
r.data.qpos[3] = Math.cos(0.4);
r.data.qpos[6] = Math.sin(0.4);
for (let j = 0; j < r.model.njnt; j++)
if (r.model.jnt_type[j] === 3)
r.data.qpos[r.model.jnt_qposadr[j]] += 0.12;
r.mujoco.mj_forward(r.model, r.data);
});
await page.waitForTimeout(200);
const posed = await inspect();
expect(posed.frameError).toBeLessThan(1e-12);
expect(posed.boundsError).toBeLessThan(1e-7);
expect(posed.handError).toBeLessThan(1e-10);
await page.evaluate(() => window.rl.resetSim());
await page.getByRole("button", { name: /Keyboard controls/ }).click();
await page.mouse.click(950, 750, { button: "right" });
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(true);
r.camera.position.set(2.4, 1.35, 1.3);
r.controls.target.set(0, 0.7, 0);
r.controls.update();
});
await page.waitForTimeout(300);
await page.screenshot({ path: info.outputPath("rp1-material-front.png") });
expect(errors).toEqual([]);
});
test("RP1 GLBs change no physics, observations or ONNX actions for walking, wave and dance", async ({
page,
}) => {
await boot(page);
let textured = false;
await page.route("**/robots/rp1/robot.json*", async (route) => {
const response = await route.fetch(),
config = await response.json();
if (!textured) delete config.render.body_visuals;
await route.fulfill({ response, json: config });
});
async function run() {
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
true,
);
return page.evaluate(async () => {
const r = window.rl,
result = [];
for (const scenario of ["walk", "wave", "dance"]) {
await r.resetSim();
await r.step(100);
if (scenario === "walk") r.command.set([0.4, 0, 0.15]);
else r.policyState.request(scenario, r.data.time);
for (let i = 0; i < 5; i++) {
await r.step(50);
result.push({
scenario,
time: r.data.time,
qpos: Array.from(r.data.qpos),
qvel: Array.from(r.data.qvel),
ctrl: Array.from(r.data.ctrl),
action: Array.from(r.lastAction),
obs: Array.from(r.buildObs()),
state: r.policyState.state,
reference: r.referenceFrame,
});
}
}
return result;
});
}
const old = await run();
await page.evaluate(() => window.__gameApi.switchRobot("microduck-legs"));
textured = true;
expect(await run()).toEqual(old);
});
test("a partial RP1 material load failure releases the candidate and retry succeeds", async ({
page,
}) => {
await boot(page);
let fail = true;
await page.route(
"**/robots/rp1/visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
(route) =>
fail ? route.fulfill({ status: 404, body: "missing" }) : route.continue(),
);
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
false,
);
await expect(page.getByRole("alert")).toContainText("404");
expect(await page.evaluate(() => window.__store.getState().robotId)).toBe(
"microduck-legs",
);
expect(await page.evaluate(() => window.rl.model.isDeleted())).toBe(false);
fail = false;
expect(await page.evaluate(() => window.__gameApi.retry())).toBe(true);
expect(
await page.evaluate(
() => window.rl.scene.getObjectByName("body-visuals").children.length,
),
).toBe(25);
});
+12 -3
View File
@@ -3,7 +3,9 @@ import { test, expect } from "@playwright/test";
test("20 robot transitions release old WASM objects and plateau scene/GPU/listeners", async ({
page,
}, info) => {
test.setTimeout(180_000);
// Twenty cold transitions now also decode RP1's ~93 MB textured GLBs.
// Keep allocation/listener assertions unchanged; allow load-time headroom.
test.setTimeout(240_000);
await page.addInitScript(() => {
const add = EventTarget.prototype.addEventListener,
remove = EventTarget.prototype.removeEventListener,
@@ -57,6 +59,9 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
dataDeleted: data.isDeleted(),
...window.__gameApi.diagnostics(),
listeners: window.listenerCount(),
toyGeometries:
window.rl.scene.getObjectByName("interactive-props")?.children
.length ?? 0,
};
}, id);
expect(sample.success).toBe(true);
@@ -70,8 +75,12 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
for (const samples of [duck, ban, rp1]) {
expect(samples.at(-1).listeners).toBe(samples[0].listeners);
expect(samples.at(-1).sceneChildren).toBe(samples[0].sceneChildren);
expect(samples.at(-1).memory.geometries).toBeLessThanOrEqual(
samples[1].memory.geometries + 1,
// Each runtime owns 5–7 independent toy geometries. Compare the fixed
// scene allocation after removing this intentionally variable count.
expect(
samples.at(-1).memory.geometries - samples.at(-1).toyGeometries,
).toBeLessThanOrEqual(
samples[1].memory.geometries - samples[1].toyGeometries + 1,
);
expect(samples.at(-1).memory.textures).toBeLessThanOrEqual(
samples[1].memory.textures + 1,
+255
View File
@@ -0,0 +1,255 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("bright scene has 5–7 free-body toys, no football, and four original-style line colour buttons", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(async () => {
const r = window.rl;
r.setPropsParked(false);
await r.step(60);
r.render();
});
const initial = await page.evaluate(() => {
const r = window.rl;
return {
toys: r.interactiveProps.map((p) => ({
type: p.type,
dof: p.dofAdr,
mass: r.model.body_mass[p.bodyId],
})),
meshes: r.scene.getObjectByName("interactive-props").children.length,
ballId: r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
),
hemisphere: r.scene.parent.children.some(
(o) => o.isHemisphereLight && o.intensity >= 0.7,
),
environment: r.scene.parent.environmentIntensity,
robotMaterials: [...r.scene.children]
.filter((o) => o.name !== "interactive-props")
.flatMap((o) => {
const result = [];
o.traverse((m) => {
if (m.isMesh && m.material?.color)
result.push([m.uuid, m.material.color.toArray()]);
});
return result;
}),
};
});
expect(initial.toys.length).toBeGreaterThanOrEqual(5);
expect(initial.toys.length).toBeLessThanOrEqual(7);
expect(initial.meshes).toBe(initial.toys.length);
expect(new Set(initial.toys.map((p) => p.type)).size).toBe(4);
expect(initial.toys.every((p) => p.mass > 0 && p.dof >= 0)).toBe(true);
expect(initial.ballId).toBe(-1);
expect(initial.hemisphere).toBe(true);
expect(initial.environment).toBeGreaterThanOrEqual(0.7);
const group = page.getByRole("group", { name: "Scene line colours" });
await expect(group.getByRole("button")).toHaveCount(4);
for (const [label, hex] of [
["Cream", "f7e6cb"],
["Graphite", "6c6a68"],
["Lavender", "bfa9cf"],
["Sky", "a9dbe8"],
]) {
const button = group.getByRole("button", { name: `${label} scene lines` });
await button.click();
await expect(button).toHaveAttribute("aria-pressed", "true");
const values = await page.evaluate(() => {
const r = window.rl,
lines = [],
robotMaterials = [];
r.scene.traverse((o) => {
if (o.material?.uniforms?.uSectionColor)
lines.push(o.material.uniforms.uSectionColor.value.getHexString());
});
for (const o of r.scene.children.filter(
(o) => o.name !== "interactive-props",
))
o.traverse((m) => {
if (m.isMesh && m.material?.color)
robotMaterials.push([m.uuid, m.material.color.toArray()]);
});
const style = getComputedStyle(
document.querySelector('[aria-label="Scene line colours"] button'),
);
return {
lines,
robotMaterials,
radius: style.borderRadius,
width: style.width,
height: style.height,
};
});
expect(values.lines).toEqual(Array(5).fill(hex));
expect(values.robotMaterials).toEqual(initial.robotMaterials);
expect(values.radius).toBe("0px");
expect(values.width).toBe(values.height);
}
await page.evaluate(() => {
const r = window.rl;
r.chaseCam = false;
r.camera.position.set(-0.2, 1.9, 2.7);
r.controls.target.set(0.1, 0.05, 0);
r.controls.update();
});
await page.screenshot({
path: "test-results/scene-interaction-overview.png",
});
expect(errors).toEqual([]);
});
test("geometry can be dragged with physical force; pause/reset clear grab and restore layout", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(false);
r.chaseCam = false;
const spec = r.interactiveProps[0],
q = r.data.qpos;
q.set([-0.25, 0, spec.pos[2]], spec.qAdr);
r.mujoco.mj_forward(r.model, r.data);
r.camera.position.set(-0.6, 0.65, 1.2);
r.controls.target.set(q[0], q[2], -q[1]);
r.controls.update();
r.camera.updateMatrixWorld();
r.render();
});
await page.waitForTimeout(250);
const point = await page.evaluate(() => {
const r = window.rl,
spec = r.interactiveProps[0],
q = r.data.qpos;
const p = r.controls.target
.clone()
.set(q[spec.qAdr], q[spec.qAdr + 2], -q[spec.qAdr + 1])
.project(r.camera);
const rect = document.querySelector("canvas").getBoundingClientRect();
return {
x: ((p.x + 1) * rect.width) / 2,
y: ((1 - p.y) * rect.height) / 2,
};
});
await page.mouse.move(point.x, point.y);
await page.mouse.down();
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(false);
await page.mouse.move(point.x + 90, point.y - 40, { steps: 6 });
const result = await page.evaluate(async () => {
const r = window.rl,
spec = r.interactiveProps[0];
await r.step(15);
return {
x: r.data.qpos[spec.qAdr],
force: Array.from(
r.data.xfrc_applied.slice(spec.bodyId * 6, spec.bodyId * 6 + 3),
),
};
});
expect(Math.abs(result.x + 0.25)).toBeGreaterThan(0.01);
expect(result.force.some((v) => Math.abs(v) > 0.001)).toBe(true);
await page.evaluate(() => window.__gameApi.pause());
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(true);
await page.mouse.up();
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).every((v) => v === 0),
),
).toBe(true);
await page.evaluate(() => window.__gameApi.reset());
expect(
await page.evaluate(() => {
const r = window.rl;
return (
r.interactiveProps.every((p) =>
p.pos.every((v, i) => r.data.qpos[p.qAdr + i] === v),
) && r.controls.enabled
);
}),
).toBe(true);
expect(errors).toEqual([]);
});
test("toys transfer collision momentum, collide with the robot and floor, and survive a mode switch", async ({
page,
}) => {
const errors = await bootDuck(page);
const collision = await page.evaluate(async () => {
const r = window.rl;
r.setPropsParked(false);
const specs = r.interactiveProps;
// Keep unrelated toys clear of a two-body collision lane.
specs.forEach((p, i) => {
r.data.qpos.set([-1 + i * 0.3, -1.1, p.pos[2]], p.qAdr);
});
const a = specs[0],
b = specs[1];
r.data.qpos.set([0.1, 0.7, a.pos[2]], a.qAdr);
r.data.qpos.set([0.35, 0.7, b.pos[2]], b.qAdr);
r.data.qvel[a.dofAdr] = 1.8;
r.mujoco.mj_forward(r.model, r.data);
await r.step(15);
const moved = r.data.qpos[b.qAdr] - 0.35;
// Launch a toy at the duck. A non-colliding body would cross x=-0.6.
r.resetSim();
const p = r.interactiveProps.find((s) => s.type === "sphere");
r.data.qpos.set([-0.35, 0, 0.09], p.qAdr);
r.data.qvel[p.dofAdr] = -1.5;
r.mujoco.mj_forward(r.model, r.data);
let contacts = 0,
minX = 10;
for (let i = 0; i < 15; i++) {
await r.step(1);
contacts = Math.max(contacts, r.data.ncon);
minX = Math.min(minX, r.data.qpos[p.qAdr]);
}
await r.step(100);
const half = r.world.boundary.half_extent;
r.data.qpos.set([half - 0.2, 0.9, p.pos[2]], p.qAdr);
r.data.qvel.fill(0, p.dofAdr, p.dofAdr + 6);
r.data.qvel[p.dofAdr] = 1.5;
r.mujoco.mj_forward(r.model, r.data);
let wallMax = -Infinity;
for (let i = 0; i < 15; i++) {
await r.step(1);
wallMax = Math.max(wallMax, r.data.qpos[p.qAdr]);
}
return {
moved,
contacts,
minX,
wallMax,
half,
finite: Array.from(r.data.qpos).every(Number.isFinite),
aboveFloor: r.interactiveProps.every(
(s) => r.data.qpos[s.qAdr + 2] > 0.025,
),
};
});
expect(collision.moved).toBeGreaterThan(0.025);
expect(collision.contacts).toBeGreaterThan(0);
expect(collision.minX).toBeGreaterThan(-0.58);
expect(collision.wallMax).toBeLessThan(collision.half - 0.03);
expect(collision.aboveFloor).toBe(true);
expect(collision.finite).toBe(true);
await page.evaluate(() => window.rl.setLoco("rollers", { force: true }));
const switched = await page.evaluate(async () => {
const r = window.rl;
await r.step(100);
r.render();
return {
count: r.interactiveProps.length,
meshes: r.scene.getObjectByName("interactive-props").children.length,
finite: Array.from(r.data.qpos).every(Number.isFinite),
};
});
expect(switched.count).toBe(switched.meshes);
expect(switched.finite).toBe(true);
expect(errors).toEqual([]);
});
+68
View File
@@ -0,0 +1,68 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("two simulated minutes of movement with physical toys stay finite in feet and rollers; escape recovery is local", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(false);
// A deterministic controller source exercises actual commanded motion,
// independent of frame-rate keyboard polling during manual stepping.
r.controller.addSource({
id: "stability-drive",
isActive: () => true,
command: new Float32Array([0.25, 0, 0.4]),
});
});
for (const mode of ["legs", "rollers"]) {
await page.evaluate(async (mode) => {
const r = window.rl;
await r.setLoco(mode, { force: true });
r.controller.sources
.find((s) => s.id === "stability-drive")
.command.set(mode === "legs" ? [0.25, 0, 0.4] : [0.4, 0, 0.15]);
}, mode);
let moved = false;
for (let chunk = 0; chunk < 30; chunk++) {
const sample = await page.evaluate(async () => {
const r = window.rl;
await r.step(100);
return {
finite: [r.data.qpos, r.data.qvel, r.data.ctrl].every((array) =>
Array.from(array).every(Number.isFinite),
),
moved: Math.hypot(r.data.qpos[0] + 0.6, r.data.qpos[1]) > 0.04,
inside: r.interactiveProps.every(
(p) =>
Math.abs(r.data.qpos[p.qAdr]) <
r.world.boundary.half_extent + 0.11 &&
Math.abs(r.data.qpos[p.qAdr + 1]) <
r.world.boundary.half_extent + 0.11,
),
};
});
expect(sample.finite).toBe(true);
expect(sample.inside).toBe(true);
moved ||= sample.moved;
}
expect(moved).toBe(true);
}
const recovery = await page.evaluate(async () => {
const r = window.rl,
p = r.interactiveProps[0];
r.data.qpos[p.qAdr] = r.world.boundary.half_extent + 1;
r.mujoco.mj_forward(r.model, r.data);
await r.step(1);
return {
actual: Array.from(r.data.qpos.slice(p.qAdr, p.qAdr + 3)),
expected: p.pos,
count: r.interactiveProps.length,
};
});
expect(recovery.actual).toEqual(recovery.expected);
expect(recovery.count).toBeGreaterThanOrEqual(5);
expect(recovery.count).toBeLessThanOrEqual(7);
expect(errors).toEqual([]);
});
+157
View File
@@ -0,0 +1,157 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("all robots share the same scaled random layout, lighting, line palette and global colour selection", async ({
page,
}) => {
const errors = await bootDuck(page);
let reference;
let expected = { name: "Sky", key: "blue", hex: "a9dbe8" };
for (const id of ["microduck-legs", "microban", "rp1", "microduck-legs"]) {
if ((await page.evaluate(() => window.__store.getState().robotId)) !== id)
await page.evaluate((id) => window.__gameApi.switchRobot(id), id);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
if (!reference)
await page.getByRole("button", { name: "Sky scene lines" }).click();
const state = await page.evaluate(() => {
const r = window.rl,
scale = r.world.boundary.half_extent / 2.25;
const lines = [];
r.scene.traverse((o) => {
if (o.material?.uniforms?.uSectionColor)
lines.push(o.material.uniforms.uSectionColor.value.getHexString());
});
return {
specs: r.interactiveProps.map((p) => ({
name: p.name,
type: p.type,
color: p.color,
pos: p.pos.map((v) => v / scale),
size: p.size.map((v) => v / scale),
})),
count: r.scene.getObjectByName("interactive-props").children.length,
lines,
lineColor: window.__store.getState().lineColor,
lights: r.scene.parent.children
.filter((o) => o.isLight)
.map((o) => [o.type, o.intensity, o.color.getHex()]),
environment: r.scene.parent.environmentIntensity,
drag: window.__store.getState().runtimeCapabilities.physicalDrag,
};
});
expect(state.count).toBeGreaterThanOrEqual(5);
expect(state.count).toBeLessThanOrEqual(7);
expect(state.lines).toEqual(Array(5).fill(expected.hex));
expect(state.lineColor).toBe(expected.key);
expect(state.drag).toBe(true);
if (!reference) reference = state;
state.specs.forEach((p, i) => {
expect(p.type).toBe(reference.specs[i].type);
expect(p.color).toBe(reference.specs[i].color);
p.pos.forEach((v, j) =>
expect(v).toBeCloseTo(reference.specs[i].pos[j], 12),
);
p.size.forEach((v, j) =>
expect(v).toBeCloseTo(reference.specs[i].size[j], 12),
);
});
expect(state.lights).toEqual(reference.lights);
expect(state.environment).toBe(reference.environment);
await expect(
page.getByRole("button", { name: `${expected.name} scene lines` }),
).toHaveAttribute("aria-pressed", "true");
await expect(
page.getByRole("button", { name: "Rollers", exact: true }),
).toHaveCount(id === "microduck-legs" ? 1 : 0);
// Also carry a choice made in another robot back to Microduck.
if (id === "rp1") {
await page.getByRole("button", { name: "Lavender scene lines" }).click();
expected = { name: "Lavender", key: "purple", hex: "bfa9cf" };
}
}
expect(errors).toEqual([]);
});
for (const id of ["microban", "rp1"]) {
test(`${id}: shared toys collide with walls, drag with real force and release on pause`, async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate((id) => window.__gameApi.switchRobot(id), id);
const wall = await page.evaluate(async () => {
const r = window.rl,
scale = r.world.boundary.half_extent / 2.25,
p = r.interactiveProps.find((s) => s.type === "sphere"),
half = r.world.boundary.half_extent;
r.data.qpos.set([half - 0.2 * scale, 0.9 * scale, p.pos[2]], p.qAdr);
r.data.qvel[p.dofAdr] = 1.5 * scale;
r.mujoco.mj_forward(r.model, r.data);
let max = -Infinity;
for (let i = 0; i < 15; i++) {
await r.step(1);
max = Math.max(max, r.data.qpos[p.qAdr]);
}
await r.resetSim();
return { max, limit: half - 0.025 * scale };
});
expect(wall.max).toBeLessThan(wall.limit);
await page.getByRole("button", { name: /Keyboard controls/ }).click();
await page.evaluate(() => {
const r = window.rl,
scale = r.world.scale ?? 1,
p = r.interactiveProps[0],
q = r.data.qpos;
q.set([0.25 * scale, -0.3 * scale, p.pos[2]], p.qAdr);
r.mujoco.mj_forward(r.model, r.data);
r.camera.position.set(q[0], q[2] + 0.6 * scale, 1.5 * scale);
r.controls.target.set(q[0], q[2], -q[1]);
r.controls.update();
window.__gameApi.resume();
});
await page.waitForTimeout(200);
const point = await page.evaluate(() => {
const r = window.rl,
p = r.interactiveProps[0],
q = r.data.qpos,
rect = document.querySelector("canvas").getBoundingClientRect();
const v = r.controls.target
.clone()
.set(q[p.qAdr], q[p.qAdr + 2], -q[p.qAdr + 1])
.project(r.camera);
return {
x: ((v.x + 1) * rect.width) / 2,
y: ((1 - v.y) * rect.height) / 2,
};
});
await page.mouse.move(point.x, point.y);
await page.mouse.down();
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(false);
await page.mouse.move(point.x + 70, point.y - 35, { steps: 5 });
await page.waitForTimeout(200);
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).some((v) => Math.abs(v) > 1e-5),
),
).toBe(true);
await page.evaluate(() => window.__gameApi.pause());
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(true);
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).every((v) => v === 0),
),
).toBe(true);
await page.mouse.up();
await page.evaluate(() => window.__gameApi.reset());
expect(
await page.evaluate(() => {
const r = window.rl;
return r.interactiveProps.every((p) =>
p.pos.every((v, i) => r.data.qpos[p.qAdr + i] === v),
);
}),
).toBe(true);
expect(errors).toEqual([]);
});
}
+2 -2
View File
@@ -1,5 +1,5 @@
import { expect } from "@playwright/test";
export async function bootDuck(page) {
export async function bootDuck(page, { legacyPhysics = false } = {}) {
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await page.addInitScript(() => {
@@ -7,7 +7,7 @@ export async function bootDuck(page) {
Math.random = () =>
(seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
});
await page.goto("/?boot=1&test=1");
await page.goto(`/?boot=1&test=1${legacyPhysics ? "&oracle=legacy" : ""}`);
await page.waitForFunction(
() => window.rl?.prepareTest || window.__store?.getState().bootFailed,
);
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "microduck-sandbox",
"version": "0.3.0",
"version": "0.5.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "microduck-sandbox",
"version": "0.3.0",
"version": "0.5.0",
"dependencies": {
"@emotion/react": "^11.14.0",
"@emotion/styled": "^11.14.1",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "microduck-sandbox",
"private": true,
"version": "0.3.0",
"version": "0.5.0",
"type": "module",
"scripts": {
"dev": "vite",
+8
View File
@@ -11,4 +11,12 @@ The precise origin remote, HEAD, dirty-worktree flag, per-file SHA-256 and behav
Browser adaptations (not edits to the copied XML): 0.005 s physics timestep; separate flat ground; explicit 18-joint policy order excluding head; body-frame gravity and IMU-site gyroscope following source observation semantics; limited joint-PD torque control instead of BAM. Default PD kp=0.3 Nm/rad, kd=0.02 Nm·s/rad, torque limit=0.65 Nm are experimentally selected simplified parameters, not metadata gains and not a claim of hardware equivalence.
## Textured visual replacement
`visuals/*.glb` are the 20 files copied byte-for-byte from the user-supplied `microban2.zip`, replacing the earlier `microban.zip` visual export, not from the upstream checkout above. They retain their embedded PBR materials and 46 textures (45 normal maps and one screen base-colour image, all 1024 × 1024). The archive contains no license notice; **confirm public redistribution rights with the asset owner** rather than assuming the upstream model license covers these replacements. Archive and file SHA-256 values are recorded separately in `visual-source-manifest.json`.
Reimport with `python3 app/scripts/import-microban-visuals.py /path/to/microban2.zip`. This does not change `model.xml`, the original STL files, policy, original source manifest or the reviewed `render.body_visuals` mapping in `robot.json`.
The GLBs have different local frames from the original MJCF. `robot.json` records fixed, visual-only rigid transforms (metres and xyzw quaternions), checked against the original per-body geometry; body-local bounding-box endpoint differences are at most 3.01 mm, allowing the replacement's added details. Mapping: BASE → trunk; LINK1 → head; LINK2–4 → right arm; LINK5–7 → left arm; LINK8–13 → left leg; LINK14–19 → right leg. Vertex positions and node transforms in all 20 GLBs are unchanged from the previous visual export, so the existing body registration is retained. Runtime follows the existing MuJoCo body poses and hides only the replaced visual geoms; collisions, inertias, joints, policy inputs and actuation are untouched. This is geometric registration, not a claim that the archive supplied authoritative joint-frame metadata.
No Placo, real-robot control, source Python code, CAD project, BAM battery model or communication delays are bundled into the browser runtime.
+113
View File
@@ -315,6 +315,119 @@
},
"render": {
"source": "mjcf_visual",
"body_visuals": [
{
"body": "trunk",
"path": "visuals/BASE__visual_0__BASE.glb",
"quaternion": [0.7071067811865476, 0, 0, 0.7071067811865476]
},
{
"body": "head",
"path": "visuals/LINK1__visual_0__LINK1.glb",
"quaternion": [0.5, -0.5, -0.5, 0.5]
},
{
"body": "shoulder",
"path": "visuals/LINK2__visual_0__LINK2.glb"
},
{
"body": "humerus__configuration_right",
"path": "visuals/LINK3__visual_0__LINK3.glb",
"position": [0, 0, -0.02755],
"quaternion": [0, -0.7071067811865476, 0, 0.7071067811865476]
},
{
"body": "radius",
"path": "visuals/LINK4__visual_0__LINK4.glb",
"position": [-0.00529885, -0.00021476, 0.00145]
},
{
"body": "shoulder_2",
"path": "visuals/LINK5__visual_0__LINK5.glb"
},
{
"body": "humerus__configuration_left",
"path": "visuals/LINK6__visual_0__LINK6.glb",
"position": [-0.00529885, -0.00021476, -0.02755],
"quaternion": [0.7071067811865476, 0, -0.7071067811865476, 0]
},
{
"body": "radius_2",
"path": "visuals/LINK7__visual_0__LINK7.glb",
"position": [-0.00529885, 0.00021476, -0.00145],
"quaternion": [0, 0, 1, 0]
},
{
"body": "hip_2",
"path": "visuals/LINK8__visual_0__LINK8.glb",
"quaternion": [0, 0.7071067811865476, 0.7071067811865476, 0]
},
{
"body": "hip_block__configuration_left",
"path": "visuals/LINK9__visual_0__LINK9.glb",
"position": [0, 0, 0.00027893],
"quaternion": [0.7071067811865476, 0, 0.7071067811865476, 0]
},
{
"body": "femur__configuration_left",
"path": "visuals/LINK10__visual_0__LINK10.glb",
"position": [0, 0, -0.00145],
"quaternion": [1, 0, 0, 0]
},
{
"body": "tibia__configuration_left",
"path": "visuals/LINK11__visual_0__LINK11.glb",
"position": [0, 0, -0.00145],
"quaternion": [1, 0, 0, 0]
},
{
"body": "ankle_block__configuration_right",
"path": "visuals/LINK12__visual_0__LINK12.glb",
"position": [0, 0, -0.00145],
"quaternion": [0, 1, 0, 0]
},
{
"body": "foot_2",
"path": "visuals/LINK13__visual_0__LINK13.glb",
"position": [0, 0, 0.0008],
"quaternion": [0, -0.7071067811865476, 0, 0.7071067811865476]
},
{
"body": "hip",
"path": "visuals/LINK14__visual_0__LINK14.glb",
"quaternion": [0.7071067811865476, 0, 0, 0.7071067811865476]
},
{
"body": "hip_block__configuration_right",
"path": "visuals/LINK15__visual_0__LINK15.glb",
"position": [0, 0, 0.0008],
"quaternion": [0.7071067811865476, 0, 0.7071067811865476, 0]
},
{
"body": "femur__configuration_right",
"path": "visuals/LINK16__visual_0__LINK16.glb",
"position": [0, 0, -0.02755],
"quaternion": [0, 1, 0, 0]
},
{
"body": "tibia__configuration_right",
"path": "visuals/LINK17__visual_0__LINK17.glb",
"position": [0, 0, -0.02745],
"quaternion": [0, 1, 0, 0]
},
{
"body": "ankle_block__configuration_left",
"path": "visuals/LINK18__visual_0__LINK18.glb",
"position": [0, 0, -0.02755],
"quaternion": [1, 0, 0, 0]
},
{
"body": "foot",
"path": "visuals/LINK19__visual_0__LINK19.glb",
"position": [0, 0, 0.0008],
"quaternion": [0, -0.7071067811865476, 0, 0.7071067811865476]
}
],
"groups": [
2
],
@@ -0,0 +1,87 @@
{
"source": "User-provided microban2.zip (textured visual replacement)",
"archive_sha256": "dfb20a34103afecdc7996cc5ac9f6a61705560717f9c6aac420f3d6307d3e6d4",
"license": "Not supplied in the archive; confirm redistribution rights with the asset owner.",
"files": [
{
"file": "visuals/BASE__visual_0__BASE.glb",
"sha256": "32cc5d7df7ab62779cb61ae777dafe485fc357489998020fba86e2bb94d968a6"
},
{
"file": "visuals/LINK1__visual_0__LINK1.glb",
"sha256": "33f7776ad30bc04fb0a93a1a80641636d5cca382c12093bf97df9d9cd1416553"
},
{
"file": "visuals/LINK2__visual_0__LINK2.glb",
"sha256": "ffbdd241b96d8e9d371bbddb57908912e95edb4c7ce88e9e20a79c663f80058f"
},
{
"file": "visuals/LINK3__visual_0__LINK3.glb",
"sha256": "01718a43c6b8936543c6c2e47b00150907c7ac7c892c90f9d612a21d6fc566b9"
},
{
"file": "visuals/LINK4__visual_0__LINK4.glb",
"sha256": "7dad340b731e9f5ed8fe37bc77dd2b7b5ff1ac540cc501dad35b487438ab4e0d"
},
{
"file": "visuals/LINK5__visual_0__LINK5.glb",
"sha256": "aa54506ab2503ac34cb4d8240772798d353f72a04b81224543b9347df38a68ba"
},
{
"file": "visuals/LINK6__visual_0__LINK6.glb",
"sha256": "2bed471a7c3e51107d5307130c342696b3e986384584c4891b4b7a161380aeb4"
},
{
"file": "visuals/LINK7__visual_0__LINK7.glb",
"sha256": "909cc20e48dad6dd8478c8dd19499ffc49c8427305ead62e1b267ae8a3939d9e"
},
{
"file": "visuals/LINK8__visual_0__LINK8.glb",
"sha256": "d2a43a6606906fd8bd8a982208ea2121245969bded44a3f81ee074e1968c5abf"
},
{
"file": "visuals/LINK9__visual_0__LINK9.glb",
"sha256": "33f34ba76488c0dfe99bcf80bb48ff3ba0c0240e506a79203d380e20aa17f952"
},
{
"file": "visuals/LINK10__visual_0__LINK10.glb",
"sha256": "4bf30a00a35ec588a74476cefc6cff79b14f16ebf8136a55ab8392c8b2b5a6be"
},
{
"file": "visuals/LINK11__visual_0__LINK11.glb",
"sha256": "04e8b462a92a6689074211bc1243a28bd031148c6f390cf00d9af977e33d4616"
},
{
"file": "visuals/LINK12__visual_0__LINK12.glb",
"sha256": "e1549141dbd462ef4aa31be2f5bb8ae2f31a0d0b49da3de41911ef482c008cd7"
},
{
"file": "visuals/LINK13__visual_0__LINK13.glb",
"sha256": "ded693e8bc53cdda7ac36b5f4d49c56391eaa283f6549872e2eeaa33d0b43afd"
},
{
"file": "visuals/LINK14__visual_0__LINK14.glb",
"sha256": "03f70f51f86a4eb67b069629ed89576cae71910f903d34638dc8fd5fc8b9bb4d"
},
{
"file": "visuals/LINK15__visual_0__LINK15.glb",
"sha256": "87202389c4bc6dade765d805f399f8577add3c2b3a10a317264dbe4e4e99f54d"
},
{
"file": "visuals/LINK16__visual_0__LINK16.glb",
"sha256": "eb8a287ca42899b728973690352974a9351fcbc7a7f9e7e632376f4aebdad3d0"
},
{
"file": "visuals/LINK17__visual_0__LINK17.glb",
"sha256": "1f942e545727ce342c97827f06d33d6ab8186120ea6dbcf36369eb2b4e3e54e5"
},
{
"file": "visuals/LINK18__visual_0__LINK18.glb",
"sha256": "808675acb0fddbbacdbed43ad71de1df7ffd0b898a8bf7814098cbe7d1d1ad29"
},
{
"file": "visuals/LINK19__visual_0__LINK19.glb",
"sha256": "b00cec2f4a0924efd7805f405ab0bb103e1ecdf1d56310caa3c275c7ef0a7c1b"
}
]
}
+12
View File
@@ -30,6 +30,18 @@ Source archives under `robolab/data/motions/rp1_bm/`:
Both skills use exclusive full-body policies, a physical stop gate, entry/return blends and locomotion interruption. Validation includes three initial headings, walking before/after, repeated episodes without robot reset, keyboard/pause/reset, numerical reference checks and malformed-asset rollback. Initial integration: `docs/superpowers/verification/2026-10-29-rp1-completed-skills.md`; current dance weights: `docs/superpowers/verification/2026-10-29-rp1-dance-23998.md`. UI labels are simply `Dance` and `Wave`; durations and transition behavior remain unchanged.
## User-provided GLB material replacement
`RP1机器人glb.zip` supplies 63 Blender-exported, self-contained GLBs. `visual-source-manifest.json` records the archive SHA-256, all 63 source-part hashes, 25 output hashes and the exact body mapping. Original MJCF/STL, policy and motion files retain their existing hashes; the only manifest change is `render.body_visuals`.
- Twenty-three bodies use byte-identical source GLBs. Their Blender Y-up coordinates are rotated +90° about X into MuJoCo body Z-up. All 63 source-part bounds were independently checked against the upstream STL bounds after this axis conversion: exact agreement.
- The existing 24-DOF model has only one `left_wrist_link` / `right_wrist_link` body per side. Each visual wrist therefore composes **20 parts**: wrist roll/yaw/pitch plus the palm and 16 finger pieces. Transforms come from the copied upstream `visuals/rp1_hand_fixed.urdf`; extra wrist yaw/pitch stay neutral, fingers stay in their fixed poses. There are **no additional physical joints or finger-control claims**.
- The assembler preserves source vertex/index/image bytes, remaps GLTF indices and retains original PBR colours, roughness/metalness, normal maps, face screen texture and emissive torso logo. It adds only visual transform wrapper nodes.
- Reproduce with `python3 app/scripts/import-rp1-visuals.py /path/to/RP1机器人glb.zip app/public/robots/rp1/visuals/rp1_hand_fixed.urdf` from the repo root. All inputs/outputs validate before installation; ZIP paths are never extracted.
- Geometry additions in the hands are visual only. Existing collision envelopes intentionally remain those used by the trained 24-DOF model.
The ZIP supplied no license/attribution file. Do not infer redistribution rights from receipt of these assets. It remains a local-evaluation package under the limitations below.
## License limitation — local evaluation only
The RP1 description submodule does not supply a license file in this checkout. Do **not** infer an Apache/BSD license for its meshes/model or motion data from the platform or RoboLab. Confirm upstream rights before redistributing this package or deploying publicly.
+253 -1
View File
@@ -678,7 +678,259 @@
],
"camera": {
"distance": 2.4
}
},
"body_visuals": [
{
"body": "base_link",
"path": "visuals/base_link__visual_0__base_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_pitch_link",
"path": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_roll_link",
"path": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_yaw_link",
"path": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_knee_link",
"path": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_ankle_pitch_link",
"path": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_ankle_roll_link",
"path": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_pitch_link",
"path": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_roll_link",
"path": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_yaw_link",
"path": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_knee_link",
"path": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_ankle_pitch_link",
"path": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_ankle_roll_link",
"path": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "waist_roll_link",
"path": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "waist_yaw_link",
"path": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_pitch_link",
"path": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_roll_link",
"path": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_yaw_link",
"path": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_elbow_link",
"path": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_wrist_link",
"path": "visuals/left_wrist_link__fixed_hand.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_pitch_link",
"path": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_roll_link",
"path": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_yaw_link",
"path": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_elbow_link",
"path": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_wrist_link",
"path": "visuals/right_wrist_link__fixed_hand.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
}
]
},
"presentation": {
"theme": "comic"
@@ -0,0 +1,557 @@
{
"source": "User-provided RP1\u673a\u5668\u4ebaglb.zip",
"archive_sha256": "8610aac8432203bf68d0efe234062c551ca805ebbe72a92d5fc7350730ebdea2",
"license": "Not supplied in the archive; local evaluation only. Confirm redistribution rights.",
"coordinate_conversion": "Source GLBs are Blender Y-up; attachments rotate +90 degrees about X into MuJoCo body Z-up.",
"assembly": "25 physical body attachments; 20 source parts per wrist (neutral extra wrist axes and fixed fingers). No physical joints added.",
"input_files": [
{
"file": "base_link__visual_0__base_link.glb",
"sha256": "3db82d0e5fa1fa48fa6f4fe4460bba14568a39b5933acff8b4b1879f582e06b0"
},
{
"file": "left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"sha256": "d8cb2aff7273185a89eba99aab47ce6bf4716f25e70f34afa1529acb802c6193"
},
{
"file": "left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"sha256": "35361b79fb69096039eb077018c25ae489f16cc5229dcd82256b3a573b168208"
},
{
"file": "left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"sha256": "19fe908274f9fe49b5a8317652e4fe02a6ff93802de87ad7d0383fa1d142b03b"
},
{
"file": "left_knee_link__visual_0__left_knee_link.glb",
"sha256": "1c01f5c03837d24287d79eff48a9e4bd4203d8516d0375592b159d49fb83f6c9"
},
{
"file": "left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"sha256": "4bab717bf7073e28754d1c42f4409141ea775987284ff8575f9c7aef25561e8c"
},
{
"file": "left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"sha256": "76144b6ebf17b2eb5875168cbe6a3c4c12eaeebdd4bd277864887b3894f781b1"
},
{
"file": "right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"sha256": "e299f46bff44d6bbf03592cd6caba3b418fc022260746554e6d2013f31793df4"
},
{
"file": "right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"sha256": "7cb006f9ba7f950c98287b59695938cfd6460d8fa4546ae6c95bc65ed5c6c6b8"
},
{
"file": "right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"sha256": "35393465aa8103ab436ecc3a635cdce7477600cf65eb03a2942af546cc39c79c"
},
{
"file": "right_knee_link__visual_0__right_knee_link.glb",
"sha256": "2a5f1f91f6db0cfac3a0e88038613be89cb94066d7f103876b455823edbfb8f3"
},
{
"file": "right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"sha256": "c43112750efd953f29dfea5d8d2bb925eb847f74b679fa1d22f734b1e3094b91"
},
{
"file": "right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"sha256": "3856c0074bbead9ade091b2bef4df165d9c1d4b03e0c0f1d8c284cddf091a9f6"
},
{
"file": "waist_roll_link__visual_0__waist_roll_link.glb",
"sha256": "a4cd49014951ed5c7836bde4a7138eab4119f0ca7249a69e26add388d5fedf32"
},
{
"file": "waist_yaw_link__visual_0__waist_yaw_link.glb",
"sha256": "cb50b0354f8a0a6d06ff486d46cb849462b84471c20a350b1e4c5a86577ce230"
},
{
"file": "left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"sha256": "9eb76bc0675578e28707120d64cb042b948312ddc9233e4c28de02ee04a3dbf9"
},
{
"file": "left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"sha256": "f72e3ac73f3804acd5e5384c2d10d48aacacb95bf9f98ea0f2426f1b92add0de"
},
{
"file": "left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"sha256": "909f0dbd40db57b223143c1c89c740c3ea636e5d9b066eb6fe26dab37e2f4975"
},
{
"file": "left_elbow_link__visual_0__left_elbow_link.glb",
"sha256": "f98a4446175c7b1d98c5a664fabcb6698635c99ccfc8261dedbef9155e1a2043"
},
{
"file": "left_wrist_roll_link__visual_0__left_wrist_roll_link.glb",
"sha256": "fd4dff45bb8c6caa79ceff1286cf8d8d69de22f781fbbd12a91baf87636e0ad2"
},
{
"file": "left_wrist_yaw_link__visual_0__left_wrist_yaw_link.glb",
"sha256": "65835f9275362e18801a3247191323f0661f538a3c24836cc78fdd1ec5560a3e"
},
{
"file": "left_wrist_pitch_link__visual_0__left_wrist_pitch_link.glb",
"sha256": "e036068c1f36349c26a8daba9472fd40cfec5ae55424a55a95a4c75472c9dd48"
},
{
"file": "right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"sha256": "21d9410ca84dcd3bfea3bd8c08a682d4ff640c79f68e5d9f93ea26bcf904c3ac"
},
{
"file": "right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"sha256": "89b5da7da64e2d3207291c7847d347d30fe0b40014ea6d6c4fc9e4bf4e03682c"
},
{
"file": "right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"sha256": "2f0c17876e2607e8ab4282fadefba6fee6c2a5446b416cce6d0b91f796da2893"
},
{
"file": "right_elbow_link__visual_0__right_elbow_link.glb",
"sha256": "ef1e9dc1afcf147e78902039f49ef0131f114a11afe873b5095a6c4f509b89b2"
},
{
"file": "right_wrist_roll_link__visual_0__right_wrist_roll_link.glb",
"sha256": "5efc62e433d732ec26a5b45d3be64f421fb9c26176f168bdde74a9737e48eae3"
},
{
"file": "right_wrist_yaw_link__visual_0__right_wrist_yaw_link.glb",
"sha256": "f6f7d91665a8645a0b5cbd2e311c78aaf39f9f2554ac232af0425bad7a38164b"
},
{
"file": "right_wrist_pitch_link__visual_0__right_wrist_pitch_link.glb",
"sha256": "48d0a47d41569a24a8ea4b83e7218084395f9ef96c4dcbcdca1a3acbc060ad64"
},
{
"file": "lh_base_link__visual_0__base_link.glb",
"sha256": "886152daa0f4982f20d346639c4fd29f779b0d8fbf8e8d4942684bbc0cbe9946"
},
{
"file": "lh_finger11_link__visual_0__finger11_link.glb",
"sha256": "c59c8b126390c3593f63c92c6b3ea0198174342f0da2a7e63f3f6d6c17c76b80"
},
{
"file": "lh_finger12_link__visual_0__finger12_link.glb",
"sha256": "dea1f32f1f098fb26e6163abeefb98d4d2e6dfb0c187446b5a3b3dc83c40b4a1"
},
{
"file": "lh_finger13_link__visual_0__finger13_link.glb",
"sha256": "688ab4e9a8191ae6bda909d099ade525be02a7de758144863dc44d731e4180fe"
},
{
"file": "lh_finger14_link__visual_0__finger14_link.glb",
"sha256": "3ebdfb11c136e4e990fc99d0af6ed9a46713c799332ecd311b402b562697592c"
},
{
"file": "lh_finger21_link__visual_0__finger21_link.glb",
"sha256": "7ac15bcfa68e38e33a49942f7a2772e63778d82b7181494a0eacfe1a73510721"
},
{
"file": "lh_finger22_link__visual_0__finger22_link.glb",
"sha256": "358757dbce84eb6cb81a6da3d2f54c467e395306819382c534624e2c67f4ae73"
},
{
"file": "lh_finger23_link__visual_0__finger23_link.glb",
"sha256": "0ff49e9c01a744186246671737bc2e45c3ec95980538851a0a142030100db16f"
},
{
"file": "lh_finger31_link__visual_0__finger31_link.glb",
"sha256": "4ed855cb9634a928da50b6aa645a00b244455c4541403a65dc451f78c941fc1d"
},
{
"file": "lh_finger32_link__visual_0__finger32_link.glb",
"sha256": "9d688372b46fd24ca06fb4767d2b7b462054949467b9f49b37532ade5827b076"
},
{
"file": "lh_finger33_link__visual_0__finger33_link.glb",
"sha256": "cb0695f569da22e06d69f9110f0779534bdf64e185d6f27b54ff89dde1f31275"
},
{
"file": "lh_finger41_link__visual_0__finger41_link.glb",
"sha256": "270a0e7aaae7cee7d0853e0300686d202aa6d92167645dbc6318ddc35ce3e61a"
},
{
"file": "lh_finger42_link__visual_0__finger42_link.glb",
"sha256": "40cf87025b184bcd92ecc00980c83449c1a62c5e7a849311c960c7b51f871e76"
},
{
"file": "lh_finger43_link__visual_0__finger43_link.glb",
"sha256": "ac791ee4a9b1877892539cb6d4e5f8145d6ad199cefb2e04ebba449c59d2416d"
},
{
"file": "lh_finger51_link__visual_0__finger51_link.glb",
"sha256": "e43276c24c230bd35459a5b4f442e0be7840c4db106ada75a51ab83ff4a17dcd"
},
{
"file": "lh_finger52_link__visual_0__finger52_link.glb",
"sha256": "3c156a7b898e695b3e34287ad12e6b5437bcd478823baffe2cd76702c6dd9626"
},
{
"file": "lh_finger53_link__visual_0__finger53_link.glb",
"sha256": "140fe9939ed6b58539a77ef0c4cf1608dc0cc16b5c8d043564dbab9b1842dd63"
},
{
"file": "rh_base_link__visual_0__base_link.glb",
"sha256": "ae7328599013944ceb0ac4e4bb0106c0e58526122624a7bb6e650b138b07840d"
},
{
"file": "rh_finger11_link__visual_0__finger11_link.glb",
"sha256": "289cdf7c384b181df0d64f47db579af622db3bcbdc42236e667dcb64c494377c"
},
{
"file": "rh_finger12_link__visual_0__finger12_link.glb",
"sha256": "505bc65a50290aefe7b1882b5ffad4e88bf7059ec9dee355014592a2f21fdd2d"
},
{
"file": "rh_finger13_link__visual_0__finger13_link.glb",
"sha256": "324cf4f7e6207912d5a709ddc85f53b76be930208c27bff8c46f8d8ec5a61922"
},
{
"file": "rh_finger14_link__visual_0__finger14_link.glb",
"sha256": "f41140c191686f6afd809f51aa3bd1341517337c160ef019980b7c278dce3296"
},
{
"file": "rh_finger21_link__visual_0__finger21_link.glb",
"sha256": "f6e9a6ce375be6107158d00df23ab416445eb6970abe25e0f80c840da4611684"
},
{
"file": "rh_finger22_link__visual_0__finger22_link.glb",
"sha256": "9e89e30d9c3d0115bcfc85b05a6b7761991443df5cd74a75b305b5025a855245"
},
{
"file": "rh_finger23_link__visual_0__finger23_link.glb",
"sha256": "2c7529aa60a4286e2630146371f6dd77228a5ffd1c8760aa0d34169aa1aeae2c"
},
{
"file": "rh_finger31_link__visual_0__finger31_link.glb",
"sha256": "253bef5027cbce1a055f4650134bafec3f58dbc8f0a7e417ddc338e3de3d514c"
},
{
"file": "rh_finger32_link__visual_0__finger32_link.glb",
"sha256": "5b3ff8e747900ebfb2fb36ff5ee1035e741e780dd3613169058643436575e4aa"
},
{
"file": "rh_finger33_link__visual_0__finger33_link.glb",
"sha256": "d440a8b8bcd8b3e3de414598af99ebfefd214d4eb58d7b4d77d1bd67a81f75cf"
},
{
"file": "rh_finger41_link__visual_0__finger41_link.glb",
"sha256": "0a2d053155922ebcfaccf6f893665beb72f7ee2b3a9a66694b695995f233c7cf"
},
{
"file": "rh_finger42_link__visual_0__finger42_link.glb",
"sha256": "aa517ec5881c1ca9840c21ae50461eacba59d76cd835fe735eba19030d49cbbc"
},
{
"file": "rh_finger43_link__visual_0__finger43_link.glb",
"sha256": "f1d4c2936c26267a20da3c1a4016e51316ac100e46e310ffb57af5debd111ef6"
},
{
"file": "rh_finger51_link__visual_0__finger51_link.glb",
"sha256": "3800f27cbbd3c7bc79ad72def333d841001614d6e9ba16ce3b9502385102771d"
},
{
"file": "rh_finger52_link__visual_0__finger52_link.glb",
"sha256": "24a3fd05bfe9273f150c93c2ccb79b723c02e70169c0cb2ed7432ced45be50c8"
},
{
"file": "rh_finger53_link__visual_0__finger53_link.glb",
"sha256": "98cd0bde6c4d27ad04bd3b567ec9e88a9ed849fe786f2bfdb775c33baacc3638"
}
],
"body_mapping": {
"base_link": {
"path": "visuals/base_link__visual_0__base_link.glb",
"parts": [
"base_link"
]
},
"left_hip_pitch_link": {
"path": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"parts": [
"left_hip_pitch_link"
]
},
"left_hip_roll_link": {
"path": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"parts": [
"left_hip_roll_link"
]
},
"left_hip_yaw_link": {
"path": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"parts": [
"left_hip_yaw_link"
]
},
"left_knee_link": {
"path": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"parts": [
"left_knee_link"
]
},
"left_ankle_pitch_link": {
"path": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"parts": [
"left_ankle_pitch_link"
]
},
"left_ankle_roll_link": {
"path": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"parts": [
"left_ankle_roll_link"
]
},
"right_hip_pitch_link": {
"path": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"parts": [
"right_hip_pitch_link"
]
},
"right_hip_roll_link": {
"path": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"parts": [
"right_hip_roll_link"
]
},
"right_hip_yaw_link": {
"path": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"parts": [
"right_hip_yaw_link"
]
},
"right_knee_link": {
"path": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"parts": [
"right_knee_link"
]
},
"right_ankle_pitch_link": {
"path": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"parts": [
"right_ankle_pitch_link"
]
},
"right_ankle_roll_link": {
"path": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"parts": [
"right_ankle_roll_link"
]
},
"waist_roll_link": {
"path": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"parts": [
"waist_roll_link"
]
},
"waist_yaw_link": {
"path": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"parts": [
"waist_yaw_link"
]
},
"left_shoulder_pitch_link": {
"path": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"parts": [
"left_shoulder_pitch_link"
]
},
"left_shoulder_roll_link": {
"path": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"parts": [
"left_shoulder_roll_link"
]
},
"left_shoulder_yaw_link": {
"path": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"parts": [
"left_shoulder_yaw_link"
]
},
"left_elbow_link": {
"path": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"parts": [
"left_elbow_link"
]
},
"left_wrist_link": {
"path": "visuals/left_wrist_link__fixed_hand.glb",
"parts": [
"left_wrist_roll_link",
"left_wrist_yaw_link",
"left_wrist_pitch_link",
"lh_base_link",
"lh_finger11_link",
"lh_finger12_link",
"lh_finger13_link",
"lh_finger14_link",
"lh_finger21_link",
"lh_finger22_link",
"lh_finger23_link",
"lh_finger31_link",
"lh_finger32_link",
"lh_finger33_link",
"lh_finger41_link",
"lh_finger42_link",
"lh_finger43_link",
"lh_finger51_link",
"lh_finger52_link",
"lh_finger53_link"
]
},
"right_shoulder_pitch_link": {
"path": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"parts": [
"right_shoulder_pitch_link"
]
},
"right_shoulder_roll_link": {
"path": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"parts": [
"right_shoulder_roll_link"
]
},
"right_shoulder_yaw_link": {
"path": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"parts": [
"right_shoulder_yaw_link"
]
},
"right_elbow_link": {
"path": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"parts": [
"right_elbow_link"
]
},
"right_wrist_link": {
"path": "visuals/right_wrist_link__fixed_hand.glb",
"parts": [
"right_wrist_roll_link",
"right_wrist_yaw_link",
"right_wrist_pitch_link",
"rh_base_link",
"rh_finger11_link",
"rh_finger12_link",
"rh_finger13_link",
"rh_finger14_link",
"rh_finger21_link",
"rh_finger22_link",
"rh_finger23_link",
"rh_finger31_link",
"rh_finger32_link",
"rh_finger33_link",
"rh_finger41_link",
"rh_finger42_link",
"rh_finger43_link",
"rh_finger51_link",
"rh_finger52_link",
"rh_finger53_link"
]
}
},
"files": [
{
"file": "visuals/rp1_hand_fixed.urdf",
"sha256": "2c1f91a82f39298fdda7ec9531d045caff745ba12157f0bb0fcb2f4cf1102948"
},
{
"file": "visuals/base_link__visual_0__base_link.glb",
"sha256": "3db82d0e5fa1fa48fa6f4fe4460bba14568a39b5933acff8b4b1879f582e06b0"
},
{
"file": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"sha256": "d8cb2aff7273185a89eba99aab47ce6bf4716f25e70f34afa1529acb802c6193"
},
{
"file": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"sha256": "35361b79fb69096039eb077018c25ae489f16cc5229dcd82256b3a573b168208"
},
{
"file": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"sha256": "19fe908274f9fe49b5a8317652e4fe02a6ff93802de87ad7d0383fa1d142b03b"
},
{
"file": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"sha256": "1c01f5c03837d24287d79eff48a9e4bd4203d8516d0375592b159d49fb83f6c9"
},
{
"file": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"sha256": "4bab717bf7073e28754d1c42f4409141ea775987284ff8575f9c7aef25561e8c"
},
{
"file": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"sha256": "76144b6ebf17b2eb5875168cbe6a3c4c12eaeebdd4bd277864887b3894f781b1"
},
{
"file": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"sha256": "e299f46bff44d6bbf03592cd6caba3b418fc022260746554e6d2013f31793df4"
},
{
"file": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"sha256": "7cb006f9ba7f950c98287b59695938cfd6460d8fa4546ae6c95bc65ed5c6c6b8"
},
{
"file": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"sha256": "35393465aa8103ab436ecc3a635cdce7477600cf65eb03a2942af546cc39c79c"
},
{
"file": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"sha256": "2a5f1f91f6db0cfac3a0e88038613be89cb94066d7f103876b455823edbfb8f3"
},
{
"file": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"sha256": "c43112750efd953f29dfea5d8d2bb925eb847f74b679fa1d22f734b1e3094b91"
},
{
"file": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"sha256": "3856c0074bbead9ade091b2bef4df165d9c1d4b03e0c0f1d8c284cddf091a9f6"
},
{
"file": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"sha256": "a4cd49014951ed5c7836bde4a7138eab4119f0ca7249a69e26add388d5fedf32"
},
{
"file": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"sha256": "cb50b0354f8a0a6d06ff486d46cb849462b84471c20a350b1e4c5a86577ce230"
},
{
"file": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"sha256": "9eb76bc0675578e28707120d64cb042b948312ddc9233e4c28de02ee04a3dbf9"
},
{
"file": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"sha256": "f72e3ac73f3804acd5e5384c2d10d48aacacb95bf9f98ea0f2426f1b92add0de"
},
{
"file": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"sha256": "909f0dbd40db57b223143c1c89c740c3ea636e5d9b066eb6fe26dab37e2f4975"
},
{
"file": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"sha256": "f98a4446175c7b1d98c5a664fabcb6698635c99ccfc8261dedbef9155e1a2043"
},
{
"file": "visuals/left_wrist_link__fixed_hand.glb",
"sha256": "7b7bee1c2aa6ba74bc2824bf3ec5c465c33a97e1e0409ccdc551fae74a6218e5"
},
{
"file": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"sha256": "21d9410ca84dcd3bfea3bd8c08a682d4ff640c79f68e5d9f93ea26bcf904c3ac"
},
{
"file": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"sha256": "89b5da7da64e2d3207291c7847d347d30fe0b40014ea6d6c4fc9e4bf4e03682c"
},
{
"file": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"sha256": "2f0c17876e2607e8ab4282fadefba6fee6c2a5446b416cce6d0b91f796da2893"
},
{
"file": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"sha256": "ef1e9dc1afcf147e78902039f49ef0131f114a11afe873b5095a6c4f509b89b2"
},
{
"file": "visuals/right_wrist_link__fixed_hand.glb",
"sha256": "57a47c99b86d4efea1cd891147a0aafc86395daa31332a88b6653e44be1449c1"
}
]
}
File diff suppressed because it is too large Load Diff
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""Import the user-supplied, self-contained Microban visual GLBs only.
Usage: python3 app/scripts/import-microban-visuals.py /path/to/microban2.zip
Does not modify MJCF, policies or the reviewed body transforms in robot.json.
"""
import hashlib
import json
from pathlib import Path
import struct
import sys
import zipfile
def import_visuals(archive):
target = Path(__file__).resolve().parents[1] / "public/robots/microban"
links = ["BASE", *(f"LINK{i}" for i in range(1, 20))]
names = [f"{link}__visual_0__{link}.glb" for link in links]
with zipfile.ZipFile(archive) as source:
if sorted(source.namelist()) != sorted(names):
raise ValueError("Expected exactly BASE and LINK1..LINK19 GLBs")
files = {name: source.read(name) for name in names}
# Validate everything before copying any file. Never extract ZIP paths.
for name, data in files.items():
magic, version, length, json_length, chunk_type = struct.unpack_from(
"<5I", data
)
if (magic, version, length, chunk_type) != (
0x46546C67, 2, len(data), 0x4E4F534A
):
raise ValueError(f"{name}: invalid GLB")
gltf = json.loads(data[20:20 + json_length])
for asset in [*gltf.get("buffers", []), *gltf.get("images", [])]:
if "uri" in asset and not asset["uri"].startswith("data:"):
raise ValueError(f"{name}: external buffer/image is unsupported")
(target / "visuals").mkdir(exist_ok=True)
records = []
for name, data in files.items():
(target / "visuals" / name).write_bytes(data)
records.append({
"file": f"visuals/{name}",
"sha256": hashlib.sha256(data).hexdigest(),
})
manifest = {
"source": f"User-provided {archive.name} (textured visual replacement)",
"archive_sha256": hashlib.sha256(archive.read_bytes()).hexdigest(),
"license": "Not supplied in the archive; confirm redistribution rights with the asset owner.",
"files": records,
}
(target / "visual-source-manifest.json").write_text(
json.dumps(manifest, indent=2) + "\n"
)
print(f"Imported {len(records)} visual GLBs; physics/policy files unchanged")
if __name__ == "__main__":
import_visuals(Path(sys.argv[1]))
+208
View File
@@ -0,0 +1,208 @@
#!/usr/bin/env python3
"""Import the user's 63 textured parts onto the existing 25-body/24-DOF RP1.
Usage: python3 app/scripts/import-rp1-visuals.py ARCHIVE HAND_FIXED_URDF
No MJCF, STL, policy, motion or physical joint is changed. The source's fixed
finger poses and neutral wrist yaw/pitch are composed into two visual-only
wrist GLBs. Source images/materials/binary vertex data are retained verbatim.
"""
import copy
import hashlib
import json
import math
from pathlib import Path
import struct
import sys
import xml.etree.ElementTree as ET
import zipfile
def sha(data):
return hashlib.sha256(data).hexdigest()
def read_glb(data):
if len(data) < 20:
raise ValueError("Truncated GLB")
magic, version, length, size, kind = struct.unpack_from("<5I", data)
if (magic, version, length, kind) != (0x46546C67, 2, len(data), 0x4E4F534A):
raise ValueError("Invalid GLB (possibly an LFS pointer)")
doc = json.loads(data[20:20 + size])
offset = 20 + size
binary_size, binary_kind = struct.unpack_from("<2I", data, offset)
if binary_kind != 0x004E4942 or offset + 8 + binary_size != len(data):
raise ValueError("Expected one embedded BIN chunk")
for asset in [*doc.get("buffers", []), *doc.get("images", [])]:
if "uri" in asset:
raise ValueError("GLBs must embed all images and buffers")
if len(doc["buffers"]) != 1 or any(k in doc for k in ["skins", "animations", "extensionsRequired"]):
raise ValueError("Expected static, self-contained body GLB")
return doc, data[offset + 8:offset + 8 + binary_size]
def identity():
return [[float(i == j) for j in range(4)] for i in range(4)]
def multiply(a, b):
return [[sum(a[i][k] * b[k][j] for k in range(4)) for j in range(4)] for i in range(4)]
def origin_matrix(origin):
result = identity()
if origin is None:
return result
xyz = list(map(float, origin.get("xyz", "0 0 0").split()))
roll, pitch, yaw = map(float, origin.get("rpy", "0 0 0").split())
cr, sr, cp, sp, cy, sy = math.cos(roll), math.sin(roll), math.cos(pitch), math.sin(pitch), math.cos(yaw), math.sin(yaw)
# URDF fixed-axis XYZ = Rz(yaw) Ry(pitch) Rx(roll).
rotation = [[cy * cp, cy * sp * sr - sy * cr, cy * sp * cr + sy * sr],
[sy * cp, sy * sp * sr + cy * cr, sy * sp * cr - cy * sr],
[-sp, cp * sr, cp * cr]]
for i in range(3):
result[i][:3] = rotation[i]
result[i][3] = xyz[i]
return result
def assemble(parts):
keys = ["nodes", "materials", "meshes", "textures", "images", "accessors", "bufferViews", "samplers"]
out = {"asset": {"version": "2.0", "generator": "microduck RP1 visual-only fixed-hand assembler"},
"scene": 0, "scenes": [{"name": "Fixed wrist and hand", "nodes": []}],
**{key: [] for key in keys}}
binary = bytearray()
# Blender's source GLBs are Y-up; the URDF matrices are Z-up.
c = [[1, 0, 0, 0], [0, 0, 1, 0], [0, -1, 0, 0], [0, 0, 0, 1]]
cinv = [[1, 0, 0, 0], [0, 0, -1, 0], [0, 1, 0, 0], [0, 0, 0, 1]]
for name, data, transform in parts:
doc, payload = read_glb(data)
if doc.get("extensionsUsed"):
raise ValueError("Unexpected extension in wrist part; review remapping")
offsets = {key: len(out[key]) for key in keys}
start = len(binary)
binary.extend(payload)
binary.extend(b"\0" * (-len(binary) % 4))
for key in keys:
for original in doc[key]:
item = copy.deepcopy(original)
if key == "bufferViews":
item["buffer"] = 0
item["byteOffset"] = item.get("byteOffset", 0) + start
elif key == "accessors":
if "sparse" in item:
raise ValueError("Sparse accessor requires explicit remapping")
if "bufferView" in item:
item["bufferView"] += offsets["bufferViews"]
elif key == "images":
item["bufferView"] += offsets["bufferViews"]
elif key == "textures":
item["source"] += offsets["images"]
if "sampler" in item:
item["sampler"] += offsets["samplers"]
elif key == "materials":
def textures(value):
if isinstance(value, dict):
for field, child in value.items():
if field.endswith("Texture") and isinstance(child, dict) and "index" in child:
child["index"] += offsets["textures"]
else:
textures(child)
elif isinstance(value, list):
for child in value:
textures(child)
textures(item)
elif key == "meshes":
for primitive in item["primitives"]:
primitive["attributes"] = {k: v + offsets["accessors"] for k, v in primitive["attributes"].items()}
for field, table in [("indices", "accessors"), ("material", "materials")]:
if field in primitive:
primitive[field] += offsets[table]
elif key == "nodes":
if "mesh" in item:
item["mesh"] += offsets["meshes"]
if "children" in item:
item["children"] = [v + offsets["nodes"] for v in item["children"]]
out[key].append(item)
matrix = multiply(multiply(c, transform), cinv)
out["scenes"][0]["nodes"].append(len(out["nodes"]))
out["nodes"].append({"name": f"source-part:{name}", "extras": {"sourcePart": name}, "matrix": [matrix[i][j] for j in range(4) for i in range(4)],
"children": [v + offsets["nodes"] for v in doc["scenes"][doc.get("scene", 0)]["nodes"]]})
out["buffers"] = [{"byteLength": len(binary)}]
encoded = json.dumps(out, separators=(",", ":")).encode()
encoded += b" " * (-len(encoded) % 4)
total = 12 + 8 + len(encoded) + 8 + len(binary)
return struct.pack("<5I", 0x46546C67, 2, total, len(encoded), 0x4E4F534A) + encoded + struct.pack("<2I", len(binary), 0x004E4942) + binary
def import_visuals(archive, urdf):
target = Path(__file__).resolve().parents[1] / "public/robots/rp1"
source_root = ET.fromstring(urdf.read_bytes())
links = {link.get("name"): link for link in source_root.findall("link")}
expected = {}
for name, link in links.items():
mesh = Path(link.find("visual/geometry/mesh").get("filename")).stem
expected[name] = f"{name}__visual_0__{mesh}.glb"
with zipfile.ZipFile(archive) as source:
if sorted(source.namelist()) != sorted(expected.values()) or len(expected) != 63:
raise ValueError("Archive must contain exactly the 63 hand-model body GLBs")
inputs = {name: source.read(file) for name, file in expected.items()}
for data in inputs.values():
read_glb(data)
model = ET.parse(target / "model.xml").getroot()
physical = [body.get("name") for body in model.findall(".//worldbody//body")]
parents = {j.find("child").get("link"): j for j in source_root.findall("joint")}
output, mappings = {}, {}
for body in physical:
if not body.endswith("_wrist_link"):
name = expected[body]
output[name] = inputs[body]
mappings[body] = {"path": f"visuals/{name}", "parts": [body]}
continue
side = body.split("_")[0]
anchor = f"{side}_wrist_roll_link"
prefix = "lh_" if side == "left" else "rh_"
names = [name for name in links if name.startswith(f"{side}_wrist_") or name.startswith(prefix)]
parts = []
for name in names:
relative = identity()
node = name
while node != anchor:
joint = parents[node]
# The extra two wrist axes are visual-only at neutral; hand
# joints come from the upstream fixed-hand URDF.
if joint.get("type") != "fixed" and "wrist" not in joint.get("name", ""):
raise ValueError("Unexpected moving hand joint")
relative = multiply(origin_matrix(joint.find("origin")), relative)
node = joint.find("parent").get("link")
parts.append((name, inputs[name], relative))
file = f"{body}__fixed_hand.glb"
output[file] = assemble(parts)
mappings[body] = {"path": f"visuals/{file}", "parts": names}
used = [name for mapping in mappings.values() for name in mapping["parts"]]
if sorted(used) != sorted(links) or len(physical) != 25:
raise ValueError("Every source part must map exactly once to 25 physical bodies")
for data in output.values():
read_glb(data)
# Commit only after all inputs and assembled outputs validate. No ZIP extraction.
(target / "visuals").mkdir(exist_ok=True)
(target / "visuals/rp1_hand_fixed.urdf").write_bytes(urdf.read_bytes())
records = [{"file": "visuals/rp1_hand_fixed.urdf", "sha256": sha(urdf.read_bytes())}]
for name, data in output.items():
(target / "visuals" / name).write_bytes(data)
records.append({"file": f"visuals/{name}", "sha256": sha(data)})
config = json.loads((target / "robot.json").read_text())
config["render"]["body_visuals"] = [{"body": body, "path": mappings[body]["path"],
"quaternion": [math.sqrt(0.5), 0, 0, math.sqrt(0.5)]} for body in physical]
(target / "robot.json").write_text(json.dumps(config, indent=2) + "\n")
manifest = {"source": f"User-provided {archive.name}", "archive_sha256": sha(archive.read_bytes()),
"license": "Not supplied in the archive; local evaluation only. Confirm redistribution rights.",
"coordinate_conversion": "Source GLBs are Blender Y-up; attachments rotate +90 degrees about X into MuJoCo body Z-up.",
"assembly": "25 physical body attachments; 20 source parts per wrist (neutral extra wrist axes and fixed fingers). No physical joints added.",
"input_files": [{"file": expected[name], "sha256": sha(data)} for name, data in inputs.items()],
"body_mapping": mappings, "files": records}
(target / "visual-source-manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
print(f"Imported {len(inputs)} source GLBs into {len(physical)} body attachments; physics/policies untouched")
if __name__ == "__main__":
import_visuals(Path(sys.argv[1]), Path(sys.argv[2]))
+22 -10
View File
@@ -8,6 +8,7 @@ import {
} from "../src/platform/loader/SchemaValidator.js";
import { resolveWorldId } from "../src/platform/worlds/arena.js";
import { JointMotionReference } from "../src/platform/policy/JointMotionReference.js";
import { validateBodyGlb } from "../src/platform/render/BodyVisuals.js";
const root = fileURLToPath(new URL("../public/robots/", import.meta.url));
const entries = JSON.parse(await readFile(resolve(root, "index.json"), "utf8"));
for (const entry of entries) {
@@ -18,6 +19,7 @@ for (const entry of entries) {
const paths = [
config.model,
...config.assets.map((a) => a.path),
...(config.render.body_visuals ?? []).map((v) => v.path),
...config.skills
.filter((s) => s.controller.type === "onnx" && s.available !== false)
.map((s) => s.controller.policy),
@@ -39,6 +41,12 @@ for (const entry of entries) {
paths.push(skill.reference, spec.data);
}
for (const path of paths) await access(resolve(dirname(file), path));
for (const visual of config.render.body_visuals ?? []) {
const bytes = await readFile(resolve(dirname(file), visual.path));
validateBodyGlb(
bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength),
);
}
validateWorld(
JSON.parse(
await readFile(
@@ -47,15 +55,18 @@ for (const entry of entries) {
),
),
);
let provenance;
try {
provenance = JSON.parse(
await readFile(resolve(dirname(file), "source-manifest.json"), "utf8"),
);
} catch (error) {
if (error.code !== "ENOENT") throw error;
}
if (provenance)
let hasProvenance = false;
for (const name of ["source-manifest.json", "visual-source-manifest.json"]) {
let provenance;
try {
provenance = JSON.parse(
await readFile(resolve(dirname(file), name), "utf8"),
);
} catch (error) {
if (error.code !== "ENOENT") throw error;
continue;
}
hasProvenance = true;
for (const record of [
...provenance.files,
...(provenance.additionalNotices ?? []),
@@ -68,7 +79,8 @@ for (const entry of entries) {
`${entry.id}/${record.file}: source SHA-256 mismatch (check git lfs pull or document asset changes)`,
);
}
}
console.log(
`${config.id}: schema, World, ${paths.length} resource paths${provenance ? " and source hashes" : ""} OK`,
`${config.id}: schema, World, ${paths.length} resource paths${hasProvenance ? " and source hashes" : ""} OK`,
);
}
+151
View File
@@ -0,0 +1,151 @@
import * as THREE from "three";
const TYPES = ["box", "sphere", "cylinder", "capsule"];
const COLORS = [
0xffb52e, 0x81cbb7, 0xb4a4d4, 0x83bfff, 0xf2efe8, 0xff7a2f, 0xf09caf,
];
// One layout per runtime, shared by both locomotion models. A shuffled,
// jittered lattice gives random placement without overlaps or retry loops.
export function generateInteractiveProps({
halfExtent,
spawn,
avoid = [],
random = Math.random,
}) {
const count = 5 + Math.floor(random() * 3);
const candidates = [];
const limit = halfExtent - 0.2;
for (
let x = Math.max(-limit, spawn[0] - 0.4);
x <= Math.min(limit, spawn[0] + 1.8);
x += 0.4
) {
for (let y = -Math.min(limit, 1.2); y <= Math.min(limit, 1.2); y += 0.4) {
const px = x + (random() - 0.5) * 0.06;
const py = y + (random() - 0.5) * 0.06;
if (
[spawn, ...avoid].every(([x, y]) => Math.hypot(px - x, py - y) > 0.34)
)
candidates.push([px, py]);
}
}
for (let i = candidates.length - 1; i > 0; i--) {
const j = Math.floor(random() * (i + 1));
[candidates[i], candidates[j]] = [candidates[j], candidates[i]];
}
// Keep at least one toy within immediate walking / kicking reach.
const near = candidates.findIndex(
([x, y]) =>
x > spawn[0] + 0.3 &&
x < spawn[0] + 0.85 &&
Math.abs(y - spawn[1]) < 0.42,
);
if (near > 0)
[candidates[0], candidates[near]] = [candidates[near], candidates[0]];
if (candidates.length < count)
throw new Error("Arena too small for interactive props");
const offset = Math.floor(random() * TYPES.length);
return candidates.slice(0, count).map(([x, y], i) => {
const type = TYPES[(i + offset) % TYPES.length];
const r = 0.045 + random() * 0.015;
const h = 0.045 + random() * 0.02;
const size =
type === "box" ? [r, r * 0.85, h] : type === "sphere" ? [r] : [r, h];
const bottom = type === "sphere" ? r : type === "capsule" ? r + h : h;
const yaw = random() * Math.PI * 2;
return {
name: `interactive_${i}`,
type,
size,
pos: [x, y, bottom + 0.006],
quat: [Math.cos(yaw / 2), 0, 0, Math.sin(yaw / 2)],
mass: 0.04 + random() * 0.04,
color: COLORS[i % COLORS.length],
};
});
}
export function resolvePropAddresses(mujoco, model, specs) {
return specs.map((spec) => {
const jointId = mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_JOINT.value,
`${spec.name}_joint`,
);
const bodyId = mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_BODY.value,
spec.name,
);
return {
bodyId,
qAdr: model.jnt_qposadr[jointId],
dofAdr: model.jnt_dofadr[jointId],
mass: model.body_mass[bodyId],
};
});
}
export function resetProp(data, address, spec, parked = false, index = 0) {
const q = data.qpos,
v = data.qvel;
q.set(parked ? [50 + index, 0, spec.pos[2]] : spec.pos, address.qAdr);
q.set(spec.quat, address.qAdr + 3);
v.fill(0, address.dofAdr, address.dofAdr + 6);
}
export function createInteractivePropVisuals(specs) {
const root = new THREE.Group();
root.name = "interactive-props";
root.rotation.x = -Math.PI / 2; // MJCF Z-up -> Three Y-up
const meshes = specs.map((spec, index) => {
const [r, h, z] = spec.size;
let geometry;
if (spec.type === "box")
geometry = new THREE.BoxGeometry(2 * r, 2 * h, 2 * z);
else if (spec.type === "sphere")
geometry = new THREE.SphereGeometry(r, 24, 16);
else {
geometry =
spec.type === "cylinder"
? new THREE.CylinderGeometry(r, r, 2 * h, 24)
: new THREE.CapsuleGeometry(r, 2 * h, 8, 16);
geometry.rotateX(Math.PI / 2);
}
const mesh = new THREE.Mesh(
geometry,
new THREE.MeshStandardMaterial({
color: spec.color,
roughness: 0.38,
metalness: 0.08,
}),
);
mesh.name = spec.name;
mesh.userData.interactiveIndex = index;
mesh.position.fromArray(spec.pos);
mesh.quaternion.set(spec.quat[1], spec.quat[2], spec.quat[3], spec.quat[0]);
root.add(mesh);
return mesh;
});
return {
root,
meshes,
dispose() {
root.removeFromParent();
meshes.forEach((mesh) => {
mesh.geometry.dispose();
mesh.material.dispose();
});
},
sync(data, addresses, visible = true) {
root.visible = visible;
const q = data.qpos;
meshes.forEach((mesh, i) => {
const a = addresses[i].qAdr;
mesh.position.set(q[a], q[a + 1], q[a + 2]);
mesh.quaternion.set(q[a + 4], q[a + 5], q[a + 6], q[a + 3]);
});
},
};
}
+11 -18
View File
@@ -1,9 +1,8 @@
import { propColliders } from "../../props.js";
import { appendInteractivePhysics } from "./session-environment.js";
import { arenaWalls } from "../../../platform/worlds/arena.js";
import {
ARENA_HALF,
BALL_PARK_POS,
BALL_RADIUS,
RELIEF_GRID,
RELIEF_HMAX,
RELIEF_SINK,
@@ -17,6 +16,8 @@ export function composeArenaPhysics(
jointNames,
defaultPose,
spawn,
interactiveProps = [],
floor,
boundary = {
half_extent: ARENA_HALF,
wall_height: 0.25,
@@ -30,7 +31,11 @@ export function composeArenaPhysics(
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v);
return e;
};
root.appendChild(el("option", { timestep: String(timestep) }));
// Adding free toys can cross MuJoCo's automatic sparse-Jacobian cutoff
// (nv >= 60). Keep the original small-robot dense path in both modes.
root.appendChild(
el("option", { timestep: String(timestep), jacobian: "dense" }),
);
const body = doc.querySelector("worldbody");
body.appendChild(
el("geom", {
@@ -38,6 +43,7 @@ export function composeArenaPhysics(
type: "plane",
size: "0 0 0.05",
pos: "0 0 0",
...(floor?.friction ? { friction: floor.friction.join(" ") } : {}),
}),
);
for (const wall of arenaWalls(boundary))
@@ -54,20 +60,7 @@ export function composeArenaPhysics(
if (c.euler) attrs.euler = c.euler;
body.appendChild(el("geom", attrs));
}
const ball = el("body", { name: "ball", pos: BALL_PARK_POS });
ball.appendChild(el("freejoint", { name: "ball_freejoint" }));
ball.appendChild(
el("geom", {
name: "ball_geom",
type: "sphere",
size: String(BALL_RADIUS),
mass: "0.03",
friction: "0.4 0.01 0.003",
solref: "0.03 0.4",
condim: "6",
}),
);
body.appendChild(ball);
appendInteractivePhysics(doc, interactiveProps, { group: 0 });
doc.querySelector("asset").appendChild(
el("hfield", {
name: "terrain",
@@ -93,7 +86,7 @@ export function composeArenaPhysics(
keyframe.appendChild(
el("key", {
name: "STAND",
qpos: `${qposFree} ${qposJoints} ${BALL_PARK_POS} 1 0 0 0`,
qpos: `${qposFree} ${qposJoints} ${interactiveProps.map((p) => [...p.pos, ...p.quat].join(" ")).join(" ")}`,
ctrl: Array.from(defaultPose).join(" "),
}),
);
+15
View File
@@ -0,0 +1,15 @@
import * as THREE from "three";
import { VARIANT_SWATCH_HEX } from "../../variants.js";
// The existing four HUD swatches now tint both the floor and enclosure.
// Keep minor lines quieter than section lines, without touching reveal/fade.
export function setArenaLineColor(materials, name) {
const hex = VARIANT_SWATCH_HEX[name];
if (!hex) return false;
const color = new THREE.Color(hex);
for (const material of materials) {
material.uniforms.uSectionColor.value.copy(color);
material.uniforms.uCellColor.value.copy(color).multiplyScalar(0.55);
}
return true;
}
@@ -1,5 +1,6 @@
import * as THREE from "three";
import { makeInfiniteGrid, makeArenaWalls } from "./render.js";
import { setArenaLineColor } from "./line-color.js";
import { arenaWalls } from "../../../platform/worlds/arena.js";
// Shared appearance; physical collision boxes come only from the World composer.
export function createArenaFloorPresentation({ sim, rig, world }) {
@@ -43,6 +44,8 @@ export function createArenaFloorPresentation({ sim, rig, world }) {
root,
grid,
walls: wallMeshes,
setLineColor: (name) =>
setArenaLineColor([grid.material, ...wallMats], name),
update(position) {
grid.position.set(position.x, 0, position.z);
grid.material.uniforms.uFocus.value.copy(position);
@@ -0,0 +1,75 @@
import { generateInteractiveProps } from "../../interactive-props.js";
// One random arrangement for the whole browser session, not one per robot.
// Positions/sizes share the arena's existing robot-relative scale. Reserve
// both bundled spawn anchors so changing robots never starts inside a toy.
export function createSessionEnvironment(random = Math.random) {
let template;
return {
propsFor(world) {
if (world.presentation !== "arena-grid" || !world.boundary) return [];
template ??= generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
avoid: [[0, 0]],
random,
});
const scale = world.boundary.half_extent / 2.25;
return template.map((spec) => ({
...spec,
pos: spec.pos.map((v) => v * scale),
size: spec.size.map((v) => v * scale),
quat: [...spec.quat],
// Keep larger robots' toys readily pushable, rather than scaling
// mass cubically into heavy obstacles for the humanoid.
mass: spec.mass * scale * scale,
}));
},
};
}
let seed = 123456789;
const testRandom = () =>
(seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
const session = createSessionEnvironment(
typeof location !== "undefined" &&
new URLSearchParams(location.search).get("test") === "1"
? testRandom
: Math.random,
);
export const sharedInteractiveProps = (world) => session.propsFor(world);
export function appendInteractivePhysics(doc, specs, { group = 5 } = {}) {
const body = doc.querySelector("worldbody");
const el = (tag, attrs) => {
const node = doc.createElement(tag);
for (const [key, value] of Object.entries(attrs))
node.setAttribute(key, value);
return node;
};
for (const spec of specs) {
if (doc.querySelector(`[name="${spec.name}"]`))
throw new Error(`World name collision: ${spec.name}`);
const prop = el("body", {
name: spec.name,
pos: spec.pos.join(" "),
quat: spec.quat.join(" "),
});
prop.appendChild(el("freejoint", { name: `${spec.name}_joint` }));
prop.appendChild(
el("geom", {
name: `${spec.name}_geom`,
type: spec.type,
size: spec.size.join(" "),
mass: String(spec.mass),
friction: spec.friction ?? "0.5 0.01 0.003",
solref: spec.solref ?? "0.02 1",
condim: "6",
contype: "1",
conaffinity: "1",
group: String(group),
}),
);
body.appendChild(prop);
}
}
+158
View File
@@ -0,0 +1,158 @@
import * as THREE from "three";
// Same mass-scaled spring perturbation as Microduck, applied every physics
// substep. Shared by generic robots for both their body and the arena toys.
export class PhysicsDrag {
constructor({
sim,
camera,
controls,
canvas,
objects,
addressFor,
enabled,
halfExtent,
scale = 1,
}) {
Object.assign(this, {
sim,
camera,
controls,
canvas,
objects,
addressFor,
enabled,
halfExtent,
scale,
});
this.ray = new THREE.Raycaster();
this.ndc = new THREE.Vector2();
this.plane = new THREE.Plane();
this.point = new THREE.Vector3();
this.direction = new THREE.Vector3();
this.listeners = [];
const listen = (type, fn, capture = false) => {
window.addEventListener(type, fn, capture);
this.listeners.push(() => window.removeEventListener(type, fn, capture));
};
listen(
"pointerdown",
(e) => {
if (
e.target !== canvas ||
e.pointerType !== "mouse" ||
e.button !== 0 ||
!enabled()
)
return;
this.setRay(e);
const hit = this.pick();
if (!hit) return;
const address = addressFor(hit.object);
if (!address) return;
this.grab = { ...address, target: [0, 0, 0] };
camera.getWorldDirection(this.direction);
this.plane.setFromNormalAndCoplanarPoint(
this.direction.negate(),
hit.point,
);
this.updateTarget();
controls.enabled = false;
canvas.style.cursor = "grabbing";
this.pointerId = e.pointerId;
try {
canvas.setPointerCapture(e.pointerId);
} catch {
/* unavailable */
}
},
true,
);
listen("pointermove", (e) => {
if (this.grab) {
this.setRay(e);
this.updateTarget();
return;
}
if (
e.target !== canvas ||
e.pointerType !== "mouse" ||
e.buttons ||
!enabled()
)
return;
if (performance.now() - (this.hoverAt ?? 0) < 80) return;
this.hoverAt = performance.now();
this.setRay(e);
canvas.style.cursor = this.pick() ? "grab" : "";
});
for (const type of ["pointerup", "pointercancel", "blur"])
listen(type, () => this.release());
canvas.addEventListener(
"lostpointercapture",
(this.lostCapture = () => this.release()),
);
}
setRay(e) {
const rect = this.canvas.getBoundingClientRect();
this.ndc.set(
((e.clientX - rect.left) / rect.width) * 2 - 1,
-((e.clientY - rect.top) / rect.height) * 2 + 1,
);
this.ray.setFromCamera(this.ndc, this.camera);
}
pick() {
return this.ray.intersectObjects(this.objects(), true).find((hit) => {
for (let o = hit.object; o; o = o.parent) if (!o.visible) return false;
return !!this.addressFor(hit.object);
});
}
updateTarget() {
if (!this.ray.ray.intersectPlane(this.plane, this.point)) return;
const lim = (this.halfExtent ?? 100) - 0.05 * this.scale;
this.grab.target = [
Math.max(-lim, Math.min(lim, this.point.x)),
Math.max(-lim, Math.min(lim, -this.point.z)),
Math.max(0.02 * this.scale, Math.min(0.45 * this.scale, this.point.y)),
];
}
apply() {
const g = this.grab;
if (!g) return;
const { data } = this.sim,
force = [];
for (let i = 0; i < 3; i++)
force[i] =
100 * (g.target[i] - data.qpos[g.qAdr + i]) -
10 * data.qvel[g.dofAdr + i];
const norm = Math.hypot(...force),
clamp = Math.min(1, 200 / (norm || 1));
for (let i = 0; i < 3; i++)
data.xfrc_applied[g.bodyId * 6 + i] = g.mass * force[i] * clamp;
}
release() {
if (this.grab)
this.sim.data.xfrc_applied.fill(
0,
this.grab.bodyId * 6,
this.grab.bodyId * 6 + 6,
);
this.grab = null;
this.controls.enabled = true;
this.canvas.style.cursor = "";
const id = this.pointerId;
this.pointerId = undefined;
if (id !== undefined) {
try {
this.canvas.releasePointerCapture(id);
} catch {
/* already released */
}
}
}
dispose() {
this.release();
this.listeners.forEach((remove) => remove());
this.canvas.removeEventListener("lostpointercapture", this.lostCapture);
}
}
@@ -218,6 +218,21 @@ export function validatePackage(config) {
}
if (config.adapter === "generic" && config.render.source !== "mjcf_visual")
throw new Error("render.source: generic runtime requires mjcf_visual");
const visualBodies = new Set();
for (const visual of config.render.body_visuals ?? []) {
if (config.adapter !== "generic")
throw new Error("render.body_visuals: requires the generic runtime");
if (visualBodies.has(visual.body))
throw new Error(`render.body_visuals: duplicate body ${visual.body}`);
visualBodies.add(visual.body);
if (
visual.quaternion &&
Math.abs(Math.hypot(...visual.quaternion) - 1) > 1e-6
)
throw new Error(
"render.body_visuals: quaternion must have unit length (xyzw)",
);
}
return config;
}
export { schema as robotSchema };
+28
View File
@@ -379,6 +379,34 @@
"source": {
"enum": ["glb", "mjcf_visual"]
},
"body_visuals": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": ["body", "path"],
"additionalProperties": false,
"properties": {
"body": { "type": "string", "minLength": 1 },
"path": {
"type": "string",
"pattern": "^(?!/)(?!.*\\.\\.)[^\\\\:?#]+\\.glb$"
},
"position": {
"type": "array",
"minItems": 3,
"maxItems": 3,
"items": { "type": "number" }
},
"quaternion": {
"type": "array",
"minItems": 4,
"maxItems": 4,
"items": { "type": "number" }
}
}
}
},
"groups": {
"type": "array",
"items": {
+112
View File
@@ -0,0 +1,112 @@
import * as THREE from "three";
import { GLTFLoader } from "three/addons/loaders/GLTFLoader.js";
import { fetchChecked } from "../loader/PackageLoader.js";
// Body GLBs must be self-contained: external image/buffer URIs would bypass
// PackageLoader's same-origin, hosting-prefix and signed-URL handling.
export function validateBodyGlb(buffer) {
const view = new DataView(buffer);
if (
buffer.byteLength < 20 ||
view.getUint32(0, true) !== 0x46546c67 ||
view.getUint32(4, true) !== 2 ||
view.getUint32(8, true) !== buffer.byteLength ||
view.getUint32(16, true) !== 0x4e4f534a ||
view.getUint32(12, true) > buffer.byteLength - 20
)
throw new Error("Body visual: invalid GLB (check git lfs pull)");
const json = JSON.parse(
new TextDecoder().decode(
new Uint8Array(buffer, 20, view.getUint32(12, true)),
),
);
for (const asset of [...(json.buffers ?? []), ...(json.images ?? [])])
if (asset.uri && !asset.uri.startsWith("data:"))
throw new Error("Body visual: GLB must embed all buffers and images");
return json;
}
// Visual-only layer. Nothing here changes the compiled model, qpos, collision
// geoms or policy inputs. Each attachment follows a MuJoCo body, not an
// independently animated skeleton or the compiler-recentered STL geom frame.
export class BodyVisuals {
static async load(mujoco, model, data, pkg) {
const specs = pkg.config.render.body_visuals ?? [];
const ids = specs.map(({ body }) => {
const id = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_BODY.value, body);
if (id <= 0) throw new Error(`Body visual: missing robot body ${body}`);
return id;
});
const visuals = new BodyVisuals(data);
const loader = new GLTFLoader();
try {
// Sequential parsing keeps ownership unambiguous on a failed candidate:
// no outstanding parse may allocate resources after dispose() returns.
for (let i = 0; i < specs.length; i++) {
const spec = specs[i];
const bytes = await fetchChecked(pkg.asset(spec.path), "bytes");
validateBodyGlb(bytes);
const gltf = await loader.parseAsync(bytes, "");
// Retain every scene for cleanup, including non-default glTF scenes.
visuals.scenes.push(...gltf.scenes);
const frame = new THREE.Group();
frame.name = `body-visual:${spec.body}`;
frame.userData.bodyId = ids[i];
gltf.scene.position.fromArray(spec.position ?? [0, 0, 0]);
gltf.scene.quaternion.fromArray(spec.quaternion ?? [0, 0, 0, 1]);
frame.add(gltf.scene);
visuals.root.add(frame);
visuals.items.push({ id: ids[i], frame });
}
visuals.sync();
return visuals;
} catch (error) {
visuals.dispose();
throw error;
}
}
constructor(data) {
this.data = data;
this.root = new THREE.Group();
this.root.name = "body-visuals";
this.items = [];
this.scenes = [];
}
sync() {
for (const { id, frame } of this.items) {
frame.position.fromArray(this.data.xpos, id * 3);
const a = id * 4,
q = this.data.xquat;
frame.quaternion.set(q[a + 1], q[a + 2], q[a + 3], q[a]);
}
}
dispose() {
if (this.disposed) return;
this.disposed = true;
const geometries = new Set(),
materials = new Set(),
textures = new Set(),
images = new Set();
for (const scene of this.scenes)
scene.traverse((object) => {
if (object.geometry) geometries.add(object.geometry);
for (const material of [object.material].flat().filter(Boolean)) {
materials.add(material);
for (const value of Object.values(material))
if (value?.isTexture) textures.add(value);
}
});
for (const geometry of geometries) geometry.dispose();
for (const material of materials) material.dispose();
for (const texture of textures) {
for (const image of [texture.source?.data].flat().filter(Boolean))
images.add(image);
texture.dispose();
}
for (const image of images) image.close?.();
this.root.removeFromParent();
this.root.clear();
this.scenes = [];
this.items = [];
}
}
+16
View File
@@ -1,5 +1,6 @@
import * as THREE from "three";
import { applyWorldGrid } from "./WorldRenderer.js";
import { BodyVisuals } from "./BodyVisuals.js";
// Geometry is compiled MuJoCo geometry, not raw STL: compiler-applied mesh
// scale/centering is already represented by vertices + geom transforms.
export class RigRenderer {
@@ -91,7 +92,21 @@ export class RigRenderer {
throw error;
}
}
async loadBodyVisuals(mujoco, pkg) {
if (!pkg.config.render.body_visuals?.length) return;
this.bodyVisuals = await BodyVisuals.load(
mujoco,
this.model,
this.data,
pkg,
);
const replaced = new Set(this.bodyVisuals.items.map(({ id }) => id));
for (const { id, mesh } of this.items)
if (replaced.has(this.model.geom_bodyid[id])) mesh.visible = false;
this.root.add(this.bodyVisuals.root);
}
sync() {
this.bodyVisuals?.sync();
const p = this.data.geom_xpos,
r = this.data.geom_xmat,
m = this.matrix;
@@ -120,6 +135,7 @@ export class RigRenderer {
}
}
dispose() {
this.bodyVisuals?.dispose();
for (const g of this.geometries.values()) g.dispose();
for (const m of this.materials) m.dispose();
this.geometries.clear();
+76 -1
View File
@@ -16,6 +16,8 @@ import { SkillManager } from "../skills/SkillManager.js";
import { InputRouter } from "../input/InputRouter.js";
import { RigRenderer } from "../render/RigRenderer.js";
import { ChaseCamera } from "../render/ChaseCamera.js";
import { PhysicsDrag } from "../input/PhysicsDrag.js";
import { createInteractivePropVisuals } from "../../game/interactive-props.js";
export class RobotRuntime {
static async load(options) {
@@ -89,6 +91,7 @@ export class RobotRuntime {
}
const { ctx, obs, actions, controller } = policies.get(primary.id);
rig = new RigRenderer(sim.model, sim.data, { ...config.render, world });
await rig.loadBodyVisuals(mujoco, pkg);
worldPresentation = await options.createWorldPresentation?.({
sim,
rig,
@@ -174,7 +177,14 @@ export class RobotRuntime {
},
});
this.scene.add(this.rig.root);
if (this.worldPresentation) this.scene.add(this.worldPresentation.root);
this.lineColor = this.environmentSettings?.lineColor ?? "classic";
if (this.worldPresentation) {
this.scene.add(this.worldPresentation.root);
this.worldPresentation.setLineColor(this.lineColor);
}
this.propsVisuals = createInteractivePropVisuals(this.sim.interactiveSpecs);
this.scene.add(this.propsVisuals.root);
this.syncProps();
this.follow = new THREE.Vector3();
}
activate({ paused = false } = {}) {
@@ -191,6 +201,35 @@ export class RobotRuntime {
this.config.render.camera,
);
this.restoreCamera();
this.drag = new PhysicsDrag({
sim: this.sim,
camera: this.camera,
controls: this.controls,
canvas: this.renderer.domElement,
objects: () => [this.rig.root, this.propsVisuals.root],
enabled: () => this.active && this.scheduler.running && !this.disposed,
halfExtent: this.world.boundary?.half_extent,
scale: this.world.scale ?? 1,
addressFor: (object) => {
const index = object.userData.interactiveIndex;
if (Number.isInteger(index)) return this.sim.propAddresses[index];
const id = object.userData.geomId;
let visualBody = false;
for (let node = object; node; node = node.parent)
if (node.userData.bodyId > 0) visualBody = true;
if (
!visualBody &&
(id === undefined || this.sim.model.geom_bodyid[id] === 0)
)
return null;
return {
bodyId: this.sim.root,
qAdr: this.sim.rootQpos,
dofAdr: this.sim.rootDof,
mass: this.sim.model.body_subtreemass[this.sim.root],
};
},
});
this.input.init();
if (!paused) this.resume();
else this.onStatus({ paused: true });
@@ -212,6 +251,11 @@ export class RobotRuntime {
}
status() {
return {
lineColor: this.lineColor,
runtimeCapabilities: {
environmentControls: !!this.worldPresentation,
physicalDrag: true,
},
policyStatus: this.policyStatus(),
actuationLabel:
this.config.actuation.type === "joint_pd"
@@ -219,6 +263,18 @@ export class RobotRuntime {
: "",
};
}
setLineColor(name) {
if (!this.worldPresentation?.setLineColor(name)) return;
this.lineColor = name;
this.onStatus({ lineColor: name });
}
syncProps() {
this.propsVisuals.sync(
this.sim.data,
this.sim.propAddresses,
!this.sim.propsParked,
);
}
policyStatus() {
return this.policyState.episodes.size
? `Skill: ${this.policyState.active} · ${this.policyState.state}`
@@ -240,6 +296,7 @@ export class RobotRuntime {
if (!paused) this.resume();
}
async pause() {
this.drag?.release();
this.input.clear();
await this.scheduler.pause();
this.onStatus({ paused: true });
@@ -259,6 +316,7 @@ export class RobotRuntime {
async #reset() {
const running = this.scheduler.running;
await this.scheduler.pause();
this.drag?.release();
this.input.clear();
this.ctx.command.fill(0);
for (const policy of this.policies.values()) policy.reset();
@@ -277,6 +335,7 @@ export class RobotRuntime {
this.onStatus({ fallen: false });
this.publishPolicyStatus();
this.rig.sync();
this.syncProps();
if (running) this.scheduler.start();
}
triggerAction(action) {
@@ -312,8 +371,10 @@ export class RobotRuntime {
}
for (let i = 0; i < this.config.sim.decimation; i++) {
this.actuation.apply();
this.drag?.apply();
this.sim.step();
}
this.sim.restoreEscapedProps(this.world.boundary?.half_extent);
const q = this.sim.data.qpos,
a = this.sim.rootQpos,
quat = this.sim.data.xquat,
@@ -355,6 +416,7 @@ export class RobotRuntime {
frame() {
if (this.disposed) return;
this.rig.sync();
this.syncProps();
if (this.chaseCamera) {
const yaw = this.cameraPose();
this.worldPresentation?.update(this.follow);
@@ -394,6 +456,17 @@ export class RobotRuntime {
},
scene: this.scene,
world: this.world,
get interactiveProps() {
return self.sim.interactiveSpecs.map((spec, i) => ({
...spec,
...self.sim.propAddresses[i],
}));
},
setPropsParked: (parked) => {
this.drag?.release();
this.sim.setPropsParked(parked);
this.syncProps();
},
command: this.input.command,
get lastAction() {
return self.policies.get(self.activePolicy).ctx.lastAction;
@@ -443,6 +516,8 @@ export class RobotRuntime {
this.active = false;
await this.scheduler.dispose();
this.input.dispose();
this.drag?.dispose();
this.propsVisuals.dispose();
this.chaseCamera?.dispose();
this.controls?.dispose();
this.worldPresentation?.dispose();
@@ -82,6 +82,9 @@ export class RuntimeManager {
camera,
renderer: this.renderer,
autoStart: false,
environmentSettings: {
lineColor: this.getState().lineColor ?? "classic",
},
publish,
onStatus: publish,
onTelemetry: (telemetry) => publish({ telemetry }),
+4 -1
View File
@@ -1,5 +1,6 @@
import { arenaWalls } from "../worlds/arena.js";
export function composeScene(xml, config, world) {
import { appendInteractivePhysics } from "../../game/worlds/arena/session-environment.js";
export function composeScene(xml, config, world, interactiveProps = []) {
const doc = new DOMParser().parseFromString(xml, "text/xml");
if (
doc.querySelector("parsererror") ||
@@ -9,6 +10,7 @@ export function composeScene(xml, config, world) {
const option =
doc.querySelector("mujoco > option") ?? doc.createElement("option");
option.setAttribute("timestep", String(config.sim.timestep));
if (interactiveProps.length) option.setAttribute("jacobian", "dense");
if (!option.parentNode) doc.documentElement.append(option);
if (world.type !== "flat")
throw new Error(`World ${world.id}: generic composer requires flat World`);
@@ -44,5 +46,6 @@ export function composeScene(xml, config, world) {
worldbody.append(geom);
}
}
appendInteractivePhysics(doc, interactiveProps);
return new XMLSerializer().serializeToString(doc);
}
+47 -3
View File
@@ -1,5 +1,10 @@
import { fetchChecked } from "../loader/PackageLoader.js";
import { composeScene } from "./SceneComposer.js";
import { sharedInteractiveProps } from "../../game/worlds/arena/session-environment.js";
import {
resolvePropAddresses,
resetProp,
} from "../../game/interactive-props.js";
export function resolveAddresses(mujoco, model, names) {
const joints = new Map();
for (const name of names) {
@@ -42,14 +47,23 @@ export class SimCore {
pkg.config.assets.forEach((a, i) =>
vfs.addBuffer(a.vfs, new Uint8Array(buffers[i])),
);
const interactiveSpecs = sharedInteractiveProps(world);
const xml = composeScene(
await fetchChecked(pkg.asset(pkg.config.model)),
pkg.config,
world,
interactiveSpecs,
);
model = mujoco.MjModel.from_xml_string(xml, vfs);
data = new mujoco.MjData(model);
return new SimCore(mujoco, model, data, vfs, pkg.config);
return new SimCore(
mujoco,
model,
data,
vfs,
pkg.config,
interactiveSpecs,
);
} catch (error) {
data?.delete();
model?.delete();
@@ -57,8 +71,10 @@ export class SimCore {
throw error;
}
}
constructor(mujoco, model, data, vfs, config) {
Object.assign(this, { mujoco, model, data, vfs, config });
constructor(mujoco, model, data, vfs, config, interactiveSpecs = []) {
Object.assign(this, { mujoco, model, data, vfs, config, interactiveSpecs });
this.propAddresses = resolvePropAddresses(mujoco, model, interactiveSpecs);
this.propsParked = false;
const names = [...new Set(Object.values(config.joints.groups).flat())];
this.joints = resolveAddresses(mujoco, model, names);
for (const [name, j] of this.joints) {
@@ -119,8 +135,36 @@ export class SimCore {
for (const [name, joint] of this.joints)
data.qpos[joint.qpos] =
config.base.initial?.[name] ?? defaults.get(name) ?? 0;
this.resetProps();
mujoco.mj_forward(model, data);
}
resetProps() {
this.interactiveSpecs.forEach((spec, i) =>
resetProp(this.data, this.propAddresses[i], spec, this.propsParked, i),
);
}
setPropsParked(parked) {
this.propsParked = parked;
this.resetProps();
this.mujoco.mj_forward(this.model, this.data);
}
restoreEscapedProps(halfExtent) {
if (this.propsParked || !halfExtent) return;
let restored = false;
this.interactiveSpecs.forEach((spec, i) => {
const a = this.propAddresses[i].qAdr,
q = this.data.qpos;
if (
Math.abs(q[a]) > halfExtent + 0.1 ||
Math.abs(q[a + 1]) > halfExtent + 0.1 ||
q[a + 2] < -0.2
) {
resetProp(this.data, this.propAddresses[i], spec);
restored = true;
}
});
if (restored) this.mujoco.mj_forward(this.model, this.data);
}
step() {
this.mujoco.mj_step(this.model, this.data);
}
+107 -199
View File
@@ -38,8 +38,6 @@ import { ObservationPipeline } from "../../platform/policy/ObservationPipeline.j
import { ActionPipeline } from "../../platform/policy/ActionPipeline.js";
import { NativeActuation } from "../../platform/sim/actuation/JointPdActuation.js";
import {
BALL_RADIUS,
BALL_PARK_POS,
ARENA_HALF as LEGACY_ARENA_HALF,
SPAWN_X,
SPAWN_Y,
@@ -91,7 +89,13 @@ import {
playLineBlip,
setRumble,
} from "../../game/audio.js";
import { createBallActor } from "../../game/ball-actor.js";
import {
resolvePropAddresses,
resetProp,
createInteractivePropVisuals,
} from "../../game/interactive-props.js";
import { setArenaLineColor } from "../../game/worlds/arena/line-color.js";
import { sharedInteractiveProps } from "../../game/worlds/arena/session-environment.js";
import {
loadCustomPolicy as loadCustomPolicyModule,
scriptCommandAt,
@@ -100,7 +104,6 @@ import {
makeInfiniteGrid,
makeArenaWalls,
} from "../../game/worlds/arena/render.js";
import { createBallVisual } from "../../game/ball-visual.js";
import {
useGame,
gameApi,
@@ -133,6 +136,7 @@ export async function bootGame({
world,
autoStart = true,
publish,
environmentSettings,
}) {
const scope = new Disposables();
try {
@@ -149,6 +153,7 @@ export async function bootGame({
world,
autoStart,
publish,
environmentSettings,
});
} catch (err) {
await scope.dispose().catch(console.error);
@@ -167,6 +172,7 @@ async function boot({
world,
autoStart,
publish,
environmentSettings,
}) {
const robot = pkg.config;
const boundary = world.boundary ?? {
@@ -175,6 +181,14 @@ async function boot({
wall_thickness: 0.05,
};
const ARENA_HALF = boundary.half_extent;
const params = new URLSearchParams(location.search);
const testMode = params.get("test") === "1";
const legacyOracle = testMode && params.get("oracle") === "legacy";
const interactiveSpecs = legacyOracle
? (await import("./test-oracle.js")).originalPhysicsProbe()
: sharedInteractiveProps(world);
let propsParked = false;
let propVisuals = null;
const rollerPkg = await loadPackage(
`${import.meta.env.BASE_URL}robots/microduck-rollers/robot.json`,
);
@@ -214,7 +228,6 @@ async function boot({
const bootHalt = (detail) => {
if (!publish) globalBootHalt(detail);
};
const testMode = new URLSearchParams(location.search).get("test") === "1";
const clock = new SimulationClock();
const listen = (target, event, callback, options) =>
scope.listen(target, event, callback, options);
@@ -295,6 +308,8 @@ async function boot({
jointNames: JOINT_NAMES,
defaultPose: DEFAULT_POSE,
spawn: robot.base.spawn,
interactiveProps: interactiveSpecs,
floor: legacyOracle ? undefined : world.floor,
boundary,
});
}
@@ -344,8 +359,10 @@ async function boot({
doneMeshes(`${meshFiles.length} FILES`);
const sessions = {};
// Always boot on the classic (orange) colourway; the quickbar re-skins live.
// The robot retains its palette; the quickbar now tints arena lines.
let currentVariant = DEFAULT_VARIANT;
let lineColor =
environmentSettings?.lineColor ?? store().lineColor ?? DEFAULT_VARIANT;
const rigPromise = (async () => {
const doneRig = bootLine("RENDER RIG");
try {
@@ -566,8 +583,7 @@ async function boot({
mujoco.mjtObj.mjOBJ_KEY.value,
"STAND",
),
ballQposAdr: model.jnt_qposadr[jointId("ball_freejoint")],
ballDofAdr: model.jnt_dofadr[jointId("ball_freejoint")],
propAddresses: resolvePropAddresses(mujoco, model, interactiveSpecs),
// Foot bodies for the footstep audio heuristic (-1 when a variant
// has no ankles, e.g. if a future model renames them).
ankleIds: ["ankle_left", "ankle_right"].map((n) =>
@@ -594,8 +610,7 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
} = resolveAddrs(model, k);
@@ -651,11 +666,10 @@ async function boot({
// Right-stick camera state, read by the telemetry before the camera-orbit
// section below has evaluated.
let padOrbitLive = false;
// Robot input gate: twist commands, mode changes, rolls, kicks and ball
// spawns all stay inert until the entrance sequence has fully played out.
// Robot input gate: twist commands, mode changes, rolls and kicks
// stay inert until the entrance sequence has fully played out.
let inputLocked = true;
let ceremony = null;
let ball = null;
let stickers = null; // comic popups, currently disabled
let mode = "walk"; // "walk" | "sitstand" | "roll" | "kickL" | "kickR" | "crouch" | "groundpick"
@@ -678,9 +692,6 @@ async function boot({
const HEAD_SIGNS = new Float32Array([1, -1, 1, -1]);
const headTarget = new Float32Array(4);
const headSmooth = new Float32Array(4);
// Local-only kickable ball: false while parked at the keyframe spot
// (mesh hidden), true once popped in front of the duck.
let ballActive = false;
// The twist the policy actually receives. Mid-roll every movement input
// is ignored (zero twist) until the roll hands back to walk on its own.
@@ -823,74 +834,26 @@ async function boot({
headTarget.fill(0);
headSmooth.fill(0);
mujoco.mj_resetDataKeyframe(model, data, standKeyId);
interactiveSpecs.forEach((spec, i) =>
resetProp(data, propAddresses[i], spec, propsParked, i),
);
mujoco.mj_forward(model, data);
lastAction.fill(0);
sitFlag = 0;
// Park the ball in physics immediately; if it was on screen, the
// reverse scan peels it away at its last pose. A queued B-respawn is
// cancelled: a reset means no ball.
ball?.despawn({ cancelQueued: true, parkPhysics: parkBallPhysics });
ballActive = false;
propVisuals?.sync(data, propAddresses, !propsParked);
syncButtons();
if (!testMode && ceremony) {
startCameraReset();
spawnBall({ fromQueue: true });
}
if (!testMode && ceremony) startCameraReset();
}
resetSim();
function parkBallPhysics() {
const qpos = data.qpos,
qvel = data.qvel;
qpos[ballQposAdr] = 50;
qpos[ballQposAdr + 1] = 0;
qpos[ballQposAdr + 2] = BALL_RADIUS;
qpos[ballQposAdr + 3] = 1;
qpos[ballQposAdr + 4] = 0;
qpos[ballQposAdr + 5] = 0;
qpos[ballQposAdr + 6] = 0;
for (let i = 0; i < 6; i++) qvel[ballDofAdr + i] = 0;
mujoco.mj_forward(model, data);
ballActive = false;
}
// Pop / respawn the ball ~0.35 m in front of the duck, with a small
// random heading + distance jitter. If the ball is already on screen,
// peel it away first (reverse scan) and pop the new one when that
// finishes - same appear/disappear pair as the duck's wireframe ceremony.
function spawnBall(opts = {}) {
if (inputLocked && !opts.fromQueue) return;
if (!ball) return;
if (ball.visual !== "hidden") {
ball.queueRespawn();
ball.despawn({ parkPhysics: parkBallPhysics });
return;
}
const qpos = data.qpos,
qvel = data.qvel;
const yaw = Math.atan2(
2 * (qpos[3] * qpos[6] + qpos[4] * qpos[5]),
1 - 2 * (qpos[5] * qpos[5] + qpos[6] * qpos[6]),
function setPropsParked(parked) {
endGrabHook();
propsParked = parked;
interactiveSpecs.forEach((spec, i) =>
resetProp(data, propAddresses[i], spec, parked, i),
);
const heading = yaw + (Math.random() - 0.5) * 0.7;
const dist = 0.35 + (Math.random() - 0.5) * 0.1;
const lim = ARENA_HALF - BALL_RADIUS - 0.05;
const clamp = (v) => Math.min(lim, Math.max(-lim, v));
qpos[ballQposAdr] = clamp(qpos[0] + Math.cos(heading) * dist);
qpos[ballQposAdr + 1] = clamp(qpos[1] + Math.sin(heading) * dist);
qpos[ballQposAdr + 2] = BALL_RADIUS + 0.02;
qpos[ballQposAdr + 3] = 1;
qpos[ballQposAdr + 4] = 0;
qpos[ballQposAdr + 5] = 0;
qpos[ballQposAdr + 6] = 0;
for (let i = 0; i < 6; i++) qvel[ballDofAdr + i] = 0;
mujoco.mj_forward(model, data);
ballActive = true;
// Snap the mesh to the new pose BEFORE the scan starts: the FX
// recomputes its bbox from the live mesh.
ball.poseFromQpos(qpos, ballQposAdr);
ball.appear();
stickers?.pop("spawn");
propVisuals?.sync(data, propAddresses, !parked);
}
// ── Observation ─────────────────────────────────────────────────────
@@ -1015,14 +978,13 @@ async function boot({
return null;
}
// ── Sim-driven audio (footsteps / roller rumble / ball impacts) ──────
// ── Sim-driven audio (footsteps / roller rumble / body bumps) ───────
// Footsteps use a per-foot height heuristic instead of MuJoCo contacts
// (the WASM bindings expose no contact array): each ankle's height
// RELATIVE to the lower ankle (the planted foot approximates the local
// ground, so the measure self-calibrates on the relief terrain), with
// lift/contact hysteresis and a per-foot debounce. Tap gain scales with
// the landing speed. Rollers get a speed-following rumble loop instead,
// and the ball thumps on velocity deltas between control steps.
// the landing speed. Rollers get a speed-following rumble loop instead.
const STEP_LIFT = 0.012; // m above the planted foot = foot in swing
const STEP_CONTACT = 0.005; // m: dropping below this while in swing = step
const STEP_DEBOUNCE_MS = 130;
@@ -1031,18 +993,11 @@ async function boot({
{ air: false, prevZ: 0, lastAt: 0 },
];
const duckEmitter = createEmitter({ refDistance: 0.5 });
const ballEmitter = createEmitter({ refDistance: 0.5 });
scope.add(() => {
duckEmitter.node.disconnect();
ballEmitter.node.disconnect();
});
const ballPrevV = [0, 0, 0];
let ballPrevValid = false;
let ballThumpAt = 0;
scope.add(() => duckEmitter.node.disconnect());
// Body bumps: trunk velocity deltas gated on obstacle proximity (walls
// or prop collider footprints) so the walking gait's own accelerations
// (comparable in magnitude to a slow wall hit) never false-trigger.
// Duller than the ball thumps: same samples pitched way down.
// Low-pitched impact samples for body bumps.
const BUMP_DV = 0.35; // m/s per control step
const BUMP_DEBOUNCE_MS = 300; // rubbing a wall must not machine-gun
const BUMP_WALL_PAD = 0.16; // trunk half-width-ish reach to a wall
@@ -1126,7 +1081,7 @@ async function boot({
const u = Math.min(1, (dv - BUMP_DV) / 1.5);
playSfx("thump", {
gain: 0.12 + 0.3 * u,
rate: 0.55 + 0.15 * u + Math.random() * 0.06, // way below the ball's range
rate: 0.55 + 0.15 * u + Math.random() * 0.06, // low-pitched body impact
out: duckEmitter.node,
});
haptics.pulse("bump", 0.5 + 0.5 * u); // shove felt in the hands
@@ -1139,42 +1094,6 @@ async function boot({
} else {
bumpPrevValid = false;
}
// Ball impacts: |dv| between control steps. Gravity alone accounts for
// ~0.2 m/s per 20 ms step; the 0.5 threshold clears it and rolling noise.
if (ballActive) {
const v = data.qvel;
const b = ballDofAdr;
if (ballPrevValid) {
const dv = Math.hypot(
v[b] - ballPrevV[0],
v[b + 1] - ballPrevV[1],
v[b + 2] - ballPrevV[2],
);
if (dv > 0.5 && now - ballThumpAt > 90) {
ballThumpAt = now;
const u = Math.min(1, (dv - 0.5) / 3.5); // full at kick-grade hits
playSfx("thump", {
gain: 0.14 + 0.4 * u,
rate: 1.25 - 0.45 * u + Math.random() * 0.08,
out: ballEmitter.node,
});
// Haptics only when the duck plausibly caused/received the hit:
// a far bounce off a wall shouldn't shake the hands.
const bq = ballQposAdr;
const dBall = Math.hypot(
data.qpos[bq] - data.qpos[0],
data.qpos[bq + 1] - data.qpos[1],
);
if (dBall < 0.6) haptics.pulse("ballHit", 0.4 + 0.6 * u);
}
}
ballPrevV[0] = v[b];
ballPrevV[1] = v[b + 1];
ballPrevV[2] = v[b + 2];
ballPrevValid = true;
} else {
ballPrevValid = false;
}
// Haptic channel scheduler: keeps the roller bed alive between pulses
// and cuts the motors when it falls silent. 50 Hz, like everything here.
haptics.tick();
@@ -1220,7 +1139,7 @@ async function boot({
applyGrabForce(); // mouse perturbation, fresh velocity every substep
mujoco.mj_step(model, data);
}
stepAudioSim(); // footsteps / rumble / ball thumps off the fresh state
stepAudioSim(); // footsteps / rumble / body bumps off the fresh state
// Command script clock: 50 Hz like everything here. A non-looping
// script ends on its own; a looping one runs until R (or a reset).
@@ -1317,14 +1236,22 @@ async function boot({
fallenSince = null;
}
// Ball respawn watchdog: outside the arena bounds means "escaped
// through a solver glitch", bring it back near the duck.
if (ballActive) {
const q = data.qpos;
const escaped =
Math.abs(q[ballQposAdr]) > ARENA_HALF + 0.1 ||
Math.abs(q[ballQposAdr + 1]) > ARENA_HALF + 0.1;
if (escaped) spawnBall();
// Return escaped toys individually without disturbing the robot or peers.
if (!propsParked) {
let restored = false;
interactiveSpecs.forEach((spec, i) => {
const a = propAddresses[i].qAdr,
q = data.qpos;
if (
Math.abs(q[a]) > ARENA_HALF + 0.1 ||
Math.abs(q[a + 1]) > ARENA_HALF + 0.1 ||
q[a + 2] < -0.2
) {
resetProp(data, propAddresses[i], spec);
restored = true;
}
});
if (restored) mujoco.mj_forward(model, data);
}
if (postKickLock > 0 && mode === "walk") postKickLock--;
@@ -1469,7 +1396,7 @@ async function boot({
// Released before models/sessions, even when boot or a later lazy load fails.
scope.add(() => scheduler.dispose());
// ── Scene wiring (grid, walls, rig, ball, arcade row) ────────────────
// ── Scene wiring (grid, walls, rig, interactive toys, arcade row) ────
// The grid/walls carry ceremony-driven uReveal uniforms and per-frame
// focus updates, so the game owns them; lights and environment live in
// the R3F layer.
@@ -1478,6 +1405,11 @@ async function boot({
reliefGridMat = grid.material; // relief drive mirrors uTopoScale into it
const { wallMats, wallMeshes } = makeArenaWalls(boundary);
for (const m of wallMeshes) scene.add(m);
grid.name = "arena-grid";
setArenaLineColor([grid.material, ...wallMats], lineColor);
propVisuals = createInteractivePropVisuals(interactiveSpecs);
scene.add(propVisuals.root);
propVisuals.sync(data, propAddresses, !propsParked);
let rig = await rigPromise;
scene.add(rig.placer);
@@ -1493,9 +1425,9 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
};
// ── Locomotion variant switching (legs <-> rollers) ──────────────────
@@ -1572,8 +1504,7 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
} = L);
@@ -1664,10 +1595,6 @@ async function boot({
setLocked: (v) => {
inputLocked = v;
controller.setLocked(v);
// A ball is always in play: pop one the moment the entrance or a
// respawn ceremony hands control back.
if (!v && ball && !ballActive && !testMode)
spawnBall({ fromQueue: true });
},
flashReset: () => {},
// Audio twins of the wireframe materialize FX (entrance and respawns):
@@ -1693,19 +1620,6 @@ async function boot({
{ fireImmediately: true },
);
const { group: ballGroup, mesh: ballMesh } = createBallVisual(renderer);
scene.add(ballGroup);
ball = createBallActor({
THREE,
scene,
camera,
renderer,
fxModule: fx,
mesh: ballMesh,
group: ballGroup,
});
scope.add(() => ball.dispose());
// ── Prop library (wall/corner dressing + entrance FX) ────────────────
// Every enabled def in props.js: loaded, real-size scaled, floor
// snapped, wireframe-materialized with the ceremony (staggered after
@@ -1876,7 +1790,7 @@ async function boot({
}
// ── Mouse grab, pointer side (pick + drag target + cursor) ────────────
// Pointer-down on the duck (or the live ball) grabs it; anywhere else
// Pointer-down on the duck or an interactive toy grabs it; anywhere else
// falls through to OrbitControls untouched. The pick is a three.js
// raycast against the render rig (the WASM bindings expose no
// mjv_select, and the rig IS the duck's collision-accurate silhouette
@@ -1902,13 +1816,19 @@ async function boot({
}
function grabPick() {
const duckHit = _grabRaycaster.intersectObject(rig.placer, true)[0];
const ballHit = ballActive
? _grabRaycaster.intersectObject(ballMesh, true)[0]
const propHit = !propsParked
? _grabRaycaster.intersectObjects(propVisuals.meshes, false)[0]
: undefined;
if (duckHit && (!ballHit || duckHit.distance <= ballHit.distance)) {
if (duckHit && (!propHit || duckHit.distance <= propHit.distance)) {
return { kind: "duck", point: duckHit.point };
}
return ballHit ? { kind: "ball", point: ballHit.point } : null;
return propHit
? {
kind: "prop",
point: propHit.point,
index: propHit.object.userData.interactiveIndex,
}
: null;
}
function updateGrabTarget() {
if (!_grabRaycaster.ray.intersectPlane(_grabPlane, _grabHit)) return;
@@ -1956,18 +1876,7 @@ async function boot({
dofAdr: 0,
mass: model.body_subtreemass[trunkId],
}
: {
bodyId: mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
),
qAdr: ballQposAdr,
dofAdr: ballDofAdr,
mass: model.body_mass[
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_BODY.value, "ball")
],
};
: propAddresses[pick.index];
camera.getWorldDirection(_grabCamDir);
_grabPlane.setFromNormalAndCoplanarPoint(
_grabCamDir.negate(),
@@ -2050,8 +1959,7 @@ async function boot({
setJoint(rig, JOINT_NAMES[j], qpos[qposAdr[j]]);
// Passive hinges (roller wheels): purely visual, driven straight from qpos.
for (const ej of extraJoints) setJoint(rig, ej.name, qpos[ej.adr]);
// Ball: live follows qpos; ghost freeze is owned by the ball actor.
if (ball) ball.sync(qpos, ballQposAdr, ballActive);
propVisuals?.sync(data, propAddresses, !propsParked);
// Follow cam: ease the orbit target toward the trunk and translate the
// camera by the same delta, so the camera-to-duck distance and viewing
// angle stay constant while the duck walks. Paused while the reset
@@ -2372,18 +2280,9 @@ async function boot({
syncRig();
syncJaw();
ghosts?.update();
// Spatial audio follows the movers: listener on the camera, emitters
// on the duck trunk and the ball (MJCF Z-up -> three Y-up).
// Spatial audio: listener on the camera, emitter on the duck trunk.
updateListener(camera);
duckEmitter.setPosition(data.qpos[0], data.qpos[2], -data.qpos[1]);
if (ballActive) {
const q = data.qpos;
ballEmitter.setPosition(
q[ballQposAdr],
q[ballQposAdr + 2],
-q[ballQposAdr + 1],
);
}
controls.update();
updateChaseCam();
if (helperCamera !== camera) {
@@ -2391,7 +2290,6 @@ async function boot({
helperCamera.copy(camera);
}
ceremony.drive();
ball.drive(() => spawnBall({ fromQueue: true }));
waypointMarker.update(dt, waypointSource.target);
renderTelemetry();
}
@@ -2408,7 +2306,6 @@ async function boot({
controller.on("reset", () => gameApi.reset?.());
controller.on("reset", () => waypointSource.cancel());
controller.on("spawnBall", () => spawnBall());
controller.on("headToggle", () => toggleHeadMode());
controller.on("chaseToggle", () => startCameraReset());
controller.on("locoToggle", () => toggleLoco());
@@ -2604,7 +2501,7 @@ async function boot({
mode = foot === "left" ? "kickL" : "kickR";
sitFlag = 0;
kickRun = { steps: 0 };
haptics.pulse("kick"); // swing launch; ball contact adds ballHit
haptics.pulse("kick"); // swing launch
syncButtons();
stickers?.pop("kick");
return true;
@@ -2634,7 +2531,6 @@ async function boot({
? "Drive"
: "Run";
if (store().modeLabel !== label) setStore({ modeLabel: label });
if (store().ballActive !== ballActive) setStore({ ballActive });
}
// ── Public surface for the React UI ──────────────────────────────────
@@ -2666,9 +2562,14 @@ async function boot({
loco,
locoWant: loco,
variant: currentVariant,
lineColor,
modeLabel: store().modeLabel,
inputBindings: robot.input.bindings,
runtimeCapabilities: { duckControls: true, physicalDrag: true },
runtimeCapabilities: {
duckControls: true,
environmentControls: true,
physicalDrag: true,
},
}),
triggerAction: (action) => {
if (
@@ -2678,7 +2579,10 @@ async function boot({
)
kbSource.onAction(action);
},
saveSettings: () => ({ variant: currentVariant, loco: store().locoWant }),
saveSettings: () => ({
variant: currentVariant,
loco: store().locoWant,
}),
restoreSettings: async (settings) => {
if (settings?.variant) runtime.setVariant(settings.variant);
if (settings?.loco)
@@ -2727,13 +2631,17 @@ async function boot({
applyVariant(rig, name);
setStore({ variant: name });
},
setLineColor: (name) => {
if (!setArenaLineColor([grid.material, ...wallMats], name)) return;
lineColor = name;
setStore({ lineColor: name });
},
requestLoco: (name) => {
if (name !== "legs" && name !== "rollers") return;
setStore({ locoWant: name });
reconcileLoco();
},
resetSim,
spawnBall: () => spawnBall(),
startEntrance: () => ceremony.startEntrance(),
loadCustomPolicy: (ref) => loadCustomPolicyIntoGame(ref),
clearCustomPolicy: () => revertCustomPolicy(),
@@ -2768,7 +2676,13 @@ async function boot({
controller,
kbSource,
padSource,
spawnBall,
get interactiveProps() {
return interactiveSpecs.map((spec, i) => ({
...spec,
...propAddresses[i],
}));
},
setPropsParked,
triggerKick,
triggerRoll,
sessions,
@@ -2828,12 +2742,6 @@ async function boot({
toggleHeadMode,
headTarget,
headSmooth,
get ballActive() {
return ballActive;
},
get ballQposAdr() {
return ballQposAdr;
},
get chaseCam() {
return chaseCam;
},
@@ -2884,7 +2792,7 @@ async function boot({
await scheduler.pause();
ceremony.setReveal(1);
ceremony.setFx(1);
parkBallPhysics();
setPropsParked(true);
inputLocked = false;
controller.setLocked(false);
}
+20
View File
@@ -0,0 +1,20 @@
// Archived policy goldens were recorded with exactly one parked sphere.
// Extra free DOFs change MuJoCo floating-point accumulation (especially
// wheel contact), even when the new toys are parked far away. Reproduce
// that original laboratory world ONLY for ?test=1&oracle=legacy. Normal
// simulation and scene-interaction tests always use the 5–7 toy layout.
export function originalPhysicsProbe() {
return [
{
name: "interactive_0",
type: "sphere",
size: [0.05],
pos: [50, 0, 0.05],
quat: [1, 0, 0, 0],
mass: 0.03,
friction: "0.4 0.01 0.003",
solref: "0.03 0.4",
color: 0xf2efe8,
},
];
}
+8 -7
View File
@@ -17,7 +17,7 @@ function Game() {
room.dispose();
pmrem.dispose();
scene.environment = environment.texture;
scene.environmentIntensity = 0.45;
scene.environmentIntensity = 0.7;
bootLog.length = 0;
const manager = new RuntimeManager({
scene,
@@ -44,7 +44,7 @@ function Game() {
for (const method of [
"setVariant",
"requestLoco",
"spawnBall",
"setLineColor",
"loadCustomPolicy",
"clearCustomPolicy",
"triggerAction",
@@ -68,20 +68,21 @@ function Game() {
}, [scene, camera, gl]);
useEffect(() => {
scene.background = new THREE.Color(
theme === "neutral" ? 0x18202b : 0x08080c,
theme === "neutral" ? 0x18202b : 0x151923,
);
gl.setClearColor(scene.background, 1);
}, [scene, gl, theme]);
useFrame((_, dt) => gameApi.frame?.(Math.min(dt, 0.05)));
return (
<>
<ambientLight intensity={0.6} />
<directionalLight position={[2, 4, 2]} intensity={1.6} />
<directionalLight position={[-2, 2, 1.5]} intensity={0.4} />
<ambientLight intensity={0.85} />
<hemisphereLight args={[0xe4edff, 0x66605a, 0.75]} />
<directionalLight position={[2, 4, 2]} intensity={2.0} />
<directionalLight position={[-2, 2, 1.5]} intensity={0.8} />
<directionalLight
color={theme === "neutral" ? 0xbcdcff : 0xffb366}
position={[0, 3, -2]}
intensity={0.7}
intensity={0.85}
/>
</>
);
+1 -1
View File
@@ -38,6 +38,7 @@ export const useGame = create(
locoSwitching: false,
rollersLoading: false, // OSD line while the roller stack streams in
variant: "classic",
lineColor: "classic",
// Community move: null = all-official slots. Written by the game
// (game.js loadCustomPolicyIntoGame / the watchdog) and by App.jsx for
// the boot-time ?move= request.
@@ -47,7 +48,6 @@ export const useGame = create(
customPolicy: null,
padConnected: false,
touchMode: false,
ballActive: false,
// Throttled telemetry block (4 Hz), bottom-right OSD
telemetry: { fps: 0, ctrlHz: 0, speed: 0, odo: 0, peers: 0 },
})),
+1 -1
View File
@@ -125,7 +125,7 @@ export default function ControlsHelp() {
{physicalDrag && (
<>
<br />
Drag robot / ball: apply force
Drag robot / geometry: apply force
</>
)}
</div>
+88 -59
View File
@@ -182,7 +182,8 @@ const hudHitSx = {
};
function Quickbar() {
const variant = useGame((s) => s.variant);
const lineColor = useGame((s) => s.lineColor);
const duckControls = useGame((s) => s.runtimeCapabilities.duckControls);
const locoWant = useGame((s) => s.locoWant);
return (
<Box
@@ -198,10 +199,10 @@ function Quickbar() {
maxWidth: "calc(100vw - 14.5rem)",
}}
>
<HudPlate caption="Color">
<HudPlate caption="Line color">
<Box
role="group"
aria-label="Duck colours"
aria-label="Scene line colours"
sx={{
display: "inline-flex",
alignItems: "stretch",
@@ -210,17 +211,20 @@ function Quickbar() {
}}
>
{Object.entries(VARIANT_SWATCH_HEX).map(([name, hex]) => {
const selected = name === variant;
const selected = name === lineColor;
const label = VARIANT_LABELS[name] ?? name;
return (
<Box
key={name}
component="button"
type="button"
title={label}
aria-label={`${label} colours`}
title={`${label} scene lines`}
aria-label={`${label} scene lines`}
aria-pressed={selected}
onClick={() => { uiClick(); gameApi.setVariant?.(name); }}
onClick={() => {
uiClick();
gameApi.setLineColor?.(name);
}}
sx={{
appearance: "none",
width: CELL_H, // square cell inside frame + padding
@@ -241,55 +245,64 @@ function Quickbar() {
outline: `2px dashed ${CREAM}`,
outlineOffset: -5,
},
"@media (prefers-reduced-motion: reduce)": { animation: "none" },
"@media (prefers-reduced-motion: reduce)": {
animation: "none",
},
}}
/>
);
})}
</Box>
</HudPlate>
<HudPlate caption="Mode">
<Box
role="group"
aria-label="Locomotion mode"
sx={{
display: "inline-flex",
alignItems: "stretch",
height: "100%",
gap: `${CELL_GAP}px`,
}}
>
{[
{ name: "legs", label: "Feet" },
{ name: "rollers", label: "Rollers" },
].map(({ name, label }) => {
const selected = locoWant === name;
return (
<Box
key={name}
component="button"
type="button"
aria-pressed={selected}
onClick={() => { uiClick(); gameApi.requestLoco?.(name); }}
sx={{
...hudHitSx,
px: "1.05rem",
background: selected ? ORANGE : "transparent",
color: selected ? COMIC_INK : "rgba(250, 248, 242, 0.72)",
"&:hover": {
color: selected ? COMIC_INK : CREAM,
background: selected ? ORANGE : "rgba(255, 122, 47, 0.12)",
},
"&:active": { filter: "brightness(0.92)" },
}}
>
{label}
</Box>
);
})}
</Box>
</HudPlate>
<PolicyChip />
{duckControls && (
<HudPlate caption="Mode">
<Box
role="group"
aria-label="Locomotion mode"
sx={{
display: "inline-flex",
alignItems: "stretch",
height: "100%",
gap: `${CELL_GAP}px`,
}}
>
{[
{ name: "legs", label: "Feet" },
{ name: "rollers", label: "Rollers" },
].map(({ name, label }) => {
const selected = locoWant === name;
return (
<Box
key={name}
component="button"
type="button"
aria-pressed={selected}
onClick={() => {
uiClick();
gameApi.requestLoco?.(name);
}}
sx={{
...hudHitSx,
px: "1.05rem",
background: selected ? ORANGE : "transparent",
color: selected ? COMIC_INK : "rgba(250, 248, 242, 0.72)",
"&:hover": {
color: selected ? COMIC_INK : CREAM,
background: selected
? ORANGE
: "rgba(255, 122, 47, 0.12)",
},
"&:active": { filter: "brightness(0.92)" },
}}
>
{label}
</Box>
);
})}
</Box>
</HudPlate>
)}
{duckControls && <PolicyChip />}
{/* Sound is fully disabled for now (SOUND_DISABLED in audio.js) -
the mute toggle comes back with the finished sound design. */}
{/* <SoundToggle /> */}
@@ -307,10 +320,15 @@ function PolicyChip() {
if (cp?.status !== "active") return null;
const title = cp.title || cp.name;
const by = cp.author ? ` by ${cp.author}` : "";
const hint = cp.slot === "trick" || cp.slot === "script" || cp.slot === "sitstand" ? " \u00b7 R" : "";
const hint =
cp.slot === "trick" || cp.slot === "script" || cp.slot === "sitstand"
? " \u00b7 R"
: "";
return (
<HudPlate caption="Playing" captionFill="orange">
<Box sx={{ display: "inline-flex", alignItems: "stretch", height: "100%" }}>
<Box
sx={{ display: "inline-flex", alignItems: "stretch", height: "100%" }}
>
<Box
sx={{
...hudHitSx,
@@ -331,7 +349,10 @@ function PolicyChip() {
component="button"
type="button"
aria-label={`Unload move ${cp.name}`}
onClick={() => { uiClick(); gameApi.clearCustomPolicy?.(); }}
onClick={() => {
uiClick();
gameApi.clearCustomPolicy?.();
}}
sx={{
...hudHitSx,
px: "0.6rem",
@@ -388,7 +409,8 @@ function SoundToggle() {
function Telemetry() {
const t = useGame((s) => s.telemetry);
const odo = t.odo < 1000 ? `${t.odo.toFixed(1)}M` : `${(t.odo / 1000).toFixed(2)}KM`;
const odo =
t.odo < 1000 ? `${t.odo.toFixed(1)}M` : `${(t.odo / 1000).toFixed(2)}KM`;
const lines = [];
if (t.peers) lines.push(`${t.peers + 1} ONLINE`);
lines.push(`${t.speed.toFixed(2)}M/S \u00b7 ODO ${odo}`);
@@ -446,7 +468,11 @@ function OsdLoad() {
{l}
<Box
component="span"
sx={{ display: "inline-block", ml: "0.15em", animation: `${recBlink} 0.8s steps(1) infinite` }}
sx={{
display: "inline-block",
ml: "0.15em",
animation: `${recBlink} 0.8s steps(1) infinite`,
}}
>
{"\u2588"}
</Box>
@@ -500,19 +526,22 @@ function PolicyToast() {
}
export default function Hud() {
const ready = useGame((s) => s.runtimePhase === 'ready');
const ready = useGame((s) => s.runtimePhase === "ready");
const touchMode = useGame((s) => s.touchMode);
const duckControls = useGame((s) => s.runtimeCapabilities.duckControls);
const environmentControls = useGame(
(s) => s.runtimeCapabilities.environmentControls,
);
if (!ready) return null;
if (!duckControls) return <Telemetry />;
if (!duckControls && !environmentControls) return <Telemetry />;
return (
<>
{!touchMode && (
<>
<Quickbar />
<Telemetry />
<OsdLoad />
<PolicyToast />
{duckControls && <OsdLoad />}
{duckControls && <PolicyToast />}
</>
)}
</>
+139
View File
@@ -0,0 +1,139 @@
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import * as THREE from "three";
import {
BodyVisuals,
validateBodyGlb,
} from "../src/platform/render/BodyVisuals.js";
import { validatePackage } from "../src/platform/loader/SchemaValidator.js";
const config = JSON.parse(
await fs.readFile(
new URL("../public/robots/microban/robot.json", import.meta.url),
),
);
function glb(json) {
const text = new TextEncoder().encode(JSON.stringify(json));
const buffer = new ArrayBuffer(20 + text.length);
const view = new DataView(buffer);
[0x46546c67, 2, buffer.byteLength, text.length, 0x4e4f534a].forEach(
(value, i) => view.setUint32(i * 4, value, true),
);
new Uint8Array(buffer, 20).set(text);
return buffer;
}
test("all 20 Microban body GLBs embed their materials/textures and stay visual-only", async () => {
validatePackage(config);
assert.equal(config.render.body_visuals.length, 20);
let textures = 0;
for (const { body, path } of config.render.body_visuals) {
assert.ok(!config.assets.some((asset) => asset.path === path));
const bytes = await fs.readFile(
new URL(`../public/robots/microban/${path}`, import.meta.url),
);
const json = validateBodyGlb(
bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength),
);
assert.ok(json.materials.length > 0);
if (body === "head")
assert.ok(
json.materials.some(
(material) => material.pbrMetallicRoughness?.baseColorTexture,
),
"microban2 screen colour texture must be preserved",
);
textures += json.images?.length ?? 0;
}
assert.ok(textures >= 20);
});
test("visual schema rejects duplicate bodies, non-unit quaternions and external/escaping paths", () => {
const duplicate = structuredClone(config);
duplicate.render.body_visuals.push(duplicate.render.body_visuals[0]);
assert.throws(() => validatePackage(duplicate), /duplicate body/);
const zero = structuredClone(config);
zero.render.body_visuals[0].quaternion = [0, 0, 0, 0];
assert.throws(() => validatePackage(zero), /unit length/);
for (const path of [
"https://host/a.glb",
"../a.glb",
"/a.glb",
"a.glb?x=1",
]) {
const invalid = structuredClone(config);
invalid.render.body_visuals[0].path = path;
assert.throws(() => validatePackage(invalid), /body_visuals/);
}
});
test("GLB validation fails closed on LFS pointers, truncation and external image/buffer requests", () => {
for (const bytes of [
new ArrayBuffer(0),
new TextEncoder().encode("version https://git-lfs.github.com/spec/v1")
.buffer,
glb({}).slice(0, -1),
])
assert.throws(() => validateBodyGlb(bytes), /invalid GLB/);
for (const json of [
{ images: [{ uri: "https://host/normal.png" }] },
{ buffers: [{ uri: "../buffer.bin" }] },
])
assert.throws(() => validateBodyGlb(glb(json)), /embed all/);
});
test("body attachments track MuJoCo body transforms without touching simulation arrays", () => {
const data = {
xpos: [0, 0, 0, 1, 2, 3],
xquat: [1, 0, 0, 0, 0.5, 0.5, 0.5, 0.5],
};
const before = structuredClone(data);
const visuals = new BodyVisuals(data),
frame = new THREE.Group();
visuals.items.push({ id: 1, frame });
visuals.sync();
assert.deepEqual(frame.position.toArray(), [1, 2, 3]);
assert.deepEqual(frame.quaternion.toArray(), [0.5, 0.5, 0.5, 0.5]);
assert.deepEqual(data, before);
});
test("shared GLB geometries, materials, textures and ImageBitmaps are released exactly once", () => {
const visuals = new BodyVisuals({}),
scene = new THREE.Group();
const geometry = new THREE.BoxGeometry(),
texture = new THREE.Texture();
let closed = 0;
texture.source.data = { close: () => closed++ };
const material = new THREE.MeshStandardMaterial({
map: texture,
normalMap: texture,
});
const counts = { geometry: 0, material: 0, texture: 0 };
geometry.addEventListener("dispose", () => counts.geometry++);
material.addEventListener("dispose", () => counts.material++);
texture.addEventListener("dispose", () => counts.texture++);
scene.add(
new THREE.Mesh(geometry, material),
new THREE.Mesh(geometry, material),
);
visuals.scenes.push(scene, scene);
const parent = new THREE.Group();
parent.add(visuals.root);
visuals.dispose();
visuals.dispose();
assert.deepEqual(counts, { geometry: 1, material: 1, texture: 1 });
assert.equal(closed, 1);
assert.equal(parent.children.length, 0);
});
test("missing body bindings fail before any asset request", async () => {
const mujoco = { mj_name2id: () => -1, mjtObj: { mjOBJ_BODY: { value: 1 } } };
await assert.rejects(
BodyVisuals.load(
mujoco,
{},
{},
{
config,
asset: () => {
throw new Error("must not fetch");
},
},
),
/missing robot body trunk/,
);
});
+2
View File
@@ -35,6 +35,8 @@ not proof that every possible long-running skill sequence is covered.
After the user-requested expansion to a 4.5 m default arena, this original-code oracle explicitly pins the **original 3 m World** through a test-only World response override. The production default remains 4.5 m, tested independently in `bounded-world.spec.js`. No accepted trajectory or tolerance was replaced to hide the intentional World-size change.
After replacing the production football with 5–7 random free-body geometries, the archived oracle additionally uses **`?test=1&oracle=legacy`**. `src/robots/microduck/test-oracle.js` supplies the original single parked sphere's dimensions, mass and contact parameters, with no football texture or gameplay. This keeps the oracle's model DOF count and floating-point accumulation unchanged; the nine accepted trajectories and exact comparisons remain untouched. `?test=1` alone still builds the real 5–7 toy scene; `prepareTest()` parks those toys for isolated stepping, and `scene-interaction.spec.js` explicitly restores them for real collision / drag tests. Production never selects the legacy probe unless both explicit test parameters are present. The Microduck composer fixes `jacobian="dense"` to retain the original solver path when the added toy DOFs cross MuJoCo's automatic sparse threshold.
`npm run test:regression` reads these files. `npm run baseline:record` writes
candidate files under ignored `test-results/baseline-candidates/`, never here.
A baseline replacement needs an explicit reviewed cause and updated provenance.
+141
View File
@@ -0,0 +1,141 @@
import test from "node:test";
import assert from "node:assert/strict";
import * as THREE from "three";
import {
generateInteractiveProps,
createInteractivePropVisuals,
resetProp,
} from "../src/game/interactive-props.js";
import { setArenaLineColor } from "../src/game/worlds/arena/line-color.js";
import {
makeInfiniteGrid,
makeArenaWalls,
} from "../src/game/worlds/arena/render.js";
import { VARIANT_SWATCH_HEX } from "../src/game/variants.js";
function rng(seed) {
return () => (seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
}
test("random layouts have 5–7 varied, separated, lightweight toys inside walls and clear of spawn", () => {
const counts = new Set();
for (let seed = 0; seed < 1000; seed++) {
for (const halfExtent of [1.5, 2.25]) {
const specs = generateInteractiveProps({
halfExtent,
spawn: [-0.6, 0, 0.12],
random: rng(seed * 7919),
});
counts.add(specs.length);
assert.ok(specs.length >= 5 && specs.length <= 7);
assert.equal(new Set(specs.map((p) => p.type)).size, 4);
assert.equal(new Set(specs.map((p) => p.name)).size, specs.length);
assert.ok(specs[0].pos[0] > -0.3 && specs[0].pos[0] < 0.25);
for (const [i, spec] of specs.entries()) {
assert.ok(spec.mass >= 0.04 && spec.mass <= 0.08);
assert.ok(Math.abs(spec.pos[0]) < halfExtent - 0.1);
assert.ok(Math.abs(spec.pos[1]) < halfExtent - 0.1);
assert.ok(Math.hypot(spec.pos[0] + 0.6, spec.pos[1]) > 0.34);
for (const other of specs.slice(i + 1))
assert.ok(
Math.hypot(spec.pos[0] - other.pos[0], spec.pos[1] - other.pos[1]) >
0.3,
);
}
}
}
assert.deepEqual([...counts].sort(), [5, 6, 7]);
assert.notDeepEqual(
generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(1),
}),
generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(2),
}),
);
});
test("visual geometry matches MJCF dimensions, poses sync through Z-up root, reset clears only prop DOFs", () => {
const specs = generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(1),
});
const view = createInteractivePropVisuals(specs);
const addresses = specs.map((_, i) => ({
qAdr: 7 + i * 7,
dofAdr: 6 + i * 6,
}));
const data = {
qpos: new Float64Array(7 + specs.length * 7).fill(123),
qvel: new Float64Array(6 + specs.length * 6).fill(42),
};
specs.forEach((spec, i) => resetProp(data, addresses[i], spec));
view.sync(data, addresses);
view.root.updateMatrixWorld(true);
view.meshes.forEach((mesh, i) => {
assert.equal(mesh.name, specs[i].name);
assert.equal(mesh.userData.interactiveIndex, i);
assert.deepEqual(
mesh
.getWorldPosition(new THREE.Vector3())
.toArray()
.map((v) => +v.toFixed(6)),
[specs[i].pos[0], specs[i].pos[2], -specs[i].pos[1]].map(
(v) => +v.toFixed(6),
),
);
mesh.geometry.computeBoundingBox();
const size = mesh.geometry.boundingBox.getSize(new THREE.Vector3());
const spec = specs[i];
const height =
2 *
(spec.type === "sphere"
? spec.size[0]
: spec.type === "capsule"
? spec.size[0] + spec.size[1]
: spec.size.at(-1));
assert.ok(Math.abs(size.z - height) < 1e-7);
});
assert.deepEqual(Array.from(data.qpos.slice(0, 7)), Array(7).fill(123));
assert.deepEqual(Array.from(data.qvel.slice(0, 6)), Array(6).fill(42));
assert.ok(Array.from(data.qvel.slice(6)).every((v) => v === 0));
resetProp(data, addresses[0], specs[0], true, 0);
view.sync(data, addresses, false);
assert.equal(view.root.visible, false);
assert.equal(data.qpos[addresses[0].qAdr], 50);
for (const mesh of view.meshes) {
mesh.geometry.dispose();
mesh.material.dispose();
}
});
test("the original four swatches tint floor AND all walls without altering fades or reveal", () => {
const grid = makeInfiniteGrid({ subdivisions: 1 });
const { wallMats, wallMeshes } = makeArenaWalls();
const materials = [grid.material, ...wallMats];
for (const [name, hex] of Object.entries(VARIANT_SWATCH_HEX)) {
assert.equal(setArenaLineColor(materials, name), true);
for (const mat of materials) {
assert.equal(
mat.uniforms.uSectionColor.value.getHexString(),
hex.slice(1),
);
assert.equal(mat.uniforms.uReveal.value, 0);
assert.ok(
mat.uniforms.uCellColor.value.equals(
mat.uniforms.uSectionColor.value.clone().multiplyScalar(0.55),
),
);
}
}
assert.equal(setArenaLineColor(materials, "unknown"), false);
for (const mesh of [grid, ...wallMeshes]) {
mesh.geometry.dispose();
mesh.material.dispose();
}
});
+108
View File
@@ -0,0 +1,108 @@
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import { createHash } from "node:crypto";
import { validateBodyGlb } from "../src/platform/render/BodyVisuals.js";
import { validatePackage } from "../src/platform/loader/SchemaValidator.js";
const root = new URL("../public/robots/rp1/", import.meta.url);
const config = JSON.parse(await fs.readFile(new URL("robot.json", root)));
const manifest = JSON.parse(
await fs.readFile(new URL("visual-source-manifest.json", root)),
);
const load = async (path) => {
const b = await fs.readFile(new URL(path, root));
return validateBodyGlb(
b.buffer.slice(b.byteOffset, b.byteOffset + b.byteLength),
);
};
test("RP1 maps all 63 supplied parts exactly once onto 25 visual-only body attachments", async () => {
validatePackage(config);
assert.equal(config.render.body_visuals.length, 25);
assert.equal(manifest.input_files.length, 63);
const parts = Object.values(manifest.body_mapping).flatMap((m) => m.parts);
assert.equal(parts.length, 63);
assert.equal(new Set(parts).size, 63);
let normals = 0,
colourTextures = 0,
emissive = 0;
for (const spec of config.render.body_visuals) {
assert.ok(!config.assets.some((a) => a.path === spec.path));
assert.deepEqual(spec.quaternion, [Math.SQRT1_2, 0, 0, Math.SQRT1_2]);
const doc = await load(spec.path);
for (const mat of doc.materials) {
if (mat.normalTexture) {
normals++;
assert.ok(doc.textures[mat.normalTexture.index]);
}
if (mat.pbrMetallicRoughness?.baseColorTexture) colourTextures++;
if (mat.extensions?.KHR_materials_emissive_strength?.emissiveStrength > 1)
emissive++;
}
for (const texture of doc.textures) assert.ok(doc.images[texture.source]);
if (spec.body.endsWith("_wrist_link")) {
const wrappers = doc.nodes.filter((n) => n.extras?.sourcePart);
assert.equal(wrappers.length, 20);
assert.equal(new Set(wrappers.map((n) => n.extras.sourcePart)).size, 20);
assert.deepEqual(
wrappers.map((n) => n.extras.sourcePart).sort(),
[...manifest.body_mapping[spec.body].parts].sort(),
);
}
}
assert.ok(normals >= 63);
assert.ok(colourTextures >= 1, "face screen texture retained");
assert.ok(emissive >= 1, "torso logo emissive strength retained");
});
test("23 unchanged body shapes align exactly with original STL bounds after Y-up conversion", async () => {
for (const { body, path } of config.render.body_visuals.filter(
(s) => !s.body.endsWith("_wrist_link"),
)) {
const doc = await load(path),
lo = [Infinity, Infinity, Infinity],
hi = [-Infinity, -Infinity, -Infinity];
for (const mesh of doc.meshes)
for (const p of mesh.primitives) {
const a = doc.accessors[p.attributes.POSITION];
for (let i = 0; i < 3; i++) {
lo[i] = Math.min(lo[i], a.min[i]);
hi[i] = Math.max(hi[i], a.max[i]);
}
}
const transformed = [lo[0], -hi[2], lo[1], hi[0], -lo[2], hi[1]];
const stl = await fs.readFile(new URL(`meshes/${body}.STL`, root));
const expected = [
Infinity,
Infinity,
Infinity,
-Infinity,
-Infinity,
-Infinity,
];
for (let t = 0; t < stl.readUInt32LE(80); t++)
for (let v = 0; v < 3; v++)
for (let axis = 0; axis < 3; axis++) {
const x = stl.readFloatLE(84 + t * 50 + 12 + v * 12 + axis * 4);
expected[axis] = Math.min(expected[axis], x);
expected[axis + 3] = Math.max(expected[axis + 3], x);
}
assert.deepEqual(transformed, expected, body);
}
});
test("original model, physics meshes, policies/motions and imported visual provenance retain reviewed hashes", async () => {
const physical = JSON.parse(
await fs.readFile(new URL("source-manifest.json", root)),
);
for (const file of [...physical.files, ...manifest.files]) {
assert.equal(
createHash("sha256")
.update(await fs.readFile(new URL(file.file, root)))
.digest("hex"),
file.sha256,
file.file,
);
}
});
+36
View File
@@ -0,0 +1,36 @@
import test from "node:test";
import assert from "node:assert/strict";
import { createSessionEnvironment } from "../src/game/worlds/arena/session-environment.js";
const world = (scale) => ({
presentation: "arena-grid",
boundary: { half_extent: 2.25 * scale },
});
test("all robot sizes share one random layout, with scaled geometry and spawn clearances", () => {
let seed = 42;
const scene = createSessionEnvironment(
() => (seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296,
);
const duck = scene.propsFor(world(1));
for (const scale of [1, 0.33 / 0.27, 1.32 / 0.27]) {
const props = scene.propsFor(world(scale));
assert.equal(props.length, duck.length);
props.forEach((p, i) => {
assert.equal(p.type, duck[i].type);
assert.equal(p.color, duck[i].color);
p.pos.forEach((v, j) =>
assert.ok(Math.abs(v / scale - duck[i].pos[j]) < 1e-12),
);
p.size.forEach((v, j) =>
assert.ok(Math.abs(v / scale - duck[i].size[j]) < 1e-12),
);
assert.ok(Math.hypot(p.pos[0] / scale, p.pos[1] / scale) > 0.34);
assert.ok(Math.hypot(p.pos[0] / scale + 0.6, p.pos[1] / scale) > 0.34);
});
}
const copy = scene.propsFor(world(1));
copy[0].pos[0] = 100;
copy[0].quat[0] = 0;
assert.deepEqual(scene.propsFor(world(1)), duck);
assert.deepEqual(scene.propsFor({ type: "flat" }), []);
});
+16
View File
@@ -16,6 +16,8 @@
- `model.xml` 与它实际声明的 25 个 STL;原模型不修改,运行时组合空地与 timestep。
- `policies/walk.onnx`。
- `visuals/*.glb`:用户提供的新版 `microban2.zip` 中 20 个原始 GLB,保留 PBR 材质和 46 张内嵌纹理(45 张法线贴图及 1 张屏幕颜色贴图);替代白模显示,不替换物理网格。
- `visual-source-manifest.json`:视觉压缩包和逐文件 SHA-256,与原物理 / 策略来源记录分开。压缩包没有许可说明,公开分发前需向资产所有者确认。
- `source-manifest.json`:源 remote、HEAD、dirty 标志、逐文件 SHA-256、行为参考代码哈希。
- `SOURCE.md` 和 `LICENSES/`:原项目混合许可文本、XML 的 Apache-2.0 许可文本和归属说明。
@@ -28,6 +30,20 @@ npm --prefix app run validate:packages
导入是显式离线维护操作,不是运行或构建的依赖;日常运行不读取源仓库、不要求 Python / BAM / Placo。导入后需检查是否有实质变化,不自动接受策略 / 行为更新。
## 带纹理外观
刷新页面后选择 Microban 即使用新外观:浅紫外壳、蓝色金属支架、黑色电机及带颜色贴图的屏幕。新版 20 个 GLB 的顶点位置和节点变换与上一版一致,保留现有部件绑定。`render.body_visuals` 将 BASE、LINK1–19 映射到已有的 20 个 MuJoCo body,显式记录米制平移和 xyzw 单位四元数。坐标变换经原几何对齐检查;部件包围盒端点差异最大约 3 mm,包含新外观的螺丝等细节,不是运行时自动猜测比例或改变关节姿态。
所有部件每帧读取原 `xpos/xquat`,包括独立转头;原 visual geom 隐藏,碰撞、质量、惯量、关节、策略和 PD 均保留。GLB 加载失败会放弃候选机器人并恢复旧机器人,不会以静默白模掩盖缺失资源。切换时释放几何、材质、纹理和 ImageBitmap。
```bash
python3 app/scripts/import-microban-visuals.py /path/to/microban2.zip
npm --prefix app run validate:packages
npm --prefix app run test:e2e -- microban-visuals.spec.js microban-walking.spec.js runtime-leaks.spec.js
```
`microban-visuals.spec.js` 验证 20 个 body 的材质/纹理、原 visual 隐藏、非零根 yaw 和关节角下的对齐、失败回退,以及开关新视觉后 80 个控制周期的 qpos / qvel / ONNX action 逐项完全一致。截图保存于该测试的 `test-results/` 输出目录。行走原有三轮有界 70 s、三轮无界诊断 84 s 和 H / reset / 近 50 Hz 验证通过;20 次跨 Microduck / Microban / RP1 切换后,Microban 的 GPU 纹理数稳定为 48(46 张模型纹理加场景资源),无增长。
## 策略接口
| 项目 | 数值 |
+21
View File
@@ -41,6 +41,27 @@ app/public/robots/example/
- `app/src/platform/loader/world.schema.json`
- `SchemaValidator.js` 的跨字段检查(所有权、输入绑定、命令尺寸等)。
## 可选分部件 GLB 外观
通用 runtime 保留 `render.source: "mjcf_visual"`,可增加 `render.body_visuals`,仅替换指定 body 的可见几何:
```json
"body_visuals": [
{
"body": "trunk",
"path": "visuals/trunk.glb",
"position": [0, 0, 0],
"quaternion": [0, 0, 0, 1]
}
]
```
`body` 必须是 MJCF 中存在的非 world body,且不能重复。`position` 为米制 body-local 平移;`quaternion` 为 xyzw 单位四元数;省略时分别使用零平移 / 单位旋转。不按导出文件名猜 body,不使用 MuJoCo 编译器重心校正后的 geom 坐标绑定 GLB。模型单位须预先为米,不支持隐式自动缩放。
GLB 必须自包含(纹理/二进制内嵌),不能引用外部图片或 buffer;`path` 是包内相对 `.glb` 路径。资源经 PackageLoader 加载,保留部署前缀与同源签名。GLB 不加入 `assets` / VFS,因为它们不参与物理编译。隐藏原 visual 不修改原碰撞、惯量、关节、策略或动作。缺失 body 或资源导致候选加载失败,并通过现有切换流程回退。
新增视觉资源应单独保存 `visual-source-manifest.json`,包含 `files: [{"file":"visuals/trunk.glb","sha256":"..."}]`;`validate:packages` 同时验证原来源记录、视觉记录和 GLB 自包含约束。示例与实测记录见 [Microban](microban.md)。
## 场地与机器人尺寸
在 Robot Package 顶层声明参考站姿的全身高度(米),例如:
+17
View File
@@ -19,6 +19,23 @@ npm --prefix app run dev -- --host 127.0.0.1
**限制:完整舞蹈参考长 131.46 秒;旧版 model_7999 的长程测试在参考帧 994(约 19.9 秒)失稳。新版 model_23998 目前只验证并开放前 10 秒,完整时长尚未重新验收,不自动延长 duration。** 训练完成不等于任意长度播放已收敛;不能仅延长 duration 就宣称稳定。
## GLB 材质与外观
已接入用户提供的 `RP1机器人glb.zip`:白/灰/蓝 PBR 材质、法线贴图、面部屏幕和发光标识均保留。63 个部件映射到现有 25 个物理 body,物理模型仍为 24 自由度;23 个 body 直接使用原 GLB,双手各 20 个部件按原始固定手 URDF 组合到手腕 body。手指随手腕整体运动,不新增独立手指控制,也不改变行走、舞蹈或挥手策略。
模型 XML、原始碰撞体、惯性、关节及全部 ONNX / motion 文件不变;仅新增 `render.body_visuals`。原 GLB 为 Y-up,绑定时统一绕 X 轴 +90°,不能当作 Z-up 直接贴到 body。原始 STL 可视 mesh 会隐藏,而不是与新模型重叠绘制。
复现导入:
```bash
python3 app/scripts/import-rp1-visuals.py "/home/cen/Downloads/RP1机器人glb.zip" \
app/public/robots/rp1/visuals/rp1_hand_fixed.urdf
npm --prefix app run validate:packages
npm --prefix app run test:e2e -- rp1-visuals
```
来源和 SHA-256 见 `app/public/robots/rp1/visual-source-manifest.json`;包内未附再分发许可。首次 RP1 加载会额外读取约 93 MB 的自包含 GLB,具体帧率取决于 GPU;本次没有简化或改写用户的网格、贴图和材质。验证记录见 [RP1 GLB 材质替换](../superpowers/verification/rp1-glb-materials.md)。
## 已完成权重
均来自 `/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl`,显式选择最终、完整且稳定的文件,不动态追踪目录中的最新文件:
@@ -0,0 +1,38 @@
# Microduck 场景提亮、线条调色和交互几何体
> 本文记录最初的 Microduck 单机器人改动。后续根据“每个机器人的环境都保持一致”的要求,已推广到所有机器人,并改为会话共享布局和全局线条色;当前规则及验证见 [共享环境验证](shared-robot-environment.md)。
## 实现范围
- `app/src/scene/GameCanvas.jsx`:新增半球补光,提高环境贴图、环境光、主光和侧光强度;漫画场景背景从近黑调整为深蓝灰。没有添加外部光照资源或网络依赖。
- `app/src/ui/Hud.jsx`、`app/src/game/worlds/arena/line-color.js`:原左下角四个色块改为 **Line color**,仍使用原 Cream / Graphite / Lavender / Sky 色值、尺寸、边框、选中动画和漫画面板。同步改变地板与四面围墙的粗细网格线,不修改机器人或几何体材质。颜色在 reset、feet / rollers 切换和机器人往返切换时保留。
- `app/src/game/interactive-props.js`、`app/src/game/worlds/arena/compose.js`、`app/src/robots/microduck/runtime.js`:生产场景删除足球刚体、足球纹理渲染、生成逻辑和足球专属音频/拖拽分支,替换为每次新建 Microduck runtime 随机生成的 **5~7 个**轻量几何体,包括方块、球体、圆柱和胶囊体。
- 几何体具有独立 MuJoCo freejoint、真实质量、摩擦和碰撞;支持机器人推动/踢动、物体间碰撞、地面/围墙碰撞、鼠标弹簧力拖拽与抬起。生成位置互不重叠并避开机器人出生点,至少一个位于机器人附近。reset / locomotion 切换恢复当前布局;重新进入 Microduck 随机生成新布局。越界物体单独复位。
- 原有足球素材/历史库文件未做无关删除,但生产 runtime 不再加载或创建足球。
## 物理与回归说明
新增自由刚体会改变 MuJoCo 的 DOF 数。轮滑模型可能跨越自动 sparse Jacobian 阈值,因此显式保留原来的 `jacobian="dense"`,避免几何体数量影响求解路径。
九份历史动作轨迹全部保留,未更新基线或放宽浮点比较。`?test=1&oracle=legacy` 专门复现历史回归使用的 3 m 实验场及一个远处停放的无足球纹理球形物理探针,保留原始 DOF 数和接触参数,从而继续精确比对全部状态。正常运行和 `?test=1` 单独使用时都构建真实的 5~7 个几何体场景。新场景的碰撞、鼠标拖拽、物体数量、材质、光照和运行稳定性由独立浏览器测试覆盖,不以历史实验场测试替代。
## 验证覆盖
- `app/test/interactive-props.test.js`:2,000 个布局(1,000 个随机种子 × 3 m / 4.5 m 两种边界),验证 5/6/7 数量、四种形状、出生点净空、物体间距、边界和质量;验证可视几何体尺寸、Z-up 转换、刚体状态同步与复位;验证四种颜色同步更新地面与四面墙。
- `app/e2e/scene-interaction.spec.js`:真实 Chromium + MuJoCo WASM,验证新光源与场景初始内容、四个色块外观与功能、机器人材质不变、鼠标物理拖拽、暂停释放抓取力、重置布局、物体间动量传递、机器人/地面碰撞、腿/轮滑切换后的物体数量及状态。
- `app/e2e/scene-stability.spec.js`:真实新场景中腿/轮滑各 60 秒模拟时间的连续运动,检查状态有限、物体边界、确实产生移动,以及越界物体单独恢复。
- `app/e2e/deployment.spec.js`:正常生产入口自动显示 5~7 个物体且不存在足球;继续覆盖三种机器人与静态资源路径。
- `app/e2e/runtime-leaks.spec.js`:20 次机器人切换后检查 WASM 对象释放、GPU 几何体、纹理和事件监听器无增长;对当前 5~7 个物体带来的预期几何体数量差异做精确扣除,而非放宽泄漏上限。
## 执行结果
- `cd app && npm test`:**62/62 通过**。
- `npm run validate:packages`:四个 bundled Package 的 schema、World、资源路径与既有来源校验全部通过。
- `npm run format:check`、`npm run build`、`git diff --check`:通过。
- `npm run test:e2e`:**48/48 通过**(13.8 分钟;包含九项精确历史动作回归、三机器人场景/动作、20 次资源切换及新场景交互)。该批启动后补充的连续运动用例另外独立执行并通过。
- 对最终 `dist/` 启动本地 Vite preview,以 `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- scene-interaction scene-stability deployment robot-switching microduck-regression --output=/tmp/microduck-scene-production-final-artifacts` 执行生产包复验:**16/16 通过**(2.7 分钟)。覆盖全部新交互、围墙阻挡、正常入口无足球、两分钟新场景模拟运动和九项历史动作精确回归。
- 人工查看新场景截图:`app/test-results/scene-interaction-overview.png`,确认补光后的机器人及彩色几何体、浅蓝网格线和保留原样的四格色板显示正常。截图属于忽略的本地测试产物,不提交大尺寸图片。
日志:`/tmp/microduck-scene-unit-tests.log`、`/tmp/microduck-scene-full-e2e.log`、`/tmp/microduck-scene-production-final-tests.log`。全量浏览器 JSON 结果另存于 `/tmp/microduck-scene-full-results.json`。
边界:没有声称执行连续 15 小时实时时长的压力测试;新增长序列检查为真实 WASM 中累计两分钟的模拟运动。
@@ -0,0 +1,64 @@
# RP1 用户 GLB 材质替换
## 输入与范围
- 输入:`/home/cen/Downloads/RP1机器人glb.zip`,63 个自包含 GLB,未包含许可文件。
- ZIP SHA-256:`8610aac8432203bf68d0efe234062c551ca805ebbe72a92d5fc7350730ebdea2`。
- 仅新增 RP1 `render.body_visuals` 和可视资源;**原 `model.xml`、STL、惯性、碰撞体、24 个驱动关节、ONNX、motion 文件均未修改**。它们仍通过原 `source-manifest.json` 哈希验证。
- 复用已有 `BodyVisuals` 渲染替换层,没有为本次替换修改共享渲染器、物理引擎或控制器。其他机器人材质、共享环境及色板不受影响。
## 坐标与部件适配
GLB 来自 Blender,顶点为 Y-up;现有 body 帧为 MuJoCo Z-up。所有挂载使用绕 X 轴 +90° 四元数。对全部 63 个输入 GLB,按此变换得到的包围盒与原始描述仓库对应 STL 包围盒逐项比较,最大差为 **0**。
当前训练模型是 25 个 body / 24 个受控关节,而该可视包包括完整的手腕和灵巧手:
- 23 个普通 body 使用输入 GLB 原始字节,不改颜色或贴图。
- 每侧手腕合并 20 个输入部件:wrist roll / yaw / pitch 三段、手掌及 16 个手指部件。
- 按原始 `rp1_hand_fixed.urdf` 的相对原点和固定指节姿态组合;原 24-DOF 手腕对应 wrist-roll,额外 yaw/pitch 轴固定在中立位。
- 合成 GLB 仅追加可视变换节点并重映射 GLTF 索引;顶点、索引、图像二进制数据以及 PBR 材质内容保留。
- 最终为 25 个挂载 GLB,共覆盖 63 个输入部件,且每个部件只使用一次。**没有新增手指/手腕物理自由度,也不声称支持独立手指控制**。
- 新手部的可视细节不会自动改变原训练模型的碰撞包络。
`app/scripts/import-rp1-visuals.py` 是可复现导入器;不会解压任意 ZIP 路径,在完成输入和合成输出验证后才写入包目录。`visual-source-manifest.json` 保存所有输入/输出 SHA-256、部件映射和复制的固定手 URDF 哈希。
## 已测定量结果
真实 Chromium / MuJoCo WASM / Three.js 中:
| 指标 | 结果 |
|---|---:|
| 可视 body 挂载 | 25 |
| 手腕/手部源变换节点 | 40(每侧 20) |
| 新可视 mesh | 79 |
| 加载的法线贴图材质 | 77 |
| 面部屏幕颜色贴图 | 1 |
| 保留发光强度的标识材质 | 1 |
| 隐藏的原 STL 可视 mesh | 25 |
| body 姿态跟随误差 | 0 |
| 23 个原形状的三维包围盒最大误差 | `1.3124658082563911e-8 m` |
| 双手变换相对独立 URDF 计算最大误差 | `5.551115123125783e-16` |
| 行走观测 / 动作维度 | 243 / 24(不变) |
| MuJoCo actuator 数 | 24(不变) |
根朝向及所有 24 个关节改变后再次验证上述坐标关系,不仅验证绑定初始姿态。
`rp1-visuals.spec.js` 还在同一会话、同一几何体布局和相同输入下,逐项**精确比较**替换前/后 walk / wave / dance 的 qpos、qvel、ctrl、观测、lastAction、状态机及参考帧;无任何容差放宽。部分 GLB 加载失败时旧机器人继续保留,释放候选资源后可直接 retry 成功。
## 测试与限制
- 新增三个纯逻辑/资源测试与三个真实浏览器测试;首轮全部通过。
- 所有 PBR 参数及贴图遵循用户原包,不另行重着色或简化。约 93 MB GLB、约 113 万三角形,首次加载量和 GPU 负担增加;不声称在所有设备具有相同帧率。
- 资源未附许可,继续仅供本地评估;没有推断公开再分发授权,也没有公开发布。
## 最终执行结果
- `npm test`:**66/66 通过**。
- `npm run validate:packages`:四个 Package 均通过;RP1 为 58 条资源路径,原始物理/策略及新增 GLB 的来源哈希全部通过。
- `npm run format:check`、`npm run build`、Python 语法检查、`git diff --check`:通过。
- `npm run test:e2e`:**55/55 通过**(20.3 分钟)。含原有所有动作、共享环境、实时控制频率、新材质、加载失败恢复、20 次资源释放检查。
- 20 次机器人切换时,RP1 GPU 资源保持 **125 geometries / 80 textures / 21 listeners**,各次完全相同;旧 model/data 均删除,未增加资源泄漏容差。该用例冷加载约三分钟,为避免新 93 MB 材质包在慢设备上贴近旧超时上限,测试时限单独由 180 秒调为 240 秒,资源断言原样保留。
- 最终 `dist/` 生产包复验:`TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- rp1-visuals shared-environment deployment --output=/tmp/rp1-material-production-artifacts`,**7/7 通过**(3.9 分钟)。再次验证坐标/材质、替换前后逐项数值一致性、错误恢复、三机器人共享环境以及静态资源路径。
- 人工查看前侧材质截图:蓝白灰配色、脸部屏幕、发光标识和双手外观正常;截图为 `/tmp/rp1-material-production-artifacts/rp1-visuals-all-63-RP1-mat-8a18f-ith-original-shapes-aligned/rp1-material-front.png`。对应开发测试截图亦在 `app/test-results/rp1-visuals-all-63-RP1-mat-8a18f-ith-original-shapes-aligned/rp1-material-front.png`。
日志:`/tmp/rp1-visuals-unit.log`、`/tmp/rp1-material-full-e2e.log`、`/tmp/rp1-material-production-e2e.log`;全量 JSON 结果另存 `/tmp/rp1-material-full-results.json`。

Some files were not shown because too many files have changed in this diff Show More