1 Commits

Author SHA1 Message Date
chenlin a8be5e0134 feat: release v0.5 with shared interactive arenas and textured RP1
Unify lighting and line-colour controls across robots, replace the football with a shared layout of draggable physical props, and import the supplied RP1 GLBs without changing trained physics or policies.

Include reproducible visual import/provenance, regression coverage and verification records. Verified 66 unit tests, 55 browser tests and production-build checks.
2026-10-30 14:14:03 +08:00
69 changed files with 5125 additions and 317 deletions
+18 -5
View File
@@ -15,18 +15,31 @@ MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), *
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
## v0.5
- Brighter shared lighting and the original four-swatch **Line color** control for every robot.
- One session-wide layout of 5–7 physical, draggable geometries replaces the football and scales consistently across robots.
- RP1 uses the supplied textured GLBs, including fixed-hand detail, without changing its 24-DOF physics or trained skills.
- Verified with 66 unit tests, 55 browser tests and production-build checks. See [shared environment](docs/superpowers/verification/shared-robot-environment.md) and [RP1 materials](docs/superpowers/verification/rp1-glb-materials.md) verification records.
## Robots
| Robot | Default scene / presentation | Controls |
|---|---|---|
| Microduck | Shared 4.5 × 4.5 m bounded arena, ball, original GLB/palette/comic effects | Existing walking, skating, skills and recovery |
| Microban | Robot-scaled 5.5 × 5.5 m arena, dark/orange grid and comic UI, textured per-body GLB visuals | Walking policy + independent head rotation |
| RP1 | Robot-scaled 22 × 22 m arena/comic UI, original 24-DOF MJCF visuals | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
| Microduck | Shared 4.5 × 4.5 m bounded arena, 5–7 random interactive geometries, original GLB/comic effects | Existing walking, skating, skills and recovery |
| Microban | Shared robot-scaled 5.5 × 5.5 m arena, interactive geometries, textured per-body GLB visuals | Walking policy + independent head rotation |
| RP1 | Shared robot-scaled 22 × 22 m arena, interactive geometries, user-supplied textured GLB visuals on original 24-DOF physics | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's scene-line colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban physics / policy assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); its textured visual replacement comes from the user-supplied `microban2.zip`. Exact hashes, dirty-worktree provenance and license caveats are included in its [Package source notes](app/public/robots/microban/SOURCE.md).
### Shared scene controls (all robots)
The scene has brighter ambient / environment illumination plus a hemisphere fill light. The lower-left **Line color** palette retains the original Cream / Graphite / Lavender / Sky swatches and square comic buttons, but now changes **floor and wall grid lines**, not robot materials. The same palette is available for every robot, and its selection is global across robot switches, reset and feet / rollers changes.
The football is removed. One browser session randomly generates **5–7 lightweight boxes, spheres, cylinders and capsules** shared by **all robots**, clear of both bundled spawn anchors and each other. Their types, colours and relative layout stay identical when switching robots; geometry and positions scale with the existing arena-to-robot ratio. They have real MuJoCo free joints: every robot can push them, they collide with one another, the floor and walls, and the mouse can grab / lift them using spring forces. Reset, locomotion changes and robot switches restore the same session layout; refreshing generates a new layout. Escaped toys are returned individually. This shares environment configuration/layout, not live object momentum between different-scale robot simulations.
## Keyboard and mouse
| Action | Binding |
@@ -47,7 +60,7 @@ Microduck policies / assets originate from [pollen-robotics/microduck](https://g
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Microduck mouse dragging still applies physical forces to the robot / ball. Microban / RP1 mouse dragging orbits the camera. Gamepad / touch are not part of this release's acceptance scope.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Dragging any robot or interactive geometry applies physical forces; dragging empty space orbits the camera, consistently for all three robots. Gamepad / touch are not part of this release's acceptance scope.
## Microban fidelity and limits
+1 -1
View File
@@ -59,7 +59,7 @@ test("arena appearance matches Microduck while Microban physics stays flat and W
expect(state).toEqual({
cell: 0.1 * (0.33 / 0.27),
section: 0.6 * (0.33 / 0.27),
color: 0xffb366,
color: 0xf7e6cb,
reveal: 1,
topo: 0,
hiddenFloor: true,
+9 -5
View File
@@ -60,13 +60,15 @@ test("both robots default to the same rear camera and comic UI; Microban keeps f
await expect(
page.getByRole("navigation", { name: "Simulation controls" }),
).toHaveCSS("border-top-width", "2px");
await expect(page.getByRole("group", { name: "Duck colours" })).toHaveCount(
0,
);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
await expect(
page.getByRole("button", { name: "Rollers", exact: true }),
).toHaveCount(0);
await expect(page.getByText("Drag robot / ball: apply force")).toHaveCount(0);
await expect(
page.getByText("Drag robot / geometry: apply force"),
).toBeVisible();
await expect(page.getByRole("status")).toContainText("Simplified actuation");
expect(
await page.evaluate(() => window.rl.model.geom("platform_floor").id),
@@ -74,7 +76,9 @@ test("both robots default to the same rear camera and comic UI; Microban keeps f
await page.screenshot({ path: info.outputPath("microban-chase.png") });
await page.evaluate(() => window.__gameApi.switchRobot("microduck-legs"));
await rear(page);
await expect(page.getByRole("group", { name: "Duck colours" })).toBeVisible();
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
expect(errors).toEqual([]);
});
for (const id of ["microban", "microduck-legs"])
+18 -1
View File
@@ -20,7 +20,24 @@ test("deployed assets respect the base path and all three Robot Packages boot",
await expect(
page.getByRole("button", { name: "Pause", exact: true }),
).toBeEnabled();
await page.waitForTimeout(500); // capture after the ball's existing appear FX
await page.waitForTimeout(500); // let the physical toys settle
const toys = await page.evaluate(() => {
const r = window.rl;
return {
count: r.interactiveProps.length,
visible: r.scene.getObjectByName("interactive-props").visible,
hasBall:
r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
) >= 0,
};
});
expect(toys.count).toBeGreaterThanOrEqual(5);
expect(toys.count).toBeLessThanOrEqual(7);
expect(toys.visible).toBe(true);
expect(toys.hasBall).toBe(false);
await page.screenshot({ path: info.outputPath("microduck.png") });
await page.getByLabel("Robot", { exact: true }).selectOption("microban");
await expect
+1 -1
View File
@@ -26,7 +26,7 @@ for (const scenario of [
world.boundary.half_extent = 1.5;
await route.fulfill({ response, json: world });
});
const errors = await bootDuck(page);
const errors = await bootDuck(page, { legacyPhysics: true });
await page.evaluate(async (scenario) => {
const r = window.rl;
if (scenario === "rollers" || scenario === "crouch") {
+5 -5
View File
@@ -11,7 +11,7 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await ready(page);
await page.getByRole("button", { name: "Lavender colours" }).click();
await page.getByRole("button", { name: "Lavender scene lines" }).click();
await page.getByRole("button", { name: "Rollers", exact: true }).click();
await page.waitForFunction(() => window.rl?.loco === "rollers");
await page.getByLabel("Robot", { exact: true }).selectOption("microban");
@@ -26,9 +26,9 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
await expect(
page.getByText("Toggle head rotation", { exact: false }),
).toBeVisible();
await expect(page.getByRole("group", { name: "Duck colours" })).toHaveCount(
0,
);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
expect(await page.evaluate(() => window.rl.buildObs().length)).toBe(63);
await page.getByLabel("Robot", { exact: true }).selectOption("microduck");
await page.waitForFunction(
@@ -37,7 +37,7 @@ test("Microduck → Microban → Microduck restores theme, colours and locomotio
window.rl?.loco === "rollers",
);
await expect(
page.getByRole("button", { name: "Lavender colours" }),
page.getByRole("button", { name: "Lavender scene lines" }),
).toHaveAttribute("aria-pressed", "true");
expect(await page.evaluate(() => window.rl.data.time)).toBeLessThan(3);
expect(errors).toEqual([]);
+274
View File
@@ -0,0 +1,274 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
async function boot(page) {
await bootDuck(page);
await page.evaluate(() => window.__gameApi.pause());
}
test("all 63 RP1 material parts follow 25 physical bodies, including fixed fingers, with original shapes aligned", async ({
page,
}, info) => {
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await boot(page);
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
true,
);
const inspect = () =>
page.evaluate(async () => {
const r = window.rl,
layer = r.scene.getObjectByName("body-visuals"),
rig = layer.parent;
rig.updateMatrixWorld(true);
const xml = new DOMParser().parseFromString(
await (await fetch("robots/rp1/visuals/rp1_hand_fixed.urdf")).text(),
"text/xml",
);
const joints = new Map(
[...xml.querySelectorAll("joint")].map((j) => [
j.querySelector("child").getAttribute("link"),
j,
]),
);
const Matrix = rig.matrix.constructor,
Euler = r.controls.object.rotation.constructor;
const correction = new Matrix().makeRotationX(Math.PI / 2);
const urdfRelative = (name, anchor) => {
const result = new Matrix();
while (name !== anchor) {
const joint = joints.get(name),
origin = joint.querySelector("origin");
const xyz = (origin?.getAttribute("xyz") ?? "0 0 0")
.split(/\s+/)
.map(Number);
const angles = (origin?.getAttribute("rpy") ?? "0 0 0")
.split(/\s+/)
.map(Number);
const matrix = new Matrix().makeRotationFromEuler(
new Euler(...angles, "ZYX"),
);
matrix.setPosition(...xyz);
result.premultiply(matrix);
name = joint.querySelector("parent").getAttribute("link");
}
return result.multiply(correction);
};
const bounds = (objects, frame) => {
const lo = [Infinity, Infinity, Infinity],
hi = [-Infinity, -Infinity, -Infinity];
const inverse = frame.matrixWorld.clone().invert(),
vector = frame.position.clone();
for (const mesh of objects) {
const transform = inverse.clone().multiply(mesh.matrixWorld),
attr = mesh.geometry.attributes.position;
for (let i = 0; i < attr.count; i++) {
vector.fromBufferAttribute(attr, i).applyMatrix4(transform);
vector.toArray().forEach((v, a) => {
lo[a] = Math.min(lo[a], v);
hi[a] = Math.max(hi[a], v);
});
}
}
return [...lo, ...hi];
};
let frameError = 0,
boundsError = 0,
handError = 0,
handParts = 0,
normals = 0,
screens = 0,
emissive = 0,
hidden = 0,
meshes = 0;
for (const frame of layer.children) {
const body = frame.name.slice("body-visual:".length),
id = r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
body,
);
const p = Array.from(r.data.xpos.slice(id * 3, id * 3 + 3)),
q = Array.from(r.data.xquat.slice(id * 4, id * 4 + 4));
frameError = Math.max(
frameError,
...p.map((v, i) => Math.abs(v - frame.position.toArray()[i])),
...[q[1], q[2], q[3], q[0]].map((v, i) =>
Math.abs(v - frame.quaternion.toArray()[i]),
),
);
const originals = rig.children.filter(
(m) =>
m.userData.geomId !== undefined &&
r.model.geom_bodyid[m.userData.geomId] === id,
);
hidden += originals.filter((m) => !m.visible).length;
const replacements = [];
frame.traverse((node) => {
if (node.userData.sourcePart) {
handParts++;
const actual = frame.matrixWorld
.clone()
.invert()
.multiply(node.matrixWorld);
const expected = urdfRelative(
node.userData.sourcePart,
`${body.split("_")[0]}_wrist_roll_link`,
);
handError = Math.max(
handError,
...actual.elements.map((v, i) =>
Math.abs(v - expected.elements[i]),
),
);
}
if (!node.isMesh) return;
meshes++;
replacements.push(node);
for (const m of [node.material].flat()) {
if (m.normalMap?.image?.width > 0) normals++;
if (m.map?.image?.width > 0) screens++;
if (m.emissiveIntensity > 1) emissive++;
}
});
if (!body.endsWith("_wrist_link")) {
const before = bounds(originals, frame),
after = bounds(replacements, frame);
boundsError = Math.max(
boundsError,
...before.map((v, i) => Math.abs(v - after[i])),
);
}
}
return {
bodies: layer.children.length,
frameError,
boundsError,
handError,
handParts,
normals,
screens,
emissive,
hidden,
meshes,
obs: r.buildObs().length,
actions: r.lastAction.length,
actuators: r.model.nu,
};
});
const initial = await inspect();
console.log("RP1 GLB alignment", initial);
expect(initial.bodies).toBe(25);
expect(initial.handParts).toBe(40);
expect(initial.frameError).toBeLessThan(1e-12);
expect(initial.boundsError).toBeLessThan(1e-7);
expect(initial.handError).toBeLessThan(1e-10);
expect(initial.normals).toBeGreaterThanOrEqual(63);
expect(initial.screens).toBeGreaterThanOrEqual(1);
expect(initial.emissive).toBeGreaterThanOrEqual(1);
expect(initial.hidden).toBe(25);
expect(initial.obs).toBe(243);
expect(initial.actions).toBe(24);
expect(initial.actuators).toBe(24);
await page.evaluate(() => {
const r = window.rl;
r.data.qpos[3] = Math.cos(0.4);
r.data.qpos[6] = Math.sin(0.4);
for (let j = 0; j < r.model.njnt; j++)
if (r.model.jnt_type[j] === 3)
r.data.qpos[r.model.jnt_qposadr[j]] += 0.12;
r.mujoco.mj_forward(r.model, r.data);
});
await page.waitForTimeout(200);
const posed = await inspect();
expect(posed.frameError).toBeLessThan(1e-12);
expect(posed.boundsError).toBeLessThan(1e-7);
expect(posed.handError).toBeLessThan(1e-10);
await page.evaluate(() => window.rl.resetSim());
await page.getByRole("button", { name: /Keyboard controls/ }).click();
await page.mouse.click(950, 750, { button: "right" });
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(true);
r.camera.position.set(2.4, 1.35, 1.3);
r.controls.target.set(0, 0.7, 0);
r.controls.update();
});
await page.waitForTimeout(300);
await page.screenshot({ path: info.outputPath("rp1-material-front.png") });
expect(errors).toEqual([]);
});
test("RP1 GLBs change no physics, observations or ONNX actions for walking, wave and dance", async ({
page,
}) => {
await boot(page);
let textured = false;
await page.route("**/robots/rp1/robot.json*", async (route) => {
const response = await route.fetch(),
config = await response.json();
if (!textured) delete config.render.body_visuals;
await route.fulfill({ response, json: config });
});
async function run() {
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
true,
);
return page.evaluate(async () => {
const r = window.rl,
result = [];
for (const scenario of ["walk", "wave", "dance"]) {
await r.resetSim();
await r.step(100);
if (scenario === "walk") r.command.set([0.4, 0, 0.15]);
else r.policyState.request(scenario, r.data.time);
for (let i = 0; i < 5; i++) {
await r.step(50);
result.push({
scenario,
time: r.data.time,
qpos: Array.from(r.data.qpos),
qvel: Array.from(r.data.qvel),
ctrl: Array.from(r.data.ctrl),
action: Array.from(r.lastAction),
obs: Array.from(r.buildObs()),
state: r.policyState.state,
reference: r.referenceFrame,
});
}
}
return result;
});
}
const old = await run();
await page.evaluate(() => window.__gameApi.switchRobot("microduck-legs"));
textured = true;
expect(await run()).toEqual(old);
});
test("a partial RP1 material load failure releases the candidate and retry succeeds", async ({
page,
}) => {
await boot(page);
let fail = true;
await page.route(
"**/robots/rp1/visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
(route) =>
fail ? route.fulfill({ status: 404, body: "missing" }) : route.continue(),
);
expect(await page.evaluate(() => window.__gameApi.switchRobot("rp1"))).toBe(
false,
);
await expect(page.getByRole("alert")).toContainText("404");
expect(await page.evaluate(() => window.__store.getState().robotId)).toBe(
"microduck-legs",
);
expect(await page.evaluate(() => window.rl.model.isDeleted())).toBe(false);
fail = false;
expect(await page.evaluate(() => window.__gameApi.retry())).toBe(true);
expect(
await page.evaluate(
() => window.rl.scene.getObjectByName("body-visuals").children.length,
),
).toBe(25);
});
+12 -3
View File
@@ -3,7 +3,9 @@ import { test, expect } from "@playwright/test";
test("20 robot transitions release old WASM objects and plateau scene/GPU/listeners", async ({
page,
}, info) => {
test.setTimeout(180_000);
// Twenty cold transitions now also decode RP1's ~93 MB textured GLBs.
// Keep allocation/listener assertions unchanged; allow load-time headroom.
test.setTimeout(240_000);
await page.addInitScript(() => {
const add = EventTarget.prototype.addEventListener,
remove = EventTarget.prototype.removeEventListener,
@@ -57,6 +59,9 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
dataDeleted: data.isDeleted(),
...window.__gameApi.diagnostics(),
listeners: window.listenerCount(),
toyGeometries:
window.rl.scene.getObjectByName("interactive-props")?.children
.length ?? 0,
};
}, id);
expect(sample.success).toBe(true);
@@ -70,8 +75,12 @@ test("20 robot transitions release old WASM objects and plateau scene/GPU/listen
for (const samples of [duck, ban, rp1]) {
expect(samples.at(-1).listeners).toBe(samples[0].listeners);
expect(samples.at(-1).sceneChildren).toBe(samples[0].sceneChildren);
expect(samples.at(-1).memory.geometries).toBeLessThanOrEqual(
samples[1].memory.geometries + 1,
// Each runtime owns 5–7 independent toy geometries. Compare the fixed
// scene allocation after removing this intentionally variable count.
expect(
samples.at(-1).memory.geometries - samples.at(-1).toyGeometries,
).toBeLessThanOrEqual(
samples[1].memory.geometries - samples[1].toyGeometries + 1,
);
expect(samples.at(-1).memory.textures).toBeLessThanOrEqual(
samples[1].memory.textures + 1,
+255
View File
@@ -0,0 +1,255 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("bright scene has 5–7 free-body toys, no football, and four original-style line colour buttons", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(async () => {
const r = window.rl;
r.setPropsParked(false);
await r.step(60);
r.render();
});
const initial = await page.evaluate(() => {
const r = window.rl;
return {
toys: r.interactiveProps.map((p) => ({
type: p.type,
dof: p.dofAdr,
mass: r.model.body_mass[p.bodyId],
})),
meshes: r.scene.getObjectByName("interactive-props").children.length,
ballId: r.mujoco.mj_name2id(
r.model,
r.mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
),
hemisphere: r.scene.parent.children.some(
(o) => o.isHemisphereLight && o.intensity >= 0.7,
),
environment: r.scene.parent.environmentIntensity,
robotMaterials: [...r.scene.children]
.filter((o) => o.name !== "interactive-props")
.flatMap((o) => {
const result = [];
o.traverse((m) => {
if (m.isMesh && m.material?.color)
result.push([m.uuid, m.material.color.toArray()]);
});
return result;
}),
};
});
expect(initial.toys.length).toBeGreaterThanOrEqual(5);
expect(initial.toys.length).toBeLessThanOrEqual(7);
expect(initial.meshes).toBe(initial.toys.length);
expect(new Set(initial.toys.map((p) => p.type)).size).toBe(4);
expect(initial.toys.every((p) => p.mass > 0 && p.dof >= 0)).toBe(true);
expect(initial.ballId).toBe(-1);
expect(initial.hemisphere).toBe(true);
expect(initial.environment).toBeGreaterThanOrEqual(0.7);
const group = page.getByRole("group", { name: "Scene line colours" });
await expect(group.getByRole("button")).toHaveCount(4);
for (const [label, hex] of [
["Cream", "f7e6cb"],
["Graphite", "6c6a68"],
["Lavender", "bfa9cf"],
["Sky", "a9dbe8"],
]) {
const button = group.getByRole("button", { name: `${label} scene lines` });
await button.click();
await expect(button).toHaveAttribute("aria-pressed", "true");
const values = await page.evaluate(() => {
const r = window.rl,
lines = [],
robotMaterials = [];
r.scene.traverse((o) => {
if (o.material?.uniforms?.uSectionColor)
lines.push(o.material.uniforms.uSectionColor.value.getHexString());
});
for (const o of r.scene.children.filter(
(o) => o.name !== "interactive-props",
))
o.traverse((m) => {
if (m.isMesh && m.material?.color)
robotMaterials.push([m.uuid, m.material.color.toArray()]);
});
const style = getComputedStyle(
document.querySelector('[aria-label="Scene line colours"] button'),
);
return {
lines,
robotMaterials,
radius: style.borderRadius,
width: style.width,
height: style.height,
};
});
expect(values.lines).toEqual(Array(5).fill(hex));
expect(values.robotMaterials).toEqual(initial.robotMaterials);
expect(values.radius).toBe("0px");
expect(values.width).toBe(values.height);
}
await page.evaluate(() => {
const r = window.rl;
r.chaseCam = false;
r.camera.position.set(-0.2, 1.9, 2.7);
r.controls.target.set(0.1, 0.05, 0);
r.controls.update();
});
await page.screenshot({
path: "test-results/scene-interaction-overview.png",
});
expect(errors).toEqual([]);
});
test("geometry can be dragged with physical force; pause/reset clear grab and restore layout", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(false);
r.chaseCam = false;
const spec = r.interactiveProps[0],
q = r.data.qpos;
q.set([-0.25, 0, spec.pos[2]], spec.qAdr);
r.mujoco.mj_forward(r.model, r.data);
r.camera.position.set(-0.6, 0.65, 1.2);
r.controls.target.set(q[0], q[2], -q[1]);
r.controls.update();
r.camera.updateMatrixWorld();
r.render();
});
await page.waitForTimeout(250);
const point = await page.evaluate(() => {
const r = window.rl,
spec = r.interactiveProps[0],
q = r.data.qpos;
const p = r.controls.target
.clone()
.set(q[spec.qAdr], q[spec.qAdr + 2], -q[spec.qAdr + 1])
.project(r.camera);
const rect = document.querySelector("canvas").getBoundingClientRect();
return {
x: ((p.x + 1) * rect.width) / 2,
y: ((1 - p.y) * rect.height) / 2,
};
});
await page.mouse.move(point.x, point.y);
await page.mouse.down();
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(false);
await page.mouse.move(point.x + 90, point.y - 40, { steps: 6 });
const result = await page.evaluate(async () => {
const r = window.rl,
spec = r.interactiveProps[0];
await r.step(15);
return {
x: r.data.qpos[spec.qAdr],
force: Array.from(
r.data.xfrc_applied.slice(spec.bodyId * 6, spec.bodyId * 6 + 3),
),
};
});
expect(Math.abs(result.x + 0.25)).toBeGreaterThan(0.01);
expect(result.force.some((v) => Math.abs(v) > 0.001)).toBe(true);
await page.evaluate(() => window.__gameApi.pause());
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(true);
await page.mouse.up();
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).every((v) => v === 0),
),
).toBe(true);
await page.evaluate(() => window.__gameApi.reset());
expect(
await page.evaluate(() => {
const r = window.rl;
return (
r.interactiveProps.every((p) =>
p.pos.every((v, i) => r.data.qpos[p.qAdr + i] === v),
) && r.controls.enabled
);
}),
).toBe(true);
expect(errors).toEqual([]);
});
test("toys transfer collision momentum, collide with the robot and floor, and survive a mode switch", async ({
page,
}) => {
const errors = await bootDuck(page);
const collision = await page.evaluate(async () => {
const r = window.rl;
r.setPropsParked(false);
const specs = r.interactiveProps;
// Keep unrelated toys clear of a two-body collision lane.
specs.forEach((p, i) => {
r.data.qpos.set([-1 + i * 0.3, -1.1, p.pos[2]], p.qAdr);
});
const a = specs[0],
b = specs[1];
r.data.qpos.set([0.1, 0.7, a.pos[2]], a.qAdr);
r.data.qpos.set([0.35, 0.7, b.pos[2]], b.qAdr);
r.data.qvel[a.dofAdr] = 1.8;
r.mujoco.mj_forward(r.model, r.data);
await r.step(15);
const moved = r.data.qpos[b.qAdr] - 0.35;
// Launch a toy at the duck. A non-colliding body would cross x=-0.6.
r.resetSim();
const p = r.interactiveProps.find((s) => s.type === "sphere");
r.data.qpos.set([-0.35, 0, 0.09], p.qAdr);
r.data.qvel[p.dofAdr] = -1.5;
r.mujoco.mj_forward(r.model, r.data);
let contacts = 0,
minX = 10;
for (let i = 0; i < 15; i++) {
await r.step(1);
contacts = Math.max(contacts, r.data.ncon);
minX = Math.min(minX, r.data.qpos[p.qAdr]);
}
await r.step(100);
const half = r.world.boundary.half_extent;
r.data.qpos.set([half - 0.2, 0.9, p.pos[2]], p.qAdr);
r.data.qvel.fill(0, p.dofAdr, p.dofAdr + 6);
r.data.qvel[p.dofAdr] = 1.5;
r.mujoco.mj_forward(r.model, r.data);
let wallMax = -Infinity;
for (let i = 0; i < 15; i++) {
await r.step(1);
wallMax = Math.max(wallMax, r.data.qpos[p.qAdr]);
}
return {
moved,
contacts,
minX,
wallMax,
half,
finite: Array.from(r.data.qpos).every(Number.isFinite),
aboveFloor: r.interactiveProps.every(
(s) => r.data.qpos[s.qAdr + 2] > 0.025,
),
};
});
expect(collision.moved).toBeGreaterThan(0.025);
expect(collision.contacts).toBeGreaterThan(0);
expect(collision.minX).toBeGreaterThan(-0.58);
expect(collision.wallMax).toBeLessThan(collision.half - 0.03);
expect(collision.aboveFloor).toBe(true);
expect(collision.finite).toBe(true);
await page.evaluate(() => window.rl.setLoco("rollers", { force: true }));
const switched = await page.evaluate(async () => {
const r = window.rl;
await r.step(100);
r.render();
return {
count: r.interactiveProps.length,
meshes: r.scene.getObjectByName("interactive-props").children.length,
finite: Array.from(r.data.qpos).every(Number.isFinite),
};
});
expect(switched.count).toBe(switched.meshes);
expect(switched.finite).toBe(true);
expect(errors).toEqual([]);
});
+68
View File
@@ -0,0 +1,68 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("two simulated minutes of movement with physical toys stay finite in feet and rollers; escape recovery is local", async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate(() => {
const r = window.rl;
r.setPropsParked(false);
// A deterministic controller source exercises actual commanded motion,
// independent of frame-rate keyboard polling during manual stepping.
r.controller.addSource({
id: "stability-drive",
isActive: () => true,
command: new Float32Array([0.25, 0, 0.4]),
});
});
for (const mode of ["legs", "rollers"]) {
await page.evaluate(async (mode) => {
const r = window.rl;
await r.setLoco(mode, { force: true });
r.controller.sources
.find((s) => s.id === "stability-drive")
.command.set(mode === "legs" ? [0.25, 0, 0.4] : [0.4, 0, 0.15]);
}, mode);
let moved = false;
for (let chunk = 0; chunk < 30; chunk++) {
const sample = await page.evaluate(async () => {
const r = window.rl;
await r.step(100);
return {
finite: [r.data.qpos, r.data.qvel, r.data.ctrl].every((array) =>
Array.from(array).every(Number.isFinite),
),
moved: Math.hypot(r.data.qpos[0] + 0.6, r.data.qpos[1]) > 0.04,
inside: r.interactiveProps.every(
(p) =>
Math.abs(r.data.qpos[p.qAdr]) <
r.world.boundary.half_extent + 0.11 &&
Math.abs(r.data.qpos[p.qAdr + 1]) <
r.world.boundary.half_extent + 0.11,
),
};
});
expect(sample.finite).toBe(true);
expect(sample.inside).toBe(true);
moved ||= sample.moved;
}
expect(moved).toBe(true);
}
const recovery = await page.evaluate(async () => {
const r = window.rl,
p = r.interactiveProps[0];
r.data.qpos[p.qAdr] = r.world.boundary.half_extent + 1;
r.mujoco.mj_forward(r.model, r.data);
await r.step(1);
return {
actual: Array.from(r.data.qpos.slice(p.qAdr, p.qAdr + 3)),
expected: p.pos,
count: r.interactiveProps.length,
};
});
expect(recovery.actual).toEqual(recovery.expected);
expect(recovery.count).toBeGreaterThanOrEqual(5);
expect(recovery.count).toBeLessThanOrEqual(7);
expect(errors).toEqual([]);
});
+157
View File
@@ -0,0 +1,157 @@
import { test, expect } from "@playwright/test";
import { bootDuck } from "./support/simulation.js";
test("all robots share the same scaled random layout, lighting, line palette and global colour selection", async ({
page,
}) => {
const errors = await bootDuck(page);
let reference;
let expected = { name: "Sky", key: "blue", hex: "a9dbe8" };
for (const id of ["microduck-legs", "microban", "rp1", "microduck-legs"]) {
if ((await page.evaluate(() => window.__store.getState().robotId)) !== id)
await page.evaluate((id) => window.__gameApi.switchRobot(id), id);
await expect(
page.getByRole("group", { name: "Scene line colours" }),
).toBeVisible();
if (!reference)
await page.getByRole("button", { name: "Sky scene lines" }).click();
const state = await page.evaluate(() => {
const r = window.rl,
scale = r.world.boundary.half_extent / 2.25;
const lines = [];
r.scene.traverse((o) => {
if (o.material?.uniforms?.uSectionColor)
lines.push(o.material.uniforms.uSectionColor.value.getHexString());
});
return {
specs: r.interactiveProps.map((p) => ({
name: p.name,
type: p.type,
color: p.color,
pos: p.pos.map((v) => v / scale),
size: p.size.map((v) => v / scale),
})),
count: r.scene.getObjectByName("interactive-props").children.length,
lines,
lineColor: window.__store.getState().lineColor,
lights: r.scene.parent.children
.filter((o) => o.isLight)
.map((o) => [o.type, o.intensity, o.color.getHex()]),
environment: r.scene.parent.environmentIntensity,
drag: window.__store.getState().runtimeCapabilities.physicalDrag,
};
});
expect(state.count).toBeGreaterThanOrEqual(5);
expect(state.count).toBeLessThanOrEqual(7);
expect(state.lines).toEqual(Array(5).fill(expected.hex));
expect(state.lineColor).toBe(expected.key);
expect(state.drag).toBe(true);
if (!reference) reference = state;
state.specs.forEach((p, i) => {
expect(p.type).toBe(reference.specs[i].type);
expect(p.color).toBe(reference.specs[i].color);
p.pos.forEach((v, j) =>
expect(v).toBeCloseTo(reference.specs[i].pos[j], 12),
);
p.size.forEach((v, j) =>
expect(v).toBeCloseTo(reference.specs[i].size[j], 12),
);
});
expect(state.lights).toEqual(reference.lights);
expect(state.environment).toBe(reference.environment);
await expect(
page.getByRole("button", { name: `${expected.name} scene lines` }),
).toHaveAttribute("aria-pressed", "true");
await expect(
page.getByRole("button", { name: "Rollers", exact: true }),
).toHaveCount(id === "microduck-legs" ? 1 : 0);
// Also carry a choice made in another robot back to Microduck.
if (id === "rp1") {
await page.getByRole("button", { name: "Lavender scene lines" }).click();
expected = { name: "Lavender", key: "purple", hex: "bfa9cf" };
}
}
expect(errors).toEqual([]);
});
for (const id of ["microban", "rp1"]) {
test(`${id}: shared toys collide with walls, drag with real force and release on pause`, async ({
page,
}) => {
const errors = await bootDuck(page);
await page.evaluate((id) => window.__gameApi.switchRobot(id), id);
const wall = await page.evaluate(async () => {
const r = window.rl,
scale = r.world.boundary.half_extent / 2.25,
p = r.interactiveProps.find((s) => s.type === "sphere"),
half = r.world.boundary.half_extent;
r.data.qpos.set([half - 0.2 * scale, 0.9 * scale, p.pos[2]], p.qAdr);
r.data.qvel[p.dofAdr] = 1.5 * scale;
r.mujoco.mj_forward(r.model, r.data);
let max = -Infinity;
for (let i = 0; i < 15; i++) {
await r.step(1);
max = Math.max(max, r.data.qpos[p.qAdr]);
}
await r.resetSim();
return { max, limit: half - 0.025 * scale };
});
expect(wall.max).toBeLessThan(wall.limit);
await page.getByRole("button", { name: /Keyboard controls/ }).click();
await page.evaluate(() => {
const r = window.rl,
scale = r.world.scale ?? 1,
p = r.interactiveProps[0],
q = r.data.qpos;
q.set([0.25 * scale, -0.3 * scale, p.pos[2]], p.qAdr);
r.mujoco.mj_forward(r.model, r.data);
r.camera.position.set(q[0], q[2] + 0.6 * scale, 1.5 * scale);
r.controls.target.set(q[0], q[2], -q[1]);
r.controls.update();
window.__gameApi.resume();
});
await page.waitForTimeout(200);
const point = await page.evaluate(() => {
const r = window.rl,
p = r.interactiveProps[0],
q = r.data.qpos,
rect = document.querySelector("canvas").getBoundingClientRect();
const v = r.controls.target
.clone()
.set(q[p.qAdr], q[p.qAdr + 2], -q[p.qAdr + 1])
.project(r.camera);
return {
x: ((v.x + 1) * rect.width) / 2,
y: ((1 - v.y) * rect.height) / 2,
};
});
await page.mouse.move(point.x, point.y);
await page.mouse.down();
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(false);
await page.mouse.move(point.x + 70, point.y - 35, { steps: 5 });
await page.waitForTimeout(200);
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).some((v) => Math.abs(v) > 1e-5),
),
).toBe(true);
await page.evaluate(() => window.__gameApi.pause());
expect(await page.evaluate(() => window.rl.controls.enabled)).toBe(true);
expect(
await page.evaluate(() =>
Array.from(window.rl.data.xfrc_applied).every((v) => v === 0),
),
).toBe(true);
await page.mouse.up();
await page.evaluate(() => window.__gameApi.reset());
expect(
await page.evaluate(() => {
const r = window.rl;
return r.interactiveProps.every((p) =>
p.pos.every((v, i) => r.data.qpos[p.qAdr + i] === v),
);
}),
).toBe(true);
expect(errors).toEqual([]);
});
}
+2 -2
View File
@@ -1,5 +1,5 @@
import { expect } from "@playwright/test";
export async function bootDuck(page) {
export async function bootDuck(page, { legacyPhysics = false } = {}) {
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await page.addInitScript(() => {
@@ -7,7 +7,7 @@ export async function bootDuck(page) {
Math.random = () =>
(seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
});
await page.goto("/?boot=1&test=1");
await page.goto(`/?boot=1&test=1${legacyPhysics ? "&oracle=legacy" : ""}`);
await page.waitForFunction(
() => window.rl?.prepareTest || window.__store?.getState().bootFailed,
);
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "microduck-sandbox",
"version": "0.4.0",
"version": "0.5.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "microduck-sandbox",
"version": "0.4.0",
"version": "0.5.0",
"dependencies": {
"@emotion/react": "^11.14.0",
"@emotion/styled": "^11.14.1",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "microduck-sandbox",
"private": true,
"version": "0.4.0",
"version": "0.5.0",
"type": "module",
"scripts": {
"dev": "vite",
+12
View File
@@ -30,6 +30,18 @@ Source archives under `robolab/data/motions/rp1_bm/`:
Both skills use exclusive full-body policies, a physical stop gate, entry/return blends and locomotion interruption. Validation includes three initial headings, walking before/after, repeated episodes without robot reset, keyboard/pause/reset, numerical reference checks and malformed-asset rollback. Initial integration: `docs/superpowers/verification/2026-10-29-rp1-completed-skills.md`; current dance weights: `docs/superpowers/verification/2026-10-29-rp1-dance-23998.md`. UI labels are simply `Dance` and `Wave`; durations and transition behavior remain unchanged.
## User-provided GLB material replacement
`RP1机器人glb.zip` supplies 63 Blender-exported, self-contained GLBs. `visual-source-manifest.json` records the archive SHA-256, all 63 source-part hashes, 25 output hashes and the exact body mapping. Original MJCF/STL, policy and motion files retain their existing hashes; the only manifest change is `render.body_visuals`.
- Twenty-three bodies use byte-identical source GLBs. Their Blender Y-up coordinates are rotated +90° about X into MuJoCo body Z-up. All 63 source-part bounds were independently checked against the upstream STL bounds after this axis conversion: exact agreement.
- The existing 24-DOF model has only one `left_wrist_link` / `right_wrist_link` body per side. Each visual wrist therefore composes **20 parts**: wrist roll/yaw/pitch plus the palm and 16 finger pieces. Transforms come from the copied upstream `visuals/rp1_hand_fixed.urdf`; extra wrist yaw/pitch stay neutral, fingers stay in their fixed poses. There are **no additional physical joints or finger-control claims**.
- The assembler preserves source vertex/index/image bytes, remaps GLTF indices and retains original PBR colours, roughness/metalness, normal maps, face screen texture and emissive torso logo. It adds only visual transform wrapper nodes.
- Reproduce with `python3 app/scripts/import-rp1-visuals.py /path/to/RP1机器人glb.zip app/public/robots/rp1/visuals/rp1_hand_fixed.urdf` from the repo root. All inputs/outputs validate before installation; ZIP paths are never extracted.
- Geometry additions in the hands are visual only. Existing collision envelopes intentionally remain those used by the trained 24-DOF model.
The ZIP supplied no license/attribution file. Do not infer redistribution rights from receipt of these assets. It remains a local-evaluation package under the limitations below.
## License limitation — local evaluation only
The RP1 description submodule does not supply a license file in this checkout. Do **not** infer an Apache/BSD license for its meshes/model or motion data from the platform or RoboLab. Confirm upstream rights before redistributing this package or deploying publicly.
+253 -1
View File
@@ -678,7 +678,259 @@
],
"camera": {
"distance": 2.4
}
},
"body_visuals": [
{
"body": "base_link",
"path": "visuals/base_link__visual_0__base_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_pitch_link",
"path": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_roll_link",
"path": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_hip_yaw_link",
"path": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_knee_link",
"path": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_ankle_pitch_link",
"path": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_ankle_roll_link",
"path": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_pitch_link",
"path": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_roll_link",
"path": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_hip_yaw_link",
"path": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_knee_link",
"path": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_ankle_pitch_link",
"path": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_ankle_roll_link",
"path": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "waist_roll_link",
"path": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "waist_yaw_link",
"path": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_pitch_link",
"path": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_roll_link",
"path": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_shoulder_yaw_link",
"path": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_elbow_link",
"path": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "left_wrist_link",
"path": "visuals/left_wrist_link__fixed_hand.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_pitch_link",
"path": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_roll_link",
"path": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_shoulder_yaw_link",
"path": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_elbow_link",
"path": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
},
{
"body": "right_wrist_link",
"path": "visuals/right_wrist_link__fixed_hand.glb",
"quaternion": [
0.7071067811865476,
0,
0,
0.7071067811865476
]
}
]
},
"presentation": {
"theme": "comic"
@@ -0,0 +1,557 @@
{
"source": "User-provided RP1\u673a\u5668\u4ebaglb.zip",
"archive_sha256": "8610aac8432203bf68d0efe234062c551ca805ebbe72a92d5fc7350730ebdea2",
"license": "Not supplied in the archive; local evaluation only. Confirm redistribution rights.",
"coordinate_conversion": "Source GLBs are Blender Y-up; attachments rotate +90 degrees about X into MuJoCo body Z-up.",
"assembly": "25 physical body attachments; 20 source parts per wrist (neutral extra wrist axes and fixed fingers). No physical joints added.",
"input_files": [
{
"file": "base_link__visual_0__base_link.glb",
"sha256": "3db82d0e5fa1fa48fa6f4fe4460bba14568a39b5933acff8b4b1879f582e06b0"
},
{
"file": "left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"sha256": "d8cb2aff7273185a89eba99aab47ce6bf4716f25e70f34afa1529acb802c6193"
},
{
"file": "left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"sha256": "35361b79fb69096039eb077018c25ae489f16cc5229dcd82256b3a573b168208"
},
{
"file": "left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"sha256": "19fe908274f9fe49b5a8317652e4fe02a6ff93802de87ad7d0383fa1d142b03b"
},
{
"file": "left_knee_link__visual_0__left_knee_link.glb",
"sha256": "1c01f5c03837d24287d79eff48a9e4bd4203d8516d0375592b159d49fb83f6c9"
},
{
"file": "left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"sha256": "4bab717bf7073e28754d1c42f4409141ea775987284ff8575f9c7aef25561e8c"
},
{
"file": "left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"sha256": "76144b6ebf17b2eb5875168cbe6a3c4c12eaeebdd4bd277864887b3894f781b1"
},
{
"file": "right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"sha256": "e299f46bff44d6bbf03592cd6caba3b418fc022260746554e6d2013f31793df4"
},
{
"file": "right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"sha256": "7cb006f9ba7f950c98287b59695938cfd6460d8fa4546ae6c95bc65ed5c6c6b8"
},
{
"file": "right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"sha256": "35393465aa8103ab436ecc3a635cdce7477600cf65eb03a2942af546cc39c79c"
},
{
"file": "right_knee_link__visual_0__right_knee_link.glb",
"sha256": "2a5f1f91f6db0cfac3a0e88038613be89cb94066d7f103876b455823edbfb8f3"
},
{
"file": "right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"sha256": "c43112750efd953f29dfea5d8d2bb925eb847f74b679fa1d22f734b1e3094b91"
},
{
"file": "right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"sha256": "3856c0074bbead9ade091b2bef4df165d9c1d4b03e0c0f1d8c284cddf091a9f6"
},
{
"file": "waist_roll_link__visual_0__waist_roll_link.glb",
"sha256": "a4cd49014951ed5c7836bde4a7138eab4119f0ca7249a69e26add388d5fedf32"
},
{
"file": "waist_yaw_link__visual_0__waist_yaw_link.glb",
"sha256": "cb50b0354f8a0a6d06ff486d46cb849462b84471c20a350b1e4c5a86577ce230"
},
{
"file": "left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"sha256": "9eb76bc0675578e28707120d64cb042b948312ddc9233e4c28de02ee04a3dbf9"
},
{
"file": "left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"sha256": "f72e3ac73f3804acd5e5384c2d10d48aacacb95bf9f98ea0f2426f1b92add0de"
},
{
"file": "left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"sha256": "909f0dbd40db57b223143c1c89c740c3ea636e5d9b066eb6fe26dab37e2f4975"
},
{
"file": "left_elbow_link__visual_0__left_elbow_link.glb",
"sha256": "f98a4446175c7b1d98c5a664fabcb6698635c99ccfc8261dedbef9155e1a2043"
},
{
"file": "left_wrist_roll_link__visual_0__left_wrist_roll_link.glb",
"sha256": "fd4dff45bb8c6caa79ceff1286cf8d8d69de22f781fbbd12a91baf87636e0ad2"
},
{
"file": "left_wrist_yaw_link__visual_0__left_wrist_yaw_link.glb",
"sha256": "65835f9275362e18801a3247191323f0661f538a3c24836cc78fdd1ec5560a3e"
},
{
"file": "left_wrist_pitch_link__visual_0__left_wrist_pitch_link.glb",
"sha256": "e036068c1f36349c26a8daba9472fd40cfec5ae55424a55a95a4c75472c9dd48"
},
{
"file": "right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"sha256": "21d9410ca84dcd3bfea3bd8c08a682d4ff640c79f68e5d9f93ea26bcf904c3ac"
},
{
"file": "right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"sha256": "89b5da7da64e2d3207291c7847d347d30fe0b40014ea6d6c4fc9e4bf4e03682c"
},
{
"file": "right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"sha256": "2f0c17876e2607e8ab4282fadefba6fee6c2a5446b416cce6d0b91f796da2893"
},
{
"file": "right_elbow_link__visual_0__right_elbow_link.glb",
"sha256": "ef1e9dc1afcf147e78902039f49ef0131f114a11afe873b5095a6c4f509b89b2"
},
{
"file": "right_wrist_roll_link__visual_0__right_wrist_roll_link.glb",
"sha256": "5efc62e433d732ec26a5b45d3be64f421fb9c26176f168bdde74a9737e48eae3"
},
{
"file": "right_wrist_yaw_link__visual_0__right_wrist_yaw_link.glb",
"sha256": "f6f7d91665a8645a0b5cbd2e311c78aaf39f9f2554ac232af0425bad7a38164b"
},
{
"file": "right_wrist_pitch_link__visual_0__right_wrist_pitch_link.glb",
"sha256": "48d0a47d41569a24a8ea4b83e7218084395f9ef96c4dcbcdca1a3acbc060ad64"
},
{
"file": "lh_base_link__visual_0__base_link.glb",
"sha256": "886152daa0f4982f20d346639c4fd29f779b0d8fbf8e8d4942684bbc0cbe9946"
},
{
"file": "lh_finger11_link__visual_0__finger11_link.glb",
"sha256": "c59c8b126390c3593f63c92c6b3ea0198174342f0da2a7e63f3f6d6c17c76b80"
},
{
"file": "lh_finger12_link__visual_0__finger12_link.glb",
"sha256": "dea1f32f1f098fb26e6163abeefb98d4d2e6dfb0c187446b5a3b3dc83c40b4a1"
},
{
"file": "lh_finger13_link__visual_0__finger13_link.glb",
"sha256": "688ab4e9a8191ae6bda909d099ade525be02a7de758144863dc44d731e4180fe"
},
{
"file": "lh_finger14_link__visual_0__finger14_link.glb",
"sha256": "3ebdfb11c136e4e990fc99d0af6ed9a46713c799332ecd311b402b562697592c"
},
{
"file": "lh_finger21_link__visual_0__finger21_link.glb",
"sha256": "7ac15bcfa68e38e33a49942f7a2772e63778d82b7181494a0eacfe1a73510721"
},
{
"file": "lh_finger22_link__visual_0__finger22_link.glb",
"sha256": "358757dbce84eb6cb81a6da3d2f54c467e395306819382c534624e2c67f4ae73"
},
{
"file": "lh_finger23_link__visual_0__finger23_link.glb",
"sha256": "0ff49e9c01a744186246671737bc2e45c3ec95980538851a0a142030100db16f"
},
{
"file": "lh_finger31_link__visual_0__finger31_link.glb",
"sha256": "4ed855cb9634a928da50b6aa645a00b244455c4541403a65dc451f78c941fc1d"
},
{
"file": "lh_finger32_link__visual_0__finger32_link.glb",
"sha256": "9d688372b46fd24ca06fb4767d2b7b462054949467b9f49b37532ade5827b076"
},
{
"file": "lh_finger33_link__visual_0__finger33_link.glb",
"sha256": "cb0695f569da22e06d69f9110f0779534bdf64e185d6f27b54ff89dde1f31275"
},
{
"file": "lh_finger41_link__visual_0__finger41_link.glb",
"sha256": "270a0e7aaae7cee7d0853e0300686d202aa6d92167645dbc6318ddc35ce3e61a"
},
{
"file": "lh_finger42_link__visual_0__finger42_link.glb",
"sha256": "40cf87025b184bcd92ecc00980c83449c1a62c5e7a849311c960c7b51f871e76"
},
{
"file": "lh_finger43_link__visual_0__finger43_link.glb",
"sha256": "ac791ee4a9b1877892539cb6d4e5f8145d6ad199cefb2e04ebba449c59d2416d"
},
{
"file": "lh_finger51_link__visual_0__finger51_link.glb",
"sha256": "e43276c24c230bd35459a5b4f442e0be7840c4db106ada75a51ab83ff4a17dcd"
},
{
"file": "lh_finger52_link__visual_0__finger52_link.glb",
"sha256": "3c156a7b898e695b3e34287ad12e6b5437bcd478823baffe2cd76702c6dd9626"
},
{
"file": "lh_finger53_link__visual_0__finger53_link.glb",
"sha256": "140fe9939ed6b58539a77ef0c4cf1608dc0cc16b5c8d043564dbab9b1842dd63"
},
{
"file": "rh_base_link__visual_0__base_link.glb",
"sha256": "ae7328599013944ceb0ac4e4bb0106c0e58526122624a7bb6e650b138b07840d"
},
{
"file": "rh_finger11_link__visual_0__finger11_link.glb",
"sha256": "289cdf7c384b181df0d64f47db579af622db3bcbdc42236e667dcb64c494377c"
},
{
"file": "rh_finger12_link__visual_0__finger12_link.glb",
"sha256": "505bc65a50290aefe7b1882b5ffad4e88bf7059ec9dee355014592a2f21fdd2d"
},
{
"file": "rh_finger13_link__visual_0__finger13_link.glb",
"sha256": "324cf4f7e6207912d5a709ddc85f53b76be930208c27bff8c46f8d8ec5a61922"
},
{
"file": "rh_finger14_link__visual_0__finger14_link.glb",
"sha256": "f41140c191686f6afd809f51aa3bd1341517337c160ef019980b7c278dce3296"
},
{
"file": "rh_finger21_link__visual_0__finger21_link.glb",
"sha256": "f6e9a6ce375be6107158d00df23ab416445eb6970abe25e0f80c840da4611684"
},
{
"file": "rh_finger22_link__visual_0__finger22_link.glb",
"sha256": "9e89e30d9c3d0115bcfc85b05a6b7761991443df5cd74a75b305b5025a855245"
},
{
"file": "rh_finger23_link__visual_0__finger23_link.glb",
"sha256": "2c7529aa60a4286e2630146371f6dd77228a5ffd1c8760aa0d34169aa1aeae2c"
},
{
"file": "rh_finger31_link__visual_0__finger31_link.glb",
"sha256": "253bef5027cbce1a055f4650134bafec3f58dbc8f0a7e417ddc338e3de3d514c"
},
{
"file": "rh_finger32_link__visual_0__finger32_link.glb",
"sha256": "5b3ff8e747900ebfb2fb36ff5ee1035e741e780dd3613169058643436575e4aa"
},
{
"file": "rh_finger33_link__visual_0__finger33_link.glb",
"sha256": "d440a8b8bcd8b3e3de414598af99ebfefd214d4eb58d7b4d77d1bd67a81f75cf"
},
{
"file": "rh_finger41_link__visual_0__finger41_link.glb",
"sha256": "0a2d053155922ebcfaccf6f893665beb72f7ee2b3a9a66694b695995f233c7cf"
},
{
"file": "rh_finger42_link__visual_0__finger42_link.glb",
"sha256": "aa517ec5881c1ca9840c21ae50461eacba59d76cd835fe735eba19030d49cbbc"
},
{
"file": "rh_finger43_link__visual_0__finger43_link.glb",
"sha256": "f1d4c2936c26267a20da3c1a4016e51316ac100e46e310ffb57af5debd111ef6"
},
{
"file": "rh_finger51_link__visual_0__finger51_link.glb",
"sha256": "3800f27cbbd3c7bc79ad72def333d841001614d6e9ba16ce3b9502385102771d"
},
{
"file": "rh_finger52_link__visual_0__finger52_link.glb",
"sha256": "24a3fd05bfe9273f150c93c2ccb79b723c02e70169c0cb2ed7432ced45be50c8"
},
{
"file": "rh_finger53_link__visual_0__finger53_link.glb",
"sha256": "98cd0bde6c4d27ad04bd3b567ec9e88a9ed849fe786f2bfdb775c33baacc3638"
}
],
"body_mapping": {
"base_link": {
"path": "visuals/base_link__visual_0__base_link.glb",
"parts": [
"base_link"
]
},
"left_hip_pitch_link": {
"path": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"parts": [
"left_hip_pitch_link"
]
},
"left_hip_roll_link": {
"path": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"parts": [
"left_hip_roll_link"
]
},
"left_hip_yaw_link": {
"path": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"parts": [
"left_hip_yaw_link"
]
},
"left_knee_link": {
"path": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"parts": [
"left_knee_link"
]
},
"left_ankle_pitch_link": {
"path": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"parts": [
"left_ankle_pitch_link"
]
},
"left_ankle_roll_link": {
"path": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"parts": [
"left_ankle_roll_link"
]
},
"right_hip_pitch_link": {
"path": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"parts": [
"right_hip_pitch_link"
]
},
"right_hip_roll_link": {
"path": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"parts": [
"right_hip_roll_link"
]
},
"right_hip_yaw_link": {
"path": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"parts": [
"right_hip_yaw_link"
]
},
"right_knee_link": {
"path": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"parts": [
"right_knee_link"
]
},
"right_ankle_pitch_link": {
"path": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"parts": [
"right_ankle_pitch_link"
]
},
"right_ankle_roll_link": {
"path": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"parts": [
"right_ankle_roll_link"
]
},
"waist_roll_link": {
"path": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"parts": [
"waist_roll_link"
]
},
"waist_yaw_link": {
"path": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"parts": [
"waist_yaw_link"
]
},
"left_shoulder_pitch_link": {
"path": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"parts": [
"left_shoulder_pitch_link"
]
},
"left_shoulder_roll_link": {
"path": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"parts": [
"left_shoulder_roll_link"
]
},
"left_shoulder_yaw_link": {
"path": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"parts": [
"left_shoulder_yaw_link"
]
},
"left_elbow_link": {
"path": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"parts": [
"left_elbow_link"
]
},
"left_wrist_link": {
"path": "visuals/left_wrist_link__fixed_hand.glb",
"parts": [
"left_wrist_roll_link",
"left_wrist_yaw_link",
"left_wrist_pitch_link",
"lh_base_link",
"lh_finger11_link",
"lh_finger12_link",
"lh_finger13_link",
"lh_finger14_link",
"lh_finger21_link",
"lh_finger22_link",
"lh_finger23_link",
"lh_finger31_link",
"lh_finger32_link",
"lh_finger33_link",
"lh_finger41_link",
"lh_finger42_link",
"lh_finger43_link",
"lh_finger51_link",
"lh_finger52_link",
"lh_finger53_link"
]
},
"right_shoulder_pitch_link": {
"path": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"parts": [
"right_shoulder_pitch_link"
]
},
"right_shoulder_roll_link": {
"path": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"parts": [
"right_shoulder_roll_link"
]
},
"right_shoulder_yaw_link": {
"path": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"parts": [
"right_shoulder_yaw_link"
]
},
"right_elbow_link": {
"path": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"parts": [
"right_elbow_link"
]
},
"right_wrist_link": {
"path": "visuals/right_wrist_link__fixed_hand.glb",
"parts": [
"right_wrist_roll_link",
"right_wrist_yaw_link",
"right_wrist_pitch_link",
"rh_base_link",
"rh_finger11_link",
"rh_finger12_link",
"rh_finger13_link",
"rh_finger14_link",
"rh_finger21_link",
"rh_finger22_link",
"rh_finger23_link",
"rh_finger31_link",
"rh_finger32_link",
"rh_finger33_link",
"rh_finger41_link",
"rh_finger42_link",
"rh_finger43_link",
"rh_finger51_link",
"rh_finger52_link",
"rh_finger53_link"
]
}
},
"files": [
{
"file": "visuals/rp1_hand_fixed.urdf",
"sha256": "2c1f91a82f39298fdda7ec9531d045caff745ba12157f0bb0fcb2f4cf1102948"
},
{
"file": "visuals/base_link__visual_0__base_link.glb",
"sha256": "3db82d0e5fa1fa48fa6f4fe4460bba14568a39b5933acff8b4b1879f582e06b0"
},
{
"file": "visuals/left_hip_pitch_link__visual_0__left_hip_pitch_link.glb",
"sha256": "d8cb2aff7273185a89eba99aab47ce6bf4716f25e70f34afa1529acb802c6193"
},
{
"file": "visuals/left_hip_roll_link__visual_0__left_hip_roll_link.glb",
"sha256": "35361b79fb69096039eb077018c25ae489f16cc5229dcd82256b3a573b168208"
},
{
"file": "visuals/left_hip_yaw_link__visual_0__left_hip_yaw_link.glb",
"sha256": "19fe908274f9fe49b5a8317652e4fe02a6ff93802de87ad7d0383fa1d142b03b"
},
{
"file": "visuals/left_knee_link__visual_0__left_knee_link.glb",
"sha256": "1c01f5c03837d24287d79eff48a9e4bd4203d8516d0375592b159d49fb83f6c9"
},
{
"file": "visuals/left_ankle_pitch_link__visual_0__left_ankle_pitch_link.glb",
"sha256": "4bab717bf7073e28754d1c42f4409141ea775987284ff8575f9c7aef25561e8c"
},
{
"file": "visuals/left_ankle_roll_link__visual_0__left_ankle_roll_link.glb",
"sha256": "76144b6ebf17b2eb5875168cbe6a3c4c12eaeebdd4bd277864887b3894f781b1"
},
{
"file": "visuals/right_hip_pitch_link__visual_0__right_hip_pitch_link.glb",
"sha256": "e299f46bff44d6bbf03592cd6caba3b418fc022260746554e6d2013f31793df4"
},
{
"file": "visuals/right_hip_roll_link__visual_0__right_hip_roll_link.glb",
"sha256": "7cb006f9ba7f950c98287b59695938cfd6460d8fa4546ae6c95bc65ed5c6c6b8"
},
{
"file": "visuals/right_hip_yaw_link__visual_0__right_hip_yaw_link.glb",
"sha256": "35393465aa8103ab436ecc3a635cdce7477600cf65eb03a2942af546cc39c79c"
},
{
"file": "visuals/right_knee_link__visual_0__right_knee_link.glb",
"sha256": "2a5f1f91f6db0cfac3a0e88038613be89cb94066d7f103876b455823edbfb8f3"
},
{
"file": "visuals/right_ankle_pitch_link__visual_0__right_ankle_pitch_link.glb",
"sha256": "c43112750efd953f29dfea5d8d2bb925eb847f74b679fa1d22f734b1e3094b91"
},
{
"file": "visuals/right_ankle_roll_link__visual_0__right_ankle_roll_link.glb",
"sha256": "3856c0074bbead9ade091b2bef4df165d9c1d4b03e0c0f1d8c284cddf091a9f6"
},
{
"file": "visuals/waist_roll_link__visual_0__waist_roll_link.glb",
"sha256": "a4cd49014951ed5c7836bde4a7138eab4119f0ca7249a69e26add388d5fedf32"
},
{
"file": "visuals/waist_yaw_link__visual_0__waist_yaw_link.glb",
"sha256": "cb50b0354f8a0a6d06ff486d46cb849462b84471c20a350b1e4c5a86577ce230"
},
{
"file": "visuals/left_shoulder_pitch_link__visual_0__left_shoulder_pitch_link.glb",
"sha256": "9eb76bc0675578e28707120d64cb042b948312ddc9233e4c28de02ee04a3dbf9"
},
{
"file": "visuals/left_shoulder_roll_link__visual_0__left_shoulder_roll_link.glb",
"sha256": "f72e3ac73f3804acd5e5384c2d10d48aacacb95bf9f98ea0f2426f1b92add0de"
},
{
"file": "visuals/left_shoulder_yaw_link__visual_0__left_shoulder_yaw_link.glb",
"sha256": "909f0dbd40db57b223143c1c89c740c3ea636e5d9b066eb6fe26dab37e2f4975"
},
{
"file": "visuals/left_elbow_link__visual_0__left_elbow_link.glb",
"sha256": "f98a4446175c7b1d98c5a664fabcb6698635c99ccfc8261dedbef9155e1a2043"
},
{
"file": "visuals/left_wrist_link__fixed_hand.glb",
"sha256": "7b7bee1c2aa6ba74bc2824bf3ec5c465c33a97e1e0409ccdc551fae74a6218e5"
},
{
"file": "visuals/right_shoulder_pitch_link__visual_0__right_shoulder_pitch_link.glb",
"sha256": "21d9410ca84dcd3bfea3bd8c08a682d4ff640c79f68e5d9f93ea26bcf904c3ac"
},
{
"file": "visuals/right_shoulder_roll_link__visual_0__right_shoulder_roll_link.glb",
"sha256": "89b5da7da64e2d3207291c7847d347d30fe0b40014ea6d6c4fc9e4bf4e03682c"
},
{
"file": "visuals/right_shoulder_yaw_link__visual_0__right_shoulder_yaw_link.glb",
"sha256": "2f0c17876e2607e8ab4282fadefba6fee6c2a5446b416cce6d0b91f796da2893"
},
{
"file": "visuals/right_elbow_link__visual_0__right_elbow_link.glb",
"sha256": "ef1e9dc1afcf147e78902039f49ef0131f114a11afe873b5095a6c4f509b89b2"
},
{
"file": "visuals/right_wrist_link__fixed_hand.glb",
"sha256": "57a47c99b86d4efea1cd891147a0aafc86395daa31332a88b6653e44be1449c1"
}
]
}
File diff suppressed because it is too large Load Diff
+208
View File
@@ -0,0 +1,208 @@
#!/usr/bin/env python3
"""Import the user's 63 textured parts onto the existing 25-body/24-DOF RP1.
Usage: python3 app/scripts/import-rp1-visuals.py ARCHIVE HAND_FIXED_URDF
No MJCF, STL, policy, motion or physical joint is changed. The source's fixed
finger poses and neutral wrist yaw/pitch are composed into two visual-only
wrist GLBs. Source images/materials/binary vertex data are retained verbatim.
"""
import copy
import hashlib
import json
import math
from pathlib import Path
import struct
import sys
import xml.etree.ElementTree as ET
import zipfile
def sha(data):
return hashlib.sha256(data).hexdigest()
def read_glb(data):
if len(data) < 20:
raise ValueError("Truncated GLB")
magic, version, length, size, kind = struct.unpack_from("<5I", data)
if (magic, version, length, kind) != (0x46546C67, 2, len(data), 0x4E4F534A):
raise ValueError("Invalid GLB (possibly an LFS pointer)")
doc = json.loads(data[20:20 + size])
offset = 20 + size
binary_size, binary_kind = struct.unpack_from("<2I", data, offset)
if binary_kind != 0x004E4942 or offset + 8 + binary_size != len(data):
raise ValueError("Expected one embedded BIN chunk")
for asset in [*doc.get("buffers", []), *doc.get("images", [])]:
if "uri" in asset:
raise ValueError("GLBs must embed all images and buffers")
if len(doc["buffers"]) != 1 or any(k in doc for k in ["skins", "animations", "extensionsRequired"]):
raise ValueError("Expected static, self-contained body GLB")
return doc, data[offset + 8:offset + 8 + binary_size]
def identity():
return [[float(i == j) for j in range(4)] for i in range(4)]
def multiply(a, b):
return [[sum(a[i][k] * b[k][j] for k in range(4)) for j in range(4)] for i in range(4)]
def origin_matrix(origin):
result = identity()
if origin is None:
return result
xyz = list(map(float, origin.get("xyz", "0 0 0").split()))
roll, pitch, yaw = map(float, origin.get("rpy", "0 0 0").split())
cr, sr, cp, sp, cy, sy = math.cos(roll), math.sin(roll), math.cos(pitch), math.sin(pitch), math.cos(yaw), math.sin(yaw)
# URDF fixed-axis XYZ = Rz(yaw) Ry(pitch) Rx(roll).
rotation = [[cy * cp, cy * sp * sr - sy * cr, cy * sp * cr + sy * sr],
[sy * cp, sy * sp * sr + cy * cr, sy * sp * cr - cy * sr],
[-sp, cp * sr, cp * cr]]
for i in range(3):
result[i][:3] = rotation[i]
result[i][3] = xyz[i]
return result
def assemble(parts):
keys = ["nodes", "materials", "meshes", "textures", "images", "accessors", "bufferViews", "samplers"]
out = {"asset": {"version": "2.0", "generator": "microduck RP1 visual-only fixed-hand assembler"},
"scene": 0, "scenes": [{"name": "Fixed wrist and hand", "nodes": []}],
**{key: [] for key in keys}}
binary = bytearray()
# Blender's source GLBs are Y-up; the URDF matrices are Z-up.
c = [[1, 0, 0, 0], [0, 0, 1, 0], [0, -1, 0, 0], [0, 0, 0, 1]]
cinv = [[1, 0, 0, 0], [0, 0, -1, 0], [0, 1, 0, 0], [0, 0, 0, 1]]
for name, data, transform in parts:
doc, payload = read_glb(data)
if doc.get("extensionsUsed"):
raise ValueError("Unexpected extension in wrist part; review remapping")
offsets = {key: len(out[key]) for key in keys}
start = len(binary)
binary.extend(payload)
binary.extend(b"\0" * (-len(binary) % 4))
for key in keys:
for original in doc[key]:
item = copy.deepcopy(original)
if key == "bufferViews":
item["buffer"] = 0
item["byteOffset"] = item.get("byteOffset", 0) + start
elif key == "accessors":
if "sparse" in item:
raise ValueError("Sparse accessor requires explicit remapping")
if "bufferView" in item:
item["bufferView"] += offsets["bufferViews"]
elif key == "images":
item["bufferView"] += offsets["bufferViews"]
elif key == "textures":
item["source"] += offsets["images"]
if "sampler" in item:
item["sampler"] += offsets["samplers"]
elif key == "materials":
def textures(value):
if isinstance(value, dict):
for field, child in value.items():
if field.endswith("Texture") and isinstance(child, dict) and "index" in child:
child["index"] += offsets["textures"]
else:
textures(child)
elif isinstance(value, list):
for child in value:
textures(child)
textures(item)
elif key == "meshes":
for primitive in item["primitives"]:
primitive["attributes"] = {k: v + offsets["accessors"] for k, v in primitive["attributes"].items()}
for field, table in [("indices", "accessors"), ("material", "materials")]:
if field in primitive:
primitive[field] += offsets[table]
elif key == "nodes":
if "mesh" in item:
item["mesh"] += offsets["meshes"]
if "children" in item:
item["children"] = [v + offsets["nodes"] for v in item["children"]]
out[key].append(item)
matrix = multiply(multiply(c, transform), cinv)
out["scenes"][0]["nodes"].append(len(out["nodes"]))
out["nodes"].append({"name": f"source-part:{name}", "extras": {"sourcePart": name}, "matrix": [matrix[i][j] for j in range(4) for i in range(4)],
"children": [v + offsets["nodes"] for v in doc["scenes"][doc.get("scene", 0)]["nodes"]]})
out["buffers"] = [{"byteLength": len(binary)}]
encoded = json.dumps(out, separators=(",", ":")).encode()
encoded += b" " * (-len(encoded) % 4)
total = 12 + 8 + len(encoded) + 8 + len(binary)
return struct.pack("<5I", 0x46546C67, 2, total, len(encoded), 0x4E4F534A) + encoded + struct.pack("<2I", len(binary), 0x004E4942) + binary
def import_visuals(archive, urdf):
target = Path(__file__).resolve().parents[1] / "public/robots/rp1"
source_root = ET.fromstring(urdf.read_bytes())
links = {link.get("name"): link for link in source_root.findall("link")}
expected = {}
for name, link in links.items():
mesh = Path(link.find("visual/geometry/mesh").get("filename")).stem
expected[name] = f"{name}__visual_0__{mesh}.glb"
with zipfile.ZipFile(archive) as source:
if sorted(source.namelist()) != sorted(expected.values()) or len(expected) != 63:
raise ValueError("Archive must contain exactly the 63 hand-model body GLBs")
inputs = {name: source.read(file) for name, file in expected.items()}
for data in inputs.values():
read_glb(data)
model = ET.parse(target / "model.xml").getroot()
physical = [body.get("name") for body in model.findall(".//worldbody//body")]
parents = {j.find("child").get("link"): j for j in source_root.findall("joint")}
output, mappings = {}, {}
for body in physical:
if not body.endswith("_wrist_link"):
name = expected[body]
output[name] = inputs[body]
mappings[body] = {"path": f"visuals/{name}", "parts": [body]}
continue
side = body.split("_")[0]
anchor = f"{side}_wrist_roll_link"
prefix = "lh_" if side == "left" else "rh_"
names = [name for name in links if name.startswith(f"{side}_wrist_") or name.startswith(prefix)]
parts = []
for name in names:
relative = identity()
node = name
while node != anchor:
joint = parents[node]
# The extra two wrist axes are visual-only at neutral; hand
# joints come from the upstream fixed-hand URDF.
if joint.get("type") != "fixed" and "wrist" not in joint.get("name", ""):
raise ValueError("Unexpected moving hand joint")
relative = multiply(origin_matrix(joint.find("origin")), relative)
node = joint.find("parent").get("link")
parts.append((name, inputs[name], relative))
file = f"{body}__fixed_hand.glb"
output[file] = assemble(parts)
mappings[body] = {"path": f"visuals/{file}", "parts": names}
used = [name for mapping in mappings.values() for name in mapping["parts"]]
if sorted(used) != sorted(links) or len(physical) != 25:
raise ValueError("Every source part must map exactly once to 25 physical bodies")
for data in output.values():
read_glb(data)
# Commit only after all inputs and assembled outputs validate. No ZIP extraction.
(target / "visuals").mkdir(exist_ok=True)
(target / "visuals/rp1_hand_fixed.urdf").write_bytes(urdf.read_bytes())
records = [{"file": "visuals/rp1_hand_fixed.urdf", "sha256": sha(urdf.read_bytes())}]
for name, data in output.items():
(target / "visuals" / name).write_bytes(data)
records.append({"file": f"visuals/{name}", "sha256": sha(data)})
config = json.loads((target / "robot.json").read_text())
config["render"]["body_visuals"] = [{"body": body, "path": mappings[body]["path"],
"quaternion": [math.sqrt(0.5), 0, 0, math.sqrt(0.5)]} for body in physical]
(target / "robot.json").write_text(json.dumps(config, indent=2) + "\n")
manifest = {"source": f"User-provided {archive.name}", "archive_sha256": sha(archive.read_bytes()),
"license": "Not supplied in the archive; local evaluation only. Confirm redistribution rights.",
"coordinate_conversion": "Source GLBs are Blender Y-up; attachments rotate +90 degrees about X into MuJoCo body Z-up.",
"assembly": "25 physical body attachments; 20 source parts per wrist (neutral extra wrist axes and fixed fingers). No physical joints added.",
"input_files": [{"file": expected[name], "sha256": sha(data)} for name, data in inputs.items()],
"body_mapping": mappings, "files": records}
(target / "visual-source-manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
print(f"Imported {len(inputs)} source GLBs into {len(physical)} body attachments; physics/policies untouched")
if __name__ == "__main__":
import_visuals(Path(sys.argv[1]), Path(sys.argv[2]))
+151
View File
@@ -0,0 +1,151 @@
import * as THREE from "three";
const TYPES = ["box", "sphere", "cylinder", "capsule"];
const COLORS = [
0xffb52e, 0x81cbb7, 0xb4a4d4, 0x83bfff, 0xf2efe8, 0xff7a2f, 0xf09caf,
];
// One layout per runtime, shared by both locomotion models. A shuffled,
// jittered lattice gives random placement without overlaps or retry loops.
export function generateInteractiveProps({
halfExtent,
spawn,
avoid = [],
random = Math.random,
}) {
const count = 5 + Math.floor(random() * 3);
const candidates = [];
const limit = halfExtent - 0.2;
for (
let x = Math.max(-limit, spawn[0] - 0.4);
x <= Math.min(limit, spawn[0] + 1.8);
x += 0.4
) {
for (let y = -Math.min(limit, 1.2); y <= Math.min(limit, 1.2); y += 0.4) {
const px = x + (random() - 0.5) * 0.06;
const py = y + (random() - 0.5) * 0.06;
if (
[spawn, ...avoid].every(([x, y]) => Math.hypot(px - x, py - y) > 0.34)
)
candidates.push([px, py]);
}
}
for (let i = candidates.length - 1; i > 0; i--) {
const j = Math.floor(random() * (i + 1));
[candidates[i], candidates[j]] = [candidates[j], candidates[i]];
}
// Keep at least one toy within immediate walking / kicking reach.
const near = candidates.findIndex(
([x, y]) =>
x > spawn[0] + 0.3 &&
x < spawn[0] + 0.85 &&
Math.abs(y - spawn[1]) < 0.42,
);
if (near > 0)
[candidates[0], candidates[near]] = [candidates[near], candidates[0]];
if (candidates.length < count)
throw new Error("Arena too small for interactive props");
const offset = Math.floor(random() * TYPES.length);
return candidates.slice(0, count).map(([x, y], i) => {
const type = TYPES[(i + offset) % TYPES.length];
const r = 0.045 + random() * 0.015;
const h = 0.045 + random() * 0.02;
const size =
type === "box" ? [r, r * 0.85, h] : type === "sphere" ? [r] : [r, h];
const bottom = type === "sphere" ? r : type === "capsule" ? r + h : h;
const yaw = random() * Math.PI * 2;
return {
name: `interactive_${i}`,
type,
size,
pos: [x, y, bottom + 0.006],
quat: [Math.cos(yaw / 2), 0, 0, Math.sin(yaw / 2)],
mass: 0.04 + random() * 0.04,
color: COLORS[i % COLORS.length],
};
});
}
export function resolvePropAddresses(mujoco, model, specs) {
return specs.map((spec) => {
const jointId = mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_JOINT.value,
`${spec.name}_joint`,
);
const bodyId = mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_BODY.value,
spec.name,
);
return {
bodyId,
qAdr: model.jnt_qposadr[jointId],
dofAdr: model.jnt_dofadr[jointId],
mass: model.body_mass[bodyId],
};
});
}
export function resetProp(data, address, spec, parked = false, index = 0) {
const q = data.qpos,
v = data.qvel;
q.set(parked ? [50 + index, 0, spec.pos[2]] : spec.pos, address.qAdr);
q.set(spec.quat, address.qAdr + 3);
v.fill(0, address.dofAdr, address.dofAdr + 6);
}
export function createInteractivePropVisuals(specs) {
const root = new THREE.Group();
root.name = "interactive-props";
root.rotation.x = -Math.PI / 2; // MJCF Z-up -> Three Y-up
const meshes = specs.map((spec, index) => {
const [r, h, z] = spec.size;
let geometry;
if (spec.type === "box")
geometry = new THREE.BoxGeometry(2 * r, 2 * h, 2 * z);
else if (spec.type === "sphere")
geometry = new THREE.SphereGeometry(r, 24, 16);
else {
geometry =
spec.type === "cylinder"
? new THREE.CylinderGeometry(r, r, 2 * h, 24)
: new THREE.CapsuleGeometry(r, 2 * h, 8, 16);
geometry.rotateX(Math.PI / 2);
}
const mesh = new THREE.Mesh(
geometry,
new THREE.MeshStandardMaterial({
color: spec.color,
roughness: 0.38,
metalness: 0.08,
}),
);
mesh.name = spec.name;
mesh.userData.interactiveIndex = index;
mesh.position.fromArray(spec.pos);
mesh.quaternion.set(spec.quat[1], spec.quat[2], spec.quat[3], spec.quat[0]);
root.add(mesh);
return mesh;
});
return {
root,
meshes,
dispose() {
root.removeFromParent();
meshes.forEach((mesh) => {
mesh.geometry.dispose();
mesh.material.dispose();
});
},
sync(data, addresses, visible = true) {
root.visible = visible;
const q = data.qpos;
meshes.forEach((mesh, i) => {
const a = addresses[i].qAdr;
mesh.position.set(q[a], q[a + 1], q[a + 2]);
mesh.quaternion.set(q[a + 4], q[a + 5], q[a + 6], q[a + 3]);
});
},
};
}
+11 -18
View File
@@ -1,9 +1,8 @@
import { propColliders } from "../../props.js";
import { appendInteractivePhysics } from "./session-environment.js";
import { arenaWalls } from "../../../platform/worlds/arena.js";
import {
ARENA_HALF,
BALL_PARK_POS,
BALL_RADIUS,
RELIEF_GRID,
RELIEF_HMAX,
RELIEF_SINK,
@@ -17,6 +16,8 @@ export function composeArenaPhysics(
jointNames,
defaultPose,
spawn,
interactiveProps = [],
floor,
boundary = {
half_extent: ARENA_HALF,
wall_height: 0.25,
@@ -30,7 +31,11 @@ export function composeArenaPhysics(
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v);
return e;
};
root.appendChild(el("option", { timestep: String(timestep) }));
// Adding free toys can cross MuJoCo's automatic sparse-Jacobian cutoff
// (nv >= 60). Keep the original small-robot dense path in both modes.
root.appendChild(
el("option", { timestep: String(timestep), jacobian: "dense" }),
);
const body = doc.querySelector("worldbody");
body.appendChild(
el("geom", {
@@ -38,6 +43,7 @@ export function composeArenaPhysics(
type: "plane",
size: "0 0 0.05",
pos: "0 0 0",
...(floor?.friction ? { friction: floor.friction.join(" ") } : {}),
}),
);
for (const wall of arenaWalls(boundary))
@@ -54,20 +60,7 @@ export function composeArenaPhysics(
if (c.euler) attrs.euler = c.euler;
body.appendChild(el("geom", attrs));
}
const ball = el("body", { name: "ball", pos: BALL_PARK_POS });
ball.appendChild(el("freejoint", { name: "ball_freejoint" }));
ball.appendChild(
el("geom", {
name: "ball_geom",
type: "sphere",
size: String(BALL_RADIUS),
mass: "0.03",
friction: "0.4 0.01 0.003",
solref: "0.03 0.4",
condim: "6",
}),
);
body.appendChild(ball);
appendInteractivePhysics(doc, interactiveProps, { group: 0 });
doc.querySelector("asset").appendChild(
el("hfield", {
name: "terrain",
@@ -93,7 +86,7 @@ export function composeArenaPhysics(
keyframe.appendChild(
el("key", {
name: "STAND",
qpos: `${qposFree} ${qposJoints} ${BALL_PARK_POS} 1 0 0 0`,
qpos: `${qposFree} ${qposJoints} ${interactiveProps.map((p) => [...p.pos, ...p.quat].join(" ")).join(" ")}`,
ctrl: Array.from(defaultPose).join(" "),
}),
);
+15
View File
@@ -0,0 +1,15 @@
import * as THREE from "three";
import { VARIANT_SWATCH_HEX } from "../../variants.js";
// The existing four HUD swatches now tint both the floor and enclosure.
// Keep minor lines quieter than section lines, without touching reveal/fade.
export function setArenaLineColor(materials, name) {
const hex = VARIANT_SWATCH_HEX[name];
if (!hex) return false;
const color = new THREE.Color(hex);
for (const material of materials) {
material.uniforms.uSectionColor.value.copy(color);
material.uniforms.uCellColor.value.copy(color).multiplyScalar(0.55);
}
return true;
}
@@ -1,5 +1,6 @@
import * as THREE from "three";
import { makeInfiniteGrid, makeArenaWalls } from "./render.js";
import { setArenaLineColor } from "./line-color.js";
import { arenaWalls } from "../../../platform/worlds/arena.js";
// Shared appearance; physical collision boxes come only from the World composer.
export function createArenaFloorPresentation({ sim, rig, world }) {
@@ -43,6 +44,8 @@ export function createArenaFloorPresentation({ sim, rig, world }) {
root,
grid,
walls: wallMeshes,
setLineColor: (name) =>
setArenaLineColor([grid.material, ...wallMats], name),
update(position) {
grid.position.set(position.x, 0, position.z);
grid.material.uniforms.uFocus.value.copy(position);
@@ -0,0 +1,75 @@
import { generateInteractiveProps } from "../../interactive-props.js";
// One random arrangement for the whole browser session, not one per robot.
// Positions/sizes share the arena's existing robot-relative scale. Reserve
// both bundled spawn anchors so changing robots never starts inside a toy.
export function createSessionEnvironment(random = Math.random) {
let template;
return {
propsFor(world) {
if (world.presentation !== "arena-grid" || !world.boundary) return [];
template ??= generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
avoid: [[0, 0]],
random,
});
const scale = world.boundary.half_extent / 2.25;
return template.map((spec) => ({
...spec,
pos: spec.pos.map((v) => v * scale),
size: spec.size.map((v) => v * scale),
quat: [...spec.quat],
// Keep larger robots' toys readily pushable, rather than scaling
// mass cubically into heavy obstacles for the humanoid.
mass: spec.mass * scale * scale,
}));
},
};
}
let seed = 123456789;
const testRandom = () =>
(seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
const session = createSessionEnvironment(
typeof location !== "undefined" &&
new URLSearchParams(location.search).get("test") === "1"
? testRandom
: Math.random,
);
export const sharedInteractiveProps = (world) => session.propsFor(world);
export function appendInteractivePhysics(doc, specs, { group = 5 } = {}) {
const body = doc.querySelector("worldbody");
const el = (tag, attrs) => {
const node = doc.createElement(tag);
for (const [key, value] of Object.entries(attrs))
node.setAttribute(key, value);
return node;
};
for (const spec of specs) {
if (doc.querySelector(`[name="${spec.name}"]`))
throw new Error(`World name collision: ${spec.name}`);
const prop = el("body", {
name: spec.name,
pos: spec.pos.join(" "),
quat: spec.quat.join(" "),
});
prop.appendChild(el("freejoint", { name: `${spec.name}_joint` }));
prop.appendChild(
el("geom", {
name: `${spec.name}_geom`,
type: spec.type,
size: spec.size.join(" "),
mass: String(spec.mass),
friction: spec.friction ?? "0.5 0.01 0.003",
solref: spec.solref ?? "0.02 1",
condim: "6",
contype: "1",
conaffinity: "1",
group: String(group),
}),
);
body.appendChild(prop);
}
}
+158
View File
@@ -0,0 +1,158 @@
import * as THREE from "three";
// Same mass-scaled spring perturbation as Microduck, applied every physics
// substep. Shared by generic robots for both their body and the arena toys.
export class PhysicsDrag {
constructor({
sim,
camera,
controls,
canvas,
objects,
addressFor,
enabled,
halfExtent,
scale = 1,
}) {
Object.assign(this, {
sim,
camera,
controls,
canvas,
objects,
addressFor,
enabled,
halfExtent,
scale,
});
this.ray = new THREE.Raycaster();
this.ndc = new THREE.Vector2();
this.plane = new THREE.Plane();
this.point = new THREE.Vector3();
this.direction = new THREE.Vector3();
this.listeners = [];
const listen = (type, fn, capture = false) => {
window.addEventListener(type, fn, capture);
this.listeners.push(() => window.removeEventListener(type, fn, capture));
};
listen(
"pointerdown",
(e) => {
if (
e.target !== canvas ||
e.pointerType !== "mouse" ||
e.button !== 0 ||
!enabled()
)
return;
this.setRay(e);
const hit = this.pick();
if (!hit) return;
const address = addressFor(hit.object);
if (!address) return;
this.grab = { ...address, target: [0, 0, 0] };
camera.getWorldDirection(this.direction);
this.plane.setFromNormalAndCoplanarPoint(
this.direction.negate(),
hit.point,
);
this.updateTarget();
controls.enabled = false;
canvas.style.cursor = "grabbing";
this.pointerId = e.pointerId;
try {
canvas.setPointerCapture(e.pointerId);
} catch {
/* unavailable */
}
},
true,
);
listen("pointermove", (e) => {
if (this.grab) {
this.setRay(e);
this.updateTarget();
return;
}
if (
e.target !== canvas ||
e.pointerType !== "mouse" ||
e.buttons ||
!enabled()
)
return;
if (performance.now() - (this.hoverAt ?? 0) < 80) return;
this.hoverAt = performance.now();
this.setRay(e);
canvas.style.cursor = this.pick() ? "grab" : "";
});
for (const type of ["pointerup", "pointercancel", "blur"])
listen(type, () => this.release());
canvas.addEventListener(
"lostpointercapture",
(this.lostCapture = () => this.release()),
);
}
setRay(e) {
const rect = this.canvas.getBoundingClientRect();
this.ndc.set(
((e.clientX - rect.left) / rect.width) * 2 - 1,
-((e.clientY - rect.top) / rect.height) * 2 + 1,
);
this.ray.setFromCamera(this.ndc, this.camera);
}
pick() {
return this.ray.intersectObjects(this.objects(), true).find((hit) => {
for (let o = hit.object; o; o = o.parent) if (!o.visible) return false;
return !!this.addressFor(hit.object);
});
}
updateTarget() {
if (!this.ray.ray.intersectPlane(this.plane, this.point)) return;
const lim = (this.halfExtent ?? 100) - 0.05 * this.scale;
this.grab.target = [
Math.max(-lim, Math.min(lim, this.point.x)),
Math.max(-lim, Math.min(lim, -this.point.z)),
Math.max(0.02 * this.scale, Math.min(0.45 * this.scale, this.point.y)),
];
}
apply() {
const g = this.grab;
if (!g) return;
const { data } = this.sim,
force = [];
for (let i = 0; i < 3; i++)
force[i] =
100 * (g.target[i] - data.qpos[g.qAdr + i]) -
10 * data.qvel[g.dofAdr + i];
const norm = Math.hypot(...force),
clamp = Math.min(1, 200 / (norm || 1));
for (let i = 0; i < 3; i++)
data.xfrc_applied[g.bodyId * 6 + i] = g.mass * force[i] * clamp;
}
release() {
if (this.grab)
this.sim.data.xfrc_applied.fill(
0,
this.grab.bodyId * 6,
this.grab.bodyId * 6 + 6,
);
this.grab = null;
this.controls.enabled = true;
this.canvas.style.cursor = "";
const id = this.pointerId;
this.pointerId = undefined;
if (id !== undefined) {
try {
this.canvas.releasePointerCapture(id);
} catch {
/* already released */
}
}
}
dispose() {
this.release();
this.listeners.forEach((remove) => remove());
this.canvas.removeEventListener("lostpointercapture", this.lostCapture);
}
}
+1
View File
@@ -51,6 +51,7 @@ export class BodyVisuals {
visuals.scenes.push(...gltf.scenes);
const frame = new THREE.Group();
frame.name = `body-visual:${spec.body}`;
frame.userData.bodyId = ids[i];
gltf.scene.position.fromArray(spec.position ?? [0, 0, 0]);
gltf.scene.quaternion.fromArray(spec.quaternion ?? [0, 0, 0, 1]);
frame.add(gltf.scene);
+75 -1
View File
@@ -16,6 +16,8 @@ import { SkillManager } from "../skills/SkillManager.js";
import { InputRouter } from "../input/InputRouter.js";
import { RigRenderer } from "../render/RigRenderer.js";
import { ChaseCamera } from "../render/ChaseCamera.js";
import { PhysicsDrag } from "../input/PhysicsDrag.js";
import { createInteractivePropVisuals } from "../../game/interactive-props.js";
export class RobotRuntime {
static async load(options) {
@@ -175,7 +177,14 @@ export class RobotRuntime {
},
});
this.scene.add(this.rig.root);
if (this.worldPresentation) this.scene.add(this.worldPresentation.root);
this.lineColor = this.environmentSettings?.lineColor ?? "classic";
if (this.worldPresentation) {
this.scene.add(this.worldPresentation.root);
this.worldPresentation.setLineColor(this.lineColor);
}
this.propsVisuals = createInteractivePropVisuals(this.sim.interactiveSpecs);
this.scene.add(this.propsVisuals.root);
this.syncProps();
this.follow = new THREE.Vector3();
}
activate({ paused = false } = {}) {
@@ -192,6 +201,35 @@ export class RobotRuntime {
this.config.render.camera,
);
this.restoreCamera();
this.drag = new PhysicsDrag({
sim: this.sim,
camera: this.camera,
controls: this.controls,
canvas: this.renderer.domElement,
objects: () => [this.rig.root, this.propsVisuals.root],
enabled: () => this.active && this.scheduler.running && !this.disposed,
halfExtent: this.world.boundary?.half_extent,
scale: this.world.scale ?? 1,
addressFor: (object) => {
const index = object.userData.interactiveIndex;
if (Number.isInteger(index)) return this.sim.propAddresses[index];
const id = object.userData.geomId;
let visualBody = false;
for (let node = object; node; node = node.parent)
if (node.userData.bodyId > 0) visualBody = true;
if (
!visualBody &&
(id === undefined || this.sim.model.geom_bodyid[id] === 0)
)
return null;
return {
bodyId: this.sim.root,
qAdr: this.sim.rootQpos,
dofAdr: this.sim.rootDof,
mass: this.sim.model.body_subtreemass[this.sim.root],
};
},
});
this.input.init();
if (!paused) this.resume();
else this.onStatus({ paused: true });
@@ -213,6 +251,11 @@ export class RobotRuntime {
}
status() {
return {
lineColor: this.lineColor,
runtimeCapabilities: {
environmentControls: !!this.worldPresentation,
physicalDrag: true,
},
policyStatus: this.policyStatus(),
actuationLabel:
this.config.actuation.type === "joint_pd"
@@ -220,6 +263,18 @@ export class RobotRuntime {
: "",
};
}
setLineColor(name) {
if (!this.worldPresentation?.setLineColor(name)) return;
this.lineColor = name;
this.onStatus({ lineColor: name });
}
syncProps() {
this.propsVisuals.sync(
this.sim.data,
this.sim.propAddresses,
!this.sim.propsParked,
);
}
policyStatus() {
return this.policyState.episodes.size
? `Skill: ${this.policyState.active} · ${this.policyState.state}`
@@ -241,6 +296,7 @@ export class RobotRuntime {
if (!paused) this.resume();
}
async pause() {
this.drag?.release();
this.input.clear();
await this.scheduler.pause();
this.onStatus({ paused: true });
@@ -260,6 +316,7 @@ export class RobotRuntime {
async #reset() {
const running = this.scheduler.running;
await this.scheduler.pause();
this.drag?.release();
this.input.clear();
this.ctx.command.fill(0);
for (const policy of this.policies.values()) policy.reset();
@@ -278,6 +335,7 @@ export class RobotRuntime {
this.onStatus({ fallen: false });
this.publishPolicyStatus();
this.rig.sync();
this.syncProps();
if (running) this.scheduler.start();
}
triggerAction(action) {
@@ -313,8 +371,10 @@ export class RobotRuntime {
}
for (let i = 0; i < this.config.sim.decimation; i++) {
this.actuation.apply();
this.drag?.apply();
this.sim.step();
}
this.sim.restoreEscapedProps(this.world.boundary?.half_extent);
const q = this.sim.data.qpos,
a = this.sim.rootQpos,
quat = this.sim.data.xquat,
@@ -356,6 +416,7 @@ export class RobotRuntime {
frame() {
if (this.disposed) return;
this.rig.sync();
this.syncProps();
if (this.chaseCamera) {
const yaw = this.cameraPose();
this.worldPresentation?.update(this.follow);
@@ -395,6 +456,17 @@ export class RobotRuntime {
},
scene: this.scene,
world: this.world,
get interactiveProps() {
return self.sim.interactiveSpecs.map((spec, i) => ({
...spec,
...self.sim.propAddresses[i],
}));
},
setPropsParked: (parked) => {
this.drag?.release();
this.sim.setPropsParked(parked);
this.syncProps();
},
command: this.input.command,
get lastAction() {
return self.policies.get(self.activePolicy).ctx.lastAction;
@@ -444,6 +516,8 @@ export class RobotRuntime {
this.active = false;
await this.scheduler.dispose();
this.input.dispose();
this.drag?.dispose();
this.propsVisuals.dispose();
this.chaseCamera?.dispose();
this.controls?.dispose();
this.worldPresentation?.dispose();
@@ -82,6 +82,9 @@ export class RuntimeManager {
camera,
renderer: this.renderer,
autoStart: false,
environmentSettings: {
lineColor: this.getState().lineColor ?? "classic",
},
publish,
onStatus: publish,
onTelemetry: (telemetry) => publish({ telemetry }),
+4 -1
View File
@@ -1,5 +1,6 @@
import { arenaWalls } from "../worlds/arena.js";
export function composeScene(xml, config, world) {
import { appendInteractivePhysics } from "../../game/worlds/arena/session-environment.js";
export function composeScene(xml, config, world, interactiveProps = []) {
const doc = new DOMParser().parseFromString(xml, "text/xml");
if (
doc.querySelector("parsererror") ||
@@ -9,6 +10,7 @@ export function composeScene(xml, config, world) {
const option =
doc.querySelector("mujoco > option") ?? doc.createElement("option");
option.setAttribute("timestep", String(config.sim.timestep));
if (interactiveProps.length) option.setAttribute("jacobian", "dense");
if (!option.parentNode) doc.documentElement.append(option);
if (world.type !== "flat")
throw new Error(`World ${world.id}: generic composer requires flat World`);
@@ -44,5 +46,6 @@ export function composeScene(xml, config, world) {
worldbody.append(geom);
}
}
appendInteractivePhysics(doc, interactiveProps);
return new XMLSerializer().serializeToString(doc);
}
+47 -3
View File
@@ -1,5 +1,10 @@
import { fetchChecked } from "../loader/PackageLoader.js";
import { composeScene } from "./SceneComposer.js";
import { sharedInteractiveProps } from "../../game/worlds/arena/session-environment.js";
import {
resolvePropAddresses,
resetProp,
} from "../../game/interactive-props.js";
export function resolveAddresses(mujoco, model, names) {
const joints = new Map();
for (const name of names) {
@@ -42,14 +47,23 @@ export class SimCore {
pkg.config.assets.forEach((a, i) =>
vfs.addBuffer(a.vfs, new Uint8Array(buffers[i])),
);
const interactiveSpecs = sharedInteractiveProps(world);
const xml = composeScene(
await fetchChecked(pkg.asset(pkg.config.model)),
pkg.config,
world,
interactiveSpecs,
);
model = mujoco.MjModel.from_xml_string(xml, vfs);
data = new mujoco.MjData(model);
return new SimCore(mujoco, model, data, vfs, pkg.config);
return new SimCore(
mujoco,
model,
data,
vfs,
pkg.config,
interactiveSpecs,
);
} catch (error) {
data?.delete();
model?.delete();
@@ -57,8 +71,10 @@ export class SimCore {
throw error;
}
}
constructor(mujoco, model, data, vfs, config) {
Object.assign(this, { mujoco, model, data, vfs, config });
constructor(mujoco, model, data, vfs, config, interactiveSpecs = []) {
Object.assign(this, { mujoco, model, data, vfs, config, interactiveSpecs });
this.propAddresses = resolvePropAddresses(mujoco, model, interactiveSpecs);
this.propsParked = false;
const names = [...new Set(Object.values(config.joints.groups).flat())];
this.joints = resolveAddresses(mujoco, model, names);
for (const [name, j] of this.joints) {
@@ -119,8 +135,36 @@ export class SimCore {
for (const [name, joint] of this.joints)
data.qpos[joint.qpos] =
config.base.initial?.[name] ?? defaults.get(name) ?? 0;
this.resetProps();
mujoco.mj_forward(model, data);
}
resetProps() {
this.interactiveSpecs.forEach((spec, i) =>
resetProp(this.data, this.propAddresses[i], spec, this.propsParked, i),
);
}
setPropsParked(parked) {
this.propsParked = parked;
this.resetProps();
this.mujoco.mj_forward(this.model, this.data);
}
restoreEscapedProps(halfExtent) {
if (this.propsParked || !halfExtent) return;
let restored = false;
this.interactiveSpecs.forEach((spec, i) => {
const a = this.propAddresses[i].qAdr,
q = this.data.qpos;
if (
Math.abs(q[a]) > halfExtent + 0.1 ||
Math.abs(q[a + 1]) > halfExtent + 0.1 ||
q[a + 2] < -0.2
) {
resetProp(this.data, this.propAddresses[i], spec);
restored = true;
}
});
if (restored) this.mujoco.mj_forward(this.model, this.data);
}
step() {
this.mujoco.mj_step(this.model, this.data);
}
+107 -199
View File
@@ -38,8 +38,6 @@ import { ObservationPipeline } from "../../platform/policy/ObservationPipeline.j
import { ActionPipeline } from "../../platform/policy/ActionPipeline.js";
import { NativeActuation } from "../../platform/sim/actuation/JointPdActuation.js";
import {
BALL_RADIUS,
BALL_PARK_POS,
ARENA_HALF as LEGACY_ARENA_HALF,
SPAWN_X,
SPAWN_Y,
@@ -91,7 +89,13 @@ import {
playLineBlip,
setRumble,
} from "../../game/audio.js";
import { createBallActor } from "../../game/ball-actor.js";
import {
resolvePropAddresses,
resetProp,
createInteractivePropVisuals,
} from "../../game/interactive-props.js";
import { setArenaLineColor } from "../../game/worlds/arena/line-color.js";
import { sharedInteractiveProps } from "../../game/worlds/arena/session-environment.js";
import {
loadCustomPolicy as loadCustomPolicyModule,
scriptCommandAt,
@@ -100,7 +104,6 @@ import {
makeInfiniteGrid,
makeArenaWalls,
} from "../../game/worlds/arena/render.js";
import { createBallVisual } from "../../game/ball-visual.js";
import {
useGame,
gameApi,
@@ -133,6 +136,7 @@ export async function bootGame({
world,
autoStart = true,
publish,
environmentSettings,
}) {
const scope = new Disposables();
try {
@@ -149,6 +153,7 @@ export async function bootGame({
world,
autoStart,
publish,
environmentSettings,
});
} catch (err) {
await scope.dispose().catch(console.error);
@@ -167,6 +172,7 @@ async function boot({
world,
autoStart,
publish,
environmentSettings,
}) {
const robot = pkg.config;
const boundary = world.boundary ?? {
@@ -175,6 +181,14 @@ async function boot({
wall_thickness: 0.05,
};
const ARENA_HALF = boundary.half_extent;
const params = new URLSearchParams(location.search);
const testMode = params.get("test") === "1";
const legacyOracle = testMode && params.get("oracle") === "legacy";
const interactiveSpecs = legacyOracle
? (await import("./test-oracle.js")).originalPhysicsProbe()
: sharedInteractiveProps(world);
let propsParked = false;
let propVisuals = null;
const rollerPkg = await loadPackage(
`${import.meta.env.BASE_URL}robots/microduck-rollers/robot.json`,
);
@@ -214,7 +228,6 @@ async function boot({
const bootHalt = (detail) => {
if (!publish) globalBootHalt(detail);
};
const testMode = new URLSearchParams(location.search).get("test") === "1";
const clock = new SimulationClock();
const listen = (target, event, callback, options) =>
scope.listen(target, event, callback, options);
@@ -295,6 +308,8 @@ async function boot({
jointNames: JOINT_NAMES,
defaultPose: DEFAULT_POSE,
spawn: robot.base.spawn,
interactiveProps: interactiveSpecs,
floor: legacyOracle ? undefined : world.floor,
boundary,
});
}
@@ -344,8 +359,10 @@ async function boot({
doneMeshes(`${meshFiles.length} FILES`);
const sessions = {};
// Always boot on the classic (orange) colourway; the quickbar re-skins live.
// The robot retains its palette; the quickbar now tints arena lines.
let currentVariant = DEFAULT_VARIANT;
let lineColor =
environmentSettings?.lineColor ?? store().lineColor ?? DEFAULT_VARIANT;
const rigPromise = (async () => {
const doneRig = bootLine("RENDER RIG");
try {
@@ -566,8 +583,7 @@ async function boot({
mujoco.mjtObj.mjOBJ_KEY.value,
"STAND",
),
ballQposAdr: model.jnt_qposadr[jointId("ball_freejoint")],
ballDofAdr: model.jnt_dofadr[jointId("ball_freejoint")],
propAddresses: resolvePropAddresses(mujoco, model, interactiveSpecs),
// Foot bodies for the footstep audio heuristic (-1 when a variant
// has no ankles, e.g. if a future model renames them).
ankleIds: ["ankle_left", "ankle_right"].map((n) =>
@@ -594,8 +610,7 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
} = resolveAddrs(model, k);
@@ -651,11 +666,10 @@ async function boot({
// Right-stick camera state, read by the telemetry before the camera-orbit
// section below has evaluated.
let padOrbitLive = false;
// Robot input gate: twist commands, mode changes, rolls, kicks and ball
// spawns all stay inert until the entrance sequence has fully played out.
// Robot input gate: twist commands, mode changes, rolls and kicks
// stay inert until the entrance sequence has fully played out.
let inputLocked = true;
let ceremony = null;
let ball = null;
let stickers = null; // comic popups, currently disabled
let mode = "walk"; // "walk" | "sitstand" | "roll" | "kickL" | "kickR" | "crouch" | "groundpick"
@@ -678,9 +692,6 @@ async function boot({
const HEAD_SIGNS = new Float32Array([1, -1, 1, -1]);
const headTarget = new Float32Array(4);
const headSmooth = new Float32Array(4);
// Local-only kickable ball: false while parked at the keyframe spot
// (mesh hidden), true once popped in front of the duck.
let ballActive = false;
// The twist the policy actually receives. Mid-roll every movement input
// is ignored (zero twist) until the roll hands back to walk on its own.
@@ -823,74 +834,26 @@ async function boot({
headTarget.fill(0);
headSmooth.fill(0);
mujoco.mj_resetDataKeyframe(model, data, standKeyId);
interactiveSpecs.forEach((spec, i) =>
resetProp(data, propAddresses[i], spec, propsParked, i),
);
mujoco.mj_forward(model, data);
lastAction.fill(0);
sitFlag = 0;
// Park the ball in physics immediately; if it was on screen, the
// reverse scan peels it away at its last pose. A queued B-respawn is
// cancelled: a reset means no ball.
ball?.despawn({ cancelQueued: true, parkPhysics: parkBallPhysics });
ballActive = false;
propVisuals?.sync(data, propAddresses, !propsParked);
syncButtons();
if (!testMode && ceremony) {
startCameraReset();
spawnBall({ fromQueue: true });
}
if (!testMode && ceremony) startCameraReset();
}
resetSim();
function parkBallPhysics() {
const qpos = data.qpos,
qvel = data.qvel;
qpos[ballQposAdr] = 50;
qpos[ballQposAdr + 1] = 0;
qpos[ballQposAdr + 2] = BALL_RADIUS;
qpos[ballQposAdr + 3] = 1;
qpos[ballQposAdr + 4] = 0;
qpos[ballQposAdr + 5] = 0;
qpos[ballQposAdr + 6] = 0;
for (let i = 0; i < 6; i++) qvel[ballDofAdr + i] = 0;
mujoco.mj_forward(model, data);
ballActive = false;
}
// Pop / respawn the ball ~0.35 m in front of the duck, with a small
// random heading + distance jitter. If the ball is already on screen,
// peel it away first (reverse scan) and pop the new one when that
// finishes - same appear/disappear pair as the duck's wireframe ceremony.
function spawnBall(opts = {}) {
if (inputLocked && !opts.fromQueue) return;
if (!ball) return;
if (ball.visual !== "hidden") {
ball.queueRespawn();
ball.despawn({ parkPhysics: parkBallPhysics });
return;
}
const qpos = data.qpos,
qvel = data.qvel;
const yaw = Math.atan2(
2 * (qpos[3] * qpos[6] + qpos[4] * qpos[5]),
1 - 2 * (qpos[5] * qpos[5] + qpos[6] * qpos[6]),
function setPropsParked(parked) {
endGrabHook();
propsParked = parked;
interactiveSpecs.forEach((spec, i) =>
resetProp(data, propAddresses[i], spec, parked, i),
);
const heading = yaw + (Math.random() - 0.5) * 0.7;
const dist = 0.35 + (Math.random() - 0.5) * 0.1;
const lim = ARENA_HALF - BALL_RADIUS - 0.05;
const clamp = (v) => Math.min(lim, Math.max(-lim, v));
qpos[ballQposAdr] = clamp(qpos[0] + Math.cos(heading) * dist);
qpos[ballQposAdr + 1] = clamp(qpos[1] + Math.sin(heading) * dist);
qpos[ballQposAdr + 2] = BALL_RADIUS + 0.02;
qpos[ballQposAdr + 3] = 1;
qpos[ballQposAdr + 4] = 0;
qpos[ballQposAdr + 5] = 0;
qpos[ballQposAdr + 6] = 0;
for (let i = 0; i < 6; i++) qvel[ballDofAdr + i] = 0;
mujoco.mj_forward(model, data);
ballActive = true;
// Snap the mesh to the new pose BEFORE the scan starts: the FX
// recomputes its bbox from the live mesh.
ball.poseFromQpos(qpos, ballQposAdr);
ball.appear();
stickers?.pop("spawn");
propVisuals?.sync(data, propAddresses, !parked);
}
// ── Observation ─────────────────────────────────────────────────────
@@ -1015,14 +978,13 @@ async function boot({
return null;
}
// ── Sim-driven audio (footsteps / roller rumble / ball impacts) ──────
// ── Sim-driven audio (footsteps / roller rumble / body bumps) ───────
// Footsteps use a per-foot height heuristic instead of MuJoCo contacts
// (the WASM bindings expose no contact array): each ankle's height
// RELATIVE to the lower ankle (the planted foot approximates the local
// ground, so the measure self-calibrates on the relief terrain), with
// lift/contact hysteresis and a per-foot debounce. Tap gain scales with
// the landing speed. Rollers get a speed-following rumble loop instead,
// and the ball thumps on velocity deltas between control steps.
// the landing speed. Rollers get a speed-following rumble loop instead.
const STEP_LIFT = 0.012; // m above the planted foot = foot in swing
const STEP_CONTACT = 0.005; // m: dropping below this while in swing = step
const STEP_DEBOUNCE_MS = 130;
@@ -1031,18 +993,11 @@ async function boot({
{ air: false, prevZ: 0, lastAt: 0 },
];
const duckEmitter = createEmitter({ refDistance: 0.5 });
const ballEmitter = createEmitter({ refDistance: 0.5 });
scope.add(() => {
duckEmitter.node.disconnect();
ballEmitter.node.disconnect();
});
const ballPrevV = [0, 0, 0];
let ballPrevValid = false;
let ballThumpAt = 0;
scope.add(() => duckEmitter.node.disconnect());
// Body bumps: trunk velocity deltas gated on obstacle proximity (walls
// or prop collider footprints) so the walking gait's own accelerations
// (comparable in magnitude to a slow wall hit) never false-trigger.
// Duller than the ball thumps: same samples pitched way down.
// Low-pitched impact samples for body bumps.
const BUMP_DV = 0.35; // m/s per control step
const BUMP_DEBOUNCE_MS = 300; // rubbing a wall must not machine-gun
const BUMP_WALL_PAD = 0.16; // trunk half-width-ish reach to a wall
@@ -1126,7 +1081,7 @@ async function boot({
const u = Math.min(1, (dv - BUMP_DV) / 1.5);
playSfx("thump", {
gain: 0.12 + 0.3 * u,
rate: 0.55 + 0.15 * u + Math.random() * 0.06, // way below the ball's range
rate: 0.55 + 0.15 * u + Math.random() * 0.06, // low-pitched body impact
out: duckEmitter.node,
});
haptics.pulse("bump", 0.5 + 0.5 * u); // shove felt in the hands
@@ -1139,42 +1094,6 @@ async function boot({
} else {
bumpPrevValid = false;
}
// Ball impacts: |dv| between control steps. Gravity alone accounts for
// ~0.2 m/s per 20 ms step; the 0.5 threshold clears it and rolling noise.
if (ballActive) {
const v = data.qvel;
const b = ballDofAdr;
if (ballPrevValid) {
const dv = Math.hypot(
v[b] - ballPrevV[0],
v[b + 1] - ballPrevV[1],
v[b + 2] - ballPrevV[2],
);
if (dv > 0.5 && now - ballThumpAt > 90) {
ballThumpAt = now;
const u = Math.min(1, (dv - 0.5) / 3.5); // full at kick-grade hits
playSfx("thump", {
gain: 0.14 + 0.4 * u,
rate: 1.25 - 0.45 * u + Math.random() * 0.08,
out: ballEmitter.node,
});
// Haptics only when the duck plausibly caused/received the hit:
// a far bounce off a wall shouldn't shake the hands.
const bq = ballQposAdr;
const dBall = Math.hypot(
data.qpos[bq] - data.qpos[0],
data.qpos[bq + 1] - data.qpos[1],
);
if (dBall < 0.6) haptics.pulse("ballHit", 0.4 + 0.6 * u);
}
}
ballPrevV[0] = v[b];
ballPrevV[1] = v[b + 1];
ballPrevV[2] = v[b + 2];
ballPrevValid = true;
} else {
ballPrevValid = false;
}
// Haptic channel scheduler: keeps the roller bed alive between pulses
// and cuts the motors when it falls silent. 50 Hz, like everything here.
haptics.tick();
@@ -1220,7 +1139,7 @@ async function boot({
applyGrabForce(); // mouse perturbation, fresh velocity every substep
mujoco.mj_step(model, data);
}
stepAudioSim(); // footsteps / rumble / ball thumps off the fresh state
stepAudioSim(); // footsteps / rumble / body bumps off the fresh state
// Command script clock: 50 Hz like everything here. A non-looping
// script ends on its own; a looping one runs until R (or a reset).
@@ -1317,14 +1236,22 @@ async function boot({
fallenSince = null;
}
// Ball respawn watchdog: outside the arena bounds means "escaped
// through a solver glitch", bring it back near the duck.
if (ballActive) {
const q = data.qpos;
const escaped =
Math.abs(q[ballQposAdr]) > ARENA_HALF + 0.1 ||
Math.abs(q[ballQposAdr + 1]) > ARENA_HALF + 0.1;
if (escaped) spawnBall();
// Return escaped toys individually without disturbing the robot or peers.
if (!propsParked) {
let restored = false;
interactiveSpecs.forEach((spec, i) => {
const a = propAddresses[i].qAdr,
q = data.qpos;
if (
Math.abs(q[a]) > ARENA_HALF + 0.1 ||
Math.abs(q[a + 1]) > ARENA_HALF + 0.1 ||
q[a + 2] < -0.2
) {
resetProp(data, propAddresses[i], spec);
restored = true;
}
});
if (restored) mujoco.mj_forward(model, data);
}
if (postKickLock > 0 && mode === "walk") postKickLock--;
@@ -1469,7 +1396,7 @@ async function boot({
// Released before models/sessions, even when boot or a later lazy load fails.
scope.add(() => scheduler.dispose());
// ── Scene wiring (grid, walls, rig, ball, arcade row) ────────────────
// ── Scene wiring (grid, walls, rig, interactive toys, arcade row) ────
// The grid/walls carry ceremony-driven uReveal uniforms and per-frame
// focus updates, so the game owns them; lights and environment live in
// the R3F layer.
@@ -1478,6 +1405,11 @@ async function boot({
reliefGridMat = grid.material; // relief drive mirrors uTopoScale into it
const { wallMats, wallMeshes } = makeArenaWalls(boundary);
for (const m of wallMeshes) scene.add(m);
grid.name = "arena-grid";
setArenaLineColor([grid.material, ...wallMats], lineColor);
propVisuals = createInteractivePropVisuals(interactiveSpecs);
scene.add(propVisuals.root);
propVisuals.sync(data, propAddresses, !propsParked);
let rig = await rigPromise;
scene.add(rig.placer);
@@ -1493,9 +1425,9 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
};
// ── Locomotion variant switching (legs <-> rollers) ──────────────────
@@ -1572,8 +1504,7 @@ async function boot({
gyroAdr,
trunkId,
standKeyId,
ballQposAdr,
ballDofAdr,
propAddresses,
extraJoints,
ankleIds,
} = L);
@@ -1664,10 +1595,6 @@ async function boot({
setLocked: (v) => {
inputLocked = v;
controller.setLocked(v);
// A ball is always in play: pop one the moment the entrance or a
// respawn ceremony hands control back.
if (!v && ball && !ballActive && !testMode)
spawnBall({ fromQueue: true });
},
flashReset: () => {},
// Audio twins of the wireframe materialize FX (entrance and respawns):
@@ -1693,19 +1620,6 @@ async function boot({
{ fireImmediately: true },
);
const { group: ballGroup, mesh: ballMesh } = createBallVisual(renderer);
scene.add(ballGroup);
ball = createBallActor({
THREE,
scene,
camera,
renderer,
fxModule: fx,
mesh: ballMesh,
group: ballGroup,
});
scope.add(() => ball.dispose());
// ── Prop library (wall/corner dressing + entrance FX) ────────────────
// Every enabled def in props.js: loaded, real-size scaled, floor
// snapped, wireframe-materialized with the ceremony (staggered after
@@ -1876,7 +1790,7 @@ async function boot({
}
// ── Mouse grab, pointer side (pick + drag target + cursor) ────────────
// Pointer-down on the duck (or the live ball) grabs it; anywhere else
// Pointer-down on the duck or an interactive toy grabs it; anywhere else
// falls through to OrbitControls untouched. The pick is a three.js
// raycast against the render rig (the WASM bindings expose no
// mjv_select, and the rig IS the duck's collision-accurate silhouette
@@ -1902,13 +1816,19 @@ async function boot({
}
function grabPick() {
const duckHit = _grabRaycaster.intersectObject(rig.placer, true)[0];
const ballHit = ballActive
? _grabRaycaster.intersectObject(ballMesh, true)[0]
const propHit = !propsParked
? _grabRaycaster.intersectObjects(propVisuals.meshes, false)[0]
: undefined;
if (duckHit && (!ballHit || duckHit.distance <= ballHit.distance)) {
if (duckHit && (!propHit || duckHit.distance <= propHit.distance)) {
return { kind: "duck", point: duckHit.point };
}
return ballHit ? { kind: "ball", point: ballHit.point } : null;
return propHit
? {
kind: "prop",
point: propHit.point,
index: propHit.object.userData.interactiveIndex,
}
: null;
}
function updateGrabTarget() {
if (!_grabRaycaster.ray.intersectPlane(_grabPlane, _grabHit)) return;
@@ -1956,18 +1876,7 @@ async function boot({
dofAdr: 0,
mass: model.body_subtreemass[trunkId],
}
: {
bodyId: mujoco.mj_name2id(
model,
mujoco.mjtObj.mjOBJ_BODY.value,
"ball",
),
qAdr: ballQposAdr,
dofAdr: ballDofAdr,
mass: model.body_mass[
mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_BODY.value, "ball")
],
};
: propAddresses[pick.index];
camera.getWorldDirection(_grabCamDir);
_grabPlane.setFromNormalAndCoplanarPoint(
_grabCamDir.negate(),
@@ -2050,8 +1959,7 @@ async function boot({
setJoint(rig, JOINT_NAMES[j], qpos[qposAdr[j]]);
// Passive hinges (roller wheels): purely visual, driven straight from qpos.
for (const ej of extraJoints) setJoint(rig, ej.name, qpos[ej.adr]);
// Ball: live follows qpos; ghost freeze is owned by the ball actor.
if (ball) ball.sync(qpos, ballQposAdr, ballActive);
propVisuals?.sync(data, propAddresses, !propsParked);
// Follow cam: ease the orbit target toward the trunk and translate the
// camera by the same delta, so the camera-to-duck distance and viewing
// angle stay constant while the duck walks. Paused while the reset
@@ -2372,18 +2280,9 @@ async function boot({
syncRig();
syncJaw();
ghosts?.update();
// Spatial audio follows the movers: listener on the camera, emitters
// on the duck trunk and the ball (MJCF Z-up -> three Y-up).
// Spatial audio: listener on the camera, emitter on the duck trunk.
updateListener(camera);
duckEmitter.setPosition(data.qpos[0], data.qpos[2], -data.qpos[1]);
if (ballActive) {
const q = data.qpos;
ballEmitter.setPosition(
q[ballQposAdr],
q[ballQposAdr + 2],
-q[ballQposAdr + 1],
);
}
controls.update();
updateChaseCam();
if (helperCamera !== camera) {
@@ -2391,7 +2290,6 @@ async function boot({
helperCamera.copy(camera);
}
ceremony.drive();
ball.drive(() => spawnBall({ fromQueue: true }));
waypointMarker.update(dt, waypointSource.target);
renderTelemetry();
}
@@ -2408,7 +2306,6 @@ async function boot({
controller.on("reset", () => gameApi.reset?.());
controller.on("reset", () => waypointSource.cancel());
controller.on("spawnBall", () => spawnBall());
controller.on("headToggle", () => toggleHeadMode());
controller.on("chaseToggle", () => startCameraReset());
controller.on("locoToggle", () => toggleLoco());
@@ -2604,7 +2501,7 @@ async function boot({
mode = foot === "left" ? "kickL" : "kickR";
sitFlag = 0;
kickRun = { steps: 0 };
haptics.pulse("kick"); // swing launch; ball contact adds ballHit
haptics.pulse("kick"); // swing launch
syncButtons();
stickers?.pop("kick");
return true;
@@ -2634,7 +2531,6 @@ async function boot({
? "Drive"
: "Run";
if (store().modeLabel !== label) setStore({ modeLabel: label });
if (store().ballActive !== ballActive) setStore({ ballActive });
}
// ── Public surface for the React UI ──────────────────────────────────
@@ -2666,9 +2562,14 @@ async function boot({
loco,
locoWant: loco,
variant: currentVariant,
lineColor,
modeLabel: store().modeLabel,
inputBindings: robot.input.bindings,
runtimeCapabilities: { duckControls: true, physicalDrag: true },
runtimeCapabilities: {
duckControls: true,
environmentControls: true,
physicalDrag: true,
},
}),
triggerAction: (action) => {
if (
@@ -2678,7 +2579,10 @@ async function boot({
)
kbSource.onAction(action);
},
saveSettings: () => ({ variant: currentVariant, loco: store().locoWant }),
saveSettings: () => ({
variant: currentVariant,
loco: store().locoWant,
}),
restoreSettings: async (settings) => {
if (settings?.variant) runtime.setVariant(settings.variant);
if (settings?.loco)
@@ -2727,13 +2631,17 @@ async function boot({
applyVariant(rig, name);
setStore({ variant: name });
},
setLineColor: (name) => {
if (!setArenaLineColor([grid.material, ...wallMats], name)) return;
lineColor = name;
setStore({ lineColor: name });
},
requestLoco: (name) => {
if (name !== "legs" && name !== "rollers") return;
setStore({ locoWant: name });
reconcileLoco();
},
resetSim,
spawnBall: () => spawnBall(),
startEntrance: () => ceremony.startEntrance(),
loadCustomPolicy: (ref) => loadCustomPolicyIntoGame(ref),
clearCustomPolicy: () => revertCustomPolicy(),
@@ -2768,7 +2676,13 @@ async function boot({
controller,
kbSource,
padSource,
spawnBall,
get interactiveProps() {
return interactiveSpecs.map((spec, i) => ({
...spec,
...propAddresses[i],
}));
},
setPropsParked,
triggerKick,
triggerRoll,
sessions,
@@ -2828,12 +2742,6 @@ async function boot({
toggleHeadMode,
headTarget,
headSmooth,
get ballActive() {
return ballActive;
},
get ballQposAdr() {
return ballQposAdr;
},
get chaseCam() {
return chaseCam;
},
@@ -2884,7 +2792,7 @@ async function boot({
await scheduler.pause();
ceremony.setReveal(1);
ceremony.setFx(1);
parkBallPhysics();
setPropsParked(true);
inputLocked = false;
controller.setLocked(false);
}
+20
View File
@@ -0,0 +1,20 @@
// Archived policy goldens were recorded with exactly one parked sphere.
// Extra free DOFs change MuJoCo floating-point accumulation (especially
// wheel contact), even when the new toys are parked far away. Reproduce
// that original laboratory world ONLY for ?test=1&oracle=legacy. Normal
// simulation and scene-interaction tests always use the 5–7 toy layout.
export function originalPhysicsProbe() {
return [
{
name: "interactive_0",
type: "sphere",
size: [0.05],
pos: [50, 0, 0.05],
quat: [1, 0, 0, 0],
mass: 0.03,
friction: "0.4 0.01 0.003",
solref: "0.03 0.4",
color: 0xf2efe8,
},
];
}
+8 -7
View File
@@ -17,7 +17,7 @@ function Game() {
room.dispose();
pmrem.dispose();
scene.environment = environment.texture;
scene.environmentIntensity = 0.45;
scene.environmentIntensity = 0.7;
bootLog.length = 0;
const manager = new RuntimeManager({
scene,
@@ -44,7 +44,7 @@ function Game() {
for (const method of [
"setVariant",
"requestLoco",
"spawnBall",
"setLineColor",
"loadCustomPolicy",
"clearCustomPolicy",
"triggerAction",
@@ -68,20 +68,21 @@ function Game() {
}, [scene, camera, gl]);
useEffect(() => {
scene.background = new THREE.Color(
theme === "neutral" ? 0x18202b : 0x08080c,
theme === "neutral" ? 0x18202b : 0x151923,
);
gl.setClearColor(scene.background, 1);
}, [scene, gl, theme]);
useFrame((_, dt) => gameApi.frame?.(Math.min(dt, 0.05)));
return (
<>
<ambientLight intensity={0.6} />
<directionalLight position={[2, 4, 2]} intensity={1.6} />
<directionalLight position={[-2, 2, 1.5]} intensity={0.4} />
<ambientLight intensity={0.85} />
<hemisphereLight args={[0xe4edff, 0x66605a, 0.75]} />
<directionalLight position={[2, 4, 2]} intensity={2.0} />
<directionalLight position={[-2, 2, 1.5]} intensity={0.8} />
<directionalLight
color={theme === "neutral" ? 0xbcdcff : 0xffb366}
position={[0, 3, -2]}
intensity={0.7}
intensity={0.85}
/>
</>
);
+1 -1
View File
@@ -38,6 +38,7 @@ export const useGame = create(
locoSwitching: false,
rollersLoading: false, // OSD line while the roller stack streams in
variant: "classic",
lineColor: "classic",
// Community move: null = all-official slots. Written by the game
// (game.js loadCustomPolicyIntoGame / the watchdog) and by App.jsx for
// the boot-time ?move= request.
@@ -47,7 +48,6 @@ export const useGame = create(
customPolicy: null,
padConnected: false,
touchMode: false,
ballActive: false,
// Throttled telemetry block (4 Hz), bottom-right OSD
telemetry: { fps: 0, ctrlHz: 0, speed: 0, odo: 0, peers: 0 },
})),
+1 -1
View File
@@ -125,7 +125,7 @@ export default function ControlsHelp() {
{physicalDrag && (
<>
<br />
Drag robot / ball: apply force
Drag robot / geometry: apply force
</>
)}
</div>
+88 -59
View File
@@ -182,7 +182,8 @@ const hudHitSx = {
};
function Quickbar() {
const variant = useGame((s) => s.variant);
const lineColor = useGame((s) => s.lineColor);
const duckControls = useGame((s) => s.runtimeCapabilities.duckControls);
const locoWant = useGame((s) => s.locoWant);
return (
<Box
@@ -198,10 +199,10 @@ function Quickbar() {
maxWidth: "calc(100vw - 14.5rem)",
}}
>
<HudPlate caption="Color">
<HudPlate caption="Line color">
<Box
role="group"
aria-label="Duck colours"
aria-label="Scene line colours"
sx={{
display: "inline-flex",
alignItems: "stretch",
@@ -210,17 +211,20 @@ function Quickbar() {
}}
>
{Object.entries(VARIANT_SWATCH_HEX).map(([name, hex]) => {
const selected = name === variant;
const selected = name === lineColor;
const label = VARIANT_LABELS[name] ?? name;
return (
<Box
key={name}
component="button"
type="button"
title={label}
aria-label={`${label} colours`}
title={`${label} scene lines`}
aria-label={`${label} scene lines`}
aria-pressed={selected}
onClick={() => { uiClick(); gameApi.setVariant?.(name); }}
onClick={() => {
uiClick();
gameApi.setLineColor?.(name);
}}
sx={{
appearance: "none",
width: CELL_H, // square cell inside frame + padding
@@ -241,55 +245,64 @@ function Quickbar() {
outline: `2px dashed ${CREAM}`,
outlineOffset: -5,
},
"@media (prefers-reduced-motion: reduce)": { animation: "none" },
"@media (prefers-reduced-motion: reduce)": {
animation: "none",
},
}}
/>
);
})}
</Box>
</HudPlate>
<HudPlate caption="Mode">
<Box
role="group"
aria-label="Locomotion mode"
sx={{
display: "inline-flex",
alignItems: "stretch",
height: "100%",
gap: `${CELL_GAP}px`,
}}
>
{[
{ name: "legs", label: "Feet" },
{ name: "rollers", label: "Rollers" },
].map(({ name, label }) => {
const selected = locoWant === name;
return (
<Box
key={name}
component="button"
type="button"
aria-pressed={selected}
onClick={() => { uiClick(); gameApi.requestLoco?.(name); }}
sx={{
...hudHitSx,
px: "1.05rem",
background: selected ? ORANGE : "transparent",
color: selected ? COMIC_INK : "rgba(250, 248, 242, 0.72)",
"&:hover": {
color: selected ? COMIC_INK : CREAM,
background: selected ? ORANGE : "rgba(255, 122, 47, 0.12)",
},
"&:active": { filter: "brightness(0.92)" },
}}
>
{label}
</Box>
);
})}
</Box>
</HudPlate>
<PolicyChip />
{duckControls && (
<HudPlate caption="Mode">
<Box
role="group"
aria-label="Locomotion mode"
sx={{
display: "inline-flex",
alignItems: "stretch",
height: "100%",
gap: `${CELL_GAP}px`,
}}
>
{[
{ name: "legs", label: "Feet" },
{ name: "rollers", label: "Rollers" },
].map(({ name, label }) => {
const selected = locoWant === name;
return (
<Box
key={name}
component="button"
type="button"
aria-pressed={selected}
onClick={() => {
uiClick();
gameApi.requestLoco?.(name);
}}
sx={{
...hudHitSx,
px: "1.05rem",
background: selected ? ORANGE : "transparent",
color: selected ? COMIC_INK : "rgba(250, 248, 242, 0.72)",
"&:hover": {
color: selected ? COMIC_INK : CREAM,
background: selected
? ORANGE
: "rgba(255, 122, 47, 0.12)",
},
"&:active": { filter: "brightness(0.92)" },
}}
>
{label}
</Box>
);
})}
</Box>
</HudPlate>
)}
{duckControls && <PolicyChip />}
{/* Sound is fully disabled for now (SOUND_DISABLED in audio.js) -
the mute toggle comes back with the finished sound design. */}
{/* <SoundToggle /> */}
@@ -307,10 +320,15 @@ function PolicyChip() {
if (cp?.status !== "active") return null;
const title = cp.title || cp.name;
const by = cp.author ? ` by ${cp.author}` : "";
const hint = cp.slot === "trick" || cp.slot === "script" || cp.slot === "sitstand" ? " \u00b7 R" : "";
const hint =
cp.slot === "trick" || cp.slot === "script" || cp.slot === "sitstand"
? " \u00b7 R"
: "";
return (
<HudPlate caption="Playing" captionFill="orange">
<Box sx={{ display: "inline-flex", alignItems: "stretch", height: "100%" }}>
<Box
sx={{ display: "inline-flex", alignItems: "stretch", height: "100%" }}
>
<Box
sx={{
...hudHitSx,
@@ -331,7 +349,10 @@ function PolicyChip() {
component="button"
type="button"
aria-label={`Unload move ${cp.name}`}
onClick={() => { uiClick(); gameApi.clearCustomPolicy?.(); }}
onClick={() => {
uiClick();
gameApi.clearCustomPolicy?.();
}}
sx={{
...hudHitSx,
px: "0.6rem",
@@ -388,7 +409,8 @@ function SoundToggle() {
function Telemetry() {
const t = useGame((s) => s.telemetry);
const odo = t.odo < 1000 ? `${t.odo.toFixed(1)}M` : `${(t.odo / 1000).toFixed(2)}KM`;
const odo =
t.odo < 1000 ? `${t.odo.toFixed(1)}M` : `${(t.odo / 1000).toFixed(2)}KM`;
const lines = [];
if (t.peers) lines.push(`${t.peers + 1} ONLINE`);
lines.push(`${t.speed.toFixed(2)}M/S \u00b7 ODO ${odo}`);
@@ -446,7 +468,11 @@ function OsdLoad() {
{l}
<Box
component="span"
sx={{ display: "inline-block", ml: "0.15em", animation: `${recBlink} 0.8s steps(1) infinite` }}
sx={{
display: "inline-block",
ml: "0.15em",
animation: `${recBlink} 0.8s steps(1) infinite`,
}}
>
{"\u2588"}
</Box>
@@ -500,19 +526,22 @@ function PolicyToast() {
}
export default function Hud() {
const ready = useGame((s) => s.runtimePhase === 'ready');
const ready = useGame((s) => s.runtimePhase === "ready");
const touchMode = useGame((s) => s.touchMode);
const duckControls = useGame((s) => s.runtimeCapabilities.duckControls);
const environmentControls = useGame(
(s) => s.runtimeCapabilities.environmentControls,
);
if (!ready) return null;
if (!duckControls) return <Telemetry />;
if (!duckControls && !environmentControls) return <Telemetry />;
return (
<>
{!touchMode && (
<>
<Quickbar />
<Telemetry />
<OsdLoad />
<PolicyToast />
{duckControls && <OsdLoad />}
{duckControls && <PolicyToast />}
</>
)}
</>
+2
View File
@@ -35,6 +35,8 @@ not proof that every possible long-running skill sequence is covered.
After the user-requested expansion to a 4.5 m default arena, this original-code oracle explicitly pins the **original 3 m World** through a test-only World response override. The production default remains 4.5 m, tested independently in `bounded-world.spec.js`. No accepted trajectory or tolerance was replaced to hide the intentional World-size change.
After replacing the production football with 5–7 random free-body geometries, the archived oracle additionally uses **`?test=1&oracle=legacy`**. `src/robots/microduck/test-oracle.js` supplies the original single parked sphere's dimensions, mass and contact parameters, with no football texture or gameplay. This keeps the oracle's model DOF count and floating-point accumulation unchanged; the nine accepted trajectories and exact comparisons remain untouched. `?test=1` alone still builds the real 5–7 toy scene; `prepareTest()` parks those toys for isolated stepping, and `scene-interaction.spec.js` explicitly restores them for real collision / drag tests. Production never selects the legacy probe unless both explicit test parameters are present. The Microduck composer fixes `jacobian="dense"` to retain the original solver path when the added toy DOFs cross MuJoCo's automatic sparse threshold.
`npm run test:regression` reads these files. `npm run baseline:record` writes
candidate files under ignored `test-results/baseline-candidates/`, never here.
A baseline replacement needs an explicit reviewed cause and updated provenance.
+141
View File
@@ -0,0 +1,141 @@
import test from "node:test";
import assert from "node:assert/strict";
import * as THREE from "three";
import {
generateInteractiveProps,
createInteractivePropVisuals,
resetProp,
} from "../src/game/interactive-props.js";
import { setArenaLineColor } from "../src/game/worlds/arena/line-color.js";
import {
makeInfiniteGrid,
makeArenaWalls,
} from "../src/game/worlds/arena/render.js";
import { VARIANT_SWATCH_HEX } from "../src/game/variants.js";
function rng(seed) {
return () => (seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296;
}
test("random layouts have 5–7 varied, separated, lightweight toys inside walls and clear of spawn", () => {
const counts = new Set();
for (let seed = 0; seed < 1000; seed++) {
for (const halfExtent of [1.5, 2.25]) {
const specs = generateInteractiveProps({
halfExtent,
spawn: [-0.6, 0, 0.12],
random: rng(seed * 7919),
});
counts.add(specs.length);
assert.ok(specs.length >= 5 && specs.length <= 7);
assert.equal(new Set(specs.map((p) => p.type)).size, 4);
assert.equal(new Set(specs.map((p) => p.name)).size, specs.length);
assert.ok(specs[0].pos[0] > -0.3 && specs[0].pos[0] < 0.25);
for (const [i, spec] of specs.entries()) {
assert.ok(spec.mass >= 0.04 && spec.mass <= 0.08);
assert.ok(Math.abs(spec.pos[0]) < halfExtent - 0.1);
assert.ok(Math.abs(spec.pos[1]) < halfExtent - 0.1);
assert.ok(Math.hypot(spec.pos[0] + 0.6, spec.pos[1]) > 0.34);
for (const other of specs.slice(i + 1))
assert.ok(
Math.hypot(spec.pos[0] - other.pos[0], spec.pos[1] - other.pos[1]) >
0.3,
);
}
}
}
assert.deepEqual([...counts].sort(), [5, 6, 7]);
assert.notDeepEqual(
generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(1),
}),
generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(2),
}),
);
});
test("visual geometry matches MJCF dimensions, poses sync through Z-up root, reset clears only prop DOFs", () => {
const specs = generateInteractiveProps({
halfExtent: 2.25,
spawn: [-0.6, 0],
random: rng(1),
});
const view = createInteractivePropVisuals(specs);
const addresses = specs.map((_, i) => ({
qAdr: 7 + i * 7,
dofAdr: 6 + i * 6,
}));
const data = {
qpos: new Float64Array(7 + specs.length * 7).fill(123),
qvel: new Float64Array(6 + specs.length * 6).fill(42),
};
specs.forEach((spec, i) => resetProp(data, addresses[i], spec));
view.sync(data, addresses);
view.root.updateMatrixWorld(true);
view.meshes.forEach((mesh, i) => {
assert.equal(mesh.name, specs[i].name);
assert.equal(mesh.userData.interactiveIndex, i);
assert.deepEqual(
mesh
.getWorldPosition(new THREE.Vector3())
.toArray()
.map((v) => +v.toFixed(6)),
[specs[i].pos[0], specs[i].pos[2], -specs[i].pos[1]].map(
(v) => +v.toFixed(6),
),
);
mesh.geometry.computeBoundingBox();
const size = mesh.geometry.boundingBox.getSize(new THREE.Vector3());
const spec = specs[i];
const height =
2 *
(spec.type === "sphere"
? spec.size[0]
: spec.type === "capsule"
? spec.size[0] + spec.size[1]
: spec.size.at(-1));
assert.ok(Math.abs(size.z - height) < 1e-7);
});
assert.deepEqual(Array.from(data.qpos.slice(0, 7)), Array(7).fill(123));
assert.deepEqual(Array.from(data.qvel.slice(0, 6)), Array(6).fill(42));
assert.ok(Array.from(data.qvel.slice(6)).every((v) => v === 0));
resetProp(data, addresses[0], specs[0], true, 0);
view.sync(data, addresses, false);
assert.equal(view.root.visible, false);
assert.equal(data.qpos[addresses[0].qAdr], 50);
for (const mesh of view.meshes) {
mesh.geometry.dispose();
mesh.material.dispose();
}
});
test("the original four swatches tint floor AND all walls without altering fades or reveal", () => {
const grid = makeInfiniteGrid({ subdivisions: 1 });
const { wallMats, wallMeshes } = makeArenaWalls();
const materials = [grid.material, ...wallMats];
for (const [name, hex] of Object.entries(VARIANT_SWATCH_HEX)) {
assert.equal(setArenaLineColor(materials, name), true);
for (const mat of materials) {
assert.equal(
mat.uniforms.uSectionColor.value.getHexString(),
hex.slice(1),
);
assert.equal(mat.uniforms.uReveal.value, 0);
assert.ok(
mat.uniforms.uCellColor.value.equals(
mat.uniforms.uSectionColor.value.clone().multiplyScalar(0.55),
),
);
}
}
assert.equal(setArenaLineColor(materials, "unknown"), false);
for (const mesh of [grid, ...wallMeshes]) {
mesh.geometry.dispose();
mesh.material.dispose();
}
});
+108
View File
@@ -0,0 +1,108 @@
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import { createHash } from "node:crypto";
import { validateBodyGlb } from "../src/platform/render/BodyVisuals.js";
import { validatePackage } from "../src/platform/loader/SchemaValidator.js";
const root = new URL("../public/robots/rp1/", import.meta.url);
const config = JSON.parse(await fs.readFile(new URL("robot.json", root)));
const manifest = JSON.parse(
await fs.readFile(new URL("visual-source-manifest.json", root)),
);
const load = async (path) => {
const b = await fs.readFile(new URL(path, root));
return validateBodyGlb(
b.buffer.slice(b.byteOffset, b.byteOffset + b.byteLength),
);
};
test("RP1 maps all 63 supplied parts exactly once onto 25 visual-only body attachments", async () => {
validatePackage(config);
assert.equal(config.render.body_visuals.length, 25);
assert.equal(manifest.input_files.length, 63);
const parts = Object.values(manifest.body_mapping).flatMap((m) => m.parts);
assert.equal(parts.length, 63);
assert.equal(new Set(parts).size, 63);
let normals = 0,
colourTextures = 0,
emissive = 0;
for (const spec of config.render.body_visuals) {
assert.ok(!config.assets.some((a) => a.path === spec.path));
assert.deepEqual(spec.quaternion, [Math.SQRT1_2, 0, 0, Math.SQRT1_2]);
const doc = await load(spec.path);
for (const mat of doc.materials) {
if (mat.normalTexture) {
normals++;
assert.ok(doc.textures[mat.normalTexture.index]);
}
if (mat.pbrMetallicRoughness?.baseColorTexture) colourTextures++;
if (mat.extensions?.KHR_materials_emissive_strength?.emissiveStrength > 1)
emissive++;
}
for (const texture of doc.textures) assert.ok(doc.images[texture.source]);
if (spec.body.endsWith("_wrist_link")) {
const wrappers = doc.nodes.filter((n) => n.extras?.sourcePart);
assert.equal(wrappers.length, 20);
assert.equal(new Set(wrappers.map((n) => n.extras.sourcePart)).size, 20);
assert.deepEqual(
wrappers.map((n) => n.extras.sourcePart).sort(),
[...manifest.body_mapping[spec.body].parts].sort(),
);
}
}
assert.ok(normals >= 63);
assert.ok(colourTextures >= 1, "face screen texture retained");
assert.ok(emissive >= 1, "torso logo emissive strength retained");
});
test("23 unchanged body shapes align exactly with original STL bounds after Y-up conversion", async () => {
for (const { body, path } of config.render.body_visuals.filter(
(s) => !s.body.endsWith("_wrist_link"),
)) {
const doc = await load(path),
lo = [Infinity, Infinity, Infinity],
hi = [-Infinity, -Infinity, -Infinity];
for (const mesh of doc.meshes)
for (const p of mesh.primitives) {
const a = doc.accessors[p.attributes.POSITION];
for (let i = 0; i < 3; i++) {
lo[i] = Math.min(lo[i], a.min[i]);
hi[i] = Math.max(hi[i], a.max[i]);
}
}
const transformed = [lo[0], -hi[2], lo[1], hi[0], -lo[2], hi[1]];
const stl = await fs.readFile(new URL(`meshes/${body}.STL`, root));
const expected = [
Infinity,
Infinity,
Infinity,
-Infinity,
-Infinity,
-Infinity,
];
for (let t = 0; t < stl.readUInt32LE(80); t++)
for (let v = 0; v < 3; v++)
for (let axis = 0; axis < 3; axis++) {
const x = stl.readFloatLE(84 + t * 50 + 12 + v * 12 + axis * 4);
expected[axis] = Math.min(expected[axis], x);
expected[axis + 3] = Math.max(expected[axis + 3], x);
}
assert.deepEqual(transformed, expected, body);
}
});
test("original model, physics meshes, policies/motions and imported visual provenance retain reviewed hashes", async () => {
const physical = JSON.parse(
await fs.readFile(new URL("source-manifest.json", root)),
);
for (const file of [...physical.files, ...manifest.files]) {
assert.equal(
createHash("sha256")
.update(await fs.readFile(new URL(file.file, root)))
.digest("hex"),
file.sha256,
file.file,
);
}
});
+36
View File
@@ -0,0 +1,36 @@
import test from "node:test";
import assert from "node:assert/strict";
import { createSessionEnvironment } from "../src/game/worlds/arena/session-environment.js";
const world = (scale) => ({
presentation: "arena-grid",
boundary: { half_extent: 2.25 * scale },
});
test("all robot sizes share one random layout, with scaled geometry and spawn clearances", () => {
let seed = 42;
const scene = createSessionEnvironment(
() => (seed = (1664525 * seed + 1013904223) >>> 0) / 4294967296,
);
const duck = scene.propsFor(world(1));
for (const scale of [1, 0.33 / 0.27, 1.32 / 0.27]) {
const props = scene.propsFor(world(scale));
assert.equal(props.length, duck.length);
props.forEach((p, i) => {
assert.equal(p.type, duck[i].type);
assert.equal(p.color, duck[i].color);
p.pos.forEach((v, j) =>
assert.ok(Math.abs(v / scale - duck[i].pos[j]) < 1e-12),
);
p.size.forEach((v, j) =>
assert.ok(Math.abs(v / scale - duck[i].size[j]) < 1e-12),
);
assert.ok(Math.hypot(p.pos[0] / scale, p.pos[1] / scale) > 0.34);
assert.ok(Math.hypot(p.pos[0] / scale + 0.6, p.pos[1] / scale) > 0.34);
});
}
const copy = scene.propsFor(world(1));
copy[0].pos[0] = 100;
copy[0].quat[0] = 0;
assert.deepEqual(scene.propsFor(world(1)), duck);
assert.deepEqual(scene.propsFor({ type: "flat" }), []);
});
+17
View File
@@ -19,6 +19,23 @@ npm --prefix app run dev -- --host 127.0.0.1
**限制:完整舞蹈参考长 131.46 秒;旧版 model_7999 的长程测试在参考帧 994(约 19.9 秒)失稳。新版 model_23998 目前只验证并开放前 10 秒,完整时长尚未重新验收,不自动延长 duration。** 训练完成不等于任意长度播放已收敛;不能仅延长 duration 就宣称稳定。
## GLB 材质与外观
已接入用户提供的 `RP1机器人glb.zip`:白/灰/蓝 PBR 材质、法线贴图、面部屏幕和发光标识均保留。63 个部件映射到现有 25 个物理 body,物理模型仍为 24 自由度;23 个 body 直接使用原 GLB,双手各 20 个部件按原始固定手 URDF 组合到手腕 body。手指随手腕整体运动,不新增独立手指控制,也不改变行走、舞蹈或挥手策略。
模型 XML、原始碰撞体、惯性、关节及全部 ONNX / motion 文件不变;仅新增 `render.body_visuals`。原 GLB 为 Y-up,绑定时统一绕 X 轴 +90°,不能当作 Z-up 直接贴到 body。原始 STL 可视 mesh 会隐藏,而不是与新模型重叠绘制。
复现导入:
```bash
python3 app/scripts/import-rp1-visuals.py "/home/cen/Downloads/RP1机器人glb.zip" \
app/public/robots/rp1/visuals/rp1_hand_fixed.urdf
npm --prefix app run validate:packages
npm --prefix app run test:e2e -- rp1-visuals
```
来源和 SHA-256 见 `app/public/robots/rp1/visual-source-manifest.json`;包内未附再分发许可。首次 RP1 加载会额外读取约 93 MB 的自包含 GLB,具体帧率取决于 GPU;本次没有简化或改写用户的网格、贴图和材质。验证记录见 [RP1 GLB 材质替换](../superpowers/verification/rp1-glb-materials.md)。
## 已完成权重
均来自 `/home/cen/Embodied_Workspace/Mujoco_Projects/roboparty_train/logs/rsl_rl`,显式选择最终、完整且稳定的文件,不动态追踪目录中的最新文件:
@@ -0,0 +1,38 @@
# Microduck 场景提亮、线条调色和交互几何体
> 本文记录最初的 Microduck 单机器人改动。后续根据“每个机器人的环境都保持一致”的要求,已推广到所有机器人,并改为会话共享布局和全局线条色;当前规则及验证见 [共享环境验证](shared-robot-environment.md)。
## 实现范围
- `app/src/scene/GameCanvas.jsx`:新增半球补光,提高环境贴图、环境光、主光和侧光强度;漫画场景背景从近黑调整为深蓝灰。没有添加外部光照资源或网络依赖。
- `app/src/ui/Hud.jsx`、`app/src/game/worlds/arena/line-color.js`:原左下角四个色块改为 **Line color**,仍使用原 Cream / Graphite / Lavender / Sky 色值、尺寸、边框、选中动画和漫画面板。同步改变地板与四面围墙的粗细网格线,不修改机器人或几何体材质。颜色在 reset、feet / rollers 切换和机器人往返切换时保留。
- `app/src/game/interactive-props.js`、`app/src/game/worlds/arena/compose.js`、`app/src/robots/microduck/runtime.js`:生产场景删除足球刚体、足球纹理渲染、生成逻辑和足球专属音频/拖拽分支,替换为每次新建 Microduck runtime 随机生成的 **5~7 个**轻量几何体,包括方块、球体、圆柱和胶囊体。
- 几何体具有独立 MuJoCo freejoint、真实质量、摩擦和碰撞;支持机器人推动/踢动、物体间碰撞、地面/围墙碰撞、鼠标弹簧力拖拽与抬起。生成位置互不重叠并避开机器人出生点,至少一个位于机器人附近。reset / locomotion 切换恢复当前布局;重新进入 Microduck 随机生成新布局。越界物体单独复位。
- 原有足球素材/历史库文件未做无关删除,但生产 runtime 不再加载或创建足球。
## 物理与回归说明
新增自由刚体会改变 MuJoCo 的 DOF 数。轮滑模型可能跨越自动 sparse Jacobian 阈值,因此显式保留原来的 `jacobian="dense"`,避免几何体数量影响求解路径。
九份历史动作轨迹全部保留,未更新基线或放宽浮点比较。`?test=1&oracle=legacy` 专门复现历史回归使用的 3 m 实验场及一个远处停放的无足球纹理球形物理探针,保留原始 DOF 数和接触参数,从而继续精确比对全部状态。正常运行和 `?test=1` 单独使用时都构建真实的 5~7 个几何体场景。新场景的碰撞、鼠标拖拽、物体数量、材质、光照和运行稳定性由独立浏览器测试覆盖,不以历史实验场测试替代。
## 验证覆盖
- `app/test/interactive-props.test.js`:2,000 个布局(1,000 个随机种子 × 3 m / 4.5 m 两种边界),验证 5/6/7 数量、四种形状、出生点净空、物体间距、边界和质量;验证可视几何体尺寸、Z-up 转换、刚体状态同步与复位;验证四种颜色同步更新地面与四面墙。
- `app/e2e/scene-interaction.spec.js`:真实 Chromium + MuJoCo WASM,验证新光源与场景初始内容、四个色块外观与功能、机器人材质不变、鼠标物理拖拽、暂停释放抓取力、重置布局、物体间动量传递、机器人/地面碰撞、腿/轮滑切换后的物体数量及状态。
- `app/e2e/scene-stability.spec.js`:真实新场景中腿/轮滑各 60 秒模拟时间的连续运动,检查状态有限、物体边界、确实产生移动,以及越界物体单独恢复。
- `app/e2e/deployment.spec.js`:正常生产入口自动显示 5~7 个物体且不存在足球;继续覆盖三种机器人与静态资源路径。
- `app/e2e/runtime-leaks.spec.js`:20 次机器人切换后检查 WASM 对象释放、GPU 几何体、纹理和事件监听器无增长;对当前 5~7 个物体带来的预期几何体数量差异做精确扣除,而非放宽泄漏上限。
## 执行结果
- `cd app && npm test`:**62/62 通过**。
- `npm run validate:packages`:四个 bundled Package 的 schema、World、资源路径与既有来源校验全部通过。
- `npm run format:check`、`npm run build`、`git diff --check`:通过。
- `npm run test:e2e`:**48/48 通过**(13.8 分钟;包含九项精确历史动作回归、三机器人场景/动作、20 次资源切换及新场景交互)。该批启动后补充的连续运动用例另外独立执行并通过。
- 对最终 `dist/` 启动本地 Vite preview,以 `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- scene-interaction scene-stability deployment robot-switching microduck-regression --output=/tmp/microduck-scene-production-final-artifacts` 执行生产包复验:**16/16 通过**(2.7 分钟)。覆盖全部新交互、围墙阻挡、正常入口无足球、两分钟新场景模拟运动和九项历史动作精确回归。
- 人工查看新场景截图:`app/test-results/scene-interaction-overview.png`,确认补光后的机器人及彩色几何体、浅蓝网格线和保留原样的四格色板显示正常。截图属于忽略的本地测试产物,不提交大尺寸图片。
日志:`/tmp/microduck-scene-unit-tests.log`、`/tmp/microduck-scene-full-e2e.log`、`/tmp/microduck-scene-production-final-tests.log`。全量浏览器 JSON 结果另存于 `/tmp/microduck-scene-full-results.json`。
边界:没有声称执行连续 15 小时实时时长的压力测试;新增长序列检查为真实 WASM 中累计两分钟的模拟运动。
@@ -0,0 +1,64 @@
# RP1 用户 GLB 材质替换
## 输入与范围
- 输入:`/home/cen/Downloads/RP1机器人glb.zip`,63 个自包含 GLB,未包含许可文件。
- ZIP SHA-256:`8610aac8432203bf68d0efe234062c551ca805ebbe72a92d5fc7350730ebdea2`。
- 仅新增 RP1 `render.body_visuals` 和可视资源;**原 `model.xml`、STL、惯性、碰撞体、24 个驱动关节、ONNX、motion 文件均未修改**。它们仍通过原 `source-manifest.json` 哈希验证。
- 复用已有 `BodyVisuals` 渲染替换层,没有为本次替换修改共享渲染器、物理引擎或控制器。其他机器人材质、共享环境及色板不受影响。
## 坐标与部件适配
GLB 来自 Blender,顶点为 Y-up;现有 body 帧为 MuJoCo Z-up。所有挂载使用绕 X 轴 +90° 四元数。对全部 63 个输入 GLB,按此变换得到的包围盒与原始描述仓库对应 STL 包围盒逐项比较,最大差为 **0**。
当前训练模型是 25 个 body / 24 个受控关节,而该可视包包括完整的手腕和灵巧手:
- 23 个普通 body 使用输入 GLB 原始字节,不改颜色或贴图。
- 每侧手腕合并 20 个输入部件:wrist roll / yaw / pitch 三段、手掌及 16 个手指部件。
- 按原始 `rp1_hand_fixed.urdf` 的相对原点和固定指节姿态组合;原 24-DOF 手腕对应 wrist-roll,额外 yaw/pitch 轴固定在中立位。
- 合成 GLB 仅追加可视变换节点并重映射 GLTF 索引;顶点、索引、图像二进制数据以及 PBR 材质内容保留。
- 最终为 25 个挂载 GLB,共覆盖 63 个输入部件,且每个部件只使用一次。**没有新增手指/手腕物理自由度,也不声称支持独立手指控制**。
- 新手部的可视细节不会自动改变原训练模型的碰撞包络。
`app/scripts/import-rp1-visuals.py` 是可复现导入器;不会解压任意 ZIP 路径,在完成输入和合成输出验证后才写入包目录。`visual-source-manifest.json` 保存所有输入/输出 SHA-256、部件映射和复制的固定手 URDF 哈希。
## 已测定量结果
真实 Chromium / MuJoCo WASM / Three.js 中:
| 指标 | 结果 |
|---|---:|
| 可视 body 挂载 | 25 |
| 手腕/手部源变换节点 | 40(每侧 20) |
| 新可视 mesh | 79 |
| 加载的法线贴图材质 | 77 |
| 面部屏幕颜色贴图 | 1 |
| 保留发光强度的标识材质 | 1 |
| 隐藏的原 STL 可视 mesh | 25 |
| body 姿态跟随误差 | 0 |
| 23 个原形状的三维包围盒最大误差 | `1.3124658082563911e-8 m` |
| 双手变换相对独立 URDF 计算最大误差 | `5.551115123125783e-16` |
| 行走观测 / 动作维度 | 243 / 24(不变) |
| MuJoCo actuator 数 | 24(不变) |
根朝向及所有 24 个关节改变后再次验证上述坐标关系,不仅验证绑定初始姿态。
`rp1-visuals.spec.js` 还在同一会话、同一几何体布局和相同输入下,逐项**精确比较**替换前/后 walk / wave / dance 的 qpos、qvel、ctrl、观测、lastAction、状态机及参考帧;无任何容差放宽。部分 GLB 加载失败时旧机器人继续保留,释放候选资源后可直接 retry 成功。
## 测试与限制
- 新增三个纯逻辑/资源测试与三个真实浏览器测试;首轮全部通过。
- 所有 PBR 参数及贴图遵循用户原包,不另行重着色或简化。约 93 MB GLB、约 113 万三角形,首次加载量和 GPU 负担增加;不声称在所有设备具有相同帧率。
- 资源未附许可,继续仅供本地评估;没有推断公开再分发授权,也没有公开发布。
## 最终执行结果
- `npm test`:**66/66 通过**。
- `npm run validate:packages`:四个 Package 均通过;RP1 为 58 条资源路径,原始物理/策略及新增 GLB 的来源哈希全部通过。
- `npm run format:check`、`npm run build`、Python 语法检查、`git diff --check`:通过。
- `npm run test:e2e`:**55/55 通过**(20.3 分钟)。含原有所有动作、共享环境、实时控制频率、新材质、加载失败恢复、20 次资源释放检查。
- 20 次机器人切换时,RP1 GPU 资源保持 **125 geometries / 80 textures / 21 listeners**,各次完全相同;旧 model/data 均删除,未增加资源泄漏容差。该用例冷加载约三分钟,为避免新 93 MB 材质包在慢设备上贴近旧超时上限,测试时限单独由 180 秒调为 240 秒,资源断言原样保留。
- 最终 `dist/` 生产包复验:`TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- rp1-visuals shared-environment deployment --output=/tmp/rp1-material-production-artifacts`,**7/7 通过**(3.9 分钟)。再次验证坐标/材质、替换前后逐项数值一致性、错误恢复、三机器人共享环境以及静态资源路径。
- 人工查看前侧材质截图:蓝白灰配色、脸部屏幕、发光标识和双手外观正常;截图为 `/tmp/rp1-material-production-artifacts/rp1-visuals-all-63-RP1-mat-8a18f-ith-original-shapes-aligned/rp1-material-front.png`。对应开发测试截图亦在 `app/test-results/rp1-visuals-all-63-RP1-mat-8a18f-ith-original-shapes-aligned/rp1-material-front.png`。
日志:`/tmp/rp1-visuals-unit.log`、`/tmp/rp1-material-full-e2e.log`、`/tmp/rp1-material-production-e2e.log`;全量 JSON 结果另存 `/tmp/rp1-material-full-results.json`。
@@ -0,0 +1,36 @@
# 所有机器人共享环境
## 当前规则
- Microduck、Microban、RP1 使用相同的补光、环境贴图亮度、背景、网格线色板和漫画式场景显示。
- **Line color** 在每个机器人界面都可用,仍为原四种色块及原按钮样式;颜色为全局会话设置,从任意机器人更改后切换其他机器人不会恢复成各自旧颜色。
- 同一浏览器会话只随机生成一套 **5~7 个**交互几何体。各机器人的种类、色彩、旋转及相对布局相同,保留原先的尺寸适配:4.5 m / 5.5 m / 22 m 场地按机器人尺度缩放,几何体大小与坐标同步缩放。质量按尺度平方增长,保持大机器人场景中的物体轻便可推。
- 默认出生点 `(-0.6, 0)` 和 `(0, 0)` 均预留净空。不同机器人有独立的物理仿真,切换恢复这套共同的初始布局,不跨不同尺度仿真搬运实时物体速度。刷新页面才重新随机布局。
- 所有机器人都可与几何体真实碰撞、推移;几何体也与地面、墙壁及其他几何体碰撞。鼠标拖动物体或机器人施加物理弹簧力;拖动空白区域仍然旋转相机。暂停、重置、切换及资源清理时释放抓取力。
- 仅场景控制共用:Microduck 特有的 Feet / Rollers 按钮不会出现在 Microban / RP1 上;机器人动作、策略、关节控制和尺寸缩放契约不变。
- 显式无边界诊断 World 不添加干扰物;历史 Microduck 动作 oracle 仍保持原始实验模型,以便继续精确对比九份未修改的轨迹基线。
## 关键实现
- `app/src/game/worlds/arena/session-environment.js`:会话共享模板及统一 MuJoCo 几何体注入。
- `app/src/game/interactive-props.js`:统一几何体形状、材质、刚体状态同步和资源清理;生成器支持多个出生点净空。
- `app/src/platform/sim/{SceneComposer,SimCore}.js`:generic 机器人同样包含交互刚体,重置、越界恢复和显式测试隔离。
- `app/src/platform/runtime/{RobotRuntime,RuntimeManager}.js`:统一线条颜色、几何体渲染、设置传递、物理拖拽和资源所有权。
- `app/src/platform/input/PhysicsDrag.js`:generic 机器人/几何体的真实拖拽及每个物理子步施力。
- `app/src/game/worlds/arena/presentation.js`、`app/src/ui/Hud.jsx`:所有机器人共用地面/围墙调色入口,机器人特有操作仍按 capability 控制。
## 新增验证
- `app/test/session-environment.test.js`:三个机器人比例使用同一模板;坐标、几何体尺寸按同一尺度变化;避开两个出生点;返回值相互隔离、不污染共享模板;诊断 World 不添加物体。
- `app/e2e/shared-environment.spec.js`:跨 Microduck → Microban → RP1 → Microduck,检查同一几何体数量、形状、颜色、归一化布局、光源及环境亮度,验证全局颜色继承和机器人特有按钮边界。
- 同一浏览器用例验证 Microban 和 RP1 的真实围墙碰撞、鼠标施力拖拽、暂停清除抓取力和重置布局。
## 执行结果
- `cd app && npm test`:**63/63 通过**。
- `npm run format:check`、`npm run validate:packages`、`npm run build`、`git diff --check`:通过。
- `npm run test:e2e`:**52/52 通过**(15.0 分钟)。没有为障碍物放宽步行/技能断言;Microban 长序列步行、RP1 行走/舞蹈/挥手、九项 Microduck 精确历史回归、新场景交互、两分钟运动稳定性、20 次切换资源释放均通过。
- 对最终生产包再次执行 `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- shared-environment scene-interaction deployment microduck-regression --output=/tmp/shared-environment-production-artifacts`:**16/16 通过**(3.1 分钟)。其中共享颜色测试还覆盖在 RP1 选择 Lavender 后返回 Microduck,验证不会被 Microduck 的旧设置覆盖。
- 人工查看生产入口的 Microban / RP1 截图,确认两者均显示相同网格、原四格色板及同布局的彩色几何体,机器人自身呈现不变。截图位于 `/tmp/shared-environment-production-artifacts/deployment-deployed-assets-19ea7-l-three-Robot-Packages-boot/`。
日志:`/tmp/shared-environment-unit.log`、`/tmp/shared-environment-full-e2e.log`、`/tmp/shared-environment-production-e2e.log`;全量浏览器 JSON 结果另存于 `/tmp/shared-environment-full-results.json`。