153 lines
16 KiB
Markdown
153 lines
16 KiB
Markdown
---
|
||
title: Microduck Sandbox
|
||
emoji: 🐤
|
||
colorFrom: yellow
|
||
colorTo: gray
|
||
sdk: docker
|
||
app_port: 8080
|
||
pinned: false
|
||
license: apache-2.0
|
||
---
|
||
|
||
# Browser robot simulation platform
|
||
|
||
MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), **Microban**, **RP1**, **Booster T1**, and **Apptronik Apollo** Robot Packages. Physics and ONNX inference run locally in the browser; no simulation backend is required.
|
||
|
||
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
|
||
|
||
## v0.6
|
||
|
||
- Added Booster T1 and Apptronik Apollo packages with browser-native ONNX locomotion and WASD control.
|
||
- Apollo uses the explicitly selected upright-v3 `model_9999.pt`, including its calibrated stand pose, spawn height and fall threshold.
|
||
- Added body-frame linear-velocity observations and explicit World floor collision settings for imported models.
|
||
- Reproducible import scripts, source hashes and browser regression tests accompany both new robots.
|
||
- Verified: 68 unit tests, 19 targeted browser regression tests (all five robot families), package/hash validation, formatting and production build. See [v0.6 verification](docs/robots/v0.6-verification.md).
|
||
|
||
## v0.5
|
||
|
||
- Brighter shared lighting and the original four-swatch **Line color** control for every robot.
|
||
- One session-wide layout of 5–7 physical, draggable geometries replaces the football and scales consistently across robots.
|
||
- RP1 uses the supplied textured GLBs, including fixed-hand detail, without changing its 24-DOF physics or trained skills.
|
||
- Verified with 66 unit tests, 55 browser tests and production-build checks. See [shared environment](docs/superpowers/verification/shared-robot-environment.md) and [RP1 materials](docs/superpowers/verification/rp1-glb-materials.md) verification records.
|
||
|
||
## Robots
|
||
|
||
| Robot | Default scene / presentation | Controls |
|
||
|---|---|---|
|
||
| Microduck | Shared 4.5 × 4.5 m bounded arena, 5–7 random interactive geometries, original GLB/comic effects | Existing walking, skating, skills and recovery |
|
||
| Microban | Shared robot-scaled 5.5 × 5.5 m arena, interactive geometries, textured per-body GLB visuals | Walking policy + independent head rotation |
|
||
| RP1 | Shared robot-scaled 22 × 22 m arena, interactive geometries, user-supplied textured GLB visuals on original 24-DOF physics | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
|
||
|
||
**Booster T1** is also available in the Robot selector: W/S walks forward/backward, A/D turns, Space resets, Escape pauses, and C restores the chase camera. It uses the latest checkpoint at import (`model_6700.pt`, run `2026-11-02_15-03-39_t1_flat`) via ONNX, with original 23-DOF physics and STL visuals. See [T1 provenance, re-import and verification](app/public/robots/t1/SOURCE.md).
|
||
|
||
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's scene-line colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
|
||
|
||
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban physics / policy assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); its textured visual replacement comes from the user-supplied `microban2.zip`. Exact hashes, dirty-worktree provenance and license caveats are included in its [Package source notes](app/public/robots/microban/SOURCE.md).
|
||
|
||
### Shared scene controls (all robots)
|
||
|
||
The scene has brighter ambient / environment illumination plus a hemisphere fill light. The lower-left **Line color** palette retains the original Cream / Graphite / Lavender / Sky swatches and square comic buttons, but now changes **floor and wall grid lines**, not robot materials. The same palette is available for every robot, and its selection is global across robot switches, reset and feet / rollers changes.
|
||
|
||
The football is removed. One browser session randomly generates **5–7 lightweight boxes, spheres, cylinders and capsules** shared by **all robots**, clear of both bundled spawn anchors and each other. Their types, colours and relative layout stay identical when switching robots; geometry and positions scale with the existing arena-to-robot ratio. They have real MuJoCo free joints: every robot can push them, they collide with one another, the floor and walls, and the mouse can grab / lift them using spring forces. Reset, locomotion changes and robot switches restore the same session layout; refreshing generates a new layout. Escaped toys are returned individually. This shares environment configuration/layout, not live object momentum between different-scale robot simulations.
|
||
|
||
## Keyboard and mouse
|
||
|
||
| Action | Binding |
|
||
|---|---|
|
||
| Forward / backward / turn | Arrow keys or physical WASD positions (layout labels adapt when supported) |
|
||
| Pause / continue | Esc or toolbar |
|
||
| Reset current robot | Space or toolbar |
|
||
| Orbit / zoom | Mouse drag / wheel |
|
||
| Microduck left / right kick | Q / E |
|
||
| Microduck alternate kick | F |
|
||
| Microduck sit / stand; roller crouch | R |
|
||
| Microduck ground pick | G |
|
||
| Microduck forward roll; roller crouch | T |
|
||
| Microduck feet / rollers | M or quickbar |
|
||
| Restore rear chase camera (all robots) | C |
|
||
| Microban independent head rotation | H |
|
||
| RP1 trained dance (10 s segment) / wave (4.36 s) | J / H |
|
||
|
||
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
|
||
|
||
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Dragging any robot or interactive geometry applies physical forces; dragging empty space orbits the camera, consistently for all robots. Gamepad / touch are not part of this release's acceptance scope.
|
||
|
||
## Microban fidelity and limits
|
||
|
||
- Actual ONNX interface: `obs [1, 63]` → `actions [1, 18]`.
|
||
- The model has 19 actuated joints; the policy controls 18 body joints, **not head**. Metadata's 19 names are resolved by name, not mistaken for actor output order.
|
||
- Projected gravity uses the trunk frame; the gyro retains the original IMU-site frame. Current policy has no extra reference-phase input.
|
||
- Physics timestep: 0.005 s; control decimation: 4 (50 Hz).
|
||
- **Simplified actuation**, not BAM: joint PD at every physics substep, kp=0.3, kd=0.02, torque limit=0.65 Nm.
|
||
- Default commands: forward 0.15 m/s, backward −0.1 m/s, turn ±0.6 rad/s. These are commanded, not guaranteed measured tracking speeds. Drift remains possible.
|
||
- Head follows the source's 0.35 Hz / ±π/4 sinusoid, with 0.5 s start / stop blending. It does not restart the walking controller.
|
||
- Falling stops policy updates and requests manual reset. There is no automatic stand-up policy for Microban.
|
||
|
||
BAM battery / friction / delay modeling, Placo squat, training, hardware control, ROS, camera observations, external Packages and online sharing are **not implemented** in this release. Ghost networking, stickers and celebration / entrance sequences are deferred. Audio retains the existing `SOUND_DISABLED=true` master setting; simulation never waits for an audio-unlock gesture.
|
||
|
||
## Apptronik Apollo
|
||
|
||
Select **Apptronik Apollo** in the Robot menu. **W/S** walks forward/backward, **A/D** turns, **Space** resets, **Escape** pauses, and **C** restores the chase camera. Releasing movement keys stops the command. The package uses the explicitly selected checkpoint `apollo_flat_upright_v3/2026-11-02_17-39-21_apollo_upright_v3/model_9999.pt` (32 actions, 108 observations), its calibrated upright stand pose, original STL visuals and native position actuators. No scripted motion is substituted. See [Apollo source, re-import and verification](app/public/robots/apollo/SOURCE.md).
|
||
|
||
## RP1 trained skills
|
||
|
||
Walking uses final `model_9000.pt` from `rp1_24dof_flat/2026-10-28_15-47-49`, float32 `[1,243]` (three term-major history frames) → `[1,24]`. Dance uses resumed `2026-10-29_11-05-14_dance_resume/model_23998.pt`; wave retains its separate final `model_7999.pt`. Their actual actor branch is **student**, float32 `[1,126]` → `[1,24]`, with joint-reference position/velocity commands. All exports include the empirical normalizer and use their own reference defaults, named joint order and action history.
|
||
|
||
Independent ONNX skills share `load/reset/run/dispose` and an exclusive full-body FSM: walk → settle → blend in → play → blend back to walk. Release movement keys before requesting a skill; movement interrupts it. Reference playback is simulation-clocked, freezes on pause, restarts on each activation, and never writes kinematic poses. **Wave plays its complete 4.36 s clip. Dance exposes only the validated first 10 s**: the previous dance checkpoint fell near 20 s during full 131.46 s playback; full-length playback of the updated checkpoint has not been qualified. These are trained policies, not scripted motion substitutes. Completed training does not imply exact training-simulator or hardware equivalence. See [RP1 integration and weight updates](docs/robots/rp1.md) and [completed-skills verification](docs/superpowers/verification/2026-10-29-rp1-completed-skills.md), plus the [23998 dance update](docs/superpowers/verification/2026-10-29-rp1-dance-23998.md).
|
||
|
||
**RP1 description asset licensing is not supplied in the upstream checkout. Local evaluation only; confirm rights before public deployment/redistribution.** Provenance and caveats are recorded in [the package SOURCE.md](app/public/robots/rp1/SOURCE.md).
|
||
|
||
## Development
|
||
|
||
Binary robot meshes, ONNX policies and `.f32` motion samples use Git LFS:
|
||
|
||
```bash
|
||
git lfs install
|
||
git lfs pull
|
||
cd app
|
||
npm ci
|
||
npm run dev # http://localhost:5173
|
||
npm run build # app/dist/
|
||
npm run preview # production bundle
|
||
```
|
||
|
||
Use a Node version supported by Vite (this revision was tested with Node 24). The Dockerfile builds with Node 22 and serves the static bundle through nginx on port 8080; a Docker build itself is not part of the local browser verification.
|
||
|
||
### Verification
|
||
|
||
```bash
|
||
cd app
|
||
npm test # pure logic tests
|
||
npm run validate:packages # schemas, World, paths, source hashes
|
||
npx playwright install chromium
|
||
npm run test:e2e # real browser / WASM + UI + regression + ownership tests
|
||
npm run test:regression # accepted Microduck golden trajectories only
|
||
npm run format:check
|
||
npm audit
|
||
```
|
||
|
||
`npm run baseline:record` writes **candidates** under `app/test-results/baseline-candidates/`; it never overwrites accepted fixtures. Review changes before accepting any new baseline. See [baseline provenance](app/test/fixtures/microduck/README.md).
|
||
|
||
For a running production server, set `TEST_BASE_URL` and run the product-facing tests, e.g. `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- chase-camera deployment simulation-shell robot-switching microban-walking microban-observation microduck-regression runtime-smoke`. Development-only fixture pages are deliberately not shipped in `dist/`.
|
||
|
||
For prefixed hosting, build and preview with `VITE_BASE=/play/`; `DEPLOY_PATH=/play/` selects that entry path for `deployment.spec.js`. Private HF authentication behavior beyond the retained same-origin asset signing has not been independently deployment-tested.
|
||
|
||
`?test=1` is an internal deterministic-step hook, not a normal launch mode. `?boot=1` is no longer required. Legacy `?move`, `?policy` and `?session` requests remain limited to the Microduck adapter; they do not load arbitrary Robot Packages or replace Microban's controller.
|
||
|
||
## Architecture and extension
|
||
|
||
- `app/public/robots/*/robot.json`: model, joint groups, observations, actions, skills, bindings and presentation.
|
||
- `app/public/worlds/`: arena / flat World manifests.
|
||
- `app/src/platform/`: schemas / loading, SimCore, observation and action pipelines, term registries, ONNX, PD backend, overlay skills, input, MJCF renderer and transactional runtime manager.
|
||
- `app/src/robots/registry.js`: trusted bundled adapters; no remote JS execution.
|
||
- `app/src/robots/microduck/runtime.js`: compatibility adapter retaining original skill / recovery quirks and GLB presentation while using shared runtimes, scheduler, input, observation / action mapping and actuation.
|
||
- `app/src/game/worlds/arena/`: arena physics composition and rendering. `game/game.js` and `game/arena.js` are compatibility entry points, not generic engines.
|
||
|
||
A robot expressible with existing capabilities can be added by a Package and index entry, without a robot-id branch in the platform. Current generic scope is floating-base robots with scalar joint groups, unit-gear actuators, supported proprioceptive / joint-reference terms, ONNX primary control, independent episodic policies, simulation-clocked motion references, position targets and optional sinusoidal overlays. Unsupported capabilities are rejected, not silently simulated incorrectly. New capabilities require explicit registry / schema work.
|
||
|
||
See [Package authoring](docs/robots/package-authoring.md), [Microban details](docs/robots/microban.md), [approved design](docs/superpowers/specs/2026-10-27-multi-robot-platform-design.md), [implementation plan](docs/superpowers/plans/2026-10-27-multi-robot-platform-implementation.md) and [initial verification](docs/superpowers/verification/2026-10-27-multi-robot-platform.md) and [style/camera verification](docs/superpowers/verification/2026-10-28-style-and-chase-camera.md), plus [arena-style verification](docs/superpowers/verification/2026-10-28-arena-presentation.md) and [initial bounded-default verification](docs/superpowers/verification/2026-10-28-default-bounded-arena.md). Current size-adaptive behavior is documented in [robot-scaled arena verification](docs/superpowers/verification/2026-10-28-robot-scaled-arena.md).
|
||
|
||
## License
|
||
|
||
The original application code is [Apache-2.0](LICENSE). Bundled third-party models, meshes, policies and samples retain their respective licenses; do **not** assume that every asset is Apache-2.0. Microban's copied XML retains its Apache-2.0 notice, and its upstream repository's mixed-license text, provenance and additional Apache notice are included in [`app/public/robots/microban/`](app/public/robots/microban/). Review those file-specific terms before redistribution. The bundled Anton font is SIL OFL-1.1; its notice is included at [`app/public/licenses/Anton-OFL.txt`](app/public/licenses/Anton-OFL.txt).
|