Files

153 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Microduck Sandbox
emoji: 🐤
colorFrom: yellow
colorTo: gray
sdk: docker
app_port: 8080
pinned: false
license: apache-2.0
---
# Browser robot simulation platform
MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), **Microban**, **RP1**, **Booster T1**, and **Apptronik Apollo** Robot Packages. Physics and ONNX inference run locally in the browser; no simulation backend is required.
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
## v0.6
- Added Booster T1 and Apptronik Apollo packages with browser-native ONNX locomotion and WASD control.
- Apollo uses the explicitly selected upright-v3 `model_9999.pt`, including its calibrated stand pose, spawn height and fall threshold.
- Added body-frame linear-velocity observations and explicit World floor collision settings for imported models.
- Reproducible import scripts, source hashes and browser regression tests accompany both new robots.
- Verified: 68 unit tests, 19 targeted browser regression tests (all five robot families), package/hash validation, formatting and production build. See [v0.6 verification](docs/robots/v0.6-verification.md).
## v0.5
- Brighter shared lighting and the original four-swatch **Line color** control for every robot.
- One session-wide layout of 5–7 physical, draggable geometries replaces the football and scales consistently across robots.
- RP1 uses the supplied textured GLBs, including fixed-hand detail, without changing its 24-DOF physics or trained skills.
- Verified with 66 unit tests, 55 browser tests and production-build checks. See [shared environment](docs/superpowers/verification/shared-robot-environment.md) and [RP1 materials](docs/superpowers/verification/rp1-glb-materials.md) verification records.
## Robots
| Robot | Default scene / presentation | Controls |
|---|---|---|
| Microduck | Shared 4.5 × 4.5 m bounded arena, 5–7 random interactive geometries, original GLB/comic effects | Existing walking, skating, skills and recovery |
| Microban | Shared robot-scaled 5.5 × 5.5 m arena, interactive geometries, textured per-body GLB visuals | Walking policy + independent head rotation |
| RP1 | Shared robot-scaled 22 × 22 m arena, interactive geometries, user-supplied textured GLB visuals on original 24-DOF physics | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
**Booster T1** is also available in the Robot selector: W/S walks forward/backward, A/D turns, Space resets, Escape pauses, and C restores the chase camera. It uses the latest checkpoint at import (`model_6700.pt`, run `2026-11-02_15-03-39_t1_flat`) via ONNX, with original 23-DOF physics and STL visuals. See [T1 provenance, re-import and verification](app/public/robots/t1/SOURCE.md).
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's scene-line colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban physics / policy assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); its textured visual replacement comes from the user-supplied `microban2.zip`. Exact hashes, dirty-worktree provenance and license caveats are included in its [Package source notes](app/public/robots/microban/SOURCE.md).
### Shared scene controls (all robots)
The scene has brighter ambient / environment illumination plus a hemisphere fill light. The lower-left **Line color** palette retains the original Cream / Graphite / Lavender / Sky swatches and square comic buttons, but now changes **floor and wall grid lines**, not robot materials. The same palette is available for every robot, and its selection is global across robot switches, reset and feet / rollers changes.
The football is removed. One browser session randomly generates **5–7 lightweight boxes, spheres, cylinders and capsules** shared by **all robots**, clear of both bundled spawn anchors and each other. Their types, colours and relative layout stay identical when switching robots; geometry and positions scale with the existing arena-to-robot ratio. They have real MuJoCo free joints: every robot can push them, they collide with one another, the floor and walls, and the mouse can grab / lift them using spring forces. Reset, locomotion changes and robot switches restore the same session layout; refreshing generates a new layout. Escaped toys are returned individually. This shares environment configuration/layout, not live object momentum between different-scale robot simulations.
## Keyboard and mouse
| Action | Binding |
|---|---|
| Forward / backward / turn | Arrow keys or physical WASD positions (layout labels adapt when supported) |
| Pause / continue | Esc or toolbar |
| Reset current robot | Space or toolbar |
| Orbit / zoom | Mouse drag / wheel |
| Microduck left / right kick | Q / E |
| Microduck alternate kick | F |
| Microduck sit / stand; roller crouch | R |
| Microduck ground pick | G |
| Microduck forward roll; roller crouch | T |
| Microduck feet / rollers | M or quickbar |
| Restore rear chase camera (all robots) | C |
| Microban independent head rotation | H |
| RP1 trained dance (10 s segment) / wave (4.36 s) | J / H |
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Dragging any robot or interactive geometry applies physical forces; dragging empty space orbits the camera, consistently for all robots. Gamepad / touch are not part of this release's acceptance scope.
## Microban fidelity and limits
- Actual ONNX interface: `obs [1, 63]` → `actions [1, 18]`.
- The model has 19 actuated joints; the policy controls 18 body joints, **not head**. Metadata's 19 names are resolved by name, not mistaken for actor output order.
- Projected gravity uses the trunk frame; the gyro retains the original IMU-site frame. Current policy has no extra reference-phase input.
- Physics timestep: 0.005 s; control decimation: 4 (50 Hz).
- **Simplified actuation**, not BAM: joint PD at every physics substep, kp=0.3, kd=0.02, torque limit=0.65 Nm.
- Default commands: forward 0.15 m/s, backward −0.1 m/s, turn ±0.6 rad/s. These are commanded, not guaranteed measured tracking speeds. Drift remains possible.
- Head follows the source's 0.35 Hz / ±π/4 sinusoid, with 0.5 s start / stop blending. It does not restart the walking controller.
- Falling stops policy updates and requests manual reset. There is no automatic stand-up policy for Microban.
BAM battery / friction / delay modeling, Placo squat, training, hardware control, ROS, camera observations, external Packages and online sharing are **not implemented** in this release. Ghost networking, stickers and celebration / entrance sequences are deferred. Audio retains the existing `SOUND_DISABLED=true` master setting; simulation never waits for an audio-unlock gesture.
## Apptronik Apollo
Select **Apptronik Apollo** in the Robot menu. **W/S** walks forward/backward, **A/D** turns, **Space** resets, **Escape** pauses, and **C** restores the chase camera. Releasing movement keys stops the command. The package uses the explicitly selected checkpoint `apollo_flat_upright_v3/2026-11-02_17-39-21_apollo_upright_v3/model_9999.pt` (32 actions, 108 observations), its calibrated upright stand pose, original STL visuals and native position actuators. No scripted motion is substituted. See [Apollo source, re-import and verification](app/public/robots/apollo/SOURCE.md).
## RP1 trained skills
Walking uses final `model_9000.pt` from `rp1_24dof_flat/2026-10-28_15-47-49`, float32 `[1,243]` (three term-major history frames) → `[1,24]`. Dance uses resumed `2026-10-29_11-05-14_dance_resume/model_23998.pt`; wave retains its separate final `model_7999.pt`. Their actual actor branch is **student**, float32 `[1,126]` → `[1,24]`, with joint-reference position/velocity commands. All exports include the empirical normalizer and use their own reference defaults, named joint order and action history.
Independent ONNX skills share `load/reset/run/dispose` and an exclusive full-body FSM: walk → settle → blend in → play → blend back to walk. Release movement keys before requesting a skill; movement interrupts it. Reference playback is simulation-clocked, freezes on pause, restarts on each activation, and never writes kinematic poses. **Wave plays its complete 4.36 s clip. Dance exposes only the validated first 10 s**: the previous dance checkpoint fell near 20 s during full 131.46 s playback; full-length playback of the updated checkpoint has not been qualified. These are trained policies, not scripted motion substitutes. Completed training does not imply exact training-simulator or hardware equivalence. See [RP1 integration and weight updates](docs/robots/rp1.md) and [completed-skills verification](docs/superpowers/verification/2026-10-29-rp1-completed-skills.md), plus the [23998 dance update](docs/superpowers/verification/2026-10-29-rp1-dance-23998.md).
**RP1 description asset licensing is not supplied in the upstream checkout. Local evaluation only; confirm rights before public deployment/redistribution.** Provenance and caveats are recorded in [the package SOURCE.md](app/public/robots/rp1/SOURCE.md).
## Development
Binary robot meshes, ONNX policies and `.f32` motion samples use Git LFS:
```bash
git lfs install
git lfs pull
cd app
npm ci
npm run dev # http://localhost:5173
npm run build # app/dist/
npm run preview # production bundle
```
Use a Node version supported by Vite (this revision was tested with Node 24). The Dockerfile builds with Node 22 and serves the static bundle through nginx on port 8080; a Docker build itself is not part of the local browser verification.
### Verification
```bash
cd app
npm test # pure logic tests
npm run validate:packages # schemas, World, paths, source hashes
npx playwright install chromium
npm run test:e2e # real browser / WASM + UI + regression + ownership tests
npm run test:regression # accepted Microduck golden trajectories only
npm run format:check
npm audit
```
`npm run baseline:record` writes **candidates** under `app/test-results/baseline-candidates/`; it never overwrites accepted fixtures. Review changes before accepting any new baseline. See [baseline provenance](app/test/fixtures/microduck/README.md).
For a running production server, set `TEST_BASE_URL` and run the product-facing tests, e.g. `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- chase-camera deployment simulation-shell robot-switching microban-walking microban-observation microduck-regression runtime-smoke`. Development-only fixture pages are deliberately not shipped in `dist/`.
For prefixed hosting, build and preview with `VITE_BASE=/play/`; `DEPLOY_PATH=/play/` selects that entry path for `deployment.spec.js`. Private HF authentication behavior beyond the retained same-origin asset signing has not been independently deployment-tested.
`?test=1` is an internal deterministic-step hook, not a normal launch mode. `?boot=1` is no longer required. Legacy `?move`, `?policy` and `?session` requests remain limited to the Microduck adapter; they do not load arbitrary Robot Packages or replace Microban's controller.
## Architecture and extension
- `app/public/robots/*/robot.json`: model, joint groups, observations, actions, skills, bindings and presentation.
- `app/public/worlds/`: arena / flat World manifests.
- `app/src/platform/`: schemas / loading, SimCore, observation and action pipelines, term registries, ONNX, PD backend, overlay skills, input, MJCF renderer and transactional runtime manager.
- `app/src/robots/registry.js`: trusted bundled adapters; no remote JS execution.
- `app/src/robots/microduck/runtime.js`: compatibility adapter retaining original skill / recovery quirks and GLB presentation while using shared runtimes, scheduler, input, observation / action mapping and actuation.
- `app/src/game/worlds/arena/`: arena physics composition and rendering. `game/game.js` and `game/arena.js` are compatibility entry points, not generic engines.
A robot expressible with existing capabilities can be added by a Package and index entry, without a robot-id branch in the platform. Current generic scope is floating-base robots with scalar joint groups, unit-gear actuators, supported proprioceptive / joint-reference terms, ONNX primary control, independent episodic policies, simulation-clocked motion references, position targets and optional sinusoidal overlays. Unsupported capabilities are rejected, not silently simulated incorrectly. New capabilities require explicit registry / schema work.
See [Package authoring](docs/robots/package-authoring.md), [Microban details](docs/robots/microban.md), [approved design](docs/superpowers/specs/2026-10-27-multi-robot-platform-design.md), [implementation plan](docs/superpowers/plans/2026-10-27-multi-robot-platform-implementation.md) and [initial verification](docs/superpowers/verification/2026-10-27-multi-robot-platform.md) and [style/camera verification](docs/superpowers/verification/2026-10-28-style-and-chase-camera.md), plus [arena-style verification](docs/superpowers/verification/2026-10-28-arena-presentation.md) and [initial bounded-default verification](docs/superpowers/verification/2026-10-28-default-bounded-arena.md). Current size-adaptive behavior is documented in [robot-scaled arena verification](docs/superpowers/verification/2026-10-28-robot-scaled-arena.md).
## License
The original application code is [Apache-2.0](LICENSE). Bundled third-party models, meshes, policies and samples retain their respective licenses; do **not** assume that every asset is Apache-2.0. Microban's copied XML retains its Apache-2.0 notice, and its upstream repository's mixed-license text, provenance and additional Apache notice are included in [`app/public/robots/microban/`](app/public/robots/microban/). Review those file-specific terms before redistribution. The bundled Anton font is SIL OFL-1.1; its notice is included at [`app/public/licenses/Anton-OFL.txt`](app/public/licenses/Anton-OFL.txt).