Files
VibemakerSandbox/README.md
T

126 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Microduck Sandbox
emoji: 🐤
colorFrom: yellow
colorTo: gray
sdk: docker
app_port: 8080
pinned: false
license: apache-2.0
---
# Browser robot simulation platform
MuJoCo WASM + onnxruntime-web + three.js, with **Microduck** (legs / rollers), **Microban**, and **RP1** Robot Packages. Physics and ONNX inference run locally in the browser; no simulation backend is required.
The application opens directly into simulation, defaults to Microduck, and keeps pause, reset, robot selection and a collapsible keyboard reference on that same screen. There is no welcome / RESUME menu or BIOS animation. Anton is bundled locally, so the default interface does not wait for an external font service.
## Robots
| Robot | Default scene / presentation | Controls |
|---|---|---|
| Microduck | Shared 4.5 × 4.5 m bounded arena, ball, original GLB/palette/comic effects | Existing walking, skating, skills and recovery |
| Microban | Robot-scaled 5.5 × 5.5 m arena, dark/orange grid and comic UI, textured per-body GLB visuals | Walking policy + independent head rotation |
| RP1 | Robot-scaled 22 × 22 m arena/comic UI, original 24-DOF MJCF visuals | Completed walking policy; trained 10 s dance segment (J) and full 4.36 s wave (H) |
Switch robots using **Robot** in the toolbar. Returning to Microduck restores the session's colour and feet / rollers choice, but initializes a fresh physical state. Failed loads keep the previous robot available and show an inline retry action.
Microduck policies / assets originate from [pollen-robotics/microduck](https://github.com/pollen-robotics/microduck) and [pollen-robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl). Microban physics / policy assets originate from [Rhoban/microban](https://github.com/Rhoban/microban); its textured visual replacement comes from the user-supplied `microban2.zip`. Exact hashes, dirty-worktree provenance and license caveats are included in its [Package source notes](app/public/robots/microban/SOURCE.md).
## Keyboard and mouse
| Action | Binding |
|---|---|
| Forward / backward / turn | Arrow keys or physical WASD positions (layout labels adapt when supported) |
| Pause / continue | Esc or toolbar |
| Reset current robot | Space or toolbar |
| Orbit / zoom | Mouse drag / wheel |
| Microduck left / right kick | Q / E |
| Microduck alternate kick | F |
| Microduck sit / stand; roller crouch | R |
| Microduck ground pick | G |
| Microduck forward roll; roller crouch | T |
| Microduck feet / rollers | M or quickbar |
| Restore rear chase camera (all robots) | C |
| Microban independent head rotation | H |
| RP1 trained dance (10 s segment) / wave (4.36 s) | J / H |
The on-screen reference comes from the same Package bindings as the keyboard, and its skill entries are also clickable buttons. Text fields / selectors retain their normal keyboard behavior. Focused buttons retain native Space / Enter behavior without also resetting the robot. Losing focus, pausing, resetting or switching clears held movement keys.
All bundled robots, and future Packages omitting `presentation.world`, use the same **robot-scaled arena template**. Microduck remains at **4.5 × 4.5 m**, with 0.25 m high / 0.05 m thick collision walls. World lengths scale by `dimensions.height / 0.27`: Microban gets **5.5 × 5.5 m**, walls ≈0.306 m high; RP1 gets **22 × 22 m**, walls ≈1.222 m high. Wall thickness, visual geometry, grid spacing and fade distances scale together, preserving Microduck's scene-to-body proportions. Nominal standing height is fixed per package, not inferred from the live pose or root spawn height; missing dimensions retain the reference scale. Interiors stay flat; no relief or automatic coordinate clamping is added. Explicit `world: "flat"` remains available for generic-runtime diagnostics. All robots start with the camera behind the body. Mouse orbit detaches chase mode; **C** restores rear following and the default zoom, including while paused (pressing C again does not detach). Microduck mouse dragging still applies physical forces to the robot / ball. Microban / RP1 mouse dragging orbits the camera. Gamepad / touch are not part of this release's acceptance scope.
## Microban fidelity and limits
- Actual ONNX interface: `obs [1, 63]` → `actions [1, 18]`.
- The model has 19 actuated joints; the policy controls 18 body joints, **not head**. Metadata's 19 names are resolved by name, not mistaken for actor output order.
- Projected gravity uses the trunk frame; the gyro retains the original IMU-site frame. Current policy has no extra reference-phase input.
- Physics timestep: 0.005 s; control decimation: 4 (50 Hz).
- **Simplified actuation**, not BAM: joint PD at every physics substep, kp=0.3, kd=0.02, torque limit=0.65 Nm.
- Default commands: forward 0.15 m/s, backward −0.1 m/s, turn ±0.6 rad/s. These are commanded, not guaranteed measured tracking speeds. Drift remains possible.
- Head follows the source's 0.35 Hz / ±π/4 sinusoid, with 0.5 s start / stop blending. It does not restart the walking controller.
- Falling stops policy updates and requests manual reset. There is no automatic stand-up policy for Microban.
BAM battery / friction / delay modeling, Placo squat, training, hardware control, ROS, camera observations, external Packages and online sharing are **not implemented** in this release. Ghost networking, stickers and celebration / entrance sequences are deferred. Audio retains the existing `SOUND_DISABLED=true` master setting; simulation never waits for an audio-unlock gesture.
## RP1 trained skills
Walking uses final `model_9000.pt` from `rp1_24dof_flat/2026-10-28_15-47-49`, float32 `[1,243]` (three term-major history frames) → `[1,24]`. Dance uses resumed `2026-10-29_11-05-14_dance_resume/model_23998.pt`; wave retains its separate final `model_7999.pt`. Their actual actor branch is **student**, float32 `[1,126]` → `[1,24]`, with joint-reference position/velocity commands. All exports include the empirical normalizer and use their own reference defaults, named joint order and action history.
Independent ONNX skills share `load/reset/run/dispose` and an exclusive full-body FSM: walk → settle → blend in → play → blend back to walk. Release movement keys before requesting a skill; movement interrupts it. Reference playback is simulation-clocked, freezes on pause, restarts on each activation, and never writes kinematic poses. **Wave plays its complete 4.36 s clip. Dance exposes only the validated first 10 s**: the previous dance checkpoint fell near 20 s during full 131.46 s playback; full-length playback of the updated checkpoint has not been qualified. These are trained policies, not scripted motion substitutes. Completed training does not imply exact training-simulator or hardware equivalence. See [RP1 integration and weight updates](docs/robots/rp1.md) and [completed-skills verification](docs/superpowers/verification/2026-10-29-rp1-completed-skills.md), plus the [23998 dance update](docs/superpowers/verification/2026-10-29-rp1-dance-23998.md).
**RP1 description asset licensing is not supplied in the upstream checkout. Local evaluation only; confirm rights before public deployment/redistribution.** Provenance and caveats are recorded in [the package SOURCE.md](app/public/robots/rp1/SOURCE.md).
## Development
Binary robot meshes, ONNX policies and `.f32` motion samples use Git LFS:
```bash
git lfs install
git lfs pull
cd app
npm ci
npm run dev # http://localhost:5173
npm run build # app/dist/
npm run preview # production bundle
```
Use a Node version supported by Vite (this revision was tested with Node 24). The Dockerfile builds with Node 22 and serves the static bundle through nginx on port 8080; a Docker build itself is not part of the local browser verification.
### Verification
```bash
cd app
npm test # pure logic tests
npm run validate:packages # schemas, World, paths, source hashes
npx playwright install chromium
npm run test:e2e # real browser / WASM + UI + regression + ownership tests
npm run test:regression # accepted Microduck golden trajectories only
npm run format:check
npm audit
```
`npm run baseline:record` writes **candidates** under `app/test-results/baseline-candidates/`; it never overwrites accepted fixtures. Review changes before accepting any new baseline. See [baseline provenance](app/test/fixtures/microduck/README.md).
For a running production server, set `TEST_BASE_URL` and run the product-facing tests, e.g. `TEST_BASE_URL=http://127.0.0.1:5175 npm run test:e2e -- chase-camera deployment simulation-shell robot-switching microban-walking microban-observation microduck-regression runtime-smoke`. Development-only fixture pages are deliberately not shipped in `dist/`.
For prefixed hosting, build and preview with `VITE_BASE=/play/`; `DEPLOY_PATH=/play/` selects that entry path for `deployment.spec.js`. Private HF authentication behavior beyond the retained same-origin asset signing has not been independently deployment-tested.
`?test=1` is an internal deterministic-step hook, not a normal launch mode. `?boot=1` is no longer required. Legacy `?move`, `?policy` and `?session` requests remain limited to the Microduck adapter; they do not load arbitrary Robot Packages or replace Microban's controller.
## Architecture and extension
- `app/public/robots/*/robot.json`: model, joint groups, observations, actions, skills, bindings and presentation.
- `app/public/worlds/`: arena / flat World manifests.
- `app/src/platform/`: schemas / loading, SimCore, observation and action pipelines, term registries, ONNX, PD backend, overlay skills, input, MJCF renderer and transactional runtime manager.
- `app/src/robots/registry.js`: trusted bundled adapters; no remote JS execution.
- `app/src/robots/microduck/runtime.js`: compatibility adapter retaining original skill / recovery quirks and GLB presentation while using shared runtimes, scheduler, input, observation / action mapping and actuation.
- `app/src/game/worlds/arena/`: arena physics composition and rendering. `game/game.js` and `game/arena.js` are compatibility entry points, not generic engines.
A robot expressible with existing capabilities can be added by a Package and index entry, without a robot-id branch in the platform. Current generic scope is floating-base robots with scalar joint groups, unit-gear actuators, supported proprioceptive / joint-reference terms, ONNX primary control, independent episodic policies, simulation-clocked motion references, position targets and optional sinusoidal overlays. Unsupported capabilities are rejected, not silently simulated incorrectly. New capabilities require explicit registry / schema work.
See [Package authoring](docs/robots/package-authoring.md), [Microban details](docs/robots/microban.md), [approved design](docs/superpowers/specs/2026-10-27-multi-robot-platform-design.md), [implementation plan](docs/superpowers/plans/2026-10-27-multi-robot-platform-implementation.md) and [initial verification](docs/superpowers/verification/2026-10-27-multi-robot-platform.md) and [style/camera verification](docs/superpowers/verification/2026-10-28-style-and-chase-camera.md), plus [arena-style verification](docs/superpowers/verification/2026-10-28-arena-presentation.md) and [initial bounded-default verification](docs/superpowers/verification/2026-10-28-default-bounded-arena.md). Current size-adaptive behavior is documented in [robot-scaled arena verification](docs/superpowers/verification/2026-10-28-robot-scaled-arena.md).
## License
The original application code is [Apache-2.0](LICENSE). Bundled third-party models, meshes, policies and samples retain their respective licenses; do **not** assume that every asset is Apache-2.0. Microban's copied XML retains its Apache-2.0 notice, and its upstream repository's mixed-license text, provenance and additional Apache notice are included in [`app/public/robots/microban/`](app/public/robots/microban/). Review those file-specific terms before redistribution. The bundled Anton font is SIL OFL-1.1; its notice is included at [`app/public/licenses/Anton-OFL.txt`](app/public/licenses/Anton-OFL.txt).