diff --git a/README.md b/README.md index 73745e5..ff70a75 100644 --- a/README.md +++ b/README.md @@ -13,14 +13,21 @@ Real trained RL policies for the Microduck robot, running fully in the browser: MuJoCo compiled to WebAssembly steps the physics, onnxruntime-web runs the policy network at 50 Hz. No server, no backend. +Two locomotion variants of the same robot are included: **legs** (walking, +the default) and **rollers** (the wheeled skating variant). Press `M` (or +hold D-pad up ~1 s on a gamepad) to switch; the roller model, meshes and +policies are lazy-loaded on the first switch. + ## Policies -| Button | Checkpoint | What it does | +| Mode | Checkpoint | What it does | |--------|-----------|--------------| -| Run | `BEST_alpha_walking.onnx` | Velocity-tracking locomotion (arrows / WASD to steer) | +| Run (legs) | `BEST_alpha_walking.onnx` | Velocity-tracking locomotion (arrows / WASD to steer) | | Sit | `BEST_alpha_sitstand.onnx` | Sits down on its hull, stands back up | | Roll | `roulade.onnx` | Rolls over and recovers | | Kick | `ball_kick_left.onnx` / `ball_kick_right.onnx` | Blind one-shot kick (0.5 s window, zeroed commands), left or right leg | +| Drive (rollers) | `BEST_roller.onnx` | Velocity-tracking skating on 4 passive wheels (higher top speed: 0.6 m/s) | +| Crouch (rollers) | `BEST_roller_crouch.onnx` | One-shot crouch-glide: sinks low over ~3.5 s and stands back up (phase-encoded command) | Policies and MJCF model from [apirrone/microduck_runtime](https://github.com/apirrone/microduck_runtime) @@ -29,14 +36,18 @@ and [apirrone/mjlab_microduck](https://github.com/apirrone/mjlab_microduck). ## Controls - Arrows or WASD (ZQSD): forward / back + turn -- Q / E (A / E on AZERTY): kick left / right -- R: roll (one barrel roll, then back to running) +- M: switch legs <-> rollers +- Q / E (A / E on AZERTY): kick left / right (legs only) +- R: roll (legs) / crouch-glide (rollers) - B: pop / respawn a kickable ball in front of the duck - C: toggle the chase camera (on by default; dragging detaches it) - Space: reset - Drag to orbit, scroll to zoom - Colour dots: repaint the duck (it quacks) +In roller mode the legs-only actions (kicks, sit) are disabled and their +hints fade out; play the ball by driving into it. + The ball is local-only: it lives in your tab's physics and is not shared with the multiplayer ghosts. A square arena (3 x 3 m) fences the play area so neither the ball nor the duck can wander off. @@ -48,10 +59,11 @@ Plug in a controller and the same mapping as the real robot runtime applies: - Left stick: forward / back + turn - Right stick: orbit the camera (detaches the chase cam) - R3 (right stick click): toggle the chase cam back on -- X: roll +- X: roll (legs) / crouch-glide (rollers) - Y: pop / respawn the ball -- RB / LB: right / left kick -- D-pad down: sit / stand toggle +- RB / LB: right / left kick (legs only) +- D-pad down: sit / stand toggle (legs only) +- D-pad up: hold ~1 s to switch legs <-> rollers (the real robot uses a 3 s hold) - Right trigger: mouth (analog) + quack ## Multiplayer ghosts @@ -60,16 +72,29 @@ Other people visiting the Space at the same time show up as translucent ducks, live. Peer-to-peer WebRTC via [Trystero](https://github.com/dmotz/trystero) (serverless signaling over public Nostr relays), so it works from a static Space with no backend. Each tab broadcasts its duck's pose (trunk + 14 -joints + jaw + colour) at 10 Hz; up to 3 ghosts are rendered, extra peers -stay connected but invisible. The camcorder-style OSD in the top-right -corner shows "N ONLINE" when peers are around. +joints + jaw + colour + locomotion variant) at 10 Hz; up to 3 ghosts are +rendered, extra peers stay connected but invisible. The camcorder-style OSD +in the top-right corner shows "N ONLINE" when peers are around. + +Ghost limitations in v1: a peer in roller mode renders with the roller rig +only if your tab has already loaded it (otherwise it falls back to the leg +rig), and ghost wheels don't spin (passive wheel joints aren't broadcast). +Old clients simply ignore the new variant flag. ## How it works -- `rl.js` fetches the MJCF (`robot_allcollisions.xml`), strips the visual - geoms, injects a floor and a STAND keyframe, and compiles it with the - official `@mujoco/mujoco` WASM bindings. -- The 61D observation (gyro, projected gravity, joint pos/vel, last action, - command) matches `mjlab_microduck/scripts/infer_policy.py`. -- Rendering is a three.js rig built from `kinematics.json` + decimated STL - meshes, driven directly from MuJoCo `qpos`. +- `rl.js` fetches the MJCF (`robot_allcollisions.xml`, or + `robot_allcollisions_rollers.xml` for the roller variant), strips the + visual geoms, injects a floor, arena walls, a ball and a STAND keyframe, + and compiles it with the official `@mujoco/mujoco` WASM bindings. +- Both variants share the exact same policy interface: 61D observation + (gyro, projected gravity, 14 joint pos/vel, last action, 13D command) + and 14 position-targets, matching `mjlab_microduck/scripts/infer_policy.py`. + The roller variant adds 4 passive wheel hinges that appear in `qpos` + (zeroed in the keyframe) but not in the observation. +- Rendering is a three.js rig built from `kinematics.json` / + `kinematics_rollers.json` + decimated STL meshes, driven directly from + MuJoCo `qpos` (including the passive wheel spin). +- Switching variants swaps the compiled model + data + rig + ONNX sessions + wholesale; both stay resident after the first load so toggling back and + forth is instant. diff --git a/ghosts.js b/ghosts.js index 4c9deb5..004202d 100644 --- a/ghosts.js +++ b/ghosts.js @@ -31,6 +31,10 @@ export async function initGhosts(env) { } const { scene, cloneRig, setJoint, setJawOpen, applyVariant, jointNames, getLocalState } = env; + // Locomotion-variant rig source: state.l 1 = rollers, 0/absent = legs + // (old clients never send it). Falls back to the leg rig when the local + // tab hasn't built the roller rig yet - known v1 limitation. + const rigFor = (l) => (env.getRigFor ? env.getRigFor(l ?? 0) : env.rig); // trystero 0.25 (backed by @trystero-p2p): makeAction returns // { send, onMessage, onReceiveProgress } where onMessage is a SETTER - // the receive handler is registered by assignment, not by calling it. @@ -60,7 +64,7 @@ export async function initGhosts(env) { const ghosts = new Map(); // peerId -> ghost const makeGhost = (state) => { - const rig = cloneRig(env.rig); + const rig = cloneRig(rigFor(state.l)); applyVariant(rig, state.v); ghostify(rig); scene.add(rig.placer); @@ -68,7 +72,10 @@ export async function initGhosts(env) { // Snap straight to the first received pose: no fly-in from the origin. trunk.position.set(state.p[0], state.p[1], state.p[2]); trunk.quaternion.set(state.p[4], state.p[5], state.p[6], state.p[3]); - return { rig, trunk, target: state, joints: state.j.slice(), jaw: state.w ?? 0, variant: state.v }; + return { + rig, trunk, target: state, joints: state.j.slice(), jaw: state.w ?? 0, + variant: state.v, loco: state.l ?? 0, + }; }; const removeGhost = (peerId) => { @@ -101,6 +108,16 @@ export async function initGhosts(env) { } g.lastSeen = performance.now(); g.target = state; + // Peer switched legs <-> rollers: rebuild its ghost on the other rig + // (cheap - cloneRig shares geometry). + if ((state.l ?? 0) !== g.loco) { + const seen = g.lastSeen; + removeGhost(peerId); + g = makeGhost(state); + g.lastSeen = seen; + ghosts.set(peerId, g); + return; + } if (state.v !== g.variant) { g.variant = state.v; applyVariant(g.rig, state.v); diff --git a/index.html b/index.html index d307e8a..da9e167 100644 --- a/index.html +++ b/index.html @@ -299,6 +299,22 @@ font-variant-numeric: tabular-nums; text-shadow: 0 0 6px rgba(255, 255, 255, 0.18); } + /* BIOS-style line while the roller variant streams in on first switch */ + #osd-load[hidden] { display: none; } + #osd-load { + font-size: 0.68rem; + letter-spacing: 0.14em; + color: var(--yellow, #ffd23f); + text-shadow: 0 0 8px rgba(255, 210, 63, 0.4); + } + .osd-load-cursor { + display: inline-block; + margin-left: 0.15em; + animation: rec-blink 0.8s steps(1) infinite; + } + /* Roller mode: legs-only actions (kicks, sit) are inert - fade their + hints; rl.js toggles body.rollers on every variant switch. */ + body.rollers .legs-only { opacity: 0.25; } @media (max-width: 700px) { #hud { flex-direction: column; align-items: flex-start; } #hud .keys { display: none; } @@ -588,6 +604,7 @@
Run
+ @@ -679,7 +697,7 @@
fresh start
-
drag to orbit · scroll to zoom
+
drag to orbit · scroll to zoom · M swaps legs for roller skates
@@ -746,7 +764,7 @@ appended: a plain src attribute can 401 when the browser blocks the *.static.hf.space auth cookie inside the hub iframe. -->