--
7f117fa7a7060b36687e7544e8d7393d14a1fa90 by Yuval Tassa <tassa@google.com>:
Handle clang AST loc nodes with elided 'line' in introspect codegen.
Apple's math.h (included directly by mujoco.h) contains an anonymous union
whose clang-JSON 'loc' lacks the 'line' key (clang elides it when unchanged
from the previous node), crashing AstProcessor._make_anonymous_key with a
KeyError during bindings regeneration on macOS. glibc's math.h has no such
union, so this never bites on Linux.
Fall back to an offset-based key for line-less nodes; these system-header
structs are never referenced by MuJoCo types, so the key only needs to be
unique.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google-deepmind/mujoco/pull/3411 from yuvaltassa:introspect-loc-fallback 7f117fa7a7060b36687e7544e8d7393d14a1fa90
PiperOrigin-RevId: 949626713
Change-Id: I9d99b427885ede18b075db6d4e507aa8bc0d049a
https://www.youtube.com/watch?v=PdSdrqhSiZA
The new geom attribute surfacevel (6 numbers: linear and angular velocity in the geom's local frame, angular about the geom frame origin) specifies the velocity of the geom's surface material relative to the geom frame. The relative surface velocity of the two geoms is added to the tangential contact rows of efc_vel in mj_referenceConstraint, so friction drives touching bodies toward the motion of the surface: objects on a conveyor are transported at belt speed, turntables impart omega x r with torsional spin-up for condim >= 4, and surface velocities compose with each other and with body motion. The component along the contact normal is projected out: probe experiments showed that velocity-space emission chatters mass-independently and ingestion merely deepens penetration; normal-direction effects belong to force-space features.
surfacevel is interpreted in the geom frame as authored: for mesh geoms, whose compiled frame absorbs the mesh centering and principal-axes transform, the compiler re-expresses the authored value in the compiled frame.
No special interaction with sleeping: objects being transported do not fall asleep because they are moving; objects at rest on an active surface may sleep like any other resting object.
Includes showcase models (model/surfacevel/): a luggage carousel whose ring is a spinning square-profile supertorus fed by a cascade of belts with matched spinning end rollers, bags dropping in and circulating indefinitely; and a treadmill with a passive humanoid.
PiperOrigin-RevId: 948647785
Change-Id: I0c6559a91cc7ece1237eb8ac2e51986e7342d962
This CL replaces the post-hoc implicit flex correction (`flexInterp_cgsolve`) with a **linearly-implicit effective metric** `M̃ = M + (h² + h·damping)·K` carried by the CG constraint solver itself. Contact/friction forces and implicit flex elasticity are now computed against one consistent metric, instead of the solver seeing `M` and a post-solve correction changing `qacc` behind its back.
Gate (unchanged semantics): `solver="CG"` + implicit/implicitfast integrator + pyramidal cones + flex stiffness present. Newton and PGS are untouched. `solver="CG"` remains the user-facing contract — the factorization is an implementation detail of the preconditioner.
### What's in the metric
- **mjData `efm_*`** (arena, efc-like lifetime/skip semantics; built in `mj_fwdPosition`, value-refreshed in `mj_fwdVelocity`): the per-step stiffness CSR `efm_B_*`, its reverse-Cholesky factor `efm_dofid` + `efm_L_*` (nested-dissection ordered, separators-first for the reverse factorization), and the smooth-force shift `efm_c = h·K·qvel`.
- **`mjd_flexStiff_assemble`** now assembles stretch (Gauss–Newton), standard dim-2 bending, and — via the cached corotated stiffness `d->flexelem_krot` — interp stiffness (all node bodies on simple sliders: point Jacobian is I₃, `flex_centered` not required; fixed nodes drop like pins) into one dof-level CSR. `mjd_effMulAdd`/`mjd_effSolve` apply the metric, with matrix-free operator fallbacks where assembly does not apply.
- **mjModel `efm0_*`** (`nefm0dof`/`nefm0L`): the constant part of the metric factor — currently the dim-2 bending factor, computed once in `mj_setConst` — so bending-only models pay zero per-step factorization cost. Naming mirrors mjData's `efm_*` with the standard `0`-suffix (reference/constant) idiom, and is deliberately not bending-specific: future constant contributors extend it without renames.
- The solver consumes the metric through pre-shifted `qfrc_smooth` and the metric products `Ma`/`Mv`/`Mgrad`; `qacc_smooth` becomes the unconstrained minimizer of the implicit dynamics, which makes the no-constraint shortcut and the warmstart choice consistent by construction.
- **`mj_inverse` adds `B·qacc − c`**, making inverse dynamics discrete-consistent with the gated forward dynamics — exact, since the gated path has no qDeriv term (new test `ForwardTest.GatedFlexInverseConsistency`).
### Performance
All numbers: ms/step over the same 2000-step window, models as shipped on each side (old code with the old model settings vs this CL with the new ones).
The new solver path activates on exactly two shipped models — the ponchos, the only flex models that need an implicit integrator (poncho on Euler degenerates to >200 ms/step). For them, this CL trades speed for consistency: the implicit bending solve now runs inside every solver iteration, where the contact solve can see the stiffness, instead of once after the solve. Solver iterations drop because the curvature is visible, but each iteration pays for the implicit solve:
| model | before | after | solver iters/step |
|---|---|---|---|
| poncho | 2.47 | 3.30 (1.33×) | 16.8 → 11.8 |
| poncho_edgeequality | 1.96 | 2.72 (1.39×) | 13.2 → 10.0 |
What that price buys: contact forces consistent with the implicit elasticity (previously the post-hoc correction changed `qacc` after the constraint solve), discrete-consistent inverse dynamics, and the removal of the post-hoc special case from the integration path. Raising poncho's timestep from 2 to 5 ms leaves its per-step cost nearly flat, so the consistency price can be recovered by taking fewer steps where accuracy allows.
Every other flex model was measured stable on Euler at its shipped timestep and switches to it (these models predate the post-hoc integrator; implicit was never load-bearing for them). They end up equal or faster than before: bunny_multicell 0.47 → 0.40, trampoline 0.28 → 0.25, plate 1.02 → 0.99, pancake 0.34 → 0.33.
Finally, the per-step factorization makes configurations practical that the old code could only integrate explicitly: implicit stretch elasticity (`elastic2d="stretch"`/`"both"`, dim-3 solids) and factorized interp stiffness. No before/after exists for these — stock has no implicit treatment of stretch at all.
### Behavior changes
- With the post-hoc correction deleted, interp/bending models running `solver="Newton"` (or elliptic cones, or islands) now integrate flex elasticity **explicitly** (previously: post-hoc implicit). Affects e.g. `gripper_trilinear` (stable, and faster, but different semantics). Follow-up options: Newton-side metric support, or a documented fallback.
- With the gate on, `mj_forward` outputs are timestep-dependent for gated models (they answer the linearly-implicit discrete problem); `qacc_smooth` and `mj_inverse` change accordingly. Non-gated models are bit-identical (full suite green throughout).
### Validation
- 1737/1737 tests, including new: `FlexStretchDerivatives` (FD-validated GN operator), `FlexStiffAssemble`/`FlexStiffAssembleInterp` (CSR ≡ operators), `GatedFlexInverseConsistency` (fails pre-change), equivalence tests vs the old post-hoc treatment (bending matches to 2e-11).
- Fingerprint discipline throughout: bending-only models bit-exact across every refactor; permutation/kernel changes verified iteration-identical.
### Known follow-ups (not in this CL)
3×3-block sparse Cholesky kernel (the numeric factorization is index-bound; projected ~3× on the factor); mjModel persistence of the factor's symbolic pattern (rest-pose ND makes sizes compile-time); the general effective-metric mode (all solvers, all PSD-safe force classes, behind an enable flag).
PiperOrigin-RevId: 948561856
Change-Id: I8b8e32ebd0428042af71647d0470d10773bf6daf
An actuator now owns a block of consecutive controls
(actuator_ctrladr/ctrlnum, width defined by the actuator type) and a block
of consecutive force outputs (actuator_outadr/outnum, width defined by the
transmission type). Force outputs are the scalars of actuation space: one
force, length, velocity and moment row each. nout = dim(actuator_force) is
derived from transmission types; all current types have width 1, so all
three counts coincide for every existing model and behavior is bit-exact.
Array re-keying: ctrlrange/ctrllimited by nu; forcerange/forcelimited/gear/
acc0/length0/lengthrange and the moment row structure by nout; everything
else per actuator. The mjModel actuator block is re-sorted by size key.
Layout-breaking, not behavior-breaking: saved .mjb files are invalidated
(size list changed) and recompilation is required.
PiperOrigin-RevId: 948351772
Change-Id: Icbc196ffa083cb1eaa6f1a3710869c89d8f62540
The mid-phase BVH descent filter prunes body pairs using body_margin,
which was compiled as the max over geom margins, excluding gap. Broadphase
and the leaf-level test both use margin+gap, so any multi-geom body relying
on gap could silently lose its in-gap contacts when raw AABBs don't
overlap. Single-geom bodies take the leaf-leaf path and were unaffected.
PiperOrigin-RevId: 946967548
Change-Id: I6d92baa296f1a83be68b4dbfd64a96d1c7efd3c4
Extend mjpResourceProvider with an optional write callback (write)
so that mj_encode, mj_saveXML, and mj_saveModel can write to any
registered provider.
PiperOrigin-RevId: 945202741
Change-Id: I37903425260932e555f4a8c2392c4ff8c2e6cc06
This example demonstrates how to use Studio's `extra_geoms` feature to render semi-transparent, time-delayed copies of the model's geoms, creating a "ghost" effect.
PiperOrigin-RevId: 943672439
Change-Id: I8ae640672945b55dd25dd3d444f52d0b2fa7f0f8
ViewerApp is simplified, interacting with the endpoint and handlers through the Viewer instance. This change makes it possible to write simulation viewers without using the UI/UX provided by ViewerApp (which should be renamed StudioApp)
PiperOrigin-RevId: 943642660
Change-Id: Id8eb3d4e6a27ceda93833367fec4699fa6bc838e
When `from __future__ import annotations` is active or forward references are used as strings, type annotations are not directly class objects. This change uses `get_type_hints` to resolve these string annotations to their actual types, ensuring correct validation of message handler signatures.
PiperOrigin-RevId: 942608060
Change-Id: Ic2b214ee85e835616cc3a18ec2699eba6deebb6a
Also add manual `Py_GIL_DISABLED` macro definition in CMakeLists as well, since there are some STATIC libraries that aren't getting it automatically through pybind11. As originally proposed in PR #3259.
Co-authored-by: Devansh Raulo <151062727+devansh0703@users.noreply.github.com>
PiperOrigin-RevId: 942216446
Change-Id: I9ef5f457491ae776c452912fbfa8617b8509d47c
This change replaces fixed callback protocols (e.g., ViewerGuiHook, ViewerUpdateHook, SimEventHandler) with a general-purpose, priority-based message and event handling mechanism.
Key changes:
- Handler decorator and registry: Introduced the `@messages.handler(priority=...)` decorator and `HandlerRegistry` (`handler_registry.py`). Methods marked as handlers are automatically discovered and dispatched by priority (CRITICAL, USER, LIBRARY, INTERNAL) or method resolution order.
- Local lifecycle events: Added `ViewerAppInitEvent`, `BuildGuiEvent`, and `UpdateEvent` to `messages.py`. Custom GUI rendering and per-frame update logic can now be implemented as standard event handlers without needing separate interface protocols.
- Streamlined launch and app APIs: Replaced individual hook and handler arguments in `launch_passive`, `ViewerApp`, and `ViewerHandle` with unified `viewer_handlers` and `sim_handlers` lists.
- Module restructuring: Extracted simulation-side message handling and `ViewerHandle` from `sim_app.py` into a dedicated `viewer_handle.py` module, removing `sim_app.py`.
- Sample updates: Migrated existing examples (such as `implot.py`) to use the new handler pattern and lifecycle events.
PiperOrigin-RevId: 941729322
Change-Id: I93e7c0edf0a13a8dc854f9e2083451f7c25a1825
Deleted async examples since they add noise and are not particularly useful
PiperOrigin-RevId: 941117641
Change-Id: I4f621801f652e0b410a7acf905cda58143dc3fb0
Introduce a new header `gil.h` defining `MutexLockIfGilDisabled` to support thread-safety in both standard and free-threaded CPython builds.
Protect critical shared states and registries:
- Guard global Python callback pointers in `callbacks.cc` using a mutex. Move `gil_scoped_acquire` into local blocks around refcount modifications to prevent `longjmp` from bypassing destructors.
- Protect raw pointer maps in `structs_wrappers.cc` with static mutexes.
- Replace TOCTOU race in `mjcb_time` initialization with thread-safe `std::call_once`.
- Add synchronization to lazy indexer array cache initialization in `indexers.cc` and `indexer_xmacro.h`.
- Protect vector mutations in `StructListBase::PopulateUpTo` in `structs.h` with a mutex.
- Revert unnecessary atomic changes to threadpool counters.
- Declare free-threading compatibility by passing `pybind11::mod_gil_not_used()` to all extension modules.
Fixes#3259Fixes#3256Fixes#2978
PiperOrigin-RevId: 941101502
Change-Id: Iec4ce58afcbc75d4b0be6a9a21fc8a47854242e3
- `sim_app.py` provides a SimEventHook and a Handle class for the simulation process to interact with the viewer, including sending state snapshots and processing viewer events.
- `viewer_app.py` implements the main viewer application logic, handling user input, building the Dear ImGui GUI, and managing communication with the simulation process. It also includes hooks for custom viewer behavior.
PiperOrigin-RevId: 940994701
Change-Id: I82786872cd885b15454843b3a4d1160ff0b4be2b
The ViewerEndpoint can now send Snapshot messages to the SimEndpoint. The SimEndpoint has a new method, `get_viewer_snapshots`, to retrieve these. Thsi channel will be used for UI state needed for the simulation (e.g., `MjOption` and `StepControl`).
PiperOrigin-RevId: 940981799
Change-Id: Ib9bd6ad848cd5479bc49492a34fddd82569cb09f
StepControl methods SetSpeed, SetNoiseParameters, and SetPauseState now return a boolean indicating whether the state was actually changed. The GUI functions NoiseGui and StepControlGui also return a boolean reflecting if any parameters were modified. Noise parameters are now directly managed by StepControl rather than being stored in UxState. The StepControlEvent message has been expanded to include all step control parameters. Keyboard event handling has been slightly refactored to better align with event-based updates.
PiperOrigin-RevId: 940656342
Change-Id: I3701623a2fdcfc4c084b5db0ae19048669040de4
- `messages.py`: Defines base classes for `Message`, `Snapshot`, and `Event`, along with `SnapshotChannel` and `EventChannel` protocols. It also includes concrete message types for state, reset, model, perturbation, pause state, and exit events.
- `endpoints.py`: Provides `ViewerEndpoint` and `SimEndpoint` classes to manage message routing between the simulation and the viewer via the defined channels. A `make_endpoints` function is included to create both endpoints.
PiperOrigin-RevId: 939659912
Change-Id: I61a52a7d25431bc4ed38a184813882145eb421d1
Introduced a ViewerConfig dataclass to encapsulate common viewer window parameters. The NativeViewer now accepts a ViewerConfig and no longer requires a model at initialization. The Viewer protocol has been updated to include a close method (renamed from stop) and an upload_image method. Added checks for None models/data in studio event handling.
PiperOrigin-RevId: 939631378
Change-Id: Ibc07d0e775efa0d79ce5552ee52102e943a9b290
The `StepControl.advance` method now checks if the provided `model` or `data` Python objects are `None` before attempting to cast them to `MjModelWrapper` or `MjDataWrapper`, passing `nullptr` to the underlying C++ function if they are `None`.
This fixes an oversight in the python bindings, C++ code already supported null models.
PiperOrigin-RevId: 937243996
Change-Id: Ie67b8603517e7390fb54bfefbfed79ccf7b2bc5e
Yuval Tassa