This change introduces a get_frame method to the viewer protocol, replacing the previous is_running check. This allows the viewer loop to skip rendering and lifecycle events when no active frame is ready (for example, when no browser is connected to the web viewer).
Additionally, this change:
- Adds a shutdown hook to ViewerHandle to wait for the viewer to cleanly release resources before exiting.
- Implements an ExitEvent handler in the viewer protocol to stop the run loop.
- Adds an http_port configuration option for the web viewer.
- Prevents double-closing in the viewer.
PiperOrigin-RevId: 954603868
Change-Id: Iaba7d131dc94191de51150464514dd10fa7834b0
Each pybind extension module holds its own copy of the ImGui/ImPlot globals (extension modules are loaded RTLD_LOCAL), so the viewer that owns the contexts must share the pointers explicitly. Without this, client plotting GUIs and user plotting scripts (e.g. the implot sample) crash on null context pointers.
PiperOrigin-RevId: 951917287
Change-Id: I639a173c2ae9f1af4feaf57aff7b5c81854b7393
https://youtu.be/17XpwnqyCXs
New transmission type mjTRN_SO3: a relative orientation, targeting a ball
joint or a site+refsite pair. It is the first transmission with more than
one force output: its length is the norm of the expmap vector of the
relative rotation and its moment axes are the 3 rows of the
relative rotational Jacobian, without projecting onto per-actuator gears.
New force law mjGAIN_SO3/mjBIAS_SO3: a geodesic PD servo, force =
kp * log(q_current^-1 * q_target) - kv * velocity, exact for arbitrary axis
combinations with a unique equilibrium at every commanded orientation.
Error, moment rows and velocity all live in the child frame (joint or
site): the right-difference error is the gradient of the geodesic
potential in that frame. The parent-frame (left) error is not: driving
child-frame torques with it pumps energy at large angles, settling into
steady-spinning limit cycles (the SO3LargeAngleConvergence test). The
integrator variant stores the 3D orientation setpoint in act (actnum = 3,
re-anchored to a bounded representative at integration time). Exposed in
MJCF as <orientation joint=|site=+refsite= kp kv|dampratio>, or via
<general gaintype="so3" biastype="so3">.
The setpoint input has two charts: an expmap target (3 controls, default)
or a quaternion target (4 controls) -- <orientation input="quat">, the
first actuator with different input and output widths. The signature is
recorded in a new per-actuator field actuator_ctrlspec (mjtCtrlChart),
whose meaning is scoped by the gain type the way gain/bias parameters are;
ctrlnum is derived from it at compile time and remains the layout
authority. An explicit field rather than width inference or a prm slot:
width-as-chart cannot express same-width signatures (upcoming servo input
subsets), and prm slots are the input_mode pattern this stack retires.
The force law normalizes the commanded quaternion, making it scale- and
antipodally-invariant. The all-zero ctrl still maps to the identity via
mju_normalize4, but it is a degenerate point (a nudge of any component
commands a half-turn), so quat inputs reset to the identity quaternion:
new mj_resetCtrl sets neutral ctrl values (zero, except qw = 1), called
by mj_resetData and the viewers' Clear All. The quat chart is
restricted to dyntype 'none': integrating a quaternion setpoint linearly
is not meaningful on the manifold. New mjsActuator.ctrlspec field carries
the signature through the spec and XML round-trip.
Actuator sensors (actuatorpos/vel/frc) now report one value per force
output; dim = 3 on an SO3 actuator.
As the first actuator with nu != nactuator, this commit also makes the
viewers multi-input aware: the control sliders in simulate and studio,
which indexed per-actuator arrays by control index (out of bounds on
this model class), are generated per control and labeled with the
actuator name plus an input suffix ("orient/qw"), via the new
introspection helper mj_actuatorInputName -- the single source of truth
for input names, extended by each new multi-input type (quaternion
components are w-first: qw, qx, qy, qz). Slider ranges now honor a
defined ctrlrange even when ctrllimited is false: range is the UI hint,
limited is the clamp -- wrapped and expmap setpoints are unbounded but
still want finite sliders, while quat components are truly bounded.
The rotational demo model is orientation.xml under
test/engine/testdata/actuation/, upgraded to a three-way contrast:
per-axis wrapped servos vs an expmap-commanded vs a quat-commanded
orientation actuator, on identical checker-textured boxes. It is loaded
by the mixed-axis contrast and input-name tests, and doubles as the
viewer test model (slider groups of 3 independent, 3 grouped, 4 grouped).
PiperOrigin-RevId: 951607063
Change-Id: If235dba8e2f2ca72672e7c62531a27e967c6a373
https://youtu.be/GioWwB36XHI
The new geom attribute adhesion (units of force, signed; pair-level
override) translates the contact friction cone along its normal so
that the force origin lies strictly inside it. Consequences: each
contact can pull with up to the given force before breaking, and the
tangential friction budget becomes mu*(f_N + adhesion) -- the
Mohr-Coulomb yield condition with cohesion c = mu*adhesion -- so
lightly-squeezed grasps retain a guaranteed friction floor.
A translated cone factors exactly into {constant attractive force}
+ {original cone}, so no solver kernels change. The implementation is
this factorization: a constant attraction along contact normals
accumulated into the new mjData.qfrc_adhesion (summed into
qfrc_passive), plus a bias of adhesive contact rows' reference
acceleration (aref += R*adhesion), which makes resting penetration
exactly independent of adhesion. Contacts of adhesive pairs remain
active throughout the gap zone, producing rows with positive violation
whose reference acceleration pulls: a tether that resists pull-off
smoothly, captures objects released within the band into steady
contact, and detaches at the specified force. Adhesion values of the
two geoms combine by sum; explicit pairs override.
mj_contactForce reports the net interface force (cone force minus the
adhesive pull), whose normal component can now be negative. Negative
adhesion is allowed and produces a repulsive offset (air hockey).
PiperOrigin-RevId: 950858148
Change-Id: I879c08eba7ae501e5c0f8c2f807167344da4c2bc
These fields (`flg_gravcomp` and `flg_surfacevel`) replace the fast-path checks originally guarded by `ngravcomp` and (recently) `nsurfacevel`. Since the engine uses these integers only as flags (zero vs non-zero), migrating them to actual booleans makes them writeable from the Python bindings at runtime without violating size/dimension constraints.
The legacy integer field `ngravcomp` is marked as deprecated and will be removed in a future release.
PiperOrigin-RevId: 949779204
Change-Id: Ifab1f026063a4239302e6ad689663b611b59dda8
--
7f117fa7a7060b36687e7544e8d7393d14a1fa90 by Yuval Tassa <tassa@google.com>:
Handle clang AST loc nodes with elided 'line' in introspect codegen.
Apple's math.h (included directly by mujoco.h) contains an anonymous union
whose clang-JSON 'loc' lacks the 'line' key (clang elides it when unchanged
from the previous node), crashing AstProcessor._make_anonymous_key with a
KeyError during bindings regeneration on macOS. glibc's math.h has no such
union, so this never bites on Linux.
Fall back to an offset-based key for line-less nodes; these system-header
structs are never referenced by MuJoCo types, so the key only needs to be
unique.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google-deepmind/mujoco/pull/3411 from yuvaltassa:introspect-loc-fallback 7f117fa7a7060b36687e7544e8d7393d14a1fa90
PiperOrigin-RevId: 949626713
Change-Id: I9d99b427885ede18b075db6d4e507aa8bc0d049a
https://www.youtube.com/watch?v=PdSdrqhSiZA
The new geom attribute surfacevel (6 numbers: linear and angular velocity in the geom's local frame, angular about the geom frame origin) specifies the velocity of the geom's surface material relative to the geom frame. The relative surface velocity of the two geoms is added to the tangential contact rows of efc_vel in mj_referenceConstraint, so friction drives touching bodies toward the motion of the surface: objects on a conveyor are transported at belt speed, turntables impart omega x r with torsional spin-up for condim >= 4, and surface velocities compose with each other and with body motion. The component along the contact normal is projected out: probe experiments showed that velocity-space emission chatters mass-independently and ingestion merely deepens penetration; normal-direction effects belong to force-space features.
surfacevel is interpreted in the geom frame as authored: for mesh geoms, whose compiled frame absorbs the mesh centering and principal-axes transform, the compiler re-expresses the authored value in the compiled frame.
No special interaction with sleeping: objects being transported do not fall asleep because they are moving; objects at rest on an active surface may sleep like any other resting object.
Includes showcase models (model/surfacevel/): a luggage carousel whose ring is a spinning square-profile supertorus fed by a cascade of belts with matched spinning end rollers, bags dropping in and circulating indefinitely; and a treadmill with a passive humanoid.
PiperOrigin-RevId: 948647785
Change-Id: I0c6559a91cc7ece1237eb8ac2e51986e7342d962
This CL replaces the post-hoc implicit flex correction (`flexInterp_cgsolve`) with a **linearly-implicit effective metric** `M̃ = M + (h² + h·damping)·K` carried by the CG constraint solver itself. Contact/friction forces and implicit flex elasticity are now computed against one consistent metric, instead of the solver seeing `M` and a post-solve correction changing `qacc` behind its back.
Gate (unchanged semantics): `solver="CG"` + implicit/implicitfast integrator + pyramidal cones + flex stiffness present. Newton and PGS are untouched. `solver="CG"` remains the user-facing contract — the factorization is an implementation detail of the preconditioner.
### What's in the metric
- **mjData `efm_*`** (arena, efc-like lifetime/skip semantics; built in `mj_fwdPosition`, value-refreshed in `mj_fwdVelocity`): the per-step stiffness CSR `efm_B_*`, its reverse-Cholesky factor `efm_dofid` + `efm_L_*` (nested-dissection ordered, separators-first for the reverse factorization), and the smooth-force shift `efm_c = h·K·qvel`.
- **`mjd_flexStiff_assemble`** now assembles stretch (Gauss–Newton), standard dim-2 bending, and — via the cached corotated stiffness `d->flexelem_krot` — interp stiffness (all node bodies on simple sliders: point Jacobian is I₃, `flex_centered` not required; fixed nodes drop like pins) into one dof-level CSR. `mjd_effMulAdd`/`mjd_effSolve` apply the metric, with matrix-free operator fallbacks where assembly does not apply.
- **mjModel `efm0_*`** (`nefm0dof`/`nefm0L`): the constant part of the metric factor — currently the dim-2 bending factor, computed once in `mj_setConst` — so bending-only models pay zero per-step factorization cost. Naming mirrors mjData's `efm_*` with the standard `0`-suffix (reference/constant) idiom, and is deliberately not bending-specific: future constant contributors extend it without renames.
- The solver consumes the metric through pre-shifted `qfrc_smooth` and the metric products `Ma`/`Mv`/`Mgrad`; `qacc_smooth` becomes the unconstrained minimizer of the implicit dynamics, which makes the no-constraint shortcut and the warmstart choice consistent by construction.
- **`mj_inverse` adds `B·qacc − c`**, making inverse dynamics discrete-consistent with the gated forward dynamics — exact, since the gated path has no qDeriv term (new test `ForwardTest.GatedFlexInverseConsistency`).
### Performance
All numbers: ms/step over the same 2000-step window, models as shipped on each side (old code with the old model settings vs this CL with the new ones).
The new solver path activates on exactly two shipped models — the ponchos, the only flex models that need an implicit integrator (poncho on Euler degenerates to >200 ms/step). For them, this CL trades speed for consistency: the implicit bending solve now runs inside every solver iteration, where the contact solve can see the stiffness, instead of once after the solve. Solver iterations drop because the curvature is visible, but each iteration pays for the implicit solve:
| model | before | after | solver iters/step |
|---|---|---|---|
| poncho | 2.47 | 3.30 (1.33×) | 16.8 → 11.8 |
| poncho_edgeequality | 1.96 | 2.72 (1.39×) | 13.2 → 10.0 |
What that price buys: contact forces consistent with the implicit elasticity (previously the post-hoc correction changed `qacc` after the constraint solve), discrete-consistent inverse dynamics, and the removal of the post-hoc special case from the integration path. Raising poncho's timestep from 2 to 5 ms leaves its per-step cost nearly flat, so the consistency price can be recovered by taking fewer steps where accuracy allows.
Every other flex model was measured stable on Euler at its shipped timestep and switches to it (these models predate the post-hoc integrator; implicit was never load-bearing for them). They end up equal or faster than before: bunny_multicell 0.47 → 0.40, trampoline 0.28 → 0.25, plate 1.02 → 0.99, pancake 0.34 → 0.33.
Finally, the per-step factorization makes configurations practical that the old code could only integrate explicitly: implicit stretch elasticity (`elastic2d="stretch"`/`"both"`, dim-3 solids) and factorized interp stiffness. No before/after exists for these — stock has no implicit treatment of stretch at all.
### Behavior changes
- With the post-hoc correction deleted, interp/bending models running `solver="Newton"` (or elliptic cones, or islands) now integrate flex elasticity **explicitly** (previously: post-hoc implicit). Affects e.g. `gripper_trilinear` (stable, and faster, but different semantics). Follow-up options: Newton-side metric support, or a documented fallback.
- With the gate on, `mj_forward` outputs are timestep-dependent for gated models (they answer the linearly-implicit discrete problem); `qacc_smooth` and `mj_inverse` change accordingly. Non-gated models are bit-identical (full suite green throughout).
### Validation
- 1737/1737 tests, including new: `FlexStretchDerivatives` (FD-validated GN operator), `FlexStiffAssemble`/`FlexStiffAssembleInterp` (CSR ≡ operators), `GatedFlexInverseConsistency` (fails pre-change), equivalence tests vs the old post-hoc treatment (bending matches to 2e-11).
- Fingerprint discipline throughout: bending-only models bit-exact across every refactor; permutation/kernel changes verified iteration-identical.
### Known follow-ups (not in this CL)
3×3-block sparse Cholesky kernel (the numeric factorization is index-bound; projected ~3× on the factor); mjModel persistence of the factor's symbolic pattern (rest-pose ND makes sizes compile-time); the general effective-metric mode (all solvers, all PSD-safe force classes, behind an enable flag).
PiperOrigin-RevId: 948561856
Change-Id: I8b8e32ebd0428042af71647d0470d10773bf6daf
An actuator now owns a block of consecutive controls
(actuator_ctrladr/ctrlnum, width defined by the actuator type) and a block
of consecutive force outputs (actuator_outadr/outnum, width defined by the
transmission type). Force outputs are the scalars of actuation space: one
force, length, velocity and moment row each. nout = dim(actuator_force) is
derived from transmission types; all current types have width 1, so all
three counts coincide for every existing model and behavior is bit-exact.
Array re-keying: ctrlrange/ctrllimited by nu; forcerange/forcelimited/gear/
acc0/length0/lengthrange and the moment row structure by nout; everything
else per actuator. The mjModel actuator block is re-sorted by size key.
Layout-breaking, not behavior-breaking: saved .mjb files are invalidated
(size list changed) and recompilation is required.
PiperOrigin-RevId: 948351772
Change-Id: Icbc196ffa083cb1eaa6f1a3710869c89d8f62540
The mid-phase BVH descent filter prunes body pairs using body_margin,
which was compiled as the max over geom margins, excluding gap. Broadphase
and the leaf-level test both use margin+gap, so any multi-geom body relying
on gap could silently lose its in-gap contacts when raw AABBs don't
overlap. Single-geom bodies take the leaf-leaf path and were unaffected.
PiperOrigin-RevId: 946967548
Change-Id: I6d92baa296f1a83be68b4dbfd64a96d1c7efd3c4
Extend mjpResourceProvider with an optional write callback (write)
so that mj_encode, mj_saveXML, and mj_saveModel can write to any
registered provider.
PiperOrigin-RevId: 945202741
Change-Id: I37903425260932e555f4a8c2392c4ff8c2e6cc06
This example demonstrates how to use Studio's `extra_geoms` feature to render semi-transparent, time-delayed copies of the model's geoms, creating a "ghost" effect.
PiperOrigin-RevId: 943672439
Change-Id: I8ae640672945b55dd25dd3d444f52d0b2fa7f0f8
ViewerApp is simplified, interacting with the endpoint and handlers through the Viewer instance. This change makes it possible to write simulation viewers without using the UI/UX provided by ViewerApp (which should be renamed StudioApp)
PiperOrigin-RevId: 943642660
Change-Id: Id8eb3d4e6a27ceda93833367fec4699fa6bc838e
When `from __future__ import annotations` is active or forward references are used as strings, type annotations are not directly class objects. This change uses `get_type_hints` to resolve these string annotations to their actual types, ensuring correct validation of message handler signatures.
PiperOrigin-RevId: 942608060
Change-Id: Ic2b214ee85e835616cc3a18ec2699eba6deebb6a
Also add manual `Py_GIL_DISABLED` macro definition in CMakeLists as well, since there are some STATIC libraries that aren't getting it automatically through pybind11. As originally proposed in PR #3259.
Co-authored-by: Devansh Raulo <151062727+devansh0703@users.noreply.github.com>
PiperOrigin-RevId: 942216446
Change-Id: I9ef5f457491ae776c452912fbfa8617b8509d47c
This change replaces fixed callback protocols (e.g., ViewerGuiHook, ViewerUpdateHook, SimEventHandler) with a general-purpose, priority-based message and event handling mechanism.
Key changes:
- Handler decorator and registry: Introduced the `@messages.handler(priority=...)` decorator and `HandlerRegistry` (`handler_registry.py`). Methods marked as handlers are automatically discovered and dispatched by priority (CRITICAL, USER, LIBRARY, INTERNAL) or method resolution order.
- Local lifecycle events: Added `ViewerAppInitEvent`, `BuildGuiEvent`, and `UpdateEvent` to `messages.py`. Custom GUI rendering and per-frame update logic can now be implemented as standard event handlers without needing separate interface protocols.
- Streamlined launch and app APIs: Replaced individual hook and handler arguments in `launch_passive`, `ViewerApp`, and `ViewerHandle` with unified `viewer_handlers` and `sim_handlers` lists.
- Module restructuring: Extracted simulation-side message handling and `ViewerHandle` from `sim_app.py` into a dedicated `viewer_handle.py` module, removing `sim_app.py`.
- Sample updates: Migrated existing examples (such as `implot.py`) to use the new handler pattern and lifecycle events.
PiperOrigin-RevId: 941729322
Change-Id: I93e7c0edf0a13a8dc854f9e2083451f7c25a1825
Deleted async examples since they add noise and are not particularly useful
PiperOrigin-RevId: 941117641
Change-Id: I4f621801f652e0b410a7acf905cda58143dc3fb0
Introduce a new header `gil.h` defining `MutexLockIfGilDisabled` to support thread-safety in both standard and free-threaded CPython builds.
Protect critical shared states and registries:
- Guard global Python callback pointers in `callbacks.cc` using a mutex. Move `gil_scoped_acquire` into local blocks around refcount modifications to prevent `longjmp` from bypassing destructors.
- Protect raw pointer maps in `structs_wrappers.cc` with static mutexes.
- Replace TOCTOU race in `mjcb_time` initialization with thread-safe `std::call_once`.
- Add synchronization to lazy indexer array cache initialization in `indexers.cc` and `indexer_xmacro.h`.
- Protect vector mutations in `StructListBase::PopulateUpTo` in `structs.h` with a mutex.
- Revert unnecessary atomic changes to threadpool counters.
- Declare free-threading compatibility by passing `pybind11::mod_gil_not_used()` to all extension modules.
Fixes#3259Fixes#3256Fixes#2978
PiperOrigin-RevId: 941101502
Change-Id: Iec4ce58afcbc75d4b0be6a9a21fc8a47854242e3
- `sim_app.py` provides a SimEventHook and a Handle class for the simulation process to interact with the viewer, including sending state snapshots and processing viewer events.
- `viewer_app.py` implements the main viewer application logic, handling user input, building the Dear ImGui GUI, and managing communication with the simulation process. It also includes hooks for custom viewer behavior.
PiperOrigin-RevId: 940994701
Change-Id: I82786872cd885b15454843b3a4d1160ff0b4be2b
The ViewerEndpoint can now send Snapshot messages to the SimEndpoint. The SimEndpoint has a new method, `get_viewer_snapshots`, to retrieve these. Thsi channel will be used for UI state needed for the simulation (e.g., `MjOption` and `StepControl`).
PiperOrigin-RevId: 940981799
Change-Id: Ib9bd6ad848cd5479bc49492a34fddd82569cb09f
Matija Kecman