- Add floating changes to changelog.

- Add sections about the state and derivatives to the Computation chapter.

PiperOrigin-RevId: 459032736
Change-Id: I0b69401688f24c84d5747f1b49980e38e19a54a3

doc/APIreference.rst

@@ -6579,6 +6579,18 @@ mju_sigmoid
 
 Sigmoid function over 0<=x<=1 constructed from half-quadratics.
 
+mjd_transitionFD
+~~~~~~~~~~~~~~~~
+
+.. code-block:: C
+
+   void mjd_transitionFD(const mjModel* m, mjData* d, mjtNum eps, mjtByte centered, mjtNum* A, mjtNum* B);
+
+Finite differenced state-transition and control-transition matrices dx(t+h) = A*dx(t) + B*du(t).
+  required output matrix dimensions:
+     A: (2*nv+na x 2*nv+na)
+     B: (2*nv+na x nu)
+
 .. _Macros:
 
 Macros

doc/changelog.rst

@@ -2,8 +2,53 @@
 Changelog
 =========
 
+Upcoming version (not yet released)
+-----------------------------------
+
+General
+^^^^^^^
+
+- Added ``mjd_transitionFD`` to compute efficient finite difference approximations of the state-transition and
+  control-transition matrices, :ref:`see here<derivatives>` for more details.
+- Added ``ctrl`` attribute to :ref:`keyframes<keyframe>`.
+- Added visualisation groups to skins.
+- Added actuator visualisation for ``free`` and ``ball`` joints and for actuators with ``site`` transmission.
+- Added visualisation for actuator activations.
+- Added ``<intvelocity>`` actuator shortcut for "integrated velocity" actuators, documented :ref:`here <intvelocity>`.
+- Added ``<damper>`` actuator shortcut for active-damping actuators, documented :ref:`here <damper>`.
+- ``mju_rotVecMat`` and ``mju_rotVecMatT`` now support in-place multiplication.
+- ``mjData.ctrl`` values are no longer clamped in-place, remain untouched by the engine.
+- Arrays in mjData's buffer now align to 64-byte boundaries rather than 8-byte.
+- Add memory poisoning when building with Address Sanitizer (ASAN) and Memory Sanitizer (MSAN). This allows ASAN to
+  detect reads and writes to regions in ``mjModel.buffer`` and ``mjData.buffer`` that do not lie within an array, and
+  for MSAN to detect reads from uninitialised fields in ``mjData`` following ``mj_resetData``.
+
+
+Bug fixes
+^^^^^^^^^
+
+- :ref:`Activation clamping <CActRange>` was not being applied in the :ref:`implicit integrator<geIntegration>`.
+- Stricter parsing of orientation specifiers. Before this change, a specification that included both ``quat`` and an
+  :ref:`alternative specifier<COrientation>` e.g., ``<geom ... quat=".1 .2 .3 .4" euler="10 20 30">``, would lead to the
+  ``quat`` being ignored and only ``euler`` being used. After this change a parse error will be thrown.
+- Stricter parsing of XML attributes. Before this change an erroneous XML snippet like ``<geom size="1/2 3 4">`` would
+  have been parsed as ``size="1 0 0"`` and no error would have been thrown. Now throws an error.
+- Trying to load a ``NaN`` via XML like ``<geom size="1 NaN 4">``, while allowed for debugging purposes, will now print
+  a warning.
+- Fixed null pointer dereference in ``mj_loadModel``.
+- Fixed memory leaks when loading an invalid model from MJB.
+- Integer overflows are now avoided when computing ``mjModel`` buffer sizes.
+- Added missing warning string for ``mjWARN_BADCTRL``.
+
+Packaging
+^^^^^^^^^
+
+- Changed MacOS packaging so that the copy of ``mujoco.framework`` embedded in ``MuJoCo.app`` can be used to build
+  applications externally.
+
+
 Version 2.2.0 (May 23, 2022)
------------------------------
+----------------------------
 
 Open Sourcing
 ^^^^^^^^^^^^^

doc/computation.rst

@@ -507,6 +507,97 @@ Implicit-in-velocity Euler method (implicit)
    time will be wasted without meaningful improvement in accuracy. There is always a comfortable range where the time
    step is "just right", but that range is model-dependent.
 
+
+.. _geState:
+
+The **state**
+~~~~~~~~~~~~~
+
+To complete our description of the general framework we will now discuss the notion of *state*. MuJoCo has a compact,
+well-defined internal state which, together with the deterministic computational pipeline, means that operations like
+resetting the state and computing dynamics derivatives are also well-defined. The state is entirely encapsulated in the
+``mjData`` struct and consists of several components:
+
+.. _gePhysicsState:
+
+Physics state
+^^^^^^^^^^^^^
+| The *physics state* contains all quantities which are time-integrated during stepping.
+| They are ``mjData.{qpos, qvel, act, time}``:
+
+  Mechanical state: ``qpos`` and ``qvel``
+    The *mechanical state* of a simulation is given by the generalized position (``mjData.qpos``) and velocity
+    (``mjData.qvel``) vectors, denoted above as :math:`q` and :math:`v`, respectively.
+
+  Actuator activations: ``act``
+    ``mjData.act`` contains the internal states of stateful actuators, denoted above as :math:`w`.
+
+  Time: ``time``
+    The time of the simulation is given by the scalar ``mjData.time``. Since physics is time-invariant, it is
+    often excluded from the *physics state*; an exception could be a time-dependent user callback (e.g., an open-loop
+    controller), in which case time should be included.
+
+.. _geInput:
+
+User inputs
+^^^^^^^^^^^
+These input fields are set by the user and affect the physics simulation, but are untouched by the simulator.
+
+  Controls: ``ctrl``
+    Controls are defined by the :ref:`actuator<actuator>` section of the XML. ``mjData.ctrl`` values either produce
+    generalized forces directly (stateless actuators), or affects the actuator activations in ``mjData.act``, which then
+    produce forces.
+
+  Auxillary Controls: ``qfrc_applied`` and ``xfrc_applied``
+    | ``mjData.qfrc_applied`` are directly applied generalised forces.
+    | ``mjData.xfrc_applied`` are Cartesian wrenches applied to the CoM of individual bodies. This field is used for
+      example, by the :ref:`native viewer<saSimulate>` to apply mouse perturbations.
+    | Note that the effects of ``qfrc_applied`` and ``xfrc_applied`` can usually be recreated by appropriate actuator
+      definitions.
+
+  MoCap poses: ``mocap_pos`` and ``mocap_quat``
+    ``mjData.mocap_pos`` and ``mjData.mocap_quat`` are special optional kinematic states :ref:`described here<CMocap>`,
+    which allow the user to set the positions and orientations of static bodies in real-time, for example when streaming
+    6D poses from a motion-capture device.
+
+  User data: ``userdata``
+    ``mjData.userdata`` acts as a user-defined memory space untouched by the engine. For example it can be used by
+    callbacks. This is described in more detail in the :ref:`Programming chapter<siSimulation>`.
+
+.. _geWarmstart:
+
+Warmstart accelerations
+^^^^^^^^^^^^^^^^^^^^^^^
+
+  ``qacc_warmstart``
+    ``mjData.qacc_warmstart`` are accelerations used to warmstart the constraint solver, saved from the previous step.
+    When using a slowly-converging :ref:`constraint solver<Solver>` like PGS, these can speed up simulation by reducing
+    the number of iterations required for convergence. Note however that the default Newton solver converges so quickly
+    (usually 2-3 iterations), that warmstarts often have no effect on speed and can be disabled.
+
+    Different warmstarts have no preceptible effect on the dynamics but should be saved if perfect numerical
+    reproducibility is required when loading a non-initial state. Note that even though their effect on physics is
+    negligible, many physical systems will accumulate small differences  `exponentially
+    <https://en.wikipedia.org/wiki/Lyapunov_exponent>`__ when time-stepping, quickly leading to divergent trajectories
+    for different warmstarts.
+
+.. _geIntegrationState:
+
+Integration state
+^^^^^^^^^^^^^^^^^
+The *integration state* is the union of all the above ``mjData`` fields and constitutes the entire set of inputs to
+the *forward dynamics*. In the case of *inverse dynamics*, ``mjData.qacc`` is also treated as an input variable. All
+other ``mjData`` fields are functions of the integration state.
+
+.. _geSimulationState:
+
+Simulation state: ``mjData``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The *simulation state* is the entirety of the ``mjData`` struct and associated memory buffer. This state includes
+all derived quantities computed during dynamics computation. Because the ``mjData`` buffers are preallocated for the
+worst case, it is often significantly faster to recompute derived quantities from the *integration state* rather than
+using ``mj_copyData``.
+
 .. _Constraint:
 
 Constraint model
@@ -1422,6 +1513,35 @@ The top-level function :ref:`mj_inverse` invokes the following sequence of compu
 #. Compute the vector ``mjData.qfrc_inverse`` by combining all results. This is the main output of inverse dynamics. It
    equals the sum of external and actuation forces.
 
+
+.. _derivatives:
+
+Derivatives
+-----------
+
+MuJoCo's entire computational pipline and uniquely -- its contraint solver -- are analytically differentiable. Writing
+efficient implementations of these derivatives is a long term goal of the development team. Analytic derivatives of the
+smooth dynamics with respect to velocity are already in place and power the :ref:`implicit integrator<geIntegration>`.
+
+The function ``mjd_transitionFD`` computes state-transition and control-transition Jacobians. Given any valid MuJoCo
+model ``mjModel* m`` with an initial :ref:`simulation state<geState>` in ``mjData* d``,
+
+- let :math:`x` denote the *physics state* of the simulation at time :math:`t` -- the concatenation of positions,
+  velocities and actuator states ``[d->qpos; d->qvel; d->act]``.
+- Let :math:`u` denote the vector of controls at time :math:`t`, corresponding to ``d->ctrl``.
+- Let :math:`y` denote the physical state of the simulation at time :math:`t+h`, where :math:`h` corresponds to
+  ``m->opt.timstep``.
+- The high level function ``mj_step(m, d)`` computes :math:`y(x, u)` -- the next state as a function of
+  the current state and control.
+- ``mjd_transitionFD`` computes the Jacobians :math:`A = \frac{\partial y}{\partial x}` and
+  :math:`B = \frac{\partial y}{\partial u}` using efficient finite-differencing of ``mj_step``.
+
+These derivatives are efficient by exploiting MuJoCo's configurable computation pipeline so that quantities are not
+recomputed when not required. For example when differencing with respect to controls, quantities which depend only on
+position and velocity are not recomputed. Additionally, solver warmstarts, quaternions and control clamping are handled
+correctly. Both forward and centered differences are supported.
+
+
 .. _References:
 
 References

doc/modeling.rst

@@ -572,13 +572,13 @@ Activation clamping
 
 As described in the :ref:`Actuation model <geActuation>` section of the Computation chapter, MuJoCo supports actuators
 with internal dynamics whose states are called "activations". One useful application of these stateful actuators is the
-"integrated-velocity" actuator. Different from the :ref:`pure velocity<velocity>` actuators, which implement direct
+"integrated-velocity" actuator, implemented by the :ref:`intvelocity<intvelocity>` shortcut. Different from the
-feedback on transmission target's velocity, *integrated-velocity* actuators couple an *integrator* with a *position-
+:ref:`pure velocity<velocity>` actuators, which implement direct feedback on transmission target's velocity,
-feedback* actuator. In this case the semantics of the activation state are "the target of the position actuator", and
+*integrated-velocity* actuators couple an *integrator* with a *position-feedback* actuator. In this case the semantics
-the semantics of the control signal are "the velocity of the target of the position actuator". Note that in real robotic
+of the activation state are "the target of the position actuator", and the semantics of the control signal are "the
-systems this integrated-velocity actuator is the most common implementation of actuators with velocity semantics, rather
+velocity of the target of the position actuator". Note that in real robotic systems this integrated-velocity actuator is
-than pure feedback on velocity which is often quite unstable (both in real life and in simulation). This actuator type
+the most common implementation of actuators with velocity semantics, rather than pure feedback on velocity which is
-is implemented by the :ref:`intvelocity<intvelocity>` shortcut.
+often quite unstable (both in real life and in simulation).
 
 In the case of integrated-velocity actuators, it is often desirable to *clamp* the activation state, since otherwise the
 position target would keep integrating beyond the joint limits, leading to loss of controllabillity. To see the effect
@@ -1223,6 +1223,49 @@ corresponding MJCF can be easily re-created. In our experience though, URDF file
 often edited. Thus in practice it is usually sufficient to convert the URDF to MJCF once and after that only work with
 the MJCF.
 
+.. _CMocap:
+
+MoCap bodies
+~~~~~~~~~~~~
+
+``mocap`` bodies are static children of the world (i.e., have no joints) and their :at:`mocap` attribute is set to
+"true". They can be used to input a data stream from a motion capture device into a MuJoCo simulation. Suppose you are
+holding a VR controller, or an object instrumented with motion capture markers (e.g. Vicon), and want to have a
+simulated object moving in the same way but also interacting with other simulated objects. There is a dilemma here:
+virtual objects cannot push on your physical hand, so your hand (and thereby the object you are controlling) can
+violate the simulated physics. But at the same time we want the resulting simulation to be reasonable. How do we do
+this?
+
+The first step is to define a mocap body in the MJCF model, and implement code that reads the data stream at runtime and
+sets mjModel.mocap_pos and mjModel.mocap_quat to the position and orientation received from the motion capture system.
+The `simulate.cc <https://github.com/deepmind/mujoco/blob/main/sample/simulate.cc>`_ code sample uses the mouse as a
+motion capture device, allowing the user to move mocap bodies around:
+
+|particle|
+
+The key thing to understand about mocap bodies is that the simulator treats them as being fixed. We are causing them
+to move from one simulation time step to the next by updating their position and orientation directly, but as far as
+the physics model is concerned their position and orientation are constant. So what happens if we make contact with a
+regular dynamic body, as in the composite object examples provided with the MuJoCo 2.0 distribution (recall that in
+those example we have a capsule probe which is a mocap body that we move with the mouse). A contact between two
+regular bodies will experience penetration as well as relative velocity, while contact with a mocap body is missing
+the relative velocity component because the simulator does not know that the mocap body itself is moving. So the
+resulting contact force is smaller and it takes longer for the contact to push the dynamic object away. Also, in more
+complex simulations the fact that we are doing something inconsistent with the physics can cause instabilities.
+
+There is however a better-behaved alternative. In addition to the mocap body, we include a second regular body and
+connect it to the mocap body with a weld equality constraint. In the plots below, the pink box is the mocap body and
+it is connected to the base of the hand. In the absence of other constraints, the hand tracks the mocap body almost
+perfectly (and much better than a spring-damper would) because the constraints are handled implicitly and can produce
+large forces without destabilizing the simulation. But if the hand is forced to make contact with the table for example
+(right plot) it cannot simultaneously respect the contact constraint and track the mocap body. This is because the
+mocap body is free to go through the table. So which constraint wins? That depends on the softness of the weld
+constraint realtive to the contact constraint. The corresponding :at:`solref` and :at:`solimp` parameters need to be
+adjusted so as to achieve the desired trade-off. See the Modular Prosthetic Limb (MPL) hand model available on the
+MuJoCo Forum for an example; the plots below are generated with that model.
+
+|image18| |image19|
+
 .. _Tips:
 
 Tips and tricks
@@ -1351,47 +1394,6 @@ For example if the stack size is just sufficient for the CG solver, the Newton a
 When we design models, we usually aim for 50% utilization in the worst-case scenario encountered while exploring the
 model. If you only intend to use the CG solver, you can get away with significantly smaller stack allocation.
 
-.. _CMocap:
-
-Motion capture
-~~~~~~~~~~~~~~
-
-Mocap bodies are static children of the world (i.e., have no joints) and their :at:`mocap` attribute is set to
-"true". They can be used to input a data stream from a motion capture device into a MuJoCo simulation. Suppose you are
-holding a VR controller, or an object instrumented with motion capture markers (e.g. Vicon), and want to have a
-simulated object moving in the same way but also interacting with other simulated objects. There is a dilemma here:
-virtual objects cannot push on your physical hand, so your hand (and thereby the object you are controlling) can
-violate the simulated physics. But at the same time we want the resulting simulation to be reasonable. How do we do
-this?
-
-The first step is to define a mocap body in the MJCF model, and implement code that reads the data stream at runtime and
-sets mjModel.mocap_pos and mjModel.mocap_quat to the position and orientation received from the motion capture system.
-The `simulate.cc <https://github.com/deepmind/mujoco/blob/main/sample/simulate.cc>`_ code sample uses the mouse as a
-motion capture device, allowing the user to move mocap bodies around.
-
-The key thing to understand about mocap bodies is that the simulator treats them as being fixed. We are causing them
-to move from one simulation time step to the next by updating their position and orientation directly, but as far as
-the physics model is concerned their position and orientation are constant. So what happens if we make contact with a
-regular dynamic body, as in the composite object examples provided with the MuJoCo 2.0 distribution (recall that in
-those example we have a capsule probe which is a mocap body that we move with the mouse). A contact between two
-regular bodies will experience penetration as well as relative velocity, while contact with a mocap body is missing
-the relative velocity component because the simulator does not know that the mocap body itself is moving. So the
-resulting contact force is smaller and it takes longer for the contact to push the dynamic object away. Also, in more
-complex simulations the fact that we are doing something inconsistent with the physics can cause instabilities.
-
-There is however a better-behaved alternative. In addition to the mocap body, we include a second regular body and
-connect it to the mocap body with a weld equality constraint. In the plots below, the pink box is the mocap body and
-it is connected to the base of the hand. In the absence of other constraints, the hand tracks the mocap body almost
-perfectly (and much better than a spring-damper would) because the constraints are handled implicitly and can produce
-large forces without destabilizing the simulation. But if the hand is forced to make contact with the table for example
-(right plot) it cannot simultaneously respect the contact constraint and track the mocap body. This is because the
-mocap body is free to go through the table. So which constraint wins? That depends on the softness of the weld
-constraint realtive to the contact constraint. The corresponding :at:`solref` and :at:`solimp` parameters need to be
-adjusted so as to achieve the desired trade-off. See the Modular Prosthetic Limb (MPL) hand model available on the
-MuJoCo Forum for an example; the plots below are generated with that model.
-
-|image18| |image19|
-
 .. |image0| image:: images/modeling/impedance.png
    :width: 600px
 .. |image1| image:: images/modeling/musclemodel.png
@@ -1432,4 +1434,6 @@ MuJoCo Forum for an example; the plots below are generated with that model.
    :height: 250px
 .. |image19| image:: images/modeling/mocap2.png
    :height: 250px
+.. |particle| image:: images/models/particle.gif
+   :width: 270px
 .. _simulate.cc: https://github.com/deepmind/mujoco/blob/main/sample/simulate.cc

doc/programming.rst

@@ -340,11 +340,11 @@ illustration of how the new UI framework is intended to be used. Below is a scre
 ..  youtube:: 0ORsj_E17B0
     :align: center
 
-Interaction is done with the mouse; see the built-in help for summary of available commands. Briefly, an object is
+Interaction is done with the mouse; built-in help with a summary of available commands is available by pressing the 'F1'
-selected by left-double-click. The user can then apply forces and torques on the selected object by holding Ctrl and
+key. Briefly, an object is selected by left-double-click. The user can then apply forces and torques on the selected
-dragging the mouse. Dragging the mouse alone (without Ctrl) moves the camera. There are keyboard shortcuts for pausing
+object by holding Ctrl and dragging the mouse. Dragging the mouse alone (without Ctrl) moves the camera. There are
-the simulation, resetting, and re-loading the model file. The latter functionality is very useful while editing the
+keyboard shortcuts for pausing the simulation, resetting, and re-loading the model file. The latter functionality is
-model in an XML editor.
+very useful while editing the model in an XML editor.
 
 The code is quite long yet reasonably commented, so it is best to just read it. Here we provide a high-level overview.
 The ``main()`` function initializes both MuJoCo and GLFW, opens a window, and install GLFW callbacks for mouse and

include/mujoco/mujoco.h

@@ -1085,7 +1085,10 @@ MJAPI mjtNum mju_sigmoid(mjtNum x);
 
 //---------------------- Derivatives ---------------------------------------------------------------
 
-// finite differenced state-transition and control-transition matrices dy = A*dx + B*du
+// Finite differenced state-transition and control-transition matrices dx(t+h) = A*dx(t) + B*du(t).
+//   required output matrix dimensions:
+//      A: (2*nv+na x 2*nv+na)
+//      B: (2*nv+na x nu)
 MJAPI void mjd_transitionFD(const mjModel* m, mjData* d, mjtNum eps, mjtByte centered,
                             mjtNum* A, mjtNum* B);
 

introspect/functions.py

@@ -6931,6 +6931,6 @@ FUNCTIONS: Mapping[str, FunctionDecl] = dict([
                  ),
              ),
          ),
-         doc='finite differenced state-transition and control-transition matrices dy = A*dx + B*du',  # pylint: disable=line-too-long
+         doc='Finite differenced state-transition and control-transition matrices dx(t+h) = A*dx(t) + B*du(t).   required output matrix dimensions:      A: (2*nv+na x 2*nv+na)      B: (2*nv+na x nu)',  # pylint: disable=line-too-long
      )),
 ])