From f54cc58dc5ce797262e9ad789add00a4926cbe95 Mon Sep 17 00:00:00 2001 From: Balint-H Date: Mon, 25 Nov 2024 14:39:17 +0000 Subject: [PATCH 1/6] Reference the tutorial in the Unity plugin docs, and update information of the existing sections. --- doc/unity.rst | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/doc/unity.rst b/doc/unity.rst index cb2c2517..61993bf5 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -10,6 +10,9 @@ runtime to use the MuJoCo physics engine. Users can import MJCF files and edit relies on Unity for most aspects -- assets, game logic, simulation time -- but uses MuJoCo to determine how objects move, giving the designer access to MuJoCo's full API. +An example project using MuJoCo's Unity plugin in a set of introductory tutorials is available at +https://github.com/Balint-H/mj-unity-tutorial. + .. _UInstallation: Installation instructions @@ -145,10 +148,6 @@ effects: material assets for geom RGBA specification. - It allows the importer to handle :ref:`\ ` elements without replicating MuJoCo’s file-system workflow. -- The current version of MuJoCo generates MJCF files with explicit :ref:`\ ` elements, even when - the original model uses geoms for implicit definition of the body inertia. If you plan to change geom properties of - an imported model, remove these auto-generated ``MjInertial`` components manually. We plan to address this in a - future release of MuJoCo. In Unity, there is no equivalent to MJCF’s “cascading” :ref:`\ ` clauses. Therefore, components in Unity reflect the corresponding elements’ state after applying all the relevant default classes, and the class structure @@ -177,9 +176,11 @@ Scene recreation maintains continuity of physics and state in the following way: persisted. 4. The MuJoCo state (for the joints that persisted) is set from the cache, and Unity transforms are synchronized. -Because the MuJoCo library doesn’t (yet) expose an API for scene editing, adding and removing MuJoCo components causes -complete scene recreation. This can be expensive for large models or if it happens frequently. We expect this -performance limitation to be lifted in future versions of MuJoCo. +MuJoCo library has functionality for dynamic scene editing (through `mjSpec +`_), however, this is not yet +supported in the Unity plugin. Therefore, adding and removing MuJoCo components causes complete scene recreation. This +can be expensive for large models or if it happens frequently. We intend to lift this performance limitation to be in a +future versions of the plugin. Global Settings _______________ @@ -323,6 +324,20 @@ The plug-in allows using arbitrary Unity meshes for MuJoCo collision. At model `__ to create a convex hull of the mesh, and uses that for collisions. Currently the computed convex hull is not visible in Unity, but we intend to expose it in future versions. +Height fields +_____________ + +MuJoCo hfields are represented in Unity through terrain gameobjects. This allows the use of the terrain editing tools +available in Unity to generate shapes for collisions with MuJoCo. When selecting hfield type in the Unity geom +component, the right click context menu provides utility to add the corresponding Unity terrain to the scene. The data +from the terrain is dynamically kept in sync with the simulation. + +MuJoCo plugins +______________ + +The current version of the Unity package does not support loading MJCF scenes that use MuJoCo plugins such as +``elasticity``. Adding basic functionality to do this will be part of an upcoming release. + Interaction with External Processes ___________________________________ From 50bfbc91485123e02792a4c7bec9876ae017eeb3 Mon Sep 17 00:00:00 2001 From: Balint-H Date: Mon, 25 Nov 2024 17:15:42 +0000 Subject: [PATCH 2/6] Use RST link instead of explicit html link. --- doc/unity.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/unity.rst b/doc/unity.rst index 61993bf5..ffa72b4b 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -176,8 +176,7 @@ Scene recreation maintains continuity of physics and state in the following way: persisted. 4. The MuJoCo state (for the joints that persisted) is set from the cache, and Unity transforms are synchronized. -MuJoCo library has functionality for dynamic scene editing (through `mjSpec -`_), however, this is not yet +MuJoCo has functionality for dynamic scene editing (through :ref:`mjSpec`), however, this is not yet supported in the Unity plugin. Therefore, adding and removing MuJoCo components causes complete scene recreation. This can be expensive for large models or if it happens frequently. We intend to lift this performance limitation to be in a future versions of the plugin. From f8843166136046e49126a376a82c2e9cad730ddc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A1lint=20Hodossy?= Date: Sat, 4 Jan 2025 11:18:53 +0000 Subject: [PATCH 3/6] Add links to the engine plugins and warning about repo versions --- doc/unity.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/unity.rst b/doc/unity.rst index ffa72b4b..3a282555 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -21,7 +21,10 @@ Installation instructions The plug-in directory (available at https://github.com/google-deepmind/mujoco/tree/main/unity) includes a ``package.json`` file. Unity's package manager recognizes this file and will import the plug-in's C# codebase to your project. In addition, Unity also needs the native MuJoCo library, which can be found in the specific platform archive at -https://github.com/google-deepmind/mujoco/releases. +https://github.com/google-deepmind/mujoco/releases. If you wish to simply use the plug-in and not develop it, you should +use one of the version-specific stable commits of the repository, identified by git tags. Check out the relevant version +of the cloned repository with git (``git checkout 3.X.Y`` where X and Y specify the engine version). Simply using the +``main`` branch of the repository may not be compatible with the most recent release binary of MuJoCo. On Unity version 2020.2 and later, the Package Manager will look for the native library file and copy it to the package directory when the package is imported. Alternatively, you can manually copy the native library to the package directory @@ -334,8 +337,8 @@ from the terrain is dynamically kept in sync with the simulation. MuJoCo plugins ______________ -The current version of the Unity package does not support loading MJCF scenes that use MuJoCo plugins such as -``elasticity``. Adding basic functionality to do this will be part of an upcoming release. +The current version of the Unity package does not support loading MJCF scenes that use :ref:`MuJoCo plugins` such as +`elasticity/ `__ . Adding basic functionality to do this will be part of an upcoming release. Interaction with External Processes ___________________________________ From 364ab5947d5d3c1e3d5639652ff2b05525e9dbd4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A1lint=20Hodossy?= Date: Sat, 4 Jan 2025 11:21:54 +0000 Subject: [PATCH 4/6] Fix typo --- doc/unity.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/unity.rst b/doc/unity.rst index 3a282555..1264ea64 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -338,7 +338,7 @@ MuJoCo plugins ______________ The current version of the Unity package does not support loading MJCF scenes that use :ref:`MuJoCo plugins` such as -`elasticity/ `__ . Adding basic functionality to do this will be part of an upcoming release. +`elasticity `__ . Adding basic functionality to do this will be part of an upcoming release. Interaction with External Processes ___________________________________ From 394cc61192923ecd69a03f612b6b7f2b29ad5c3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A1lint=20Hodossy?= Date: Thu, 16 Jan 2025 14:33:08 +0000 Subject: [PATCH 5/6] Use RST links throughout the Unity doc. --- doc/unity.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/doc/unity.rst b/doc/unity.rst index 1264ea64..390ac52c 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -5,26 +5,27 @@ Unity Plug-in Introduction ------------ -The MuJoCo `Unity plug-in `_ allows the Unity Editor and +The MuJoCo `Unity plug-in `__ allows the Unity Editor and runtime to use the MuJoCo physics engine. Users can import MJCF files and edit the models in the Editor. The plug-in relies on Unity for most aspects -- assets, game logic, simulation time -- but uses MuJoCo to determine how objects move, giving the designer access to MuJoCo's full API. -An example project using MuJoCo's Unity plugin in a set of introductory tutorials is available at -https://github.com/Balint-H/mj-unity-tutorial. +An example project using MuJoCo's Unity plugin in a set of introductory tutorials are also available as a `standalone +repository `__. .. _UInstallation: Installation instructions ------------------------- -The plug-in directory (available at https://github.com/google-deepmind/mujoco/tree/main/unity) includes a +The `plug-in directory `__ includes a ``package.json`` file. Unity's package manager recognizes this file and will import the plug-in's C# codebase to your -project. In addition, Unity also needs the native MuJoCo library, which can be found in the specific platform archive at -https://github.com/google-deepmind/mujoco/releases. If you wish to simply use the plug-in and not develop it, you should -use one of the version-specific stable commits of the repository, identified by git tags. Check out the relevant version -of the cloned repository with git (``git checkout 3.X.Y`` where X and Y specify the engine version). Simply using the -``main`` branch of the repository may not be compatible with the most recent release binary of MuJoCo. +project. In addition, Unity also needs the native MuJoCo library, which can be found in the corrsponding `platform +archive `__. If you wish to simply use the plug-in and not +develop it, you should use one of the version-specific stable commits of the repository, identified by git tags. Check +out the relevant version of the cloned repository with git (``git checkout 3.X.Y`` where X and Y specify the engine +version). Simply using the ``main`` branch of the repository may not be compatible with the most recent release binary +of MuJoCo. On Unity version 2020.2 and later, the Package Manager will look for the native library file and copy it to the package directory when the package is imported. Alternatively, you can manually copy the native library to the package directory @@ -121,7 +122,7 @@ This design principle has several implications: - The layout of MuJoCo components in the GameObject hierarchy determines the layout of the resulting MuJoCo model. Therefore, we adopt a design rule that **every game object must have at most one MuJoCo component**. - We rely on Unity for spatial configuration, which requires vector components to be `swizzled - `_ since Unity uses left-handed frames with Y as the + `__ since Unity uses left-handed frames with Y as the vertical axis, while MuJoCo uses right-handed frames with Z as the vertical axis. - Unity transform scaling affects positions, orientations, and scale of the entire game object subtree. However, MuJoCo doesn’t support collision of skewed cylinders and capsules (skewed spheres are supported via the ellipsoid primitive). @@ -343,9 +344,9 @@ The current version of the Unity package does not support loading MJCF scenes th Interaction with External Processes ___________________________________ -Roboti’s `MuJoCo plug-in for Unity `_ steps the simulation in an external Python +Roboti’s `MuJoCo plug-in for Unity `__ steps the simulation in an external Python process, and uses Unity only for rendering. In contrast, our plug-in relies on Unity to step the simulation. It should be possible to use our plug-in while an external process "drives" the simulation, for example by setting ``qpos``, calling ``mj_kinematics``, synchronizing the transforms, and then using Unity to render or compute game logic. In order to establish communication with an external process, you can use Unity's `ML-Agents -`_ package. +`__ package. From 203e9bc29298d250195632fd207ce4484c84a018 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A1lint=20Hodossy?= Date: Sun, 19 Jan 2025 13:24:36 +0000 Subject: [PATCH 6/6] Remove trailing spaces --- doc/unity.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/unity.rst b/doc/unity.rst index 390ac52c..78fe2ef4 100644 --- a/doc/unity.rst +++ b/doc/unity.rst @@ -20,10 +20,10 @@ Installation instructions The `plug-in directory `__ includes a ``package.json`` file. Unity's package manager recognizes this file and will import the plug-in's C# codebase to your -project. In addition, Unity also needs the native MuJoCo library, which can be found in the corrsponding `platform -archive `__. If you wish to simply use the plug-in and not +project. In addition, Unity also needs the native MuJoCo library, which can be found in the corrsponding `platform +archive `__. If you wish to simply use the plug-in and not develop it, you should use one of the version-specific stable commits of the repository, identified by git tags. Check -out the relevant version of the cloned repository with git (``git checkout 3.X.Y`` where X and Y specify the engine +out the relevant version of the cloned repository with git (``git checkout 3.X.Y`` where X and Y specify the engine version). Simply using the ``main`` branch of the repository may not be compatible with the most recent release binary of MuJoCo.