Add initial documentation for experimental OpenUSD support in MuJoCo.
PiperOrigin-RevId: 789365870 Change-Id: I19cce03ec2ffc28cdf4252bfd02c8b5163bb5ba2
This commit is contained in:
Sam Haves
committed by
Copybara-Service
Copybara-Service
parent
32c7d3f085
commit
700da7c5bd
@@ -0,0 +1,57 @@
|
|||||||
|
Building
|
||||||
|
========
|
||||||
|
|
||||||
|
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||||
|
|
||||||
|
Advanced users can start testing out USD support by building against their own USD libraries or USD built from source.
|
||||||
|
|
||||||
|
This assumes that you have built MuJoCo in ``~/mujoco`` and have a build directory at ``~/mujoco/build``
|
||||||
|
|
||||||
|
Building USD
|
||||||
|
------------
|
||||||
|
|
||||||
|
USD has a pretty streamlined installation via their ``build_usd.py`` script. It's recommended to use a separate
|
||||||
|
installation directory that exists outside of the cloned repository directory.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
git clone https://github.com/PixarAnimationStudios/OpenUSD
|
||||||
|
python OpenUSD/build_scripts/build_usd.py /path/to/my_usd_install_dir
|
||||||
|
|
||||||
|
Enabling USD
|
||||||
|
------------
|
||||||
|
|
||||||
|
USD is comprised of many plugins. When USD enabled application starts up it looks for an environment variable called
|
||||||
|
``PXR_PLUGINPATH_NAME``. Below is an example where we build MuJoCo with USD enabled and set this variable.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
cd ~/mujoco/build
|
||||||
|
cmake .. -DCMAKE_BUILD_TYPE=Release -DUSD_DIR=/path/to/my_usd_install_dir
|
||||||
|
cmake --build . -j 30; sudo cmake --install .
|
||||||
|
export PXR_PLUGINPATH_NAME=/usr/local/lib/mujocoUsd/resources/*/plugInfo.json
|
||||||
|
|
||||||
|
If we now run :ref:`simulate.cc <saSimulate>`, we will be able to drag and drop USD files.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
simulate
|
||||||
|
|
||||||
|
Enabling plugins in Houdini
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Houdini is a procedural content authoring tool with extensive support for USD workflows via their Solaris context. It's
|
||||||
|
highly popular in the VFX industry, and it's easy to imagine procedural generation tools for simulation ready assets and
|
||||||
|
scenes.
|
||||||
|
|
||||||
|
To allow support for loading MJCF files in Solaris, and usage of the mjhcPhysics schemas you can build against Houdini's
|
||||||
|
USD libraries. To do so, simply run `source ./houdini_setup` as descrived in the `SideFX documentation
|
||||||
|
<https://www.sidefx.com/faq/question/how-do-i-set-up-the-houdini-environment-for-command-line-tools>`__.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
cd ~/mujoco/build
|
||||||
|
cmake .. -DCMAKE_BUILD_TYPE=Release -DHOUDINI_HFS_DIR=$HFS
|
||||||
|
cmake --build . -j 30; sudo cmake --install .
|
||||||
|
export PXR_PLUGINPATH_NAME=/usr/local/lib/mujocoUsd/resources/*/plugInfo.json
|
||||||
|
houdini
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
Exporting
|
||||||
|
=========
|
||||||
|
|
||||||
|
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||||
|
|
||||||
|
Currently, exporting MuJoCo scenes to OpenUSD format is an area of active development. The primary method for exporting
|
||||||
|
is expected to be through the Python API.
|
||||||
|
|
||||||
|
USDExporter
|
||||||
|
-----------
|
||||||
|
|
||||||
|
At the moment the only way to export USD from MuJoCo is to use the existing USDExporter in :doc:`../python`.
|
||||||
|
|
||||||
|
We are working on native support for writing simulations as animations to existing USD scenes, please check back here
|
||||||
|
for updates.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Importing
|
||||||
|
=========
|
||||||
|
|
||||||
|
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||||
|
|
||||||
|
MuJoCo can load assets from OpenUSD files (``.usd``, ``.usda``, ``.usdc``, ``.usdz``). This allows you to incorporate
|
||||||
|
assets and scenes defined in USD into your MuJoCo simulations.
|
||||||
|
|
||||||
|
USD in MJCF
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
If you have built mujoco with USD enabled, you can reference USD assets from MJCF via the ``<model`` tag with content
|
||||||
|
type ``text/usd``.
|
||||||
|
|
||||||
|
.. code-block:: xml
|
||||||
|
:caption: example.xml
|
||||||
|
|
||||||
|
<mujoco>
|
||||||
|
<asset>
|
||||||
|
<model file="chair.usdz" name="chair" content_type="text/usd"/>
|
||||||
|
</asset>
|
||||||
|
|
||||||
|
<worldbody>
|
||||||
|
...
|
||||||
|
</worldbody>
|
||||||
|
</mujoco>
|
||||||
|
|
||||||
|
In this example the ``<model file="chair.usdz"/>`` line in ``<asset>`` tells MuJoCo to load and process the USD file.
|
||||||
|
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
OpenUSD
|
||||||
|
===========
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:hidden:
|
||||||
|
|
||||||
|
building
|
||||||
|
mjcPhysics
|
||||||
|
mjcf_file_format_plugin
|
||||||
|
importing
|
||||||
|
exporting
|
||||||
|
|
||||||
|
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||||
|
|
||||||
|
Introduction
|
||||||
|
------------
|
||||||
|
|
||||||
|
This chapter describes MuJoCo's support for `OpenUSD <https://openusd.org/release/intro.html>`__. USD (Universal Scene
|
||||||
|
Description) is an open-source framework developed by Pixar for describing 3D scenes. MuJoCo's integration allows users
|
||||||
|
to leverage USD's rich ecosystem and tooling.
|
||||||
|
|
||||||
|
What is OpenUSD?
|
||||||
|
----------------
|
||||||
|
|
||||||
|
USD is a high-performance, extensible system for describing, composing, simulating, and collaborating on 3D data.
|
||||||
|
Originally developed by Pixar Animation Studios, USD is now used across various industries, including visual effects,
|
||||||
|
animation, gaming, and robotics, to streamline complex 3D workflows. It provides a common language for different
|
||||||
|
software applications to exchange 3D scene information.
|
||||||
|
|
||||||
|
Why do we care about OpenUSD?
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
Integrating USD with MuJoCo offers several advantages:
|
||||||
|
|
||||||
|
* **Interoperability:** USD is supported by a wide range of 3D content creation tools (e.g., Houdini, Maya, Blender).
|
||||||
|
This allows MuJoCo users to easily import scenes and assets created in these tools.
|
||||||
|
* **Rich Scene Description:** USD provides a powerful and flexible way to represent complex scenes, including
|
||||||
|
geometry, materials, lighting, and hierarchies.
|
||||||
|
* **Collaboration:** USD's layering and composition features enable powerful and efficient
|
||||||
|
non-destructive authoring pipelines.
|
||||||
|
|
||||||
|
USD support overview
|
||||||
|
------------------------------------------------------
|
||||||
|
|
||||||
|
* **Import:** You can load USD assets (specifically ``.usd``, ``.usda``, ``.usdc``, ``.usdz`` files) into MuJoCo via
|
||||||
|
MJCF or dragging and dropping into :ref:`simulate.cc <saSimulate>`.
|
||||||
|
* **Schemas:** MuJoCo primarily uses the standard `UsdPhysics
|
||||||
|
<https://openusd.org/dev/api/usd_physics_page_front.html>`__ schemas for representing physics properties.
|
||||||
|
* **Extensions:** Custom :doc:`mjcPhysics` schemas are provided to cover MuJoCo-specific features not available in
|
||||||
|
``UsdPhysics``.
|
||||||
|
* **MJCF File Format Plugin:** A :doc:`mjcf_file_format_plugin` allows treating MJCF files as USD layers in any native USD
|
||||||
|
application.
|
||||||
|
* **Export:** MuJoCo scenes can be exported to USD.
|
||||||
|
|
||||||
|
Where do I learn more about USD?
|
||||||
|
------------------------------------------
|
||||||
|
|
||||||
|
* `Remedy's Book of USD <https://remedy-entertainment.github.io/USDBook>`__: Friendly introduction to USD.
|
||||||
|
* `Official OpenUSD Documentation <https://openusd.org/release/intro.html>`__: Official documentation for API and
|
||||||
|
implementation details.
|
||||||
|
* `Pixar's USD Introduction <https://graphics.pixar.com/usd/release/index.html>`__: Simple example usage of USD.
|
||||||
|
* `NVIDIA's USD Resources <https://developer.nvidia.com/usd>`__: Set of USD resources primarily concerned with asset
|
||||||
|
structure.
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
mjcPhysics
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||||
|
|
||||||
|
The ``mjcPhysics`` `schema <https://openusd.org/release/api/_usd__page__generating_schemas.html>`__ allows for detailed
|
||||||
|
specification of a MuJoCo simulation environment directly within a USD file. The aim is not to replace `UsdPhysics
|
||||||
|
<https://openusd.org/release/api/usd_physics_page_front.html>`__, but to extend existing concepts and create new types
|
||||||
|
only where is necessary.
|
||||||
|
|
||||||
|
The schema can be use `codeless <https://openusd.org/dev/api/_usd__page__generating_schemas.html#Codeless_Schemas>`__,
|
||||||
|
or can be built with its C++ bindings. We've pre-generated `the code
|
||||||
|
<https://github.com/google-deepmind/mujoco/tree/main/src/experimental/usd/mjcPhysics>`__ via `usdGenSchema
|
||||||
|
<https://openusd.org/dev/api/_usd__page__generating_schemas.html>`_ for internal MuJoCo usage, but it should also work
|
||||||
|
outside of MuJoCo.
|
||||||
|
|
||||||
|
API Schemas
|
||||||
|
-----------
|
||||||
|
|
||||||
|
MjcSceneAPI
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API schema provides global options for the MuJoCo simulation. It is an
|
||||||
|
amalgamation of the ``<option>``, ``<option/flag>`` and ``<compiler>`` elements in
|
||||||
|
MJCF. Users should apply this to an existing
|
||||||
|
`UsdPhysicsScene <https://openusd.org/dev/api/class_usd_physics_scene.html>`__
|
||||||
|
prim.
|
||||||
|
|
||||||
|
Key attributes include:
|
||||||
|
|
||||||
|
- **mjc:option**: Attributes in this namespace map to the ``<option>`` element.
|
||||||
|
- **mjc:flag**: Attributes in this namespace map to the ``<option/flag>`` element.
|
||||||
|
- **mjc:compiler**: Attributes in this namespace map to the ``<compiler>`` element.
|
||||||
|
|
||||||
|
MjcSiteAPI
|
||||||
|
^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class is used to define a MuJoCo site, it can be applied to
|
||||||
|
`UsdGeomSphere <https://openusd.org/dev/api/class_usd_geom_sphere.html>`__,
|
||||||
|
`UsdGeomCapsule <https://openusd.org/dev/api/class_usd_geom_capsule.html>`__,
|
||||||
|
`UsdGeomCylinder <https://openusd.org/dev/api/class_usd_geom_cylinder.html>`__, and
|
||||||
|
`UsdGeomCube <https://openusd.org/dev/api/class_usd_geom_cube.html>`__.
|
||||||
|
|
||||||
|
MjcImageableAPI
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class provides attributes for strictly visual entities in MuJoCo, in
|
||||||
|
MuJoCo terms we would quantify these has having ``contype = conaffinity = 0``.
|
||||||
|
|
||||||
|
MjcCollisionAPI
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class is applied to prims that represent collision geometry and should
|
||||||
|
be applied alongside
|
||||||
|
`UsdPhysicsCollisionAPI <https://openusd.org/dev/api/class_usd_physics_collision_a_p_i.html>`__.
|
||||||
|
|
||||||
|
MjcMeshCollisionAPI
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class is applied to prims that represent mesh collision geometry and
|
||||||
|
should be applied alongside
|
||||||
|
`UsdPhysicsMeshCollisionAPI <https://openusd.org/dev/api/class_usd_physics_mesh_collision_a_p_i.html>`__.
|
||||||
|
|
||||||
|
MjcJointAPI
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class is applied to `UsdPhysicsJoint <https://openusd.org/dev/api/class_usd_physics_joint.html>`__ prims,
|
||||||
|
adding extra attributes to fully describe MuJoCo joints.
|
||||||
|
|
||||||
|
MjcMaterialAPI
|
||||||
|
^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This API class provides attributes for physical materials and is an extension of `UsdPhysicsMaterialAPI
|
||||||
|
<https://openusd.org/dev/api/class_usd_physics_material_a_p_i.html>`__
|
||||||
|
|
||||||
|
Type Schemas
|
||||||
|
------------
|
||||||
|
|
||||||
|
MjcActuator
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
This class represents a MuJoCo actuator, which is responsible for applying force to a transmission target joint, body,
|
||||||
|
or site specific via a `relationship <https://openusd.org/dev/api/class_usd_relationship.html>`__.
|
||||||
|
|
||||||
|
We do not use the existing `UsdPhysicsDriveAPI <https://openusd.org/dev/api/class_usd_physics_drive_a_p_i.html>`__ as it
|
||||||
|
is closer to a runtime construct and the concepts do not map very closely.
|
||||||
|
|
||||||
|
MjcKeyframe
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
This type holds tensor values representing simulator state at specific time values.
|
||||||
|
|
||||||
|
In MJCF this is the ``<keyframe>`` element and has a ``time`` attribute. In USD we map the time attribute to
|
||||||
|
`timeSamples <https://openusd.org/release/tut_xforms.html>`__ instead.
|
||||||
|
|
||||||
|
The order of the values in the keyframes should map to the depth first ordered traversal of rigidbodies in the composed
|
||||||
|
stage.
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
File Format Plugin
|
||||||
|
=========================
|
||||||
|
|
||||||
|
What is an SdfFileFormat plugin?
|
||||||
|
--------------------------------
|
||||||
|
|
||||||
|
In the OpenUSD framework, ``Sdf`` stands for Scene Description Foundations. It's the underlying layer that handles the
|
||||||
|
serialization and composition of scene data. A ``SdfFileFormat`` plugin is a component that teaches USD how to read and
|
||||||
|
write a specific file format.
|
||||||
|
|
||||||
|
By default, USD comes with plugins for its own formats (``.usda``, ``.usdc``, ``.usdz``) and the community has created
|
||||||
|
several plugin extension such as the `Adobe File Format Plugins
|
||||||
|
<https://github.com/adobe/USD-Fileformat-plugins/tree/main>`__.
|
||||||
|
|
||||||
|
The MJCF ``SdfFileFormat`` plugin allows USD-aware applications to directly understand and interact with MuJoCo's native
|
||||||
|
``.xml`` (MJCF) files as if they were native USD files.
|
||||||
|
|
||||||
|
What does it enable?
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
This plugin enables:
|
||||||
|
|
||||||
|
1. **Referencing MJCF files in USD:** Using standard USD composition arcs (like references, payloads) to include an
|
||||||
|
MJCF file directly within a larger USD scene. For example, you can place a MuJoCo robot defined in an ``.xml`` file
|
||||||
|
into a room scene modeled in USD.
|
||||||
|
2. **Load MJCF files in USD tools:** Tools like ``usdview`` or other USD-based
|
||||||
|
applications can open, inspect, and render MJCF files, translating the MJCF elements into USD prims and attributes
|
||||||
|
on the fly.
|
||||||
|
3. **Convert MJCF to USD:** The plugin can be used as a basis for converting MJCF files to persistent
|
||||||
|
USD files (e.g., ``.usda`` or ``.usdc``).
|
||||||
|
|
||||||
|
Essentially, it makes MJCF a first-class citizen in the USD ecosystem.
|
||||||
|
|
||||||
|
Usage
|
||||||
|
------------------
|
||||||
|
|
||||||
|
1. **Installation:** refer to :doc:`building`.
|
||||||
|
|
||||||
|
2. **Referencing in a USD file (e.g., ``.usda``):**
|
||||||
|
|
||||||
|
.. code-block:: usd
|
||||||
|
:caption: example.usda
|
||||||
|
|
||||||
|
#usda 1.0
|
||||||
|
(
|
||||||
|
upAxis = "Z"
|
||||||
|
)
|
||||||
|
|
||||||
|
def Xform "world"
|
||||||
|
{
|
||||||
|
def "robot" (
|
||||||
|
prepend references = @./my_robot.xml@
|
||||||
|
)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
In this example, ``my_robot.xml`` is an MJCF file in the same directory. USD will use the plugin to load and
|
||||||
|
interpret its contents.
|
||||||
|
|
||||||
|
3. **Opening in usdview:**
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
usdview my_robot.xml
|
||||||
|
|
||||||
|
If the plugin is correctly set up, ``usdview`` will render the robot defined in the MJCF file.
|
||||||
|
|
||||||
|
4. **Using in Python (with USD API):**
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from pxr import Usd
|
||||||
|
|
||||||
|
# Load an MJCF file as a USD stage
|
||||||
|
stage = Usd.Stage.Open('my_robot.xml')
|
||||||
|
|
||||||
|
if stage:
|
||||||
|
print(f"Successfully opened {stage.GetRootLayer().identifier}")
|
||||||
|
# You can now inspect the stage as any other USD stage
|
||||||
|
for prim in stage.TraverseAll():
|
||||||
|
print(prim.GetPath())
|
||||||
|
else:
|
||||||
|
print("Failed to open MJCF file")
|
||||||
|
|
||||||
|
This plugin significantly enhances the interoperability between MuJoCo and USD-based workflows, allowing
|
||||||
|
seamless integration of physics assets defined in MJCF into broader 3D environments.
|
||||||
@@ -16,6 +16,7 @@
|
|||||||
python
|
python
|
||||||
MJX <mjx>
|
MJX <mjx>
|
||||||
unity
|
unity
|
||||||
|
OpenUSD/index.rst
|
||||||
models
|
models
|
||||||
changelog
|
changelog
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user