Add initial documentation for experimental OpenUSD support in MuJoCo.
PiperOrigin-RevId: 789365870 Change-Id: I19cce03ec2ffc28cdf4252bfd02c8b5163bb5ba2
This commit is contained in:
Sam Haves
committed by
Copybara-Service
Copybara-Service
parent
32c7d3f085
commit
700da7c5bd
@@ -0,0 +1,57 @@
|
||||
Building
|
||||
========
|
||||
|
||||
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||
|
||||
Advanced users can start testing out USD support by building against their own USD libraries or USD built from source.
|
||||
|
||||
This assumes that you have built MuJoCo in ``~/mujoco`` and have a build directory at ``~/mujoco/build``
|
||||
|
||||
Building USD
|
||||
------------
|
||||
|
||||
USD has a pretty streamlined installation via their ``build_usd.py`` script. It's recommended to use a separate
|
||||
installation directory that exists outside of the cloned repository directory.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/PixarAnimationStudios/OpenUSD
|
||||
python OpenUSD/build_scripts/build_usd.py /path/to/my_usd_install_dir
|
||||
|
||||
Enabling USD
|
||||
------------
|
||||
|
||||
USD is comprised of many plugins. When USD enabled application starts up it looks for an environment variable called
|
||||
``PXR_PLUGINPATH_NAME``. Below is an example where we build MuJoCo with USD enabled and set this variable.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/mujoco/build
|
||||
cmake .. -DCMAKE_BUILD_TYPE=Release -DUSD_DIR=/path/to/my_usd_install_dir
|
||||
cmake --build . -j 30; sudo cmake --install .
|
||||
export PXR_PLUGINPATH_NAME=/usr/local/lib/mujocoUsd/resources/*/plugInfo.json
|
||||
|
||||
If we now run :ref:`simulate.cc <saSimulate>`, we will be able to drag and drop USD files.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
simulate
|
||||
|
||||
Enabling plugins in Houdini
|
||||
---------------------------
|
||||
|
||||
Houdini is a procedural content authoring tool with extensive support for USD workflows via their Solaris context. It's
|
||||
highly popular in the VFX industry, and it's easy to imagine procedural generation tools for simulation ready assets and
|
||||
scenes.
|
||||
|
||||
To allow support for loading MJCF files in Solaris, and usage of the mjhcPhysics schemas you can build against Houdini's
|
||||
USD libraries. To do so, simply run `source ./houdini_setup` as descrived in the `SideFX documentation
|
||||
<https://www.sidefx.com/faq/question/how-do-i-set-up-the-houdini-environment-for-command-line-tools>`__.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/mujoco/build
|
||||
cmake .. -DCMAKE_BUILD_TYPE=Release -DHOUDINI_HFS_DIR=$HFS
|
||||
cmake --build . -j 30; sudo cmake --install .
|
||||
export PXR_PLUGINPATH_NAME=/usr/local/lib/mujocoUsd/resources/*/plugInfo.json
|
||||
houdini
|
||||
@@ -0,0 +1,15 @@
|
||||
Exporting
|
||||
=========
|
||||
|
||||
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||
|
||||
Currently, exporting MuJoCo scenes to OpenUSD format is an area of active development. The primary method for exporting
|
||||
is expected to be through the Python API.
|
||||
|
||||
USDExporter
|
||||
-----------
|
||||
|
||||
At the moment the only way to export USD from MuJoCo is to use the existing USDExporter in :doc:`../python`.
|
||||
|
||||
We are working on native support for writing simulations as animations to existing USD scenes, please check back here
|
||||
for updates.
|
||||
@@ -0,0 +1,29 @@
|
||||
Importing
|
||||
=========
|
||||
|
||||
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||
|
||||
MuJoCo can load assets from OpenUSD files (``.usd``, ``.usda``, ``.usdc``, ``.usdz``). This allows you to incorporate
|
||||
assets and scenes defined in USD into your MuJoCo simulations.
|
||||
|
||||
USD in MJCF
|
||||
-----------------------------
|
||||
|
||||
If you have built mujoco with USD enabled, you can reference USD assets from MJCF via the ``<model`` tag with content
|
||||
type ``text/usd``.
|
||||
|
||||
.. code-block:: xml
|
||||
:caption: example.xml
|
||||
|
||||
<mujoco>
|
||||
<asset>
|
||||
<model file="chair.usdz" name="chair" content_type="text/usd"/>
|
||||
</asset>
|
||||
|
||||
<worldbody>
|
||||
...
|
||||
</worldbody>
|
||||
</mujoco>
|
||||
|
||||
In this example the ``<model file="chair.usdz"/>`` line in ``<asset>`` tells MuJoCo to load and process the USD file.
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
OpenUSD
|
||||
===========
|
||||
|
||||
.. toctree::
|
||||
:hidden:
|
||||
|
||||
building
|
||||
mjcPhysics
|
||||
mjcf_file_format_plugin
|
||||
importing
|
||||
exporting
|
||||
|
||||
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
This chapter describes MuJoCo's support for `OpenUSD <https://openusd.org/release/intro.html>`__. USD (Universal Scene
|
||||
Description) is an open-source framework developed by Pixar for describing 3D scenes. MuJoCo's integration allows users
|
||||
to leverage USD's rich ecosystem and tooling.
|
||||
|
||||
What is OpenUSD?
|
||||
----------------
|
||||
|
||||
USD is a high-performance, extensible system for describing, composing, simulating, and collaborating on 3D data.
|
||||
Originally developed by Pixar Animation Studios, USD is now used across various industries, including visual effects,
|
||||
animation, gaming, and robotics, to streamline complex 3D workflows. It provides a common language for different
|
||||
software applications to exchange 3D scene information.
|
||||
|
||||
Why do we care about OpenUSD?
|
||||
-----------------------------
|
||||
|
||||
Integrating USD with MuJoCo offers several advantages:
|
||||
|
||||
* **Interoperability:** USD is supported by a wide range of 3D content creation tools (e.g., Houdini, Maya, Blender).
|
||||
This allows MuJoCo users to easily import scenes and assets created in these tools.
|
||||
* **Rich Scene Description:** USD provides a powerful and flexible way to represent complex scenes, including
|
||||
geometry, materials, lighting, and hierarchies.
|
||||
* **Collaboration:** USD's layering and composition features enable powerful and efficient
|
||||
non-destructive authoring pipelines.
|
||||
|
||||
USD support overview
|
||||
------------------------------------------------------
|
||||
|
||||
* **Import:** You can load USD assets (specifically ``.usd``, ``.usda``, ``.usdc``, ``.usdz`` files) into MuJoCo via
|
||||
MJCF or dragging and dropping into :ref:`simulate.cc <saSimulate>`.
|
||||
* **Schemas:** MuJoCo primarily uses the standard `UsdPhysics
|
||||
<https://openusd.org/dev/api/usd_physics_page_front.html>`__ schemas for representing physics properties.
|
||||
* **Extensions:** Custom :doc:`mjcPhysics` schemas are provided to cover MuJoCo-specific features not available in
|
||||
``UsdPhysics``.
|
||||
* **MJCF File Format Plugin:** A :doc:`mjcf_file_format_plugin` allows treating MJCF files as USD layers in any native USD
|
||||
application.
|
||||
* **Export:** MuJoCo scenes can be exported to USD.
|
||||
|
||||
Where do I learn more about USD?
|
||||
------------------------------------------
|
||||
|
||||
* `Remedy's Book of USD <https://remedy-entertainment.github.io/USDBook>`__: Friendly introduction to USD.
|
||||
* `Official OpenUSD Documentation <https://openusd.org/release/intro.html>`__: Official documentation for API and
|
||||
implementation details.
|
||||
* `Pixar's USD Introduction <https://graphics.pixar.com/usd/release/index.html>`__: Simple example usage of USD.
|
||||
* `NVIDIA's USD Resources <https://developer.nvidia.com/usd>`__: Set of USD resources primarily concerned with asset
|
||||
structure.
|
||||
@@ -0,0 +1,97 @@
|
||||
mjcPhysics
|
||||
==========
|
||||
|
||||
.. WARNING:: OpenUSD support is currently experimental and subject to frequent change.
|
||||
|
||||
The ``mjcPhysics`` `schema <https://openusd.org/release/api/_usd__page__generating_schemas.html>`__ allows for detailed
|
||||
specification of a MuJoCo simulation environment directly within a USD file. The aim is not to replace `UsdPhysics
|
||||
<https://openusd.org/release/api/usd_physics_page_front.html>`__, but to extend existing concepts and create new types
|
||||
only where is necessary.
|
||||
|
||||
The schema can be use `codeless <https://openusd.org/dev/api/_usd__page__generating_schemas.html#Codeless_Schemas>`__,
|
||||
or can be built with its C++ bindings. We've pre-generated `the code
|
||||
<https://github.com/google-deepmind/mujoco/tree/main/src/experimental/usd/mjcPhysics>`__ via `usdGenSchema
|
||||
<https://openusd.org/dev/api/_usd__page__generating_schemas.html>`_ for internal MuJoCo usage, but it should also work
|
||||
outside of MuJoCo.
|
||||
|
||||
API Schemas
|
||||
-----------
|
||||
|
||||
MjcSceneAPI
|
||||
^^^^^^^^^^^
|
||||
|
||||
This API schema provides global options for the MuJoCo simulation. It is an
|
||||
amalgamation of the ``<option>``, ``<option/flag>`` and ``<compiler>`` elements in
|
||||
MJCF. Users should apply this to an existing
|
||||
`UsdPhysicsScene <https://openusd.org/dev/api/class_usd_physics_scene.html>`__
|
||||
prim.
|
||||
|
||||
Key attributes include:
|
||||
|
||||
- **mjc:option**: Attributes in this namespace map to the ``<option>`` element.
|
||||
- **mjc:flag**: Attributes in this namespace map to the ``<option/flag>`` element.
|
||||
- **mjc:compiler**: Attributes in this namespace map to the ``<compiler>`` element.
|
||||
|
||||
MjcSiteAPI
|
||||
^^^^^^^^^^
|
||||
|
||||
This API class is used to define a MuJoCo site, it can be applied to
|
||||
`UsdGeomSphere <https://openusd.org/dev/api/class_usd_geom_sphere.html>`__,
|
||||
`UsdGeomCapsule <https://openusd.org/dev/api/class_usd_geom_capsule.html>`__,
|
||||
`UsdGeomCylinder <https://openusd.org/dev/api/class_usd_geom_cylinder.html>`__, and
|
||||
`UsdGeomCube <https://openusd.org/dev/api/class_usd_geom_cube.html>`__.
|
||||
|
||||
MjcImageableAPI
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
This API class provides attributes for strictly visual entities in MuJoCo, in
|
||||
MuJoCo terms we would quantify these has having ``contype = conaffinity = 0``.
|
||||
|
||||
MjcCollisionAPI
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
This API class is applied to prims that represent collision geometry and should
|
||||
be applied alongside
|
||||
`UsdPhysicsCollisionAPI <https://openusd.org/dev/api/class_usd_physics_collision_a_p_i.html>`__.
|
||||
|
||||
MjcMeshCollisionAPI
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This API class is applied to prims that represent mesh collision geometry and
|
||||
should be applied alongside
|
||||
`UsdPhysicsMeshCollisionAPI <https://openusd.org/dev/api/class_usd_physics_mesh_collision_a_p_i.html>`__.
|
||||
|
||||
MjcJointAPI
|
||||
^^^^^^^^^^^
|
||||
|
||||
This API class is applied to `UsdPhysicsJoint <https://openusd.org/dev/api/class_usd_physics_joint.html>`__ prims,
|
||||
adding extra attributes to fully describe MuJoCo joints.
|
||||
|
||||
MjcMaterialAPI
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
This API class provides attributes for physical materials and is an extension of `UsdPhysicsMaterialAPI
|
||||
<https://openusd.org/dev/api/class_usd_physics_material_a_p_i.html>`__
|
||||
|
||||
Type Schemas
|
||||
------------
|
||||
|
||||
MjcActuator
|
||||
^^^^^^^^^^^
|
||||
|
||||
This class represents a MuJoCo actuator, which is responsible for applying force to a transmission target joint, body,
|
||||
or site specific via a `relationship <https://openusd.org/dev/api/class_usd_relationship.html>`__.
|
||||
|
||||
We do not use the existing `UsdPhysicsDriveAPI <https://openusd.org/dev/api/class_usd_physics_drive_a_p_i.html>`__ as it
|
||||
is closer to a runtime construct and the concepts do not map very closely.
|
||||
|
||||
MjcKeyframe
|
||||
^^^^^^^^^^^
|
||||
|
||||
This type holds tensor values representing simulator state at specific time values.
|
||||
|
||||
In MJCF this is the ``<keyframe>`` element and has a ``time`` attribute. In USD we map the time attribute to
|
||||
`timeSamples <https://openusd.org/release/tut_xforms.html>`__ instead.
|
||||
|
||||
The order of the values in the keyframes should map to the depth first ordered traversal of rigidbodies in the composed
|
||||
stage.
|
||||
@@ -0,0 +1,87 @@
|
||||
File Format Plugin
|
||||
=========================
|
||||
|
||||
What is an SdfFileFormat plugin?
|
||||
--------------------------------
|
||||
|
||||
In the OpenUSD framework, ``Sdf`` stands for Scene Description Foundations. It's the underlying layer that handles the
|
||||
serialization and composition of scene data. A ``SdfFileFormat`` plugin is a component that teaches USD how to read and
|
||||
write a specific file format.
|
||||
|
||||
By default, USD comes with plugins for its own formats (``.usda``, ``.usdc``, ``.usdz``) and the community has created
|
||||
several plugin extension such as the `Adobe File Format Plugins
|
||||
<https://github.com/adobe/USD-Fileformat-plugins/tree/main>`__.
|
||||
|
||||
The MJCF ``SdfFileFormat`` plugin allows USD-aware applications to directly understand and interact with MuJoCo's native
|
||||
``.xml`` (MJCF) files as if they were native USD files.
|
||||
|
||||
What does it enable?
|
||||
------------------------
|
||||
|
||||
This plugin enables:
|
||||
|
||||
1. **Referencing MJCF files in USD:** Using standard USD composition arcs (like references, payloads) to include an
|
||||
MJCF file directly within a larger USD scene. For example, you can place a MuJoCo robot defined in an ``.xml`` file
|
||||
into a room scene modeled in USD.
|
||||
2. **Load MJCF files in USD tools:** Tools like ``usdview`` or other USD-based
|
||||
applications can open, inspect, and render MJCF files, translating the MJCF elements into USD prims and attributes
|
||||
on the fly.
|
||||
3. **Convert MJCF to USD:** The plugin can be used as a basis for converting MJCF files to persistent
|
||||
USD files (e.g., ``.usda`` or ``.usdc``).
|
||||
|
||||
Essentially, it makes MJCF a first-class citizen in the USD ecosystem.
|
||||
|
||||
Usage
|
||||
------------------
|
||||
|
||||
1. **Installation:** refer to :doc:`building`.
|
||||
|
||||
2. **Referencing in a USD file (e.g., ``.usda``):**
|
||||
|
||||
.. code-block:: usd
|
||||
:caption: example.usda
|
||||
|
||||
#usda 1.0
|
||||
(
|
||||
upAxis = "Z"
|
||||
)
|
||||
|
||||
def Xform "world"
|
||||
{
|
||||
def "robot" (
|
||||
prepend references = @./my_robot.xml@
|
||||
)
|
||||
{
|
||||
}
|
||||
}
|
||||
|
||||
In this example, ``my_robot.xml`` is an MJCF file in the same directory. USD will use the plugin to load and
|
||||
interpret its contents.
|
||||
|
||||
3. **Opening in usdview:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
usdview my_robot.xml
|
||||
|
||||
If the plugin is correctly set up, ``usdview`` will render the robot defined in the MJCF file.
|
||||
|
||||
4. **Using in Python (with USD API):**
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from pxr import Usd
|
||||
|
||||
# Load an MJCF file as a USD stage
|
||||
stage = Usd.Stage.Open('my_robot.xml')
|
||||
|
||||
if stage:
|
||||
print(f"Successfully opened {stage.GetRootLayer().identifier}")
|
||||
# You can now inspect the stage as any other USD stage
|
||||
for prim in stage.TraverseAll():
|
||||
print(prim.GetPath())
|
||||
else:
|
||||
print("Failed to open MJCF file")
|
||||
|
||||
This plugin significantly enhances the interoperability between MuJoCo and USD-based workflows, allowing
|
||||
seamless integration of physics assets defined in MJCF into broader 3D environments.
|
||||
@@ -16,6 +16,7 @@
|
||||
python
|
||||
MJX <mjx>
|
||||
unity
|
||||
OpenUSD/index.rst
|
||||
models
|
||||
changelog
|
||||
|
||||
|
||||
Reference in New Issue
Block a user