Merge pull request #3111 from StafaH:main

PiperOrigin-RevId: 871825088
Change-Id: I1e9c4a559a19427753cfdbacc7c7254ad0cb9c6c

doc/mjwarp/index.rst

@@ -21,15 +21,16 @@ MJWarp is developed and maintained as a joint effort by `NVIDIA <https://nvidia.
 Tutorial notebook
 =================
 
-The MJWarp basics are covered in a
+The MJWarp basics are covered in a tutorial |notebook| |open in colab|.
-`tutorial
+
-notebook <https://colab.research.google.com/github/google-deepmind/mujoco_warp/blob/main/notebooks/tutorial.ipynb>`__.
+.. |notebook| replace:: `[notebook] <https://github.com/google-deepmind/mujoco_warp/blob/main/notebooks/tutorial.ipynb>`__
+
+.. |open in colab| replace:: `[open in colab] <https://colab.research.google.com/github/google-deepmind/mujoco_warp/blob/main/notebooks/tutorial.ipynb>`__
+
 
 When To Use MJWarp?
 ===================
 
-.. TODO(robotics-simulation): batch renderer
-
 High throughput
 ---------------
 
@@ -365,9 +366,6 @@ To enable this routine set ``Model.opt.ls_parallel=True`` or add a custom numeri
 
   The parallel linesearch is currently an experimental feature.
 
-.. _mjwBatch:
-
-
 Memory
 ------
 
@@ -409,6 +407,8 @@ total contacts with per-collider contacts: mesh-mesh: 30 (CCD), ellipsoid-ellips
 (primitive) should set `nconmax`_ / `naconmax`_ to at least 8 / 80 (may require more for broadphase) and ``nccdmax`` /
 ``naccdmax`` to 3 / 30.
 
+.. _mjwBatch:
+
 Batched :class:`Model <mujoco_warp.Model>` Fields
 =================================================
 
@@ -464,6 +464,104 @@ subset of fields.
 
    Heterogeneous worlds, for example: per-world meshes or number of degrees of freedom, are not currently available.
 
+Batch Rendering
+===============
+
+MJWarp provides a high-throughput ray-tracing batch renderer built on
+`Warp's accelerated BVHs <https://nvidia.github.io/warp/api_reference/_generated/warp.Bvh.html#warp.Bvh>`__ for
+rendering worlds with multiple cameras in parallel on device.
+
+Key features:
+
+- **Mesh rendering with textures**: BVH-accelerated mesh rendering with full texture support.
+- **Heightfield rendering**: Optimized rendering for heightfields.
+- **Flex rendering**: Render 2D and 3D :ref:`flex<deformable-flex>` objects.
+- **Lighting and shadows**: Dynamic lighting with configurable shadows; domain randomizable: `light_active`,
+  `light_type`, `light_castshadow`, `light_xpos`, `light_xdir`.
+- **Heterogeneous multi-camera**: Multiple cameras per world and each camera can have a different resolution
+  (`cam_resolution`), field of view (`cam_fovy`, `cam_sensorsize`, `cam_intrinsic`), and output mode (`cam_output`).
+- **Domain Randomization**: Per-world :class:`mjw.Model <mujoco_warp.Model>` fields (see
+  :ref:`Batched Model Fields <mjwBatch>` above): `geom_matid`, `geom_size`, `geom_rgba`, `mat_texid`, `mat_texrepeat`,
+  `mat_rgba`.
+- **BVH-accelerated ray/rays API**: Ray casting: Accelerated :func:`mjw.ray <mujoco_warp.ray>`,
+  :func:`mjw.rays <mujoco_warp.rays>`, and :ref:`rangefinder sensors <sensor-rangefinder>` via
+  `Warp's BVHs <https://nvidia.github.io/warp/api_reference/_generated/warp.Bvh.html#warp.Bvh>`__.
+
+Basic Usage
+----------
+
+Rendering or raycasting requires a :class:`mjw.RenderContext <mujoco_warp.RenderContext>` which contains BVH structures
+and rendering and output buffers.
+
+.. code-block:: python
+
+    rc = mjw.create_render_context(
+        mjm,
+        nworld=1,
+        cam_res=(256, 256),           # Override camera resolution (or per-camera list)
+        render_rgb=True,              # Enable RGB output (or per-camera list)
+        render_depth=True,            # Enable depth output (or per-camera list)
+        use_textures=True,            # Apply material textures
+        use_shadows=False,            # Enable shadow casting (slower)
+        enabled_geom_groups=[0, 1],   # Only render geoms in groups 0 and 1
+        cam_active=[True, False],     # Selectively enable/disable cameras
+        flex_render_smooth=True,      # Smooth shading for soft bodies
+    )
+
+Each :class:`mjw.RenderContext <mujoco_warp.RenderContext>` parameter can be applied globally or per camera.
+Additionally, values for :class:`mjw.RenderContext <mujoco_warp.RenderContext>` parameters can be parsed from XML:
+
+.. code-block:: xml
+
+    <camera name="front_camera" pos="3 0 2" xyaxes="0 1 0 -0.6 0 0.8" resolution="64 64" output="rgb depth"/>
+
+or set via :ref:`mjSpec <mjspec>` for camera customization.
+
+To render, first call :func:`mjw.refit_bvh <mujoco_warp.refit_bvh>` to update the BVH trees,
+followed by :func:`mjw.render <mujoco_warp.render>` to write to output buffers.
+
+.. code-block:: python
+
+    mjw.refit_bvh(m, d, rc)
+    mjw.render(m, d, rc)
+
+The output buffers contain stacked pixels for all cameras with shape `(nworld, npixel)` and RGB data is
+packed into one `unit32` variable. `RenderContext.rgb_adr` and `RenderContext.depth_adr` provide per-camera indexing.
+For convenience, :func:`mjw.get_rgb <mujoco_warp.get_rgb>` and :func:`mjw.get_depth <mujoco_warp.get_depth>`
+provide per-camera batched post-processing.
+
+.. code-block:: python
+
+    nworld = 1
+    cam_index = 0
+    resolution = rc.cam_res.numpy()[cam_index]
+    rgb_data = wp.zeros((nworld, resolution[1], resolution[0]), dtype=wp.vec3)
+    mjw.get_rgb(rc, rgb_data=rgb_data, cam_id=cam_index)
+
+A complete example can be found in the MJWarp tutorial |notebook| |open in colab|.
+
+Benchmarks
+----------
+
+Rendering can be benchmarked using `testspeed`_:
+
+.. code-block:: shell
+
+    mjwarp-testspeed benchmarks/primitives.xml --function=render
+
+For benchmark results across a variety of scenes, see the
+`released benchmarks <https://github.com/google-deepmind/mujoco_warp/pull/1113>`__.
+
+Notes
+-----
+
+- **Meshes**: Rendering computation scales with mesh complexity. A primitive is expected to have better
+  performance (i.e., higher throughput) compared to a similar sized :ref:`mesh<body-geom-mesh>` or
+  :ref:`heightfield <body-geom-hfield>`.
+- **Flex**: Currently limited to 2D and 3D :ref:`flex<deformable-flex>` objects. Performance is expected to improved as
+  this feature is further developed.
+- **Scaling**: Rendering scales linearly with resolution (total number of pixels) and number of cameras.
+
 .. _mjwFAQ:
 
 Frequently Asked Questions