Mujoco_WASM/wasm/README.md

MuJoCo JavaScript Bindings

Caution

These bindings are not yet ready for general use. They have been added without announcement while we develop the CI we need to accept pull requests.

These are the canonical JavaScript and TypeScript bindings for the MuJoCo physics engine.

This package provides a high-level API that allows you to interact with the core MuJoCo engine compiled into a high-performance WebAssembly (WASM) module. These bindings are developed and maintained by Google DeepMind and are always up to date with the latest developments in MuJoCo. For brevity, the documentation below will often refer to “JavaScript” but the concepts apply equally to TypeScript.

Important

These bindings are still a WIP. For details, see the Future Work section. Also note that development has primarily taken place on Linux using Google Chrome. If you're working on a different OS or browser, you may encounter some rough edges.

Prerequisites

Note

Run all the commands in this README from the top-level directory.

• To compile the bindings.cc file, which generates the .wasm WebAssembly file, .js JavaScript import, and .d.ts TypeScript declaration file, you will need Emscripten SDK version 4.0.10. Later versions may work but are untested. To set up the SDK, do the following, you can run this anywhere but the rest of the commands in this README only work in the shell where you source the emsdk_env.sh script.

  git clone https://github.com/emscripten-core/emsdk.git
  ./emsdk/emsdk install 4.0.10
  ./emsdk/emsdk activate 4.0.10
  source ./emsdk/emsdk_env.sh
  

• To easily run the JavaScript tests and the demo application, node and npm are required. We recommend managing these using nvm. There are also various JavaScript dependencies needed for the tests, demo, and bindings build process. These dependencies are expected to be located in the wasm folder. To install them and ensure they can be found by later commands, run the following:

  npm install --prefix ./wasm
  export PATH="$(pwd)/wasm/node_modules/.bin:$PATH"
  

• To modify the bindings python3 is required because the bindings.cc file is generated by a Python script. To run the bindings generator tests, absl is required and pytest will be helpful. Set up a Python environment with these dependencies as follows:

  python3 -m venv .venv
  source .venv/bin/activate
  pip install -r python/build_requirements.txt
  

Tip

Emscripten is well-documented. We recommend reading the sections covering the Emscripten Compiler Settings, the Emscripten SDK, and the Embind library. To understand the limitations and caveats related to using the browser as a platform, see the Porting section.

User Guide

Bindings Generation

The bindings.cc file is compiled to generate to .wasm WebAssembly file, .js JavaScript import, and .d.ts TypeScript declaration file. These are the files you'll use to call MuJoCo from JavaScript. To generate them ensure the npm and Emscripten SDK prerequisites are set up and then run the following:

emcmake cmake -B build && cmake --build build

This command will generate the following folders under the project root:

• build: contains MuJoCo compiled using Emscripten.
• wasm/dist: contains the WebAssembly module, .js and .d.ts files.

Example Application

After generating the bindings you will be ready to write web applications using MuJoCo. We have provided a basic web application that uses Three.js to render a simple simulation, to try it run this command:

npm run dev:demo --prefix ./wasm

You may prefer to write your entire app in C++ and compile it using Emscripten. If you do this, you won’t need to use these bindings, since you’ll be writing minimal JavaScript, and the granularity of these bindings may be inappropriate (e.g., you might want to call multiple MuJoCo functions in the C++ callback invoked by requestAnimationFrame).

We have also found that a hybrid approach can be helpful, as it is often more convenient to work with browser APIs directly in JavaScript. If you choose to write your application in C++ and compile it using Emscripten, you may want to copy a subset of the EMSCRIPTEN_BINDINGS from bindings.cc into your application’s source file.

Development

In order to change the bindings you will need to change the bindings.cc file but this should not be done manually. The file is generated using the Python scripts and template files in the codegen folder, to edit the bindings you will need to change those files and re-generate bindings.cc using this command:

PYTHONPATH=python/mujoco python3 -m wasm.codegen.update

The codegen scripts use MuJoCo’s Python introspect library to generate the Embind EMSCRIPTEN_BINDINGS block that binds C++ functions and classes to JavaScript. The functions and classes that are bound are wrappers around MuJoCo's C API. These wrappers provide a convenient place to add features like bounds checking and nice error reporting.

Testing

1. JavaScript API tests. These verify that a wide variety of MuJoCo functions and classes work correctly when called from JavaScript. Run the tests as follows:

   npm run test --prefix ./wasm
   

   Note

   The above command runs the enums_tests.ts file which checks that all the enums in the API are bound. This test is auto-generated so if you add/edit an enum to MuJoCo then you will need to run the following command to re-create the test:

   PYTHONPATH=python/mujoco python3 -m wasm.codegen.enums_test_generator
   

2. JavaScript API benchmark tests. The current benchmark tests check JavaScript/C++ shared memory buffers performance. We will increase the coverage of the benchmarks overtime. Run the benchmarks using this command:

   npm run benchmark --prefix ./wasm
   

3. Bindings generator tests. These are relevant when developing or extending the bindings. The following command finds and runs all test_*.py or *_test.py files in the wasm folder:

   PYTHONPATH=python/mujoco python3 -m pytest ./wasm
   

Debugging

We provide a “sandbox” app where you can quickly write code to run in your browser. Write your code in the main.ts file and use the following command to execute it in your browser:

npm run dev:sandbox --prefix ./wasm

The page will be blank since the script only logs to the console output. You can add your code at the indicated placeholder and use Chrome DevTools for debugging. It is possible to set up a debug workflow where stack traces and stepping through code across language boundaries work correctly. Our current method to do this only works internally at Google, but it should be possible to replicate the experience with open-source tooling — community suggestions are welcome!

Future Work

1. Bind all useful APIs. These bindings are not yet complete. While the main MuJoCo APIs (mj_step, mj_loadXML, etc.) are well tested, other APIs (e.g., functions from mjspec.h) remain untested in real web applications (though test code for the mjspec bindings does exist). One notable feature not yet supported in the WASM bindings, which has proved very useful in the Python bindings, is named access methods — where data distributed across multiple arrays in C can be conveniently accessed by name, e.g., model.geom('mygeom') or data.joint('myjoint'). Currently, this data must be accessed via the mj_name2id function. Adding support for these features is a high priority, as it affects user code written in JavaScript.

2. Improve the developer experience. There is still work to be done to improve the developer experience when developing the WASM bindings. The most obvious issue is that bindings generation is not yet fully automated. As a result, it is currently less convenient than we'd like to identify and apply the changes needed to update the bindings. The goal is to eventually automate all binding code generation and clearly communicate what changes are required in the WASM bindings as a result of C++ updates. This problem should only affect developers working on the MuJoCo engine in C++, not end users writing JavaScript.

3. Improve the documentation. The documentation in this README will eventually be merged into the main MuJoCo documentation once the bindings are complete and named access is implemented. We also intend to review the bindings APIs and make adjustments to minimize differences with the Python bindings (while respecting language idioms) to reduce the amount of additional documentation required.

4. Improve the example. We aim to provide an example application that can be easily modified and embedded into a paper project page (see this example). This could be achieved by extending the Three.js example or by compiling the MuJoCo toolbox C++ code using the Emscripten toolchain. Community suggestions and contributions are welcome!