chenlin/Mujoco_WASM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Copybara import of the project:

Browse Source 
--
3439d8a5cd745dfd2776593916eead65cb456afc by Kevin Zakka <kevinarmandzakka@gmail.com>:

Respect an explicit CMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF

The default-LTO guard checked the value rather than whether it was DEFINED, so an
explicit -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF (used in CI to cut build time)
was silently flipped back ON. Check DEFINED instead; default-on for Release is
preserved when the caller makes no choice.

--
bd15d1b264e5ab9594669b58fa1e0388d0a16f6b by Kevin Zakka <kevinarmandzakka@gmail.com>:

Speed up CI

~10x faster on the POSIX jobs (~45m -> ~4-5m) and ~2x overall wall-clock.

- Build Studio/Filament and WASM only in their dedicated canary jobs, not in
  every matrix job (WASM uses emcc, which ignores the host compiler).
- Compiler matrix -> build_matrix.json with core/extended tiers: PRs build the
  core set, pushes to main run the full sweep.
- ccache across build jobs (studio cache keyed on the Filament pin); fix the
  Python-bindings build so ccache hits (CCACHE_BASEDIR).
- Disable IPO/LTO on POSIX (it was silently on); keep it on Windows where /GL-off
  exposes a latent test bug and build time is not the bottleneck.
- Run ctest in parallel on POSIX (~2x); uv for Python installs (~29s -> ~8s).
- Move MJX (compiler-independent) to a single dedicated job.
- Restrict GITHUB_TOKEN to contents: read; bump actions off deprecated Node20.

--
c0618ab7aef524b238e0a30f2f23c50c4ce0c9b1 by Kevin Zakka <kevinarmandzakka@gmail.com>:

Build the gcc jobs with LTO off too

Restores LTO-off for gcc (recovering the build-time win). That surfaces known
gcc-12 -Wrestrict false positives in libstdc++ <char_traits> at -O3; two clean,
behavior-preserving rewrites in the XML writer avoid them. Confirmed gcc-12 and
gcc-14 build clean with LTO off (PR #3325).

--
2c38da0f48f77635e58e4b7931b86d60f3c253c7 by Kevin Zakka <kevinarmandzakka@gmail.com>:

Respect explicit IPO=OFF in the Simulate and Sample options too

Apply the same DEFINED check as cmake/MujocoOptions.cmake to the simulate/ and
sample/ subprojects (which carry identical copies of the guard) to keep the three
in sync, as the internal import requires. Since the guard now honors =OFF, stop
forcing IPO=OFF on the tiny samples/simulate CI builds so they keep their default
LTO and don't expose the gcc -Werror false positives that -O3-without-LTO triggers.

COPYBARA_INTEGRATE_REVIEW=https://github.com/google-deepmind/mujoco/pull/3315 from google-deepmind:speed-up-ci 2c38da0f48f77635e58e4b7931b86d60f3c253c7
PiperOrigin-RevId: 930008698
Change-Id: Ic41dc87881833c95f2ebfc0602de1ed438db3d4f
This commit is contained in:
Kevin Zakka
2026-06-10 12:26:36 -07:00
committed by Copybara-Service
parent 6957966c7d
commit 1aaced190a
11 changed files with 386 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build_steps.sh
+87 -10
View File
@@ -17,6 +17,34 @@
# consider making the builds parallel.
# Wrap the compiler with ccache when it is available (set up by ccache-action in
# CI). This makes warm rebuilds - including the expensive, pinned Filament build -
# much faster. ccache is content-addressed on the full compiler invocation, so
# changing a flag or source forces a recompile: a stale object is never reused.
# Guarded by `command -v` so the script still works locally without ccache.
CCACHE_ARGS=""
if command -v ccache >/dev/null 2>&1; then
CCACHE_ARGS="-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache"
fi
# Emit the build matrix for build.yml as a step output. On pull_request we run
# only the representative "core" compiler set; on push (e.g. to main) we run the
# full compiler sweep. Tiers are defined in build_matrix.json.
generate_matrix() {
echo "Generating build matrix for event '${GITHUB_EVENT_NAME}'..."
local file=".github/workflows/build_matrix.json"
local matrix
if [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
matrix="$(jq -c '{include: [.include[] | select(.tier == "core") | del(.tier)]}' "${file}")"
else
matrix="$(jq -c '{include: [.include[] | del(.tier)]}' "${file}")"
fi
echo "matrix=${matrix}" >> "${GITHUB_OUTPUT}"
echo "${matrix}" | jq .
}
prepare_linux() {
echo "Preparing Linux..."
sudo apt-get update && sudo apt-get install \
@@ -43,8 +71,17 @@ prepare_python() {
ln -s ../Scripts/activate venv/bin/activate
fi
source venv/bin/activate
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements.txt"
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements_usd.txt"
# Install build deps with uv when available (set up by setup-uv in CI on
# POSIX) - much faster than pip. Fall back to pip otherwise (e.g. Windows,
# local dev). The venv is still created by `python -m venv`, so pip stays
# available for later steps (pip wheel / python -m build).
if command -v uv > /dev/null 2>&1; then
uv pip install --require-hashes -r "${repo}/python/build_requirements.txt"
uv pip install --require-hashes -r "${repo}/python/build_requirements_usd.txt"
else
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements.txt"
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements_usd.txt"
fi
popd > /dev/null
}
@@ -73,13 +110,22 @@ setup_emsdk() {
configure_mujoco() {
echo "Configuring MuJoCo..."
# Disable IPO/LTO to cut build time. Skip this on Windows: turning off MSVC's
# whole-program optimization (/GL) exposes a latent heap corruption in
# SetConstTest.SleepingNotAllowed (a real bug worth a separate investigation),
# and Windows build time is not a CI bottleneck.
local ipo_off="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF"
if [[ "${RUNNER_OS}" == "Windows" ]]; then
ipo_off=""
fi
mkdir build &&
cd build &&
cmake .. \
-DCMAKE_BUILD_TYPE:STRING=Release \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
${ipo_off} \
-DCMAKE_INSTALL_PREFIX:STRING=${TMPDIR}/mujoco_install \
-DMUJOCO_BUILD_EXAMPLES:BOOL=OFF \
${CCACHE_ARGS} \
${CMAKE_ARGS}
}
@@ -92,7 +138,18 @@ build_mujoco() {
test_mujoco() {
echo "Testing MuJoCo..."
ctest -C Release --output-on-failure .
# ctest defaults to serial. The suite is ~1650 independent tests that use
# unique temp files (mkstemp / testing::TempDir) and declare no RUN_SERIAL /
# RESOURCE_LOCK, so running them in parallel is safe and ~2x faster on POSIX.
# Windows is kept serial conservatively: parallel-safety on the Windows file
# system is unverified and its test time is not a CI bottleneck.
if [[ "${RUNNER_OS}" == "Windows" ]]; then
ctest -C Release --output-on-failure .
else
local ncpu
ncpu="$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo "${NUMBER_OF_PROCESSORS:-2}")"
ctest -C Release --output-on-failure --parallel "${ncpu}" .
fi
}
@@ -123,24 +180,28 @@ copy_plugins_window() {
configure_samples() {
echo "Configuring samples..."
# Samples are tiny, so they keep the default IPO/LTO: disabling it saves no
# meaningful build time and would expose the same gcc -Werror false positives
# that -O3-without-LTO triggers (see configure_mujoco).
mkdir build &&
cd build &&
cmake .. \
-DCMAKE_BUILD_TYPE:STRING=Release \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
-Dmujoco_ROOT:STRING=${TMPDIR}/mujoco_install \
${CCACHE_ARGS} \
${CMAKE_ARGS}
}
configure_simulate() {
echo "Configuring simulate..."
# See configure_samples: keep the default IPO/LTO for this small build.
mkdir build &&
cd build &&
cmake .. \
-DCMAKE_BUILD_TYPE:STRING=Release \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
-Dmujoco_ROOT:STRING=${TMPDIR}/mujoco_install \
${CCACHE_ARGS} \
${CMAKE_ARGS}
}
@@ -165,6 +226,7 @@ configure_studio() {
-DMUJOCO_TEST_PYTHON_UTIL=OFF \
-DMUJOCO_WITH_USD=OFF \
-DMUJOCO_USE_FILAMENT=ON \
${CCACHE_ARGS} \
${CMAKE_ARGS}
echo "Configuring Studio... DONE"
}
@@ -186,10 +248,17 @@ make_python_sdist() {
build_python_bindings() {
echo "Building Python bindings..."
source ${TMPDIR}/venv/bin/activate &&
source ${TMPDIR}/venv/bin/activate
# pip unpacks the sdist into a randomized temp dir every run, so the absolute
# source/include paths differ each time and defeat ccache (0% hit, full
# recompile). CCACHE_BASEDIR rewrites absolute paths under it to paths relative
# to the (also-in-temp) build cwd, cancelling the random component so objects
# hash identically across runs. CCACHE_SLOPPINESS ignores timestamp/path noise.
export CCACHE_BASEDIR="${TMPDIR}"
export CCACHE_SLOPPINESS="time_macros,include_file_mtime,include_file_ctime,pch_defines,locale,system_headers"
MUJOCO_PATH="${TMPDIR}/mujoco_install" \
MUJOCO_PLUGIN_PATH="${TMPDIR}/mujoco_install/mujoco_plugin" \
MUJOCO_CMAKE_ARGS="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF ${CMAKE_ARGS}" \
MUJOCO_CMAKE_ARGS="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF ${CCACHE_ARGS} ${CMAKE_ARGS}" \
pip wheel -v --no-deps mujoco-*.tar.gz
}
@@ -216,6 +285,7 @@ build_test_wasm() {
emcmake cmake -B build_wasm_mt \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
-DMUJOCO_WASM_THREADS=ON \
${CCACHE_ARGS} \
$WASM_CMAKE_ARGS
cmake --build build_wasm_mt --parallel $(nproc)
@@ -230,6 +300,7 @@ build_test_wasm() {
emcmake cmake -B build_wasm_st \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
-DMUJOCO_WASM_THREADS=OFF \
${CCACHE_ARGS} \
$WASM_CMAKE_ARGS
cmake --build build_wasm_st --parallel $(nproc)
@@ -257,8 +328,14 @@ package_mjx() {
install_mjx() {
echo "Installing MJX..."
source ${TMPDIR}/venv/bin/activate &&
pip install --require-hashes -r requirements.txt &&
source ${TMPDIR}/venv/bin/activate
# The MJX requirements (jax, jaxlib, scipy, ...) are a big install; use uv when
# available. Keep pip for the local --no-index wheel.
if command -v uv > /dev/null 2>&1; then
uv pip install --require-hashes -r requirements.txt
else
pip install --require-hashes -r requirements.txt
fi
pip install --no-index dist/mujoco_mjx-*.whl
}