Kevin Zakka
1aaced190a
-- 3439d8a5cd745dfd2776593916eead65cb456afc by Kevin Zakka <kevinarmandzakka@gmail.com>: Respect an explicit CMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF The default-LTO guard checked the value rather than whether it was DEFINED, so an explicit -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF (used in CI to cut build time) was silently flipped back ON. Check DEFINED instead; default-on for Release is preserved when the caller makes no choice. -- bd15d1b264e5ab9594669b58fa1e0388d0a16f6b by Kevin Zakka <kevinarmandzakka@gmail.com>: Speed up CI ~10x faster on the POSIX jobs (~45m -> ~4-5m) and ~2x overall wall-clock. - Build Studio/Filament and WASM only in their dedicated canary jobs, not in every matrix job (WASM uses emcc, which ignores the host compiler). - Compiler matrix -> build_matrix.json with core/extended tiers: PRs build the core set, pushes to main run the full sweep. - ccache across build jobs (studio cache keyed on the Filament pin); fix the Python-bindings build so ccache hits (CCACHE_BASEDIR). - Disable IPO/LTO on POSIX (it was silently on); keep it on Windows where /GL-off exposes a latent test bug and build time is not the bottleneck. - Run ctest in parallel on POSIX (~2x); uv for Python installs (~29s -> ~8s). - Move MJX (compiler-independent) to a single dedicated job. - Restrict GITHUB_TOKEN to contents: read; bump actions off deprecated Node20. -- c0618ab7aef524b238e0a30f2f23c50c4ce0c9b1 by Kevin Zakka <kevinarmandzakka@gmail.com>: Build the gcc jobs with LTO off too Restores LTO-off for gcc (recovering the build-time win). That surfaces known gcc-12 -Wrestrict false positives in libstdc++ <char_traits> at -O3; two clean, behavior-preserving rewrites in the XML writer avoid them. Confirmed gcc-12 and gcc-14 build clean with LTO off (PR #3325). -- 2c38da0f48f77635e58e4b7931b86d60f3c253c7 by Kevin Zakka <kevinarmandzakka@gmail.com>: Respect explicit IPO=OFF in the Simulate and Sample options too Apply the same DEFINED check as cmake/MujocoOptions.cmake to the simulate/ and sample/ subprojects (which carry identical copies of the guard) to keep the three in sync, as the internal import requires. Since the guard now honors =OFF, stop forcing IPO=OFF on the tiny samples/simulate CI builds so they keep their default LTO and don't expose the gcc -Werror false positives that -O3-without-LTO triggers. COPYBARA_INTEGRATE_REVIEW=https://github.com/google-deepmind/mujoco/pull/3315 from google-deepmind:speed-up-ci 2c38da0f48f77635e58e4b7931b86d60f3c253c7 PiperOrigin-RevId: 930008698 Change-Id: Ic41dc87881833c95f2ebfc0602de1ed438db3d4f
448 lines
16 KiB
YAML
448 lines
16 KiB
YAML
# This configuration is used to build and test on GitHub Actions only.
|
|
# It is not the same configuration that is used by Google DeepMind to create release binaries.
|
|
# The "official" binaries are built with Clang 13 on all platforms, and are linked against libc++
|
|
# on Linux.
|
|
#
|
|
# We set CMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF here to reduce build time.
|
|
# It is highly recommended that this is set to ON for production builds.
|
|
#
|
|
# Build speed notes:
|
|
# * The compiler matrix lives in build_matrix.json. Pull requests build the
|
|
# representative "core" subset; pushes to main run the full compiler sweep.
|
|
# (See generate_matrix() in build_steps.sh.)
|
|
# * Studio/Filament and WASM are NOT built in the main matrix - they are covered
|
|
# by the dedicated `studio` and `wasm` jobs below. WASM in particular is built
|
|
# with emcc, which ignores the host compiler, so building it per-compiler added
|
|
# no coverage.
|
|
# * ccache is used to wrap the compiler. PR branches fall back to main's cache, so
|
|
# the expensive (and pinned) Filament build is restored rather than recompiled.
|
|
|
|
# TODO(matijak): Consider switching Windows to Ninja builds only, this configures slightly faster
|
|
# and brings Windows in line with other OSes. Also we wouldn't need to specify the --config option
|
|
# in the build step because Ninja doesn't support multiple configurations (unlike MSBuild).
|
|
|
|
name: build
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
paths-ignore:
|
|
- "doc/**"
|
|
- "**/README.md"
|
|
pull_request:
|
|
paths-ignore:
|
|
- "doc/**"
|
|
- "**/README.md"
|
|
|
|
# Limit the default GITHUB_TOKEN to read-only. No job here writes via the token:
|
|
# checkout reads the repo; ccache/cache and upload-artifact use their own backend
|
|
# tokens; the chat notification uses a secret webhook.
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
# Compute the build matrix based on the event: PRs get the "core" representative
|
|
# compiler set, pushes to main get the full sweep.
|
|
setup:
|
|
name: "setup (compute matrix)"
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
matrix: ${{ steps.generate.outputs.matrix }}
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- name: Generate build matrix
|
|
id: generate
|
|
env:
|
|
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
|
run: bash ./.github/workflows/build_steps.sh generate_matrix
|
|
|
|
mujoco:
|
|
needs: setup
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJson(needs.setup.outputs.matrix) }}
|
|
|
|
name: "${{ matrix.label }}"
|
|
runs-on: ${{ matrix.os }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Prepare Linux
|
|
if: ${{ runner.os == 'Linux' }}
|
|
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
|
|
|
- name: Prepare macOS
|
|
if: ${{ runner.os == 'macOS' }}
|
|
run: brew install ninja
|
|
|
|
- uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
# uv makes the dependency install much faster, with a cross-run cache. POSIX
|
|
# only: Windows stays on pip (it is not a Python-setup bottleneck, and this
|
|
# keeps the venv activate fixup path untouched there).
|
|
- name: Install uv
|
|
if: ${{ runner.os != 'Windows' }}
|
|
uses: astral-sh/setup-uv@v8.2.0
|
|
with:
|
|
enable-cache: true
|
|
cache-dependency-glob: "python/build_requirements*.txt"
|
|
|
|
- name: Prepare Python
|
|
shell: bash
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ./.github/workflows/build_steps.sh prepare_python
|
|
|
|
# ccache dramatically speeds up warm rebuilds. The key is the matrix label so
|
|
# each compiler gets its own cache; the action's restore-keys let PR branches
|
|
# fall back to main's cache. ccache on Windows/MSVC is finicky and Windows
|
|
# isn't the bottleneck, so we skip it there.
|
|
- name: Setup ccache
|
|
if: ${{ runner.os != 'Windows' }}
|
|
uses: hendrikmuhs/ccache-action@v1.2.23
|
|
with:
|
|
key: ${{ matrix.label }}
|
|
max-size: "1.5G"
|
|
|
|
- name: Configure MuJoCo
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
|
run: bash ./.github/workflows/build_steps.sh configure_mujoco
|
|
|
|
- name: Upload CMake Configure Log
|
|
if: always()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: cmake-configure-log-${{ matrix.label }}
|
|
path: build/CMakeFiles/CMakeConfigureLog.yaml
|
|
retention-days: 5
|
|
|
|
- name: Build MuJoCo
|
|
working-directory: build
|
|
env:
|
|
CMAKE_BUILD_ARGS: ${{ matrix.cmake_build_args }}
|
|
run: bash ../.github/workflows/build_steps.sh build_mujoco
|
|
|
|
- name: Test MuJoCo
|
|
working-directory: build
|
|
run: bash ../.github/workflows/build_steps.sh test_mujoco
|
|
|
|
- name: Install MuJoCo
|
|
working-directory: build
|
|
run: bash ../.github/workflows/build_steps.sh install_mujoco
|
|
|
|
- name: Copy plugins (POSIX)
|
|
if: ${{ runner.os != 'Windows' }}
|
|
working-directory: build
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ../.github/workflows/build_steps.sh copy_plugins_posix
|
|
|
|
- name: Copy plugins (Windows)
|
|
if: ${{ runner.os == 'Windows' }}
|
|
working-directory: build
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ../.github/workflows/build_steps.sh copy_plugins_window
|
|
|
|
- name: Configure samples
|
|
working-directory: sample
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
|
run: bash ../.github/workflows/build_steps.sh configure_samples
|
|
|
|
- name: Build samples
|
|
working-directory: sample/build
|
|
run: cmake --build . --config=Release ${{ matrix.cmake_build_args }}
|
|
|
|
- name: Configure simulate
|
|
working-directory: simulate
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
|
run: bash ../.github/workflows/build_steps.sh configure_simulate
|
|
|
|
- name: Build simulate
|
|
working-directory: simulate/build
|
|
env:
|
|
CMAKE_BUILD_ARGS: ${{ matrix.cmake_build_args }}
|
|
run: bash ../../.github/workflows/build_steps.sh build_simulate
|
|
|
|
- name: Make Python sdist
|
|
shell: bash
|
|
working-directory: python
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ../.github/workflows/build_steps.sh make_python_sdist
|
|
|
|
- name: Build Python bindings
|
|
if: ${{ runner.os != 'Windows' }}
|
|
shell: bash
|
|
working-directory: python/dist
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
|
run: bash ../../.github/workflows/build_steps.sh build_python_bindings
|
|
|
|
- name: Install Python bindings
|
|
if: ${{ runner.os != 'Windows' }}
|
|
shell: bash
|
|
working-directory: python/dist
|
|
env:
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ../../.github/workflows/build_steps.sh install_python_bindings
|
|
|
|
- name: Test Python bindings
|
|
if: ${{ runner.os != 'Windows' }}
|
|
shell: bash
|
|
env:
|
|
MUJOCO_GL: disable
|
|
TMPDIR: ${{ matrix.tmpdir }}
|
|
run: bash ./.github/workflows/build_steps.sh test_python_bindings
|
|
|
|
# MJX (mujoco.mjx) is covered by the dedicated `mjx` job below; it is
|
|
# compiler-independent so it does not need to run in every matrix job.
|
|
|
|
- name: Notify team chat
|
|
shell: bash
|
|
env:
|
|
GCHAT_API_URL: ${{ secrets.GCHAT_API }}
|
|
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
|
CHATMSG_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
|
|
CHATMSG_AUTHOR_EMAIL: ${{ github.event.head_commit.author.email }}
|
|
CHATMSG_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
|
|
CHATMSG_JOB_ID: ${{ matrix.label }}
|
|
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
|
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
|
|
|
# This job quickly determines if MuJoCo Studio is broken. It is the only place
|
|
# Filament is compiled in CI, so ccache here is what keeps the long pole short.
|
|
studio:
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: ubuntu-24.04
|
|
label: "ubuntu-24.04-clang-18-studio"
|
|
cmake_args: >-
|
|
-G Ninja
|
|
-DCMAKE_C_COMPILER:STRING=clang-18
|
|
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
|
-DMUJOCO_HARDEN:BOOL=ON
|
|
# gcc canary: Filament is compiler-dependent and historically finicky on
|
|
# gcc, so keep one gcc Studio build even though the matrix no longer does.
|
|
- os: ubuntu-24.04
|
|
label: "ubuntu-24.04-gcc-14-studio"
|
|
cmake_args: >-
|
|
-G Ninja
|
|
-DCMAKE_C_COMPILER:STRING=gcc-14
|
|
-DCMAKE_CXX_COMPILER:STRING=g++-14
|
|
-DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed
|
|
- os: windows-2025
|
|
label: "windows-2025-ninja-studio"
|
|
cmake_args: >-
|
|
-G Ninja
|
|
-DCMAKE_SYSTEM_VERSION=10.0.26100.0
|
|
- os: macos-15
|
|
label: "macos-15-arm64-studio"
|
|
cmake_args: >-
|
|
-G Ninja
|
|
-DCMAKE_CXX_FLAGS="-Wno-error=deprecated-declarations"
|
|
-DMUJOCO_HARDEN:BOOL=ON
|
|
|
|
name: "${{ matrix.label }}"
|
|
runs-on: ${{ matrix.os }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Prepare Windows (setup MSVC)
|
|
if: ${{ runner.os == 'Windows' }}
|
|
uses: ilammy/msvc-dev-cmd@v1
|
|
|
|
- name: Prepare Linux
|
|
if: ${{ runner.os == 'Linux' }}
|
|
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
|
|
|
# Key the cache on the Filament pin: hashing cmake/third_party_deps/filament.cmake
|
|
# captures BOTH the pinned commit hash and the FILAMENT_* build flags, so bumping
|
|
# either starts a fresh cache namespace instead of restoring stale artifacts.
|
|
# (Correctness does not depend on this: ccache is content-addressed on the full
|
|
# compiler invocation, so changing any build flag forces a recompile regardless.)
|
|
- name: Setup ccache
|
|
if: ${{ runner.os != 'Windows' }}
|
|
uses: hendrikmuhs/ccache-action@v1.2.23
|
|
with:
|
|
key: ${{ matrix.label }}-filament-${{ hashFiles('cmake/third_party_deps/filament.cmake') }}
|
|
restore-keys: ${{ matrix.label }}-filament-
|
|
max-size: "2.0G"
|
|
|
|
- name: Configure Studio
|
|
env:
|
|
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
|
run: bash ./.github/workflows/build_steps.sh configure_studio
|
|
|
|
- name: Build Studio
|
|
run: bash ./.github/workflows/build_steps.sh build_studio
|
|
|
|
- name: Notify team chat
|
|
shell: bash
|
|
env:
|
|
GCHAT_API_URL: ${{ secrets.GCHAT_API }}
|
|
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
|
CHATMSG_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
|
|
CHATMSG_AUTHOR_EMAIL: ${{ github.event.head_commit.author.email }}
|
|
CHATMSG_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
|
|
CHATMSG_JOB_ID: ${{ matrix.label }}
|
|
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
|
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
|
|
|
|
|
# This job quickly determines if WASM bindings are broken.
|
|
wasm:
|
|
name: "ubuntu-24.04-clang-18-wasm"
|
|
runs-on: ubuntu-24.04
|
|
env:
|
|
label: "ubuntu-24.04-clang-18-wasm"
|
|
cmake_args: >-
|
|
-G Ninja
|
|
-DCMAKE_C_COMPILER:STRING=clang-18
|
|
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
|
-DMUJOCO_HARDEN:BOOL=ON
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Setup Node.js for WASM bindings
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Prepare Linux
|
|
if: ${{ runner.os == 'Linux' }}
|
|
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
|
|
|
- name: Setup ccache
|
|
uses: hendrikmuhs/ccache-action@v1.2.23
|
|
with:
|
|
key: ${{ env.label }}
|
|
max-size: "1.5G"
|
|
|
|
# For convenience run multiple build_steps functions in a single step.
|
|
- name: Prepare, Build and Test WASM bindings
|
|
env:
|
|
CMAKE_ARGS: ${{ env.cmake_args }}
|
|
run: |
|
|
bash ./.github/workflows/build_steps.sh npm_ci
|
|
bash ./.github/workflows/build_steps.sh setup_emsdk
|
|
bash ./.github/workflows/build_steps.sh build_test_wasm
|
|
|
|
- name: Notify team chat
|
|
shell: bash
|
|
env:
|
|
GCHAT_API_URL: ${{ secrets.GCHAT_API }}
|
|
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
|
CHATMSG_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
|
|
CHATMSG_AUTHOR_EMAIL: ${{ github.event.head_commit.author.email }}
|
|
CHATMSG_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
|
|
CHATMSG_JOB_ID: ${{ env.label }}
|
|
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
|
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
|
|
|
|
|
# MJX (mujoco.mjx) is pure JAX/XLA and compiler-independent: it imports the
|
|
# mujoco Python package only as a reference, so its tests run once here instead
|
|
# of in every matrix job. MuJoCo is built WITHOUT the C++ test suite (MJX does
|
|
# not need it) to keep the build lean.
|
|
mjx:
|
|
name: "ubuntu-24.04-clang-18-mjx"
|
|
runs-on: ubuntu-24.04
|
|
env:
|
|
TMPDIR: "/tmp"
|
|
CMAKE_ARGS: >-
|
|
-G Ninja
|
|
-DCMAKE_C_COMPILER:STRING=clang-18
|
|
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
|
-DMUJOCO_HARDEN:BOOL=ON
|
|
-DMUJOCO_BUILD_TESTS:BOOL=OFF
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Prepare Linux
|
|
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
|
|
|
- uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v8.2.0
|
|
with:
|
|
enable-cache: true
|
|
cache-dependency-glob: "python/build_requirements*.txt"
|
|
|
|
- name: Prepare Python
|
|
run: bash ./.github/workflows/build_steps.sh prepare_python
|
|
|
|
- name: Setup ccache
|
|
uses: hendrikmuhs/ccache-action@v1.2.23
|
|
with:
|
|
key: ubuntu-24.04-clang-18-mjx
|
|
max-size: "1.0G"
|
|
|
|
- name: Configure MuJoCo
|
|
run: bash ./.github/workflows/build_steps.sh configure_mujoco
|
|
|
|
- name: Build MuJoCo
|
|
working-directory: build
|
|
run: bash ../.github/workflows/build_steps.sh build_mujoco
|
|
|
|
- name: Install MuJoCo
|
|
working-directory: build
|
|
run: bash ../.github/workflows/build_steps.sh install_mujoco
|
|
|
|
- name: Copy plugins
|
|
working-directory: build
|
|
run: bash ../.github/workflows/build_steps.sh copy_plugins_posix
|
|
|
|
- name: Make Python sdist
|
|
working-directory: python
|
|
run: bash ../.github/workflows/build_steps.sh make_python_sdist
|
|
|
|
- name: Build Python bindings
|
|
working-directory: python/dist
|
|
run: bash ../../.github/workflows/build_steps.sh build_python_bindings
|
|
|
|
- name: Install Python bindings
|
|
working-directory: python/dist
|
|
run: bash ../../.github/workflows/build_steps.sh install_python_bindings
|
|
|
|
- name: Package MJX
|
|
working-directory: mjx
|
|
run: bash ../.github/workflows/build_steps.sh package_mjx
|
|
|
|
- name: Install MJX
|
|
working-directory: mjx
|
|
run: bash ../.github/workflows/build_steps.sh install_mjx
|
|
|
|
- name: Test MJX
|
|
working-directory: mjx
|
|
run: bash ../.github/workflows/build_steps.sh test_mjx
|
|
|
|
- name: Notify team chat
|
|
shell: bash
|
|
env:
|
|
GCHAT_API_URL: ${{ secrets.GCHAT_API }}
|
|
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
|
CHATMSG_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
|
|
CHATMSG_AUTHOR_EMAIL: ${{ github.event.head_commit.author.email }}
|
|
CHATMSG_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
|
|
CHATMSG_JOB_ID: "ubuntu-24.04-clang-18-mjx"
|
|
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
|
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|