Copybara import of the project:
-- 3439d8a5cd745dfd2776593916eead65cb456afc by Kevin Zakka <kevinarmandzakka@gmail.com>: Respect an explicit CMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF The default-LTO guard checked the value rather than whether it was DEFINED, so an explicit -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF (used in CI to cut build time) was silently flipped back ON. Check DEFINED instead; default-on for Release is preserved when the caller makes no choice. -- bd15d1b264e5ab9594669b58fa1e0388d0a16f6b by Kevin Zakka <kevinarmandzakka@gmail.com>: Speed up CI ~10x faster on the POSIX jobs (~45m -> ~4-5m) and ~2x overall wall-clock. - Build Studio/Filament and WASM only in their dedicated canary jobs, not in every matrix job (WASM uses emcc, which ignores the host compiler). - Compiler matrix -> build_matrix.json with core/extended tiers: PRs build the core set, pushes to main run the full sweep. - ccache across build jobs (studio cache keyed on the Filament pin); fix the Python-bindings build so ccache hits (CCACHE_BASEDIR). - Disable IPO/LTO on POSIX (it was silently on); keep it on Windows where /GL-off exposes a latent test bug and build time is not the bottleneck. - Run ctest in parallel on POSIX (~2x); uv for Python installs (~29s -> ~8s). - Move MJX (compiler-independent) to a single dedicated job. - Restrict GITHUB_TOKEN to contents: read; bump actions off deprecated Node20. -- c0618ab7aef524b238e0a30f2f23c50c4ce0c9b1 by Kevin Zakka <kevinarmandzakka@gmail.com>: Build the gcc jobs with LTO off too Restores LTO-off for gcc (recovering the build-time win). That surfaces known gcc-12 -Wrestrict false positives in libstdc++ <char_traits> at -O3; two clean, behavior-preserving rewrites in the XML writer avoid them. Confirmed gcc-12 and gcc-14 build clean with LTO off (PR #3325). -- 2c38da0f48f77635e58e4b7931b86d60f3c253c7 by Kevin Zakka <kevinarmandzakka@gmail.com>: Respect explicit IPO=OFF in the Simulate and Sample options too Apply the same DEFINED check as cmake/MujocoOptions.cmake to the simulate/ and sample/ subprojects (which carry identical copies of the guard) to keep the three in sync, as the internal import requires. Since the guard now honors =OFF, stop forcing IPO=OFF on the tiny samples/simulate CI builds so they keep their default LTO and don't expose the gcc -Werror false positives that -O3-without-LTO triggers. COPYBARA_INTEGRATE_REVIEW=https://github.com/google-deepmind/mujoco/pull/3315 from google-deepmind:speed-up-ci 2c38da0f48f77635e58e4b7931b86d60f3c253c7 PiperOrigin-RevId: 930008698 Change-Id: Ic41dc87881833c95f2ebfc0602de1ed438db3d4f
This commit is contained in:
committed by
Copybara-Service
parent
6957966c7d
commit
1aaced190a
+183
-164
@@ -5,6 +5,17 @@
|
||||
#
|
||||
# We set CMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF here to reduce build time.
|
||||
# It is highly recommended that this is set to ON for production builds.
|
||||
#
|
||||
# Build speed notes:
|
||||
# * The compiler matrix lives in build_matrix.json. Pull requests build the
|
||||
# representative "core" subset; pushes to main run the full compiler sweep.
|
||||
# (See generate_matrix() in build_steps.sh.)
|
||||
# * Studio/Filament and WASM are NOT built in the main matrix - they are covered
|
||||
# by the dedicated `studio` and `wasm` jobs below. WASM in particular is built
|
||||
# with emcc, which ignores the host compiler, so building it per-compiler added
|
||||
# no coverage.
|
||||
# * ccache is used to wrap the compiler. PR branches fall back to main's cache, so
|
||||
# the expensive (and pinned) Filament build is restored rather than recompiled.
|
||||
|
||||
# TODO(matijak): Consider switching Windows to Ninja builds only, this configures slightly faster
|
||||
# and brings Windows in line with other OSes. Also we wouldn't need to specify the --config option
|
||||
@@ -24,110 +35,39 @@ on:
|
||||
- "doc/**"
|
||||
- "**/README.md"
|
||||
|
||||
# Limit the default GITHUB_TOKEN to read-only. No job here writes via the token:
|
||||
# checkout reads the repo; ccache/cache and upload-artifact use their own backend
|
||||
# tokens; the chat notification uses a secret webhook.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# Compute the build matrix based on the event: PRs get the "core" representative
|
||||
# compiler set, pushes to main get the full sweep.
|
||||
setup:
|
||||
name: "setup (compute matrix)"
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
matrix: ${{ steps.generate.outputs.matrix }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Generate build matrix
|
||||
id: generate
|
||||
env:
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
run: bash ./.github/workflows/build_steps.sh generate_matrix
|
||||
|
||||
mujoco:
|
||||
needs: setup
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-gcc-14"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=gcc-14
|
||||
-DCMAKE_CXX_COMPILER:STRING=g++-14
|
||||
-DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-gcc-13"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=gcc-13
|
||||
-DCMAKE_CXX_COMPILER:STRING=g++-13
|
||||
-DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-22.04
|
||||
label: "ubuntu-22.04-gcc-12"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=gcc-12
|
||||
-DCMAKE_CXX_COMPILER:STRING=g++-12
|
||||
-DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-clang-18"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-18
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-clang-17"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-17
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-17
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-clang-16"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-16
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-16
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-22.04
|
||||
label: "ubuntu-22.04-clang-15"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-15
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-15
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-22.04
|
||||
label: "ubuntu-22.04-clang-14"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-14
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-14
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: ubuntu-22.04
|
||||
label: "ubuntu-22.04-clang-13"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-13
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-13
|
||||
-DCMAKE_EXE_LINKER_FLAGS:STRING=-rtlib=compiler-rt
|
||||
-DCMAKE_SHARED_LINKER_FLAGS:STRING=-rtlib=compiler-rt
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: macos-15
|
||||
label: "macos-15-arm64"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: macos-15-large
|
||||
label: "macos-15-x86_64"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
tmpdir: "/tmp"
|
||||
- os: windows-2025
|
||||
label: "windows-2025"
|
||||
cmake_args: >-
|
||||
-DCMAKE_SYSTEM_VERSION="10.0.26100.0"
|
||||
cmake_build_args: "-- -m"
|
||||
tmpdir: "C:/Temp"
|
||||
matrix: ${{ fromJson(needs.setup.outputs.matrix) }}
|
||||
|
||||
name: "${{ matrix.label }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Prepare Linux
|
||||
if: ${{ runner.os == 'Linux' }}
|
||||
@@ -137,31 +77,36 @@ jobs:
|
||||
if: ${{ runner.os == 'macOS' }}
|
||||
run: brew install ninja
|
||||
|
||||
- uses: actions/setup-python@v4
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
# uv makes the dependency install much faster, with a cross-run cache. POSIX
|
||||
# only: Windows stays on pip (it is not a Python-setup bottleneck, and this
|
||||
# keeps the venv activate fixup path untouched there).
|
||||
- name: Install uv
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
uses: astral-sh/setup-uv@v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: "python/build_requirements*.txt"
|
||||
|
||||
- name: Prepare Python
|
||||
shell: bash
|
||||
env:
|
||||
TMPDIR: ${{ matrix.tmpdir }}
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_python
|
||||
|
||||
- name: Setup Node.js for WASM bindings
|
||||
if: ${{ runner.os == 'Linux' && matrix.label != 'ubuntu-24.04-clang-18' }}
|
||||
uses: actions/setup-node@v4
|
||||
# ccache dramatically speeds up warm rebuilds. The key is the matrix label so
|
||||
# each compiler gets its own cache; the action's restore-keys let PR branches
|
||||
# fall back to main's cache. ccache on Windows/MSVC is finicky and Windows
|
||||
# isn't the bottleneck, so we skip it there.
|
||||
- name: Setup ccache
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
uses: hendrikmuhs/ccache-action@v1.2.23
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
# Exclude platforms covered by the `wasm` job.
|
||||
- name: Install NPM Dependencies for WASM bindings
|
||||
if: ${{ runner.os == 'Linux' && matrix.label != 'ubuntu-24.04-clang-18' }}
|
||||
run: bash ./.github/workflows/build_steps.sh npm_ci
|
||||
|
||||
# Exclude platforms covered by the `wasm` job.
|
||||
- name: Setup Emscripten for WASM bindings
|
||||
if: ${{ runner.os == 'Linux' && matrix.label != 'ubuntu-24.04-clang-18' }}
|
||||
run: bash ./.github/workflows/build_steps.sh setup_emsdk
|
||||
key: ${{ matrix.label }}
|
||||
max-size: "1.5G"
|
||||
|
||||
- name: Configure MuJoCo
|
||||
env:
|
||||
@@ -171,7 +116,7 @@ jobs:
|
||||
|
||||
- name: Upload CMake Configure Log
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: cmake-configure-log-${{ matrix.label }}
|
||||
path: build/CMakeFiles/CMakeConfigureLog.yaml
|
||||
@@ -229,26 +174,6 @@ jobs:
|
||||
CMAKE_BUILD_ARGS: ${{ matrix.cmake_build_args }}
|
||||
run: bash ../../.github/workflows/build_steps.sh build_simulate
|
||||
|
||||
# Exclude platforms covered by the `studio` job.
|
||||
# Exclude macos-15-x86_64 because we don't have a way maintain/develop the build locally.
|
||||
- name: Configure Studio
|
||||
if: ${{ matrix.label != 'windows-2025' &&
|
||||
matrix.label != 'macos-15-arm64' &&
|
||||
matrix.label != 'macos-15-x86_64' }}
|
||||
env:
|
||||
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
||||
run: bash ./.github/workflows/build_steps.sh configure_studio
|
||||
|
||||
# Exclude platforms covered by the `studio` job.
|
||||
# Exclude macos-15-x86_64 because we don't have a way maintain/develop the build locally.
|
||||
# TODO(haroonq): Fix Filament compilation for GCC<=12?
|
||||
- name: Build Studio
|
||||
if: ${{ matrix.label != 'windows-2025' &&
|
||||
matrix.label != 'macos-15-arm64' &&
|
||||
matrix.label != 'macos-15-x86_64' &&
|
||||
matrix.label != 'ubuntu-22.04-gcc-12' }}
|
||||
run: bash ./.github/workflows/build_steps.sh build_studio
|
||||
|
||||
- name: Make Python sdist
|
||||
shell: bash
|
||||
working-directory: python
|
||||
@@ -281,35 +206,8 @@ jobs:
|
||||
TMPDIR: ${{ matrix.tmpdir }}
|
||||
run: bash ./.github/workflows/build_steps.sh test_python_bindings
|
||||
|
||||
# Exclude platforms covered by the `wasm` job.
|
||||
- name: Build and Test WASM bindings
|
||||
if: ${{ runner.os == 'Linux' && matrix.label != 'ubuntu-24.04-clang-18' }}
|
||||
shell: bash
|
||||
run: bash ./.github/workflows/build_steps.sh build_test_wasm
|
||||
|
||||
- name: Package MJX
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
shell: bash
|
||||
working-directory: mjx
|
||||
env:
|
||||
TMPDIR: ${{ matrix.tmpdir }}
|
||||
run: bash ../.github/workflows/build_steps.sh package_mjx
|
||||
|
||||
- name: Install MJX
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
shell: bash
|
||||
working-directory: mjx
|
||||
env:
|
||||
TMPDIR: ${{ matrix.tmpdir }}
|
||||
run: bash ../.github/workflows/build_steps.sh install_mjx
|
||||
|
||||
- name: Test MJX
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
shell: bash
|
||||
working-directory: mjx
|
||||
env:
|
||||
TMPDIR: ${{ matrix.tmpdir }}
|
||||
run: bash ../.github/workflows/build_steps.sh test_mjx
|
||||
# MJX (mujoco.mjx) is covered by the dedicated `mjx` job below; it is
|
||||
# compiler-independent so it does not need to run in every matrix job.
|
||||
|
||||
- name: Notify team chat
|
||||
shell: bash
|
||||
@@ -323,7 +221,8 @@ jobs:
|
||||
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
||||
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
||||
|
||||
# This job quickly determines if MuJoCo Studio is broken.
|
||||
# This job quickly determines if MuJoCo Studio is broken. It is the only place
|
||||
# Filament is compiled in CI, so ccache here is what keeps the long pole short.
|
||||
studio:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
@@ -336,6 +235,15 @@ jobs:
|
||||
-DCMAKE_C_COMPILER:STRING=clang-18
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
# gcc canary: Filament is compiler-dependent and historically finicky on
|
||||
# gcc, so keep one gcc Studio build even though the matrix no longer does.
|
||||
- os: ubuntu-24.04
|
||||
label: "ubuntu-24.04-gcc-14-studio"
|
||||
cmake_args: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=gcc-14
|
||||
-DCMAKE_CXX_COMPILER:STRING=g++-14
|
||||
-DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed
|
||||
- os: windows-2025
|
||||
label: "windows-2025-ninja-studio"
|
||||
cmake_args: >-
|
||||
@@ -352,7 +260,7 @@ jobs:
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Prepare Windows (setup MSVC)
|
||||
if: ${{ runner.os == 'Windows' }}
|
||||
@@ -362,6 +270,19 @@ jobs:
|
||||
if: ${{ runner.os == 'Linux' }}
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
||||
|
||||
# Key the cache on the Filament pin: hashing cmake/third_party_deps/filament.cmake
|
||||
# captures BOTH the pinned commit hash and the FILAMENT_* build flags, so bumping
|
||||
# either starts a fresh cache namespace instead of restoring stale artifacts.
|
||||
# (Correctness does not depend on this: ccache is content-addressed on the full
|
||||
# compiler invocation, so changing any build flag forces a recompile regardless.)
|
||||
- name: Setup ccache
|
||||
if: ${{ runner.os != 'Windows' }}
|
||||
uses: hendrikmuhs/ccache-action@v1.2.23
|
||||
with:
|
||||
key: ${{ matrix.label }}-filament-${{ hashFiles('cmake/third_party_deps/filament.cmake') }}
|
||||
restore-keys: ${{ matrix.label }}-filament-
|
||||
max-size: "2.0G"
|
||||
|
||||
- name: Configure Studio
|
||||
env:
|
||||
CMAKE_ARGS: ${{ matrix.cmake_args }}
|
||||
@@ -395,10 +316,10 @@ jobs:
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Node.js for WASM bindings
|
||||
uses: actions/setup-node@v4
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
@@ -406,6 +327,12 @@ jobs:
|
||||
if: ${{ runner.os == 'Linux' }}
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
||||
|
||||
- name: Setup ccache
|
||||
uses: hendrikmuhs/ccache-action@v1.2.23
|
||||
with:
|
||||
key: ${{ env.label }}
|
||||
max-size: "1.5G"
|
||||
|
||||
# For convenience run multiple build_steps functions in a single step.
|
||||
- name: Prepare, Build and Test WASM bindings
|
||||
env:
|
||||
@@ -426,3 +353,95 @@ jobs:
|
||||
CHATMSG_JOB_ID: ${{ env.label }}
|
||||
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
||||
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
||||
|
||||
|
||||
# MJX (mujoco.mjx) is pure JAX/XLA and compiler-independent: it imports the
|
||||
# mujoco Python package only as a reference, so its tests run once here instead
|
||||
# of in every matrix job. MuJoCo is built WITHOUT the C++ test suite (MJX does
|
||||
# not need it) to keep the build lean.
|
||||
mjx:
|
||||
name: "ubuntu-24.04-clang-18-mjx"
|
||||
runs-on: ubuntu-24.04
|
||||
env:
|
||||
TMPDIR: "/tmp"
|
||||
CMAKE_ARGS: >-
|
||||
-G Ninja
|
||||
-DCMAKE_C_COMPILER:STRING=clang-18
|
||||
-DCMAKE_CXX_COMPILER:STRING=clang++-18
|
||||
-DMUJOCO_HARDEN:BOOL=ON
|
||||
-DMUJOCO_BUILD_TESTS:BOOL=OFF
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Prepare Linux
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
||||
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: "python/build_requirements*.txt"
|
||||
|
||||
- name: Prepare Python
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_python
|
||||
|
||||
- name: Setup ccache
|
||||
uses: hendrikmuhs/ccache-action@v1.2.23
|
||||
with:
|
||||
key: ubuntu-24.04-clang-18-mjx
|
||||
max-size: "1.0G"
|
||||
|
||||
- name: Configure MuJoCo
|
||||
run: bash ./.github/workflows/build_steps.sh configure_mujoco
|
||||
|
||||
- name: Build MuJoCo
|
||||
working-directory: build
|
||||
run: bash ../.github/workflows/build_steps.sh build_mujoco
|
||||
|
||||
- name: Install MuJoCo
|
||||
working-directory: build
|
||||
run: bash ../.github/workflows/build_steps.sh install_mujoco
|
||||
|
||||
- name: Copy plugins
|
||||
working-directory: build
|
||||
run: bash ../.github/workflows/build_steps.sh copy_plugins_posix
|
||||
|
||||
- name: Make Python sdist
|
||||
working-directory: python
|
||||
run: bash ../.github/workflows/build_steps.sh make_python_sdist
|
||||
|
||||
- name: Build Python bindings
|
||||
working-directory: python/dist
|
||||
run: bash ../../.github/workflows/build_steps.sh build_python_bindings
|
||||
|
||||
- name: Install Python bindings
|
||||
working-directory: python/dist
|
||||
run: bash ../../.github/workflows/build_steps.sh install_python_bindings
|
||||
|
||||
- name: Package MJX
|
||||
working-directory: mjx
|
||||
run: bash ../.github/workflows/build_steps.sh package_mjx
|
||||
|
||||
- name: Install MJX
|
||||
working-directory: mjx
|
||||
run: bash ../.github/workflows/build_steps.sh install_mjx
|
||||
|
||||
- name: Test MJX
|
||||
working-directory: mjx
|
||||
run: bash ../.github/workflows/build_steps.sh test_mjx
|
||||
|
||||
- name: Notify team chat
|
||||
shell: bash
|
||||
env:
|
||||
GCHAT_API_URL: ${{ secrets.GCHAT_API }}
|
||||
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
||||
CHATMSG_AUTHOR_NAME: ${{ github.event.head_commit.author.name }}
|
||||
CHATMSG_AUTHOR_EMAIL: ${{ github.event.head_commit.author.email }}
|
||||
CHATMSG_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
|
||||
CHATMSG_JOB_ID: "ubuntu-24.04-clang-18-mjx"
|
||||
if: failure() && github.ref_name == 'main' && github.event_name == 'push' && env.GCHAT_API_URL != ''
|
||||
run: bash ./.github/workflows/build_steps.sh notify_team_chat
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
{
|
||||
"comment": "Build matrix for build.yml. 'tier' controls when an entry runs: 'core' runs on every pull_request and push; 'extended' runs only on push to main (full compiler sweep). See generate_matrix() in build_steps.sh.",
|
||||
"include": [
|
||||
{
|
||||
"os": "ubuntu-24.04",
|
||||
"label": "ubuntu-24.04-gcc-14",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=gcc-14 -DCMAKE_CXX_COMPILER:STRING=g++-14 -DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-24.04",
|
||||
"label": "ubuntu-24.04-gcc-13",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=gcc-13 -DCMAKE_CXX_COMPILER:STRING=g++-13 -DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "extended"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-22.04",
|
||||
"label": "ubuntu-22.04-gcc-12",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=gcc-12 -DCMAKE_CXX_COMPILER:STRING=g++-12 -DCMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--no-as-needed",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-24.04",
|
||||
"label": "ubuntu-24.04-clang-18",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-18 -DCMAKE_CXX_COMPILER:STRING=clang++-18 -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-24.04",
|
||||
"label": "ubuntu-24.04-clang-17",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-17 -DCMAKE_CXX_COMPILER:STRING=clang++-17 -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "extended"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-24.04",
|
||||
"label": "ubuntu-24.04-clang-16",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-16 -DCMAKE_CXX_COMPILER:STRING=clang++-16 -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "extended"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-22.04",
|
||||
"label": "ubuntu-22.04-clang-15",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-15 -DCMAKE_CXX_COMPILER:STRING=clang++-15 -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "extended"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-22.04",
|
||||
"label": "ubuntu-22.04-clang-14",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-14 -DCMAKE_CXX_COMPILER:STRING=clang++-14 -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "extended"
|
||||
},
|
||||
{
|
||||
"os": "ubuntu-22.04",
|
||||
"label": "ubuntu-22.04-clang-13",
|
||||
"cmake_args": "-G Ninja -DCMAKE_C_COMPILER:STRING=clang-13 -DCMAKE_CXX_COMPILER:STRING=clang++-13 -DCMAKE_EXE_LINKER_FLAGS:STRING=-rtlib=compiler-rt -DCMAKE_SHARED_LINKER_FLAGS:STRING=-rtlib=compiler-rt -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "macos-15",
|
||||
"label": "macos-15-arm64",
|
||||
"cmake_args": "-G Ninja -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "macos-15-large",
|
||||
"label": "macos-15-x86_64",
|
||||
"cmake_args": "-G Ninja -DMUJOCO_HARDEN:BOOL=ON",
|
||||
"tmpdir": "/tmp",
|
||||
"tier": "core"
|
||||
},
|
||||
{
|
||||
"os": "windows-2025",
|
||||
"label": "windows-2025",
|
||||
"cmake_args": "-DCMAKE_SYSTEM_VERSION=\"10.0.26100.0\"",
|
||||
"cmake_build_args": "-- -m",
|
||||
"tmpdir": "C:/Temp",
|
||||
"tier": "core"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -17,6 +17,34 @@
|
||||
# consider making the builds parallel.
|
||||
|
||||
|
||||
# Wrap the compiler with ccache when it is available (set up by ccache-action in
|
||||
# CI). This makes warm rebuilds - including the expensive, pinned Filament build -
|
||||
# much faster. ccache is content-addressed on the full compiler invocation, so
|
||||
# changing a flag or source forces a recompile: a stale object is never reused.
|
||||
# Guarded by `command -v` so the script still works locally without ccache.
|
||||
CCACHE_ARGS=""
|
||||
if command -v ccache >/dev/null 2>&1; then
|
||||
CCACHE_ARGS="-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache"
|
||||
fi
|
||||
|
||||
|
||||
# Emit the build matrix for build.yml as a step output. On pull_request we run
|
||||
# only the representative "core" compiler set; on push (e.g. to main) we run the
|
||||
# full compiler sweep. Tiers are defined in build_matrix.json.
|
||||
generate_matrix() {
|
||||
echo "Generating build matrix for event '${GITHUB_EVENT_NAME}'..."
|
||||
local file=".github/workflows/build_matrix.json"
|
||||
local matrix
|
||||
if [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
|
||||
matrix="$(jq -c '{include: [.include[] | select(.tier == "core") | del(.tier)]}' "${file}")"
|
||||
else
|
||||
matrix="$(jq -c '{include: [.include[] | del(.tier)]}' "${file}")"
|
||||
fi
|
||||
echo "matrix=${matrix}" >> "${GITHUB_OUTPUT}"
|
||||
echo "${matrix}" | jq .
|
||||
}
|
||||
|
||||
|
||||
prepare_linux() {
|
||||
echo "Preparing Linux..."
|
||||
sudo apt-get update && sudo apt-get install \
|
||||
@@ -43,8 +71,17 @@ prepare_python() {
|
||||
ln -s ../Scripts/activate venv/bin/activate
|
||||
fi
|
||||
source venv/bin/activate
|
||||
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements.txt"
|
||||
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements_usd.txt"
|
||||
# Install build deps with uv when available (set up by setup-uv in CI on
|
||||
# POSIX) - much faster than pip. Fall back to pip otherwise (e.g. Windows,
|
||||
# local dev). The venv is still created by `python -m venv`, so pip stays
|
||||
# available for later steps (pip wheel / python -m build).
|
||||
if command -v uv > /dev/null 2>&1; then
|
||||
uv pip install --require-hashes -r "${repo}/python/build_requirements.txt"
|
||||
uv pip install --require-hashes -r "${repo}/python/build_requirements_usd.txt"
|
||||
else
|
||||
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements.txt"
|
||||
python -m pip install --upgrade --require-hashes -r "${repo}/python/build_requirements_usd.txt"
|
||||
fi
|
||||
popd > /dev/null
|
||||
}
|
||||
|
||||
@@ -73,13 +110,22 @@ setup_emsdk() {
|
||||
|
||||
configure_mujoco() {
|
||||
echo "Configuring MuJoCo..."
|
||||
# Disable IPO/LTO to cut build time. Skip this on Windows: turning off MSVC's
|
||||
# whole-program optimization (/GL) exposes a latent heap corruption in
|
||||
# SetConstTest.SleepingNotAllowed (a real bug worth a separate investigation),
|
||||
# and Windows build time is not a CI bottleneck.
|
||||
local ipo_off="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF"
|
||||
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
||||
ipo_off=""
|
||||
fi
|
||||
mkdir build &&
|
||||
cd build &&
|
||||
cmake .. \
|
||||
-DCMAKE_BUILD_TYPE:STRING=Release \
|
||||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
|
||||
${ipo_off} \
|
||||
-DCMAKE_INSTALL_PREFIX:STRING=${TMPDIR}/mujoco_install \
|
||||
-DMUJOCO_BUILD_EXAMPLES:BOOL=OFF \
|
||||
${CCACHE_ARGS} \
|
||||
${CMAKE_ARGS}
|
||||
}
|
||||
|
||||
@@ -92,7 +138,18 @@ build_mujoco() {
|
||||
|
||||
test_mujoco() {
|
||||
echo "Testing MuJoCo..."
|
||||
ctest -C Release --output-on-failure .
|
||||
# ctest defaults to serial. The suite is ~1650 independent tests that use
|
||||
# unique temp files (mkstemp / testing::TempDir) and declare no RUN_SERIAL /
|
||||
# RESOURCE_LOCK, so running them in parallel is safe and ~2x faster on POSIX.
|
||||
# Windows is kept serial conservatively: parallel-safety on the Windows file
|
||||
# system is unverified and its test time is not a CI bottleneck.
|
||||
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
||||
ctest -C Release --output-on-failure .
|
||||
else
|
||||
local ncpu
|
||||
ncpu="$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo "${NUMBER_OF_PROCESSORS:-2}")"
|
||||
ctest -C Release --output-on-failure --parallel "${ncpu}" .
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
@@ -123,24 +180,28 @@ copy_plugins_window() {
|
||||
|
||||
configure_samples() {
|
||||
echo "Configuring samples..."
|
||||
# Samples are tiny, so they keep the default IPO/LTO: disabling it saves no
|
||||
# meaningful build time and would expose the same gcc -Werror false positives
|
||||
# that -O3-without-LTO triggers (see configure_mujoco).
|
||||
mkdir build &&
|
||||
cd build &&
|
||||
cmake .. \
|
||||
-DCMAKE_BUILD_TYPE:STRING=Release \
|
||||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
|
||||
-Dmujoco_ROOT:STRING=${TMPDIR}/mujoco_install \
|
||||
${CCACHE_ARGS} \
|
||||
${CMAKE_ARGS}
|
||||
}
|
||||
|
||||
|
||||
configure_simulate() {
|
||||
echo "Configuring simulate..."
|
||||
# See configure_samples: keep the default IPO/LTO for this small build.
|
||||
mkdir build &&
|
||||
cd build &&
|
||||
cmake .. \
|
||||
-DCMAKE_BUILD_TYPE:STRING=Release \
|
||||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
|
||||
-Dmujoco_ROOT:STRING=${TMPDIR}/mujoco_install \
|
||||
${CCACHE_ARGS} \
|
||||
${CMAKE_ARGS}
|
||||
}
|
||||
|
||||
@@ -165,6 +226,7 @@ configure_studio() {
|
||||
-DMUJOCO_TEST_PYTHON_UTIL=OFF \
|
||||
-DMUJOCO_WITH_USD=OFF \
|
||||
-DMUJOCO_USE_FILAMENT=ON \
|
||||
${CCACHE_ARGS} \
|
||||
${CMAKE_ARGS}
|
||||
echo "Configuring Studio... DONE"
|
||||
}
|
||||
@@ -186,10 +248,17 @@ make_python_sdist() {
|
||||
|
||||
build_python_bindings() {
|
||||
echo "Building Python bindings..."
|
||||
source ${TMPDIR}/venv/bin/activate &&
|
||||
source ${TMPDIR}/venv/bin/activate
|
||||
# pip unpacks the sdist into a randomized temp dir every run, so the absolute
|
||||
# source/include paths differ each time and defeat ccache (0% hit, full
|
||||
# recompile). CCACHE_BASEDIR rewrites absolute paths under it to paths relative
|
||||
# to the (also-in-temp) build cwd, cancelling the random component so objects
|
||||
# hash identically across runs. CCACHE_SLOPPINESS ignores timestamp/path noise.
|
||||
export CCACHE_BASEDIR="${TMPDIR}"
|
||||
export CCACHE_SLOPPINESS="time_macros,include_file_mtime,include_file_ctime,pch_defines,locale,system_headers"
|
||||
MUJOCO_PATH="${TMPDIR}/mujoco_install" \
|
||||
MUJOCO_PLUGIN_PATH="${TMPDIR}/mujoco_install/mujoco_plugin" \
|
||||
MUJOCO_CMAKE_ARGS="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF ${CMAKE_ARGS}" \
|
||||
MUJOCO_CMAKE_ARGS="-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF ${CCACHE_ARGS} ${CMAKE_ARGS}" \
|
||||
pip wheel -v --no-deps mujoco-*.tar.gz
|
||||
}
|
||||
|
||||
@@ -216,6 +285,7 @@ build_test_wasm() {
|
||||
emcmake cmake -B build_wasm_mt \
|
||||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
|
||||
-DMUJOCO_WASM_THREADS=ON \
|
||||
${CCACHE_ARGS} \
|
||||
$WASM_CMAKE_ARGS
|
||||
cmake --build build_wasm_mt --parallel $(nproc)
|
||||
|
||||
@@ -230,6 +300,7 @@ build_test_wasm() {
|
||||
emcmake cmake -B build_wasm_st \
|
||||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF \
|
||||
-DMUJOCO_WASM_THREADS=OFF \
|
||||
${CCACHE_ARGS} \
|
||||
$WASM_CMAKE_ARGS
|
||||
cmake --build build_wasm_st --parallel $(nproc)
|
||||
|
||||
@@ -257,8 +328,14 @@ package_mjx() {
|
||||
|
||||
install_mjx() {
|
||||
echo "Installing MJX..."
|
||||
source ${TMPDIR}/venv/bin/activate &&
|
||||
pip install --require-hashes -r requirements.txt &&
|
||||
source ${TMPDIR}/venv/bin/activate
|
||||
# The MJX requirements (jax, jaxlib, scipy, ...) are a big install; use uv when
|
||||
# available. Keep pip for the local --no-index wheel.
|
||||
if command -v uv > /dev/null 2>&1; then
|
||||
uv pip install --require-hashes -r requirements.txt
|
||||
else
|
||||
pip install --require-hashes -r requirements.txt
|
||||
fi
|
||||
pip install --no-index dist/mujoco_mjx-*.whl
|
||||
}
|
||||
|
||||
|
||||
@@ -9,10 +9,10 @@ jobs:
|
||||
name: pre-commit
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- uses: pre-commit/action@v3.0.1
|
||||
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
build-and-upload-artifacts:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Prepare Linux
|
||||
run: bash ./.github/workflows/build_steps.sh prepare_linux
|
||||
|
||||
@@ -25,10 +25,10 @@ jobs:
|
||||
VERSION: ${{ github.event_name == 'workflow_dispatch' && inputs.tag_version || github.ref_name }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Node.js for WASM bindings
|
||||
uses: actions/setup-node@v4
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '24.x'
|
||||
|
||||
|
||||
@@ -108,7 +108,12 @@ if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR (CMAKE_CXX_COMPILER_ID MATCHES "Clang
|
||||
endif()
|
||||
endif()
|
||||
|
||||
if(NOT CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
# Enable interprocedural optimization (LTO) by default for non-Debug builds, but
|
||||
# only when the caller has not made an explicit choice. Checking the value (rather
|
||||
# than whether it is DEFINED) meant an explicit `-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF`
|
||||
# was silently overridden back to ON, e.g. in CI where LTO is disabled to cut
|
||||
# build time.
|
||||
if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON)
|
||||
endif()
|
||||
|
||||
|
||||
@@ -108,7 +108,12 @@ if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR (CMAKE_CXX_COMPILER_ID MATCHES "Clang
|
||||
endif()
|
||||
endif()
|
||||
|
||||
if(NOT CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
# Enable interprocedural optimization (LTO) by default for non-Debug builds, but
|
||||
# only when the caller has not made an explicit choice. Checking the value (rather
|
||||
# than whether it is DEFINED) meant an explicit `-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF`
|
||||
# was silently overridden back to ON, e.g. in CI where LTO is disabled to cut
|
||||
# build time.
|
||||
if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON)
|
||||
endif()
|
||||
|
||||
|
||||
@@ -108,7 +108,12 @@ if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR (CMAKE_CXX_COMPILER_ID MATCHES "Clang
|
||||
endif()
|
||||
endif()
|
||||
|
||||
if(NOT CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
# Enable interprocedural optimization (LTO) by default for non-Debug builds, but
|
||||
# only when the caller has not made an explicit choice. Checking the value (rather
|
||||
# than whether it is DEFINED) meant an explicit `-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF`
|
||||
# was silently overridden back to ON, e.g. in CI where LTO is disabled to cut
|
||||
# build time.
|
||||
if(NOT DEFINED CMAKE_INTERPROCEDURAL_OPTIMIZATION AND (CMAKE_BUILD_TYPE AND NOT CMAKE_BUILD_TYPE STREQUAL "Debug"))
|
||||
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON)
|
||||
endif()
|
||||
|
||||
|
||||
@@ -97,7 +97,7 @@ static string WriteDoc(XMLDocument& doc, char *error, size_t error_sz) {
|
||||
|
||||
// insert another newline
|
||||
if (line_pos != string::npos) {
|
||||
str.insert(line_pos + 1, "\n");
|
||||
str.insert(line_pos + 1, 1, '\n');
|
||||
pos++; // account for inserted newline
|
||||
}
|
||||
|
||||
|
||||
+2
-1
@@ -1165,7 +1165,8 @@ void mjXUtil::WriteAttrKeys(XMLElement* elem, std::string name, const mjMap* map
|
||||
|
||||
std::string text = FindValue(map, mapsz, data[0]);
|
||||
for (int i = 1; i < ndata; ++i) {
|
||||
text += " " + FindValue(map, mapsz, data[i]);
|
||||
text += " ";
|
||||
text += FindValue(map, mapsz, data[i]);
|
||||
}
|
||||
|
||||
WriteAttrTxt(elem, name, text);
|
||||
|
||||
Reference in New Issue
Block a user