Wave 351/509: Kernel rig: every area and the ladder as a parallel gate on pull requests that touch the kernel (and 3 more)
4 commits, squashed; the originals stay reachable from tag archive/merge-main-develop. - 6c5537df6 jyviko: Kernel ladder: a gate run fails on any case whose status or counts differ from the golden - ee2a1ea06 jyviko: CI: maintainers' pull requests run the tests, and the reference tests run on pull requests - b4fcd5910 jyviko: Kernel rig: every area and the ladder as a parallel gate on pull requests that touch the kernel - 18d2dddac jyviko: Release gate: every check in full and in parallel before a branch replaces main Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtL5M41G8Nua7HvuFPZkmJ
This commit is contained in:
+29
-12
@@ -12,8 +12,9 @@
|
||||
# checks (no archive to upload and download). The GPU path tracer's tests run one at a time
|
||||
# (.config/nextest.toml), so they have a job of their own in four shards (GPU path tests). A few
|
||||
# heavy tests (see .config/nextest.toml) are skipped on pull requests and run on pushes to main and
|
||||
# nightly. The reference tests and the doc tests run in the merge queue, on main and nightly, not
|
||||
# on pull requests (the fast lane, in Select checks); a break there is fixed forward on main. The
|
||||
# nightly. The reference tests run on a pull request that selects solidcraft-cli (they replay
|
||||
# recipes through it). The doc tests run in the merge queue, on main and nightly, not on pull
|
||||
# requests (the fast lane, in Select checks); a break there is fixed forward on main. The
|
||||
# Windows jobs (D3D12 shaders and app startup) run nightly, and in the merge queue only for a group
|
||||
# that changes what they exercise. Only main saves the build caches; pull requests restore them.
|
||||
# sccache is on a Blacksmith sticky disk that only main commits.
|
||||
@@ -21,8 +22,9 @@
|
||||
# commit, or when nothing a build reads changed since the last push to main whose run passed.
|
||||
# Otherwise it runs everything.
|
||||
# Heavy checks wait for review: a pull request without the `approved-for-ci` label runs only
|
||||
# Select checks, the format and repository checks and a quick compile. Adding the label (after
|
||||
# review) runs everything above, and every later push to the PR does too, except a push that only
|
||||
# Select checks, the format and repository checks and a quick compile, unless its author is an
|
||||
# owner, member or collaborator of the repository (they could push the code to a branch here
|
||||
# anyway). Adding the label (after review) runs everything above, and every later push to the PR does too, except a push that only
|
||||
# merges main into a change that already passed (scripts/ci/main_merge.py): it carries that
|
||||
# result and runs the repository checks. Merge groups, the nightly run and Run workflow are not
|
||||
# pull requests and always run the full suite.
|
||||
@@ -31,6 +33,8 @@
|
||||
# `ci` job still runs and reports the commit's real result (see the `ci` job): the merge queue
|
||||
# reads `ci` from the newest run of a commit, so a run without it would block a PR that passed.
|
||||
# Cross-compilation is opt-in through Run workflow; release builds remain in release.yml.
|
||||
# The release gate (release-gate.yml) calls this workflow (workflow_call) as a Run workflow with
|
||||
# cross-compilation on.
|
||||
# Every Linux job runs on Blacksmith's Ubuntu 24.04 runners (blacksmith-<N>vcpu-ubuntu-2404), sized
|
||||
# per job: 32 vCPU for the tests, 16 for Clippy, the GPU path tests, the archive build (nightly) and the merge queue's, 8 for compiles, doc and reference tests, 4 for the light checks, 2 for Select checks and ci.
|
||||
# The Windows shader job runs on Blacksmith's Windows Server 2025 runners (blacksmith-8vcpu-windows-2025);
|
||||
@@ -64,6 +68,14 @@ on:
|
||||
description: "Also check WebAssembly and RISC-V (before a release)"
|
||||
type: boolean
|
||||
default: false
|
||||
# The release gate (release-gate.yml): the event stays the caller's (workflow_dispatch), so a
|
||||
# call runs exactly what Run workflow runs.
|
||||
workflow_call:
|
||||
inputs:
|
||||
cross_compile:
|
||||
description: "Also check WebAssembly and RISC-V (before a release)"
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -137,6 +149,9 @@ jobs:
|
||||
HEAD_REF: ${{ github.event.pull_request.head.ref }}
|
||||
REPO: ${{ github.repository }}
|
||||
APPROVED: ${{ contains(github.event.pull_request.labels.*.name, 'approved-for-ci') }}
|
||||
# An owner, member or collaborator could push the change to a branch here: their pull
|
||||
# requests need no label.
|
||||
TRUSTED: ${{ contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association) }}
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
native=false
|
||||
@@ -230,16 +245,18 @@ jobs:
|
||||
if [ "$EVENT_NAME" = pull_request ] && [ "$full" != true ]; then
|
||||
case " $packages " in *" solidcraft-cli "*) ;; *) oracle=false ;; esac
|
||||
fi
|
||||
# The pull request fast lane: Windows, the reference tests and the doc tests run on main
|
||||
# (and the nightly), not on every pull request push; a break there is fixed forward on
|
||||
# main. Over three days of pull requests the doc tests were the only failing job 3 times
|
||||
# in about 7,900 runner-minutes, and the reference tests never were.
|
||||
# The pull request fast lane: Windows and the doc tests run on main (and the nightly),
|
||||
# not on every pull request push; a break there is fixed forward on main. Over three
|
||||
# days of pull requests the doc tests were the only failing job 3 times in about 7,900
|
||||
# runner-minutes. The reference tests run when solidcraft-cli is selected (above): a
|
||||
# kernel change that broke them went unseen until a full run on develop.
|
||||
doctests=$native
|
||||
if [ "$EVENT_NAME" = pull_request ]; then windows=false; oracle=false; doctests=false; fi
|
||||
# Until a maintainer adds approved-for-ci, a pull request gets a quick compile instead of
|
||||
# Clippy, the tests and the doc tests.
|
||||
if [ "$EVENT_NAME" = pull_request ]; then windows=false; doctests=false; fi
|
||||
# Until a maintainer adds approved-for-ci, a pull request from outside the repository's
|
||||
# owners, members and collaborators gets a quick compile instead of Clippy, the tests,
|
||||
# the reference tests and the doc tests.
|
||||
quick=false
|
||||
if [ "$EVENT_NAME" = pull_request ] && [ "$APPROVED" != true ]; then
|
||||
if [ "$EVENT_NAME" = pull_request ] && [ "$APPROVED" != true ] && [ "$TRUSTED" != true ]; then
|
||||
quick=$native
|
||||
native=false
|
||||
oracle=false
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
# The kernel rig and the complexity ladder as a regression gate. A pull request that touches the
|
||||
# kernel surface (the paths below) builds solidcraft-cli once (release, SMLib prebuilt), hands it
|
||||
# to a matrix of shards as an artifact, and each shard runs one area of tests/kernel-cases/ (or
|
||||
# the ladder) in parallel, so the wall time is the build plus the slowest shard. See
|
||||
# tests/kernel-cases/README.md.
|
||||
#
|
||||
# - An area fails on a regression (a case that fails and its tests/kernel-cases/<area>/KNOWN_FAILING
|
||||
# does not list) or on a listed case that passes (it is fixed: remove its line in the same pull
|
||||
# request, so the list only shrinks).
|
||||
# - The ladder fails on any case whose status or counts differ from
|
||||
# tests/kernel-cases/ladder/golden.json; a right change is re-recorded (`--record "why"`).
|
||||
#
|
||||
# GitHub filters a workflow by whole files, so any change to crates/doc/src/eval*.rs or
|
||||
# crates/engine/src/cmd/features*.rs selects it. Pushes to main and develop that touch the same
|
||||
# paths run it too and commit the compiler cache (a Blacksmith sticky disk) that every other run
|
||||
# mounts read-only. A merge group (once the queue is on) always runs it: it is the last check
|
||||
# before the change lands. The release gate (release-gate.yml) calls it (workflow_call).
|
||||
name: Kernel rig
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, ready_for_review]
|
||||
paths:
|
||||
- "crates/kernel/**"
|
||||
- "crates/geom/**"
|
||||
- "crates/smlib/**"
|
||||
- "crates/step/**"
|
||||
- "crates/doc/src/eval*.rs"
|
||||
- "crates/engine/src/cmd/features*.rs"
|
||||
- "tests/kernel-cases/**"
|
||||
# The rig itself: the grading, the runner and this workflow.
|
||||
- "apps/solidcraft-cli/src/kernel_cases.rs"
|
||||
- "xtask/src/kernel_cases.rs"
|
||||
- "xtask/src/kernel_ladder.rs"
|
||||
- "scripts/fetch-smlib.sh"
|
||||
- ".github/workflows/kernel-rig.yml"
|
||||
push:
|
||||
branches: [main, develop]
|
||||
# The same paths as pull_request.
|
||||
paths:
|
||||
- "crates/kernel/**"
|
||||
- "crates/geom/**"
|
||||
- "crates/smlib/**"
|
||||
- "crates/step/**"
|
||||
- "crates/doc/src/eval*.rs"
|
||||
- "crates/engine/src/cmd/features*.rs"
|
||||
- "tests/kernel-cases/**"
|
||||
# The rig itself: the grading, the runner and this workflow.
|
||||
- "apps/solidcraft-cli/src/kernel_cases.rs"
|
||||
- "xtask/src/kernel_cases.rs"
|
||||
- "xtask/src/kernel_ladder.rs"
|
||||
- "scripts/fetch-smlib.sh"
|
||||
- ".github/workflows/kernel-rig.yml"
|
||||
merge_group:
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
# A new push to a pull request replaces its old run.
|
||||
group: kernel-rig-${{ github.event_name }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build the CLI
|
||||
# Draft pull requests run no CI, as in ci.yml.
|
||||
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
|
||||
runs-on: blacksmith-8vcpu-ubuntu-2404
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
CARGO_INCREMENTAL: "0"
|
||||
RUSTC_WRAPPER: sccache
|
||||
SCCACHE_DIR: /home/runner/.cache/sccache
|
||||
SCCACHE_CACHE_SIZE: 10G
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- name: Size parallel compiles to the job's memory
|
||||
run: scripts/ci/build-jobs.sh
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
# Only pushes to main and develop commit the cache; pull requests, merge groups and forks
|
||||
# mount a clone and never write back.
|
||||
- name: Mount the compiler cache
|
||||
uses: useblacksmith/stickydisk@3f6be1451e3cad893b778bc93495c80452a5da26 # v1.7.2
|
||||
continue-on-error: true
|
||||
with:
|
||||
key: ${{ github.repository }}-sccache-kernel-rig
|
||||
path: /home/runner/.cache/sccache
|
||||
commit: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop') && 'on-change' || 'false' }}
|
||||
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
|
||||
with:
|
||||
key: kernel-rig
|
||||
save-if: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop') }}
|
||||
- name: Fetch SMLib (crates/smlib/SMLIB_REV)
|
||||
shell: bash
|
||||
run: scripts/fetch-smlib.sh
|
||||
- name: Set up compiler cache
|
||||
uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
|
||||
with:
|
||||
version: v0.16.0
|
||||
- name: Build the CLI (release, SMLib) and the rig's runner
|
||||
run: |
|
||||
cargo build --release --locked -p solidcraft-cli --features smlib
|
||||
cargo build --locked -p xtask
|
||||
# A tar keeps the executable bits, which an artifact drops. xtask finds the repository from
|
||||
# the path it was built in (CARGO_MANIFEST_DIR), which is the same checkout path in every
|
||||
# job of this workflow.
|
||||
- name: Pack the binaries
|
||||
run: tar -cf "$RUNNER_TEMP/kernel-rig-bin.tar" -C target release/solidcraft-cli debug/xtask
|
||||
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
with:
|
||||
name: kernel-rig-bin
|
||||
path: ${{ runner.temp }}/kernel-rig-bin.tar
|
||||
retention-days: 1
|
||||
- name: Compiler cache statistics
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
shard:
|
||||
name: Kernel rig (${{ matrix.shard }})
|
||||
needs: build
|
||||
runs-on: blacksmith-4vcpu-ubuntu-2404
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Every area of tests/kernel-cases/ and the ladder. On the build box (aarch64, 4 cases at
|
||||
# a time) the slowest areas took about 1 minute (booleans, 212 s of cases with one of
|
||||
# 84 s; threads, 161 s) and the ladder, one case at a time, 92 s: no area needs splitting.
|
||||
shard: [booleans, continuity, crashes, fillets, import, ladder, sheet-metal, shell, sweeps, tessellation, threads]
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
|
||||
with:
|
||||
name: kernel-rig-bin
|
||||
path: ${{ runner.temp }}
|
||||
- name: Unpack the binaries
|
||||
run: mkdir -p target && tar -xf "$RUNNER_TEMP/kernel-rig-bin.tar" -C target
|
||||
- name: Run the cases against the known-failing list (the ladder against its golden)
|
||||
env:
|
||||
SHARD: ${{ matrix.shard }}
|
||||
run: |
|
||||
if [ "$SHARD" = ladder ]; then
|
||||
set -- --ladder --gate
|
||||
else
|
||||
set -- --area "$SHARD" --known-failing "tests/kernel-cases/$SHARD/KNOWN_FAILING"
|
||||
fi
|
||||
target/debug/xtask kernel-cases "$@" --cli target/release/solidcraft-cli --out "$RUNNER_TEMP/kernel-rig"
|
||||
- name: Keep the results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
with:
|
||||
name: kernel-rig-results-${{ matrix.shard }}
|
||||
path: ${{ runner.temp }}/kernel-rig/results.json
|
||||
if-no-files-found: ignore
|
||||
|
||||
# One check for the whole gate, for branch protection and the merge queue.
|
||||
kernel-rig:
|
||||
name: kernel-rig
|
||||
needs: [build, shard]
|
||||
if: always() && (github.event_name != 'pull_request' || !github.event.pull_request.draft)
|
||||
runs-on: blacksmith-2vcpu-ubuntu-2404
|
||||
steps:
|
||||
- name: Every shard passed
|
||||
env:
|
||||
BUILD: ${{ needs.build.result }}
|
||||
SHARDS: ${{ needs.shard.result }}
|
||||
run: |
|
||||
test "$BUILD" = success
|
||||
test "$SHARDS" = success
|
||||
@@ -0,0 +1,70 @@
|
||||
# The release gate: everything, in full and in parallel, on one branch's head, before that branch
|
||||
# replaces main as a release (develop, rebased onto main). Run it by hand:
|
||||
# gh workflow run release-gate.yml --ref develop
|
||||
# It calls the workflows that own each check instead of copying their jobs:
|
||||
# - ci.yml as Run workflow with cross-compilation: every native suite in full (not scoped), the
|
||||
# reference tests, the doc tests, the GPU path tests, the action matrix (all 16 shards) and its
|
||||
# determinism check, the property tests, the repository checks (web-offline among them),
|
||||
# WebAssembly and RISC-V, Windows D3D12 FXC and Windows app startup;
|
||||
# - kernel-rig.yml: every rig area against its KNOWN_FAILING and the ladder against its golden;
|
||||
# - release.yml as a dry run (draft false): the Linux, Flatpak, Snap, FreeBSD, macOS, Windows and
|
||||
# web packages with their smoke tests and package checks.
|
||||
# The gate passes only when all three pass. Before main is replaced, tag the old main
|
||||
# (main-before-develop-<date>) and push the tag.
|
||||
name: Release gate
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: release-gate-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
ci:
|
||||
name: CI
|
||||
uses: ./.github/workflows/ci.yml
|
||||
permissions:
|
||||
contents: read
|
||||
# Select checks reads this commit's other runs.
|
||||
actions: read
|
||||
with:
|
||||
cross_compile: true
|
||||
|
||||
kernel-rig:
|
||||
name: Kernel rig
|
||||
uses: ./.github/workflows/kernel-rig.yml
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
packages:
|
||||
name: Packages
|
||||
uses: ./.github/workflows/release.yml
|
||||
# release.yml's last job asks for contents: write (it creates releases); a dry run only keeps
|
||||
# the files as an artifact.
|
||||
permissions:
|
||||
contents: write
|
||||
actions: read
|
||||
secrets: inherit
|
||||
with:
|
||||
draft: false
|
||||
|
||||
release-gate:
|
||||
name: release-gate
|
||||
needs: [ci, kernel-rig, packages]
|
||||
if: always()
|
||||
runs-on: blacksmith-2vcpu-ubuntu-2404
|
||||
steps:
|
||||
- name: Every part passed
|
||||
env:
|
||||
CI: ${{ needs.ci.result }}
|
||||
KERNEL_RIG: ${{ needs.kernel-rig.result }}
|
||||
PACKAGES: ${{ needs.packages.result }}
|
||||
run: |
|
||||
echo "ci=$CI kernel-rig=$KERNEL_RIG packages=$PACKAGES"
|
||||
test "$CI" = success
|
||||
test "$KERNEL_RIG" = success
|
||||
test "$PACKAGES" = success
|
||||
@@ -20,6 +20,7 @@
|
||||
# releases/latest/download/<name> links in the README keep working across releases.
|
||||
#
|
||||
# A dry run: dispatch with draft=false (the default). Only the release branch may create drafts.
|
||||
# The release gate (release-gate.yml) calls it as such a dry run.
|
||||
# A dry run from any branch also test-signs the Windows files when SIGNPATH_API_TOKEN exists:
|
||||
# gh workflow run release.yml --ref <branch> -f draft=false [-f signpath_signing_policy=test-signing]
|
||||
name: Release
|
||||
@@ -40,6 +41,18 @@ on:
|
||||
description: "SignPath signing policy for the Windows files (empty: the repository variable, else test-signing)"
|
||||
type: string
|
||||
default: ""
|
||||
# The release gate (release-gate.yml): a dry run (draft false) of every package and its checks.
|
||||
# The event stays the caller's (workflow_dispatch), so a call runs as a dispatched dry run does.
|
||||
workflow_call:
|
||||
inputs:
|
||||
draft:
|
||||
description: "Create/update a draft (release branch only; off: build artifacts only)"
|
||||
type: boolean
|
||||
default: false
|
||||
signpath_signing_policy:
|
||||
description: "SignPath signing policy for the Windows files (empty: the repository variable, else test-signing)"
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
Reference in New Issue
Block a user