Wave 352/509: Kernel rig: each area runs only when the files its PATHS lists change, every area on the shared core (and 3 more)

4 commits, squashed; the originals stay reachable from tag archive/merge-main-develop.

- e71177769 jyviko: Docs: the kernel rig gate, updating a known-failing list, re-recording the ladder golden
- d84d00bb4 jyviko: Release gate: the packages' dry run is unsigned and skips the release job
- 02e1f1841 jyviko: Kernel rig: each area runs only when the files its PATHS lists change, every area on the shared core
- b67e93211 jyviko: Kernel rig: an area's PATHS names only its own files, never the shared core

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtL5M41G8Nua7HvuFPZkmJ
This commit is contained in:
jyviko
2026-10-29 16:55:07 -04:00
parent 524b25b8ba
commit e909f3e013
19 changed files with 687 additions and 51 deletions
+86 -27
View File
@@ -1,8 +1,11 @@
# The kernel rig and the complexity ladder as a regression gate. A pull request that touches the
# kernel surface (the paths below) builds solidcraft-cli once (release, SMLib prebuilt), hands it
# to a matrix of shards as an artifact, and each shard runs one area of tests/kernel-cases/ (or
# the ladder) in parallel, so the wall time is the build plus the slowest shard. See
# tests/kernel-cases/README.md.
# The kernel rig and the complexity ladder as a regression gate. A plan job maps a pull request's
# changed files to the rig areas they can affect (scripts/ci/kernel_rig_plan.py): an area runs
# when a file its tests/kernel-cases/<area>/PATHS lists changes, or one of its cases; every area
# runs when the shared core in tests/kernel-cases/PATHS-ALL changes; the ladder runs whenever any
# area does. One job then builds solidcraft-cli once (release, SMLib prebuilt), checks that every
# registry command has a case or an exemption (tests/kernel-cases/EXEMPT), and hands the CLI to a
# matrix of shards as an artifact, one per selected area plus the ladder, in parallel, so the
# wall time is the build plus the slowest shard. See tests/kernel-cases/README.md.
#
# - An area fails on a regression (a case that fails and its tests/kernel-cases/<area>/KNOWN_FAILING
# does not list) or on a listed case that passes (it is fixed: remove its line in the same pull
@@ -10,11 +13,13 @@
# - The ladder fails on any case whose status or counts differ from
# tests/kernel-cases/ladder/golden.json; a right change is re-recorded (`--record "why"`).
#
# GitHub filters a workflow by whole files, so any change to crates/doc/src/eval*.rs or
# crates/engine/src/cmd/features*.rs selects it. Pushes to main and develop that touch the same
# paths run it too and commit the compiler cache (a Blacksmith sticky disk) that every other run
# mounts read-only. A merge group (once the queue is on) always runs it: it is the last check
# before the change lands. The release gate (release-gate.yml) calls it (workflow_call).
# The trigger paths are whole folders that hold every area's PATHS (a self-test,
# scripts/ci/test_kernel_rig_plan.py, checks they do), so a pull request that touches none of
# them (the UI, docs, packaging) starts no runner at all, and a new area needs no edit here: the
# plan finds the areas from the folders. Pushes to main and develop on the same paths, merge groups
# (once the queue is on), the release gate (release-gate.yml, workflow_call) and a run by hand run
# every area; those pushes also commit the compiler cache (a Blacksmith sticky disk) that every
# other run mounts read-only.
name: Kernel rig
on:
@@ -25,14 +30,17 @@ on:
- "crates/geom/**"
- "crates/smlib/**"
- "crates/step/**"
- "crates/doc/src/eval*.rs"
- "crates/engine/src/cmd/features*.rs"
- "crates/io/src/**"
- "crates/doc/src/**"
- "crates/engine/src/**"
- "tests/kernel-cases/**"
# The rig itself: the grading, the runner and this workflow.
# The rig itself: the grading, the runner, the selection and this workflow.
- "apps/solidcraft-cli/src/kernel_cases.rs"
- "xtask/src/kernel_cases.rs"
- "xtask/src/kernel_ladder.rs"
- "scripts/fetch-smlib.sh"
- "scripts/ci/kernel_rig_plan.py"
- "scripts/ci/test_kernel_rig_plan.py"
- ".github/workflows/kernel-rig.yml"
push:
branches: [main, develop]
@@ -42,14 +50,17 @@ on:
- "crates/geom/**"
- "crates/smlib/**"
- "crates/step/**"
- "crates/doc/src/eval*.rs"
- "crates/engine/src/cmd/features*.rs"
- "crates/io/src/**"
- "crates/doc/src/**"
- "crates/engine/src/**"
- "tests/kernel-cases/**"
# The rig itself: the grading, the runner and this workflow.
# The rig itself: the grading, the runner, the selection and this workflow.
- "apps/solidcraft-cli/src/kernel_cases.rs"
- "xtask/src/kernel_cases.rs"
- "xtask/src/kernel_ladder.rs"
- "scripts/fetch-smlib.sh"
- "scripts/ci/kernel_rig_plan.py"
- "scripts/ci/test_kernel_rig_plan.py"
- ".github/workflows/kernel-rig.yml"
merge_group:
workflow_call:
@@ -67,10 +78,44 @@ env:
CARGO_TERM_COLOR: always
jobs:
build:
name: Build the CLI
plan:
name: Select the rig areas
# Draft pull requests run no CI, as in ci.yml.
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
runs-on: blacksmith-2vcpu-ubuntu-2404
timeout-minutes: 5
outputs:
# The shard matrix (JSON list of areas and the ladder; [] when nothing is selected).
shards: ${{ steps.plan.outputs.shards }}
# Whether the build job, and with it the coverage check, runs.
build: ${{ steps.plan.outputs.build }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
# A pull request's diff needs its base's history.
fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }}
- name: Selection self-test
run: python3 -I scripts/ci/test_kernel_rig_plan.py
- id: plan
name: Map the changed files to areas
env:
EVENT_NAME: ${{ github.event_name }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
# Pull requests run the areas their diff reaches; pushes, merge groups, the release
# gate and a run by hand run every area.
if [ "$EVENT_NAME" = pull_request ]; then
git diff --no-renames --name-only -z "$BASE_SHA...$HEAD_SHA" > "$RUNNER_TEMP/rig-paths"
python3 -I scripts/ci/kernel_rig_plan.py --paths "$RUNNER_TEMP/rig-paths" --github-output "$GITHUB_OUTPUT"
else
python3 -I scripts/ci/kernel_rig_plan.py --all --github-output "$GITHUB_OUTPUT"
fi
build:
name: Build the CLI
needs: plan
if: needs.plan.outputs.build == 'true'
runs-on: blacksmith-8vcpu-ubuntu-2404
timeout-minutes: 60
env:
@@ -110,9 +155,15 @@ jobs:
# A tar keeps the executable bits, which an artifact drops. xtask finds the repository from
# the path it was built in (CARGO_MANIFEST_DIR), which is the same checkout path in every
# job of this workflow.
# Every registry command is used by a case or listed in tests/kernel-cases/EXEMPT.
- name: Every command has a case
run: target/debug/xtask kernel-cases --coverage --cli target/release/solidcraft-cli
# A coverage-only run (the registry or the cases changed, no area selected) runs no shard.
- name: Pack the binaries
if: needs.plan.outputs.shards != '[]'
run: tar -cf "$RUNNER_TEMP/kernel-rig-bin.tar" -C target release/solidcraft-cli debug/xtask
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: needs.plan.outputs.shards != '[]'
with:
name: kernel-rig-bin
path: ${{ runner.temp }}/kernel-rig-bin.tar
@@ -123,16 +174,18 @@ jobs:
shard:
name: Kernel rig (${{ matrix.shard }})
needs: build
needs: [plan, build]
if: needs.plan.outputs.shards != '[]'
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
# Every area of tests/kernel-cases/ and the ladder. On the build box (aarch64, 4 cases at
# a time) the slowest areas took about 1 minute (booleans, 212 s of cases with one of
# 84 s; threads, 161 s) and the ladder, one case at a time, 92 s: no area needs splitting.
shard: [booleans, continuity, crashes, fillets, import, ladder, sheet-metal, shell, sweeps, tessellation, threads]
# The areas of tests/kernel-cases/ the plan selected, and the ladder. On the build box
# (aarch64, 4 cases at a time) the slowest areas took about 1 minute (booleans, 212 s of
# cases with one of 84 s; threads, 161 s) and the ladder, one case at a time, 92 s: no area
# needs splitting.
shard: ${{ fromJSON(needs.plan.outputs.shards) }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
@@ -162,14 +215,20 @@ jobs:
# One check for the whole gate, for branch protection and the merge queue.
kernel-rig:
name: kernel-rig
needs: [build, shard]
needs: [plan, build, shard]
if: always() && (github.event_name != 'pull_request' || !github.event.pull_request.draft)
runs-on: blacksmith-2vcpu-ubuntu-2404
steps:
- name: Every shard passed
# A skipped build or shard counts as passed only when the plan selected none.
- name: Every selected job passed
env:
PLAN: ${{ needs.plan.result }}
WANT_BUILD: ${{ needs.plan.outputs.build }}
WANT_SHARDS: ${{ needs.plan.outputs.shards }}
BUILD: ${{ needs.build.result }}
SHARDS: ${{ needs.shard.result }}
run: |
test "$BUILD" = success
test "$SHARDS" = success
echo "plan=$PLAN build=$BUILD shards=$SHARDS selected=$WANT_SHARDS"
test "$PLAN" = success
if [ "$WANT_BUILD" = true ]; then test "$BUILD" = success; else test "$BUILD" = skipped; fi
if [ "$WANT_SHARDS" != '[]' ]; then test "$SHARDS" = success; else test "$SHARDS" = skipped; fi
+9 -5
View File
@@ -7,8 +7,10 @@
# determinism check, the property tests, the repository checks (web-offline among them),
# WebAssembly and RISC-V, Windows D3D12 FXC and Windows app startup;
# - kernel-rig.yml: every rig area against its KNOWN_FAILING and the ladder against its golden;
# - release.yml as a dry run (draft false): the Linux, Flatpak, Snap, FreeBSD, macOS, Windows and
# web packages with their smoke tests and package checks.
# - release.yml as a dry run (draft false, packages_only): the Linux, Flatpak, Snap, FreeBSD,
# macOS, Windows and web packages with their smoke tests and package checks, unsigned (no
# secrets are passed, so no signing secret or SignPath quota is used) and without the release
# job (checksums, update signing, publishing).
# The gate passes only when all three pass. Before main is replaced, tag the old main
# (main-before-develop-<date>) and push the tag.
name: Release gate
@@ -43,14 +45,16 @@ jobs:
packages:
name: Packages
uses: ./.github/workflows/release.yml
# release.yml's last job asks for contents: write (it creates releases); a dry run only keeps
# the files as an artifact.
# No secrets: the signing steps skip without them. GitHub checks a called workflow's job
# permissions before the run, skipped jobs included, so the release job's contents: write must
# be granted here; packages_only skips that job, and no step that runs writes to the
# repository. actions: read is the Windows job's.
permissions:
contents: write
actions: read
secrets: inherit
with:
draft: false
packages_only: true
release-gate:
name: release-gate
+11 -2
View File
@@ -20,7 +20,8 @@
# releases/latest/download/<name> links in the README keep working across releases.
#
# A dry run: dispatch with draft=false (the default). Only the release branch may create drafts.
# The release gate (release-gate.yml) calls it as such a dry run.
# The release gate (release-gate.yml) calls it as such a dry run, without secrets (unsigned) and
# without the release job (packages_only).
# A dry run from any branch also test-signs the Windows files when SIGNPATH_API_TOKEN exists:
# gh workflow run release.yml --ref <branch> -f draft=false [-f signpath_signing_policy=test-signing]
name: Release
@@ -43,8 +44,15 @@ on:
default: ""
# The release gate (release-gate.yml): a dry run (draft false) of every package and its checks.
# The event stays the caller's (workflow_dispatch), so a call runs as a dispatched dry run does.
# The gate passes no secrets, so nothing is signed (the signing steps skip without their
# secrets, as on a fork), and packages_only skips the release job (checksums, update signing,
# publishing).
workflow_call:
inputs:
packages_only:
description: "Build and check every package only: no release job"
type: boolean
default: false
draft:
description: "Create/update a draft (release branch only; off: build artifacts only)"
type: boolean
@@ -600,7 +608,8 @@ jobs:
release:
name: GitHub Release or dry-run artifacts
needs: [version, linux, flatpak, snap, freebsd, macos, windows, web]
if: needs.version.outputs.build == 'true'
# Not for the release gate (packages_only): it builds and checks the packages, nothing more.
if: needs.version.outputs.build == 'true' && !inputs.packages_only
runs-on: ubuntu-latest
permissions:
contents: write
+2 -1
View File
@@ -241,7 +241,8 @@ the surface itself). Expected values are either worked out in closed form
(`tools/oracle-analytic/`) or computed by building the same recipe in FreeCAD, run as a separate
program (`tools/freecad-oracle/`); each file says which. The generated
[reference test report](docs/oracle.md) has the current case count and results. The reference-test
CI job runs `cargo xtask oracle` during full validation; pull-request runs skip it. A case that
CI job runs `cargo xtask oracle` during full validation, and on pull requests that reach
`solidcraft-cli`. A case that
fails because of a known bug names its issue in `expected.json` (`"known_failure": "#123"`);
never change an expected value to match SolidCraft's output.
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""Select the kernel rig areas a pull request affects (.github/workflows/kernel-rig.yml).
Usage: kernel_rig_plan.py (--all | --paths FILE) [--root DIR] [--github-output FILE]
FILE holds the changed paths relative to the repository root, separated by NUL (git diff -z);
"-" reads standard input. --all selects every area (pushes, merge groups, the release gate, a run
by hand). --root is the repository (default: two folders up from this script).
The areas are the folders of tests/kernel-cases/ (the ladder aside), so a new area joins by
having a folder with its cases, its KNOWN_FAILING and its PATHS. A changed path selects:
- every area, when it matches tests/kernel-cases/PATHS-ALL (the shared core: SMLib, geom, the
kernel's core, the rig's grader and runner, the workflow);
- an area, when it is under tests/kernel-cases/<area>/ or matches that area's PATHS: one git
pathspec glob a line (`*` within a folder, `**` across folders), `#` starts a comment.
The ladder spans every operation: it runs whenever any area does, and on a change under
tests/kernel-cases/ladder/.
The coverage check (`cargo xtask kernel-cases --coverage`) runs in the build job. The build runs
when a shard does, or when the command registry or the cases may have changed (COVERAGE below),
so a new command without a case fails on its own pull request.
Prints and optionally appends to $GITHUB_OUTPUT:
shards=["booleans", ..., "ladder"] the shard matrix, JSON (empty list: no shard runs)
build=true|false the build job (and the coverage check) runs
"""
import argparse
import json
import os
import re
import sys
CASES = "tests/kernel-cases"
LADDER = "ladder"
# Changes here may add or remove a registry command, or the case that covers one: the coverage
# check must see them even when no area runs.
COVERAGE = (
"crates/engine/src/cmd/**",
"tests/kernel-cases/EXEMPT",
"tests/kernel-cases/**/recipe.json",
"xtask/src/kernel_cases.rs",
)
def glob_regex(glob):
"""A git pathspec glob (`:(glob)` magic) as a regular expression over a whole path: `**/`
matches any folders (none too), a trailing `/**` everything below, `*` and `?` stay within
one folder."""
out, i = [], 0
while i < len(glob):
if glob.startswith("**/", i):
out.append("(?:.*/)?")
i += 3
elif glob.startswith("**", i):
out.append(".*")
i += 2
elif glob[i] == "*":
out.append("[^/]*")
i += 1
elif glob[i] == "?":
out.append("[^/]")
i += 1
else:
out.append(re.escape(glob[i]))
i += 1
return re.compile("".join(out) + r"\Z")
def read_globs(path):
"""The globs of a PATHS file: one a line, blank lines and `#` comments skipped."""
with open(path, encoding="utf-8") as f:
lines = [line.strip() for line in f]
globs = [line for line in lines if line and not line.startswith("#")]
for g in globs:
if g.startswith("/") or g.startswith(":") or "\\" in g:
raise SystemExit(f"{path}: `{g}`: a glob is relative to the repository root, with / and no pathspec magic")
return globs
def areas(root):
"""Every area (a folder of tests/kernel-cases/ but the ladder) with its PATHS globs."""
base = os.path.join(root, CASES)
out = {}
for name in sorted(os.listdir(base)):
if name == LADDER or not os.path.isdir(os.path.join(base, name)):
continue
paths = os.path.join(base, name, "PATHS")
if not os.path.isfile(paths):
raise SystemExit(f"{CASES}/{name}/PATHS is missing: every area lists the source files whose change reruns it")
out[name] = read_globs(paths)
return out
def matches(globs, path):
return any(glob_regex(g).match(path) for g in globs)
def select(root, paths, everything=False):
"""Return (shards, build): the areas to run, in order and the ladder last, and whether the
build job runs."""
by_area = areas(root)
if everything:
return [*by_area, LADDER], True
shared = read_globs(os.path.join(root, CASES, "PATHS-ALL"))
hit, ladder = set(), False
for path in paths:
if matches(shared, path):
return [*by_area, LADDER], True
if path.startswith(f"{CASES}/{LADDER}/"):
ladder = True
for area, globs in by_area.items():
if path.startswith(f"{CASES}/{area}/") or matches(globs, path):
hit.add(area)
shards = [a for a in by_area if a in hit]
if shards or ladder:
shards.append(LADDER)
return shards, bool(shards) or any(matches(COVERAGE, p) for p in paths)
def read_paths(src):
data = sys.stdin.buffer.read() if src == "-" else open(src, "rb").read()
return [p.decode() for p in data.split(b"\0") if p]
def main():
ap = argparse.ArgumentParser()
group = ap.add_mutually_exclusive_group(required=True)
group.add_argument("--paths")
group.add_argument("--all", action="store_true")
ap.add_argument("--root", default=os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
ap.add_argument("--github-output")
a = ap.parse_args()
shards, build = select(a.root, [] if a.all else read_paths(a.paths), everything=a.all)
lines = [f"shards={json.dumps(shards)}", f"build={'true' if build else 'false'}"]
print("\n".join(lines))
if a.github_output:
with open(a.github_output, "a", encoding="utf-8") as f:
f.write("\n".join(lines) + "\n")
if __name__ == "__main__":
main()
+156
View File
@@ -0,0 +1,156 @@
#!/usr/bin/env python3
"""Self-test for kernel_rig_plan.py. Run: python3 -I scripts/ci/test_kernel_rig_plan.py (from the
repo root).
Uses a small synthetic tests/kernel-cases/, then the real one: every area has its PATHS and
KNOWN_FAILING, every glob names files that exist, and the workflow's trigger paths cover them all.
"""
import os
import re
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import kernel_rig_plan as plan # noqa: E402
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
WORKFLOW = os.path.join(ROOT, ".github", "workflows", "kernel-rig.yml")
def write(root, rel, text=""):
path = os.path.join(root, rel)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text)
class Glob(unittest.TestCase):
def test_star_stays_in_a_folder_and_double_star_crosses(self):
self.assertTrue(plan.glob_regex("crates/kernel/src/blend*.rs").match("crates/kernel/src/blend_smlib.rs"))
self.assertFalse(plan.glob_regex("crates/kernel/src/*.rs").match("crates/kernel/src/tess/grid.rs"))
self.assertTrue(plan.glob_regex("crates/kernel/src/tess/**").match("crates/kernel/src/tess/a/b.rs"))
self.assertFalse(plan.glob_regex("crates/kernel/src/tess/**").match("crates/kernel/src/tess.rs"))
self.assertTrue(plan.glob_regex("tests/**/recipe.json").match("tests/recipe.json"))
self.assertTrue(plan.glob_regex("tests/**/recipe.json").match("tests/a/b/recipe.json"))
self.assertFalse(plan.glob_regex("a.rs").match("a.rsx"))
self.assertFalse(plan.glob_regex("a.rs").match("b/a.rs"))
class Synthetic(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
r = self.root = self.tmp.name
write(r, "tests/kernel-cases/PATHS-ALL", "# shared\ncrates/geom/**\n\n.github/workflows/kernel-rig.yml\n")
write(r, "tests/kernel-cases/fillets/PATHS", "# blends\ncrates/kernel/src/blend*.rs\n")
write(r, "tests/kernel-cases/shell/PATHS", "crates/kernel/src/offset*.rs\ncrates/kernel/src/blend_smlib.rs\n")
write(r, "tests/kernel-cases/ladder/ladder.json", "{}")
write(r, "tests/kernel-cases/README.md")
def tearDown(self):
self.tmp.cleanup()
def select(self, *paths):
return plan.select(self.root, list(paths))
def test_an_area_runs_on_its_paths_with_the_ladder(self):
self.assertEqual(self.select("crates/kernel/src/blend.rs"), (["fillets", "ladder"], True))
self.assertEqual(self.select("crates/kernel/src/blend_smlib.rs"), (["fillets", "shell", "ladder"], True))
def test_a_change_to_an_areas_cases_runs_it(self):
self.assertEqual(self.select("tests/kernel-cases/shell/1-x/expected.json"), (["shell", "ladder"], True))
self.assertEqual(self.select("tests/kernel-cases/shell/PATHS"), (["shell", "ladder"], True))
self.assertEqual(self.select("tests/kernel-cases/ladder/golden.json"), (["ladder"], True))
def test_the_shared_core_runs_every_area(self):
for p in ["crates/geom/src/brep.rs", ".github/workflows/kernel-rig.yml"]:
self.assertEqual(self.select("docs/a.md", p), (["fillets", "shell", "ladder"], True), p)
def test_unrelated_changes_run_nothing(self):
self.assertEqual(self.select("crates/ui-egui/src/app.rs", "docs/a.md", "tests/kernel-cases/README.md"), ([], False))
def test_a_registry_or_case_change_builds_for_the_coverage_check(self):
self.assertEqual(self.select("crates/engine/src/cmd/view_navigation.rs"), ([], True))
self.assertEqual(self.select("tests/kernel-cases/EXEMPT"), ([], True))
def test_everything_runs_every_area(self):
self.assertEqual(plan.select(self.root, [], everything=True), (["fillets", "shell", "ladder"], True))
def test_a_new_area_joins_from_its_folder(self):
write(self.root, "tests/kernel-cases/surfaces/PATHS", "crates/kernel/src/surface_*.rs\n")
self.assertEqual(self.select("crates/kernel/src/surface_trim.rs"), (["surfaces", "ladder"], True))
self.assertEqual(plan.select(self.root, [], everything=True)[0], ["fillets", "shell", "surfaces", "ladder"])
def test_an_area_without_paths_is_an_error(self):
os.makedirs(os.path.join(self.root, "tests/kernel-cases/plastic"))
with self.assertRaises(SystemExit) as e:
self.select("crates/geom/src/brep.rs")
self.assertIn("plastic/PATHS is missing", str(e.exception))
def trigger_globs():
"""The pull_request trigger paths of the workflow (the push trigger lists the same)."""
with open(WORKFLOW, encoding="utf-8") as f:
text = f.read()
events = {}
for event in ("pull_request", "push"):
block = re.search(rf"^ {event}:\n((?: .*\n|\s*\n)+)", text, re.M)
assert block, event
paths = block.group(1).split("paths:", 1)[1]
events[event] = re.findall(r'^\s+- "([^"]+)"', paths, re.M)
return events
def sample(glob):
"""A path the glob matches: each wildcard filled in."""
return glob.replace("**", "x/y").replace("*", "x").replace("?", "x")
class Real(unittest.TestCase):
def test_every_area_has_its_lists_and_every_glob_names_files(self):
# Tracked files and new ones not yet committed (not the ignored ones).
ls = ["git", "ls-files", "--cached", "--others", "--exclude-standard"]
tracked = subprocess.run(ls, cwd=ROOT, check=True, capture_output=True, text=True).stdout.split("\n")
shared = plan.read_globs(os.path.join(ROOT, plan.CASES, "PATHS-ALL"))
areas = plan.areas(ROOT)
self.assertTrue(areas)
for area, globs in {"PATHS-ALL": shared, **areas}.items():
if area != "PATHS-ALL":
self.assertTrue(os.path.isfile(os.path.join(ROOT, plan.CASES, area, "KNOWN_FAILING")), area)
for g in globs:
with self.subTest(area=area, glob=g):
rx = plan.glob_regex(g)
self.assertTrue(any(rx.match(t) for t in tracked), f"{area}: {g} matches no file")
def test_an_area_names_only_its_own_files(self):
# The shared core is in PATHS-ALL; an area's PATHS repeating it (or a whole crate's
# folder) would only blur what the area is about.
ls = ["git", "ls-files", "--cached", "--others", "--exclude-standard"]
files = subprocess.run(ls, cwd=ROOT, check=True, capture_output=True, text=True).stdout.split("\n")
shared = plan.read_globs(os.path.join(ROOT, plan.CASES, "PATHS-ALL"))
core = {f for f in files if plan.matches(shared, f)}
for area, globs in plan.areas(ROOT).items():
for g in globs:
rx = plan.glob_regex(g)
overlap = sorted(f for f in core if rx.match(f))
with self.subTest(area=area, glob=g):
self.assertFalse(overlap, f"{area}/PATHS: {g} names shared-core files (PATHS-ALL): {overlap[:3]}")
def test_the_workflow_triggers_on_every_glob(self):
events = trigger_globs()
self.assertEqual(events["pull_request"], events["push"])
triggers = [plan.glob_regex(g) for g in events["pull_request"]]
globs = [*plan.read_globs(os.path.join(ROOT, plan.CASES, "PATHS-ALL")), *plan.COVERAGE]
for area, own in plan.areas(ROOT).items():
globs += [*own, f"{plan.CASES}/{area}/**"]
for g in globs:
with self.subTest(glob=g):
self.assertTrue(any(t.match(sample(g)) for t in triggers), f"{g}: not under the workflow's trigger paths")
def test_a_ui_change_runs_nothing(self):
self.assertEqual(plan.select(ROOT, ["crates/ui-egui/src/app.rs", "docs/kernel-cases.md", "README.md"]), ([], False))
if __name__ == "__main__":
unittest.main()
+2
View File
@@ -25,6 +25,8 @@ A failing test without its fix cannot merge: it turns `main` red for everyone. I
## Goldens, matrices and skipped tests
- The reference for what the kernel builds is the kernel-cases ladder (`tests/kernel-cases/ladder/`, `cargo xtask kernel-cases --ladder`, `docs/kernel-ladder.md`).
- Kernel PRs run the kernel rig gate (`kernel-rig.yml`): each area against its `KNOWN_FAILING`, the ladder against its golden. In the same PR, a fix removes its line, a new failing case adds one, a changed ladder result is re-recorded (`--ladder --record "why"`); see `tests/kernel-cases/README.md`.
- Maintainers' PRs run the native, GPU path and reference tests on the affected packages; others wait for `approved-for-ci`. `release-gate.yml` runs everything in full before develop replaces main.
- The action matrix is a ratchet: record a newly passing action with `cargo xtask action-matrix --bless` (`docs/action-matrix.md`).
- Pull requests skip a few heavy tests, which run in the merge queue, nightly and on `main`. Check that validation after merging. The list is in `.config/nextest.toml` and the testing page.
- Mutation testing finds behaviour the suites miss: `docs/mutation-testing.md`.
+58
View File
@@ -0,0 +1,58 @@
# The shared core: a change to any of these reruns every area of the kernel rig and the ladder
# (scripts/ci/kernel_rig_plan.py). One git pathspec glob a line, relative to the repository root:
# `*` stays within a folder, `**` crosses folders. Each area's own sources are in <area>/PATHS.
#
# SMLib: the bindings, the schema 4 record bridge and the pinned revision (SMLIB_REV,
# SMLIB_PREBUILT), and the script that fetches it.
crates/smlib/**
scripts/fetch-smlib.sh
# The geometry every body is made of: B-rep record, curves, surfaces, meshes.
crates/geom/**
# The kernel's core, which every case goes through: bodies, extrude/revolve/primitives, booleans
# and placements, the planar prisms, the SMLib sweeps and booleans, history and provenance, the
# native bridge to SMLib, tessellation, and what the grader measures (measure, validity, counts).
crates/kernel/Cargo.toml
crates/kernel/src/body.rs
crates/kernel/src/boolean_record.rs
crates/kernel/src/build.rs
crates/kernel/src/freeform.rs
crates/kernel/src/heal.rs
crates/kernel/src/history.rs
crates/kernel/src/lib.rs
crates/kernel/src/measure.rs
crates/kernel/src/native.rs
crates/kernel/src/native/**
crates/kernel/src/numbered.rs
crates/kernel/src/ops.rs
crates/kernel/src/par.rs
crates/kernel/src/planar_regularize.rs
crates/kernel/src/planar_sources.rs
crates/kernel/src/plane_fit.rs
crates/kernel/src/prism.rs
crates/kernel/src/prism_curve.rs
crates/kernel/src/prism_measure.rs
crates/kernel/src/profile.rs
crates/kernel/src/provenance.rs
crates/kernel/src/record.rs
crates/kernel/src/shell_groups.rs
crates/kernel/src/smlib_boolean.rs
crates/kernel/src/smlib_sweeps.rs
crates/kernel/src/tess/**
crates/kernel/src/topo.rs
crates/kernel/src/validity.rs
crates/kernel/src/watch.rs
# Timeline evaluation and the model cache, which every feature is evaluated through.
crates/doc/src/eval.rs
crates/doc/src/model_cache.rs
crates/doc/src/walk.rs
# The command registry and the solid feature commands every case starts with (sketches made
# into boxes, cylinders, extrusions, revolutions, combines, moves; fillets, holes, shells, sweeps
# and lofts live there too).
crates/engine/src/cmd/features.rs
crates/engine/src/cmd/mod.rs
# The rig itself: the grader, the runner, the selection and this gate's workflow.
.github/workflows/kernel-rig.yml
apps/solidcraft-cli/src/kernel_cases.rs
scripts/ci/kernel_rig_plan.py
xtask/src/kernel_cases.rs
xtask/src/kernel_ladder.rs
+40 -16
View File
@@ -97,27 +97,43 @@ the frozen truck-geometry PRs, a `kernel-paused` PR), for the report's coverage
| `tolerance` | Optional overrides: `rel`, `abs_mm`, `mesh_mm`, `weld_mm`. |
| `missing` | A capability SolidCraft has no command or parameter for yet (`"solid.thread has no start count"`): the case is reported **not supported yet** without running, so the gap stays visible. |
## The fillet gate
## The kernel rig gate
The fillet cases are the fillet regression suite: each is one small case of one issue, with its
exact answer. `.github/workflows/fillet-rig.yml` runs them on every pull request (not a draft)
that touches fillet code: `crates/kernel/src/{blend*,fillet*,*corner*,continuity*}.rs`,
`crates/smlib/**` (`SMLIB_REV` and `SMLIB_PREBUILT` included), `crates/doc/src/eval*.rs`,
`crates/engine/src/cmd/features.rs`, `tests/kernel-cases/fillets/**` and the rig's own code
(GitHub filters whole files, so any change to `eval*.rs` or `features.rs` runs it). One release
build of `solidcraft-cli`, then:
Every area is a regression suite: each case is one small case of one issue, with its exact
answer. `.github/workflows/kernel-rig.yml` runs them all, and the ladder, on every pull request
(not a draft) that touches the kernel surface: `crates/kernel/**`, `crates/geom/**`,
`crates/smlib/**` (`SMLIB_REV` and `SMLIB_PREBUILT` included), `crates/step/**`,
`crates/doc/src/eval*.rs`, `crates/engine/src/cmd/features*.rs`, `tests/kernel-cases/**` and the
rig's own code (GitHub filters whole files, so any change to `eval*.rs` or `features*.rs` runs
it). Pushes to main and develop on the same paths, merge groups and the release gate
(`release-gate.yml`) run it too. One job builds `solidcraft-cli` (release, SMLib) and the rig's
runner, and hands both to a matrix of shards, one per area plus the ladder, that run in
parallel; the `kernel-rig` check passes when every shard does. (`fillet-rig.yml`, the earlier
fillet-only gate, folds into it.) Each area's shard runs:
```sh
cargo xtask kernel-cases --area fillets --cli target/release/solidcraft-cli \
--known-failing tests/kernel-cases/fillets/KNOWN_FAILING
cargo xtask kernel-cases --area <area> --cli target/release/solidcraft-cli \
--known-failing tests/kernel-cases/<area>/KNOWN_FAILING
```
`fillets/KNOWN_FAILING` lists the cases that fail today, sorted, one a line with its cause
(`<case> <why it fails today>`; `#` starts a comment). The gate fails on a **regression**, a case
that fails (or crashes, hangs, or is not supported yet) and is not listed, and on a **fixed** case,
one listed that passes: a fix removes its line in the same pull request, so the list only shrinks.
A new case that fails today goes in with its line. With `--known-failing` the run writes no
report; `docs/kernel-cases.md` stays the full run's.
`<area>/KNOWN_FAILING` lists the cases of that area that fail today, sorted, one a line with its
cause (`<case> <why it fails today>`; `#` starts a comment; the header names the baseline: the
develop commit and SMLib revision it was taken from). The gate fails on a **regression**, a
case that fails (or crashes, hangs, or is not supported yet) and is not listed, and on a
**fixed** case, one listed that passes. With `--known-failing` the run writes no report;
`docs/kernel-cases.md` stays the full run's.
**Updating a list.** The pull request that changes an outcome changes the list in the same pull
request, so the list only shrinks and never goes stale:
- a fix (yours, or an SMLib pin bump): remove the line of each case the shard reports as
`FIXED`;
- a new case that fails today: add its line, sorted, with the cause from its result
(`results.json` in the shard's `kernel-rig-results-<area>` artifact, or a local run);
- a regression: fix it in the same pull request.
To check a list locally before pushing: `cargo xtask kernel-cases --area <area> --cli
target/release/solidcraft-cli --known-failing tests/kernel-cases/<area>/KNOWN_FAILING`.
## The complexity ladder
@@ -137,9 +153,17 @@ running part grows one tier at a time through `base`), so the last case carries
```sh
cargo xtask kernel-cases --ladder --cli target/release/solidcraft-cli
cargo xtask kernel-cases --ladder --cli target/release/solidcraft-cli --gate
cargo xtask kernel-cases --ladder --cli target/release/solidcraft-cli --record "why"
```
The ladder needs no known-failing list: the golden is the reference. Its shard in the kernel rig
gate runs with `--gate`, which fails on any case whose status or counts differ from the golden,
better or worse, and on a case with no golden; times are not compared there (they are the
machine's). A pull request that changes a ladder result re-records the golden with
`--record "why the golden changes"` and commits `golden.json` and
`ladder.json` with it.
Ladder runs measure one case at a time (unless `--jobs`) and keep their output in
`target/kernel-ladder` (`--out`); `--record` rewrites the golden and sorts each tier by time,
fastest first; commit both files. The report, `docs/kernel-ladder.md`, gives per tier the counts
+33
View File
@@ -0,0 +1,33 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Combine, cuts, holes, patterns, split, mirror, draft, emboss, remove, press/pull, delete face,
# coils, text, surface extend/thicken, sketches projected from the model, and bodies moved between
# components.
crates/doc/src/body_transfer.rs
crates/doc/src/body_transfer_sheet.rs
crates/doc/src/eval_more.rs
crates/doc/src/eval_surface.rs
crates/doc/src/hole_reference.rs
crates/doc/src/project.rs
crates/doc/src/thread_catalog.rs
crates/engine/src/cmd/component.rs
crates/engine/src/cmd/delete.rs
crates/engine/src/cmd/features_more.rs
crates/engine/src/cmd/sketch_project.rs
crates/engine/src/cmd/surface.rs
crates/kernel/src/assemble.rs
crates/kernel/src/delete_face.rs
crates/kernel/src/draft_corners.rs
crates/kernel/src/emboss.rs
crates/kernel/src/fill.rs
crates/kernel/src/helix.rs
crates/kernel/src/intersection.rs
crates/kernel/src/offset.rs
crates/kernel/src/offset_backend.rs
crates/kernel/src/polybool.rs
crates/kernel/src/polyhedron.rs
crates/kernel/src/sew.rs
crates/kernel/src/splitface.rs
crates/kernel/src/surface_extend.rs
crates/kernel/src/surfaces.rs
+17
View File
@@ -0,0 +1,17 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Seams of designed order: fillets and chamfers, patches, stitches and trims, and the continuity
# measurement itself (Continuity Analysis).
crates/doc/src/eval_surface.rs
crates/engine/src/cmd/delete.rs
crates/engine/src/cmd/surface.rs
crates/engine/src/cmd/surface_analysis.rs
crates/kernel/src/blend*.rs
crates/kernel/src/continuity.rs
crates/kernel/src/continuity_order.rs
crates/kernel/src/corner_g2.rs
crates/kernel/src/delete_face.rs
crates/kernel/src/fill.rs
crates/kernel/src/surface_trim.rs
crates/kernel/src/surfaces.rs
+21
View File
@@ -0,0 +1,21 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Crashes and hangs inside kernel operations: fillets, lofts, patterns, delete face, snap fits,
# export, measurement and document inspection.
crates/doc/src/plastic.rs
crates/engine/src/cmd/delete.rs
crates/engine/src/cmd/file.rs
crates/engine/src/cmd/inspect.rs
crates/engine/src/cmd/measure_sel.rs
crates/engine/src/cmd/plastic.rs
crates/kernel/src/blend*.rs
crates/kernel/src/corner_g2.rs
crates/kernel/src/delete_face.rs
crates/kernel/src/fill.rs
crates/kernel/src/lip.rs
crates/kernel/src/loft_blend.rs
crates/kernel/src/loft_rails.rs
crates/kernel/src/step.rs
crates/kernel/src/step_out.rs
crates/step/**
+15
View File
@@ -0,0 +1,15 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Fillets and chamfers (constant, per-edge, variable, G2 corners), with the drafts, shells, holes
# and patterns they round, and the continuity the blends are graded on.
crates/doc/src/hole_reference.rs
crates/kernel/src/blend*.rs
crates/kernel/src/continuity.rs
crates/kernel/src/continuity_order.rs
crates/kernel/src/corner_g2.rs
crates/kernel/src/draft_corners.rs
crates/kernel/src/fill.rs
crates/kernel/src/offset.rs
crates/kernel/src/offset_backend.rs
crates/kernel/src/polyhedron.rs
+22
View File
@@ -0,0 +1,22 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# STEP and IGES import and export, mesh import (3MF, STL, OBJ, glTF) and mesh-to-solid conversion.
crates/engine/src/cmd/file.rs
crates/engine/src/cmd/file_import.rs
crates/engine/src/cmd/mesh.rs
crates/io/src/formats.rs
crates/io/src/gltf_in.rs
crates/io/src/obj.rs
crates/io/src/threemf.rs
crates/kernel/src/iges_in.rs
crates/kernel/src/iges_in/**
crates/kernel/src/iges_out.rs
crates/kernel/src/mesh_convert.rs
crates/kernel/src/meshbody.rs
crates/kernel/src/sew.rs
crates/kernel/src/splitface.rs
crates/kernel/src/step.rs
crates/kernel/src/step_in/**
crates/kernel/src/step_out.rs
crates/step/**
+15
View File
@@ -0,0 +1,15 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Sheet metal: rules, flanges, folds, unfold/refold, flat patterns, convert to sheet metal, with
# the parameters that drive them.
crates/doc/src/body_transfer_sheet.rs
crates/doc/src/params.rs
crates/doc/src/sheet.rs
crates/doc/src/sheet_convert.rs
crates/doc/src/sheet_face_names.rs
crates/engine/src/cmd/parameters.rs
crates/engine/src/cmd/sheet.rs
crates/kernel/src/offset_backend.rs
crates/kernel/src/polyhedron.rs
crates/kernel/src/sew.rs
+19
View File
@@ -0,0 +1,19 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Shell, offset faces, thicken, split face and body, emboss and remove on shelled parts.
crates/doc/src/eval_more.rs
crates/doc/src/eval_surface.rs
crates/doc/src/params.rs
crates/engine/src/cmd/delete.rs
crates/engine/src/cmd/features_more.rs
crates/engine/src/cmd/parameters.rs
crates/engine/src/cmd/surface.rs
crates/kernel/src/delete_face.rs
crates/kernel/src/emboss.rs
crates/kernel/src/offset.rs
crates/kernel/src/offset_backend.rs
crates/kernel/src/polyhedron.rs
crates/kernel/src/splitface.rs
crates/kernel/src/surface_offset.rs
crates/kernel/src/surfaces.rs
+20
View File
@@ -0,0 +1,20 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Sweeps, lofts (rails, multi-section blends), pipes, coils, ribs, surface lofts and sweeps, their
# 3D sketch paths and the construction planes they run on.
crates/doc/src/construct.rs
crates/doc/src/eval_more.rs
crates/doc/src/eval_surface.rs
crates/doc/src/project3d.rs
crates/engine/src/cmd/construct.rs
crates/engine/src/cmd/features_more.rs
crates/engine/src/cmd/sketch_3d.rs
crates/engine/src/cmd/surface.rs
crates/kernel/src/helix.rs
crates/kernel/src/loft_blend.rs
crates/kernel/src/loft_rails.rs
crates/kernel/src/point_path.rs
crates/kernel/src/splitface.rs
crates/kernel/src/surfaces.rs
crates/kernel/src/sweep_path.rs
+7
View File
@@ -0,0 +1,7 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Display meshes (the tessellator itself is in the shared core): which edges the shaded view
# smooths over, and mesh bodies.
crates/kernel/src/meshbody.rs
crates/kernel/src/smooth_edges.rs
+12
View File
@@ -0,0 +1,12 @@
# The source files whose change reruns this area of the kernel rig (scripts/ci/kernel_rig_plan.py),
# besides the shared core in ../PATHS-ALL. One git pathspec glob a line, relative to the
# repository root; `#` starts a comment.
# Modelled and cosmetic threads, tapped holes, and the thread and pipe-thread catalogues.
crates/doc/src/eval_thread.rs
crates/doc/src/hole_reference.rs
crates/doc/src/pipe_thread_catalog.rs
crates/doc/src/pipe_thread_sizes.rs
crates/doc/src/thread_catalog.rs
crates/doc/src/thread_forms.rs
crates/engine/src/cosmetic_threads.rs
crates/kernel/src/helix.rs