Files
makeCAD/skills/solidcraft-tests/SKILL.md
T
bherbruck 3a504f5347 Agents: AGENTS.md becomes a router to focused skills, with a staleness check (Refs #6439) (#6444)
AGENTS.md drops from 16.8k to 3.7k characters: the hard rules, each naming the
check that enforces it, and a table of what to read for each kind of work.
The rest moves into eleven new skills; the five existing ones stay.

cargo xtask skills fails when AGENTS.md or a SKILL.md cites a path, an xtask
command or a skill that does not exist, when the router table and skills/
disagree, or when the router or a new skill grows past its size cap. It runs
in cargo xtask ci and the repository checks job.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 11:51:19 +00:00

31 lines
3.0 KiB
Markdown

---
name: solidcraft-tests
description: "Use when writing, placing or running tests in SolidCraft: a regression test for a bug, an engine test for a command, a UI scenario, a reference (oracle) test, goldens, or when CI fails a test you did not touch."
---
# Where tests go and how to run them
The long form is `docs/book/src/dev/testing.md`.
## Placement
- A regression test goes into the existing test file for its area (booleans, blends, shell, sweep, STEP, naming, ...), never a new file named after the bug (#3226).
- Engine tests sit next to the code or in `*_tests.rs` files; they measure the result (volume, area, face/edge/vertex counts) against a value you can derive.
- UI scenarios are JSON in `crates/ui-egui/tests/scenarios/*.json`; a new file is picked up automatically. Put `{"pending": "why"}` in a scenario that records a bug before it is fixed, and remove it in the fixing PR.
- Reference tests are `tests/oracle/<case>/recipe.json` with an `expected.json` beside it, run by `cargo xtask oracle`. Expected values are worked out in closed form (`tools/oracle-analytic/`) or computed by FreeCAD run as a separate program (`tools/freecad-oracle/`). A case that fails on a known bug names its issue (`"known_failure": "#123"`). Never change an expected value to match SolidCraft's output, and never take one from a commercial CAD product.
- Crash fixes: a regression test, started from the crash report's `design.solidcraft` and `replay.json` (`skills/solidcraft-crash-safety/SKILL.md`).
## A failing test needs its fix
A failing test without its fix cannot merge: it turns `main` red for everyone. Include the fix in the same PR, or mark the test `#[ignore = "fails until #123 is fixed"]` and remove that line in the PR that fixes it.
## Running
- `cargo xtask test` builds with the `ci` profile and runs every test binary at once; arguments after `--` go to each binary (`cargo xtask test -- shell`). Use the `ci` profile: it builds several times faster than `--release`, and debug assertions are off because truck's reject valid geometry. If a test fails only in a plain debug build, say so in the PR.
- `SOLIDCRAFT_TEST_JOBS=N` bounds concurrent commands on a shared machine.
- `cargo xtask ci` is the gate before a commit: fmt, assets, icons, licences, lists, layers, ui-containers, clippy `-D warnings`, tests and the wasm check.
- Every test runs in its own process: do not rely on state left by another test, or on wall-clock timing.
## Goldens, matrices and skipped tests
- A geometry change that moves the robustness goldens re-records them in the same PR (`ROBUSTNESS_GOLDENS_BLESS="why it changed"`, `docs/robustness.md`).
- The action matrix is a ratchet: record a newly passing action with `cargo xtask action-matrix --bless` (`docs/action-matrix.md`).
- Pull requests skip a few heavy tests, which run in the merge queue, nightly and on `main`. Check that validation after merging. The list is in `.config/nextest.toml` and the testing page.
- Mutation testing finds behaviour the suites miss: `docs/mutation-testing.md`.