* claim: #3983 * Agent skill for reproducible bug reports (#3983) skills/solidcraft-reporting has an agent gather the version, steps and exact error, reproduce the problem in a fresh design (a command script or one batch with rollback off), shrink it while it still fails, check the numbers with measure and a screenshot, keep private designs out, search for duplicates, and write the report in one shape: Reproduced on, Steps with a minimal script, Expected, Actual, Attachments. Listed in skills/README.md and docs/mcp.md; the bug template's steps field points agents to it. The skill-name check now covers its files. * Reporting skill: GitHub takes a design only as a .zip
This commit is contained in:
@@ -19,6 +19,7 @@ body:
|
||||
label: Steps to reproduce
|
||||
description: |
|
||||
What you clicked or typed, in order. For a problem on a particular design, `solidcraft-cli script design.solidcraft --out repro.json` turns its history into a script that `solidcraft-cli eval repro.json` replays; attach it, trimmed to the failing step if you can.
|
||||
Using an AI agent with SolidCraft's MCP server? The `solidcraft-reporting` skill in [`skills/`](https://github.com/tangency-labs/solidcraft/tree/main/skills) has it reproduce the problem in a fresh design and write these steps for you.
|
||||
placeholder: |
|
||||
1. Sketch a circle on the XY plane
|
||||
2. Extrude it 10 mm
|
||||
|
||||
@@ -1472,6 +1472,8 @@ fn skills_name_only_real_tools_and_commands() {
|
||||
include_str!("../../../skills/solidcraft-checking/reference/porting.md"),
|
||||
include_str!("../../../skills/solidcraft-mechanisms/SKILL.md"),
|
||||
include_str!("../../../skills/solidcraft-mechanisms/reference.md"),
|
||||
include_str!("../../../skills/solidcraft-reporting/SKILL.md"),
|
||||
include_str!("../../../skills/solidcraft-reporting/reference/report-template.md"),
|
||||
];
|
||||
let mut s = server();
|
||||
let tools: Vec<String> = tool_definitions().iter().filter_map(|t| t["name"].as_str()).map(str::to_owned).collect();
|
||||
|
||||
+2
-1
@@ -238,11 +238,12 @@ completely (from the notification area) and start it again to load the servers.
|
||||
|
||||
### Agent skills
|
||||
|
||||
[`skills/`](https://github.com/bherbruck/solidcraft/tree/main/skills) holds four [Agent Skills](https://agentskills.io) for agents that model through this server:
|
||||
[`skills/`](https://github.com/bherbruck/solidcraft/tree/main/skills) holds five [Agent Skills](https://agentskills.io) for agents that model through this server:
|
||||
- `solidcraft-modeling` covers building a new part.
|
||||
- `solidcraft-editing` covers changing an existing design.
|
||||
- `solidcraft-checking` adds checks for errors SolidCraft does not report, regression tests and porting, next to the other two.
|
||||
- `solidcraft-mechanisms` covers assemblies: joints placed by world coordinates, motion links, closed loops such as actuators, contact sets and motion studies, checked against analytic values.
|
||||
- `solidcraft-reporting` turns a problem the user hit into a bug report: it reproduces it in a fresh design, shrinks it to a minimal command script, checks the numbers with `measure` and a screenshot, and searches existing issues first.
|
||||
|
||||
They cover the workflow, the command shapes, how to verify with `measure`, and the known traps. With them an agent builds a part in a few `batch` calls and reads far less output. To install them for Claude Code, copy the folders into `~/.claude/skills/`; [skills/README.md](https://github.com/bherbruck/solidcraft/blob/main/skills/README.md) has the details.
|
||||
|
||||
|
||||
+2
-1
@@ -15,6 +15,7 @@ agent without file access (such as the one in SolidCraft's own chat) can still r
|
||||
| [`solidcraft-editing`](solidcraft-editing/SKILL.md) | changes, fixes or extends an existing design |
|
||||
| [`solidcraft-checking`](solidcraft-checking/SKILL.md) | builds, changes or ports a design, next to the two above: parameter ranges, every allowed value, exact STEP comparison, regression tests, designs kept in git |
|
||||
| [`solidcraft-mechanisms`](solidcraft-mechanisms/SKILL.md) | builds, drives or animates an assembly: components, joints, motion links, closed loops such as actuators, contact sets, motion studies |
|
||||
| [`solidcraft-reporting`](solidcraft-reporting/SKILL.md) | hears that something went wrong (a crash, a failed command, a wrong shape or number, a hang): reproduces it in a fresh design, shrinks it to a minimal script and writes the bug report |
|
||||
|
||||
The modelling skills teach the agent to:
|
||||
- plan the part as numbers
|
||||
@@ -24,7 +25,7 @@ The modelling skills teach the agent to:
|
||||
|
||||
They also list the traps that otherwise cost failed calls. Examples are the sketch axes of the XZ plane, edge picks that snap to the nearest edge, and cuts that miss but still report success.
|
||||
|
||||
The modelling skills are written for SolidCraft 0.8 and later; `solidcraft-mechanisms` for `main` after 0.11.1 (the next release), which counts the degrees of freedom of closed loops.
|
||||
The modelling skills are written for SolidCraft 0.8 and later; `solidcraft-mechanisms` for `main` after 0.11.1 (the next release), which counts the degrees of freedom of closed loops; `solidcraft-reporting` for 0.18 and later.
|
||||
|
||||
## Install
|
||||
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: solidcraft-reporting
|
||||
description: Use when a SolidCraft user says something went wrong - the app crashed or closed, a command failed or gave the wrong shape, a number is off, a design no longer rebuilds, something hangs - and wants it reported or fixed upstream. Turns the problem into a minimal, reproducible bug report with exact steps, a command script, expected and actual results and a screenshot.
|
||||
---
|
||||
|
||||
# SolidCraft reporting
|
||||
|
||||
A report the maintainers can act on reproduces in a fresh design, in as few steps as possible, with the exact message and numbers. Build that here, so nobody has to rebuild the problem from a description. Use `reference.md` from `solidcraft-modeling` for command shapes, and `reference/report-template.md` (next to this file) for the report.
|
||||
|
||||
Written against SolidCraft 0.18.2. Report what you observed, never a guessed cause: "the fillet fails with `…`", not "the kernel can't handle tangent edges".
|
||||
|
||||
## 1. Gather (ask only for what you cannot find)
|
||||
- **Version and OS.** Help → About in the app, `solidcraft --version` or `solidcraft-cli --version`; the MCP server reports its version in `initialize` (`serverInfo.version`). Note whether the user built from source.
|
||||
- **What the user did**, in order, what they expected and what happened instead.
|
||||
- **The exact error text**, copied, not paraphrased: a failed `execute` or `batch` step's `failed.error`, the timeline feature's `error` in `inspect_design {"measure":false,"sketches":false}`, or the message in the app.
|
||||
- **If the app closed or froze:** `crash_reports {}` first. A matching report has the version, the command that crashed, the commands before it and a replay. `solidcraft-cli crashes replay <id>` checks that it still happens. Its `design.solidcraft` is the user's design: see Privacy.
|
||||
|
||||
## 2. Reproduce in a fresh design
|
||||
Start from nothing, so the report does not depend on the user's file.
|
||||
- **Headless:** write the steps as a command script `{"commands":[...]}` and run `solidcraft-cli run repro.json`. A failing feature fails the run and names the feature. This never touches the design open in the app.
|
||||
- **Over MCP:** `new_design {}`, then the steps in one `batch {"commands":[...],"rollback":false}`. With `rollback: false` the steps before the failure stay, so you can inspect the state it failed in. When the server is bridged to the app and the open design is `dirty`, ask before `new_design`.
|
||||
- Find commands and parameters with `list_commands {"filter":"fillet"}` and `describe_command {"id":"solid.fillet"}`, not the unfiltered catalog.
|
||||
- **Only in the user's design:** `solidcraft-cli script design.solidcraft --out repro.json` writes its history as a command script. `solidcraft-cli eval repro.json` replays it and prints the measurements; `solidcraft-cli run repro.json` stops at the first failing feature. If the script rebuilds without the problem, say so in the report: it then depends on something the history does not hold (an import, a file the design links to).
|
||||
- **Interactive only** (a dialog, a pick, a drag, the viewport): write the clicks as numbered app steps. A script cannot show those.
|
||||
- Check that it fails the same way twice. A problem that comes and goes is still worth reporting; say how often it happened.
|
||||
|
||||
## 3. Minimise
|
||||
Shrink while the same failure, with the same message, still happens. Re-run after every change:
|
||||
1. Drop steps from the end, then from the middle: other bodies, features after the failing one, cosmetic steps (appearance, names, views).
|
||||
2. Replace sketches with `solid.box` or `solid.cylinder` where the failure survives it.
|
||||
3. Round the dimensions (10, 20, 2.5) and replace parameter expressions with numbers, unless an expression is the problem.
|
||||
4. Stop when removing anything more makes it pass. Note the boundary when you find one ("fails at radius 5, works at 4.9").
|
||||
|
||||
**Hangs and slow commands:** never wait forever. Cap every run: `timeout 120 solidcraft-cli run repro.json --timings` prints the milliseconds of each step when the run finishes. A run that is killed prints nothing, so find the slow step by running the script one step longer at a time. Over MCP, `batch {"commands":[...],"timings":true}` reports every step's time, the failing one included. Report the step, the time, and how the time grows when you double a count or a size.
|
||||
|
||||
## 4. Check the claim
|
||||
- Wrong geometry: `measure {}` (volume, area, bounding box, centre of mass) against a value you computed by hand. Put both numbers in the report.
|
||||
- Wrong pick or wrong face: `body_topology {"body":"Body1"}` shows what was there to pick.
|
||||
- A look at it: `screenshot {"view":"iso"}` after the `rollback: false` batch, or `solidcraft-cli snapshot repro.json --out shot.png --view iso`. `snapshot` fails with the script when a step fails, so for a failing command snapshot the script without that step and show the state it fails on. Look at the image before you attach it.
|
||||
- Re-run the final minimal script from scratch once more. Its output goes into the report as it is.
|
||||
|
||||
## 5. Privacy
|
||||
- Never attach, upload or quote the user's design, a crash report's `design.solidcraft`, or file contents without asking. The minimal synthetic script is the default attachment.
|
||||
- GitHub does not accept `.solidcraft` attachments. A design the user agrees to share goes up as a `.zip`: a `.solidcraft` file is a ZIP already, so copy or rename it to `name.solidcraft.zip`, or zip it. SolidCraft opens either kind of `.zip`.
|
||||
- Strip home folders, user names, part and project names and file paths from messages and logs (`/home/alice/clients/acme/bracket.solidcraft` → `bracket.solidcraft`).
|
||||
- Check screenshots for other windows, file names and anything personal.
|
||||
|
||||
## 6. Search, then write
|
||||
- Search before writing: `gh issue list -R tangency-labs/solidcraft --state all --search "fillet radius"` with two or three words from the error and the command. If one matches, add the minimal script and your version to it as a comment, instead of filing a new one.
|
||||
- Write the report with `reference/report-template.md`. Title `<Area>: <what fails>`, such as `Fillet: a radius above half the edge's face fails with "…"`.
|
||||
- Show the user the finished report. File it only after they approve: `gh issue create -R tangency-labs/solidcraft --label bug --title "…" --body-file report.md`, or let them paste it into **Report a bug** in the app's title bar, which adds a screenshot they review first.
|
||||
|
||||
## Traps
|
||||
- **The wrong version.** The CLI on `PATH` can be older or newer than the app the user ran. Reproduce with the user's version and say which one you used.
|
||||
- **Fixed already.** If it does not reproduce on the latest release or `main` but does on the user's version, say so: it saves a report.
|
||||
- **Edge picks** snap to the nearest edge with no distance limit and no error, so a repro can pick a different edge than the user did. Check the face count or `body_topology` after the step.
|
||||
- **A success can be the bug.** A cut that misses its body still reports success. Measure, don't trust the reply.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Report template
|
||||
|
||||
Fill in every section and delete the hints in angle brackets. Keep it short: the script and the numbers say most of it.
|
||||
|
||||
````markdown
|
||||
**Reproduced on:** SolidCraft <version> (<release, or built from source at commit …>), <OS and version>, <GPU, for a display problem>
|
||||
<or: **Only in my design:** <what was tried: its history as a script rebuilds fine / fails the same way>>
|
||||
|
||||
### Steps
|
||||
1. <app step, in the app's words: Sketch → Rectangle on the XY plane, 20 × 20 mm>
|
||||
2. <…>
|
||||
3. <the step that fails>
|
||||
|
||||
Minimal script (`solidcraft-cli run repro.json`):
|
||||
|
||||
```json
|
||||
{"commands":[
|
||||
<the smallest command list that still fails>
|
||||
]}
|
||||
```
|
||||
|
||||
### Expected
|
||||
<what should happen, with the number you computed>
|
||||
|
||||
### Actual
|
||||
<what happens, with the measured numbers>
|
||||
|
||||
```text
|
||||
<the error text or command output, copied as it is>
|
||||
```
|
||||
|
||||
<how often: every time / 3 of 5 runs; the boundary if you found one>
|
||||
|
||||
### Attachments
|
||||
<screenshot of the failure; the crash report id if there is one; no private designs unless the user agreed>
|
||||
````
|
||||
|
||||
## Example
|
||||
|
||||
The shape filled in, for a command that fails with an error. (This one is expected behaviour; a real report would be one where Actual differs from Expected.)
|
||||
|
||||
````markdown
|
||||
**Reproduced on:** SolidCraft 0.18.2 (release), Linux x86_64
|
||||
|
||||
### Steps
|
||||
1. Box, 20 × 20 × 10 mm, corner at the origin.
|
||||
2. Fillet the top front edge, radius 12 mm.
|
||||
|
||||
Minimal script (`solidcraft-cli run repro.json`):
|
||||
|
||||
```json
|
||||
{"commands":[
|
||||
{"command":"solid.box","params":{"length":20,"width":20,"height":10}},
|
||||
{"command":"solid.fillet","params":{"edges":[[10,0,10]],"radius":12}}
|
||||
]}
|
||||
```
|
||||
|
||||
### Expected
|
||||
The fillet fails and names the largest radius that fits: 10 mm, the box's height.
|
||||
|
||||
### Actual
|
||||
```text
|
||||
error: step 2 (solid.fillet): not supported yet: the blend is larger than the neighbouring faces; the largest radius that fits is 10 mm
|
||||
```
|
||||
Every time. Radius 2 works: volume 3982.8 mm³, as computed by hand (4000 − 20 × 2² × (1 − π/4)).
|
||||
````
|
||||
Reference in New Issue
Block a user