likang/cdsl-cad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

重构 改为标准化输出

Browse Source
This commit is contained in:
李康
2026-09-01 14:10:23 +08:00
parent 7533b298f1
commit bc87bd6d2d
10151 changed files with 1709487 additions and 11776 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/cad_agent/evals/TOKEN_BASELINE.md
+21
View File
@@ -0,0 +1,21 @@
# Token Baseline Format
`python -m app.cad_agent.evals.live --suite release --require-live` requires
`--baseline-report` to point at a measured protocol-2 report. It is not a
fixture or an estimate. Record three independent runs of every release
scenario with the same author request configuration and runtime profile.
| Field | Requirement |
| --- | --- |
| `schema_version` | `cad.token-baseline.v1` |
| `protocol_version` | `2.0` |
| `author` | Exact `provider`, `model`, `api_style`, `reasoning_effort`, and `sampling` object from the v3 report |
| `runtime_profile_sha256` | Exact v3 `runtime_profile_sha256` |
| `measurement` | `prompt_tokens` when provider usage exists; otherwise `context_chars` |
| `scenarios[]` | One entry for every versioned release scenario, including its request SHA-256 |
| `repetitions[]` | Entries `1`, `2`, and `3`, with non-negative `author_metric` and `plan_review_metric`, plus completion, final-review, and deterministic-claim booleans |
The comparison uses `author_metric + plan_review_metric` for the v2 median and
the v3 author metric only. It fails closed when provenance differs, any
repetition is absent, completion/review/claim rates drop, plan-review calls
remain, or the median reduction is below 30%.