# Onshape to CDSL

This package produces an auditable CDSL dataset from official public Onshape
Part Studio URL mappings. It never treats a feature tree as sufficient proof:
each accepted record has its complete raw API evidence, original AP242 STEP,
CDSL-only rebuilt STEP, and an independent strict geometry report.

```text
scan -> select -> download -> convert -> validate -> dataset/<id>.cdsl.json
```

Run from the repository root. The only runtime path setup needed is the engine
and this source package; no new packaging tool is introduced.

```bash
export ONSHAPE_ACCESS_KEY='...'
export ONSHAPE_SECRET_KEY='...'
export PYTHONPATH=backend/engine:onshape_to_cdsl/src

python -m onshape_to_cdsl scan --url-file /path/to/objects.yml --limit 500 --workers 1
python -m onshape_to_cdsl select --count 100 --seed 0
python -m onshape_to_cdsl download
python -m onshape_to_cdsl convert
python -m onshape_to_cdsl validate
```

To find more than one URL range yields, scan non-overlapping batches and merge
them before deterministic selection:

```bash
python -m onshape_to_cdsl scan --url-file objects_0000.yml --offset 0 --limit 500 --out data/scan/00000.json
python -m onshape_to_cdsl scan --url-file objects_0000.yml --offset 500 --limit 500 --out data/scan/00500.json
python -m onshape_to_cdsl merge-scans --scan data/scan/00000.json --scan data/scan/00500.json --out data/scan/merged.json
python -m onshape_to_cdsl select --scan data/scan/merged.json --count 100 --seed 0
```

For hosts that limit foreground process duration, download the deterministic
selection in resumable batches. Existing artifacts are reused by SHA-256:

```bash
python -m onshape_to_cdsl download --offset 0 --limit 3
python -m onshape_to_cdsl download --offset 3 --limit 3
```

Or run all stages in sequence:

```bash
python -m onshape_to_cdsl run --url-file /path/to/objects.yml --scan-limit 500 --scan-workers 1 --count 100 --seed 0
```

`--url-file` accepts `id URL`, `id: URL`, and the same-line ID/URL records in
ABC `objects_*.yml`. URLs are deliberately not hard-coded in the repository.
Credentials are read from `ONSHAPE_ACCESS_KEY`/`ONSHAPE_SECRET_KEY` or hidden
terminal prompts, and are never written to files or logs.

The official ABC mapping archive is published by NYU as
`abc_objects_00-49.7z` and `abc_objects_50-99.7z`. Each contained
`objects_*.yml` file has 10,000 URLs. Start with `--scan-limit 500`; the
pipeline stratifies those API-inspected candidates and selects 100 complete
downloads. The client honors `429 Retry-After` responses; use one scan worker
unless the API quota explicitly permits higher concurrency.

All generated data lives in ignored `onshape_to_cdsl/data/`:

```text
scan/manifest.json          feature classifications for all source URLs
selection/manifest.json     deterministic, stratified selected candidates
raw/<id>/                   complete public API responses, meshes, STEP and hashes
converted/<id>/             candidate CDSL, rebuild STEP, conversion/validation reports
dataset/<id>.cdsl.json      only strict-validation accepted CDSL
rejected/<id>/              candidate/rebuild/report evidence for every failure
reconstruction_issues.md    generated rejection and engine capability work queue
reconstruction_issues.json  machine-readable version of the same register
```

The v1 executable converter intentionally supports only solved `newSketch`
geometry composed of lines, arcs and circles plus standard, one-direction,
no-draft `BLIND` extrudes (`NEW`, `ADD`, `REMOVE`). The registry audits all
other feature types. Unsupported standard operations are `deferred`; custom
FeatureScript, imports and external/derived dependencies are
`not_admissible`. Their raw feature trees, FeatureScript payload, B-rep and
STEP remain in `raw/`, but they cannot enter `dataset/`.

Admission uses a separate comparator, not the engine's preview comparator.
It requires bidirectional B-rep surface sample maximum and P99 distances at
most `0.01 mm`, six-coordinate bounding-box error at most `0.01 mm`, volume
and surface-area relative error at most `1e-5`, and equal solid counts.
Topology counts/types are report-only diagnostics because kernel exports may
split equivalent faces differently.

`validate` regenerates `reconstruction_issues.md` on every run. It classifies
each failure as a source dependency limitation, converter gap, engine
capability gap, engine execution failure, or strict geometric regression, and
adds engine-specific blockers to an explicit implementation work queue.