95 lines
4.4 KiB
Markdown
95 lines
4.4 KiB
Markdown
# Onshape to CDSL
|
|
|
|
This package produces an auditable CDSL dataset from official public Onshape
|
|
Part Studio URL mappings. It never treats a feature tree as sufficient proof:
|
|
each accepted record has its complete raw API evidence, original AP242 STEP,
|
|
CDSL-only rebuilt STEP, and an independent strict geometry report.
|
|
|
|
```text
|
|
scan -> select -> download -> convert -> validate -> dataset/<id>.cdsl.json
|
|
```
|
|
|
|
Run from the repository root. The only runtime path setup needed is the engine
|
|
and this source package; no new packaging tool is introduced.
|
|
|
|
```bash
|
|
export ONSHAPE_ACCESS_KEY='...'
|
|
export ONSHAPE_SECRET_KEY='...'
|
|
export PYTHONPATH=backend/engine:onshape_to_cdsl/src
|
|
|
|
python -m onshape_to_cdsl scan --url-file /path/to/objects.yml --limit 500 --workers 1
|
|
python -m onshape_to_cdsl select --count 100 --seed 0
|
|
python -m onshape_to_cdsl download
|
|
python -m onshape_to_cdsl convert
|
|
python -m onshape_to_cdsl validate
|
|
```
|
|
|
|
To find more than one URL range yields, scan non-overlapping batches and merge
|
|
them before deterministic selection:
|
|
|
|
```bash
|
|
python -m onshape_to_cdsl scan --url-file objects_0000.yml --offset 0 --limit 500 --out data/scan/00000.json
|
|
python -m onshape_to_cdsl scan --url-file objects_0000.yml --offset 500 --limit 500 --out data/scan/00500.json
|
|
python -m onshape_to_cdsl merge-scans --scan data/scan/00000.json --scan data/scan/00500.json --out data/scan/merged.json
|
|
python -m onshape_to_cdsl select --scan data/scan/merged.json --count 100 --seed 0
|
|
```
|
|
|
|
For hosts that limit foreground process duration, download the deterministic
|
|
selection in resumable batches. Existing artifacts are reused by SHA-256:
|
|
|
|
```bash
|
|
python -m onshape_to_cdsl download --offset 0 --limit 3
|
|
python -m onshape_to_cdsl download --offset 3 --limit 3
|
|
```
|
|
|
|
Or run all stages in sequence:
|
|
|
|
```bash
|
|
python -m onshape_to_cdsl run --url-file /path/to/objects.yml --scan-limit 500 --scan-workers 1 --count 100 --seed 0
|
|
```
|
|
|
|
`--url-file` accepts `id URL`, `id: URL`, and the same-line ID/URL records in
|
|
ABC `objects_*.yml`. URLs are deliberately not hard-coded in the repository.
|
|
Credentials are read from `ONSHAPE_ACCESS_KEY`/`ONSHAPE_SECRET_KEY` or hidden
|
|
terminal prompts, and are never written to files or logs.
|
|
|
|
The official ABC mapping archive is published by NYU as
|
|
`abc_objects_00-49.7z` and `abc_objects_50-99.7z`. Each contained
|
|
`objects_*.yml` file has 10,000 URLs. Start with `--scan-limit 500`; the
|
|
pipeline stratifies those API-inspected candidates and selects 100 complete
|
|
downloads. The client honors `429 Retry-After` responses; use one scan worker
|
|
unless the API quota explicitly permits higher concurrency.
|
|
|
|
All generated data lives in ignored `onshape_to_cdsl/data/`:
|
|
|
|
```text
|
|
scan/manifest.json feature classifications for all source URLs
|
|
selection/manifest.json deterministic, stratified selected candidates
|
|
raw/<id>/ complete public API responses, meshes, STEP and hashes
|
|
converted/<id>/ candidate CDSL, rebuild STEP, conversion/validation reports
|
|
dataset/<id>.cdsl.json only strict-validation accepted CDSL
|
|
rejected/<id>/ candidate/rebuild/report evidence for every failure
|
|
reconstruction_issues.md generated rejection and engine capability work queue
|
|
reconstruction_issues.json machine-readable version of the same register
|
|
```
|
|
|
|
The v1 executable converter intentionally supports only solved `newSketch`
|
|
geometry composed of lines, arcs and circles plus standard, one-direction,
|
|
no-draft `BLIND` extrudes (`NEW`, `ADD`, `REMOVE`). The registry audits all
|
|
other feature types. Unsupported standard operations are `deferred`; custom
|
|
FeatureScript, imports and external/derived dependencies are
|
|
`not_admissible`. Their raw feature trees, FeatureScript payload, B-rep and
|
|
STEP remain in `raw/`, but they cannot enter `dataset/`.
|
|
|
|
Admission uses a separate comparator, not the engine's preview comparator.
|
|
It requires bidirectional B-rep surface sample maximum and P99 distances at
|
|
most `0.01 mm`, six-coordinate bounding-box error at most `0.01 mm`, volume
|
|
and surface-area relative error at most `1e-5`, and equal solid counts.
|
|
Topology counts/types are report-only diagnostics because kernel exports may
|
|
split equivalent faces differently.
|
|
|
|
`validate` regenerates `reconstruction_issues.md` on every run. It classifies
|
|
each failure as a source dependency limitation, converter gap, engine
|
|
capability gap, engine execution failure, or strict geometric regression, and
|
|
adds engine-specific blockers to an explicit implementation work queue.
|