fix(runtime, exporter): a script can model and export STEP in a read-only, unattended run (#89)
DECISION: three changes for the three obstacles of the issue, each the smallest that removes the obstacle. No new API name, no relocatable state directory, no rewrite of rule R3. 1. A run without the cache touches nothing next to the source (the crash). PROBLEM: `run_notebook(path, cache=False)` failed on a read-only source directory (`Errno 30` on Linux, `WinError 5` here): `_run` ended with `write_dependencies`, which writes `__marimo__/.../deps.json` beside the notebook, and `exclusive()` made the state directory for its lock. REPRODUCED on Windows with an ACL that denies creating entries in the directory, on the issue's own notebook: PermissionError at runner.py `write_dependencies`. After the change the same run prints `ok` and creates nothing. WHY THIS FIX: the dependency record exists for the cache (cache entries are keyed on the files it lists); a run that keeps no entries has no reason to move it, and moving it left the record and the entries out of step. The lock exists to protect pruning and that record, neither of which a run without the cache does. A run WITH the cache still writes its state and still fails loudly when it cannot: asking for persistence and silently not getting it would be worse. REJECTED: an environment variable or argument that moves the state directory (more surface, and the problem is that there should be no state, not that it is in the wrong place); swallowing the OSError (hides real cache failures). 2. The exporters take a build result (one step, no `.scadpkg` on disk). PROBLEM: STEP from a freshly built Part needed `capture(result, path)` and then the exporter on that path, and the default capture tessellates a scene nobody reads: 1.85 s against 0.26 s for the issue's 36-hole plate. WHY THIS FIX: `build_product_package` already accepts a build result or a definition and can leave the scene out; `read_product_package_translation_units`, which every exporter and translator goes through, now builds that package in memory when it is given neither a package nor bytes nor a path. So STEP, STL, OBJ, MJCF and FreeCAD all gain the one-step form with the same validation, under the existing names. A value that is no product still raises TypeError. REJECTED: `exporter.step.export_part(part_or_solid, path)` as the issue suggests: a bare Part or solid has no definition (feature graph, topology snapshot, hashes), those come from `@scad.part`; and a second family of names beside `export_product_package_to_*` (the tests pin that these live only in `scad.exporter`). 3. The skill lists the plain script as an entry for a single-script task. PROBLEM: R3 says a model is a marimo notebook; an agent given "one script, `--output part.step`" read the skill and chose another library. WHY THIS FIX: R3 keeps its force for everything a person opens or reuses; it gains one exception, a task that itself asks for a single script, and the template (which I ran in a read-only directory: STEP written, source untouched) lives in the export domain page next to the rule it relaxes. THIS IS A POLICY DECISION OF THE OWNER, made here provisionally: R3 was a deliberate rule, and the PR says so. Not done: the issue's timings were not rerun on Linux, only on this Windows machine (same direction). Checked: test_notebook_runtime and the cache tests (123 passed), test_product_exporter/packages/capture/assembly pass with the three new exporter tests, test_skill_lint passes. test_skill_build::test_files_without_directives_copied_verbatim fails on the base commit as well. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -15,7 +15,9 @@ the durable export call `capture(result, path)`, whose two required arguments ar
|
||||
notebook; one assembly notebook per product, composing children with `scad.use(...)`; parameters live in the notebook that
|
||||
consumes them. Cell boundaries, `_private` names and `var()` follow `discipline/notebook-cell-design.md`; block structure
|
||||
and the mandatory header `# ---- feature: <slug> (<role>) ----` follow `discipline/feature-tree-convention.md`. Read both
|
||||
before writing any cell.
|
||||
before writing any cell. The one exception is a task that itself asks for a single script
|
||||
(`python model.py --output part.step`, an unattended batch): that is a plain script, see "One-shot script" in
|
||||
`domains/export-and-translation.md`; the notebook rules do not apply to it.
|
||||
|
||||
**R4. Sketch first.** 2D profiles and planar paths go through the sketch API; primitives only when the shape is entirely
|
||||
the basic form.
|
||||
|
||||
@@ -60,8 +60,10 @@ Read the exact page under `references/docs/api/` for every API used:
|
||||
|
||||
## Export rules
|
||||
|
||||
- Translate only from a validated `.scadpkg` package: `capture` the product
|
||||
first, then export from the package path.
|
||||
- Translate only from a validated product package: `capture` the product
|
||||
first, then export from the package path. In a one-shot script (below) the
|
||||
exporters also take the build result itself and package it in memory, with
|
||||
the same validation and without the scene projection nobody reads.
|
||||
- STL and OBJ share one direct OpenCASCADE tessellation of the evaluated BREP;
|
||||
both contain the same oriented triangles. Control curved-surface accuracy
|
||||
with `linear_deflection` (chordal deviation in product units) and
|
||||
@@ -75,6 +77,38 @@ Read the exact page under `references/docs/api/` for every API used:
|
||||
- Use `validate_step_roundtrip_rdescriptor` when a STEP export must survive
|
||||
re-import (roundtrip-sensitive consumers).
|
||||
|
||||
## One-shot script (no notebook, no `.scadpkg` on disk)
|
||||
|
||||
For a batch or unattended task whose deliverable is "one script, run as
|
||||
`python model.py --output part.step`" (read-only source directory, no editor),
|
||||
a plain script is the supported entry. A notebook is still the form for
|
||||
anything a person will open, edit or reuse (core rule R3). The runtime keeps
|
||||
no state next to the source when it runs without the cache.
|
||||
|
||||
```python
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import simplecadapi as scad
|
||||
|
||||
|
||||
@scad.part(id="plate", project_root=Path(__file__).parent)
|
||||
def build_plate() -> scad.Part:
|
||||
body = scad.make_box_rsolid(width=120.0, height=120.0, depth=10.0)
|
||||
return scad.make_part_rpart("plate", body)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--output", required=True)
|
||||
args = parser.parse_args()
|
||||
report = scad.exporter.export_product_package_to_step(build_plate(), args.output)
|
||||
print(report.output_path, report.definition_ids)
|
||||
```
|
||||
|
||||
A notebook run unattended in a read-only directory uses
|
||||
`run_notebook(path, cache=False)`.
|
||||
|
||||
## Validation gates
|
||||
|
||||
- Read the returned export report and print its actual fields per format
|
||||
|
||||
@@ -10,7 +10,7 @@ import numpy as np
|
||||
from ..artifacts.assembly_io import materialize_definition
|
||||
from simplecadapi._cpp import Assembly, make_compound_from_assembly_rcompound, Part
|
||||
from ..translator.package_units import (
|
||||
ProductPackageInput,
|
||||
ProductInput,
|
||||
read_product_package_translation_units,
|
||||
)
|
||||
|
||||
@@ -122,7 +122,7 @@ def tessellate_solid(
|
||||
|
||||
|
||||
def tessellate_product_surface(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
*,
|
||||
linear_deflection: float,
|
||||
angular_deflection_degrees: float,
|
||||
|
||||
@@ -20,7 +20,7 @@ from ..artifacts.part_definition import PartDefinition
|
||||
from simplecadapi._cpp import Assembly, Part
|
||||
from ..product.placement import Placement, relative_placement
|
||||
from ..translator.package_units import (
|
||||
ProductPackageInput,
|
||||
ProductInput,
|
||||
read_product_package_translation_units,
|
||||
)
|
||||
from ._surface_mesh import tessellate_solid
|
||||
@@ -339,7 +339,7 @@ def _part_materials(units: tuple[Any, ...]) -> dict[str, tuple[Part, float, bool
|
||||
|
||||
|
||||
def export_product_package_to_mjcf(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
output_path: str | Path,
|
||||
*,
|
||||
mesh_directory: str | Path | None = None,
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ..translator.package_units import ProductPackageInput
|
||||
from ..translator.package_units import ProductInput
|
||||
from ._surface_mesh import tessellate_product_surface
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ class ProductOBJExportReport:
|
||||
|
||||
|
||||
def export_product_package_to_obj(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
output_path: str | Path,
|
||||
*,
|
||||
linear_deflection: float = 0.1,
|
||||
|
||||
@@ -13,7 +13,7 @@ from typing import Any
|
||||
from ..artifacts import materialize_definition
|
||||
from simplecadapi._cpp import Assembly, Part
|
||||
from ..translator.package_units import (
|
||||
ProductPackageInput,
|
||||
ProductInput,
|
||||
read_product_package_translation_units,
|
||||
)
|
||||
|
||||
@@ -152,10 +152,11 @@ def _native_definitions(root_value: Part | Assembly, units: Any) -> list[dict[st
|
||||
|
||||
|
||||
def export_product_package_to_step(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
output_path: str | Path,
|
||||
) -> ProductSTEPExportReport:
|
||||
"""Export a validated `.scadpkg` closure as XCAF-backed AP242 STEP."""
|
||||
"""Export a product as XCAF-backed AP242 STEP: a validated `.scadpkg` closure (package, bytes or path), or the result of
|
||||
a build (``@scad.part`` / ``@scad.assemble``) or its definition, which is packaged in memory without a scene."""
|
||||
|
||||
import simplecadapi._cpp as native
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ import struct
|
||||
|
||||
import numpy as np
|
||||
|
||||
from ..translator.package_units import ProductPackageInput
|
||||
from ..translator.package_units import ProductInput
|
||||
from ._surface_mesh import TessellatedSurfaceMesh, tessellate_product_surface
|
||||
|
||||
|
||||
@@ -51,7 +51,7 @@ def _write_binary_stl(path: Path, mesh: TessellatedSurfaceMesh) -> None:
|
||||
|
||||
|
||||
def export_product_package_to_stl(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
output_path: str | Path,
|
||||
*,
|
||||
linear_deflection: float = 0.1,
|
||||
|
||||
@@ -99,7 +99,7 @@ _held_lock = threading.Lock()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def exclusive(config: NotebookConfig) -> Iterator[None]:
|
||||
def exclusive(config: NotebookConfig, *, enabled: bool = True) -> Iterator[None]:
|
||||
"""Hold the notebook's state directory against other processes.
|
||||
|
||||
Worker processes may run the same notebook at once (two notebooks that
|
||||
@@ -107,9 +107,13 @@ def exclusive(config: NotebookConfig) -> Iterator[None]:
|
||||
lock is between processes only: a run of a notebook this process is
|
||||
already running (a ``use`` cycle, an editor next to a headless run) goes
|
||||
on without it, as it always did. Without ``fcntl`` (Windows) it is a
|
||||
no-op.
|
||||
no-op. With ``enabled=False`` (a run that keeps no state: nothing to
|
||||
prune, nothing to protect) it is a no-op too, and creates nothing.
|
||||
"""
|
||||
|
||||
if not enabled:
|
||||
yield
|
||||
return
|
||||
directory = state_dir(config)
|
||||
key = str(directory)
|
||||
with _held_lock:
|
||||
|
||||
@@ -170,7 +170,7 @@ def run_notebook(
|
||||
# Errors still come through. "marimo" is the logger every marimo
|
||||
# module shares (lazy.py's record shows its file name, "lazy", which
|
||||
# is not the logger's name).
|
||||
with suppress_warnings_logs("marimo"), exclusive(config):
|
||||
with suppress_warnings_logs("marimo"), exclusive(config, enabled=cache):
|
||||
return _run(config, values, cache=cache, through=through, product=product)
|
||||
|
||||
# marimo keeps its runtime context per thread, and a notebook run needs
|
||||
@@ -322,7 +322,10 @@ def _run(
|
||||
# Cells restored from the cache ran no ``use`` this time; the files they
|
||||
# depend on are among those the previous run recorded.
|
||||
recorded = frozenset(used) | (previous if restored else frozenset())
|
||||
write_dependencies(config, recorded)
|
||||
if cache:
|
||||
# The record belongs to the cache: its entries are keyed on the files it lists, so a run that keeps no entries must
|
||||
# not move it (and a run without the cache must work where the source directory is read-only, issue #89).
|
||||
write_dependencies(config, recorded)
|
||||
dependencies = module_files | recorded
|
||||
if cache and cell_cache is not None:
|
||||
# A cell that runs a child notebook depends on the record just written:
|
||||
|
||||
@@ -11,6 +11,7 @@ from ..artifacts.feature_graph import FeatureGraphArtifact, load_feature_graph_a
|
||||
from ..artifacts.part_definition import PartDefinition
|
||||
from ..product.packages import (
|
||||
ProductPackage,
|
||||
build_product_package,
|
||||
read_product_package,
|
||||
validate_product_package,
|
||||
)
|
||||
@@ -18,6 +19,8 @@ from simplecadapi._cpp import OperationGraph
|
||||
|
||||
Definition = PartDefinition | AssemblyDefinition
|
||||
ProductPackageInput = ProductPackage | bytes | bytearray | memoryview | str | Path
|
||||
# What an exporter takes besides a package: the result of a build (``@scad.part`` / ``@scad.assemble``) or its definition.
|
||||
ProductInput = ProductPackageInput | Definition | Any
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
@@ -70,15 +73,22 @@ class ProductPackageTranslationUnit:
|
||||
|
||||
|
||||
def read_product_package_translation_units(
|
||||
data: ProductPackageInput,
|
||||
data: ProductInput,
|
||||
) -> tuple[ProductPackage, tuple[ProductPackageTranslationUnit, ...]]:
|
||||
"""Return a validated package and dependency-first definition units."""
|
||||
"""Return a validated package and dependency-first definition units.
|
||||
|
||||
*data* is a package, the bytes or path of one, or (issue #89) the result of a build or a definition: a script that
|
||||
only wants a STEP file need not write a ``.scadpkg`` first. That package is built in memory without the scene
|
||||
projection (tessellated meshes), which no exporter reads and which took most of the time of a default capture.
|
||||
"""
|
||||
|
||||
if isinstance(data, ProductPackage):
|
||||
validate_product_package(data)
|
||||
package = data
|
||||
else:
|
||||
elif isinstance(data, (bytes, bytearray, memoryview, str, Path)):
|
||||
package = read_product_package(data)
|
||||
else:
|
||||
package = build_product_package(data, include_scene=False)
|
||||
ordered: list[Definition] = []
|
||||
visited: set[tuple[str, str]] = set()
|
||||
|
||||
@@ -122,5 +132,6 @@ def read_product_package_translation_units(
|
||||
__all__ = [
|
||||
"ProductPackageInput",
|
||||
"ProductPackageTranslationUnit",
|
||||
"ProductInput",
|
||||
"read_product_package_translation_units",
|
||||
]
|
||||
|
||||
@@ -13,6 +13,7 @@ import importlib
|
||||
import importlib.metadata
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import pickle
|
||||
import re
|
||||
import sys
|
||||
@@ -326,6 +327,33 @@ class TestRunNotebook(NotebookTestCase):
|
||||
self.assertFalse((state_dir(run.config) / "cache").exists())
|
||||
self.assertEqual(self.statuses(run_notebook(self.bracket)), ["ran"] * 4)
|
||||
|
||||
def test_without_the_cache_the_source_directory_is_left_alone(self) -> None:
|
||||
"""Issue #89: a run that keeps no state must not create the state directory (``__marimo__``) next to the source,
|
||||
so a notebook can run where the source directory is read-only (a container's ``-v src:/src:ro``)."""
|
||||
|
||||
run_notebook(self.bracket, cache=False)
|
||||
self.assertFalse((self.root / "__marimo__").exists())
|
||||
self.assertFalse(state_dir(read_notebook_config(self.bracket)).exists())
|
||||
|
||||
@unittest.skipUnless(
|
||||
os.name == "posix" and hasattr(os, "geteuid") and os.geteuid() != 0, "needs a non-root POSIX user to make a directory read-only"
|
||||
)
|
||||
def test_without_the_cache_a_read_only_source_directory_runs(self) -> None:
|
||||
directories = [self.root, self.root / "lib"]
|
||||
modes = [directory.stat().st_mode for directory in directories]
|
||||
for directory in directories:
|
||||
directory.chmod(0o555)
|
||||
try:
|
||||
run = run_notebook(self.bracket, cache=False)
|
||||
finally:
|
||||
for directory, mode in zip(directories, modes):
|
||||
directory.chmod(mode)
|
||||
self.assertEqual(run.definition.definition_id, "bracket")
|
||||
|
||||
def test_with_the_cache_the_dependency_record_is_still_written(self) -> None:
|
||||
run = run_notebook(self.bracket)
|
||||
self.assertTrue((state_dir(run.config) / "deps.json").is_file())
|
||||
|
||||
def test_overrides_skip_the_defining_cell(self) -> None:
|
||||
plain = run_notebook(self.bracket)
|
||||
wide = run_notebook(self.bracket, overrides={"width": 30.0})
|
||||
|
||||
@@ -147,7 +147,7 @@ def _build_nested_package():
|
||||
return scad.build_product_package(build_root())
|
||||
|
||||
|
||||
def _build_cylinder_package():
|
||||
def _build_cylinder_result():
|
||||
@scad.part(
|
||||
id="cylinder",
|
||||
project_root=Path(__file__).parent,
|
||||
@@ -156,7 +156,11 @@ def _build_cylinder_package():
|
||||
body = scad.make_cylinder_rsolid(radius=4.0, height=8.0)
|
||||
return scad.make_part_rpart("cylinder", body)
|
||||
|
||||
return scad.build_product_package(build_cylinder())
|
||||
return build_cylinder()
|
||||
|
||||
|
||||
def _build_cylinder_package():
|
||||
return scad.build_product_package(_build_cylinder_result())
|
||||
|
||||
|
||||
class TestProductExporter(unittest.TestCase):
|
||||
@@ -182,6 +186,39 @@ class TestProductExporter(unittest.TestCase):
|
||||
self.assertFalse(hasattr(scad, "export_obj"))
|
||||
self.assertFalse(hasattr(scad.translator, "ap242_translator"))
|
||||
|
||||
def test_a_build_result_exports_in_one_step_without_a_scene(self):
|
||||
"""Issue #89: a script that only wants a file need not capture a ``.scadpkg`` first, nor pay for the scene
|
||||
(the tessellated projection no exporter reads)."""
|
||||
|
||||
from unittest import mock
|
||||
|
||||
from simplecadapi.product import packages
|
||||
|
||||
result = _build_cylinder_result()
|
||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||
root = Path(tmp_dir)
|
||||
with mock.patch.object(packages, "_scene_projection", side_effect=AssertionError("the scene was built")):
|
||||
step = scad.exporter.export_product_package_to_step(result, root / "one.step")
|
||||
stl = scad.exporter.export_product_package_to_stl(result, root / "one.stl")
|
||||
self.assertEqual(step.definition_ids, ("cylinder",))
|
||||
self.assertGreater((root / "one.step").stat().st_size, 0)
|
||||
self.assertGreater((root / "one.stl").stat().st_size, 0)
|
||||
self.assertEqual(stl.output_path, (root / "one.stl").resolve())
|
||||
|
||||
def test_a_definition_exports_like_its_build_result(self):
|
||||
result = _build_cylinder_result()
|
||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||
root = Path(tmp_dir)
|
||||
from_result = scad.exporter.export_product_package_to_step(result, root / "a.step")
|
||||
from_definition = scad.exporter.export_product_package_to_step(result.definition, root / "b.step")
|
||||
self.assertEqual(from_result.definition_ids, from_definition.definition_ids)
|
||||
self.assertEqual(from_result.metadata_item_count, from_definition.metadata_item_count)
|
||||
|
||||
def test_a_value_that_is_no_product_is_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||
with self.assertRaises(TypeError):
|
||||
scad.exporter.export_product_package_to_step(object(), Path(tmp_dir) / "x.step")
|
||||
|
||||
def test_nested_package_preserves_ap242_product_structure_and_placements(self):
|
||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||
root = Path(tmp_dir)
|
||||
|
||||
Reference in New Issue
Block a user