feat(refs): persistence across the cell cache, bodies, naming version; design summary and plan

- A test restores a cell's lineage record in a fresh process and resolves a tag and a stable reference on the
  unpickled value through the restored deltas (tags and references survive the cell cache).
- References report the body (output slot / solid) of each target; element_ref.resolve reads the slot from the
  shape, so each body of a split result resolves to its own piece. A cut that leaves several bodies is not a
  reference error.
- kNamingVersion (scad-naming/1) is stamped into stable anchors; a reference made under another version still
  resolves and says version_changed.
- graph.hpp: from_value restores the deltas (stale comment fixed, SDK_DEFECTS 22).
- Design: summary, versioning and plan updated to what is done and what is next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
NiJingzhe
2026-11-04 05:34:28 +08:00
parent facccaaef5
commit 45d441ca01
9 changed files with 178 additions and 64 deletions
+52 -55
View File
@@ -10,28 +10,25 @@ a probe run in this session, no committed test. **[I]** = inference / proposal,
## 0. Summary
1. **A reference is a query over recorded history, not a stored entity.** `ElementRef = {anchor, follow, pick, guard,
last}`: the anchor names where the entity was *born* (a node that is stable across re-execution, plus a
deterministic entity key: kernel output role, source sub-shape, or tag); resolution walks the `TopoDelta` every
later node recorded from OCCT history to the entities of the shape being built. This is Onshape's model
(`qCreatedBy` + `startTracking`/`makeRobustQuery`, `OK`/`SPLIT`/`FAILED_TO_RESOLVE`) on top of the history
FreeCAD 1.0 uses (Modified / Generated / IsDeleted), with typed records instead of an encoded name string.
2. **Names are generated in the C++ kernel, per operation, from OCCT history.** That data exists today: every tracked
operation writes witnesses `{ref, event, parent_refs, origin_role, derivation}` into the node's `TopoDelta`
**[R]** (`src/cpp/src/semantic/tracking_*.cpp`, `src/cpp/src/reactive/session.cpp` `resolve_recorded_delta`). The
resolver is C++ too (`scad::reactive::resolve_by_history`, `src/cpp/src/reactive/element_ref.cpp`); Python only
stores and edits reference records.
3. **Outcome is always one of `ok` / `split` / `merged` / `lost`**, with the entities, the hops that got there, and for
`lost` the node and the reason (`deleted`, `replaced`, `no_history`, `not_downstream`). Geometry is a *guard* and a
source of ranked candidates for the editor, never a silent fallback ("first, do no harm").
4. **Tags become a view over references.** A tag is a named `ElementRef`; `ql.tag("x")` resolves the tag's anchor
through history to the queried shape. The per-op tag projection (with its face-source gate and face-channel gap)
stops being the long-range identity mechanism.
5. The spike proves the resolver on the four known failures on the real kernel **[T]**
(`test/test_element_ref_history.py`) and on hand-built graphs for every status **[T]**
(`src/cpp/tests/cpp/reactive/test_element_ref.cpp`). The audit (section 6) shows where kernel history is
missing today: `transform_rsolid`, `offset_rsolid`, draft, scale, helical sweep record no delta, and a shell's
kept faces have no successor witness.
1. **A reference is a query over recorded history, not a stored entity.** The anchor names where the entity was
*born* (a stable node key: feature, op, occurrence among ancestors, plus a kernel role or topo id); resolution
walks the `TopoDelta` every later node recorded from OCCT history to the entities of the shape being built.
Onshape's model (`qCreatedBy` + `startTracking`, `OK`/`SPLIT`/`FAILED_TO_RESOLVE`) on FreeCAD 1.0's history
(Modified / Generated / IsDeleted), with typed records instead of an encoded name string. **[T]**
2. **Names are generated in the C++ kernel, per operation, from OCCT history** (`semantic/tracking_*.cpp`); the
resolver, the stable keys, the backward trace and the tag view are C++ too (`reactive/element_ref.cpp`,
`reactive/tag_view.cpp`). Python only stores references (`simplecadapi.topology.element_ref`). **[T]**
3. **Outcome is always `ok` / `split` / `merged` / `lost`**, with entities, hops, bodies, and for `lost` the node and
reason. A split is an error carrying the candidates (owner decision); a lost reference offers guard-ranked
look-alikes; nothing is chosen silently. **[T]**
4. **Tags are a view over the recorded history** (owner decision, switched directly): a tag assignment is a graph
node, and `ql.tag` sees every entity its targets became. Plain and op-kwarg face tags now survive every operation
family with history. **[T]**
5. **Every solid modelling op records history now:** `transform_rsolid`, `scale_rsolid`, `offset_rsolid` and
`draft_rsolid` gained it; a shell's kept faces are preserved (option a); N-ary booleans compose their steps.
**[T]**
6. Persistence: tags and references resolve on values restored from the cell cache (nodes and deltas travel in the
lineage records) and after pickling **[T]**; references carry a naming version **[T]**.
---
@@ -285,19 +282,22 @@ modelling fact. `anchor_missing` / `anchor_ambiguous` mean the anchored node its
- **Cell cache.** `runtime/lineage.py` saves each cell's nodes with `graph.to_dict()`, which includes `topo_delta`,
and `OperationGraph.from_value` restores it **[R]** (`graph.cpp` `from_value` parses `topo_delta`; the header
comment "topology/semantic deltas are not restored" is stale, see SDK_DEFECTS 22). So a reference resolves through
restored cells as long as the value carries its node object. Pinned by a test in stage 1 [I].
restored cells as long as the value carries its node object **[T]** (`test_tags_and_references_resolve_on_values_restored_from_the_cell_cache`, fresh process).
- **Model JSON / packages.** `recording/serializer.py` writes deltas per node already (`"delta"`), so the resolver
can run on a deserialised graph; refs are data in the feature params. [R for deltas, I for refs]
---
## 5. Versioning [I]
## 5. Versioning (implemented in the spike) [T]
`naming = "scad-naming/<n>"` changes whenever a tracker's witnesses or the resolver's rules change (e.g. the shell fix
in 7.2), plus the OCCT version (history changes between OCCT releases; FreeCAD versions its element map the same
way). On mismatch: resolve anyway, compare against `guard` and `last`; if they agree, rewrite the version silently;
if not, report `lost(version_changed)` with candidates. Refs never need a migration step of their own because they
store anchors and policies, not resolved names.
`kNamingVersion = "scad-naming/1"` (`element_ref.hpp`, `element_ref.NAMING_VERSION`) is stamped into every stable
anchor. It changes whenever a tracker's witnesses or the resolver's rules change (the shell fix of 7.2 would have
been a bump had references been persisted before it), and should be combined with the OCCT version when OCCT is
upgraded (FreeCAD versions its element map the same way). A reference made under another version is still resolved,
and the result says `version_changed`; the caller (editor) compares it against its guard: agreement rewrites the
version silently, disagreement shows the reference as `lost(version_changed)` with candidates [I]. References need no
migration of their own because they store anchors and policies, not resolved names. Tags need none either: they are
recomputed from the graph on every read.
## 6. Audit: tag and history propagation per family (before -> now)
@@ -331,32 +331,29 @@ by kwarg (`kw.top`) and by `apply_tag_rselection` (`user.top`). "Before" is `dev
## 7. Staged plan
### Stage 0/1 (this branch, done)
- `scad::reactive::resolve_by_history`, `anchors_for_role`, `subshapes_by_recorded_id` (C++), private bindings
`_native._resolve_by_history` / `_anchors_for_role` / `_subshapes_by_recorded_id`, Python wrapper
`simplecadapi.topology.element_ref` (private, not exported). Doctest: 6 cases over hand-built graphs. Pytest:
known failures, statuses, parameter sweep, creator caps, audit (30 tests, `test/test_element_ref_history.py`).
### Done on this branch
- Resolver (`resolve_by_history`), role anchors, recorded-id lookup; statuses ok / split / merged / lost.
- History for transform / scale / offset / draft (`modification_delta`); shell kept faces preserved (7.2); N-ary
boolean steps composed (`merge_topo_deltas`).
- Stable anchors (feature, op, occurrence-among-ancestors, params tie-break, role) and the backward trace to birth.
- Splits as errors with candidates; lost references with guard-ranked look-alikes; bodies; naming version.
- Tags as a view over history (3.1), effective includes lineage, one path per binding.
- Tests: `test/test_element_ref_history.py` (48), `test_element_ref.cpp`, `test_tag_view.cpp`,
`test_modification_delta.cpp`; goldens reviewed one by one (commits `df38241`, `facccaa`).
### Stage 1: names and pick-time trace (next)
1. Backward trace (2.4) in C++: `name_of(node, entity) -> ElementRef` with deterministic sibling ranks computed in
the tracker (add a `piece` rank to witnesses that share a parent, sorted by a kernel geometric key).
2. Stable node identity (2.2): `(cell, op, occurrence)` <-> node id map kept by the session; anchor check of `op`.
3. Cache test: a reference resolved through a cell restored from the cache equals the live one.
4. Kernel gaps, in order of user impact: `transform_rsolid` (GTransform has `Modified`), shell kept faces (see
7.2), offset, draft, scale, helical sweep. Each with a doctest and an audit row flipped.
### Stage 2: SDK and editor
5. Public `scad.ref` API (`faces`/`edges`, `ElementRefLost` with candidates), docstrings/modules.json/serializer
row/API page per the SDK checklist.
6. Editor: `selection_recipe` emits refs; reference health badges (4.2); "use all / this piece / candidate" actions
writing the ref back; ws message for statuses.
7. Sketch roots: sketch geometry ids as anchors for profile edges, so side faces are named by sketch entities.
### Stage 3: tags as a view
8. Tag bindings hold refs; `ql.tag` resolves lazily with memoisation; projection kept as a cache and compared on the
corpus; then retire the gates.
9. Corpus: parameter sweeps of the examples (insert a feature mid-cell, change widths, add an upstream fillet),
asserting zero wrong picks and the resolved-by-history rate.
### Next
1. Public `scad.ref` (`of`, `resolve`, `one`, `all`) and `ql.ref(ref)` as a predicate, with the SDK checklist
(docstrings, modules.json, serializer row, API page); `ElementRefError` exported.
2. Editor (`feat/gui-modeling`): `selection_recipe` emits `scad.ref.faces([...])` instead of tag-or-window code;
health badges and "this piece / this candidate" actions write the reference back (4.2).
3. Sketch roots: sketch geometry ids as anchors for profile edges, so side faces have names that survive profile
edits; a kernel-computed geometric rank for pieces of a split (2.4).
4. Per-node parent index for the resolver (7.1); `Query.explain(shape)` as data (the cardinality errors already
carry the per-predicate counts and near-miss inventory).
5. Multi-body (`research/multi-solid`): `ql.body(name)` once named bodies land; references already resolve per
output slot and report the body index inside a compound.
6. Corpus: parameter sweeps of the examples (insert a feature mid-cell, change widths, add an upstream fillet),
asserting zero wrong picks.
### 7.1 Performance budget
Measured **[R]** (ad-hoc probe, Release build, Apple silicon; one top face of a 400x20x10 plate through N
@@ -369,7 +366,7 @@ successive hole cuts; the resolved face area matched 8000 - N*pi/4 at every N):
| 150 | 4.75 s | 3.0 ms | 3.1 ms |
The cost is the scan of each consuming node's delta, whose size grows with the model (O(N) entries per cut, so
O(N^2) overall). Stage 1: a per-node index parent -> entries, built once per node and memoised, makes a resolve
O(N^2) overall). Next: a per-node index parent -> entries, built once per node and memoised, makes a resolve
O(hops). Budget [I]: < 1 ms per reference for 200-node histories, memoised per (ref, target node), nothing added at
recording time (the deltas are recorded already). Resolution is already negligible next to modelling (0.06 % of
build time at N = 150).
+4 -5
View File
@@ -339,13 +339,12 @@ that surfaced it is in brackets.
assignment reaches every entity its targets became, per lineage policy, and effective includes lineage.
Audit: `test_element_ref_history.py::test_audit_tag_channels_and_history_per_family` (every family 1/1).
## 22. `OperationGraph::from_value` header says deltas are not restored; they are [stable-refs research]
## 22. `OperationGraph::from_value` header said deltas are not restored; they are [stable-refs research]
- **Symptom:** `include/scad/reactive/graph.hpp` comments "topology/semantic deltas are not restored" while
- **Symptom:** `include/scad/reactive/graph.hpp` commented "topology/semantic deltas are not restored" while
`graph.cpp` `from_value` parses `topo_delta` and `semantic_delta`.
- **Trigger:** reading the header to decide whether history survives the cell cache (`runtime/lineage.py`).
- **Status:** open (comment only).
- **Fix direction:** correct the comment; pin restore of deltas with a test, since stable references rely on it.
- **Status:** fixed (comment), and pinned: `test_tags_and_references_resolve_on_values_restored_from_the_cell_cache`
restores a cell's lineage record in a fresh process and resolves a tag and a reference through its deltas.
## 23. A shell's kept faces had no successor witness [stable-refs research]
@@ -43,6 +43,7 @@ py::dict anchor_to(const sr::ElementAnchor& anchor) {
} // namespace
void bind_element_ref(py::module_& m) {
m.attr("_naming_version") = std::string(sr::kNamingVersion);
m.def(
"_resolve_by_history",
[](const sr::NodePtr& target, const py::dict& anchor, int target_slot, const std::string& follow) {
@@ -21,6 +21,11 @@
namespace scad::reactive {
// The version of the naming rules (trackers' witnesses + resolver + stable keys). Stored in every stable anchor;
// a reference made under another version is still resolved, and the caller checks it against its guard. Bump it
// whenever a tracker changes which witnesses it emits or the resolver changes how it follows them.
inline constexpr std::string_view kNamingVersion = "scad-naming/1";
enum class RefStatus { Ok, Split, Merged, Lost };
std::string_view to_string(RefStatus status) noexcept;
@@ -102,6 +107,7 @@ struct StableAnchor {
std::string role; // empty: by topo id
int role_ordinal = 0; // among the node's entities with that role, in recorded order
std::string topo_id; // the recorded topo id when it was made (a hint; the only name when `role` is empty)
std::string naming{kNamingVersion}; // the naming version it was made under
};
// The feature (namespace) part of a node id: "cellA_node_0000002a" -> "cellA", "node_00000001" -> "".
+2 -1
View File
@@ -152,7 +152,8 @@ class OperationGraph {
// ---- serialization (OperationGraph.to_dict / from_dict) --------------------
Value to_value() const;
std::string to_json(int indent = 2) const;
// Restores nodes and edges; topology/semantic deltas are not restored.
// Restores nodes and edges, with their topology and semantic deltas (what stable references and the tag view
// read on a restored cell).
// `expressions` / `frames`, when given, are what the nodes' expression and frame records are
// looked up in (a restored node carries the expressions its parameters refer to, and its frame).
static OperationGraph from_value(const Value& payload, bool strict = true,
+2
View File
@@ -565,6 +565,7 @@ Value to_value(const StableAnchor& anchor) {
out.set("role", anchor.role.empty() ? Value() : Value(anchor.role));
out.set("role_ordinal", static_cast<std::int64_t>(anchor.role_ordinal));
out.set("topo_id", anchor.topo_id);
out.set("naming", anchor.naming);
return out;
}
@@ -596,6 +597,7 @@ StableAnchor stable_anchor_from_value(const Value& value) {
out.role = text("role");
out.role_ordinal = number("role_ordinal");
out.topo_id = text("topo_id");
out.naming = text("naming"); // empty: made before versions were stamped
return out;
}
@@ -941,3 +941,4 @@ SQUARE_M: simplecadapi.params.units.Unit # value = Unit(symbol='m^2', dimension
SQUARE_MM: simplecadapi.params.units.Unit # value = Unit(symbol='mm^2', dimension=Dimension(length=2, angle=0), scale_to_canonical=1.0)
VOLUME: simplecadapi.params.units.Dimension # value = Dimension(length=3, angle=0)
WORLD_CS: simplecadapi.core.CoordinateSystem # value = CoordinateSystem(origin=(0, 0, 0), x_axis=(1, 0, 0), y_axis=(0, 1, 0))
_naming_version: str = 'scad-naming/1'
+39 -3
View File
@@ -32,7 +32,10 @@ from typing import Any, Dict, List, Optional, Sequence
from .._cpp import _native
NAMING_VERSION: str = _native._naming_version
__all__ = [
"NAMING_VERSION",
"Candidate",
"ElementRefError",
"ResolvedRef",
@@ -77,6 +80,10 @@ class ResolvedRef:
lost_at: Optional[str] = None
reason: Optional[str] = None
candidates: List[Candidate] = field(default_factory=list)
# Which body (solid of the target shape, in its solid order) each of `entities` lies in.
bodies: List[Optional[int]] = field(default_factory=list)
# The reference was made under another naming version (NAMING_VERSION): check it against its guard.
version_changed: bool = False
def describe(self) -> Dict[str, Any]:
"""The editor-facing form: status, where it was lost and why, and the candidates to choose from."""
@@ -85,6 +92,8 @@ class ResolvedRef:
"status": self.status,
"lost_at": self.lost_at,
"reason": self.reason,
"bodies": list(self.bodies),
"version_changed": self.version_changed,
"candidates": [c.describe() for c in self.candidates],
}
@@ -195,17 +204,41 @@ def _ranked_lookalikes(shape: Any, guard: Dict[str, Any], kind: str, limit: int
return out[:limit]
def _bodies_of(shape: Any, entities: List[Any]) -> List[Optional[int]]:
from .. import ql
if not entities:
return []
solids = ql.solids().resolve(shape) if type(shape).__name__ != "Solid" else [shape]
out: List[Optional[int]] = []
for entity in entities:
kind = type(entity).__name__
found = None
for index, solid in enumerate(solids):
members = ql.faces().resolve(solid) if kind == "Face" else ql.edges().resolve(solid)
if any(member == entity for member in members):
found = index
break
out.append(found)
return out
def resolve(
anchor: Dict[str, Any],
shape: Any,
*,
follow: str = "identity",
output_slot: int = 0,
output_slot: Optional[int] = None,
guard: Optional[Dict[str, Any]] = None,
) -> ResolvedRef:
"""Follow *anchor* (raw or stable) through recorded history to the entities of *shape* (a later output)."""
"""Follow *anchor* (raw or stable) through recorded history to the entities of *shape* (a later output).
*shape* names its node and output slot itself (a body of a multi-body result is its own slot).
"""
node = _node_of(shape)
if output_slot is None:
output_slot = int((shape.get_metadata("graph", None) or {}).get("output_slot", 0))
raw = anchor
if "op" in anchor: # a stable anchor: find its node in this run first
raw = _native._locate_anchor(node, anchor)
@@ -214,7 +247,8 @@ def resolve(
ambiguous = len(_native._stable_candidates(node, anchor)) > 1
lookalikes = _ranked_lookalikes(shape, guard, str(anchor["kind"]).lower()) if guard else []
reason = "anchor_ambiguous" if ambiguous else "anchor_missing"
return ResolvedRef("lost", [], [], [], None, reason, lookalikes)
changed = anchor.get("naming", "") != NAMING_VERSION
return ResolvedRef("lost", [], [], [], None, reason, lookalikes, [], changed)
data = _native._resolve_by_history(node, raw, output_slot, follow)
targets = list(data["targets"])
entities: List[Any] = []
@@ -242,6 +276,8 @@ def resolve(
lost_at=data.get("lost_at"),
reason=data.get("reason"),
candidates=candidates,
bodies=_bodies_of(shape, entities),
version_changed="op" in anchor and anchor.get("naming", "") != NAMING_VERSION,
)
+71
View File
@@ -484,3 +484,74 @@ def test_a_tag_survives_pickling_with_its_history():
assert face.get_area() == pytest.approx(800 - 9 * math.pi)
explained = scad.explain_tag(face, "user.top")
assert explained and explained[0]["lineage"][0]["evidence"]["via"] == "recorded_history"
def test_tags_and_references_resolve_on_values_restored_from_the_cell_cache(tmp_path):
# The cell cache writes a cell's nodes beside its entry (runtime/lineage.py) and pickles its values; a fresh
# process registers the nodes, then unpickles the values. The tag view and the resolver read those nodes.
import json
import subprocess
import sys
import textwrap
root = str(tmp_path)
build = textwrap.dedent(
f"""
import json, pickle, math
from pathlib import Path
import simplecadapi as scad
from simplecadapi import ql
from simplecadapi._glue.session import GraphSession
from simplecadapi.runtime import lineage
from simplecadapi.topology import element_ref as er
with GraphSession(graph_id="g1", namespace="cellA", shared_lineage=True) as session:
box = scad.make_box_rsolid(40, 20, 10)
top = ql.faces().where(ql.prop("geom.center.z", ">=", 9.99)).exactly(1).resolve(box)
box = scad.apply_tag_rselection(box, top, "user.top")
out = scad.cut_rsolid(box, scad.make_cylinder_rsolid(3, 30))
anchor = er.stable_anchor_of(top[0], out)
lineage.save(Path({root!r}), "cellA", session)
Path({root!r}, "value.pkl").write_bytes(pickle.dumps(out))
Path({root!r}, "anchor.json").write_text(json.dumps(anchor))
"""
)
restore = textwrap.dedent(
f"""
import json, pickle
from pathlib import Path
from simplecadapi import ql
from simplecadapi.runtime import lineage
from simplecadapi.topology import element_ref as er
assert lineage.load(Path({root!r}), "cellA", "g1")
out = pickle.loads(Path({root!r}, "value.pkl").read_bytes())
tagged = ql.faces().where(ql.tag("user.top")).resolve(out)
r = er.resolve(json.loads(Path({root!r}, "anchor.json").read_text()), out)
print(json.dumps({{"tagged": [f.get_area() for f in tagged], "status": r.status,
"areas": [f.get_area() for f in r.entities]}}))
"""
)
for code in (build, restore):
done = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, timeout=300)
assert done.returncode == 0, done.stderr[-1500:]
result = json.loads(done.stdout.strip().splitlines()[-1])
assert result["status"] == "ok"
assert result["tagged"] == pytest.approx([800 - 9 * math.pi])
assert result["areas"] == pytest.approx([800 - 9 * math.pi])
def test_a_reference_says_which_body_and_which_naming_version():
with scad.GraphSession():
box, anchor = _box_with_top_anchor()
stable = er.stable_anchor_of(_faces_at(box, 10)[0], box)
assert stable["naming"] == er.NAMING_VERSION
# A cut that leaves two bodies is allowed (not a reference error): each body is its own output slot, and
# the reference resolves in each body to the piece of the top face that body has.
pieces = scad.cut_rsolid_list(box, scad.make_box_rsolid(4, 30, 30, (0, 0, -10)))
assert len(pieces) == 2
halves = [er.resolve(anchor, piece) for piece in pieces]
assert [r.status for r in halves] == ["ok", "ok"]
assert sorted(round(r.entities[0].get_center().x, 6) for r in halves) == [-11.0, 11.0]
assert all(r.bodies == [0] for r in halves)
# A reference made under another naming version still resolves, and says so (the caller checks its guard).
r = er.resolve(dict(stable, naming="scad-naming/0"), box)
assert r.status == "ok" and r.version_changed and r.bodies == [0]