Model cache: each sheet body is written once, not with every state (#6152)

* Test: a saved model keeps each sheet body once

A sheet-metal design wrote its sheet bodies again with every later feature's state in the saved
model: each feature that leaves the sheet alone (a box beside it) adds more than the whole sheet
to the records. A design of many features after its sheet metal passed the cache's 32 MiB
manifest limit that way and kept only some of its states (#3220). The test also opens the saved
file and wants the sheets taken from it as evaluated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsiqyXWBFGTCX9J51bhysc

* Model cache: each sheet body is written once, not with every state

Problem: a saved model writes every feature's state, and each state wrote its sheet bodies in
full; most features leave the sheets as they were, so a sheet-metal design of many features
repeated them over and over (a 280-feature part: 30.5 MB of manifest, 29.6 MB of it 69 distinct
sheet records written 243 times). Past the manifest's 32 MiB limit the save thinned the cache
(#3220) and opening evaluated the left-out features again, although the distinct data was small.

Solution: the records keep a list of sheet bodies, each written once; a state names its sheets by
their place in it, as it already does for bodies and sketches. A thinned model keeps the whole
list and counts it once against its budget; the reader counts sheets with the bodies and sketches
a state holds; the cache format is now /4 (an older cache is a miss).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsiqyXWBFGTCX9J51bhysc

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
NOWN-DEV
2026-10-25 01:24:57 +00:00
committed by GitHub
parent 67adf5abb5
commit b88673f366
5 changed files with 85 additions and 11 deletions
+35 -6
View File
@@ -22,7 +22,7 @@ pub struct SavedModel {
/// Records refer to these at most.
const MAX_ITEMS: usize = 1_000_000;
/// Bodies and sketches all the states hold together, at most (a state is a list of clones).
/// Bodies, sketches and sheet bodies all the states hold together, at most (a state is a list of clones).
const MAX_HELD: usize = 50_000_000;
/// A feature's boolean retries as saved. The kernel leaves `rejected` out when it is empty, and
@@ -118,6 +118,10 @@ struct Writer {
names_index: HashMap<usize, usize>,
sketches: Vec<Json>,
sketch_index: HashMap<usize, usize>,
/// Sheet bodies by their record: most features leave the sheets as they were, so the states
/// share them (writing them with every state made a sheet-metal design's model too large to keep).
sheets: Vec<Json>,
sheet_index: HashMap<String, usize>,
states: Vec<Json>,
state_index: HashMap<usize, usize>,
}
@@ -184,6 +188,17 @@ impl Writer {
Some(self.sketches.len() - 1)
}
fn sheet(&mut self, sheet: &crate::sheet::SheetBody) -> Option<usize> {
let record = serde_json::to_value(sheet).ok()?;
let key = record.to_string();
if let Some(i) = self.sheet_index.get(&key) {
return Some(*i);
}
self.sheets.push(record);
self.sheet_index.insert(key, self.sheets.len() - 1);
Some(self.sheets.len() - 1)
}
fn state(&mut self, st: &Arc<ModelState>) -> Option<usize> {
let id = Arc::as_ptr(st) as usize;
if let Some(i) = self.state_index.get(&id) {
@@ -191,10 +206,11 @@ impl Writer {
}
let bodies = st.bodies.iter().map(|b| self.model_body(b)).collect::<Option<Vec<_>>>()?;
let sketches = st.sketches.iter().map(|s| self.sketch(s)).collect::<Option<Vec<_>>>()?;
let sheets = st.sheets.iter().map(|s| self.sheet(s)).collect::<Option<Vec<_>>>()?;
let r = json!({
"bodies": runs(&bodies), "sketches": runs(&sketches), "body_counter": st.body_counter,
"reserved_body_names": st.reserved_body_names.iter().collect::<Vec<_>>(),
"sheets": serde_json::to_value(&st.sheets).ok()?, "construct": serde_json::to_value(&st.construct).ok()?,
"sheets": sheets, "construct": serde_json::to_value(&st.construct).ok()?,
"threads": serde_json::to_value(&st.threads).ok()?,
"sketch_keys": st.sketch_keys.iter().map(|(f, k)| json!([f, hex(*k)])).collect::<Vec<_>>(),
"blend_edges": serde_json::to_value(&st.blend_edges).ok()?, "fitted": serde_json::to_value(&st.fitted).ok()?,
@@ -230,6 +246,7 @@ impl Model {
}
let records = json!({
"features": features, "states": w.states, "model_bodies": w.model_bodies, "names": w.names, "sketches": w.sketches,
"sheets": w.sheets,
});
Ok(SavedModel { records, bodies: w.bodies })
})
@@ -306,6 +323,10 @@ impl SavedModel {
}
model_bodies.push(b);
}
let mut sheets = Vec::new();
for s in list("sheets")? {
sheets.push(serde_json::from_value::<crate::sheet::SheetBody>(s.clone()).map_err(|_| bad("saved model: sheet"))?);
}
let mut states = Vec::new();
let mut held = 0usize;
for s in list("states")? {
@@ -339,7 +360,13 @@ impl SavedModel {
.and_then(Json::as_array)
.map(|a| a.iter().filter_map(|n| n.as_str().map(str::to_string)).collect())
.unwrap_or_default(),
sheets: serde_json::from_value(field("sheets")?).map_err(invalid)?,
sheets: s
.get("sheets")
.and_then(Json::as_array)
.ok_or_else(|| bad("saved model: state"))?
.iter()
.map(|i| index(Some(i), sheets.len()).and_then(|i| sheets.get(i).cloned().ok_or_else(|| bad("saved model: sheet"))))
.collect::<Result<Vec<_>>>()?,
construct: serde_json::from_value(field("construct")?).map_err(invalid)?,
threads: serde_json::from_value(field("threads")?).map_err(invalid)?,
sketch_keys,
@@ -347,7 +374,7 @@ impl SavedModel {
fitted: serde_json::from_value(field("fitted")?).map_err(invalid)?,
tapered: serde_json::from_value(field("tapered")?).map_err(invalid)?,
};
held = held.saturating_add(st.bodies.len()).saturating_add(st.sketches.len());
held = held.saturating_add(st.bodies.len()).saturating_add(st.sketches.len()).saturating_add(st.sheets.len());
if held > MAX_HELD {
return Err(bad("saved model: states"));
}
@@ -424,7 +451,9 @@ impl SavedModel {
let mut kept_bodies = vec![false; model_bodies.len()];
let (mut names_used, mut sketches_used) = (vec![false; names.len()], vec![false; sketches.len()]);
let mut stored = std::collections::HashSet::new();
let mut used = 0usize;
// The sheet bodies, each written once and named by place in the states: all kept.
let sheets = list("sheets").cloned().unwrap_or_default();
let mut used = bytes(sheets.as_slice()).iter().fold(0usize, |t, b| t.saturating_add(*b));
for s in std::iter::once(last).chain(order) {
let (mut adds, mut new_names, mut new_sketches) = (HashMap::new(), std::collections::HashSet::new(), std::collections::HashSet::new());
let mut records = *state_bytes.get(s)?;
@@ -516,7 +545,7 @@ impl SavedModel {
}
let records = json!({
"features": out_features, "states": out_states, "model_bodies": out_bodies, "names": kept_of(names.as_slice(), &names_used),
"sketches": kept_of(sketches.as_slice(), &sketches_used),
"sketches": kept_of(sketches.as_slice(), &sketches_used), "sheets": sheets,
});
let kept: Vec<usize> = (0..self.bodies.len()).filter(|i| kernel_used.get(*i) == Some(&true)).collect();
let bodies = kept.iter().filter_map(|i| self.bodies.get(*i).cloned()).collect();
+43
View File
@@ -923,6 +923,49 @@ mod tests {
std::fs::remove_file(path).unwrap();
}
/// A sheet-metal design keeps each sheet body once in its saved model, not again with every
/// later feature's state: a design of many features after its sheet metal wrote the sheets
/// over and over, and past the cache's size limit kept only some of its states.
#[test]
fn a_saved_model_keeps_each_sheet_body_once() {
let design = |boxes: usize| {
let mut s = Session::new(Document::new("Sheet"));
let run = |s: &mut Session, id: &str, p: Value| s.execute(id, &p).unwrap();
run(
&mut s,
"sheet.manage_rules",
json!({"name":"AL2","thickness":"2 mm","k_factor":0.5,"bend_radius":"1 mm","gap":"0.5 mm","active":true}),
);
run(&mut s, "sketch.create", json!({"plane":"XY","name":"Base"}));
run(&mut s, "sketch.rectangle.two_point", json!({"p0":[0,0],"p1":[200,150]}));
run(&mut s, "sketch.finish", json!({}));
run(&mut s, "sheet.flange", json!({"sketch":"Base"}));
run(&mut s, "sheet.flange", json!({"edges":[[100,0,2],[200,75,2],[100,150,2],[0,75,2]],"height":"20 mm","angle":"90 deg"}));
// A panel's holes: each one is a cut-out the sheet keeps.
for i in 0..16 {
run(&mut s, "solid.hole", json!({"position":[25 + 10 * i,40,2],"diameter":5}));
}
for i in 0..boxes {
run(&mut s, "solid.box", json!({"length":5,"width":5,"height":5,"corner":[300 + 10 * i,0,0],"operation":"new"}));
}
s
};
let size = |s: &Session| serde_json::to_vec(&saved_geometry(s, &s.doc).unwrap().records).unwrap().len();
let (few, many) = (design(2), design(12));
let sheet = serde_json::to_vec(&many.model.state().sheets).unwrap().len();
let per_feature = (size(&many) - size(&few)) / 10;
assert!(per_feature * 2 < sheet, "each later feature adds {per_feature} bytes, the sheet is {sheet}");
// Taken from the file as saved: the sheet is the one evaluated.
let flat = many.model.state().sheets.clone();
let path = std::env::temp_dir().join(format!("solidcraft-sheet-cache-{}.solidcraft", std::process::id()));
std::fs::write(&path, many.design_container(solidcraft_io::DesignCompression::Fast, false, true).unwrap()).unwrap();
let mut reopened = Session::default();
reopened.execute("doc.open", &json!({"path":path})).unwrap();
assert!(reopened.model.results.iter().zip(&many.model.results).all(|(a, b)| a.ms == b.ms), "taken from the file, not evaluated");
assert_eq!(serde_json::to_value(&reopened.model.state().sheets).unwrap(), serde_json::to_value(&flat).unwrap());
std::fs::remove_file(path).unwrap();
}
#[test]
#[ignore = "manual native cache save benchmark; generate issue303-step.json first"]
fn benchmark_native_cache_save() {
+2 -2
View File
@@ -954,7 +954,7 @@ mod tests {
solidcraft_doc::SavedModel {
records: json!({
"features": [{"id": 1, "fingerprint": "0000000000000000", "ms": 0.0, "skipped": false, "state": 0}],
"states": [state], "model_bodies": records, "names": [{"faces": [], "info": []}], "sketches": [],
"states": [state], "model_bodies": records, "names": [{"faces": [], "info": []}], "sketches": [], "sheets": [],
}),
bodies,
}
@@ -1060,7 +1060,7 @@ mod tests {
.map(|(i, ms)| json!({"id": i + 1, "fingerprint": "0000000000000000", "ms": ms, "skipped": false, "state": i}))
.collect();
solidcraft_doc::SavedModel {
records: json!({"features": features, "states": states, "model_bodies": records, "names": [{"faces": [], "info": []}], "sketches": []}),
records: json!({"features": features, "states": states, "model_bodies": records, "names": [{"faces": [], "info": []}], "sketches": [], "sheets": []}),
bodies,
}
}
+1 -1
View File
@@ -5,7 +5,7 @@
use super::*;
use solidcraft_doc::{ModelState, SavedModel};
const FORMAT: &str = "solidcraft-model-cache/3";
const FORMAT: &str = "solidcraft-model-cache/4";
const PRODUCER: &str = "solidcraft-truck-adapter/3";
const MANIFEST: &str = "cache/model.json";
const PAYLOAD: &str = "cache/bodies.bin";
+4 -2
View File
@@ -147,7 +147,7 @@ SHA-256 of their encoding), and a placed copy of a repeated part (an instance fr
a STEP assembly) records the part and its matrix, so it shows that part's display
again instead of being meshed on its own.
`cache/model.json` records `solidcraft-model-cache/3`, the producer revision
`cache/model.json` records `solidcraft-model-cache/4`, the producer revision
`solidcraft-truck-adapter/3`, the application version, the build (source commit,
CPU architecture and OS), the SHA-256 of the exact `document.json` (whose blob
references pin every STEP, mesh and image payload) and of `cache/bodies.bin`, the
@@ -168,7 +168,9 @@ A model whose cache would push the container past them keeps fewer states
other states, the slowest feature's first (the evaluations opening would otherwise
repeat for longest), each while what it adds still fits: the bodies no state kept
so far holds (a body is stored once however many states hold it) and the records
of the state, those bodies, their names and sketches. When the rest of the
of the state, those bodies, their names and sketches. Sheet bodies, like sketches, are
written once and named by place in the states (most features leave the sheets as they were); a
thinned cache keeps all of them. When the rest of the
manifest (the bodies' table, the slices) then leaves too little room, the choice
is made again within that much less, and the final state alone is always tried.
A feature whose state is left out records none. The cache is left out (and the