fix(editor): take in edits made to the notebook file; sca cell / sca new

The editor held the open notebook in memory and rewrote the file on each edit, so an
agent's saved change was overwritten and never noticed. The file is now watched and
applied cell by cell (agent's version wins a same-cell conflict), the workspace list
follows new files, and sca cell/new edit through the running editor.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
NiJingzhe
2026-10-14 15:30:40 +08:00
parent 94c42a3329
commit 3d4e789da6
10 changed files with 851 additions and 8 deletions
+18
View File
@@ -6,6 +6,24 @@ user-facing notes live in [`docs/updates/`](docs/updates/) (English and
## Unreleased
### Added
- Editor: `sca cell ls|get|put|insert|rm|run` edits the open notebook's cells through the
running editor (found by `<workspace>/.sca-editor/server.json`): the page shows the change live,
with no reload, the cell runs, and the command prints its status, errors and output. `sca new
NAME [--kind part|assembly]` writes a notebook the editor then lists, leaving the person's view
alone. The routes behind them are `GET /api/cells` and `POST /api/cells/{edit,insert,delete,run}`.
- Editor: a change to the open notebook's file made by someone else (an agent) is noticed
within a second and applied cell by cell (edited cells are updated and run again, added or
removed cells appear or go), without a reload; a changed setup or `@app.function` cell restarts
the kernel. Notebooks added to or removed from the workspace appear in the list by themselves.
### Fixed
- Editor: an edit made in the editor no longer writes over what an agent saved to the same
notebook file. If a cell was edited in both places, the file's version is kept and the
editor's unsaved text goes to the log.
## 2.2.2 — 2026-10-14
### Added
+3
View File
@@ -195,6 +195,9 @@ each instruction to one action and wait for them.
these edges by 1 mm"). You read the selection from the file (`selection` cell) or with
`sca snapshot`, and you can look at `.sca-editor/viewport.png` to see what they see. Then
you edit the notebook; they see your change in the editor.
- **Work through the editor.** Change cells with `sca cell` (`sca cell ls`, `put`, `insert`):
the page updates live and nothing the person typed is overwritten. Editing the file works
too: the editor takes the change in within a second.
- **"Change this cell."** Every cell has a copy button that copies a reference such as
`/path/u_link.py:L47-L48 (cell 02 ...)`. They paste it to you with what they want changed,
and you edit exactly those lines.
+4 -2
View File
@@ -50,7 +50,9 @@ Paths are relative to this file's directory.
selected, and you MUST read
`references/domains/editor-collaboration.md` first to see what they
see (selection cell, `sca snapshot`, cell references) and how to
hand the work back.
hand the work back. While the editor runs, change its notebooks with
`sca cell` (or the file; never both blindly): the editor is the
file's writer.
- Creating, installing, updating, or removing a SimpleCADAPI addon (a
third-party skill+tooling package around `.scadpkg`) enters no
workflow; you MUST use `references/domains/addon-development.md`
@@ -202,7 +204,7 @@ print(len(rebuilt))
- `references/inspect/brep-reverse-engineering.md`
- `references/domains/addon-development.md` — `sca` addon CLI and authoring guide
- `references/domains/editor-collaboration.md` — working with a person in `sca edit`:
selection cell, snapshot, cell references, parameters
selection cell, snapshot, cell references, `sca cell` / `sca new`, parameters
- `references/docs/guides/reconstruction-agent-test-prompt.md`
- `references/docs/guides/reconstruction-agent-strategy.md`
- `references/docs/guides/notebook-runtime.md`
@@ -6,11 +6,26 @@ edit code. It is not a workflow. The modeling you do still follows the workflow
you selected; this page says how to see what the person sees and how to hand
work back.
The notebook file is the only shared state. The editor autosaves every edit to
The notebook file is the shared state. The editor autosaves every edit to
it, and every message the person gives you (a selection, a cell reference, a
tuned value) is also something you can read from disk. Never rely on what the
person says they selected when the file or the snapshot can tell you.
The editor holds the open notebook in memory and is the one writer of its
file. You have two safe ways to change it while it runs; use the first:
1. **`sca cell` (preferred).** It edits through the editor: the page shows the
change live (no reload, no flicker), the cell runs, and you get its state back.
2. **Edit the file.** The editor notices a saved change within a second, compares
it with its cells and applies only what differs (changed cells are edited in
place and run again, added or removed cells appear or go away). A change to the
setup cell or an `@app.function` cell restarts the kernel instead. A file that
does not parse is ignored until the next save that does.
If the person has the same cell open with unsaved typing, **your version wins**:
their text is dropped and kept in the editor's log for them. Do not rewrite a
cell the person is typing in without telling them.
## Start the editor for a person
```bash
@@ -28,6 +43,29 @@ sca edit <workspace> --no-browser --port 7180
the assembly that fixes them together) to `./u_link_motor_mount`, then
`sca edit ./u_link_motor_mount`.
## Edit cells through the editor: `sca cell`
```bash
sca cell ls [--code] # numbered cells with status, errors and output
sca cell get 3 # one cell: code, status, errors, output
echo 'L = 90' | sca cell put 3 # replace the code of cell 3 and run it (--no-run to only save)
sca cell insert --after 3 -f new.py # add a cell (--after start|end|CELL, --name fn_name)
sca cell rm 4
sca cell run [CELL ...] # run cells (default all) and what they feed
sca new bracket --kind part # a new notebook; the open editor lists it, the person's view stays
```
- A cell is named by its number in `sca cell ls`, its `key`, its function name, or its id.
Numbers move when cells are added or removed: read the list the command prints back.
- Each write prints the touched cell's state (`status`, `errors`, `output`). That is the
feedback: read it before the next step, as with any other operation.
- `put` fails with a conflict if you pass a base that is out of date; without one it
replaces the cell, so `get` first when the person may have changed it.
- The commands find the running editor through `<workspace>/.sca-editor/server.json`. With no
editor running, or with another notebook open, `ls` and `get` read the file and the write
commands refuse: then edit the file directly, nothing else holds it.
- `@app.function` cells cannot be changed this way; edit them in the file.
## Read what the person is doing
| Need | Where |
@@ -140,10 +178,12 @@ file first, since every cell is blocked until it compiles.
## Rules of thumb
- Re-read the file before editing: the person edits and autosaves too, and the
editor rewrites the selection cell.
- Edit by cell. A change that touches many cells at once makes the person's
- Re-read the cell before editing (`sca cell get`, or the file): the person edits and
autosaves too, and the editor rewrites the selection cell.
- Edit by cell, with `sca cell`. A change that touches many cells at once makes the person's
screen reflow and loses their place.
- New notebooks: `sca new`, not a bare file write plus a request to refresh; the editor
lists a new file by itself either way.
- After an edit, say which cells you changed (by reference) and what to look
at, so the person can run them and judge the result.
- Never edit `__marimo__/` or `.sca-editor/`; both are generated.
+13
View File
@@ -186,6 +186,13 @@ export function App() {
setCells(next);
log.knowCells(next);
});
// A notebook was added or removed under the workspace (an agent wrote one).
socket.on("workspace.changed", () => refreshWorkspace());
// The server dropped an edit made on a cell that had changed on disk.
socket.on("cell.conflict", (envelope) => {
const mine = String(envelope["mine"] ?? "");
log.push("error", `a cell changed on disk while you were editing it: kept the file's version. Your unsaved text was:\n${mine.length > 600 ? `${mine.slice(0, 600)}…` : mine}`);
});
socket.on("notebook.features", (envelope) => setFeatures((envelope["features"] as NotebookFeature[]) ?? []));
socket.on("preview", (envelope) => {
@@ -355,6 +362,12 @@ export function App() {
socket={socketRef.current}
notebook={notebook}
onInterrupt={handleStop}
onConflict={(cell, mine) =>
log.push(
"error",
`${cell.title ?? "cell"} changed on disk while you were editing it: kept the file's version. Your unsaved text was:\n${mine.length > 600 ? `${mine.slice(0, 600)}…` : mine}`,
)
}
/>
</TabsContent>
<TabsContent value="workspace" className="min-h-0">
+54 -2
View File
@@ -74,6 +74,7 @@ export function CellEditor({
socket,
notebook,
onInterrupt,
onConflict,
}: {
cells: NotebookCell[];
features: NotebookFeature[];
@@ -81,6 +82,8 @@ export function CellEditor({
socket: SocketLike;
notebook: string | null;
onInterrupt: () => void;
/** A cell changed on disk while it had an unsaved edit here: the file's version was kept. */
onConflict: (cell: NotebookCell, mine: string) => void;
}) {
const [drafts, setDrafts] = useState<Map<string, string>>(new Map());
const [running, setRunning] = useState<string | null>(null);
@@ -89,14 +92,60 @@ export function CellEditor({
// the second; any click elsewhere disarms.
const [armed, setArmed] = useState<string | null>(null);
const saveTimers = useRef(new Map<string, number>());
// What each edited cell's draft started from, and the last text sent to be saved:
// a cell that arrives as something else was changed by someone else (an agent).
const bases = useRef(new Map<string, string>());
const sent = useRef(new Map<string, string>());
const featureByCell = useMemo(() => new Map(features.map((feature) => [feature.cell, feature])), [features]);
useEffect(() => {
setCollapsed(loadCollapsed(notebook));
setDrafts(new Map());
bases.current.clear();
sent.current.clear();
}, [notebook]);
// The cells arrive again after every save and every change on disk: settle each draft
// against them. Saved (equal) -> done; confirms our last save -> keep editing from it;
// anything else -> the file's version wins, the person is told.
const onConflictRef = useRef(onConflict);
onConflictRef.current = onConflict;
useEffect(() => {
const same = (a: string, b: string) => a.trim() === b.trim();
const lost: [NotebookCell, string][] = [];
setDrafts((previous) => {
let next: Map<string, string> | null = null;
const drop = (id: string) => {
next ??= new Map(previous);
next.delete(id);
bases.current.delete(id);
sent.current.delete(id);
};
for (const [id, draft] of previous) {
const cell = cells.find((candidate) => candidate.id === id);
if (!cell) {
drop(id);
continue;
}
const base = bases.current.get(id);
const last = sent.current.get(id);
if (same(cell.code, draft)) drop(id);
else if (last !== undefined && same(cell.code, last)) bases.current.set(id, cell.code);
else if (base !== undefined && !same(cell.code, base)) {
lost.push([cell, draft]);
drop(id);
}
}
return next ?? previous;
});
for (const [cell, mine] of lost) {
cancelSave(cell.id);
onConflictRef.current(cell, mine);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [cells]);
const remember = (next: Set<string>) => {
setCollapsed(next);
try {
@@ -122,7 +171,8 @@ export function CellEditor({
cancelSave(cell.id);
const code = drafts.get(cell.id) ?? cell.code;
setRunning(cell.id);
socket?.send({ type: "code.edit", cell: cell.id, code });
sent.current.set(cell.id, code);
socket?.send({ type: "code.edit", cell: cell.id, code, base: bases.current.get(cell.id) });
window.setTimeout(() => setRunning((current) => (current === cell.id ? null : current)), 2500);
};
@@ -141,6 +191,7 @@ export function CellEditor({
const insert = (after: string | null) => socket?.send({ type: "insert.request", after });
const edit = (cell: NotebookCell, value: string) => {
if (!bases.current.has(cell.id)) bases.current.set(cell.id, cell.code);
setDrafts((previous) => new Map(previous).set(cell.id, value));
if (cell.readonly) return;
cancelSave(cell.id);
@@ -148,7 +199,8 @@ export function CellEditor({
cell.id,
window.setTimeout(() => {
saveTimers.current.delete(cell.id);
socket?.send({ type: "cell.save", cell: cell.id, code: value });
sent.current.set(cell.id, value);
socket?.send({ type: "cell.save", cell: cell.id, code: value, base: bases.current.get(cell.id) });
}, AUTOSAVE_MS),
);
};
+237
View File
@@ -6,6 +6,8 @@
sca edit path/to/model.py # ... starting at one notebook
sca edit --port 7180 --no-browser
sca snapshot [WORKSPACE] # what the person has selected / is looking at
sca cell ls|get|put|insert|rm|run # edit the cells of the notebook the editor has open
sca new NAME [--kind part|assembly] # a new notebook; the editor lists it at once
"""
from __future__ import annotations
@@ -43,6 +45,63 @@ def configure(subparsers: argparse._SubParsersAction) -> None:
snapshot.add_argument("workspace", type=Path, nargs="?", default=Path.cwd())
snapshot.set_defaults(handler=_snapshot, printer=_print_snapshot)
cell = subparsers.add_parser(
"cell",
help="read and edit the cells of a notebook through the running editor",
description="Edit a notebook's cells through the editor that has it open: the page "
"shows the change live (no reload) and the editor, the one writer of the file, saves "
"it, so nothing is overwritten. Cells are named by number (see `sca cell ls`), by key "
"or by id. When the editor is not running, or has another notebook open, `ls` and "
"`get` read the file and the write commands refuse: edit the file directly then, "
"nothing else holds it.",
)
cell_sub = cell.add_subparsers(dest="cell_command", required=True)
def common(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--notebook", "-n", type=Path, default=None, help="the notebook (default: the one the editor has open)")
parser.add_argument("--workspace", "-w", type=Path, default=None, help="the workspace the editor serves (default: found from the current directory)")
ls = cell_sub.add_parser("ls", help="list the cells with their state")
common(ls)
ls.add_argument("--code", action="store_true", help="include each cell's code")
ls.set_defaults(handler=_cell_ls, printer=_print_json)
get = cell_sub.add_parser("get", help="print one cell: its code, state, errors and output")
common(get)
get.add_argument("cell")
get.set_defaults(handler=_cell_get, printer=_print_json)
put = cell_sub.add_parser("put", help="replace a cell's code (the code comes from --file or stdin) and run it")
common(put)
put.add_argument("cell")
put.add_argument("--file", "-f", type=Path, default=None, help="read the code from this file (default: stdin)")
put.add_argument("--no-run", action="store_true", help="save the code without running the cell")
put.set_defaults(handler=_cell_put, printer=_print_json)
insert = cell_sub.add_parser("insert", help="add a cell (the code comes from --file or stdin)")
common(insert)
insert.add_argument("--after", default="end", help="a cell, or `start` / `end` (default)")
insert.add_argument("--name", default=None, help="the cell's function name")
insert.add_argument("--file", "-f", type=Path, default=None)
insert.add_argument("--no-run", action="store_true")
insert.set_defaults(handler=_cell_insert, printer=_print_json)
rm = cell_sub.add_parser("rm", help="delete a cell")
common(rm)
rm.add_argument("cell")
rm.set_defaults(handler=_cell_rm, printer=_print_json)
run = cell_sub.add_parser("run", help="run cells (default: all) and what they feed")
common(run)
run.add_argument("cells", nargs="*")
run.set_defaults(handler=_cell_run, printer=_print_json)
new = subparsers.add_parser(
"new",
help="create a notebook in the workspace (the running editor lists it at once)",
description="Write a new notebook from the template into WORKSPACE (default: the "
"current directory). A running editor on that workspace lists it without a reload.",
)
new.add_argument("name")
new.add_argument("--kind", choices=("part", "assembly"), default="part")
new.add_argument("--workspace", "-w", type=Path, default=None)
new.set_defaults(handler=_new_notebook, printer=_print_json)
def _execute(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
try:
@@ -86,4 +145,182 @@ def _print_snapshot(report: dict[str, Any]) -> None:
print(json.dumps(report, indent=2, ensure_ascii=False))
# -- sca cell / sca new ---------------------------------------------------------
def _print_json(report: dict[str, Any]) -> None:
import json
print(json.dumps(report, indent=2, ensure_ascii=False))
def _find_editor(start: Path) -> tuple[Path, str] | None:
"""The running editor whose workspace holds *start*: ``(workspace, base url)``."""
import json
import urllib.request
here = start.expanduser().resolve()
for directory in (here, *here.parents):
record = directory / ".sca-editor" / "server.json"
if not record.is_file():
continue
try:
url = json.loads(record.read_text(encoding="utf-8"))["url"]
with urllib.request.urlopen(f"{url}/api/health", timeout=1.5):
pass
except (OSError, ValueError, KeyError):
continue # a stale record: the editor is gone
return directory, str(url)
return None
def _call(url: str, path: str, body: dict | None = None) -> dict[str, Any]:
import json
import urllib.error
import urllib.request
data = None if body is None else json.dumps(body).encode("utf-8")
request = urllib.request.Request(
url + path, data=data, headers={"Content-Type": "application/json"}, method="GET" if body is None else "POST"
)
try:
with urllib.request.urlopen(request, timeout=300) as response:
return json.loads(response.read())
except urllib.error.HTTPError as error:
try:
return json.loads(error.read())
except ValueError:
return {"error": f"editor answered {error.code}"}
def _session(args: argparse.Namespace, *, write: bool) -> tuple[str | None, Path | None, str]:
"""``(editor url or None, notebook path or None, relative notebook name)`` for a cell command."""
start = args.workspace or (args.notebook.parent if args.notebook else Path.cwd())
found = _find_editor(start)
notebook = args.notebook.expanduser().resolve() if args.notebook else None
if found is None:
if write:
raise ValueError(
"no editor is running on this workspace (start `sca edit`), or edit the notebook file "
"directly: nothing else holds it"
)
if notebook is None:
raise ValueError("no editor is running here; name the notebook with --notebook")
return None, notebook, notebook.name
workspace, url = found
open_path = _call(url, "/api/cells").get("notebook")
if notebook is None:
if open_path is None:
raise ValueError("the editor has no notebook open")
return url, workspace / open_path, open_path
relative = notebook.relative_to(workspace).as_posix() if notebook.is_relative_to(workspace) else notebook.name
if relative == open_path:
return url, notebook, relative
if write:
raise ValueError(
f"the editor has {open_path} open, not {relative}: edit {relative} directly "
"(the editor does not hold it), or open it in the editor first"
)
return None, notebook, relative
def _cells_from_file(notebook: Path) -> list[dict[str, Any]]:
from ..runtime._marimo import parse_notebook
parsed = parse_notebook(notebook.read_text(encoding="utf-8"))
cells = []
for cell in (parsed.cells if parsed is not None else []):
if getattr(cell, "name", None) == "setup":
continue
cells.append({"index": len(cells) + 1, "name": getattr(cell, "name", None), "code": cell.code, "source": "file"})
return cells
def _slim(report: dict[str, Any], *, code: bool) -> dict[str, Any]:
if "cells" not in report:
return report
keep = ("index", "key", "name", "title", "status", "errors", "output", "ref", "readonly")
return {
**report,
"cells": [
{**{k: c[k] for k in keep if k in c and c[k] not in (None, [], False)}, **({"code": c["code"]} if code else {})}
for c in report["cells"]
],
}
def _cell_ls(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
url, notebook, name = _session(args, write=False)
if url is None:
assert notebook is not None
return {"notebook": name, "source": "file", "cells": _cells_from_file(notebook)}, 0
return _slim(_call(url, "/api/cells"), code=args.code), 0
def _cell_get(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
url, notebook, name = _session(args, write=False)
if url is None:
assert notebook is not None
cells = _cells_from_file(notebook)
if not args.cell.isdigit() or not 1 <= int(args.cell) <= len(cells):
raise ValueError("without the editor, name a cell by its number (see `sca cell ls`)")
return {"notebook": name, "source": "file", **cells[int(args.cell) - 1]}, 0
report = _call(url, "/api/cells")
for cell in report.get("cells", []):
if args.cell in (str(cell["index"]), cell.get("key"), cell["id"]):
return {"notebook": report.get("notebook"), **cell}, 0
raise ValueError(f"no cell {args.cell!r}; run `sca cell ls`")
def _read_code(args: argparse.Namespace) -> str:
import sys
if args.file is not None:
return args.file.read_text(encoding="utf-8")
if sys.stdin.isatty():
raise ValueError("give the code with --file, or on stdin")
return sys.stdin.read()
def _write_op(args: argparse.Namespace, op: str, body: dict[str, Any]) -> tuple[dict[str, Any], int]:
url, _notebook, _name = _session(args, write=True)
assert url is not None
report = _call(url, f"/api/cells/{op}", body)
if "error" in report:
return _slim(report, code=False), 1
touched = set(report.get("touched", []))
report = _slim(report, code=False)
# What the change did: the touched cells with their state; the rest as a count.
report["cells"] = [c for c in report["cells"] if c.get("index") in touched] or report["cells"]
return report, 0
def _cell_put(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
return _write_op(args, "edit", {"cell": args.cell, "code": _read_code(args), "run": not args.no_run})
def _cell_insert(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
return _write_op(args, "insert", {"after": args.after, "name": args.name, "code": _read_code(args), "run": not args.no_run})
def _cell_rm(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
return _write_op(args, "delete", {"cell": args.cell})
def _cell_run(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
return _write_op(args, "run", {"cells": args.cells})
def _new_notebook(args: argparse.Namespace) -> tuple[dict[str, Any], int]:
from .templates import create_notebook
start = (args.workspace or Path.cwd()).expanduser().resolve()
found = _find_editor(start)
root = found[0] if found is not None else start
# Only the file is written: the running editor lists it on its own and the
# person's current notebook stays open.
path = create_notebook(root, args.name, args.kind)
return {"ok": True, "path": str(path), "editor_listed": found is not None}, 0
__all__ = ["configure"]
+44
View File
@@ -20,6 +20,7 @@ typing in the embedded marimo editor), and is what the server reacts to.
from __future__ import annotations
import functools
import json
import os
import re
@@ -43,6 +44,22 @@ class KernelError(RuntimeError):
pass
def _locked(method):
"""Run a cell-writing method under the kernel's write lock.
The editor's own writes and the ingest of an external edit of the notebook
file must not interleave: ingest compares the file with the kernel's cells,
and a write half-way through would look like an external change.
"""
@functools.wraps(method)
def wrapper(self, *args, **kwargs):
with self.write_lock:
return method(self, *args, **kwargs)
return wrapper
class MarimoKernel:
def __init__(
self,
@@ -59,6 +76,7 @@ class MarimoKernel:
self.setup_errors: list[str] = []
self._ran = threading.Event() # a run of cells completed (see wait_instantiated)
self._own_calls = 0
self.write_lock = threading.RLock()
self._own_ended = 0.0
self.workspace = self.notebook.parent
self.marimo_port = _free_port(server_port + 1)
@@ -357,6 +375,7 @@ class MarimoKernel:
"""Queue cells to run (lazy mode: their dependents turn stale)."""
return self.execute(_driver("\n".join(f" ctx.run_cell({cell_id!r})" for cell_id in cell_ids)))
@_locked
def edit_cell(self, cell_id: str, code: str) -> dict:
"""Replace a cell's code (persisted to the file) and run it."""
return self.execute(
@@ -365,6 +384,7 @@ class MarimoKernel:
+ _driver(f" ctx.edit_cell({cell_id!r}, code=__code)\n ctx.run_cell({cell_id!r})")
)
@_locked
def save_cell(self, cell_id: str, code: str) -> dict:
"""Replace a cell's code (persisted to the file) without running it."""
return self.execute(
@@ -373,6 +393,7 @@ class MarimoKernel:
+ _driver(f" ctx.edit_cell({cell_id!r}, code=__code)")
)
@_locked
def put_cell(self, code: str, *, cell_id: str | None, run: bool = True) -> str | None:
"""Write *code* into cell *cell_id*, or append a new cell holding it.
@@ -401,9 +422,11 @@ class MarimoKernel:
return line[len("SCA-CELL "):].strip()
return None
@_locked
def delete_cell(self, cell_id: str) -> dict:
return self.execute(_driver(f" ctx.delete_cell({cell_id!r})"))
@_locked
def insert_cell(self, after: str | None) -> dict:
"""Insert one empty, named cell after *after* (at the top if None).
@@ -431,6 +454,27 @@ class MarimoKernel:
)
)
@_locked
def create_cell_after(self, after: str | None, code: str, name: str | None, *, run: bool = True) -> dict:
"""Create a cell holding *code* after cell *after* (at the top if None)."""
body = (
" __names = [c.id for c in ctx.cells]\n"
+ (
f" __id = ctx.create_cell(__code, after={after!r}, name={name!r})\n"
if after
else (
f" __id = ctx.create_cell(__code, before=__names[0], name={name!r}) if __names"
f" else ctx.create_cell(__code, name={name!r})\n"
)
)
+ (" ctx.run_cell(__id)\n" if run else "")
+ "print('SCA-CELL', __id)\n"
)
return self.execute(
"import json as _json\n"
f"__code = _json.loads({json.dumps(json.dumps(code))})\n" + _driver(body)
)
def reload_modules(self, names: list[str]) -> dict:
"""Forget the local modules *names* and run every cell again.
+339
View File
@@ -144,6 +144,9 @@ class WorkspaceState:
self._generation = 0
self._switch_lock = threading.Lock()
self._booting: MarimoKernel | None = None
self._setup_code: str | None = None
self._edited_from: dict[str, set[str]] = {}
self._shutdown = threading.Event()
# -- notebook / kernel lifecycle ---------------------------------------------
@@ -263,6 +266,7 @@ class WorkspaceState:
self.cells = []
self.specs = {}
self._watched = self._watched_files()[0]
self._setup_code = _setup_code_of(notebook)
self.hub.broadcast(self.hello())
kernel.wait_instantiated()
kernel.wait_idle()
@@ -277,6 +281,9 @@ class WorkspaceState:
threading.Thread(
target=self._watch_files, args=(kernel,), name="file-watch", daemon=True
).start()
threading.Thread(
target=self._watch_notebook, args=(kernel,), name="notebook-watch", daemon=True
).start()
# -- remembering what was published -----------------------------------------------
@@ -479,6 +486,250 @@ class WorkspaceState:
self._module_busy = True
threading.Thread(target=self._propagate, name="propagate", daemon=True).start()
# -- the notebook file changed under the editor ------------------------------
def _watch_notebook(self, kernel: MarimoKernel) -> None:
"""Take in edits made to the open notebook's file by someone else (an agent).
The kernel holds the notebook in memory and writes the file on every edit
the editor makes, so an outside edit that the kernel never hears of would be
written over by the editor's next save. Poll the file; once it has been
quiet for one tick, compare it with the kernel's cells (a change the editor
itself made shows no difference) and apply what differs.
"""
stamp = self._stamp(self.notebook)
pending: tuple[int, int] | None = None
while not kernel._stop.is_set():
time.sleep(0.3)
if self.kernel is not kernel:
return
current = self._stamp(self.notebook)
if current != stamp:
stamp, pending = current, current
continue
if pending is None:
continue
pending = None
try:
self.ingest_external(kernel)
except Exception as exc:
self.hub.broadcast({"type": "status", "error": f"could not take in the file change: {str(exc)[-200:]}"})
@staticmethod
def _stamp(path: Path | None) -> tuple[int, int]:
try:
assert path is not None
info = path.stat()
return info.st_mtime_ns, info.st_size
except (OSError, AssertionError):
return 0, 0
def ingest_external(self, kernel: MarimoKernel) -> None:
"""Make the kernel's cells equal the notebook file's, changing only what differs.
Cells that differ are edited, created or deleted in place (the page updates
without a reload), then the cells that changed are run and what they feed
turns fresh. A change the cell commands cannot carry (the setup cell, an
``@app.function`` cell) restarts the kernel instead.
"""
import difflib
from ..runtime._marimo import parse_notebook
notebook = self.notebook
if notebook is None:
return
with kernel.write_lock:
try:
parsed = parse_notebook(notebook.read_text(encoding="utf-8"))
except (OSError, UnicodeError):
return
except SyntaxError as exc:
self.hub.broadcast({"type": "status", "error": f"{notebook.name} has a syntax error ({exc.msg}, line {exc.lineno}); waiting for the next save"})
return
if parsed is None:
return
setup = next((c.code.strip() for c in parsed.cells if getattr(c, "name", None) == "setup"), "")
if self._setup_code is not None and setup != self._setup_code:
self.hub.broadcast({"type": "status", "working": "the setup cell changed on disk: restarting the kernel…"})
self._in_thread(lambda: self.start_kernel(notebook))
return
disk = [
(getattr(c, "name", None) or None, c.code)
for c in parsed.cells
if getattr(c, "name", None) != "setup"
]
live = kernel.list_cells()
norm = lambda code: code.strip() # noqa: E731
matcher = difflib.SequenceMatcher(
None, [norm(c["code"]) for c in live], [norm(code) for _name, code in disk], autojunk=False
)
ops = [op for op in matcher.get_opcodes() if op[0] != "equal"]
if not ops:
return
touched = [live[i]["code"] for _t, i1, i2, _j1, _j2 in ops for i in range(i1, i2)]
touched += [code for _t, _i1, _i2, j1, j2 in ops for _name, code in disk[j1:j2]]
if any(_is_function_cell(code) for code in touched):
self.hub.broadcast({"type": "status", "working": "an @app.function cell changed on disk: restarting the kernel…"})
self._in_thread(lambda: self.start_kernel(notebook))
return
ids = [c["id"] for c in live]
changed: list[str] = []
previous: str | None = None
for tag, i1, i2, j1, j2 in matcher.get_opcodes():
if tag == "equal":
previous = ids[i2 - 1] if i2 > i1 else previous
continue
paired = min(i2 - i1, j2 - j1) if tag == "replace" else 0
for offset in range(paired):
kernel.save_cell(ids[i1 + offset], disk[j1 + offset][1])
changed.append(ids[i1 + offset])
previous = ids[i1 + offset]
for index in range(i1 + paired, i2):
kernel.delete_cell(ids[index])
for name, code in disk[j1 + paired : j2]:
created = kernel.create_cell_after(previous, code, name, run=False)
new_id = _created_id(created)
if new_id is None:
raise RuntimeError("the kernel did not create a cell for the new code on disk")
changed.append(new_id)
previous = new_id
self.hub.broadcast({"type": "status", "info": f"{notebook.name} changed on disk: {len(changed)} cell(s) updated, running them"})
self.push_cells()
self._in_thread(lambda: self._rerun_changed(kernel, changed))
# -- the same cell edits, for an agent (``sca cell``) -------------------------
def _brief(self, number: int, cell: dict) -> dict:
return {
"index": number,
"id": cell["id"],
"key": cell.get("key"),
"name": cell.get("name"),
"title": cell.get("title"),
"status": cell.get("status"),
"errors": cell.get("errors") or [],
"output": cell.get("output"),
"console": cell.get("console") or [],
"readonly": bool(cell.get("readonly")),
"ref": cell.get("ref"),
"code": cell["code"],
}
def cells_report(self, *, refresh: bool = True) -> dict:
if refresh and self.kernel is not None:
self.push_cells()
return {
"notebook": self.notebook.relative_to(self.workspace).as_posix() if self.notebook else None,
"cells": [self._brief(n, c) for n, c in enumerate(self.cells, start=1)],
}
def _find_cell(self, ref: Any) -> dict:
text = str(ref).strip()
for cell in self.cells:
if text in (cell["id"], cell.get("key")) or (text == cell.get("name") and text != "_"):
return cell
if text.isdigit() and 1 <= int(text) <= len(self.cells):
return self.cells[int(text) - 1]
raise KeyError(f"no cell {text!r} in {self.notebook.name if self.notebook else 'the notebook'}; run `sca cell ls`")
def cell_command(self, op: str, body: dict) -> dict:
"""Run one cell operation for the CLI; returns the report or ``{"error": ...}``.
The edit goes through the kernel exactly as one made in the page does, so the
page shows it live and the file is written by the one writer that holds it.
"""
kernel = self.kernel
if kernel is None:
return {"error": "the editor has no kernel yet; wait for it to start"}
self.push_cells()
touched: list[str] = []
try:
if op == "edit":
cell = self._find_cell(body.get("cell"))
code = _module_code(str(body.get("code", "")))
if _is_function_cell(cell["code"]):
return {"error": "this cell is an @app.function; edit it in the file (the editor takes the change in)"}
if body.get("base") is not None and _module_code(str(body["base"])).strip() != _module_code(cell["code"]).strip():
return {"error": "conflict: the cell changed since you read it", "theirs": cell["code"]}
if code.strip() != _module_code(cell["code"]).strip():
(kernel.edit_cell if body.get("run", True) else kernel.save_cell)(cell["id"], code)
elif body.get("run", True):
kernel.run_cells([cell["id"]])
touched = [cell["id"]]
elif op == "insert":
after = body.get("after")
if after in (None, "end"):
anchor = self.cells[-1]["id"] if self.cells else None
elif after == "start":
anchor = None
else:
anchor = self._find_cell(after)["id"]
code = _module_code(str(body.get("code", ""))) or "# new cell"
name = str(body["name"]) if body.get("name") else None
created = _created_id(kernel.create_cell_after(anchor, code, name, run=bool(body.get("run", True))))
if created is None:
return {"error": "the kernel did not create the cell"}
touched = [created]
elif op == "delete":
cell = self._find_cell(body.get("cell"))
kernel.delete_cell(cell["id"])
elif op == "run":
refs = body.get("cells") or []
ids = [self._find_cell(ref)["id"] for ref in refs] or [c["id"] for c in self.cells]
kernel.run_until_settled(ids)
touched = ids
else:
return {"error": f"unknown cell operation {op!r}"}
except KeyError as exc:
return {"error": str(exc.args[0])}
kernel.wait_idle()
self.push_cells()
self.schedule_settle()
report = self.cells_report(refresh=False)
report["touched"] = [c["index"] for c in report["cells"] if c["id"] in touched]
return report
def _rerun_changed(self, kernel: MarimoKernel, changed: list[str]) -> None:
kernel.run_until_settled(changed)
self.schedule_settle()
def _watch_workspace(self) -> None:
"""Tell the pages when notebooks appear or disappear under the workspace."""
known = self._workspace_stamps()
while not self._shutdown.is_set():
time.sleep(0.7)
try:
current = self._workspace_stamps()
except OSError:
continue
# The open notebook is rewritten by the editor on every edit; only its
# appearing or vanishing counts.
open_path = self.notebook
changed = set(current) ^ set(known)
changed |= {p for p in current if p in known and current[p] != known[p] and p != open_path}
known = current
if changed:
self.hub.broadcast({"type": "workspace.changed"})
def _workspace_stamps(self) -> dict[Path, int]:
import os
stamps: dict[Path, int] = {}
for directory, subdirectories, files in os.walk(self.workspace):
subdirectories[:] = [
name for name in subdirectories
if not name.startswith((".", "_")) and name not in _SKIPPED_DIRS
]
for name in files:
if name.endswith(".py") and not name.startswith("_"):
path = Path(directory) / name
try:
stamps[path] = path.stat().st_mtime_ns
except OSError:
continue
return stamps
def _propagate(self) -> None:
"""Serialized, coalescing: one reload covers every change that arrived meanwhile."""
try:
@@ -807,6 +1058,8 @@ class WorkspaceState:
current = next((c["code"] for c in self.cells if c["id"] == cell), None)
if current is None or code.strip() == _module_code(current).strip():
return {"type": "ack", "command": kind}
if self._conflicts(message, cell, code, current):
return {"type": "ack", "command": kind}
if _is_function_cell(current):
return {
"type": "error",
@@ -840,6 +1093,8 @@ class WorkspaceState:
elif kind == "code.edit":
code = _module_code(str(message.get("code", "")))
current = next((c["code"] for c in self.cells if c["id"] == cell), None)
if current is not None and code.strip() != _module_code(current).strip() and self._conflicts(message, cell, code, current):
return {"type": "ack", "command": kind}
if current is not None and code.strip() != _module_code(current).strip() and _is_function_cell(current):
# marimo's code mode would turn an @app.function into a plain
# cell; edit those in the file or in marimo's own editor.
@@ -870,6 +1125,27 @@ class WorkspaceState:
self._in_thread(run)
return {"type": "ack", "command": kind}
def _conflicts(self, message: dict, cell: str, mine: str, current: str) -> bool:
"""The page edited a cell that changed under it: keep what is on disk.
The page says which code its edit started from (``base``). If the cell
is something else now (an agent edited the file and the editor took the
change in), writing the page's version would erase that edit, so the
write is dropped and the page is told both versions; it shows the file's
version and keeps its own in the log.
"""
base = message.get("base")
now = _module_code(current).strip()
# A page types on: its second save still names the text it started from, which
# its own first save has replaced since. Only a text the editor did not write
# over is a change made by someone else.
mine_before = self._edited_from.setdefault(cell, set())
if base is None or _module_code(str(base)).strip() in (now, *mine_before):
mine_before.add(now)
return False
self.hub.broadcast({"type": "cell.conflict", "cell": cell, "mine": mine, "theirs": current})
return True
def _in_thread(self, work: Any) -> None:
def run() -> None:
try:
@@ -1119,6 +1395,26 @@ class WorkspaceState:
"model_revision": self.preview_revision,
}
def write_endpoint(self, host: str) -> None:
"""Say where this editor listens, so ``sca cell`` and other tools find it."""
try:
directory = self.snapshot_dir
directory.mkdir(parents=True, exist_ok=True)
import os
(directory / "server.json").write_text(
json.dumps({"url": f"http://{host}:{self.port}", "pid": os.getpid(), "workspace": str(self.workspace)}),
encoding="utf-8",
)
except OSError:
pass
def remove_endpoint(self) -> None:
try:
(self.snapshot_dir / "server.json").unlink()
except OSError:
pass
def write_snapshot(self) -> None:
"""Keep ``.sca-editor/snapshot.json`` current; ``sca snapshot`` reads it."""
try:
@@ -1192,6 +1488,28 @@ def _module_code(code: str) -> str:
return "\n".join(lines)
_SKIPPED_DIRS = frozenset({"node_modules", "venv", "site-packages", "build", "dist"})
def _setup_code_of(notebook: Path) -> str | None:
from ..runtime._marimo import parse_notebook
try:
parsed = parse_notebook(notebook.read_text(encoding="utf-8"))
except (OSError, UnicodeError, SyntaxError):
return None
if parsed is None:
return None
return next((c.code.strip() for c in parsed.cells if getattr(c, "name", None) == "setup"), "")
def _created_id(result: dict) -> str | None:
for line in (result.get("log") or "").splitlines():
if line.startswith("SCA-CELL "):
return line[len("SCA-CELL "):].strip()
return None
def _static_cells(notebook: Path) -> list[dict]:
"""The notebook's cells as the file has them, for showing before a kernel runs them."""
from ..runtime._marimo import parse_notebook
@@ -1365,6 +1683,21 @@ def create_app(state: WorkspaceState) -> Starlette:
)
return JSONResponse({"error": "no package made yet"}, status_code=404)
async def api_cells(request: Request) -> JSONResponse:
"""The open notebook's cells, with their state (for ``sca cell ls``)."""
report = await asyncio.to_thread(state.cells_report)
return JSONResponse(report)
async def api_cell_command(request: Request) -> JSONResponse:
op = request.path_params["op"]
try:
body = json.loads(await request.body() or b"{}")
except json.JSONDecodeError:
return JSONResponse({"error": "body is not JSON"}, status_code=400)
report = await asyncio.to_thread(state.cell_command, op, body if isinstance(body, dict) else {})
status = 409 if str(report.get("error", "")).startswith("conflict") else 400 if "error" in report else 200
return JSONResponse(report, status_code=status)
async def sdk_index(request: Request) -> JSONResponse:
"""The SDK's public API (names and parameters), for the cell editor's completion."""
from .apiindex import build_index
@@ -1491,6 +1824,8 @@ def create_app(state: WorkspaceState) -> Starlette:
Route("/api/runtime/package", push_package, methods=["POST"]),
Route("/api/package/latest", latest_package, methods=["GET"]),
Route("/api/sdk", sdk_index, methods=["GET"]),
Route("/api/cells", api_cells, methods=["GET"]),
Route("/api/cells/{op}", api_cell_command, methods=["POST"]),
Route("/api/workspace", workspace_list, methods=["GET"]),
Route("/api/workspace/open", workspace_open, methods=["POST"]),
Route("/api/workspace/new", workspace_new, methods=["POST"]),
@@ -1532,6 +1867,8 @@ def serve_workspace(
hub = state.hub
hub.attach_loop(asyncio.get_running_loop())
print(f"[sca edit] workspace {state.workspace}")
state.write_endpoint(host)
threading.Thread(target=state._watch_workspace, name="workspace-watch", daemon=True).start()
if state.notebook is None:
print("[sca edit] no SimpleCAD notebook found in the workspace", file=sys.stderr)
@@ -1563,6 +1900,8 @@ def serve_workspace(
original_shutdown = server.shutdown
async def shutdown(sockets=None):
state._shutdown.set()
state.remove_endpoint()
if state.kernel is not None:
state.kernel.stop()
await original_shutdown(sockets=sockets)
+95
View File
@@ -881,3 +881,98 @@ class ApiIndexTest(unittest.TestCase):
response = TestClient(app).get("/api/sdk")
self.assertEqual(response.status_code, 200)
self.assertIn("scad", response.json()["namespaces"])
class ConflictTest(unittest.TestCase):
"""A page's edit of a cell that changed on disk must not erase the change."""
def make_state(self) -> tuple[WorkspaceState, list[dict]]:
state = WorkspaceState(Path(tempfile.mkdtemp()))
sent: list[dict] = []
state.hub.broadcast = sent.append # type: ignore[method-assign]
return state, sent
def test_an_edit_that_started_from_another_text_is_dropped_and_reported(self) -> None:
state, sent = self.make_state()
self.assertTrue(state._conflicts({"base": "x = 1"}, "c1", "x = 2", "x = 9"))
self.assertEqual(sent[0]["type"], "cell.conflict")
self.assertEqual((sent[0]["mine"], sent[0]["theirs"]), ("x = 2", "x = 9"))
def test_an_edit_from_the_current_text_goes_through(self) -> None:
state, sent = self.make_state()
self.assertFalse(state._conflicts({"base": "x = 1\n"}, "c1", "x = 2", "x = 1"))
self.assertFalse(state._conflicts({}, "c1", "x = 2", "x = 1")) # an old page sends no base
self.assertEqual(sent, [])
def test_a_second_save_of_the_same_typing_is_not_a_conflict(self) -> None:
state, sent = self.make_state()
# first save: the page started from "a", the file has "a"
self.assertFalse(state._conflicts({"base": "a"}, "c1", "ab", "a"))
# second save arrives while the first is applied: the file has "ab", the page still says "a"
self.assertFalse(state._conflicts({"base": "a"}, "c1", "abc", "ab"))
# the file has "abc" now; an agent makes it "zzz": the page's next save from "abc" is a conflict
self.assertTrue(state._conflicts({"base": "abc"}, "c1", "abd", "zzz"))
self.assertEqual(len(sent), 1)
class WorkspaceWatchTest(unittest.TestCase):
def test_stamps_see_new_notebooks_and_skip_hidden_directories(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
state = WorkspaceState(root)
create_notebook(root, "first", "part")
before = state._workspace_stamps()
(root / ".sca-editor").mkdir()
(root / ".sca-editor" / "x.py").write_text("x = 1\n")
create_notebook(root, "second", "part")
after = state._workspace_stamps()
self.assertEqual({p.name for p in after} - {p.name for p in before}, {"second.py"})
class CellCliTest(unittest.TestCase):
def test_cell_commands_refuse_to_write_without_an_editor(self) -> None:
from simplecadapi.cli import main
with tempfile.TemporaryDirectory() as tmp:
path = create_notebook(Path(tmp), "bracket", "part")
self.assertEqual(main(["cell", "rm", "1", "--notebook", str(path)]), 2)
def test_cell_ls_reads_the_file_when_no_editor_runs(self) -> None:
from simplecadapi.editor import cli
with tempfile.TemporaryDirectory() as tmp:
path = create_notebook(Path(tmp), "bracket", "part")
cells = cli._cells_from_file(path)
self.assertTrue(cells)
self.assertEqual(cells[0]["index"], 1)
self.assertIsNone(cli._find_editor(Path(tmp)))
@unittest.skipUnless(os.environ.get("SCA_EDITOR_E2E"), "set SCA_EDITOR_E2E=1 to boot a marimo kernel")
class ExternalEditTest(unittest.TestCase):
def test_an_edit_of_the_file_is_taken_in_and_not_written_over(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
path = create_notebook(root, "bracket", "part")
state = WorkspaceState(root)
state.port = 17733
state.hub.attach_loop(__import__("asyncio").new_event_loop())
try:
state.start_kernel(path)
kernel = state.kernel
assert kernel is not None
before = [c["code"] for c in kernel.list_cells()]
text = path.read_text()
anchor = 'if __name__ == "__main__":'
added = '@app.cell\ndef _agent():\n agent_note = 1\n return (agent_note,)\n\n\n'
path.write_text(text.replace(anchor, added + anchor, 1))
state.ingest_external(kernel)
after = [c["code"] for c in kernel.list_cells()]
self.assertEqual(len(after), len(before) + 1)
self.assertIn("agent_note = 1", after[-1])
# the editor writes on its next save: the agent's cell is still in the file
kernel.save_cell(kernel.list_cells()[0]["id"], before[0] + "\n# touched")
self.assertIn("agent_note = 1", path.read_text())
finally:
if state.kernel is not None:
state.kernel.stop()