From c47025cc7d37774784aea11cfb08a1e0f9acc784 Mon Sep 17 00:00:00 2001 From: Byron Wall Date: Thu, 12 Nov 2026 21:56:25 -0500 Subject: [PATCH 1/2] docs: retire python-dataflow All three planned slices merged on 2026-10-25 (#119, #120, #122). Move the durable facts into the lasting docs, add the closure record to docs/initiative-history.json, and remove the planning folder. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 2 +- docs/dataflow-coverage.md | 15 ++++ docs/initiative-history.json | 17 ++++ docs/intent/python-dataflow/decision.md | 115 ------------------------ 4 files changed, 33 insertions(+), 116 deletions(-) delete mode 100644 docs/intent/python-dataflow/decision.md diff --git a/AGENTS.md b/AGENTS.md index 6581fd5e..157f9ea9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ Open work lives in a Product Grid project in `pgm/data` (project `srcly-backlog` - To add an op: a Zod contract in `engine/src/srcly/contracts/` (and its `registry.ts` entry) → the op in `engine/src/srcly/ops/` and its worker branch → `cd engine && pnpm emit-contracts` → a Pydantic mirror in `server/app/services/dataflow/models.py` (`tests/test_dataflow_models.py` checks its fields against the schema) → a route in `server/app/routers/dataflow.py` → the client call. The emitted `server/app/services/dataflow/contracts/*.schema.json` and `client/src/dataflow/contract.generated.ts` are never edited by hand; `engine/test/srcly/contracts-current.test.ts` fails when they drift. Then `cd server && SRCLY_UPDATE_SNAPSHOTS=1 uv run pytest tests/test_dataflow_outcomes.py` writes the `tests/snapshots/fake-engine.*.json` files that `client/src/dataflow/shapes.test.ts` reads. - Every result carries `generation` and `programHash`. A query with an older `expectedGeneration` is `stale`, a named outcome; never mix generations. - Not analyzed is never 0: show the outcome or reason by name. -- Python roots: `engine/python/srcly_pyfacts.py` (stdlib `ast` + mypy, run with srcly's own interpreter) emits facts in the collector's raw shape; roles and boundaries come from the element kind, so new Python recognition goes in the helper's tables, never in `engine/src/analysis`. Its tests need an interpreter with mypy (`SRCLY_PYTHON`, else `server/.venv` after `cd server && uv sync`). Decision: `docs/intent/python-dataflow/decision.md`. +- Python roots: `engine/python/srcly_pyfacts.py` (stdlib `ast` + mypy, run with srcly's own interpreter) emits facts in the collector's raw shape; roles and boundaries come from the element kind, so new Python recognition goes in the helper's tables, never in `engine/src/analysis`. Its tests need an interpreter with mypy (`SRCLY_PYTHON`, else `server/.venv` after `cd server && uv sync`). Why `ast` + mypy: `docs/dataflow-coverage.md` (Python type source). - Coverage survey: `engine/scripts/survey-blockers.ts` traces about 120 exports of one repo both ways and writes every outcome and stop as JSON; `engine/scripts/survey-report.ts ` ranks the stops by how many traces each cuts short. - Any script that builds a Program on a real repo (the survey, `pnpm memory-probe`, an ad-hoc spike) runs under `node --max-old-space-size=4096`, one repo per process, one process at a time across all sessions: a big repo's Program takes 2–3 GB, and parallel runs swap the Mac. `~/Projects/visual-notes` is the standard big-repo benchmark. - A packaging change is done when the wheel works with dev mode off: `bash build-srcly.sh`, then `uv run --no-project --with ./server/dist/srcly-*.whl pytest scripts/test_package.py` (CI runs the same). diff --git a/docs/dataflow-coverage.md b/docs/dataflow-coverage.md index cb7babb3..022de27c 100644 --- a/docs/dataflow-coverage.md +++ b/docs/dataflow-coverage.md @@ -47,6 +47,21 @@ The work came from the data-flow experience audit (October 2026). The audit trac - State: a zustand store (a `store` row) holds what its function returns and what its `set` receives; a selector reads the field it names, and calling a member read through the hook (`setCount` after `useStore((s) => s.setCount)`) runs the store's function, never the selector it is written beside. a setter's value reaches its getter's reads (#182); a function handed to a setter (`setCount((c) => c + step)`, a store path's `setState("todos", (list) => […])`) stands for what it returns, and its parameter is the current state; a Solid store path (`setStore("left", "title", v)`) writes `left.title` alone, so a sibling field never sees it, an updater there (inline or named) receives that field, not the whole store, and a predicate in the path (`(item) => item.id === id`) is a test, never stored; a store or signal's initial value seeds it field by field, so `createStore({ left, right })` keeps `left` and `right` apart; a state-pair row marked `lazy` (React's `useState`) calls a function initial value, and Solid's `createSignal` holds it; a `useReducer` dispatch binds the reducer's action, and the reducer's return is the state; state-pair factory and getter calls are framework-modelled (#188), modelled in the collector (#196). - Ends: a binding every read of which decides a branch (`if (forgotten)` in nested branches, `ready && `, ``, through `!` or an accessor's call) ends at its binding, decides a branch. A parameter or local binding the program never reads (`_level`, `const { isSuccess } = useQuery()`) ends there, named "never read": the checker proves the value goes nowhere, so the trace is `ok`, not a dead end; the value-flow ledger draws every in-place end (a branch test, a write, never read) as a terminal row, as the graph does. A service write or read is a method on a client's contract, or a function-valued field a declaration file declares (a generated client's); a callback prop the project declares (`props.save(x)`) is never one. A thrown value ends at the throw (#207). An exported value that nothing in the program reads ends at its export, the module's API, not at a dead end; a call through a callback an export nothing in the program calls was handed (`onSeen(id)`, `props.onPick(x)` in a component nothing renders) ends there too, at the module's API, never a library method on a parameter (`items.forEach`); one the program reads, by name or through `import * as`, walks on to that read. A function handed to a timer (`setTimeout`, `requestAnimationFrame`) ends there: the callback runs later. A function handed to a page element as a handler (`