diff --git a/AGENTS.md b/AGENTS.md index 6581fd5e..157f9ea9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ Open work lives in a Product Grid project in `pgm/data` (project `srcly-backlog` - To add an op: a Zod contract in `engine/src/srcly/contracts/` (and its `registry.ts` entry) → the op in `engine/src/srcly/ops/` and its worker branch → `cd engine && pnpm emit-contracts` → a Pydantic mirror in `server/app/services/dataflow/models.py` (`tests/test_dataflow_models.py` checks its fields against the schema) → a route in `server/app/routers/dataflow.py` → the client call. The emitted `server/app/services/dataflow/contracts/*.schema.json` and `client/src/dataflow/contract.generated.ts` are never edited by hand; `engine/test/srcly/contracts-current.test.ts` fails when they drift. Then `cd server && SRCLY_UPDATE_SNAPSHOTS=1 uv run pytest tests/test_dataflow_outcomes.py` writes the `tests/snapshots/fake-engine.*.json` files that `client/src/dataflow/shapes.test.ts` reads. - Every result carries `generation` and `programHash`. A query with an older `expectedGeneration` is `stale`, a named outcome; never mix generations. - Not analyzed is never 0: show the outcome or reason by name. -- Python roots: `engine/python/srcly_pyfacts.py` (stdlib `ast` + mypy, run with srcly's own interpreter) emits facts in the collector's raw shape; roles and boundaries come from the element kind, so new Python recognition goes in the helper's tables, never in `engine/src/analysis`. Its tests need an interpreter with mypy (`SRCLY_PYTHON`, else `server/.venv` after `cd server && uv sync`). Decision: `docs/intent/python-dataflow/decision.md`. +- Python roots: `engine/python/srcly_pyfacts.py` (stdlib `ast` + mypy, run with srcly's own interpreter) emits facts in the collector's raw shape; roles and boundaries come from the element kind, so new Python recognition goes in the helper's tables, never in `engine/src/analysis`. Its tests need an interpreter with mypy (`SRCLY_PYTHON`, else `server/.venv` after `cd server && uv sync`). Why `ast` + mypy: `docs/dataflow-coverage.md` (Python type source). - Coverage survey: `engine/scripts/survey-blockers.ts` traces about 120 exports of one repo both ways and writes every outcome and stop as JSON; `engine/scripts/survey-report.ts ` ranks the stops by how many traces each cuts short. - Any script that builds a Program on a real repo (the survey, `pnpm memory-probe`, an ad-hoc spike) runs under `node --max-old-space-size=4096`, one repo per process, one process at a time across all sessions: a big repo's Program takes 2–3 GB, and parallel runs swap the Mac. `~/Projects/visual-notes` is the standard big-repo benchmark. - A packaging change is done when the wheel works with dev mode off: `bash build-srcly.sh`, then `uv run --no-project --with ./server/dist/srcly-*.whl pytest scripts/test_package.py` (CI runs the same). diff --git a/docs/CODE_MAP.md b/docs/CODE_MAP.md index 53c6d63c..d7118041 100644 --- a/docs/CODE_MAP.md +++ b/docs/CODE_MAP.md @@ -453,7 +453,7 @@ The tsx-data-flow analysis lives in `engine/`, a standalone Node package with it | Fixture projects (Solid route, compact field proof, bad-ish Solid, Python FastAPI) | `engine/examples/` | | Python handler routes in Data flow mode: the route summary's `handler` (name, line) opens the handler in anchor mode from the picker or a `flow=` link; Python name reads are `transparent` (walked through, not drawn) | engine `contracts/routes.ts`, `ops/routes.ts`, `ops/anchor-graph.ts` (`walk`); client `viz/deps/flow/FlowView.tsx` (`handlerOf`, `pickRoute`); helper `expr_Name` | | Language host: what the language-neutral ops (`routes`, `selectExpression`, `valueFlow`, `anchorGraph`) ask a program besides its evidence (files, candidates, parts, identifier on a line, text, checked type, owning function, call sites, and `evidence` — whether its facts pair calls (T13) and mark real dead ends (R16)); `Tier1` is `TsTier1 \| PythonTier1`, `requireTypescript` guards the TypeScript-only ops (route graph and surface, route path, function effects, file exports, legacy report, phase profiler); engine flags `--language`, `--python` | `engine/src/srcly/host.ts`, `typescript-host.ts`, `python/host.ts`, `python/session.ts`, `session.ts`; tests `engine/test/srcly/python-session.test.ts`, server `tests/test_dataflow_availability.py`, `tests/test_dataflow_smoke.py` (FastAPI example and srcly's own server over HTTP) | -| Python evidence provider: `ast` + mypy → raw ProgramElement / ProgramRelation facts, selectable expressions with checked types, function and class scopes (a class is an entry its constructor calls relate to by `definition`), call sites, FastAPI / Flask routes (one JSON document per generation); newer syntax than the interpreter's (3.14's `except A, B:`) rewritten before parsing; loaded behind the fact index by an endpoint-map relation provider; refusals `python-not-found`, `python-checker-not-found`, `no-python-files` | `engine/python/srcly_pyfacts.py` (origins and terminals from the `py` rows of `engine/models/python.json`, passed with `--models`); `engine/src/srcly/python/run.ts` (spawn, probe, named errors), `facts.ts` (wire format, `PythonRelationProvider`); fixtures `engine/examples/python-fastapi/`, `engine/examples/python-anchors/` (every def and class anchored by name); tests `engine/test/srcly/python-facts.test.ts`, `python-anchors.test.ts`; decision and numbers `docs/intent/python-dataflow/decision.md` | +| Python evidence provider: `ast` + mypy → raw ProgramElement / ProgramRelation facts, selectable expressions with checked types, function and class scopes (a class is an entry its constructor calls relate to by `definition`), call sites, FastAPI / Flask routes (one JSON document per generation); newer syntax than the interpreter's (3.14's `except A, B:`) rewritten before parsing; loaded behind the fact index by an endpoint-map relation provider; refusals `python-not-found`, `python-checker-not-found`, `no-python-files` | `engine/python/srcly_pyfacts.py` (origins and terminals from the `py` rows of `engine/models/python.json`, passed with `--models`); `engine/src/srcly/python/run.ts` (spawn, probe, named errors), `facts.ts` (wire format, `PythonRelationProvider`); fixtures `engine/examples/python-fastapi/`, `engine/examples/python-anchors/` (every def and class anchored by name); tests `engine/test/srcly/python-facts.test.ts`, `python-anchors.test.ts`; decision and numbers in `docs/dataflow-coverage.md` (Python type source) | | Field proof acceptance runner | `engine/scripts/accept-route-field-proof.ts` | ## Tests diff --git a/docs/dataflow-coverage.md b/docs/dataflow-coverage.md index cb7babb3..022de27c 100644 --- a/docs/dataflow-coverage.md +++ b/docs/dataflow-coverage.md @@ -47,6 +47,21 @@ The work came from the data-flow experience audit (October 2026). The audit trac - State: a zustand store (a `store` row) holds what its function returns and what its `set` receives; a selector reads the field it names, and calling a member read through the hook (`setCount` after `useStore((s) => s.setCount)`) runs the store's function, never the selector it is written beside. a setter's value reaches its getter's reads (#182); a function handed to a setter (`setCount((c) => c + step)`, a store path's `setState("todos", (list) => […])`) stands for what it returns, and its parameter is the current state; a Solid store path (`setStore("left", "title", v)`) writes `left.title` alone, so a sibling field never sees it, an updater there (inline or named) receives that field, not the whole store, and a predicate in the path (`(item) => item.id === id`) is a test, never stored; a store or signal's initial value seeds it field by field, so `createStore({ left, right })` keeps `left` and `right` apart; a state-pair row marked `lazy` (React's `useState`) calls a function initial value, and Solid's `createSignal` holds it; a `useReducer` dispatch binds the reducer's action, and the reducer's return is the state; state-pair factory and getter calls are framework-modelled (#188), modelled in the collector (#196). - Ends: a binding every read of which decides a branch (`if (forgotten)` in nested branches, `ready && `, ``, through `!` or an accessor's call) ends at its binding, decides a branch. A parameter or local binding the program never reads (`_level`, `const { isSuccess } = useQuery()`) ends there, named "never read": the checker proves the value goes nowhere, so the trace is `ok`, not a dead end; the value-flow ledger draws every in-place end (a branch test, a write, never read) as a terminal row, as the graph does. A service write or read is a method on a client's contract, or a function-valued field a declaration file declares (a generated client's); a callback prop the project declares (`props.save(x)`) is never one. A thrown value ends at the throw (#207). An exported value that nothing in the program reads ends at its export, the module's API, not at a dead end; a call through a callback an export nothing in the program calls was handed (`onSeen(id)`, `props.onPick(x)` in a component nothing renders) ends there too, at the module's API, never a library method on a parameter (`items.forEach`); one the program reads, by name or through `import * as`, walks on to that read. A function handed to a timer (`setTimeout`, `requestAnimationFrame`) ends there: the callback runs later. A function handed to a page element as a handler (`