diff --git a/docs/CODE_MAP.md b/docs/CODE_MAP.md index 83ad5e83..53c6d63c 100644 --- a/docs/CODE_MAP.md +++ b/docs/CODE_MAP.md @@ -308,7 +308,14 @@ Graph-to-code rule: there is one `CodeModal`. Opening from the graph resolves th ### Symbol graph (server) -Plan: `docs/intent/symbol-graph/`. Backlog area `area-symbol-graph` in `pgm/data`. +Design rules: + +- Matching is name-based first, and every edge says how sure it is: `resolution` is `exact`, `typed` or `name`. The client draws `name` edges as partial proof. The intended upgrade is to re-resolve a neighbourhood with the data-flow engine's checker. +- The server holds the index. The client asks only for a neighbourhood (up to 4 hops). Past 400 symbols the answer says `truncated`, and the client offers filters. +- Symbol ids are `file#Qualified.name`. A top-level symbol's id equals the import graph's id, so `at`, Open code and Trace work on it. Members add `.member`. +- Symbols is a third mode of the graph view. It reuses the dependency canvas, layout, palette, node menus and rail through an adapter. Nothing new is drawn by hand. + +Open work: backlog area `area-symbol-graph` in `pgm/data`. | Feature | Where | | --- | --- | diff --git a/docs/initiative-history.json b/docs/initiative-history.json index 6d42b667..d5074084 100644 --- a/docs/initiative-history.json +++ b/docs/initiative-history.json @@ -152,5 +152,20 @@ ], "revision": "0bfb05ba", "follow_ups": [] + }, + { + "title": "Symbol graph", + "slug": "symbol-graph", + "outcome": "completed", + "retired_at": "2026-11-12", + "summary": "A symbol reference graph shipped for TypeScript and Python: a server index with a neighbourhood API, Symbols mode in the graph view, a path between two symbols, dead and hub overlays, and the HTTP seam between client requests and server routes. This folder had no initiative map; a minimal one was written only to run the retirement check. Reconciled against origin/main 0bfb05ba.", + "reason": "All six PRs of the shape brief's stack merged on 2026-11-05 (#348, #349, #352, #353, #354, #358).", + "documentation": [ + "docs/CODE_MAP.md" + ], + "revision": "0bfb05ba", + "follow_ups": [ + "Re-resolve name and typed edges with the data-flow engine's checker. The backlog area area-symbol-graph in pgm/data holds any open work." + ] } ] diff --git a/docs/intent/symbol-graph/intent-brief.md b/docs/intent/symbol-graph/intent-brief.md deleted file mode 100644 index 9cee7ad5..00000000 --- a/docs/intent/symbol-graph/intent-brief.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -title: "Symbol graph" -slug: "symbol-graph" -phase: intent -status: current -last_updated: "2026-11-05" ---- - -# Symbol graph - -## My read - -You want srcly to be the best tool for tracing how a TypeScript or Python codebase hangs together, and the import graph is not enough. It shows *module seams*: which file imports what, refined to the top-level declaration that uses it. It cannot show a method calling a method, a function calling another in the same file, who writes a piece of state, or a chain three calls deep. The data-flow engine traces *values* precisely, but it is heavy, lazy and needs Node, and a value trace answers a different question from "who calls this?". - -The missing layer is a **symbol graph**: every function, class, method, field, variable and type as a node, and each place one of them calls, instantiates, reads, writes, extends or names another as a typed edge with its line. It is always available (built by name from tree-sitter, cached like the import graph), honest about how sure it is, and drawn in the graph view you already have. - -## What matters most - -- **Members and intra-file links.** Methods are nodes with their class as container; nested functions with their parent. A call inside one file is an edge like any other. -- **Edges mean something.** `call`, `instantiate`, `read`, `write`, `type`, `extends`, `implements`, each with the lines it happens at. Filterable. -- **Honest resolution.** Every edge says `exact`, `typed` (through a declared type or `new`) or `name` (a unique member name, for calls only). A guess is drawn as a guess, never as a fact. -- **Follow the chain.** Counts on every neighbour; expand one hop in place without refocusing; find the path between two symbols. -- **Both languages.** Python reaches the same contract: methods through `self`/`cls`, class-qualified members, module attributes, `__init__` re-exports. -- **Fast.** The index of srcly itself (736 TS files) builds in about 6 s once and answers neighbourhoods in milliseconds; neighbourhoods are served, never the whole index. - -## The experience you want - -1. From the go-to palette, the treemap, the import graph's symbol boxes or a right-click in the code viewer, you land on a symbol in **Symbols** mode of the graph view: the symbol centred, what references it on the left, what it references on the right, framed by file and class. -2. Each neighbour shows its own in/out counts. Clicking a count pulls that neighbour's hop into the view; the camera stays put. -3. Edges are drawn by kind and sureness. A legend and filters (hide `type`, hide `read`, hide tests) keep a hub readable. -4. The rail lists the reference sites of the selected box or edge, one row per line, each opening the code viewer there. That list doubles as a call hierarchy. -5. Pick two symbols and the view draws the shortest reference path between them. -6. Symbols nothing references, and symbols everything references, are badged. -7. A client `fetch('/api/x')` joins the FastAPI handler of `/api/x` as one `http` edge, so a TS+Python repo reads as one graph. - -## Boundaries - -### Must be true - -- Available without Node and without the data-flow engine; the engine upgrades edges later, it never gates the view. -- Top-level symbol ids match the import graph's (`file#name`), so `at` and the existing navigation address both. -- A reference the extractor cannot resolve is dropped and counted, never attached to a plausible-looking target. -- Neighbourhoods, not whole-index payloads: a 2k-file repo has about 100k references. - -### Must be avoided - -- A flat force-directed symbol graph. The focus layout and its frames stay. -- Name-only resolution of property reads or of builtin member names (`.filter`, `.has`, `.push`): that is how a name-based index lies. -- A second toolbar or a second canvas. Symbols is the third mode beside Imports and Data flow. - -## What seems settled - -- Baseline by name first (tree-sitter on the server), engine-precise edges as a later upgrade carrying `resolution: checker`. -- Third graph mode, one adapter to the existing ViewModel, like Data flow mode. -- Members, nested functions and module-level code are nodes from the first version. - -## Current reality that matters - -- `services/deps/typescript_deps.py` attributes every use to the enclosing *top-level* declaration and skips same-file references (`emit_symbol_edges`). -- The engine's `linksOf` (`engine/src/srcly/typescript-host.ts`) already computes per-function calls, reads and writes with the checker; it is the natural source for the later precise upgrade. -- The anchor mode's symbol view (`viz/deps/flow/symbolView.ts`) draws a value trace as symbols; it is a different question (where a value goes) and stays as is. - -## Next step after confirmation - -Shape: the contract, the extractor's resolution rules, the API, and the stack of PRs. diff --git a/docs/intent/symbol-graph/shape-brief.md b/docs/intent/symbol-graph/shape-brief.md deleted file mode 100644 index f54cde5d..00000000 --- a/docs/intent/symbol-graph/shape-brief.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -title: "Symbol graph — shape brief" -slug: "symbol-graph" -phase: shape -status: current -last_updated: "2026-11-05" ---- - -# Symbol graph — what we are adding - -**Outcome:** From any symbol in a TS or Python repo you see who references it and what it references, by kind and by line, follow the chain hop by hop, and find the path to another symbol. - -**Primary flow:** go-to palette (or a symbol box, a treemap tile, a code line) → Symbols mode centred on the symbol → expand a neighbour's hop → open a reference line in the code viewer. - -## Feature scope - -```text -Symbol graph -├── Server index (services/symbols/) PR 1 -│ ├── Symbol: id file#Qualified.name, kind function|class|method|field|variable|type|enum|module, -│ │ container, line range, exported, typed_as (new X() / : X), members, bases -│ ├── Reference: source → target, kind call|instantiate|read|write|type|extends|implements, -│ │ line, resolution exact|typed|name -│ ├── TypeScript extractor (tree-sitter, scope stack, shadowing, barrels through TsProgram) -│ ├── Member settlement after all files: this/super, class-qualified, module attribute, -│ │ typed receiver, else a unique non-builtin member name for calls only -│ ├── Cache by file mtimes; GET /api/symbols/graph (neighbourhood, hops ≤ 4, kinds, direction, -│ │ tests), GET /api/symbols/search -│ └── Python extractor: same contract (self/cls, Class.member, module.attr, __init__ passes) PR 2 -├── Client: Symbols mode of the graph view PR 3 -│ ├── ?view=graph&symbols=; Imports / Data flow / Symbols switch -│ ├── adapter → ViewModel: focus centre, referrers left, referenced right, frames per file, -│ │ sub-frames per class; edge style by kind, dotted for `name` resolution -│ ├── in/out counts on every box; click a count → that hop opens in place (`open=` list) -│ ├── legend + filters (kinds, tests); rail lists reference lines, each opens the code viewer -│ └── entry points: go-to palette (server search), code viewer menu, treemap tile, import graph box -├── Path between two symbols (server BFS, drawn as a chain) PR 4 -├── Dead and hub overlays (inCount 0; top in-degree) as badges and a filter PR 5 -└── HTTP seam: client fetch('/api/x') ↔ FastAPI/Flask handler, kind `http` PR 6 -``` - -## Behaviour - -| Situation | Expected result | -| --- | --- | -| `this.save()` inside `Store.bump` | `Store.bump —call→ Store.save`, `exact` | -| `x.bump()` with `x: Store` | `—call→ Store.bump`, `typed` | -| `props.s.bump()` and exactly one member named `bump` | `—call→ Store.bump`, `name`, drawn dotted | -| `items.filter(...)` with an unknown receiver | Nothing drawn. Counted as unresolved only if some symbol is named `filter` | -| `super.row()` | The base class's `row`, never the subclass's override | -| `const api = { get() {} }` | `api.get` is a method node with `api` as container | -| Module-level `outer();` | Source is the file's `module` node | -| A focus with 400+ symbols within the hops | `truncated: true`; the client says so and offers filters | - -## Decisions and boundaries - -**Appetite:** one stack of six PRs; each lands on its own and the view is usable from PR 3. - -**Key decisions:** - -- **Name-based first, honest about it.** `resolution` is on every edge. The data-flow engine can later re-resolve a neighbourhood with the checker and replace `name`/`typed` edges with `checker` ones, the way Data flow swaps its surface for the complete graph. -- **Neighbourhood API, index on the server.** The whole index is tens of MB on a 2k-file repo; the view only ever needs a few hops. -- **Third mode, one adapter.** The dependency canvas, layout, go-to palette, node menus and rail are reused through an adapter like `flow/adapter.ts`; nothing new is drawn by hand. -- **Ids line up.** `file#name` for top-level symbols is the import graph's id, so `at`, Open code and Trace already work on them; members add `.member`. diff --git a/pgm/data/projects/srcly-backlog/nodes/area-symbol-graph.md b/pgm/data/projects/srcly-backlog/nodes/area-symbol-graph.md index 89c1749b..0c735cc3 100644 --- a/pgm/data/projects/srcly-backlog/nodes/area-symbol-graph.md +++ b/pgm/data/projects/srcly-backlog/nodes/area-symbol-graph.md @@ -4,4 +4,4 @@ label: "Symbol graph" type: area status: planned --- -A symbol reference graph (functions, classes, methods, fields, variables, types; call / instantiate / read / write / type / extends / implements edges with lines and resolution) built by name from tree-sitter, served as neighbourhoods, drawn as the Symbols mode of the graph view. Intent and shape: `docs/intent/symbol-graph/`. +A symbol reference graph (functions, classes, methods, fields, variables, types; call / instantiate / read / write / type / extends / implements edges with lines and resolution) built by name from tree-sitter, served as neighbourhoods, drawn as the Symbols mode of the graph view. Design rules: `docs/CODE_MAP.md` (Symbol graph). Closure record: `docs/initiative-history.json`. diff --git a/site/src/docs/pages/design.md b/site/src/docs/pages/design.md index c705aafd..be0cea9b 100644 --- a/site/src/docs/pages/design.md +++ b/site/src/docs/pages/design.md @@ -36,7 +36,7 @@ This prevents missing analysis from looking like improvement. Rankings can use d The TypeScript and TSX dependency graph shows imports. Data flow traces modeled values, function effects, or supported routes. An import edge does not prove value movement; a data-flow edge is not a general runtime trace. The [integration brief](https://github.com/byronwall/srcly/blob/main/docs/intent/tsx-dataflow-integration/intent-brief.md) and [logic visibility brief](https://github.com/byronwall/srcly/blob/main/docs/intent/dataflow-logic-visibility/intent-brief.md) explain the boundary. -Symbols mode shows which functions, classes, fields, or types reference a symbol. Edges carry a resolution label: exact, typed, or name-based. This gives more detail than imports without claiming value flow. See the [symbol graph brief](https://github.com/byronwall/srcly/blob/main/docs/intent/symbol-graph/intent-brief.md). +Symbols mode shows which functions, classes, fields, or types reference a symbol. Edges carry a resolution label: exact, typed, or name-based. This gives more detail than imports without claiming value flow. See the [symbol graph brief](https://github.com/byronwall/srcly/blob/0bfb05ba/docs/intent/symbol-graph/intent-brief.md). Data flow is optional. It runs through a bundled engine that requires Node.js 18 or newer. The engine's results carry outcomes and provenance, including partial, assumed, stale, and unavailable states. The rest of the scan does not depend on that engine. This keeps the base map usable when the optional analysis cannot run. Read [Data-flow analysis](../data-flow/) before relying on a trace.