diff --git a/docs/CODE_MAP.md b/docs/CODE_MAP.md index ab469f0e..d12fcacb 100644 --- a/docs/CODE_MAP.md +++ b/docs/CODE_MAP.md @@ -345,6 +345,15 @@ Plan: `docs/intent/symbol-graph/`. Backlog area `area-symbol-graph` in `pgm/data | Folder contents on graph boxes (children by LOC as a colored strip; "Largest inside" in the hover card) | `viz/deps/metrics.ts` (`viewContents`, `stripSegments`); `ContentsStrip` and `NodeCard` in `viz/deps/DependencyCanvas.tsx` | | Local dependency-graph fixtures for layout tests (gitignored `__fixtures__/local/`) | `scripts/capture-dep-graph.sh` | +Dependency view design: + +- The server emits one language-neutral graph at the finest level it can resolve: a folder → file → symbol hierarchy plus edges flagged type-only, test, external or barrel. TypeScript and Python differ only in how they fill this contract. +- The client does all roll-up and budgeting (about 30–50 labeled boxes). A drill stays local and makes no request. The server caches the graph with the analysis. +- Groups come from folders and module boundaries, never from auto-clustering. Focus sits in the center, dependents to the left and dependencies to the right. Extra columns come from layering inside a side, still one hop out. +- External packages, tests and type-only imports are hidden by default. Barrels and `__init__` re-exports pass through to the defining module. External packages roll up by package, not by import string. +- Layout is our own grid and channel router in three pure stages: view model, grid layout, render. ELK was tried and dropped, because its packing and bundling could not be steered. Boxes never move after layout, because moving them is what made edges cross boxes. +- Edge thickness or a count shows aggregation, not color. + Graph legibility rules (keep these when you change the layout or canvas): - Speed comes first. No legibility change may slow loading, focus or drill. diff --git a/docs/initiative-history.json b/docs/initiative-history.json index b9c46cd5..eed010d7 100644 --- a/docs/initiative-history.json +++ b/docs/initiative-history.json @@ -56,5 +56,18 @@ ], "revision": "0bfb05ba", "follow_ups": [] + }, + { + "title": "Multi-scale dependency view", + "slug": "dependency-view", + "outcome": "completed", + "retired_at": "2026-11-12", + "summary": "A multi-scale dependency view replaced the legacy graph: one language-neutral folder → file → symbol contract for TypeScript and Python, client-side roll-up, a custom grid and channel layout, and symbol-level focus. Reconciled against origin/main 0bfb05ba.", + "reason": "All five milestones were marked completed and their PRs merged on 2026-10-15 (#14, #16, #18, #21, #26).", + "documentation": [ + "docs/CODE_MAP.md" + ], + "revision": "0bfb05ba", + "follow_ups": [] } ] diff --git a/docs/intent/dependency-view/implementation-plan.md b/docs/intent/dependency-view/implementation-plan.md deleted file mode 100644 index f6eae2f5..00000000 --- a/docs/intent/dependency-view/implementation-plan.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -title: "Multi-scale dependency view — implementation plan" -slug: "dependency-view" -phase: plan -status: current -last_updated: "2026-10-14" ---- - -# Multi-scale dependency view — implementation plan - -## Plan at a glance - -The riskiest part is the drawing, not the data. If the view can't produce orthogonal, bundled, box-avoiding layouts in ≤50-box focus views, better import resolution won't help. So Milestone 1 builds the whole new client pipeline on the TS `/dependencies` payload we already have, as fixture JSON, and treats routing quality as automated geometry tests rather than eyeballing. Only then does the server change, in three steps that each add one unknown: a language-neutral hierarchical contract with full TS resolution, then Python, then symbol-level usage. The old `DependencyGraph.tsx` stays behind a toggle until the new view reaches parity, then it's deleted. - -Pushback against the obvious route: -- Don't fix the 2,000-line component in place. Its post-layout repositioning and super-node hacks are the problem. -- Don't start with Python. Python resolution is tractable work. Routing quality is the unknown that could change the architecture. - -## Implementation strategy - -- **First proof:** On the srcly repo, the new view shows a folder overview and a three-sided focus on `metricsStore.tsx`, with 0 edges crossing boxes. -- **Primary seams:** - - `GraphContract` (the finest-level hierarchical graph) feeds the client view model. - - `layoutView(viewModel) → positioned geometry` hides the layout engine behind one function. -- **Fast local loop:** `pnpm --dir client test -- deps` runs pure-function tests (roll-up, budget, layout, geometry checks) against checked-in fixture JSON. `cd server && uv run pytest -k dependency_graph` covers the server. -- **Local dependencies:** none; the layout is our own code. The fixtures are captured from srcly itself plus small synthetic repos under `server/tests/fixtures/`. -- **Visual confirmation:** screenshots of the running app at the overview and both focus fixtures, at 1440 and 1024 px wide. -- **Rollout and rollback:** a "New dependency view" toggle in `TreemapHeader`. The old component and the `/dependencies` endpoint stay untouched until Milestone 5. - -## Milestone 1: New focus view renders today's TS data with clean routing - -This proves the layout and routing core and the multi-scale interaction. It deliberately excludes new backend data. Symbols, Python, and edge flags come later. - -- **Change — client view-model module (`client/src/viz/deps/`)** - - `hierarchy.ts`: derive the folder → file tree from file-node paths, collapsing single-child folder chains. - - `viewModel.ts`: given a focus and a budget, choose visible nodes. The focus goes in the center, 1-hop incoming nodes on the left and outgoing on the right. Neighbors that don't fit roll up to their nearest folder, and aggregate edges carry a count. - - Vitest on fixtures: no visible node is both itself and an ancestor; every real edge maps to exactly one visible edge; the box count stays within budget. -- **Change — layout and routing (`layout.ts`, `bundle.ts`)** - - A custom grid + channel layout (decided in the spike below): uniform boxes in globally aligned rows, regions folded into columns to fit the viewport, edges routed only through channels between columns and gutters between rows, shared tracks bundled. - - Partitioning puts incoming layers, then the focus, then outgoing layers in order, and each side layers by its own internal edges. - - Routing is orthogonal with `mergeEdges`. - - A post-pass offsets coincident segments 1–2px. Nodes are never moved after layout. - - `geometry.ts` checks feed the tests: edge-segment/box intersections must be 0, plus the crossing count and the aspect-fill ratio. -- **Change — `DependencyView.tsx` behind a toggle** - - Uniform boxes with truncated labels and a hover tooltip, faint neighbor↔neighbor edges, and count badges on aggregate edges. - - Click to drill or recenter, breadcrumbs, and pan/zoom that only magnifies. - - The initial focus comes from `currentRoot().path`. -- **Spike — is ELK good enough? (resolved: no — custom layout)** - - ELK layered (partitions, Coffman–Graham, mergeEdges) never crossed boxes, but tuning packing and bundling turned into endless parameter fiddling; aspect fit and trunk bundling were not controllable. - - Replaced with `layout.ts`: grid columns per region with depth layering folded to the viewport, barycenter row ordering, channel/gutter routing with crossing-cost track ordering, and 1–2px bundle offsets. Zero box crossings holds by construction and is still asserted in tests; layout runs in ~1–10ms. - -### Desired end state - -- The toggle shows the new view on srcly. The overview and focus views stay under 50 boxes and read left to right. -- Geometry tests pass with 0 node crossings on all fixtures. -- The spike decision is recorded in the initiative map. - -## Milestone 2: Language-neutral graph contract with full TS resolution - -- **Change — `GET /api/analysis/dependency-graph`** - - Pydantic models in `server/app/models.py` cover hierarchy nodes (folder, file, symbol) and finest edges with `kind` flags: `type_only`, `test`, `external`, and `via_barrel`. Edges always point importer → imported. - - Cached through `app/services/cache.py`, keyed like the analysis cache. -- **Change — TS extractor (`server/app/services/deps/typescript.py`)** - - Merge the duplicate tsconfig helpers from `routers/analysis.py` and `focus_overlay.py`. - - Cover all tsconfig `paths` entries and `.js/.jsx/.mjs` files. - - Follow `export *` and named re-exports through barrels to the file that defines the symbol. - - Detect `import type`, key externals by package name, and classify test files by path or name convention. - - Pytest fixtures cover each case. -- **Change — client consumes the contract** - - Default filters hide externals, tests, and type-only imports, with UI to toggle them back on. - -### Desired end state - -- The new view runs on the new endpoint, and barrel imports land on the file that defines the symbol. -- All M1 geometry tests still pass when run on contract-shaped fixtures. - -## Milestone 3: Python fills the same contract - -- **Change — Python extractor (`server/app/services/deps/python.py`)** - - Detect package roots: `pyproject` and `setup.cfg` package dirs, `src/` layouts, `__init__.py` chains, and top-level modules. - - Resolve relative imports and absolute imports into repo modules. - - For `from pkg import name`, decide whether `name` is a submodule or a symbol. - - Pass `__init__` re-exports through to the defining module, flag `TYPE_CHECKING` blocks as `type_only`, and treat `conftest`, `test_*`, and `tests/` as test code. - - Pytest covers a synthetic repo with each layout, plus srcly's `server/` as a real-world check. - -### Desired end state - -- The dependency view on `server/` shows `routers → services → models` seams with correct aggregate counts. - -## Milestone 4: Focus expands into symbols with function-level usage - -- **Change — symbol edges in the contract** - - Add symbol nodes and edges from the using function to the imported symbol. - - TS reuses the scope and usage resolution in `focus_overlay.FocusAnalyzer`. Python gets an equivalent tree-sitter scope walk that handles `import x` with `x.attr` access and aliases. -- **Change — the view** - - A focused file expands into symbol rows. - - The other side shows using functions while the budget allows, otherwise it falls back to file boxes with bundled edges. - - Test on the `metricsStore.tsx` fixture, where `Explorer.tsx › SortHeader()` → `HOTSPOT_METRICS` appears. - -### Desired end state - -- Symbol-level focus works for both languages, with the M1 geometry checks still passing. - -## Milestone 5: Retire the old view - -- **Change** — Make the new view the default, then delete `DependencyGraph.tsx`, the toggle, and the old `/dependencies` route, after checking the CLI reports don't use them. - -### Desired end state - -- One dependency view, and no dead code. - -## Open decisions and spikes - -- **Python package-root detection:** if heuristics misfire on unusual layouts, the fallback is a `srcly` config entry listing roots. Decide after the M3 fixtures. - -## Below the cut line - -- Symbol explorer and search for popular symbols. The contract keeps symbols as first-class nodes so this view can reuse it later. -- Multi-hop flows, call chains, and continuous semantic zoom. -- Incremental indexing for 10k+ file repos. -- Cross-language edges (TS → Python API calls) and CSS imports. -- Animated transitions and storing the focus in the URL. diff --git a/docs/intent/dependency-view/initiative-map.json b/docs/intent/dependency-view/initiative-map.json deleted file mode 100644 index 28f4ea46..00000000 --- a/docs/intent/dependency-view/initiative-map.json +++ /dev/null @@ -1,658 +0,0 @@ -{ - "schema_version": "0.5.0", - "initiative": { - "title": "Multi-scale dependency view", - "slug": "dependency-view", - "scale": "capability", - "current_phase": "execution", - "last_updated": "2026-10-15", - "summary": "Replace the crude TS-only file dependency graph with a multi-scale, focus-centered view for TypeScript and Python: folders/modules with bundled edges when zoomed out, accurate edges when zoomed in, 30-50 labeled items on screen, uniform boxes, rectilinear routing that never crosses nodes.", - "status": "active", - "next_action": "Review and merge the stacked PRs (#14 first); then mark initiative completed" - }, - "artifacts": { - "intent_brief": { - "path": "intent-brief.md", - "status": "current", - "updated_at": "2026-10-14" - }, - "brainstorm": { - "path": "brainstorm.md", - "status": "not-requested", - "updated_at": null - }, - "shape_brief": { - "path": "shape-brief.md", - "status": "current", - "updated_at": "2026-10-14" - }, - "implementation_plan": { - "path": "implementation-plan.md", - "status": "current", - "updated_at": "2026-10-14" - } - }, - "sources": [ - { - "id": "req-1", - "type": "request", - "location": "conversation", - "anchor": "initial request 2026-10-14", - "authority": "direct current intent" - }, - { - "id": "repo-server-deps", - "type": "repository", - "location": "server/app/routers/analysis.py", - "anchor": "get_dependencies", - "authority": "current implementation reality" - }, - { - "id": "repo-client-graph", - "type": "repository", - "location": "client/src/components/DependencyGraph.tsx", - "anchor": "layoutGraph / filterGraph / super-node logic", - "authority": "current implementation reality" - }, - { - "id": "repo-python", - "type": "repository", - "location": "server/app/services/python/python_analysis.py", - "anchor": "import counting only", - "authority": "current implementation reality" - }, - { - "id": "qa-round-1", - "type": "question-round", - "location": "conversation", - "anchor": "1", - "authority": "direct current intent" - }, - { - "id": "qa-round-1b", - "type": "question-round", - "location": "conversation", - "anchor": "1 (re-asked)", - "authority": "direct current intent" - }, - { - "id": "qa-round-2", - "type": "question-round", - "location": "conversation", - "anchor": "2", - "authority": "direct current intent" - }, - { - "id": "qa-round-3", - "type": "question-round", - "location": "conversation", - "anchor": "3", - "authority": "direct current intent" - } - ], - "intent": { - "claims": [ - { - "id": "C1", - "statement": "Zoomed out, the view shows folders/modules with bundled aggregate edges; spaghetti must be avoided.", - "kind": "requirement", - "strength": "must", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C2", - "statement": "When zoomed to something specific, render accurate edges to every relevant place.", - "kind": "requirement", - "strength": "must", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C3", - "statement": "Aim for 30-50 labeled items on screen; smaller items may render unlabeled.", - "kind": "requirement", - "strength": "should", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C4", - "statement": "Focus-centered layout: whatever the user expressed interest in sits in the center.", - "kind": "requirement", - "strength": "should", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C5", - "statement": "Flows render left-to-right or top-to-bottom; call-chain/hop rendering is deferred. Current goal is immediate seams.", - "kind": "scope", - "strength": "should", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C6", - "statement": "Uniform box sizes, clean text, rectilinear edges that never cross nodes, minimized crossings, bundled edges sharing a path (1-2px apart).", - "kind": "requirement", - "strength": "must", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C7", - "statement": "Must work for both TypeScript and Python.", - "kind": "requirement", - "strength": "must", - "sources": [ - "req-1" - ], - "confidence": "high" - }, - { - "id": "C8", - "statement": "Zoomed out, collapse many real edges into a few aggregate edges showing broad themes; detail appears only as the user zooms in.", - "kind": "requirement", - "strength": "must", - "sources": [ - "qa-round-1" - ], - "confidence": "high" - }, - { - "id": "C9", - "statement": "Grouping levels come from folders or module boundaries (not auto-clustering).", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-1" - ], - "confidence": "high" - }, - { - "id": "C10", - "statement": "Focused layout is three columns: dependents left, focus center, dependencies right (resolves X1).", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-1b" - ], - "confidence": "high" - }, - { - "id": "C11", - "statement": "Detail changes by click-to-drill: clicking a folder expands it and makes it the focus, splitting its bundled edges; breadcrumbs go back out; scroll only magnifies.", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-1b" - ], - "confidence": "high" - }, - { - "id": "C12", - "statement": "The focused item expands into symbols. On the other side of each edge, show the specific symbols involved when they fit; otherwise keep them bundled as files.", - "kind": "requirement", - "strength": "must", - "sources": [ - "qa-round-1b" - ], - "confidence": "medium" - }, - { - "id": "C13", - "statement": "Focus is set from the treemap selection or by clicking in the graph.", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-1b" - ], - "confidence": "high" - }, - { - "id": "C14", - "statement": "Future: a symbol-focused explorer/search (e.g. popular or commonly used symbols) that the treemap cannot offer. Not in the first version, but the design must not preclude it.", - "kind": "later", - "strength": "later", - "sources": [ - "qa-round-1b" - ], - "confidence": "high" - }, - { - "id": "C15", - "statement": "The other side of focus edges shows the specific function/symbol in the importing file that uses the focus symbol (usage tracking in bodies, TS and Python).", - "kind": "requirement", - "strength": "must", - "sources": [ - "qa-round-2" - ], - "confidence": "high" - }, - { - "id": "C16", - "statement": "Fill the screen with nodes. More than 3 columns is allowed: incoming on the left, outgoing on the right, positioned by interconnectedness. Group by folder or aggregate only when it still does not fit. Goal: exploring how ALL things interconnect.", - "kind": "requirement", - "strength": "must", - "sources": [ - "qa-round-2" - ], - "confidence": "medium" - }, - { - "id": "C17", - "statement": "Neighbor-to-neighbor edges (not touching focus) are drawn faintly.", - "kind": "decision", - "strength": "should", - "sources": [ - "qa-round-2" - ], - "confidence": "high" - }, - { - "id": "C18", - "statement": "Hidden/collapsed by default: external packages, tests, type-only imports; barrel/__init__ re-exports are passed through to the defining file.", - "kind": "decision", - "strength": "should", - "sources": [ - "qa-round-2" - ], - "confidence": "high" - }, - { - "id": "C19", - "statement": "Extra columns come from layering within each side: still 1-hop only, and a dependent that imports another dependent sits further out.", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-3" - ], - "confidence": "high" - }, - { - "id": "C20", - "statement": "Target stays ~30-50 boxes, packed to the screen aspect ratio. Use as many columns as needed (e.g. 5); do not waste width holding to 3 strict columns; the focus column can be narrow when it holds one node.", - "kind": "requirement", - "strength": "must", - "sources": [ - "qa-round-3" - ], - "confidence": "high" - }, - { - "id": "C21", - "statement": "First proof: layout + routing on existing TS data (folder overview, drill, focus columns, orthogonal bundled routing). Then Python resolution, then symbol usage.", - "kind": "decision", - "strength": "must", - "sources": [ - "qa-round-3" - ], - "confidence": "high" - }, - { - "id": "C22", - "statement": "Must stay fast on ~2k-file repos; graph built on first open and cached with the existing analysis cache.", - "kind": "constraint", - "strength": "must", - "sources": [ - "qa-round-3" - ], - "confidence": "high" - } - ], - "themes": [ - "multi-scale aggregation", - "focus+context", - "routing quality", - "language coverage" - ], - "conflicts": [ - { - "id": "X1", - "statement": "Focus-centered (radial-ish) layout vs. directional L->R/T->B flow.", - "status": "resolved: three columns (C10)" - }, - { - "id": "X2", - "statement": "Fill the screen with nodes (C16) vs. 30-50 labeled items (C3).", - "status": "resolved: ~30-50 boxes packed to aspect ratio across as many layered columns as needed (C20)" - } - ], - "human_questions": [] - }, - "brainstorm": { - "status": "not-requested", - "requested_count": 7, - "options": [], - "selected_option": null - }, - "shape": { - "status": "current", - "appetite": "Replacement dependency view plus one language-neutral graph contract (TS + Python). No call-graph/data-flow beyond which function references an imported name.", - "problem": "The current view is a flat TS-only file graph laid out in one ELK pass with post-layout repositioning; it spaghettis, crosses nodes, and has no hierarchy, Python, or usage data.", - "recommended_shape": "Server emits finest-level hierarchical graph (folder>file>symbol, flagged edges); client does roll-up/budget view model -> ELK layered with partitions, fixed sizes, orthogonal routing, mergeEdges -> bundling offsets -> render. New component behind toggle until parity.", - "success_moves": [ - { - "id": "S1", - "move": "Split into three pure stages (view model, layout, routing/render) and never reposition after layout.", - "benefit": "Removes main cause of node-crossing edges; each stage fixture-testable in vitest." - }, - { - "id": "S2", - "move": "Client-side roll-up over one finest-level graph contract shared by TS and Python.", - "benefit": "Drill is local and instant; languages differ only in extraction." - }, - { - "id": "S3", - "move": "First proof on existing TS /dependencies data with no backend change.", - "benefit": "Proves the riskiest part (routing quality) before investing in Python/usage extraction." - }, - { - "id": "S4", - "move": "Exhaust ELK options (partitioning, mergeEdges, fixed sizes, orthogonal) before a custom router; libavoid-js/custom router is a gated spike.", - "benefit": "Avoids building a router unless evidence demands it." - }, - { - "id": "S5", - "move": "Automated geometry assertions: edge-node intersections = 0, crossing count, box count, aspect fill.", - "benefit": "Turns visual goals into regressions tests rather than eyeballing." - }, - { - "id": "S6", - "move": "Ship new view behind a toggle beside the old component.", - "benefit": "Rollback is a toggle; old view retired at parity." - } - ], - "first_proof": { - "question": "Can ELK layered (partitions, fixed sizes, orthogonal, mergeEdges) plus a bundling offset pass produce 0 node-crossing, bundled, <=50-box focus and overview layouts on real TS data?", - "artifact": "View model + rollup + ELK layout on current /dependencies data for srcly; focus fixtures client/src/components and metricsStore.tsx; geometry checks + screenshots", - "decision_rule": "0 node crossings and readable bundling at <=50 boxes -> keep ELK, proceed to graph contract + Python. Otherwise run the router spike before more work." - }, - "rabbit_holes": [ - "ELK layered may not pack sides to viewport aspect ratio; may need custom column wrapping", - "mergeEdges gives overlap, not 1-2px offset bundles; offset pass needed", - "Python absolute import resolution across package roots / src layouts", - "Function-level usage tracking (name references in bodies, aliasing, attribute access on module imports)", - "Compound focus (symbols inside file box) with ELK hierarchy handling" - ], - "open_decisions": [] - }, - "implementation": { - "status": "current", - "repository_snapshot": { - "root": "/Users/byronwall/Projects/srcly", - "revision": "2000a03", - "worktree_status": "dirty (App.tsx, Explorer.tsx, HotSpotItem.tsx)", - "subsystems": [ - "server/app/routers/analysis.py:get_dependencies", - "client/src/components/DependencyGraph.tsx" - ], - "test_commands": [ - "cd server && uv run pytest" - ], - "deployment_surfaces": [] - }, - "dependencies": [ - { - "id": "D1", - "name": "Custom grid + channel layout (replaced elkjs)", - "material": true, - "application_seam": "layoutView(viewModel) -> positioned geometry (client/src/viz/deps/layout.ts)", - "failure_modes": [ - "edges cross nodes", - "poor packing to aspect ratio", - "no 1-2px bundle offsets" - ], - "proof_rungs": [ - "geometry assertions on fixtures", - "screenshots in running app" - ], - "first_proof_role": { - "mode": "real", - "rationale": "The library itself is the risk under test." - }, - "local_proof": "vitest geometry checks", - "live_proof": "n/a (client-only)", - "rollout": "behind New dependency view toggle; swappable behind layoutView" - } - ], - "fast_local_loop": { - "command": "pnpm --dir client test -- deps; cd server && uv run pytest -k dependency_graph", - "starts": "nothing (pure functions over fixture JSON)", - "network": false, - "failure_injection": "synthetic fixtures (80-neighbor rollup, barrel chains, odd python layouts)", - "fidelity_proof": "screenshots of running app on srcly at 1440/1024px" - }, - "milestones": [ - { - "id": "m1", - "title": "New focus view renders today's TS data with clean routing", - "outcome": "Toggle shows folder overview + three-sided focus on srcly with 0 node crossings, <=50 boxes.", - "dependencies_introduced": [ - "D1" - ], - "proof": "vitest view-model + geometry checks on fixtures; screenshots", - "valid_intermediate_state": "New view behind toggle; old view default.", - "rollback": "Turn toggle off.", - "status": "completed", - "evidence": [ - "client/src/viz/deps/*.test.ts: 0 node crossings on 5 focus fixtures", - "Spike P1 resolved: ELK dropped for custom layout after user steer + measurements" - ] - }, - { - "id": "m2", - "title": "Language-neutral graph contract with full TS resolution", - "outcome": "New /dependency-graph endpoint with hierarchy and flagged importer->imported edges; barrels resolved; filters.", - "dependencies_introduced": [], - "proof": "pytest per resolution case; M1 geometry tests on contract fixtures", - "valid_intermediate_state": "Old /dependencies untouched.", - "rollback": "Point client back to adapted /dependencies payload.", - "status": "completed", - "evidence": [ - "server/tests/test_dependency_graph_ts.py (7 cases)", - "client tests on contract fixture srcly-client.json" - ] - }, - { - "id": "m3", - "title": "Python fills the same contract", - "outcome": "Python repos render with correct seams and counts.", - "dependencies_introduced": [], - "proof": "pytest on synthetic layouts + srcly server/", - "valid_intermediate_state": "Python simply absent if extractor disabled.", - "rollback": "Disable python extractor.", - "status": "completed", - "evidence": [ - "server/tests/test_dependency_graph_py.py (6 cases incl. srcly server)" - ] - }, - { - "id": "m4", - "title": "Focus expands into symbols with function-level usage", - "outcome": "Focused file shows symbols; other side shows using functions when budget allows.", - "dependencies_introduced": [], - "proof": "metricsStore fixture shows SortHeader -> HOTSPOT_METRICS; geometry checks pass", - "valid_intermediate_state": "Without symbol edges, view falls back to file-level.", - "rollback": "Omit symbol edges from contract.", - "status": "completed", - "evidence": [ - "server/tests/test_dependency_graph_symbols.py (4 cases)", - "client viewModel tests for symbol expansion and rollup" - ] - }, - { - "id": "m5", - "title": "Retire the old view", - "outcome": "Single dependency view; old component, toggle, and route removed.", - "dependencies_introduced": [], - "proof": "tests pass; reports unaffected", - "valid_intermediate_state": "n/a", - "rollback": "git revert", - "status": "completed", - "evidence": [ - "Legacy DependencyGraph.tsx, /dependencies route, helpers and elkjs removed; reporting migrated" - ] - } - ], - "cut_line": [ - "symbol explorer", - "multi-hop flows / call chains / semantic zoom", - "incremental indexing for 10k+ repos", - "cross-language and CSS edges", - "animations, focus in URL" - ], - "open_decisions": [] - }, - "history": [ - { - "date": "2026-10-14", - "event": "Initiative created from request plus repository survey." - }, - { - "date": "2026-10-14", - "event": "Shape brief written; ELK-first with gated router spike; first proof on existing TS data." - }, - { - "date": "2026-10-14", - "event": "Implementation plan written: 5 milestones, M1 client-only routing proof." - }, - { - "date": "2026-10-14", - "event": "M1 built. ELK tried, then replaced with custom grid+channel layout (user steer: avoid endless parameter tuning). Spike P1 resolved." - }, - { - "date": "2026-10-14", - "event": "M3 built: Python resolution via source-root detection (markers, src/, top-level package parents); P2 resolved without config fallback." - }, - { - "date": "2026-10-14", - "event": "All 5 milestones implemented as stacked PRs #14, #16, #18, #21 and M5; awaiting review/merge." - }, - { - "date": "2026-10-15", - "event": "2026-10-15 voiceover: follow-ups captured in dependency-graph-legibility (supersedes C6 uniform boxes for folders), unified-navigation (promotes animations/focus-in-URL from cut line), and dependency-and-treemap-quick-fixes." - } - ], - "closure": null, - "question_rounds": { - "focus": "dependency view intent", - "per_round": 5, - "planned_rounds": 3, - "status": "complete", - "rounds": [ - { - "round": 1, - "theme": "goals and stakes", - "source": "qa-round-1", - "questions": [ - { - "n": 1, - "question": "Focused layout: 3 columns / radial / layered T-B?", - "answer": "unanswered", - "effect": "carried to round 2" - }, - { - "n": 2, - "question": "Zoom model: click drill / continuous semantic / both?", - "answer": "Implied zoom drives aggregation (edges collapse when zoomed out)", - "effect": "carried to round 2 as understanding check" - }, - { - "n": 3, - "question": "Zoomed-out boxes: folders / language modules / auto-cluster?", - "answer": "Folders or module boundaries", - "effect": "C9" - }, - { - "n": 4, - "question": "Finest node: file / symbol / symbol-for-focus?", - "answer": "unanswered", - "effect": "carried to round 2" - }, - { - "n": 5, - "question": "(re-asked) layout/zoom/finest/focus source", - "answer": "3 columns; click to drill; focus expands to symbols, receiving side shows symbols if possible else files; focus from treemap + graph click; symbol explorer later", - "effect": "C10-C14" - } - ] - }, - { - "round": 2, - "theme": "behavior and limits", - "source": "qa-round-2", - "questions": [ - { - "n": 6, - "question": "Other side shows using function?", - "answer": "Yes, user symbol", - "effect": "C15" - }, - { - "n": 7, - "question": "Column overflow", - "answer": "Fill screen, >3 columns ok, position by interconnectedness, aggregate last", - "effect": "C16, X2" - }, - { - "n": 8, - "question": "Neighbor edges", - "answer": "Show faintly", - "effect": "C17" - }, - { - "n": 9, - "question": "Hidden by default", - "answer": "externals, tests, type-only, barrel pass-through", - "effect": "C18" - } - ] - }, - { - "round": 3, - "theme": "boundaries and confirmation", - "source": "qa-round-3", - "questions": [ - { - "n": 10, - "question": "What makes extra columns?", - "answer": "Layered within a side", - "effect": "C19" - }, - { - "n": 11, - "question": "Fill screen vs label cap", - "answer": "Keep 30-50 boxes; pack columns to aspect ratio, e.g. 5 columns; no wasted space for strict 3", - "effect": "C20, X2 resolved" - }, - { - "n": 12, - "question": "First proof", - "answer": "Layout + routing on TS", - "effect": "C21" - }, - { - "n": 13, - "question": "Repo size", - "answer": "~2k files", - "effect": "C22" - } - ] - } - ] - } -} diff --git a/docs/intent/dependency-view/intent-brief.md b/docs/intent/dependency-view/intent-brief.md deleted file mode 100644 index 93d5d280..00000000 --- a/docs/intent/dependency-view/intent-brief.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: "Multi-scale dependency view" -slug: "dependency-view" -phase: intent -status: current -last_updated: "2026-10-14" ---- - -# Multi-scale dependency view - -## My read - -You want to explore a codebase and understand how its parts interconnect, in both TypeScript and Python. Today the view is a single tangled file graph. The replacement should scale with attention. From far away it shows folders or modules joined by a few thick aggregate edges that tell the broad story. When you drill into something, it becomes the focus: it opens up into its symbols, and exact edges connect it to the specific functions that use it and the things it uses. The scope is **immediate seams** (direct, one-hop dependencies). Call chains and multi-hop flows are later work. - -## What matters most - -- **No spaghetti at any level.** Aggregate edges when zoomed out, and split them into real edges only when you drill in. -- **Accuracy at the focus.** The focus shows its symbols, and edges reach the actual using function on the other side, not just the importing file. -- **Visual discipline.** Uniform boxes, clean labels, orthogonal edges that never pass through a node, as few crossings as possible, and edges sharing a path bundled within 1–2px. -- **Use the screen.** About 30–50 boxes, packed to the screen's aspect ratio instead of three thin columns of empty space. - -## The experience you want - -1. The view opens on the treemap's current selection, or at the repo's top-level folders. Grouping follows folders and module boundaries, with aggregate edges between them. -2. Clicking a folder drills in: it expands, becomes the focus, and its bundled edges split. Breadcrumbs go back out. Scrolling only magnifies. -3. A focused view flows left to right. Incoming code (things that depend on the focus) is on the left, the focus is in the middle, and outgoing code (things it depends on) is on the right. Each side spreads into as many layers as its own interconnections need. For example, if dependent A imports dependent B, A sits further left. Everything is still one hop from the focus. A single focus node takes only a narrow column. -4. The focus expands into symbols. On the other side, edges land on the function that uses each symbol (`Explorer.tsx › SortHeader()` → `HOTSPOT_METRICS`) when there's room. Otherwise they stay bundled at file level. -5. Edges between neighbors that don't touch the focus are drawn faintly. -6. Folder roll-up and condensing happen only when a side still won't fit. -7. Clicking any node recenters the graph on it. - -## Boundaries - -### Must be true - -- Edges are rectilinear, never cross a node box, and are bundled when they share a path. -- All boxes are the same size. Text never sets the box width. -- Hidden or collapsed by default: external packages, tests, and type-only imports (TS `import type` and Python `TYPE_CHECKING`). Barrel/`__init__` re-exports pass through to the file that defines the symbol. -- TypeScript and Python reach parity, including symbol-level usage. -- Fast on repos of about 2k files. The graph is built once and cached. - -### Must be avoided - -- A whole-repo flat graph, or 100 real edges where one aggregate edge would do. -- Holding to a rigid three-column grid that wastes the screen. -- Edges routed through boxes, or fanned out and unbundled. - -## What seems settled - -- Grouping follows folders and module boundaries, not automatic clustering. -- Detail changes with click-to-drill, not continuous semantic zoom. -- The focus comes from the treemap selection or a click in the graph. -- The order of proof is (1) layout and routing on today's TS data, (2) Python resolution, (3) symbol-level usage. - -## Later, but don't preclude - -- A **symbol-focused explorer**: search and list symbols, perhaps surfacing popular or heavily used ones. The treemap can't offer this. The graph's data model should treat symbols as nodes from the start so this view can reuse it. -- Multi-hop flows and call-chain rendering. - -## Current reality that matters - -- **Python has no dependency graph.** `/api/analysis/dependencies` (`server/app/routers/analysis.py`) only walks `.ts/.tsx` files. The Python analyzer only counts imports. -- **TS resolution is partial.** It handles relative imports and the *first* tsconfig `paths` it finds. It skips `.js/.jsx`, ignores barrels and `export *`, and doesn't distinguish type-only imports. Externals are keyed by full import string, not by package. -- **There is no hierarchy or usage data.** The graph holds flat files plus export nodes, with no folders, no aggregate edges, and no record of which function uses an import. Edge direction is also inconsistent: file edges point importer→imported, while export edges point export→importer. -- **The client can't deliver the routing goals.** `DependencyGraph.tsx` (about 2k lines) runs one ELK layered layout over everything and hides hubs behind letter-coded "super nodes". It moves export pills *after* layout, so routed edges no longer line up with their boxes. Box sizes come from the labels. -- Nothing is cached. The graph is rebuilt on every request. - -## Next step after confirmation - -Shape: choose the layout and routing approach (packed layered sides, orthogonal routing with bundling, node avoidance) and the hierarchical graph contract. Prove it first on today's TS data. diff --git a/docs/intent/dependency-view/shape-brief.md b/docs/intent/dependency-view/shape-brief.md deleted file mode 100644 index 00e6d3cc..00000000 --- a/docs/intent/dependency-view/shape-brief.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: "Multi-scale dependency view — shape brief" -slug: "dependency-view" -phase: shape -status: current -last_updated: "2026-10-14" ---- - -# Multi-scale dependency view — what we are adding - -**Outcome:** You can explore how any part of a TS or Python codebase connects to the rest. You start from a folder overview with a few aggregate edges and drill into exact symbol-level seams, without ever seeing spaghetti. - -**Primary flow:** Open from the treemap selection → folder overview → click to drill → the focus sits center with incoming on the left and outgoing on the right → click a neighbor to recenter. - -## Feature scope - -```text -Dependency view (replaces DependencyGraph.tsx, behind a toggle until parity) -├── PLANNED ADDITIONS -│ ├── Overview: folders/modules with aggregate edges (weight = import count) -│ │ └── single-child folder chains collapse into one box (src/app/services) -│ ├── Focus view -│ │ ├── incoming side ◀ focus ▶ outgoing side, each side layered by its own internal imports -│ │ ├── focus file → its symbols; focus folder → its children -│ │ ├── other side → the using function when it fits, otherwise the file box -│ │ └── neighbor↔neighbor edges drawn faint -│ ├── Budget: ~30–50 boxes packed to the viewport's aspect ratio -│ │ └── over budget → roll up by folder, never "+N more" -│ ├── Drawing rules: one box size, truncated labels and full text on hover, -│ │ orthogonal edges, zero edges through boxes, shared paths bundled 1–2px apart -│ ├── Navigation: click to drill/recenter, breadcrumbs back out, scroll only magnifies -│ ├── Default filters: externals (one node per package), tests, type-only imports hidden; -│ │ barrel / __init__ re-exports resolved to the defining file -│ └── Language parity -│ ├── TS: all tsconfig paths, .js/.jsx, export * / barrels, import type -│ └── Python: absolute + relative imports, package roots / src layout, __init__, TYPE_CHECKING -└── LATER POSSIBILITIES - ├── symbol explorer: search and list symbols, surface popular ones - └── multi-hop flows and call chains -``` - -## Behavior - -| Situation | Expected result | -| --- | --- | -| A folder has 100 file-level imports into another | One thick edge with a count, which splits only when you drill into either end | -| A side has 80 direct neighbors | Neighbors roll up into their nearest folders until the view is under budget, and clicking a group drills into it | -| A dependent also imports another dependent | It moves one layer further out, and the edge between them is drawn faint | -| Several edges leave the same box for the same target | They share one path, offset 1–2px | -| An edge would cross a box | Routing goes around it, and this is checked automatically | -| The focus symbol is used by 3 functions in one file | The file shows those 3 function rows if the budget allows, otherwise one file box with a bundled edge | - -## Decisions and boundaries - -**Appetite:** A replacement view and a single graph contract for both languages. No call graph or data-flow analysis beyond "which function references an imported name". - -**Key decision:** -- **One finest-level graph, rolled up on the client.** The server emits a language-neutral graph: a folder → file → symbol hierarchy plus the finest edges it can resolve, each flagged as type-only, test, external, or barrel. The client does all aggregation and budgeting. TS and Python then only differ in how they fill the contract, and each drill stays local with no extra request. This fits the ~2k-file scale, cached once. -- **Our own grid + channel layout.** The pipeline runs in three pure stages: view model (budget and roll-up) → grid layout (regions folded into columns that fit the viewport, uniform boxes, edges only in channels and gutters, shared tracks for bundles) → render. ELK was tried first and dropped: it never crossed boxes but packing and bundling could not be steered without endless tuning. Boxes are never moved after layout. That post-layout repositioning is the main reason edges cross boxes today. - -**Boundary:** The current component stays in place until the new view reaches parity, so switching back is a toggle. Edge thickness or a count badge shows aggregation, not color. External packages are aggregated by package, never by import string. - -**First proof:** -- **Try:** On today's TS `/dependencies` data for srcly itself, build the view model, rollup, and layout with file-level focus only, with no backend change. Use `client/src/components` and `metricsStore.tsx` as the focus fixtures. -- **Observe:** Automated geometry checks report edges crossing boxes (must be 0), edge-crossing count, the number of boxes on screen, and aspect-ratio fill. Screenshots show the overview and both focus fixtures. -- **Decide:** 0 node crossings and readable bundling at ≤50 boxes → move on to the graph contract and Python. (Result: ELK failed the readability bar; the custom layout passed.) - -**Open choices:** None blocking. Defaults I'm taking: a focused folder shows its children (not symbols), and the aggregate edge count shows as a small badge. - -Intent: [intent-brief.md](intent-brief.md)