diff --git a/CHANGELOG.md b/CHANGELOG.md index 18cc895..fb462b6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Call-graph edge extractor** (`kb.extract.deterministic.calls`): the sixth deterministic + extractor — one `call_edge` artifact per RESOLVED caller→callee pair (`call:{caller}->{callee}`, + call-site lines aggregated in the payload, the import-edge precedent). Three deterministic + resolution tiers: same-module calls, imported-name calls (**cross-file** — `from x import f + [as g]` and `import x[.y] [as z]; x.y.f()` module-attribute forms, resolved via per-module import + tables incl. relative imports), and `self.method()` to a method of the same class. Precision-first: + only first-party-resolved edges are emitted; `obj.method(...)`, `getattr`/dynamic, `super()`, + inherited self-calls, star imports, decorator/default-arg expressions and class-body calls are + documented gaps. Grounded on the caller def span (role `caller`; module span for module-level + calls) + the callee def span (role `callee`); direct recursion is a single-span artifact; mutual + recursion yields two distinct artifacts (identity rule v2). `framework_versions` empty. Registered + in `default_extractors()`; `summarize` / `embed_text` gained a `call_edge` branch. The + deterministic foundation under the future business-process extractor (DESIGN §14 item 2). +- **Tier-1 call-graph HARD gate** (`kb.eval.tier1_calls_test`): extracted edges match a hand-labeled + oracle (11 edges across 5 modules); a mutually recursive pair stays two distinct artifacts over an + identical evidence span set (extractor-level identity-v2 regression); cross-file provenance (one + artifact spans the caller's and callee's files); six blind spots asserted as *known* gaps; nested + defs attribute to the innermost function. Headline HARD gates: twelve → **thirteen**. + ## [0.5.0] - 2026-07-07 ### Fixed diff --git a/DESIGN.md b/DESIGN.md index ca9a24b..523dea6 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -254,6 +254,12 @@ test.) artifact per handler because `artifact_id` is content-addressed over spans + extractor, §6). Grounded on the handler span + cross-file on the listened-to class; hand-labeled Tier-1 gate. Known gaps (asserted): call-form `event.listen(...)`, lifespan, pydantic-v1, dynamic names. +- **(e)** Call-graph edge extractor *(shipped)*: one `call_edge` per RESOLVED caller→callee pair, + three deterministic tiers (same-module; imported — **cross-file**; `self.` method of the same + class); precision-first — only first-party-resolved edges are emitted, recall bounded by + documented gaps (dynamic/attribute calls, inheritance); grounded on caller + callee def spans; + hand-labeled Tier-1 gate incl. a mutual-recursion identity-v2 regression. The deterministic + foundation for §14 item 2. - Single Postgres with the `≥1 derived_from` invariant enforced at write. - **Adversarial fixture:** an ungrounded artifact that the write-time check **must reject** — so the anti-hallucination discipline is tested in the MVP, not deferred with the LLM. @@ -357,7 +363,7 @@ freshness(current|stale@sha)`, with a deterministic tie-break for reproducible e | Module | Responsibility | Key tech | |--------|----------------|----------| | `kb.structural` | Parse Python without executing it; enumerate symbols/imports/call-sites with per-SHA byte/line ranges; compute content-addressed span identity; incremental reparse. Hidden behind a `StructuralIndex`/`PathEngine` interface so a SCIP backend can replace tree-sitter later. | tree-sitter + tree-sitter-python (canonical bindings) | -| `kb.extract.deterministic` | No-LLM extractors → exact artifacts (confidence=1.0): import graph; FastAPI API contract (static, cross-file grounded); domain entities (pydantic/dataclass/SQLAlchemy, static, cross-file links to referenced entities, hand-labeled gate); library public-API surface (static tree-sitter, cross-file `__init__` re-export resolution, independent griffe-oracle gate); event handlers (pydantic validators / FastAPI `on_event` / SQLAlchemy `listens_for`, static, cross-file target grounding, hand-labeled gate). | grimp, tree-sitter queries; griffe (dev-only oracle) | +| `kb.extract.deterministic` | No-LLM extractors → exact artifacts (confidence=1.0): import graph; FastAPI API contract (static, cross-file grounded); domain entities (pydantic/dataclass/SQLAlchemy, static, cross-file links to referenced entities, hand-labeled gate); library public-API surface (static tree-sitter, cross-file `__init__` re-export resolution, independent griffe-oracle gate); event handlers (pydantic validators / FastAPI `on_event` / SQLAlchemy `listens_for`, static, cross-file target grounding, hand-labeled gate); call-graph edges (per-edge artifacts, three resolution tiers, caller+callee span grounding, hand-labeled gate). | grimp, tree-sitter queries; griffe (dev-only oracle) | | `kb.introspect` | Eval-only runtime oracle: runs a FastAPI app in a network-blocked sandbox and emits `app.openapi()` for the Tier-1 API gate. Never on the index path. | subprocess sandbox, fastapi | | `kb.embed` | Replaceable embedding adapters + snapshot population for `search_knowledge`. Torch isolated behind the `embed` extra and a lazy import. | sentence-transformers (default), OpenAI (optional), pgvector | | `kb.rag` | Frozen pgvector RAG-over-source baseline — the "other arm" of the knowledge-vs-RAG A/B (no provenance/grounding). | deterministic line-window chunker, pgvector | @@ -404,6 +410,10 @@ Review fact-checked these against current (2026) sources. Caveats are first-clas velocity, small team, no tagged GitHub releases (v0.6.6 is the npm version), needs Node + an activated venv, and the Python consumer will likely compile the `.proto` itself. This *strengthens* keeping it deferred behind an interface. +- The shipped deterministic `call_edge` extractor takes the tree-sitter-call-sites road: it emits + **only resolved** edges (precision-first) with recall bounded by documented gaps (dynamic + dispatch, attribute calls, inheritance) — the "always quote recall, not just precision" rule; + PyCG-class resolvers stay deferred behind `PathEngine`. - **PyCG (call graph, deferred semantic layer)** — **archived/unmaintained**; ~99% precision is paired with only **~70% recall** (≈30% of real edges missed → "incomplete path" is the common case). Treat as a known-temporary default strictly behind `PathEngine`; evaluate maintained @@ -442,7 +452,9 @@ Review fact-checked these against current (2026) sources. Caveats are first-clas pydantic/FastAPI/SQLAlchemy, static tree-sitter, hand-labeled Tier-1 gate; call-form `event.listen` deferred). 2. The **one** grounded business-process extractor (named real path + labeler + validator + - deterministic sub-property gate). + deterministic sub-property gate). *Deterministic foundation shipped:* the `call_edge` extractor + + Tier-1 calls gate; `PathEngine` slicing, the sink registry, the labeler and the span-binding + validator are next. 3. Recursive invalidation (`artifact_depends_on`), multi-branch dedup, freshness precompute. 4. Embeddings + `search_knowledge` *(shipped v0.2)*; then `pg_search`/BM25 + RRF if vector ranking is insufficient. 5. ADR mining from git/PR history. diff --git a/README.md b/README.md index 36754f4..88fcc72 100644 --- a/README.md +++ b/README.md @@ -84,17 +84,17 @@ flowchart LR ## Status -**v0.5 — spine (artifact-identity rule v2), five deterministic extractors (cross-file grounding, incl. the library public-API surface and event handlers), LLM-grounded descriptions up to per-package architecture overviews, live incremental re-index (`kb watch`), MCP serving, the twelve knowledge gates, and published Docker images.** Everything here grounds what it claims, and nothing it cannot: +**v0.5 — spine (artifact-identity rule v2), six deterministic extractors (cross-file grounding, incl. the library public-API surface, event handlers, and call-graph edges), LLM-grounded descriptions up to per-package architecture overviews, live incremental re-index (`kb watch`), MCP serving, the thirteen knowledge gates, and published Docker images.** Everything here grounds what it claims, and nothing it cannot: - **Provenance spine** — content-addressed `span_id` (LOCKED); tree-sitter spans with a normalized S-expression fingerprint and per-SHA location; a single-Postgres, Alembic-managed store with content-addressed idempotent writes; the ≥ 1 `derived_from` anti-hallucination invariant enforced in-app *and* by a deferred DB trigger; pygit2 git ingest (no checkout) with a diff-based invalidation seed. -- **Deterministic extractors** — the **import / dependency graph** (grimp resolves the edge, tree-sitter grounds it on the exact import statement, with an honest `approximate` fallback for re-exports / relative / unmappable imports — never a silent loss); the **FastAPI API-contract** extractor, which grounds a single route **across files** (handler in `routes.py` + `response_model` class in `schemas.py`); the **domain-entity** extractor (pydantic / dataclass / SQLAlchemy classes and their fields, grounded on the class definition **and cross-file on the entities they reference** — purely static, with documented detection limits); the **library public-API-surface** extractor (what a package exposes from its `__init__.py` — `__all__`-authoritative, with `__init__` re-exports resolved **cross-file** to the defining function/class — validated against an independent **griffe** static oracle); and the **event-handler** extractor (pydantic `@field_validator`/`@model_validator`, FastAPI `@app.on_event`, SQLAlchemy `@event.listens_for` — one artifact per handler, grounded on the handler span and **cross-file** on the class it listens to; the call-form `event.listen(...)` and dynamic names are documented gaps). +- **Deterministic extractors** — the **import / dependency graph** (grimp resolves the edge, tree-sitter grounds it on the exact import statement, with an honest `approximate` fallback for re-exports / relative / unmappable imports — never a silent loss); the **FastAPI API-contract** extractor, which grounds a single route **across files** (handler in `routes.py` + `response_model` class in `schemas.py`); the **domain-entity** extractor (pydantic / dataclass / SQLAlchemy classes and their fields, grounded on the class definition **and cross-file on the entities they reference** — purely static, with documented detection limits); the **library public-API-surface** extractor (what a package exposes from its `__init__.py` — `__all__`-authoritative, with `__init__` re-exports resolved **cross-file** to the defining function/class — validated against an independent **griffe** static oracle); the **event-handler** extractor (pydantic `@field_validator`/`@model_validator`, FastAPI `@app.on_event`, SQLAlchemy `@event.listens_for` — one artifact per handler, grounded on the handler span and **cross-file** on the class it listens to; the call-form `event.listen(...)` and dynamic names are documented gaps); and the **call-graph edge** extractor (deterministic caller→callee edges — same-module, imported **cross-file**, and `self.` method calls — one `call_edge` per resolved pair with call-site lines aggregated; `obj.method(...)`, `getattr`, inherited self-calls and decorator/default-arg expressions are documented gaps; the deterministic foundation under the future business-process extractor). - **`kb introspect`** — a sandboxed, network-blocked `app.openapi()` oracle, eval-only and never on the index path, that the API gate scores the static contract against. - **Read-only MCP server** — `find_provenance`, `get_knowledge`, and `search_knowledge`, each returning provenance-carrying units (method + confidence + freshness). - **pgvector embeddings + semantic search** — a replaceable embedding provider (sentence-transformers by default, OpenAI optional) populated by a separate `kb embed` pass; torch stays out of the index path. - **A frozen RAG-over-source baseline** and the **Tier-3 knowledge-vs-RAG recall gate** — the honest A/B that backs the "knowledge > RAG" thesis. - **LLM-grounded descriptions** — an optional, key-gated `kb describe` pass has an LLM write NL summaries for routes, entities, modules (per file), and **packages** (a per-package architecture overview that synthesizes the import graph + public surface + member-module summaries, grounded on the package's own and its direct submodules' spans); every claim is validated against the target's own spans by a deterministic sub-property gate, so ungrounded claims are *dropped* (the anti-hallucination invariant, with a model in the loop). Stored as `extraction_method = "llm_grounded"`, grounded on code spans. - **Incremental re-index** — `kb index --parent ` reuses unchanged files' spans from the parent snapshot and parses only the diff; extraction stays full, so the result is identical to a full re-index (a HARD gate proves it). Auto-detects the parent; falls back to full when none is indexed. -- **Twelve HARD CI eval gates** (see [Development](#development)). +- **Thirteen HARD CI eval gates** (see [Development](#development)). - **A nightly LLM-judged A/B** (optional, key-gated, **non-gating**) — an answerer LLM answers each question from knowbase's grounded context vs a RAG-over-source context, and a judge LLM scores **answer accuracy** (against hand-written gold) + **hallucination**. Tracked metrics on top of recall; it never blocks CI. @@ -120,7 +120,7 @@ The base `--extra dev` install stays torch-free; the `embed` extra pulls sentenc ### Run the gates ```bash -uv run pytest src/kb/eval -q # the twelve HARD gates (spins an ephemeral local Postgres) +uv run pytest src/kb/eval -q # the thirteen HARD gates (spins an ephemeral local Postgres) ``` ### Index a commit @@ -233,6 +233,7 @@ A Python package `kb` (uv, src-layout). Modules and their responsibilities: | `kb.extract.deterministic.fastapi_contract` | Static FastAPI API-contract extractor; grounds a route across files (handler + `response_model` class), never imports user code. | | `kb.extract.deterministic.entities` | Static domain-entity extractor — pydantic / dataclass / SQLAlchemy classes + their fields, grounded on the class definition **and, across files, on the entities they reference** (field types + `relationship()`); detection signals and limits recorded in the payload. | | `kb.extract.deterministic.library_surface` | Static library public-API-surface extractor — one `public_symbol` per name a package exposes from its `__init__.py` (`__all__`-authoritative), `__init__` re-exports resolved **cross-file** to the defining function/class; validated by an independent griffe static oracle. Never imports user code. | +| `kb.extract.deterministic.calls` | Static call-graph edge extractor — one `call_edge` per resolved caller→callee pair (same-module / imported **cross-file** / `self.` method), call-site lines aggregated; only resolved first-party edges are emitted; dynamic/attribute/inherited calls are documented gaps. Never imports user code. | | `kb.extract.deterministic.events` | Static event-handler extractor — one `event_handler` per handler carrying pydantic validator / FastAPI `on_event` / SQLAlchemy `listens_for` decorator registrations (stacked decorators in `payload.registrations`), grounded on the handler span **and, across files, on the class it listens to**; call-form `event.listen(...)` and dynamic names are documented gaps. | | `kb.introspect` | Sandboxed, network-blocked `app.openapi()` oracle — eval-only ground truth for the API gate, never on the index path. | | `kb.mcp` | Read-only MCP server and its provenance-carrying records: `find_provenance`, `get_knowledge`, `search_knowledge`. | @@ -240,7 +241,7 @@ A Python package `kb` (uv, src-layout). Modules and their responsibilities: | `kb.rag` | The frozen pgvector RAG-over-source baseline — the "other arm" of the knowledge-vs-RAG A/B (no provenance, no grounding). | | `kb.extract.semantic` | LLM-grounded extraction (`kb describe`): NL descriptions of routes/entities/modules **and per-package architecture overviews** (rich synthesis from the import graph + public surface + member summaries, grounded on code spans) with a deterministic sub-property gate (`grounding.validate_claims`) that drops any claim not backed by the target's spans. Separate key-gated pass; never on `index`. | | `kb.daemon.cli` | The `kb` CLI: `index` (full or `--incremental`/`--parent`), `watch` (poll a local branch ref, per-commit incremental catch-up, `branch_ref` cursor), `migrate`, `embed`, `describe`, `serve` (MCP), and `introspect` — all functional. | -| `kb.eval` | Twelve HARD CI gates (identity reproducibility, adversarial grounding, Tier-1 import oracle, Tier-1 API oracle, Tier-1 entities oracle, Tier-1 library-surface oracle, Tier-1 events oracle, Tier-3 knowledge-vs-RAG recall, Tier-4 one-hop invalidation, invariants, semantic grounding floor, incremental re-index equivalence) plus the supporting MCP / embed / store suite. | +| `kb.eval` | Thirteen HARD CI gates (identity reproducibility, adversarial grounding, Tier-1 import oracle, Tier-1 API oracle, Tier-1 entities oracle, Tier-1 library-surface oracle, Tier-1 events oracle, Tier-1 call-graph oracle, Tier-3 knowledge-vs-RAG recall, Tier-4 one-hop invalidation, invariants, semantic grounding floor, incremental re-index equivalence) plus the supporting MCP / embed / store suite. | Core tables: `commit_ref`, `branch_ref`, `code_span`, `span_occurrence`, `artifact` (now with `embedding vector(384)` + `embedding_model_id`), `artifact_derived_from`, `snapshot_entry`, and `rag_chunk` (the baseline arm). @@ -250,10 +251,10 @@ Core tables: `commit_ref`, `branch_ref`, `code_span`, `span_occurrence`, `artifa uv sync --extra dev # venv + install uv run ruff check src/kb # lint uv run mypy # strict type-check -uv run pytest src/kb/eval -q # the twelve HARD eval gates +uv run pytest src/kb/eval -q # the thirteen HARD eval gates ``` -CI (GitHub Actions, workflow **"CI"**, `.github/workflows/ci.yml`) runs ruff, `mypy --strict`, and the eval gates against a `pgvector/pgvector:pg17` service (with the embedding model cached). The **twelve HARD gates** that block a merge: +CI (GitHub Actions, workflow **"CI"**, `.github/workflows/ci.yml`) runs ruff, `mypy --strict`, and the eval gates against a `pgvector/pgvector:pg17` service (with the embedding model cached). The **thirteen HARD gates** that block a merge: 1. **Identity reproducibility** — formatting / comment / docstring / location changes must NOT change `span_id`; a rename MUST. Pure identity core, no database. 2. **Adversarial grounding** — an ungrounded artifact is rejected by *both* layers (the app's `GroundingError` and the DB's deferred `artifact_grounded_check` trigger); a genuinely grounded artifact commits cleanly. @@ -262,11 +263,12 @@ CI (GitHub Actions, workflow **"CI"**, `.github/workflows/ci.yml`) runs ruff, `m 5. **Tier-1 entities oracle** — extracted pydantic / dataclass / SQLAlchemy entities + their fields match a hand-labeled oracle, each grounded on its class span; a bare declarative `Base` is correctly *not* an entity and a `create_model(...)` model is asserted as a *known* gap. 6. **Tier-1 library-surface oracle** — the statically-extracted public API surface of a package (`__all__`-authoritative, `__init__` re-exports resolved **cross-file**) equals an independent **griffe** static oracle; underscore-private names are excluded, and a third-party / dynamic-`__all__` re-export is grounded-but-flagged, asserted as a *known* gap. 7. **Tier-1 events oracle** — extracted event handlers (pydantic validators, FastAPI `on_event`, SQLAlchemy `listens_for`) match a hand-labeled oracle, each grounded on its handler span and **cross-file** on the class it listens to; the call-form `event.listen(...)` and a dynamic `@app.on_event(EVENT)` name are asserted as *known* gaps. -8. **Tier-3 knowledge-vs-RAG recall** — knowbase cross-file recall@k == 1.0 for every cross-file question (API contracts **and** domain entities: in each case one artifact already spans both files, so the floor is *structural*, independent of embedding quality); the RAG arm is reported but **never asserted**, so a model bump can't redden CI. -9. **Tier-4 one-hop invalidation** — a content diff invalidates *exactly* the artifacts whose grounding span changed (set-equality: no over-invalidation, no stale survivors); a version bump invalidates everything. -10. **Invariants** — zero orphans (every snapshot artifact is grounded), and re-indexing the same SHA yields the identical set of artifact ids. -11. **Semantic grounding floor** — the LLM-grounded describer's claims are validated against the artifact's own spans by a deterministic sub-property gate; an adversarial fabricated claim is *dropped*, never stored (run on a stub LLM, so it gates without an API key). -12. **Incremental re-index equivalence** — an incremental re-index (reuse unchanged files' spans from the parent, parse only the diff) yields the *identical* `{logical_key: artifact_id}` snapshot as a full re-index of the same tree, and the parse is provably skipped for unchanged files (counter assertions); a missing/unindexed parent falls back to full. +8. **Tier-1 call-graph oracle** — extracted call edges match a hand-labeled oracle (same-module, cross-file imported — incl. aliases and module-attribute forms — `self.` method calls, and class instantiation), each grounded on the caller **and** callee def spans (cross-file); a mutually recursive pair yields two distinct artifacts (identity-v2 regression); `obj.method(...)`, `getattr`, inherited self-calls and decorator-expression calls are asserted as *known* gaps. +9. **Tier-3 knowledge-vs-RAG recall** — knowbase cross-file recall@k == 1.0 for every cross-file question (API contracts **and** domain entities: in each case one artifact already spans both files, so the floor is *structural*, independent of embedding quality); the RAG arm is reported but **never asserted**, so a model bump can't redden CI. +10. **Tier-4 one-hop invalidation** — a content diff invalidates *exactly* the artifacts whose grounding span changed (set-equality: no over-invalidation, no stale survivors); a version bump invalidates everything. +11. **Invariants** — zero orphans (every snapshot artifact is grounded), and re-indexing the same SHA yields the identical set of artifact ids. +12. **Semantic grounding floor** — the LLM-grounded describer's claims are validated against the artifact's own spans by a deterministic sub-property gate; an adversarial fabricated claim is *dropped*, never stored (run on a stub LLM, so it gates without an API key). +13. **Incremental re-index equivalence** — an incremental re-index (reuse unchanged files' spans from the parent, parse only the diff) yields the *identical* `{logical_key: artifact_id}` snapshot as a full re-index of the same tree, and the parse is provably skipped for unchanged files (counter assertions); a missing/unindexed parent falls back to full. The identity rules in `kb.ids` (and `kb.structural`) are **LOCKED**: changing one is a breaking change, gated behind a `NORMALIZATION_VERSION` / `extractor_version` bump so existing digests are invalidated rather than silently colliding. diff --git a/src/kb/daemon/watch.py b/src/kb/daemon/watch.py index 4710bc3..e9c23fd 100644 --- a/src/kb/daemon/watch.py +++ b/src/kb/daemon/watch.py @@ -26,6 +26,7 @@ from kb.daemon.pipeline import IndexResult, index_commit from kb.extract.base import Extractor +from kb.extract.deterministic.calls import CallGraphExtractor from kb.extract.deterministic.entities import EntityExtractor from kb.extract.deterministic.events import EventExtractor from kb.extract.deterministic.fastapi_contract import FastAPIExtractor @@ -44,6 +45,7 @@ def default_extractors() -> list[Extractor]: EntityExtractor(), EventExtractor(), LibrarySurfaceExtractor(), + CallGraphExtractor(), ] diff --git a/src/kb/embed/text.py b/src/kb/embed/text.py index 493c570..92efd0e 100644 --- a/src/kb/embed/text.py +++ b/src/kb/embed/text.py @@ -57,6 +57,16 @@ def embed_text(kind: str, payload: dict[str, Any]) -> str: "fields " + " ".join(str(f) for r in regs for f in r.get("fields", [])), ] return " ".join(p for p in parts if p.strip()) + if kind == "call_edge": + parts = [ + head, + f"call {payload.get('caller', '')} {payload.get('callee', '')}", + f"caller module {payload.get('caller_module', '')}", + f"callee module {payload.get('callee_module', '')}", + f"resolution {payload.get('resolution', '')}", + f"callee kind {payload.get('callee_kind', '')}", + ] + return " ".join(p for p in parts if p.strip()) if kind == "description": claims = " ".join(str(c.get("text", "")) for c in payload.get("claims", [])) return f"{payload.get('summary', '')} {claims}".strip() or head diff --git a/src/kb/eval/tier1_calls_test.py b/src/kb/eval/tier1_calls_test.py new file mode 100644 index 0000000..11f96cd --- /dev/null +++ b/src/kb/eval/tier1_calls_test.py @@ -0,0 +1,251 @@ +"""HARD GATE — Tier 1: call-graph edges vs a hand-labeled oracle (DESIGN.md §9, §14 item 2). + +Fully static (tree-sitter re-parse; never imports/executes user code), precision-first: only edges +resolved to a first-party def span are emitted, and every documented blind spot — ``obj.method``, +``getattr``, ``super()``, inherited ``self`` calls, decorator-expression calls — is asserted as a +*known* gap, never a silent wrong guess. The mutual-recursion pair is an extractor-level regression +of identity rule v2 (``kb.ids.ARTIFACT_ID_VERSION``): two edges over the identical evidence span +set must stay two distinct artifacts. +""" + +from __future__ import annotations + +from pathlib import Path + +from sqlalchemy import Engine, func, select + +from kb.daemon.pipeline import index_commit +from kb.eval._fixtures import make_git_repo +from kb.extract.deterministic.calls import CallGraphExtractor +from kb.store import models as m +from kb.store.queries import provenance_for_artifact + +FILES = { + "src/cg/__init__.py": "", + "src/cg/util.py": ( + "def helper():\n return 1\n\n\n" + "def ping():\n return pong()\n\n\n" + "def pong():\n return ping()\n\n\n" + "def loop(n):\n return loop(n)\n" + ), + "src/cg/models.py": ( + "class Order:\n" + " def total(self):\n return self.compute()\n\n" + " def compute(self):\n return 0\n\n\n" + "class Rush(Order):\n" + " def go(self):\n return self.total()\n\n" + " def parent_total(self):\n return super().total()\n" + ), + "src/cg/svc.py": ( + "import cg.util\n" + "from cg.models import Order\n" + "from cg.util import helper as h\n\n\n" + "def build():\n" + " order = Order()\n" + " total = h()\n" + " return h(), order, total\n\n\n" + "def fanout():\n return cg.util.helper()\n\n\n" + "def local():\n return build()\n" + ), + "src/cg/rel.py": ( + "from .util import pong\n\n\n" + "def relay():\n return pong()\n" + ), + "src/cg/edge.py": ( + "import cg.util as u\n\n\n" + "def deco(fn):\n return fn\n\n\n" + "SEED = u.helper()\n\n\n" + "def use(order):\n return order.compute()\n\n\n" + "def dyn():\n f = getattr(u, 'helper')\n return f()\n\n\n" + "@deco(u.ping())\n" + "def wrapped():\n return 0\n\n\n" + "def outer():\n" + " def inner():\n return u.helper()\n" + " return inner\n" + ), +} + +# Hand-labeled oracle: (caller_fq, callee_fq, resolution). +EXPECTED_EDGES = { + ("cg.util.ping", "cg.util.pong", "same_module"), + ("cg.util.pong", "cg.util.ping", "same_module"), + ("cg.util.loop", "cg.util.loop", "same_module"), + ("cg.models.Order.total", "cg.models.Order.compute", "self"), + ("cg.svc.build", "cg.models.Order", "imported"), + ("cg.svc.build", "cg.util.helper", "imported"), + ("cg.svc.fanout", "cg.util.helper", "imported"), + ("cg.svc.local", "cg.svc.build", "same_module"), + ("cg.rel.relay", "cg.util.pong", "imported"), + ("cg.edge", "cg.util.helper", "imported"), + ("cg.edge.outer.inner", "cg.util.helper", "imported"), +} +# Deliberate blind spots — callers that must produce NO edge (obj.method, getattr, a decorator +# expression, inherited self-call, super(), and the outer function whose only call is nested). +KNOWN_GAP_CALLERS = { + "cg.edge.use", + "cg.edge.dyn", + "cg.edge.wrapped", + "cg.models.Rush.go", + "cg.models.Rush.parent_total", + "cg.edge.outer", +} + + +def _index(engine: Engine, tmp_path: Path) -> str: + sha = make_git_repo(tmp_path, [FILES])[0] + index_commit( + engine, str(tmp_path), sha, extractors=[CallGraphExtractor()], first_party_root="src" + ) + return sha + + +def _payloads(engine: Engine, sha: str) -> list[dict]: + join = m.snapshot_entry.join( + m.artifact, m.artifact.c.artifact_id == m.snapshot_entry.c.artifact_id + ) + with engine.connect() as conn: + return list( + conn.execute( + select(m.artifact.c.payload) + .select_from(join) + .where(m.snapshot_entry.c.sha == sha, m.artifact.c.kind == "call_edge") + ).scalars() + ) + + +def test_call_edges_match_oracle(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + extracted = {(p["caller"], p["callee"], p["resolution"]) for p in _payloads(engine, sha)} + assert extracted == EXPECTED_EDGES + + +def test_mutual_recursion_distinct_artifacts(engine: Engine, tmp_path: Path) -> None: + """Identity-v2 regression at the extractor level: two edges, one evidence span set.""" + sha = _index(engine, tmp_path) + join = m.snapshot_entry.join( + m.artifact, m.artifact.c.artifact_id == m.snapshot_entry.c.artifact_id + ) + with engine.connect() as conn: + rows = conn.execute( + select(m.snapshot_entry.c.logical_key, m.snapshot_entry.c.artifact_id, + m.artifact.c.payload) + .select_from(join) + .where( + m.snapshot_entry.c.sha == sha, + m.snapshot_entry.c.logical_key.in_( + ["call:cg.util.ping->cg.util.pong", "call:cg.util.pong->cg.util.ping"] + ), + ) + ).all() + by_key = {r.logical_key: r for r in rows} + assert len(by_key) == 2 # both directions survive + assert len({bytes(r.artifact_id) for r in rows}) == 2 # distinct ids, identical span set + assert by_key["call:cg.util.ping->cg.util.pong"].payload["caller"] == "cg.util.ping" + assert by_key["call:cg.util.pong->cg.util.ping"].payload["caller"] == "cg.util.pong" + + +def test_instantiation_edge(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + order = next( + p for p in _payloads(engine, sha) + if p["caller"] == "cg.svc.build" and p["callee"] == "cg.models.Order" + ) + assert order["callee_kind"] == "class" # instantiation is a real dependency edge + assert order["caller_kind"] == "function" + assert order["resolution"] == "imported" + + +def test_module_caller_and_merged_lines(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + by_edge = {(p["caller"], p["callee"]): p for p in _payloads(engine, sha)} + seed = by_edge[("cg.edge", "cg.util.helper")] + assert seed["caller_kind"] == "module" # a module-level call grounds on the module span + assert len(seed["lines"]) == 1 + build = by_edge[("cg.svc.build", "cg.util.helper")] + assert len(build["lines"]) == 2 # two h() sites merged into ONE artifact + assert build["lines"] == sorted(build["lines"]) + + +def test_self_call_grounded_on_same_class_method(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + key = "call:cg.models.Order.total->cg.models.Order.compute" + with engine.connect() as conn: + prov = {(p.file_path, p.role) for p in provenance_for_artifact(conn, sha, key)} + assert prov == {("src/cg/models.py", "caller"), ("src/cg/models.py", "callee")} + + +def test_cross_file_provenance(engine: Engine, tmp_path: Path) -> None: + """The thesis property: ONE artifact spans the caller's and the callee's files.""" + sha = _index(engine, tmp_path) + key = "call:cg.svc.build->cg.models.Order" + with engine.connect() as conn: + prov = {(p.file_path, p.role) for p in provenance_for_artifact(conn, sha, key)} + assert ("src/cg/svc.py", "caller") in prov + assert ("src/cg/models.py", "callee") in prov + + +def test_known_gaps_not_extracted(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + payloads = _payloads(engine, sha) + assert all(p["caller"] not in KNOWN_GAP_CALLERS for p in payloads) + # the decorator-expression call u.ping() produced nothing: pong stays ping's only caller + ping_callers = {p["caller"] for p in payloads if p["callee"] == "cg.util.ping"} + assert ping_callers == {"cg.util.pong"} + + +def test_nested_def_attribution(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + callers = {p["caller"] for p in _payloads(engine, sha)} + assert "cg.edge.outer.inner" in callers # the call attributes to the NESTED function ... + assert "cg.edge.outer" not in callers # ... never to its enclosing one (no-descend rule) + + +def test_every_call_edge_is_grounded(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + join = m.snapshot_entry.outerjoin( + m.artifact_derived_from, + m.artifact_derived_from.c.artifact_id == m.snapshot_entry.c.artifact_id, + ).join(m.artifact, m.artifact.c.artifact_id == m.snapshot_entry.c.artifact_id) + with engine.connect() as conn: + ungrounded = conn.execute( + select(func.count()) + .select_from(join) + .where( + m.snapshot_entry.c.sha == sha, + m.artifact.c.kind == "call_edge", + m.artifact_derived_from.c.artifact_id.is_(None), + ) + ).scalar() + assert ungrounded == 0 + join2 = ( + m.snapshot_entry.join( + m.artifact, m.artifact.c.artifact_id == m.snapshot_entry.c.artifact_id + ) + .join( + m.artifact_derived_from, + m.artifact_derived_from.c.artifact_id == m.artifact.c.artifact_id, + ) + .join(m.code_span, m.code_span.c.span_id == m.artifact_derived_from.c.span_id) + ) + with engine.connect() as conn: + rows = conn.execute( + select(m.artifact.c.payload, m.code_span.c.span_kind, m.artifact_derived_from.c.role) + .select_from(join2) + .where(m.snapshot_entry.c.sha == sha, m.artifact.c.kind == "call_edge") + ).all() + assert rows + for row in rows: + assert row.payload["span_mapping"] == "exact" + if row.role == "caller": + assert row.span_kind in ("module", "function", "method") + else: + assert row.role == "callee" + assert row.span_kind in ("function", "class", "method") + + +def test_direct_recursion_single_span(engine: Engine, tmp_path: Path) -> None: + sha = _index(engine, tmp_path) + with engine.connect() as conn: + prov = provenance_for_artifact(conn, sha, "call:cg.util.loop->cg.util.loop") + assert len(prov) == 1 # caller == callee span: one grounding row (PK-safe) ... + assert prov[0].role == "caller" # ... keeping the caller role diff --git a/src/kb/extract/deterministic/calls.py b/src/kb/extract/deterministic/calls.py new file mode 100644 index 0000000..8dae0e0 --- /dev/null +++ b/src/kb/extract/deterministic/calls.py @@ -0,0 +1,348 @@ +"""Deterministic call-graph edge extractor (DESIGN.md §8; the §14-item-2 foundation). + +Produces one ``call_edge`` artifact per RESOLVED (caller -> callee) pair — ``call:{caller_fq}-> +{callee_fq}`` — with the call-site line numbers aggregated in ``payload.lines`` (the import_edge +precedent). Three deterministic resolution tiers, precision-first (only edges resolved to a +first-party def span are emitted; never a wrong guess): + +* ``same_module`` — a bare ``f(...)`` naming a top-level function/class of the same module; +* ``imported`` — CROSS-FILE: ``from x import f [as g]; g(...)`` and module-attribute calls after + ``import x[.y] [as z]`` (``x.y.f(...)``, ``z.f(...)``), resolved via a per-module import table; +* ``self`` — ``self.method(...)`` inside a method, resolved to a method of the SAME class. + +Grounded on the caller def span (role ``caller``; a module-level call grounds on the module span) +plus the callee def span (role ``callee``, cross-file possible). Direct recursion is a single-span +artifact (one edge row, role ``caller`` — the ``(artifact_id, span_id)`` PK permits one row per +span). Mutual recursion yields two distinct artifacts (identity rule v2 folds ``logical_key`` into +``artifact_id``). ``framework_versions`` is deliberately empty: a call edge does not depend on any +framework version (the import-graph rationale). + +Fully static — re-parses span text with tree-sitter; never imports or executes user code. Documented +gaps (recall is bounded, surfaced by the eval gate — DESIGN.md §12): arbitrary ``obj.method(...)``; +dynamic calls (``getattr``, computed callables, the outer link of ``super().m()`` / ``a().b()``); +inherited ``self.method()``; star imports; re-export indirection; calls to non-first-party targets +(skipped — not first-party knowledge); decorator expressions, parameter-default expressions and +class-body statements (def-time, not call-time; decorator wiring belongs to the fastapi/events +extractors); the import table is file-scoped (no scope modeling); ``cls.method()`` / +``ClassName.method(...)`` receivers. +""" + +from __future__ import annotations + +import textwrap +from collections.abc import Iterator +from dataclasses import dataclass, field +from pathlib import Path + +import tree_sitter_python as tsp +from tree_sitter import Language, Node, Parser + +from kb.extract.base import DerivedEdge, ExtractContext, ExtractedArtifact +from kb.structural.interface import ParsedSpan + +EXTRACTOR_ID = "calls" +EXTRACTOR_VERSION = "1" + +_LANGUAGE = Language(tsp.language()) +_IMPORT_TYPES = frozenset({"import_statement", "import_from_statement"}) +_SCOPE_BARRIERS = frozenset({"function_definition", "class_definition", "decorated_definition"}) + + +@dataclass(frozen=True) +class _ImportTable: + symbols: dict[str, tuple[str, str]] # local name -> (target_module, original_symbol) + modules: dict[str, str] # local dotted name -> target module fqname + + +@dataclass +class _Edge: + caller: ParsedSpan + caller_kind: str # "module" | "function" | "method" + caller_module: str + callee: ParsedSpan + callee_module: str + resolution: str # "same_module" | "imported" | "self" + lines: set[int] = field(default_factory=set) + + +class CallGraphExtractor: + extractor_id = EXTRACTOR_ID + extractor_version = EXTRACTOR_VERSION + + def __init__(self) -> None: + self._parser = Parser(_LANGUAGE) + + def extract(self, ctx: ExtractContext) -> list[ExtractedArtifact]: + def_index: dict[tuple[str, str], ParsedSpan] = {} + method_index: dict[tuple[str, str], ParsedSpan] = {} + for module, spans in ctx.spans_by_module.items(): + for span in spans: + parent, _, name = span.fq_symbol_path.rpartition(".") + if span.span_kind in ("class", "function") and parent == module: + def_index[(module, name)] = span + elif span.span_kind == "method": + method_index[(parent, name)] = span # parent segment IS the class fq + module_set = set(ctx.spans_by_module) + + edges: dict[str, _Edge] = {} + for module, spans in ctx.spans_by_module.items(): + table = self._import_table(ctx, module, module_set) + for span in spans: + if span.span_kind not in ("module", "function", "method"): + continue + scan_root = self._caller_scan_root(span) + if scan_root is None: + continue + for call in _iter_calls(scan_root): + resolved = self._resolve(call, span, module, table, def_index, + method_index, module_set) + if resolved is None: + continue + callee, callee_module, resolution = resolved + key = f"call:{span.fq_symbol_path}->{callee.fq_symbol_path}" + edge = edges.get(key) + if edge is None: + edge = _Edge( + caller=span, caller_kind=span.span_kind, caller_module=module, + callee=callee, callee_module=callee_module, resolution=resolution, + ) + edges[key] = edge + edge.lines.add(span.start_line + call.start_point[0]) + + return [self._build_artifact(key, edge) for key, edge in edges.items()] + + # --- resolution ---------------------------------------------------------- + + def _resolve( + self, + call: Node, + span: ParsedSpan, + module: str, + table: _ImportTable, + def_index: dict[tuple[str, str], ParsedSpan], + method_index: dict[tuple[str, str], ParsedSpan], + module_set: set[str], + ) -> tuple[ParsedSpan, str, str] | None: + fn = call.child_by_field_name("function") + if fn is None: + return None + if fn.type == "identifier": + name = _text(fn) or "" + local = def_index.get((module, name)) + if local is not None: + return local, module, "same_module" + bound = table.symbols.get(name) + if bound is not None: + target = def_index.get(bound) + if target is not None: + return target, bound[0], "imported" + return None + if fn.type == "attribute": + attr = _text(fn.child_by_field_name("attribute")) + obj = fn.child_by_field_name("object") + if attr is None or obj is None: + return None + if obj.type == "identifier" and _text(obj) == "self" and span.span_kind == "method": + class_fq = span.fq_symbol_path.rsplit(".", 1)[0] + target = method_index.get((class_fq, attr)) + if target is not None: + return target, module, "self" + return None # inherited / dynamic -> documented gap + obj_text = _dotted_object_text(obj) + if obj_text is None: # call()/subscript/... receivers -> documented gap + return None + parts = obj_text.split(".") + for k in range(len(parts), 0, -1): + bound_module = table.modules.get(".".join(parts[:k])) + if bound_module is None: + continue + candidate = ".".join([bound_module, *parts[k:]]) + if candidate in module_set: + target = def_index.get((candidate, attr)) + if target is not None: + return target, candidate, "imported" + break + return None + return None # call-of-call (getattr/super/chained outer), lambda, subscript, ... + + # --- import table --------------------------------------------------------- + + def _import_table(self, ctx: ExtractContext, module: str, module_set: set[str]) -> _ImportTable: + symbols: dict[str, tuple[str, str]] = {} + modules: dict[str, str] = {} + root = self._parse_file(ctx, module) + if root is None: + return _ImportTable(symbols, modules) + base = _relative_base(module, ctx.path_by_module.get(module, "")) + for stmt in _iter_import_statements(root): + if stmt.type == "import_from_statement": + module_ref = stmt.child_by_field_name("module_name") + if module_ref is None: + continue + target = _resolve_module_ref(module_ref, base) + if target is None: + continue + for name_node in stmt.children_by_field_name("name"): + exposed, original = _import_alias(name_node) + if exposed is None or original is None: # wildcard -> nothing + continue + symbols[exposed] = (target, original) + if f"{target}.{original}" in module_set: # `from pkg import sub` (a module) + modules[exposed] = f"{target}.{original}" + else: # plain import_statement + for name_node in stmt.children_by_field_name("name"): + if name_node.type == "dotted_name": # import x.y + dotted = _text(name_node) or "" + if dotted: + modules[dotted] = dotted + modules.setdefault(dotted.split(".")[0], dotted.split(".")[0]) + elif name_node.type == "aliased_import": # import x.y as z + original = _text(name_node.child_by_field_name("name")) + alias = _text(name_node.child_by_field_name("alias")) + if original and alias: + modules[alias] = original + return _ImportTable(symbols, modules) + + def _parse_file(self, ctx: ExtractContext, module: str) -> Node | None: + path = ctx.path_by_module.get(module) + if path is None: + return None + source = (Path(ctx.materialized_root) / path).read_bytes() + return self._parser.parse(source).root_node + + def _caller_scan_root(self, span: ParsedSpan) -> Node | None: + """The subtree whose calls belong to this caller: a def's BODY (its own decorators, + parameters and annotations are excluded by construction), or the whole file for a module.""" + root = self._parser.parse(textwrap.dedent(span.raw_text).encode("utf-8")).root_node + if span.span_kind == "module": + return root + deco = _first_child_of_type(root, "decorated_definition") + host = deco if deco is not None else root + fn = _first_child_of_type(host, "function_definition") + return fn.child_by_field_name("body") if fn is not None else None + + # --- artifact assembly ----------------------------------------------------- + + def _build_artifact(self, key: str, edge: _Edge) -> ExtractedArtifact: + grounding: dict[bytes, DerivedEdge] = { + edge.caller.span_id: DerivedEdge(edge.caller.span_id, "caller") + } + # direct recursion: caller == callee span -> the single row keeps role "caller" + grounding.setdefault(edge.callee.span_id, DerivedEdge(edge.callee.span_id, "callee")) + return ExtractedArtifact( + kind="call_edge", + logical_key=key, + payload={ + "caller": edge.caller.fq_symbol_path, + "caller_kind": edge.caller_kind, + "caller_module": edge.caller_module, + "callee": edge.callee.fq_symbol_path, + "callee_module": edge.callee_module, + "callee_kind": edge.callee.span_kind, + "resolution": edge.resolution, + "lines": sorted(edge.lines), + "span_mapping": "exact", + "limitations": [], + }, + derived_from=list(grounding.values()), + extractor_id=self.extractor_id, + extractor_version=self.extractor_version, + ) + + +# --- module-level helpers (kept local; mirror the other deterministic extractors) ---- + + +def _iter_calls(node: Node) -> Iterator[Node]: + """``call`` nodes lexically owned by ONE caller: stops at nested def/class subtrees (each + nested def is its own caller span; descending would double-attribute). Recurses INTO call + nodes too, so the inner call of a chained ``a().b()`` is collected.""" + for child in node.named_children: + if child.type in _SCOPE_BARRIERS: + continue + if child.type == "call": + yield child + yield from _iter_calls(child) + + +def _iter_import_statements(node: Node) -> Iterator[Node]: + for child in node.named_children: + if child.type in _IMPORT_TYPES: + yield child + else: + yield from _iter_import_statements(child) + + +def _dotted_object_text(node: Node) -> str | None: + """Dotted text of a pure identifier/attribute chain (``a`` / ``a.b.c``), else None.""" + if node.type == "identifier": + return _text(node) + if node.type == "attribute": + obj = node.child_by_field_name("object") + attr = node.child_by_field_name("attribute") + if obj is None or attr is None: + return None + left = _dotted_object_text(obj) + right = _text(attr) + if left is None or right is None: + return None + return f"{left}.{right}" + return None + + +def _relative_base(module: str, path: str) -> str | None: + """The package a relative import resolves against: the module itself for an ``__init__``, + its parent package otherwise; None for a top-level non-package module (invalid Python).""" + if path.split("/")[-1] == "__init__.py": + return module + return module.rsplit(".", 1)[0] if "." in module else None + + +def _resolve_module_ref(node: Node, base: str | None) -> str | None: + if node.type == "dotted_name": + return _text(node) + if node.type == "relative_import": + if base is None: + return None + prefix = node.child(0) + dots = len((prefix.text or b"").decode("utf-8")) if prefix is not None else 0 + tail_node = node.child_by_field_name("module_name") or _first_child_of_type( + node, "dotted_name" + ) + return _resolve_relative(base, dots, _text(tail_node) if tail_node else None) + return None + + +def _resolve_relative(package: str, dots: int, tail: str | None) -> str | None: + parts = package.split(".") if package else [] + drop = dots - 1 + if drop > len(parts): + return None + base = parts[: len(parts) - drop] if drop else parts + full = base + (tail.split(".") if tail else []) + return ".".join(full) if full else None + + +def _import_alias(node: Node) -> tuple[str | None, str | None]: + """``(exposed, original)`` for an imported-name node; ``(None, None)`` for a wildcard.""" + if node.type == "dotted_name": + text = _text(node) + return (text, text) + if node.type == "aliased_import": + original = _text(node.child_by_field_name("name")) + alias = _text(node.child_by_field_name("alias")) + return (alias, original) + return (None, None) + + +def _first_child_of_type(node: Node, type_name: str) -> Node | None: + for child in node.named_children: + if child.type == type_name: + return child + return None + + +def _text(node: Node | None) -> str | None: + if node is None or node.text is None: + return None + return node.text.decode("utf-8") diff --git a/src/kb/mcp/records.py b/src/kb/mcp/records.py index fb5b054..da57ae8 100644 --- a/src/kb/mcp/records.py +++ b/src/kb/mcp/records.py @@ -90,6 +90,12 @@ def summarize(kind: str, payload: dict[str, Any]) -> str: suffix = f" +{len(regs) - 1} more" if len(regs) > 1 else "" family = first.get("family", "?") return f"{payload.get('handler', '?')} on {target}:{trigger} ({family}{suffix})" + if kind == "call_edge": + n = len(payload.get("lines", [])) + return ( + f"{payload.get('caller', '?')} -> {payload.get('callee', '?')} " + f"({payload.get('resolution', '?')}, {n} sites)" + ) if kind == "description": summary = str(payload.get("summary", "")).strip().splitlines() return summary[0] if summary else f"description of {payload.get('target_logical_key', '?')}"