diff --git a/docs/adr/ADR-088-amendment-1-git-digest.md b/docs/adr/ADR-088-amendment-1-git-digest.md new file mode 100644 index 00000000..545bcb1d --- /dev/null +++ b/docs/adr/ADR-088-amendment-1-git-digest.md @@ -0,0 +1,157 @@ +# ADR-088 Amendment 1: `git.digest` — Agent-Facing Digest Verb with Remote-URL Support + +**Status**: Accepted\ +**Date**: 2026-07-09\ +**Authors**: khive maintainers\ +**Amends**: [ADR-088](ADR-088-git-lifecycle-pack.md) (Git-Lifecycle Pack)\ +**Related**: ADR-016 (Request DSL), ADR-017 (Pack Standard), ADR-023 (Pack Verb Surface) + +## Context + +ADR-088 §5 deliberately shipped the git pack verb-less: population happens through the +admin CLI ingester (`kkernel git-ingest`), and the ADR states "No new agent-facing verbs." +Operational experience (2026-07-09) showed the consequence: the ingester existed for weeks +and was never run, because agents — the system's primary operators — had no surface for it. +The directive is to make digestion agent-facing, and to accept a remote URL directly so an +agent can point the verb at any repository and ingest its history without a pre-existing +local clone. + +This amendment supersedes ADR-088's "no new agent-facing verbs" clause for exactly one +verb. Note kinds, edge usage, cursor semantics, secret masking, and the `gh` access path +are unchanged. + +## Decision + +Add one verb to the git pack: + +``` +git.digest(source, project?, max_items?, include?) +``` + +- `source` (required, string) — either an absolute local path to a git repository, or an + `https://` git URL (e.g. `https://github.com/org/repo`). Any https host is accepted; + hosts other than github.com degrade to commits-only ingestion with an explicit warning + (issue/PR ingestion requires `gh`, mirroring ADR-088's gh-unavailable degradation + semantics). SSH URLs are rejected in v1 (no interactive auth in the daemon). +- `project` (optional, string) — UUID or 8+ hex prefix of the repo-anchor `project` + entity. When absent: the handler searches for a `project` entity whose + `properties.repo_url` (or name derived from the URL/path basename) matches; if none is + found it CREATES the anchor entity (`kind=project`, `name=`, + `properties.repo_url=`), returning its id in the report. Auto-creation is + reported, never silent. +- `max_items` (optional, int, default 500, clamp 1..2000) — bounded work per call, counted + across commits + issues + PRs. The existing per-repo cursors (ADR-088 §5) make the verb + resumable: each call ingests up to `max_items` and returns `done: false` with cursor + state when more remains. Agents loop until `done: true`. This keeps the verb inside MCP + call-latency envelopes instead of blocking minutes on a large repo. +- `include` (optional, array of `commits|issues|pull_requests`, default all three). + +Return shape: the existing `IngestReport` (counts, skips, warnings, `gh_available`) +extended with `done: bool`, `project_id`, and `project_created: bool`. + +### Remote-URL mode + +1. Clone to a daemon-owned scratch directory (`~/.khive/scratch/git-digest//`), + `git clone --filter=blob:none` (history + trees without file blobs — commit walking + needs messages and file lists, not contents; `git log --name-only` works against a + partial clone with lazy fetch disabled for our read pattern). +2. Derive the `owner/repo` slug from the URL for `gh`-based issue/PR ingestion (unchanged + ADR-088 Open Question 3 resolution: shell `gh`; skip with warning when unavailable). +3. The clone is cached keyed by canonical URL: subsequent digest calls `git fetch` instead + of re-cloning. An LRU cap (default 5 repos / 2GB, config `[git] digest_cache_*`) evicts + oldest; eviction is safe because cursors live in the database, not the clone. + Additionally, a per-clone size cap (operator-configurable, default 1GB) bounds any + single clone: if a clone or fetch would exceed it, the operation aborts with a clear + error before writing further — `max_items` bounds ingestion work, but only this cap + bounds disk consumption by a single large-history repository before LRU eviction can + apply. +4. Cleanup on eviction uses directory removal of the scratch path only (never touches + user-owned paths). + +### Security posture + +- `git clone` of an untrusted remote does not execute repository-supplied code (no hooks + run on clone/fetch). The handler additionally sets `GIT_TERMINAL_PROMPT=0` and + `core.hooksPath=/dev/null` on the scratch clone as defense in depth. +- Local-path mode requires an absolute path; relative paths are rejected. The path must + contain a `.git` directory; arbitrary directory walking is not performed. +- Secret masking is unchanged: ingested text goes through the same `create`-verb gate + (ADR-088 acceptance note 5). Blocked writes surface as report warnings, fail-closed. +- Namespace/attribution: writes stamp the caller's token namespace exactly as the CLI + ingester does today; no new authorization surface. The Gate (ADR-018) remains the + authorization seam for callers who should not write. + +### Surface-contract touch points + +- ADR-023 (pack verb surface): git pack's verb table gains one row; `verbs()` output and + the khive-mcp tool description regenerate (CLAUDE.md guidance: re-run + `request(ops="verbs()")` before editing the count line). +- ADR-015 product-verb table is NOT amended: `git.digest` is a pack-prefixed verb + (`pack.verb` convention, ADR-023), not one of the 15 flat product verbs. +- `kkernel git-ingest` remains as the admin path (shared implementation; the verb handler + and CLI both call `khive_pack_git::ingest::run_ingest` with the same options struct, + extended with the bounded `max_items` + remote-source support). + +## Ingest enrichment (consumer-evidence riders, 2026-07-09) + +First-consumer evidence (an agent running 14 live GQL operations against a freshly +ingested multi-repository corpus) showed the corpus is a flat property store: the only +edges the v0 ingester creates are note→project `annotates` and merge-commit→PR. The three +cross-references consumers actually want — PR-to-issue trails, commits-touching-an-issue, +fix chains — are impossible by traversal because `Closes #N` / `#M` references exist only +as unextracted body text, and issues are isolated leaves. Two riders, both ingest-side and +in scope for this amendment: + +1. **Reference-edge extraction.** At ingest, parse commit messages and PR/issue bodies for + GitHub reference grammar (`Closes/Fixes/Resolves #N`, bare `#N` mentions) and + materialize edges: closing references as `annotates` from the closing commit/PR note to + the issue note (with `properties.ref_kind = "closes" | "mentions"` on the edge's + annotating metadata), and commit `parents[]` as `precedes` between commit notes + (parent precedes child; both endpoints are same-substrate notes, legal per supersedes/ + precedes note rules — verify against `EDGE_RULES` at impl time and fall back to + `annotates` + `ref_kind="parent"` if `precedes` n→n is not in the base contract). + Cross-repo `#N` collisions resolve within the same `project_id` only; unresolved + references (issue not ingested) are skipped and counted in the report. Extraction is + fail-open: a malformed or unresolvable reference never fails an ingest batch — it is + skipped with a warning in the report. Edge extraction runs at ingest only: this + amendment adds no retroactive backfill verb. Re-digesting an already-cursored + repository picks up edges for new items; a one-shot backfill over already-ingested + notes is an admin pass (`kkernel`), out of scope here. +2. **Readable names.** Provenance notes currently carry `name=null`, so neighbors/GQL + render placeholders and force a `get()` per hop. Set `name` at ingest: issues/PRs + `"# "` (truncated), commits `"<short-sha> <subject>"` (truncated). + +Data-fidelity checks from the same evidence run were verified clean: `closed_at` values +that cluster at one instant reflect real GitHub bulk-close events (confirmed against +`gh issue view`), and `author` is the genuine GitHub login (commit author names come from +git identity, a different identity system — both correct). + +## Consequences + +- One-call adoption for agents: "digest this repo" becomes a verb loop instead of an admin + task nobody runs. Periodic re-digest can be scheduled via `schedule.schedule(action= + "git.digest(source=...)")` — composing two existing packs with zero new machinery. +- The bounded-call contract adds cursor-state plumbing to the report but removes the + worst failure mode (an MCP call blocked for minutes on a 10k-commit clone). +- Scratch-clone cache is new daemon-owned disk state; sized and evictable, documented in + the operator guide. + +## Alternatives rejected + +- **Fire-and-forget background job + status verb**: heavier (job table, lifecycle, + another verb); the cursor design already gives resumability with strictly simpler + semantics. Revisit only if per-call latency at max_items=500 proves unacceptable. +- **Full clone in URL mode**: blob download dominates clone time and disk for zero + ingestion value; `--filter=blob:none` keeps everything the ingester reads. +- **Direct GitHub REST instead of `gh`**: re-opens ADR-088 Open Question 3 for no gain; + `gh` handles auth and pagination and is already the accepted path. + +## Spec-gate rulings (2026-07-09) + +1. No host allowlist. Any https git host is accepted; non-github hosts degrade to + commits-only with an explicit warning (mirrors the gh-unavailable degradation + semantics). SSH remains hard-rejected in v1. +2. `max_items` default 500 confirmed (measured ~10s per call on repositories of 448-991 + items). +3. A per-clone size cap on the scratch cache is required in addition to the LRU cap, so a + single large-history repository cannot exhaust daemon disk before eviction applies. diff --git a/docs/adr/ADR-088-git-lifecycle-pack.md b/docs/adr/ADR-088-git-lifecycle-pack.md index 22c3d4dd..f1de3012 100644 --- a/docs/adr/ADR-088-git-lifecycle-pack.md +++ b/docs/adr/ADR-088-git-lifecycle-pack.md @@ -9,7 +9,10 @@ ADR-017 (Pack Standard — `NOTE_KINDS`, `NoteKindSpec`, `KindHook`), ADR-085 (C `finding` note kind and its Alternative A7, directly reconciled below)\ **Related**: ADR-010 (KG Versioning — "khive does not build a GitHub replacement"), ADR-087 (Workspace Mirror — sibling background-ingestion pattern this pack's ingester -reuses) +reuses)\ +**Amended by**: [Amendment 1](ADR-088-amendment-1-git-digest.md) — `git.digest` agent-facing +verb with remote-URL support (supersedes the "no new agent-facing verbs" clause for this +one verb; adds reference-edge extraction and note naming at ingest) ## Context diff --git a/docs/adr/README.md b/docs/adr/README.md index fade7027..06d9e9f3 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -114,6 +114,7 @@ For historical context, see [v0 archive](../_archive/adr_v0/README.md). v0 ADRs | [ADR-086](ADR-086-doc-file-pack.md) | Document/File Modeling — Content on the Existing `document` Entity Kind (Proposed) | | [ADR-087](ADR-087-workspace-mirror.md) | Workspace Mirror — Folding `.khive/` Into the Graph Substrate (Proposed) | | [ADR-088](ADR-088-git-lifecycle-pack.md) | Git-Lifecycle Pack — Commit and Issue Note Kinds (Proposed) | +| [ADR-088 Amendment 1](ADR-088-amendment-1-git-digest.md) | `git.digest` — Agent-Facing Digest Verb with Remote-URL Support (Accepted) | | [ADR-089](ADR-089-context-verb.md) | Context Verb — Entity-Anchored Graph Context in One Call (Proposed) | | [ADR-090](ADR-090-docs-site-standard.md) | Docs Site Standard — Navigation, Agent md/txt Surfaces, Visual Style (Proposed) | | [ADR-091](ADR-091-wal-snapshot-lifetime.md) | Bounded Read-Transaction Lifetime and WAL Checkpoint Escalation (Accepted) |