Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 157 additions & 0 deletions docs/adr/ADR-088-amendment-1-git-digest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# ADR-088 Amendment 1: `git.digest` — Agent-Facing Digest Verb with Remote-URL Support

**Status**: Accepted\
**Date**: 2026-07-09\
**Authors**: khive maintainers\
**Amends**: [ADR-088](ADR-088-git-lifecycle-pack.md) (Git-Lifecycle Pack)\
**Related**: ADR-016 (Request DSL), ADR-017 (Pack Standard), ADR-023 (Pack Verb Surface)

## Context

ADR-088 §5 deliberately shipped the git pack verb-less: population happens through the
admin CLI ingester (`kkernel git-ingest`), and the ADR states "No new agent-facing verbs."
Operational experience (2026-07-09) showed the consequence: the ingester existed for weeks
and was never run, because agents — the system's primary operators — had no surface for it.
The directive is to make digestion agent-facing, and to accept a remote URL directly so an
agent can point the verb at any repository and ingest its history without a pre-existing
local clone.

This amendment supersedes ADR-088's "no new agent-facing verbs" clause for exactly one
verb. Note kinds, edge usage, cursor semantics, secret masking, and the `gh` access path
are unchanged.

## Decision

Add one verb to the git pack:

```
git.digest(source, project?, max_items?, include?)
```

- `source` (required, string) — either an absolute local path to a git repository, or an
`https://` git URL (e.g. `https://github.com/org/repo`). Any https host is accepted;
hosts other than github.com degrade to commits-only ingestion with an explicit warning
(issue/PR ingestion requires `gh`, mirroring ADR-088's gh-unavailable degradation
semantics). SSH URLs are rejected in v1 (no interactive auth in the daemon).
- `project` (optional, string) — UUID or 8+ hex prefix of the repo-anchor `project`
entity. When absent: the handler searches for a `project` entity whose
`properties.repo_url` (or name derived from the URL/path basename) matches; if none is
found it CREATES the anchor entity (`kind=project`, `name=<repo basename>`,
`properties.repo_url=<canonical url>`), returning its id in the report. Auto-creation is
reported, never silent.
- `max_items` (optional, int, default 500, clamp 1..2000) — bounded work per call, counted
across commits + issues + PRs. The existing per-repo cursors (ADR-088 §5) make the verb
resumable: each call ingests up to `max_items` and returns `done: false` with cursor
state when more remains. Agents loop until `done: true`. This keeps the verb inside MCP
call-latency envelopes instead of blocking minutes on a large repo.
- `include` (optional, array of `commits|issues|pull_requests`, default all three).

Return shape: the existing `IngestReport` (counts, skips, warnings, `gh_available`)
extended with `done: bool`, `project_id`, and `project_created: bool`.

### Remote-URL mode

1. Clone to a daemon-owned scratch directory (`~/.khive/scratch/git-digest/<hash>/`),
`git clone --filter=blob:none` (history + trees without file blobs — commit walking
needs messages and file lists, not contents; `git log --name-only` works against a
partial clone with lazy fetch disabled for our read pattern).
2. Derive the `owner/repo` slug from the URL for `gh`-based issue/PR ingestion (unchanged
ADR-088 Open Question 3 resolution: shell `gh`; skip with warning when unavailable).
3. The clone is cached keyed by canonical URL: subsequent digest calls `git fetch` instead
of re-cloning. An LRU cap (default 5 repos / 2GB, config `[git] digest_cache_*`) evicts
oldest; eviction is safe because cursors live in the database, not the clone.
Additionally, a per-clone size cap (operator-configurable, default 1GB) bounds any
single clone: if a clone or fetch would exceed it, the operation aborts with a clear
error before writing further — `max_items` bounds ingestion work, but only this cap
bounds disk consumption by a single large-history repository before LRU eviction can
apply.
4. Cleanup on eviction uses directory removal of the scratch path only (never touches
user-owned paths).

### Security posture

- `git clone` of an untrusted remote does not execute repository-supplied code (no hooks
run on clone/fetch). The handler additionally sets `GIT_TERMINAL_PROMPT=0` and
`core.hooksPath=/dev/null` on the scratch clone as defense in depth.
- Local-path mode requires an absolute path; relative paths are rejected. The path must
contain a `.git` directory; arbitrary directory walking is not performed.
- Secret masking is unchanged: ingested text goes through the same `create`-verb gate
(ADR-088 acceptance note 5). Blocked writes surface as report warnings, fail-closed.
- Namespace/attribution: writes stamp the caller's token namespace exactly as the CLI
ingester does today; no new authorization surface. The Gate (ADR-018) remains the
authorization seam for callers who should not write.

### Surface-contract touch points

- ADR-023 (pack verb surface): git pack's verb table gains one row; `verbs()` output and
the khive-mcp tool description regenerate (CLAUDE.md guidance: re-run
`request(ops="verbs()")` before editing the count line).
- ADR-015 product-verb table is NOT amended: `git.digest` is a pack-prefixed verb
(`pack.verb` convention, ADR-023), not one of the 15 flat product verbs.
- `kkernel git-ingest` remains as the admin path (shared implementation; the verb handler
and CLI both call `khive_pack_git::ingest::run_ingest` with the same options struct,
extended with the bounded `max_items` + remote-source support).

## Ingest enrichment (consumer-evidence riders, 2026-07-09)

First-consumer evidence (an agent running 14 live GQL operations against a freshly
ingested multi-repository corpus) showed the corpus is a flat property store: the only
edges the v0 ingester creates are note→project `annotates` and merge-commit→PR. The three
cross-references consumers actually want — PR-to-issue trails, commits-touching-an-issue,
fix chains — are impossible by traversal because `Closes #N` / `#M` references exist only
as unextracted body text, and issues are isolated leaves. Two riders, both ingest-side and
in scope for this amendment:

1. **Reference-edge extraction.** At ingest, parse commit messages and PR/issue bodies for
GitHub reference grammar (`Closes/Fixes/Resolves #N`, bare `#N` mentions) and
materialize edges: closing references as `annotates` from the closing commit/PR note to
the issue note (with `properties.ref_kind = "closes" | "mentions"` on the edge's
annotating metadata), and commit `parents[]` as `precedes` between commit notes
(parent precedes child; both endpoints are same-substrate notes, legal per supersedes/
precedes note rules — verify against `EDGE_RULES` at impl time and fall back to
`annotates` + `ref_kind="parent"` if `precedes` n→n is not in the base contract).
Cross-repo `#N` collisions resolve within the same `project_id` only; unresolved
references (issue not ingested) are skipped and counted in the report. Extraction is
fail-open: a malformed or unresolvable reference never fails an ingest batch — it is
skipped with a warning in the report. Edge extraction runs at ingest only: this
amendment adds no retroactive backfill verb. Re-digesting an already-cursored
repository picks up edges for new items; a one-shot backfill over already-ingested
notes is an admin pass (`kkernel`), out of scope here.
2. **Readable names.** Provenance notes currently carry `name=null`, so neighbors/GQL
render placeholders and force a `get()` per hop. Set `name` at ingest: issues/PRs
`"#<number> <title>"` (truncated), commits `"<short-sha> <subject>"` (truncated).

Data-fidelity checks from the same evidence run were verified clean: `closed_at` values
that cluster at one instant reflect real GitHub bulk-close events (confirmed against
`gh issue view`), and `author` is the genuine GitHub login (commit author names come from
git identity, a different identity system — both correct).

## Consequences

- One-call adoption for agents: "digest this repo" becomes a verb loop instead of an admin
task nobody runs. Periodic re-digest can be scheduled via `schedule.schedule(action=
"git.digest(source=...)")` — composing two existing packs with zero new machinery.
- The bounded-call contract adds cursor-state plumbing to the report but removes the
worst failure mode (an MCP call blocked for minutes on a 10k-commit clone).
- Scratch-clone cache is new daemon-owned disk state; sized and evictable, documented in
the operator guide.

## Alternatives rejected

- **Fire-and-forget background job + status verb**: heavier (job table, lifecycle,
another verb); the cursor design already gives resumability with strictly simpler
semantics. Revisit only if per-call latency at max_items=500 proves unacceptable.
- **Full clone in URL mode**: blob download dominates clone time and disk for zero
ingestion value; `--filter=blob:none` keeps everything the ingester reads.
- **Direct GitHub REST instead of `gh`**: re-opens ADR-088 Open Question 3 for no gain;
`gh` handles auth and pagination and is already the accepted path.

## Spec-gate rulings (2026-07-09)

1. No host allowlist. Any https git host is accepted; non-github hosts degrade to
commits-only with an explicit warning (mirrors the gh-unavailable degradation
semantics). SSH remains hard-rejected in v1.
2. `max_items` default 500 confirmed (measured ~10s per call on repositories of 448-991
items).
3. A per-clone size cap on the scratch cache is required in addition to the LRU cap, so a
single large-history repository cannot exhaust daemon disk before eviction applies.
5 changes: 4 additions & 1 deletion docs/adr/ADR-088-git-lifecycle-pack.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ ADR-017 (Pack Standard — `NOTE_KINDS`, `NoteKindSpec`, `KindHook`), ADR-085 (C
`finding` note kind and its Alternative A7, directly reconciled below)\
**Related**: ADR-010 (KG Versioning — "khive does not build a GitHub replacement"),
ADR-087 (Workspace Mirror — sibling background-ingestion pattern this pack's ingester
reuses)
reuses)\
**Amended by**: [Amendment 1](ADR-088-amendment-1-git-digest.md) — `git.digest` agent-facing
verb with remote-URL support (supersedes the "no new agent-facing verbs" clause for this
one verb; adds reference-edge extraction and note naming at ingest)

## Context

Expand Down
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ For historical context, see [v0 archive](../_archive/adr_v0/README.md). v0 ADRs
| [ADR-086](ADR-086-doc-file-pack.md) | Document/File Modeling — Content on the Existing `document` Entity Kind (Proposed) |
| [ADR-087](ADR-087-workspace-mirror.md) | Workspace Mirror — Folding `.khive/` Into the Graph Substrate (Proposed) |
| [ADR-088](ADR-088-git-lifecycle-pack.md) | Git-Lifecycle Pack — Commit and Issue Note Kinds (Proposed) |
| [ADR-088 Amendment 1](ADR-088-amendment-1-git-digest.md) | `git.digest` — Agent-Facing Digest Verb with Remote-URL Support (Accepted) |
| [ADR-089](ADR-089-context-verb.md) | Context Verb — Entity-Anchored Graph Context in One Call (Proposed) |
| [ADR-090](ADR-090-docs-site-standard.md) | Docs Site Standard — Navigation, Agent md/txt Surfaces, Visual Style (Proposed) |
| [ADR-091](ADR-091-wal-snapshot-lifetime.md) | Bounded Read-Transaction Lifetime and WAL Checkpoint Escalation (Accepted) |
Expand Down
Loading