diff --git a/CHANGELOG.md b/CHANGELOG.md index 8abd9d2f..07fa4b38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,8 +7,113 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- **MCP 2026-07-28.** The server speaks the final "stateless MCP" revision on + the v2 SDK line (`@modelcontextprotocol/server` 2.0). `daftari serve` is + stateless per the spec's Decision 1: the initialize handshake, the + `Mcp-Session-Id` header, and the session table are gone — identity is + resolved from the bearer on every request against the same config-declared + map, and 2025-era traffic is refused (no dual-stacking). stdio serves both + eras from one factory, so lagging clients use stdio; the RBAC, + existence-disclosure, and process-lock invariants carry over unchanged. + Design record: + `docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md`. +- `vault_ratify` called without a `decision` now answers with a stateless + form-mode elicitation (`input_required`): approve/reject with reject + preselected, plus HMAC-signed opaque state carrying the action id, the + vault HEAD at proposal time, and the deciding user. A declined form applies + nothing and leaves the action pending; a direct call with the decision + inline keeps working — including a batch `ids` call, which always requires + an explicit decision. The server proposes, the human disposes — now on the + wire itself (Decision 5). +- The maintenance passes (`sleep`, `consolidate`, `audit`, `eval`) remain + CLI-only: the spec's Decision 4 kill condition fired — the final revision + moved Tasks to a standalone extension (removing `tasks/list`) and the + TypeScript SDK ships no tasks runtime yet. + ### Added +- **Contextual chunking + optional local reranker.** `chunkDocument` + (replacing `chunkText`) splits document bodies at ATX headings (never + packing across a heading boundary) and gives every chunk a one-line + breadcrumb context (`{collection} › {title} › {headings} · tags: a, b, c`) + that is hashed and embedded together with the chunk's body text, and + stored as a second `chunks_fts` column — contextual embeddings and + contextual BM25 from one string-prefix change, no LLM call. Displayed + snippets are read from the body column only; the synthesized breadcrumb + never appears in served content. Schema bump `SCHEMA_VERSION` 10 → 11: + every chunk's hash input changed, so the first post-upgrade reindex is a + one-time full re-embed of the whole corpus (~25 min local-minilm / ~2 min + ~$0.10 openai-3-small for the 44k-chunk reference vault). Retitling a + document, moving it between collections, or changing its tag *set* + re-embeds all of that document's chunks (the breadcrumb is part of the + hash); tag *reorder* is a no-op (tags are sorted before hashing). + Also lands an optional local cross-encoder reranker: `rerank.provider: + local-bge-m3` (default `none`) reorders the top-50 RBAC-filtered + `vault_search` hits with a ~600MB ONNX q8 model + (`onnx-community/bge-reranker-v2-m3-ONNX` via `@huggingface/transformers`, + already a dependency — zero new deps), between the RBAC filter and the + slice to `limit`. A 1.5s per-search timeout, a not-warm skip that fires a + background warm instead of blocking the call, and a provider `Result.err` + all degrade to the fused order identically. `vault_search`'s result gains + `rerankUsed: boolean`, the honest twin of `vectorUsed`. Ships opt-in; the + default flip to `local-bge-m3` is gated on a post-merge recall + measurement, same playbook as chunk-level BM25's v1.29.0 default flip. + Design record: + `docs/superpowers/specs/2026-07-26-contextual-chunking-reranker-design.md`. + +- **Two new local embedding providers + int8 vec-index quantization + (opt-in; default embedder unchanged).** `embeddings.provider` gains + `local-embeddinggemma` (`google/embeddinggemma-300m`, 768d native, + Matryoshka-truncatable to 512/768) and `local-qwen3-0.6b` + (`Qwen/Qwen3-Embedding-0.6B`, up to 768d exposed), both fully local via + the existing `@huggingface/transformers` dependency, both with + asymmetric document/query prompt prefixing. The durable `embeddings` + cache now stores the full NATIVE-dim vector under a dim-free cache id + (e.g. `local-embeddinggemma#p1`); a new `embeddings.dim` config key picks + the INDEX-time truncation. A new `embeddings.quantize: int8 | none` + config key (default `int8` for the two new providers, `none` — today's + exact behavior — for `local-minilm`/`openai-3-small`) stores the + sqlite-vec mirror as `int8[dim]` with scan-then-rescore: candidates are + selected by quantized distance, then rescored with exact float32 cosine + against the durable cache, so quantization never becomes the reported + score. Switching `provider`, `dim`, or `quantize` between server runs is + a config change plus a background reindex; a `dim`/`quantize` flip alone + needs no re-embed (`isIndexFresh` detects the change and the reindex is + all cache hits — a vec-mirror rebuild, not a cold re-embed). `local- + minilm` remains the default for `loadConfig`'s programmatic fallback; + the vault-init template default flip and any measured cold-reindex / + query-latency / RSS numbers are gated on the governing spec's Phase 0 + smoke spike and Phase 5 recall-bench, neither of which has run against a + real model download as of this entry — see `docs/architecture.md`'s + "Vec-index quantization" section and the verification-honesty notes on + each new provider file. + Design record: + `docs/superpowers/specs/2026-07-26-embedding-refresh-quantization-design.md`. + +- **MCP `content`-channel summaries for every remaining tool.** Every + registered tool now has a `summarize` (compact, model-facing text) and, + where it names documents, a `docLinks` (`resource_link` entries) — closing + the gap Decision 3 (#302) left on everything but search and lint. + `vault_read`'s body now rides the wire exactly once: `content[0].text` + carries it verbatim, `structuredContent` omits it (a `wireValue` + projection), with the doc resource (`daftari://doc/{path}`) as the + programmatic alternative. The CallTool bridge hardens presentation: a + throwing `summarize`/`docLinks` falls back to the pre-Decision-3 + `JSON.stringify` behavior and logs to stderr instead of turning a + successful tool call into an error response. + Design record: + `docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md`. + +### Fixed + +- `vault_tier2_queue`'s `field_changes` could report a field with no `before` + key at all (dropped by JSON serialization on a document's first write), + violating its own declared output shape. `before` now normalizes to `null` + when the log has no prior value, matching the schema's documented + "`null` means no prior value" contract. + - **Bi-temporal validity.** Two optional built-in frontmatter fields, `valid_from` and `valid_until`, recording when a document's claim was true *in the world* — as distinct from when the vault recorded it, which git diff --git a/CLAUDE.md b/CLAUDE.md index 85f7d19c..925fbd48 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,12 +18,14 @@ Concepts and layer boundaries: docs/architecture.md. Where things live: - `src/witness/` — per-principal track records - `src/asof/` — belief archaeology over git history (`daftari asof`) - `src/audit/` — doc-to-code coherence audit (`daftari audit`) +- `src/anchors/` — citation-anchor pin grammar and the shared JIT classifier (used by vault_read, `daftari audit`, and vault_lint) - `src/eval/` — vault quality eval: question generation + LLM judging (`daftari eval`) - `src/interview/` — principal interview: question sheet from tensions/staleness/open questions, verbatim transcript (`daftari interview`) - `src/backfill/`, `src/import/`, `src/okf/` — adoption paths: metadata backfill, foreign-vault import, OKF export/import - `src/serve/` — server mode over Streamable HTTP (`daftari serve`); `src/sync/` — push/restore against storage backends - `src/hooks/` — vault-supplied hook module loading - `src/themes/` — clustering primitives for vault_themes +- `src/context/` — context-pack assembly for vault_context: deterministic selection, budget-cut, markdown templating over enriched candidates (`src/context/assemble.ts`), chars/4 token estimation (`src/context/estimate.ts`). No I/O, no LLM call — see the standing constraints below. - `src/utils/` — config.yaml loading, git plumbing, paths, hashing Entrypoints: `src/index.ts` (stdio MCP entry), `src/server.ts` (MCP server wiring), `src/cli.ts` (CLI). @@ -43,7 +45,11 @@ Entrypoints: `src/index.ts` (stdio MCP entry), `src/server.ts` (MCP server wirin - Tension/edge visibility: omission over redaction, no existence leak. Doc lists never name docs in unreadable collections; hidden-blast remainders are reported coarsened (none/some/many), never as exact counts — small cells disclose linked existence. Vault-global lint aggregates stay unfiltered by design. See docs/superpowers/specs/2026-07-14-edge-graph-existence-disclosure-design.md. - The Tension Court is an operator-only surface. Court/docket code never takes an access context. Exposing any court surface via MCP requires revisiting the 2026-07-14 edge-graph spec first. - Storage backends (#6) are dumb sync targets — `get/put/list/delete` over opaque keys. The local git working copy is canonical; backends never understand markdown, git, or locks; index/locks stay local and never sync. See spec 2026-07-20 Decision 3. +- MCP: the server speaks the 2026-07-28 stateless revision (v2 SDK, `@modelcontextprotocol/server`). `daftari serve` resolves identity per request from the bearer — no sessions — and refuses 2025-era traffic; stdio serves both eras. `vault_ratify` without a decision elicits an approve/reject form (reject preselected) with HMAC-signed opaque state — single-`id` calls only; a batch `ids` call always requires an explicit decision. The maintenance passes stay CLI-only until the Tasks extension has a TS SDK runtime. See docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md. - Only one daftari process may hold a vault at a time. `.daftari/process.lock` is the per-vault process lock, and it records the holder's mode (stdio or serve). Live-holder precedence favors the durable tenant (2026-07-20 spec, Decision 4): stdio finding a live stdio holder SIGTERMs it and waits up to 3 seconds before taking over (the original single-user convenience — the only implicit live takeover); stdio finding a live `daftari serve` REFUSES to start; a new serve refuses against ANY live holder unless started with `--takeover`. Stale locks (dead PID, or PID recycled) are overwritten silently in every mode. The lockfile is ephemeral — never check it in. +- The staged-action queue's risk score is derived/ordinal, computed fresh on every `vault_lint` read, never stored as a `risk` field or column — the same posture `derives_from` strength takes. `vault_ratify`'s `ids` batch is an explicit, capped id list (`BATCH_RATIFY_MAX` = 20); the parameter shape has no threshold and no "all pending" sentinel, so auto-approval-by-score cannot be expressed. Proposal records carry the authenticated stager (`staged_by_principal`, from `access.user`) alongside the unauthenticated, caller-claimed `proposed_by` display string — the witness and the risk score's W term key on the former when present. Each decision record additionally carries a non-authoritative `risk_at_decision` snapshot (JSONL-only, never read for ordering) so the risk score's own predictive-power kill condition is evaluable later. See docs/superpowers/specs/2026-07-26-risk-triaged-ratification-design.md. +- Citation-anchor pins (`describes` entries suffixed `#L-@`) are advisory, annotate-only — a `moved`/`missing` pin never invalidates, demotes, or rewrites a doc, and an intact pin softens decay's banner copy but never extends the TTL clock. The read-path check is batched git plumbing only (one `hash-object` invocation per referenced repo per read, not per pin) and silent on failure (a classifier error drops that entry, never fails the read). Configuring `code_repos` makes blob-level facts about those repos (path existence, blob-match, relocated line numbers) visible to every reader whose role carries the `code_repo_visibility` grant (default off) — see docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification-design.md. +- `vault_tools` is the always-advertised in-band catalog: index mode (`{name, oneLine}` per tool) or expand mode (full schemas for named tools). It reads the FULL registry minus the vault's `exclude` list — exclude always wins (#104), but tier and `include` never affect it, since making tiered-out tools discoverable in-band is its entire purpose. `vault_context(task, budget?)` assembles a token-budgeted markdown brief: hybrid search → RBAC filter (before any budgeting) → supersession dedup (a collapsed chain's flags are ALL keyed on the head's own path, never a stale member's) → greedy budget cut at `budget * 0.9` estimated tokens (chars/4, no tokenizer) → render. No LLM call anywhere in the handler — pure selection and templating; a tension always renders both claims, never a blended verdict. `hidden_remainder` is a lower-bound signal over OBSERVABLE withholding (RBAC-dropped BM25-side pool candidates, dropped coverage additions, restricted supersession hops) — "none" means "no withholding observed," never "nothing withheld," because the vector half of retrieval is already RBAC-pushdown-scrubbed and structurally invisible to this count. Both tools join `CORE_TOOLS`; the default `tools.tier` stays `full` this wave — see docs/superpowers/specs/2026-07-26-context-packs-progressive-disclosure-design.md. ## Labeling - [DATA] for values read from files or the index diff --git a/README.md b/README.md index d26cbe18..72c98d36 100644 --- a/README.md +++ b/README.md @@ -414,6 +414,43 @@ SDK's standard environment chain, never from vault config. GCS is reached via its S3-interoperability endpoint. Restore refuses non-empty directories and reindexes when done. +## Embedding providers + +`vault_search`'s vector half is a swappable, config-driven `EmbeddingProvider`. +Four ship: + +```yaml +# .daftari/config.yaml +embeddings: + provider: local-minilm # local-minilm | openai-3-small | local-embeddinggemma | local-qwen3-0.6b + # dim: 512 # local-embeddinggemma / local-qwen3-0.6b only: 512 (default) | 768 + # quantize: int8 # int8 (default for the two new local providers) | none +``` + +- **`local-minilm`** (programmatic default) — `all-MiniLM-L6-v2`, 384d, + fully local via `@huggingface/transformers`, free, no API key. ~100MB + model footprint. +- **`local-embeddinggemma`** — `google/embeddinggemma-300m`, 768d native, + Matryoshka-truncatable to 512 (default) or 768, fully local. A + model-generation upgrade over `local-minilm`, currently opt-in — see + `docs/architecture.md` for the verification-honesty caveat before relying + on it in production. ~600MB-class footprint. +- **`local-qwen3-0.6b`** — `Qwen/Qwen3-Embedding-0.6B`, exposes up to 768d + (native 1024d not yet offered), fully local, larger footprint + (~1.5GB-class) than EmbeddingGemma. The documented alternative, not a + default candidate. +- **`openai-3-small`** — OpenAI `text-embedding-3-small`, 1536d, paid, + requires `OPENAI_API_KEY`. Untouched by the two entries above. + +`warm_embeddings: false` (default `true`) skips the background model +warm-up at startup — the escape hatch that matters more once the default +footprint options grow past `local-minilm`'s. Switching `provider`, `dim`, +or `quantize` between server runs never loses data: the durable embeddings +cache is keyed by `(content_hash, model)` and stores every provider's +vectors side by side, so switching back is free and a `dim`/`quantize` +change alone needs no re-embed at all (see `docs/architecture.md`, +"Vec-index quantization"). + ## How it compares | |AGENTS.md |RAG |Daftari | @@ -437,9 +474,14 @@ Deliberately deferred to keep the surface tight: advisory boundary warnings shipped in the meantime: `vault_lint`'s `domainLeaks` check and write-time `domain_warnings`) -(LLM reranking, deferred here originally, has since shipped as the opt-in -agent-as-judge `rerank_candidates` on `vault_search`: the server prepares the -fused candidate pool and the protocol; the calling agent is the judge.) +(LLM reranking, deferred here originally, has since shipped two ways. The +agent-as-judge `rerank_candidates` on `vault_search` is opt-in and free: the +server prepares the fused candidate pool and the protocol; the calling agent +is the judge. A second, local cross-encoder reranker (`rerank.provider: +local-bge-m3` in `.daftari/config.yaml`) also ships, opt-in and default off — +it reorders the top-50 RBAC-filtered hits with a local ONNX model before the +`vault_search` response is sliced to `limit`; see `docs/architecture.md` for +the config block and degradation behavior.) Each is a clean increment on a surface that already works. diff --git a/docs/architecture.md b/docs/architecture.md index d5f176ed..671956cf 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -182,41 +182,114 @@ Three things sit alongside the markdown: The vector embeddings are produced by a configurable **`EmbeddingProvider`** (see `src/search/embedding-provider.ts`). Each - document body is split into ~800-character chunks; every chunk is embedded - into a fixed-dimension vector by the active provider. Two providers ship - with v1.9: - - - **`local-minilm`** (default) — runs `all-MiniLM-L6-v2` in-process via - `@huggingface/transformers` (Transformers.js). 384-dimension vectors, - fully local, no embedding API call. The only network access is the - one-time download of the model weights to the Hugging Face cache on - first use. Slow on cold-start (multi-minute on large vaults) but free. + document body is split into heading-aware, ~800-character-max chunks by + `chunkDocument` (`src/search/vector.ts`, spec 2026-07-26-contextual- + chunking-reranker-design.md): a heading boundary always starts a new + chunk, never packed across sections, and every chunk carries a one-line + synthesized breadcrumb context (`{collection} › {title} › {headings} · + tags: a, b, c`, capped at 160 chars). The context is hashed and embedded + TOGETHER with the chunk's body text (`embeddingInput = context + "\n\n" + + text`) — this is the "contextual embeddings" half of Anthropic's + contextual-retrieval recipe, done with a string prefix instead of an LLM + call. It is also stored as a second `chunks_fts` column, so + `bm25(chunks_fts)` scores title/collection/tag tokens together with body + tokens — contextual BM25, the other half. Displayed snippets are read from + the `text` column only; the synthesized context never appears in served + content (Markdown is truth). Every chunk is then embedded into a + fixed-dimension vector by the active provider. Four providers ship as of + the 2026-07-26 embedding-refresh-quantization spec: + + - **`local-minilm`** (programmatic default) — runs `all-MiniLM-L6-v2` + in-process via `@huggingface/transformers` (Transformers.js). + 384-dimension vectors, fully local, no embedding API call. The only + network access is the one-time download of the model weights to the + Hugging Face cache on first use. Slow on cold-start (multi-minute on + large vaults) but free. `all-MiniLM-L6-v2` is a 2021-era model; the two + entries below are the model-generation replacement, currently opt-in. + + - **`local-embeddinggemma`** — runs `google/embeddinggemma-300m` + in-process via the same Transformers.js runtime. 768-dimension native + output, Matryoshka-truncatable; the configured index dim defaults to + 512 (`embeddings.dim: 512 | 768`). Asymmetric prompt prefixes (document + vs. query form) are applied internally by the provider. ~600MB-class + model footprint (less with the q8 ONNX variant this provider defaults + to). **Verification honesty:** the exact prompt-prefix strings and the + Transformers.js compatibility of this model are [TRAINING] hypotheses + from the governing spec, pending a smoke-spike verification that has + not been run against a real model download as of this writing — see + `src/search/providers/local-embeddinggemma.ts`'s header and the spec's + Phase 0. Cold-reindex, query-latency, and RSS numbers are therefore not + yet measured; this section will carry real figures once that spike and + the Phase 5 recall-bench (`integrations/recall-bench/embedrefresh- + runner.mjs`) have run. + + - **`local-qwen3-0.6b`** — runs `Qwen/Qwen3-Embedding-0.6B`, same + runtime, last-token pooling instead of mean pooling. Exposes up to + 768d (the model's native 1024d is deliberately not offered yet). + ~1.5GB-class model footprint — larger than EmbeddingGemma, which is + why it ships as the documented alternative rather than a default + candidate. Same verification-honesty caveat as `local-embeddinggemma` + above; additionally, its last-token-pooling implementation path is a + [HYPOTHESIS] pending the same unrun spike (see the file header of + `src/search/providers/local-transformers.ts` for the specific kill + condition). - **`openai-3-small`** — calls OpenAI's `text-embedding-3-small` (1536-dim) over HTTPS. Fast (~2 min for a 44k-chunk vault vs ~25 min locally) but paid. Requires `OPENAI_API_KEY` in the server's environment; the key is never read from config files. Batched at 96 inputs per request, with exponential backoff on 429 / 5xx (up to 3 - retries). + retries). Untouched by the embedding-refresh spec — no dim/quantize + options, 1536d float32 rows behave exactly as before. - The active provider is set in `.daftari/config.yaml`: + The active provider, its dim, and its vec-index quantization are set in + `.daftari/config.yaml`: ```yaml embeddings: - provider: local-minilm # or: openai-3-small + provider: local-minilm # local-minilm | openai-3-small | local-embeddinggemma | local-qwen3-0.6b + # dim: 512 # local-embeddinggemma / local-qwen3-0.6b only: 512 (default) | 768 + # quantize: int8 # int8 (default for the two new providers) | none ``` - An unknown provider id, or `openai-3-small` with no `OPENAI_API_KEY` - in env, is a hard config error — the server refuses to start. Embedding - is best-effort at runtime: if the model cannot load (local) or the API - is unreachable (paid), a reindex still builds the FTS5 lexical index and chunks - land with no embedding row, so search degrades to lexical-only rather - than failing. - - Switching providers between server runs is safe: the `embeddings` table - is keyed by `(content_hash, model)`, so the new provider populates a - fresh row set on first reindex while the previous provider's rows stay - in the cache as cheap insurance for switching back. + An unknown provider id, an out-of-range `dim` for the active provider (or + any `dim` at all for `local-minilm`/`openai-3-small`, which have none to + configure), an unrecognised `quantize` value, or `openai-3-small` with no + `OPENAI_API_KEY` in env, is a hard config error — the server refuses to + start. Embedding is best-effort at runtime: if the model cannot load + (local) or the API is unreachable (paid), a reindex still builds the + FTS5 lexical index and chunks land with no embedding row, so search + degrades to lexical-only rather than failing. + + Switching providers, `dim`, or `quantize` between server runs is safe. + The durable `embeddings` cache is keyed by `(content_hash, model)` and + ALWAYS stores the FULL NATIVE-dim, float32 vector regardless of the + active provider's configured dim or quantize setting — truncation to the + configured dim happens at a single choke point (`toIndexDim`, + `src/search/vector.ts`) wherever a cached vector meets the sqlite-vec + index or a query. A provider switch populates a fresh row set on first + reindex while the previous provider's rows stay in the cache as cheap + insurance for switching back; a `dim` or `quantize` flip on an UNCHANGED + provider needs no re-embed at all — it is purely a vec-index-mirror + rebuild from the existing cache (`isIndexFresh` detects the staleness and + the next reindex is all cache hits, typically minutes not hours). + + **Vec-index quantization.** `embeddings.quantize: int8` (the default for + the two new local providers) stores the sqlite-vec `embeddings_vec` + mirror as `int8[dim]` instead of `FLOAT[dim]` — a calibration-free + `round(x * 127)` clamped to `[-127, 127]`, valid because every provider + L2-normalizes its output (components live in `[-1, 1]`). Search becomes + scan-then-rescore: the KNN scan over the quantized column selects + candidates by quantized distance ONLY (4x over-fetch); each candidate is + then rescored with exact float32 cosine similarity against the durable + cache, joined in the same query — the quantized distance never becomes a + score. A candidate whose cache row is unexpectedly missing (a GC race) is + dropped from the ranking rather than approximated. `quantize: none` (the + default for `local-minilm`/`openai-3-small`) keeps today's exact float32 + index, byte-for-byte. sqlite-vec's `vec_int8(?)` SQL wrapper is required + around both inserted and queried int8 vectors — a raw int8-byte blob + bound directly against an `int8[]` column is rejected by the pinned + sqlite-vec build. The model loads **lazily**: `getExtractor()` is invoked only when `embed()` actually has texts to embed, not at startup. With the @@ -229,25 +302,43 @@ Three things sit alongside the markdown: finished, so that search does not pay the ~500ms cold start. The warm-up is gated by the optional `warm_embeddings` flag in `.daftari/config.yaml` (default `true`); set it to `false` for read-only roles that never embed, - or for low-memory deployments where the ~100MB model footprint is - unwelcome. A warm-up failure (no network on first run, model download + or for low-memory deployments where the model footprint is unwelcome — + ~100MB-class for `local-minilm`, ~600MB-class for `local-embeddinggemma`, + ~1.5GB-class for `local-qwen3-0.6b` (all q8-ONNX-variant estimates, + pending real measurement — see the provider list above). A warm-up + failure (no network on first run, model download blocked) is logged but never crashes the server — the next `embed()` call - retries. + retries. The same flag covers the optional reranker below: once the + embedder warms, the server warms `rerank.provider` too when one is + configured — no separate knob, since `warm_embeddings`'s meaning ("pay + model cold-starts at startup, not on the first query") applies to either + model equally. Embeddings are stored in a separate, **content-addressed** `embeddings` table keyed by `(content_hash, model)`, with a `dim` column recording the vector dimension as defense-in-depth against a corrupt or cross-provider - mix. A `chunks` row carries the `sha256` of its text and joins to the - `embeddings` table for the current model. The consequence is the key - idea: an embedding is the property of a chunk's *text*, not of a file path - or its mtime. - - That property is what makes reindexing cheap. A reindex hashes every - chunk, asks the cache which hashes already have a row for the current - model, and only embeds the misses — so its cost scales with the number of - *changed chunks*, not the size of the vault. Edit one paragraph and you - re-embed one chunk; rename a file and you re-embed zero; move a paragraph - verbatim to another file and you re-embed zero. (The first reindex after a + mix. A `chunks` row carries the `sha256` of `embeddingInput` — the + breadcrumb context concatenated with the chunk's body text, per the + contextual-chunking note above — and joins to the `embeddings` table for + the current model. The consequence is the key idea: an embedding is the + property of a chunk's *retrieval identity* (context + text), not of a file + path or its mtime. + + That property is what makes reindexing cheap, with one honest cost. A + reindex hashes every chunk, asks the cache which hashes already have a row + for the current model, and only embeds the misses — so its cost scales + with the number of *changed chunks*, not the size of the vault. Edit one + paragraph and you re-embed one chunk; rename a file and you re-embed zero + (title/collection/tags unchanged, so the breadcrumb is unchanged); move a + paragraph verbatim to another file that shares the SAME title, collection, + and tag set and you re-embed zero. But because the breadcrumb is part of + the hash, **retitling a document, moving it between collections, or + changing its tag *set* re-embeds every one of its chunks** — the embedding + input genuinely changed, and a stale-vector cache hit would silently serve + pre-edit semantics, which is worse than paying the recompute. Tag + *reorder* is a no-op (tags are sorted before hashing). Curation flows that + touch metadata at scale (backfill, decay retitles, tag hygiene) should + batch their edits with this cost in mind. (The first reindex after a schema bump finds an empty cache, so it pays a one-time full embed; every reindex after that is incremental.) @@ -255,9 +346,38 @@ Three things sit alongside the markdown: chunks, the reindex runs an internal `vault_gc` step that drops embeddings rows whose `content_hash` is no longer referenced by any chunk, so orphans don't accumulate across edits. And the composite primary key on - `(content_hash, model)` is deliberate: a future model migration can keep - both the old and new model's embeddings present under the same hash, so a - roll-forward never has to clear the cache first. + `(content_hash, model)` is deliberate: a model migration keeps both the + old and new model's embeddings present under the same hash, so a + roll-forward never has to clear the cache first — this is exactly the + mechanism the `local-embeddinggemma`/`local-qwen3-0.6b` migration above + rides, not a new one. + + **Optional local reranker.** A `RerankProvider` seam + (`src/search/rerank-provider.ts`) mirrors `EmbeddingProvider`: config- + selected, memoised per process, `warm()`/lazy-load, `Result`-returning + with graceful degradation. It has one real provider today, + `local-bge-m3` — `BAAI/bge-reranker-v2-m3`, ONNX q8, via the same + `@huggingface/transformers` runtime `local-minilm` uses (zero new + dependencies) — and defaults to `none` (off): + + ```yaml + rerank: + provider: none # none | local-bge-m3 + ``` + + When configured, `vault_search` scores the top-50 RBAC-filtered hits + against the query with the cross-encoder and reorders them, between the + RBAC filter and the slice to `limit` — after RBAC so cross-encoder budget + is never spent on a hit the caller cannot see, before the slice so a + fused-#12 hit can still land #1 of a limit-10 page. `vault_search`'s + result gains `rerankUsed: boolean` — `false` covers `none`, a not-yet-warm + model (a background warm fires instead of blocking the call), a provider + error, and a 1.5s-timeout alike, matching `vectorUsed`'s honest-degrade + shape. The ~600MB q8 weights are an order of magnitude past + `local-minilm`'s footprint, which is why this stays opt-in rather than + defaulting on: the same "ship behind a flag, measure, then flip the + default on evidence" playbook chunk-level BM25 used for its v1.29.0 + default flip. - **SQLite lock store** (`.daftari/locks.db`). Holds active write locks. Also ephemeral. @@ -668,22 +788,55 @@ suggest changes without ever enacting them. The queue has two ends. `vault_stage_action` is the producer (normally the loop, exposed for testing and future callers): it records a proposed `promote` / -`deprecate` / `supersede` / `merge` / `confidence-up` with a rationale, a -proposed diff, and a TTL (default 14 days). `vault_ratify` is the consumer: a -human `approve`s or `reject`s one pending action. On approve it dispatches to the -matching write tool, which auto-commits — `promote` → `vault_promote`, -`deprecate` → `vault_deprecate`, `supersede` → `vault_supersede`, -`confidence-up` → `vault_set_confidence`, `merge` → `vault_merge` (the §11.4 -write tools). A dispatch failure, including a malformed proposed diff, leaves the -action pending so it can be retried. (The legacy `ratified-pending-tool` status, -from before §11.4 wired up the last three tools, is no longer produced.) +`deprecate` / `supersede` / `merge` / `confidence-up` / `write` with a +rationale, a proposed diff, and a TTL (default 14 days). `vault_ratify` is the +consumer: a human `approve`s or `reject`s one pending action, or up to +`BATCH_RATIFY_MAX` (20) at once via an explicit `ids` list — never a threshold +or an "all pending" sentinel; each id is processed independently, so a +gate-blocked or failing id leaves only that action pending while the rest land +(2026-07-26 risk-triaged-ratification spec, Decision 2). On approve it +dispatches to the matching write tool, which auto-commits — `promote` → +`vault_promote`, `deprecate` → `vault_deprecate`, `supersede` → +`vault_supersede`, `confidence-up` → `vault_set_confidence`, `merge` → +`vault_merge`, `write` → `vault_write` (the §11.4 / #235 write tools). A +dispatch failure, including a malformed proposed diff, leaves the action +pending so it can be retried. (The legacy `ratified-pending-tool` status, from +before §11.4 wired up the last three tools, is no longer produced.) + +Every verdict carries a machine-readable `decision_kind` +(`approve` | `edit-then-approve` | `reject`, derived server-side) and, on +reject or `edit-then-approve`, a closed `reason_category` — **reject now +REQUIRES a category; this is an intentional, spec-mandated break from the +prior optional contract for reject callers only** (approve-path callers are +untouched). `vault_ratify` also accepts an optional `amended_diff` on a +single-id approve: the tier-0 gates and the dispatch run against the amendment +instead of the staged diff, and the decision record keeps both what was +proposed and what actually landed. Under `shadow_mode`, `amended_diff` errors +rather than silently discarding the amendment — shadow mode records no +decisions of any kind. The witness (below) folds `edited` and per-category +counts into each principal's proposal record. Storage mirrors the rest of Daftari: an append-only canonical log at `.daftari/staged-actions.jsonl` is the source of truth, with a derived `staged_actions` table in the ephemeral index rebuilt from it. `vault_lint` -surfaces pending actions soonest-to-expire first and expires past-TTL ones as a -housekeeping sweep on each run — the queue can grow stale, but it never grows -unbounded. +surfaces pending actions **risk descending, soonest-to-expire as the +tiebreak** (inverted from the prior expiry-only sort by the 2026-07-26 +risk-triaged-ratification spec) and expires past-TTL ones as a housekeeping +sweep on each run — the queue can grow stale, but it never grows unbounded. +The risk score — a weighted sum of six deterministic terms (action-kind +severity, diff size, blast radius, open tension, conflict/retry markers, +proposer track record) — is **derived on every read, never stored**: no +`risk` field is appended to the jsonl and no column lands in the sqlite +table, the same posture `derives_from` strength takes. Like every other +queue listing, it is filtered to the caller's vantage: an item whose target +is unreadable is omitted, and the hidden remainder is reported coarsened +(none/some/many), never as an exact count. Each ratify/reject decision +additionally carries a **non-authoritative `risk_at_decision`** snapshot +(jsonl-only, never mirrored to sqlite, never read for ordering) — a frozen +observation of the score at decision time, so the spec's first kill +condition (partition decisions by risk quartile, compare correction rates) +can be evaluated without reconstructing blast radius or tension state as of +a past instant. #### derives_from edges @@ -801,6 +954,74 @@ ratification-queue depth), and `vault_ratify` returns This is the calibration posture the cortex loop runs in until coverage/equity ratchets clear and the auto-write tier graduates. +#### Independence-aware promotion (shadow calibration) + +`k_survived` counts *attestations*, not *independent evidence* — a panel of M +votes from the same loop pass, same model, same principal, over the same two +truncated endpoint texts, differs only in which prompt template ran, and the +decorrelation verdict already measured that axis's lift at ~0. The +independence-aware-promotion spec +(`docs/superpowers/specs/2026-07-26-independence-aware-promotion-design.md`) +closes that gap: every `observe` may now carry an evidence fingerprint +`fp: { inputs, principal, model, prompt }`, `inputs` a mechanical hash over the +bytes actually read; the store partitions each edge's COUNTED votes into +equivalence classes (agree on inputs+principal+model, `prompt` excluded) and +computes a geometrically-discounted `k_eff` alongside the untouched raw +`k_survived`. `k_eff` and its aged `strengthIndependent` are shadow-only — +materialized, exported, and journaled, but live `strength`/`status` still key +on `k_survived` exclusively. + +The revision loop journals one row per panel to +`.daftari/independence-shadow.jsonl` (`would_accrue | would_needs_review | +null`, the last for fails/tie/no-vote/gated panels) and, only once graduated +(`independence_graduated: true`, default false), a correlated-only +survives-majority panel becomes `needs-review` instead of `survives`: no +observes, an interpretive tension instead +(`correlated-only survival: derives_from `), and the loop parks +that edge — it does not re-panel it — while the tension stays open. The edge +keeps aging under Decision 2's normal clock throughout the parked window: +decay-pending-adjudication is the deliberately conservative default, since the +alternative (refreshing the clock on correlated evidence) lets a suspect edge +coast trigger-bearing indefinitely under an ignored tension. + +`vault_lint`'s `independenceCalibration` section is the graduation dashboard: +the k-vs-k_eff distribution, would-drop-below-trigger counts (split +legacy-only vs signal), and the would-be needs-review rate. Graduate only +after a full quarterly shadow window and only if (a) the would-be +needs-review rate is stable and the risk-triaged queue can drain it, (b) a +hand audit of 20 would-be-demoted edges finds a majority genuinely +correlated, and (c) `ρ`, the class-key component set, and +`EDGE_NEEDS_REVIEW_MIN_GAIN` survived the window without retuning. **Warm-up +rule:** criterion (a) is judged on `wouldNeedsReviewRateInformative` +(`informativePanels` restricted to rows whose pre-panel classes already carry +a non-∅ key), not the raw rate — a fingerprinted class key can never equal +`∅`, so every legacy edge's *first* fingerprinted panel reads `would_accrue` +by construction, and the raw rate is degenerate for at least the first +post-ship revision cycle. Two independent kill conditions, evaluated +separately: if the hand audit finds the discount isn't discriminating (the +class key too coarse), recalibrate the component set once and re-shadow, or +if a second window still fails, delete the `k_eff` scoring + Decision-3 +machinery and keep only the `fp` recording (the fingerprint trail stays +valuable as provenance even if the scoring dies); if the would-be +needs-review rate exceeds what the triage queue demonstrably drains, fold +correlated-only survival into a lint counter and drop the needs-review +outcome, keeping Decisions 1-2. + +Two caveats the Decision-4 hand audit must read the sample through. **Byte- +stability, not independence:** `fp.inputs` hashes the endpoint bytes at +observe time, so it detects "did the text change", not "was this derivation +independent" — any edit mints a fresh class, so an actively-edited edge reads +`k_eff ≈ k_survived` regardless of true correlation, and the flagged +population skews toward byte-frozen docs. A frozen-doc-only sample is +evidence toward the first kill condition (the class-key design may be wrong), +not clean validation. **Attestation, not verification (C3):** `fp.inputs` is +server-computed and `fp.principal` is server-derived from the RBAC access +context, but `fp.model`/`fp.prompt` are caller-attested — the same trust +class `blind`/`varied_axis` already occupy. `nonLoopFingerprintedCountedVotes` +(view + lint section) reports how much of the class structure this attested, +not verified, component set rests on; the graduation reading treats it +accordingly. + Advisory-by-design is the point: an agent maintains the vault, but no automated process silently rewrites or deletes knowledge. Every change is a deliberate, attributable act. The staged-action queue is the same principle pushed one step @@ -929,6 +1150,76 @@ The cortex quality sampler (`daftari eval`) follows the same edge kind: vault-resident code loads as a separate, non-citable context node, so the answerer is never asked to retrieve code on the agent's behalf. +### Citation anchors — just-in-time verification at read time + +`daftari audit` is a batch sweep the operator has to run; between audits, a +`describes` binding is inert. Citation anchors close that gap. A binding may +carry a **pin** — `repo:path[#L-]@`, the git blob id (and +optional line range) the author looked at when the binding was written — +and `vault_read` verifies every pin against the locally checked-out +`code_repos` at read time, the exact moment an agent is about to act on the +doc's account of the code. See docs/superpowers/specs/2026-07-26-citation- +anchors-jit-verification-design.md for the full grammar and the four-step +git-plumbing classifier (`intact` / `moved` / `missing`). + +The pin grammar's suffix pattern is end-anchored and sha-strict, so it is +backward compatible by construction — but a path that itself *ends* in text +matching the pin shape (`#L[-]@<7-40 lowercase hex>`) is a known, +accepted ambiguity: **the pin wins**, the trailing text is parsed as a pin +rather than as part of the path. This is pathological, not silent: the +stripped path no longer resolves against the code repo, and a real +collision surfaces as a `broken_describes` finding at audit time, exactly +like any other missing target. + +Verification is advisory always (the curation house rule): a `moved` or +`missing` pin never auto-invalidates, demotes, or rewrites the doc — it +tells the reading agent to re-verify before trusting the doc's account of +the code. An intact pin on a past-TTL doc softens the decay banner's copy +(annotate, never extend — the TTL clock itself never moves) because the one +thing the pin actually verified — this code — has not changed, even though +the doc as a whole is stale by the clock. + +Configuring `code_repos` makes blob-level facts about those repos — path +existence, whether a pinned blob still matches the current file, and +relocated line numbers for a moved pin — visible to a reader. That +visibility is gated per role: a principal sees the `anchors` annotation only +where their role can already read the pinned doc **and** the role carries +the `code_repo_visibility` grant in `.daftari/config.yaml` (default off for +every non-operator role). A server run without an access context (stdio, +no `--role`) is unaffected by the gate — the operator posture the rest of +this document assumes throughout. + +## Progressive tool disclosure and task briefs + +Two coupled tools attack the context cost of a daftari session before the +first document is ever read (spec 2026-07-26-context-packs-progressive- +disclosure-design.md). + +`vault_tools` is a pure advertisement seam over the registry `src/tools/ +registry.ts` assembles from every `src/tools/*.ts` file's exports. Index mode +returns a `{name, oneLine}` line per registered tool; expand mode returns full +schemas for named tools. It reads the FULL registry minus the vault's +`exclude` config list — exclude always wins (#104) — but tier and `include` +never narrow it, because making tiered-out tools discoverable in-band is the +tool's entire purpose. `CallTool` is unaffected either way: every registered +name stays callable regardless of what any tool advertises (#103). + +`vault_context(task, budget?)` is the assembly layer over the search stack: +hybrid retrieval, RBAC filtered BEFORE any budgeting, supersession dedup (a +collapsed chain's flags are all keyed on the head's OWN index row — never a +stale member's), then a greedy budget cut at `budget * 0.9` estimated tokens +(chars/4, no tokenizer — `src/context/estimate.ts`), then markdown +templating (`src/context/assemble.ts`, pure and deterministic — no I/O, no +LLM call). The pack selects and points; it never synthesizes: an open tension +renders both claims verbatim, never a blended verdict, and a supersession +prints only the pointer and hop count, the chain head's own content carrying +the snippet. `hidden_remainder` is a LOWER-BOUND signal over observable +withholding (RBAC-dropped BM25-side pool candidates, dropped coverage +additions, restricted supersession hops) — "none" means "no withholding +observed," never "nothing withheld," because the vector half of retrieval is +already RBAC-pushdown-scrubbed (Decision 3 of the retrieval-fusion-overhaul +spec) and structurally invisible to this count. + ## A fact's life — the request path Everything above is the machinery at rest. Watch it move, and the four layers @@ -983,9 +1274,19 @@ quietly settled. That is the entire product, in one fact's lifetime. Denied collections are filtered out of results entirely. 3. **Layer 1** reads the markdown (or queries the index) and returns it, with an advisory frontmatter validation report attached. -4. (`vault_search` only) Two additive, lossless post-passes run on the - RBAC-filtered hit list — never re-ranking, never leaking content from - denied collections: +4. (`vault_search` only, optional) **Rerank stage.** When `rerank.provider` + is configured, a local cross-encoder (`local-bge-m3`) re-scores the + top-50 RBAC-filtered hits against the query and reorders them — after + RBAC, before the slice to `limit`, so cross-encoder budget is never spent + on a hit the caller cannot see and a fused-#12 hit can still land #1 of a + limit-10 page. Skipped (fused order stands, `rerankUsed: false`) when no + provider is configured, the model is not yet warm (a background warm is + fired instead — reranking never triggers a synchronous model load inside + a tool call), the provider errors, or scoring exceeds a 1.5s timeout. See + `docs/superpowers/specs/2026-07-26-contextual-chunking-reranker-design.md`. +5. (`vault_search` only) Two additive, lossless post-passes run on the + reranked (or, if skipped, fused) RBAC-filtered hit list — never + re-ranking, never leaking content from denied collections: - **Coverage pass.** When the top seeds share a frontmatter tag with at least two of the top-K, the index is queried for other docs carrying that tag inside the seeds' `created`-date window @@ -1044,6 +1345,23 @@ staged-action queue and the unresolved-tension count growing without bound acros real use. If they do, advisory restraint was a luxury for small vaults, not a principle. +The 2026-07-26 risk-triaged-ratification spec is this wager's defense: it puts +the reviewer's scarce attention on the proposals a wrong verdict costs the most, +rather than the ones that merely expire soonest. It ships with three of its own +kill conditions, checkable in the system's own numbers, not this document's +prose: **(1)** the risk score must predict corrections — after a real body of +decisions, the reject + edit-then-approve rate in the top risk quartile at +decision time must beat the bottom quartile, or the score is decorative and gets +cut while the outcome logging stays; **(2)** batch ratify must not become the +rubber stamp with better lighting — if batch approvals come to dominate with a +near-zero in-batch reject rate while the arrival rate keeps climbing, enumerated +batching failed its one job and goes; **(3)** `reviewThroughputSummary` stays the +judge — if expiries keep climbing under risk triage, ordering was never the +bottleneck and the honest fix is upstream in the proposal budget, not another +pass over the queue. Condition (1) reads `risk_at_decision`, the non-authoritative +snapshot each decision record carries — see Staged actions above — rather than +approximating a past instant's blast radius and tension state from present data. + **One identity per process** makes access control a single flag instead of a user database — but it pushes multi-tenancy out to deployment: you get N identities by running N processes, not by authenticating N callers at runtime. That trade is @@ -1056,7 +1374,29 @@ staying *cheap*. It is cheap because embeddings are content-addressed and only changed chunks re-embed — but a first cold reindex on a large vault is already multi-minute. If that ever becomes multi-hour, "delete the `.db` files and continue" stops being a real fallback and becomes a threat, and the disposability -I keep advertising is disposability you can't afford to use. +I keep advertising is disposability you can't afford to use. 1.33.0's contextual- +chunking schema bump (SCHEMA_VERSION 10 → 11, spec 2026-07-26-contextual- +chunking-reranker-design.md Decision 3) pays exactly this cost once, deliberately +and loudly, not lazily: every chunk's hash input changed, so the first post- +upgrade reindex is a full cold re-embed of the whole corpus (~25 min local- +minilm / ~2 min openai-3-small for a 44k-chunk reference vault; see the 1.33.0 +release notes for the exact numbers). The same trade recurs in smaller doses +after that: because the breadcrumb context is now part of a chunk's hash, +retitling a document, moving it between collections, or changing its tag *set* +re-embeds every one of that document's chunks — correct (the retrieval identity +genuinely changed), but a cost curation flows that touch metadata at scale +(backfill, decay retitles, tag hygiene) need to batch around rather than trigger +document-by-document. + +**The reranker is a second cost lever, opt-in.** `rerank.provider: local-bge-m3` +adds a ~600MB local cross-encoder to the search path — an order of magnitude +past local-minilm's footprint — with a hard 1.5s per-search timeout so a slow or +cold model degrades to the fused order rather than hanging a tool call. It ships +default-off, behind the same "measure before flipping the default" playbook +chunk-level BM25 used in v1.29.0: if the measured recall lift never clears the +bar, or serve-mode concurrency under real load turns out to serialize badly on +one CPU, it stays permanently opt-in rather than becoming everyone's default +latency tax. **The locks neither queue nor merge.** This is sufficient *because* agents usually write to different documents. If contention on a few hot documents turns out to be diff --git a/docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification-design.md b/docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification-design.md index 0f249fb5..ca3f70fa 100644 --- a/docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification-design.md +++ b/docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification-design.md @@ -1,7 +1,19 @@ # Citation anchors and just-in-time verification — design -2026-07-26. Status: **proposed — awaiting Mihir's review; implementation not -started.** +2026-07-26. Status: **implemented (2026-07-28)**, after Jugalbandi dialectical +review. The final plan resolved ten challenges and escalated one — C4, "no new +disclosure surface" — to Mihir (see `.jugalbandi/citation-anchors-jit/final- +plan.md`). Mihir's 2026-07-27 decision: gate the `anchors` annotation per +role/collection from day one, not as a future step (see the amended Decision +2 disclosure paragraph and the new "role gate" note below). The amendments in +this text (the read-path batching in Decision 2, the symlink-confinement and +CRLF/trivial-content hardening in Decision 2's classifier steps, the +disclosure-posture correction below, and the kill-condition amendment) are +the disposition of that review, applied in place because the spec was still +pre-implementation when the review landed — not a silent deviation from what +shipped. `daftari audit --pin`/`--pin --apply`, `vault_lint`'s malformedPins +check and Decision-4 softening, and the `code_repo_visibility` role grant +described below are all implemented. Predecessor specs: 2026-05-30 (coherence audit — the surface this extends), 2026-06-09 (backfill — the plan/apply precedent Decision 5 reuses), 2026-07-20 (self-hosted server mode — the config posture). @@ -144,12 +156,29 @@ network: 4. Otherwise (whole-file pin with a differing blob, or a pinned blob git no longer has) → **`moved`**. -**Cheap by construction.** At most two git invocations plus one bounded -file read per pin; pins per read are capped at a fixed constant (24), the -remainder reported as skipped with a count. Any git failure degrades that -binding's entry to absent, and the read never fails on the check (the -recordRead best-effort contract). And the operator holds a kill-switch: -`jit_anchors: false` removes the entire code path. +**Cheap by construction — amended (2026-07-27 plan resolution, C1).** The +original claim — "at most two git invocations per pin" — blows its own 50ms +kill threshold on the hot path: 24 sequential `execFile` spawns land at or +over budget at even ~3-5ms each, and the CI tripwire (originally 500ms) was +10x too loose to catch it. The implemented design batches instead: existence +(step 1) is a realpath/stat check, no subprocess; step 2 (current blob hash) +is ONE `git hash-object` invocation per REPO REFERENCED, not per pin, via +`hashObjects` in `src/utils/git.ts` — so the all-intact case, the common +one, costs one subprocess per referenced repo per read. Only pins whose blob +differs AND carry a range proceed to step 3 (`git cat-file -s` + a bounded +`cat-file blob`), run with a small bounded concurrency (4) so several +drifted range pins on one doc don't serialize behind each other. The +"at most two invocations per pin" claim now holds on the hot (intact) path +and is knowingly exceeded by one `cat-file -s` size gate on the cold +(drift) path. Pins per read are still capped at a fixed constant (24 +— `MAX_PINS_PER_READ`), the remainder reported as skipped with a count. Any +classifier failure (a repo's whole batch call erroring) degrades that +repo's pending entries to `errored` (C8) rather than silently vanishing, +and the read never fails on the check (the recordRead best-effort +contract). The CI tripwire tightened to 24 intact pins across 2 repos +classifying under 150ms; the live 50ms p95 measurement on a real vault +remains the authoritative post-ship check. The operator still holds a +kill-switch: `jit_anchors: false` removes the entire code path. **Annotation shape**, following the read path's null-when-silent contract (`decay`, `upstream_staleness`, `structural`): @@ -165,15 +194,50 @@ anchors: { }>; checked: number; skipped: number; // over-cap remainder + errored: number; // classifier failures, dropped from + // entries (2026-07-27 resolution, C8: + // keeps the Decision 4 softening from + // quantifying over a censored sample) banner: string | null; // the decay-banner idiom -} | null // no pinned bindings, no resolvable repo, or jit_anchors: false +} | null // no pinned bindings, no resolvable repo, jit_anchors: false, OR + // (2026-07-27 resolution) the caller's role lacks + // code_repo_visibility — see the disclosure-posture note below ``` -**No new disclosure surface.** The annotation derives solely from the -doc's own frontmatter (already visible to any caller who can read the doc) -plus a server-local code tree. It names no other vault document, so the -2026-07-14 omission/existence-disclosure rules gain no new edge here; in -serve mode the annotation is identical across sessions by construction. +**Disclosure posture — corrected.** The original text here claimed "no new +disclosure surface": the annotation names no other vault document, so the +2026-07-14 omission/existence-disclosure rules gain no new edge. That claim +is true as far as it goes but incomplete, and the Jugalbandi Challenger +(C4) caught it: the annotation is a per-read oracle over a repo that RBAC's +collection model has no vocabulary for at all. Any principal who can read a +pinned doc can, via writer-controlled pins, learn path existence in a +configured `code_repos` tree, whether a specific blob still matches the +current file, and (via step 3) where matching content now lives — facts +about a *filesystem outside the vault*, not about another vault document. +The 2026-07-14 rules were written for vault-internal edges; they say +nothing about this because this oracle didn't exist yet. + +Mihir's 2026-07-27 decision: gate it, from day one, not as a follow-up. +`.daftari/config.yaml` gains a per-role `code_repo_visibility` grant +(default **off** for every non-operator role). The `anchors` field on +`vault_read`'s result is null unless BOTH hold: the caller's role can +already read the pinned doc (the existing collection-scoped `read` grant), +AND the role carries `code_repo_visibility`. The underlying git +classification still runs regardless of the gate — kill-condition (b) +instrumentation (`anchors_moved`/`anchors_missing`/`anchors_errored` on the +read log) is local operator telemetry, unfiltered by role, the same posture +`broken_upstream` already takes — but the RETURNED annotation, and the +Decision 4 decay-banner softening derived from it, are both gated. A caller +with no `AccessContext` at all (stdio without `--role`, or any direct +in-process call) is unaffected by the gate, matching every other RBAC check +in this codebase. Configuring `code_repos` at all is therefore an explicit +operator choice with a stated cost: it makes blob-level facts about that +repo visible to every reader the operator has granted `code_repo_visibility` +— see `docs/architecture.md`'s "Citation anchors" section for the +operator-facing statement of this posture. In serve mode the annotation is +identical across sessions for the same role by construction (deterministic +git state at query time), so no session-to-session leak exists beyond the +role gate itself. ## Decision 3 — advisory consequences only; the batch audit gains the same classifier @@ -280,3 +344,25 @@ Decision 4's softened copy dies on its own if reviewers judge all-pins-intact docs stale anyway in practice — copy that teaches agents to discount TTL would be exactly the freshness-laundering the annotate-only rule exists to prevent. + +**Amendment (2026-07-27 plan resolution, C9): condition (b) cannot be fully +measured by the instrumentation this design builds.** A code re-read +happens in the agent's own tool-use loop, outside Daftari entirely — the +read log has no join key into it, and never will without instrumenting the +agent harness. The observable subset is real and worth measuring: a doc +update, a pin refresh, or a tension logged *within the same run* following +a `moved`/`missing` annotation, joined from the read log +(`anchors_moved`/`anchors_missing`, per-run via `run_id`) against +subsequent writes and `daftari audit --auto-tension` output. But absence of +those signals in that joined data is, on its own, **insufficient** to kill +the read-path check — it is equally consistent with "agents re-read the +code and said nothing else about it" (the intended, cheapest-possible +consequence: an agent that re-reads before acting has already done the +thing pins exist to prompt) and with "agents ignore the flag entirely." +Before reverting on (b), the evaluation must ALSO include a spot check of +agent transcripts, or an operator interview, specifically probing for the +invisible consequence — code re-reads that never produced a Daftari-visible +side effect. The read-log fields are documented, here and in code, as +measuring exactly what they measure (the observable subset) and nothing +more; a dashboard or report built on them must carry that caveat forward +rather than presenting the joined count as the full picture. diff --git a/docs/superpowers/specs/2026-07-26-independence-aware-promotion-design.md b/docs/superpowers/specs/2026-07-26-independence-aware-promotion-design.md index 884610ad..d9cb9edf 100644 --- a/docs/superpowers/specs/2026-07-26-independence-aware-promotion-design.md +++ b/docs/superpowers/specs/2026-07-26-independence-aware-promotion-design.md @@ -133,16 +133,36 @@ S = min(k_eff, K_max) × (1/2)^(Δt / 90d) ## Decision 3 — three-way verdict in `--mode revision`: the needs-review outcome Today the panel aggregates to `survives | fails | tie | no-vote | gated` -(`RevisionDecision`, revision.ts:70). This spec splits *survives* on independence: +(`RevisionDecision`, revision.ts:70). This spec splits *survives* on independence. -- **survives-independent** — the panel's majority survives **and** at least one surviving - vote opens a new equivalence class against the edge's existing cycle trail (its class - key is not already present). → accrue: apply the observes, exactly today's path. +**Operative rule (amended 2026-07-26, PR-2 of the resolved plan):** apply the surviving +votes' evidence-class keys, in order, against the edge's existing cycle trail and sum the +marginal `k_eff` gain (`independenceVerdict`, src/consolidate/independence.ts) — + +- **survives-independent** — the surviving votes' marginal `k_eff` gain is **≥ + `EDGE_NEEDS_REVIEW_MIN_GAIN = 0.5`**. → accrue: apply the observes, exactly today's path. - **fails** — unchanged: majority-fails → one `vault_edge_contest`, revoke + tension. -- **correlated-only survival** — the majority survives but **every** surviving vote lands - in an already-present class (marginal `k_eff` gain below - `EDGE_NEEDS_REVIEW_MIN_GAIN = 0.5`, i.e. not even one half-fresh vote). → **needs-review**: - apply **no observes**, and surface for human adjudication. +- **correlated-only survival** — the majority survives but the marginal gain is + **strictly below** `EDGE_NEEDS_REVIEW_MIN_GAIN`. → **needs-review**: apply **no + observes**, and surface for human adjudication. + +"Opens a new equivalence class against the edge's existing cycle trail" is the intuition, +not the literal test — the literal test is the marginal-gain threshold above, because a +*second* vote landing in an already-present class is not "no new class" but still carries +real (discounted) evidential weight. Boundary case: a second vote in a count-1 class gains +exactly `EDGE_INDEPENDENCE_RHO ** 1 = 0.5` — "one half-fresh vote" — which is **not** below +the floor, so it accrues (survives-independent), matching this spec's own parenthetical +above ("not even one half-fresh vote" describes what needs-review requires: strictly less +than half a fresh vote's worth of marginal gain). + +**Parking (added 2026-07-26, PR-2, disposition C1):** while a `correlated-only survival` +tension is open on an edge, the revision loop does not re-panel that edge — it reported +its doubt once and waits. The edge continues to age under Decision 2's normal clock +(decay-pending-adjudication is the conservative failure mode: the edge is a `derives_from` +claim currently underwritten only by correlated evidence, so letting it coast +trigger-bearing while the tension sits unresolved is exactly the risk this spec exists to +close). Resolution — a genuinely independent re-derivation, or a contest — unparks the +edge for the next due cycle. How needs-review surfaces — argued, not defaulted: the staged-action kinds (`promote | deprecate | supersede | merge | confidence-up | write`, diff --git a/docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md b/docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md index c8ea2bd0..98c0fac7 100644 --- a/docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md +++ b/docs/superpowers/specs/2026-07-26-mcp-2026-07-28-readiness-design.md @@ -1,7 +1,10 @@ # MCP 2026-07-28 readiness — design -2026-07-26. Status: **proposed — awaiting Mihir's review; implementation not -started.** +2026-07-26. Status: **implemented except Decision 4 — see the kill-condition +outcomes at the bottom.** Decisions 2, 3, and 6 landed 2026-07-26 (#302); +Decisions 1 and 5 landed 2026-07-28 against the final revision on the v2 SDK +line (`@modelcontextprotocol/server` 2.0). Decision 4's kill condition fired +— the passes remain CLI-only. The final "stateless MCP" protocol revision (2026-07-28, RC published 2026-05-21) lands two days after this spec's date. This document settles what daftari adopts, what it defers, and what it will never adopt. @@ -249,7 +252,9 @@ posture: **the server proposes, the human disposes**, and now the wire format itself says so. RBAC is unchanged — the resubmitted decision is enforced against the requester's `ratify` grant exactly as today, and a direct call with the decision inline keeps working for clients that don't -do elicitation. +do elicitation. This form-mode path is single-`id` only: the risk-triaged +ratification spec's batch `ids` call always requires an explicit `decision` +— a batch has no single action's rationale to put in a form. ## Decision 6 — sampling: never. Stated once, here, so it stops being implicit @@ -301,3 +306,38 @@ go direct — the transport never carries inference in either direction. Each decision lands as its own PR, in the order written; Decision 1 gates only Decision 4 (tasks ride the new transport's types). + +## Kill-condition outcomes (2026-07-28, checked against the final revision) + +- **Decision 1 — survived.** [DATA] The final revision matches the RC in the + load-bearing places: `initialize` and `Mcp-Session-Id` are gone, client + info rides `_meta`, `server/discover` replaces the upfront capability + exchange. Implemented on `@modelcontextprotocol/server` 2.0 + + `@modelcontextprotocol/node` 2.0 (the stable line published with the final + revision; the monolithic 1.x SDK survives only as a devDependency so the + e2e suite keeps proving a lagging stdio client works). Serve is + `legacy: "reject"`; stdio serves both eras from one factory. +- **Decision 4 — KILLED, for now.** [DATA] Two independent hits: + (a) the final revision moved Tasks to a standalone extension and **removed + `tasks/list` outright** (unsafe without sessions — only `tasks/get`, + `tasks/update`, `tasks/cancel` remain), so this spec's task-list + RBAC posture is moot until re-specified; (b) the v2 TypeScript SDK ships + the task types as "2025-11-25 wire vocabulary with no SDK runtime; kept + importable for interoperability only" — `execution.taskSupport` and + `capabilities.tasks` are deleted fields in the 2026 wire shape, and no + tasks-extension runtime package exists on npm. Exactly the spec's named + outcome: the four task tools do not ship; `sleep`, `consolidate`, `audit`, + and `eval` remain CLI-only — today's world, costing the wait. Revisit when + the TS SDK line ships the extension runtime; note the `tasks/list` removal + simplifies the omission-over-redaction bullet (handle-holders only). +- **Decision 5 — survived.** [DATA] Stateless elicitation landed as + `InputRequiredResult` with opaque client-resubmitted `requestState` + (field names differ from the RC-era sketch — `inputRequests` / + `requestState` / `inputResponses` — but the model is materially the one + specified: the server remembers nothing between rounds). Implemented with + the SDK's HMAC request-state codec; the state binds action id + vault HEAD + + deciding user. Landed alongside the risk-triaged-ratification spec's + batch `ids` extension to `vault_ratify` — elicitation only ever applies to + a single-`id` call; a batch call always carries an explicit `decision`. +- **Decisions 2, 3, 6 — no protocol risk materialized**, as predicted + (#302, 2026-07-26). diff --git a/docs/superpowers/specs/2026-07-26-risk-triaged-ratification-design.md b/docs/superpowers/specs/2026-07-26-risk-triaged-ratification-design.md index 6980218d..8f0e90f9 100644 --- a/docs/superpowers/specs/2026-07-26-risk-triaged-ratification-design.md +++ b/docs/superpowers/specs/2026-07-26-risk-triaged-ratification-design.md @@ -1,7 +1,14 @@ # Risk-triaged ratification — design -2026-07-26. Status: **proposed — awaiting Mihir's review; implementation not -started.** +2026-07-26. Status: **implemented (2026-07-28)**, after Jugalbandi dialectical +review. The final plan resolved eight challenges (see +`.jugalbandi/risk-triaged-ratification/final-plan.md`) and Mihir's 2026-07-27 +decision on the one escalated contradiction (Decision 1 vs. kill condition #1, +below). The amendments below (C bullet, diff-size buckets, T-term +canonicalization, W/witness keying, and the Decision 1 carve-out) are the +disposition of that review, applied to this text per the plan's Phase 6 — +corrected in place, not silently deviated from, because the spec was still +"proposed" when implementation started. ## Why @@ -48,6 +55,21 @@ recomputed estimate. Derivation also keeps the log honest: the JSONL remains a record of what was *proposed and decided*, never of what some scorer once thought. +**Amendment (Mihir, 2026-07-27), narrow carve-out:** every decision record +(ratify or reject) additionally carries a `risk_at_decision` field — +JSONL-only, still never mirrored to sqlite, and **never read to influence +queue ordering** (`rankPendingActions` does not read it; nothing does but the +kill-condition-#1 analysis). It resolves the contradiction between this +paragraph's ban and Kill condition #1's need for "risk quartile at decision +time": B and T (0.40 of the total weight) are not replayable from the +present-state log alone once the vault has moved on (a dependent gets +written, a tension resolves), so a *frozen observation* stamped at the moment +of decision is the only way to make that condition evaluable without +approximation. This is a snapshot of a fact ("what the score computed to, +right then"), not a stored authority ("what the score currently is") — the +same distinction `ratified_at` / `ratified_by` already draw for every other +decision field. + The score is a weighted sum of six deterministic terms, no LLM anywhere: $$R(a) = \mathrm{clamp}_{[0,1]}\big(w_K K + w_D D + w_B B + w_T T + w_C C + w_W W\big)$$ @@ -61,30 +83,65 @@ $$R(a) = \mathrm{clamp}_{[0,1]}\big(w_K K + w_D D + w_B B + w_T T + w_C C + w_W serialized `proposed_diff` — log-scaled so a 10 KB payload saturates and a one-field delta stays near zero. Large diffs are the literature's first routing trigger, and the size is already sitting in the record. + **Amendment (2026-07-27 final plan, C3):** the queue item's displayed + `small`/`medium`/`large` diff-size bucket is defined over **raw serialized + bytes** (`< 256` / `< 4096` / `≥ 4096`), decoupled from D's own formula + above. D<0.25 is unreachable in fewer than 9 bytes — no valid payload is + that small — so a bucket boundary drawn on D itself made `small` + mathematically unreachable and every lifecycle action read "medium." D's + formula is unchanged; only the display bucket moved to raw bytes. - **B — blast radius of the target.** Reuses the vault_tension_blast machinery (src/curation/tension-blast.ts): `min(1, primary/10 + - advisory/40)` over the target's inbound `sources` and link edges. The - reverse-source and reverse-link maps are already built on demand from - loaded docs; scoring N pending actions builds them once and probes N - targets — no new graph state. + advisory/40)` over the target's **direct (distance-1)** inbound `sources` + and link edges — the reverse-source and reverse-link maps are already + built on demand from loaded docs; scoring N pending actions builds them + once and probes N targets with no traversal (`computeBlast`'s BFS is not + called) — no new graph state. - **T — open tension.** 1 when the target sits in an unresolved tension (the `contestedDocs` set the witness already computes), else 0. Ruling on a contested doc is exactly the verdict that deserves a human's full - attention. + attention. **Amendment (2026-07-27 final plan, C5):** tension endpoints + (`sourceA`/`sourceB`) are free-form caller strings, while a staged + action's target is always a canonicalized relPath — raw string equality + between them silently understates T on aliased spellings. Endpoints are + canonicalized via the same link-resolution the codebase already uses for + aliased vault links before the comparison runs, with raw-string fallback + for an endpoint that resolves to nothing (it cannot match a live target + anyway). - **C — conflict / retry markers.** 1 when the proposal carries a non-empty `conflicts_with` (the #235 inter-proposal check in `stageActionWithConflictCheck`), or when the log holds a prior *rejected* - or *expired* proposal of the same kind against the same target — the - deterministic form of "the agent is retrying something a human already - declined." Else 0. + proposal of the same kind against the same target **with no later + *ratified* proposal for that same pair**. Else 0. **Amendment (2026-07-27 + final plan, C1):** `expired` was removed from this clause. Decision 3's own + W-term rationale (below) already states "an expiry is reviewer capacity, + not a human declining" — counting it here too both contradicted that + rationale and double-counted the same record (C's flat +0.10 on top of W's + `expired/2`), and after one TTL-cycle backlog flush, `expired` would have + become a near-constant hit on exactly the vaults that most need triage. A + later ratified proposal on the same `(actionType, targetPath)` pair clears + the mark — the human approving a subsequent proposal on that target is not + the "retrying something already declined" signal this term exists to flag. - **W — proposer track record.** The witness's per-principal proposal tallies (src/witness/track-record.ts), Laplace-smoothed so a new principal defaults to the middle instead of the extremes: `(rejected + edited + expired/2 + 1) / (ratified + rejected + edited + - expired/2 + 2)`. Expiries count half — an expiry is reviewer capacity, not - proposer fault, but a principal whose proposals *always* expire is flooding - the queue and should pay some triage cost. `edited` is new; Decision 3 - creates it. + expired/2 + 2)`, where `ratified` here means *plain* approvals + (`ratified − edited`, so an edit-then-approve is not double-counted). + Expiries count half — an expiry is reviewer capacity, not proposer fault, + but a principal whose proposals *always* expire is flooding the queue and + should pay some triage cost. `edited` is new; Decision 3 creates it. + **Amendment (2026-07-27 final plan, C4):** tallies key on the + **authenticated `staged_by_principal`** (the caller's `access.user` at + stage time) when a proposal record has one, falling back to the + unauthenticated, caller-claimed `proposed_by` string only for legacy + records or proposals staged without an access context. Keying on + `proposed_by` alone let a fresh claimed-agent string reset a principal's + history to the Laplace midpoint (laundering) and let junk staged under a + rival's claimed name count against the rival instead of the actual stager + (poisoning) — the same authenticated-identity fix already applied to the + decision side (`decided_by_principal`) but never to the proposal side + until this spec. The weights are exported constants, provisional, and to be calibrated against the outcome data Decision 3 starts collecting — the same "the @@ -240,6 +297,10 @@ Three, one per load-bearing bet, checkable in the system's own numbers: indistinguishable from the bottom, the score is decorative — reviewers' judgment and the arithmetic disagree, and the arithmetic loses. Kill the score, keep the outcome logging (Decision 3 stands on its own). + **Amendment (Mihir, 2026-07-27):** "risk quartile at decision time" is read + from each decision record's `risk_at_decision` snapshot (the Decision 1 + carve-out above), not approximated from present-state B/T — this condition + is now fully evaluable, not merely approximately so. - **Batch must not become the rubber stamp with better lighting.** If batch approvals come to dominate decisions with a near-zero in-batch reject rate while the arrival rate keeps climbing, enumerated-list batching diff --git a/integrations/recall-bench/embedrefresh-runner.mjs b/integrations/recall-bench/embedrefresh-runner.mjs new file mode 100644 index 00000000..7514a2f5 --- /dev/null +++ b/integrations/recall-bench/embedrefresh-runner.mjs @@ -0,0 +1,411 @@ +// Embedding-refresh A/B/C/D bench (spec 2026-07-26-embedding-refresh- +// quantization-design.md, Phase 5, as revised by the resolved final plan at +// .jugalbandi/embedding-refresh-quantization/final-plan.md). Cloned from +// chunkbm25-runner.mjs / fusion-runner.mjs's pattern: top-of-file constants, +// dist/ dynamic imports, --smoke, JSON outputs. +// +// Arms: +// A: local-minilm, 384d, float32 — today's shipped default +// B: local-embeddinggemma@512, float32, quantize: none — model gain at comparable scale +// C: local-embeddinggemma@512, int8 + rescore — what would actually ship +// D: local-qwen3-0.6b@512, vector-only metrics, ALWAYS run — Qwen3 user-selectability gate +// +// Gates, in order of severity (final plan Phase 5): +// - C >= A on recall@10 — ship gate (kill condition 2 fires here) +// - |C - B| <= 1pp at every K — quantize/rescore bug detector +// - B vs A reported ungated — the headline model-generation number +// - D not pathologically worse than A — gates Qwen3's user-selectability +// +// Doc-doc smoke (disposition C6): for a fixed ~50-doc sample, mean-embedding +// nearest neighbors under arms A and B; top-10 neighbor overlap plus this +// script's own overlap number stands in for the "brief qualitative spot- +// check" a human reviews in the results doc — this script cannot write +// prose commentary, only the numbers a human reads before writing it. +// +// One vault, provider-switched per arm (final plan Phase 4's own migration +// story: config change + reindexVault). The durable `embeddings` cache is +// keyed by (content_hash, model), so each arm's reindex populates its own +// row set without disturbing the others — a re-run after the first is all +// cache hits for every arm except a genuinely new model id. +// +// NOT RUN as part of implementing this scaffolding: arms B/C/D require +// downloading real ONNX weights (EmbeddingGemma ~600MB q8, Qwen3 ~1.5GB) and +// a machine-local QFILE fixture this repo does not commit (same convention +// as chunkbm25-runner.mjs / fusion-runner.mjs) — neither is available in the +// environment that wrote this file. Do not treat the presence of this +// script as evidence the spec's Phase 0 spike or Phase 5 measurement have +// happened; they have not. See the governing spec's kill conditions before +// running this for real, and DO NOT flip any default off its output without +// a human reviewing the results doc this script's output feeds. + +import { createHash } from "node:crypto"; +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; + +const ROOT = fileURLToPath(new URL("../..", import.meta.url)); +const QFILE = `${ROOT}/integrations/recall-bench/results/ea-180d-partial-2026-06-21/questions.jsonl`; +const VAULT = "/tmp/embedrefresh-recall/vault"; +const OUT = "/tmp/embedrefresh-recall"; +const SMOKE = process.argv.includes("--smoke"); +const SMOKE_CAP = 25; +const KS = [10, 20, 50]; +const LIMIT = 50; +const DOC_DOC_SAMPLE_SIZE = 50; +const DOC_DOC_TOP_K = 10; +const DOC_DOC_SEED = 20260726; + +// Ship / bug-detector / non-pathological gate thresholds. Recorded here so +// a human reviewing a real run's output sees exactly what was checked; the +// spec explicitly permits the flip review to revise these in writing with +// measurements in hand (final plan, C7 disposition, same posture applied +// here to the recall gates). +const GATE_QUANTIZE_BUG_PP = 0.01; // |C - B| <= 1pp at every K +const GATE_D_PATHOLOGICAL_DROP_PP = 0.1; // D vs A drop > 10pp at K=10 is "pathological" + +const { hybridSearch } = await import(`${ROOT}/dist/search/hybrid.js`); +const { cosineSimilarity, meanEmbedding, setProvider, getProvider, toIndexDim } = await import( + `${ROOT}/dist/search/vector.js` +); +const { openIndexForActiveProvider } = await import(`${ROOT}/dist/tools/search.js`); +const { getAllDocuments, getChunksForPath } = await import(`${ROOT}/dist/storage/index-db.js`); +const { reindexVault } = await import(`${ROOT}/dist/search/reindex.js`); + +function openVault(path) { + const r = openIndexForActiveProvider(path); + if (!r.ok) { + console.error(`open ${path} failed:`, r.error.message); + process.exit(1); + } + return r.value; +} + +function mulberry32(seed) { + let s = seed | 0; + return function rng() { + s = (s + 0x6d2b79f5) | 0; + let t = Math.imul(s ^ (s >>> 15), 1 | s); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +function shuffledIndices(n, rng) { + const idx = Array.from({ length: n }, (_, i) => i); + for (let i = idx.length - 1; i > 0; i--) { + const j = Math.floor(rng() * (i + 1)); + [idx[i], idx[j]] = [idx[j], idx[i]]; + } + return idx; +} + +const dayOf = (p) => { + const m = /day-(\d+)/.exec(p || ""); + return m ? Number(m[1]) : null; +}; +const recall = (got, rel) => + rel.length ? rel.filter((d) => got.includes(d)).length / rel.length : null; +const daysAtK = (hits, K) => [ + ...new Set( + hits + .slice(0, K) + .map((h) => dayOf(h.path)) + .filter((d) => d !== null), + ), +]; + +function sha256Hex(text) { + return createHash("sha256").update(text).digest("hex"); +} + +async function retrieve(db, q, opts) { + const res = await hybridSearch(db, q, opts); + if (!res.ok) throw new Error(`hybridSearch failed for "${q}": ${res.error.message}`); + return res.value; +} + +function loadQuestions() { + if (!existsSync(QFILE)) { + console.error( + `QFILE missing: ${QFILE}\n` + + "(machine-local fixture, same convention as chunkbm25-runner.mjs / " + + "fusion-runner.mjs — not committed; third-party corpus.)", + ); + process.exit(1); + } + const recs = readFileSync(QFILE, "utf8").split("\n").filter(Boolean).map(JSON.parse); + return recs.map((r) => ({ question: r.qa.question, relevantDays: r.qa.relevantDays || [] })); +} + +// --------------------------------------------------------------------------- +// Per-arm provider configuration. `setup` mutates the active provider (and +// triggers a reindex — the actual migration path Phase 3d makes real) so +// the SAME vault directory carries every arm's row set, keyed by model id. +// --------------------------------------------------------------------------- +const ARMS = { + A: { + label: "local-minilm 384d float32 (today's default)", + async setup() { + setProvider("local-minilm"); + }, + }, + B: { + label: "local-embeddinggemma@512 float32 (quantize: none)", + async setup() { + setProvider("local-embeddinggemma", { dim: 512, quantize: "none" }); + }, + }, + C: { + label: "local-embeddinggemma@512 int8 + rescore (proposed default)", + async setup() { + setProvider("local-embeddinggemma", { dim: 512, quantize: "int8" }); + }, + }, + D: { + label: "local-qwen3-0.6b@512, vector-only metrics, always run", + async setup() { + setProvider("local-qwen3-0.6b", { dim: 512, quantize: "int8" }); + }, + }, +}; + +async function reindexForArm(armId) { + await ARMS[armId].setup(); + const result = await reindexVault(VAULT); + if (!result.ok) throw new Error(`[arm ${armId}] reindex failed: ${result.error.message}`); + console.log( + `[arm ${armId}] ${ARMS[armId].label}: ${result.value.documentCount} docs, ` + + `${result.value.embeddedCount} embedded, ${result.value.cacheHits} cache hits`, + ); + return result.value; +} + +// --------------------------------------------------------------------------- +// Aggregation +// --------------------------------------------------------------------------- +function meanAt(rows, arm, K) { + const v = rows.map((r) => r[arm]?.[K]).filter((x) => x != null); + return v.length ? +(v.reduce((a, b) => a + b, 0) / v.length).toFixed(4) : null; +} +function curve(rows, arm) { + return Object.fromEntries(KS.map((K) => [K, meanAt(rows, arm, K)])); +} + +// --------------------------------------------------------------------------- +// Doc-doc smoke (disposition C6): relatedSearch-style mean-embedding nearest +// neighbors, arms A vs B. Uses meanEmbedding + cosineSimilarity directly +// (not relatedSearch itself) so the comparison is symmetric across the two +// otherwise-fusion-free vector-only neighbor sets. +// --------------------------------------------------------------------------- +async function docDocNeighbors(db, provider, sampleDocs) { + const vectorsByPath = new Map(); + for (const doc of sampleDocs) { + const chunks = getChunksForPath(db, doc.path, provider.id, provider.nativeDim ?? provider.dim) + .map((c) => c.embedding) + .filter((e) => e !== null) + .map((e) => toIndexDim(e, provider.dim)); + const mean = meanEmbedding(chunks); + if (mean) vectorsByPath.set(doc.path, mean); + } + const paths = [...vectorsByPath.keys()]; + const neighbors = new Map(); + for (const p of paths) { + const v = vectorsByPath.get(p); + const scored = paths + .filter((q) => q !== p) + .map((q) => ({ path: q, sim: cosineSimilarity(v, vectorsByPath.get(q)) })) + .sort((a, b) => b.sim - a.sim) + .slice(0, DOC_DOC_TOP_K) + .map((r) => r.path); + neighbors.set(p, scored); + } + return neighbors; +} + +function neighborOverlap(a, b) { + const overlaps = []; + for (const [path, neighborsA] of a) { + const neighborsB = b.get(path); + if (!neighborsB) continue; + const setB = new Set(neighborsB); + const shared = neighborsA.filter((n) => setB.has(n)).length; + overlaps.push(shared / DOC_DOC_TOP_K); + } + return overlaps.length + ? +(overlaps.reduce((s, x) => s + x, 0) / overlaps.length).toFixed(4) + : null; +} + +// --------------------------------------------------------------------------- +// Main +// --------------------------------------------------------------------------- +async function main() { + mkdirSync(OUT, { recursive: true }); + const questions = loadQuestions(); + const cases = SMOKE ? questions.slice(0, SMOKE_CAP) : questions; + console.log(`questions: ${cases.length}${SMOKE ? " (smoke)" : ""}`); + + // ---- Arms A / B / C: hybrid + vector-only recall ---- + const perQ = []; + for (const armId of ["A", "B", "C"]) { + await reindexForArm(armId); + const db = openVault(VAULT); + for (const q of cases) { + let row = perQ.find((r) => r.question === q.question); + if (!row) { + row = { question: q.question, rel: q.relevantDays, hybrid: {}, vectorOnly: {} }; + perQ.push(row); + } + const hybrid = await retrieve(db, q.question, { + limit: LIMIT, + weights: { bm25: 0.5, vector: 0.5 }, + }); + const vectorOnly = await retrieve(db, q.question, { + limit: LIMIT, + weights: { bm25: 0, vector: 1 }, + }); + row.hybrid[armId] = {}; + row.vectorOnly[armId] = {}; + for (const K of KS) { + row.hybrid[armId][K] = recall(daysAtK(hybrid.hits, K), q.relevantDays); + row.vectorOnly[armId][K] = recall(daysAtK(vectorOnly.hits, K), q.relevantDays); + } + } + db.close(); + } + + const hybridCurves = Object.fromEntries( + ["A", "B", "C"].map((arm) => [ + arm, + curve( + perQ.map((r) => ({ [arm]: r.hybrid[arm] })), + arm, + ), + ]), + ); + const vectorOnlyCurves = Object.fromEntries( + ["A", "B", "C"].map((arm) => [ + arm, + curve( + perQ.map((r) => ({ [arm]: r.vectorOnly[arm] })), + arm, + ), + ]), + ); + + // ---- Arm D: vector-only metrics only, always run, never gates the flip ---- + await reindexForArm("D"); + const dbD = openVault(VAULT); + const dPerQ = []; + for (const q of cases) { + const vectorOnly = await retrieve(dbD, q.question, { + limit: LIMIT, + weights: { bm25: 0, vector: 1 }, + }); + const row = { question: q.question, D: {} }; + for (const K of KS) row.D[K] = recall(daysAtK(vectorOnly.hits, K), q.relevantDays); + dPerQ.push(row); + } + const dCurve = curve(dPerQ, "D"); + dbD.close(); + + // ---- Gates ---- + const gates = {}; + gates.shipRecall10 = { + description: "C >= A on recall@10 (kill condition 2 — default flip)", + pass: (hybridCurves.C[10] ?? -1) >= (hybridCurves.A[10] ?? -1), + a: hybridCurves.A[10], + c: hybridCurves.C[10], + }; + gates.quantizeBugDetector = { + description: `|C - B| <= ${GATE_QUANTIZE_BUG_PP} at every K`, + pass: KS.every((K) => { + const b = hybridCurves.B[K]; + const c = hybridCurves.C[K]; + if (b == null || c == null) return true; + return Math.abs(c - b) <= GATE_QUANTIZE_BUG_PP; + }), + deltas: Object.fromEntries( + KS.map((K) => [ + K, + hybridCurves.B[K] != null && hybridCurves.C[K] != null + ? +(hybridCurves.C[K] - hybridCurves.B[K]).toFixed(4) + : null, + ]), + ), + }; + gates.headlineBvsA = { + description: "B vs A — ungated, the measured size of the model-generation claim", + delta10: + hybridCurves.B[10] != null && hybridCurves.A[10] != null + ? +(hybridCurves.B[10] - hybridCurves.A[10]).toFixed(4) + : null, + }; + gates.qwen3Selectable = { + description: `D not pathologically worse than A (drop > ${GATE_D_PATHOLOGICAL_DROP_PP} at K=10 is pathological) — gates Qwen3's user-selectability, never the flip`, + pass: (dCurve[10] ?? 0) >= (vectorOnlyCurves.A[10] ?? 0) - GATE_D_PATHOLOGICAL_DROP_PP, + a: vectorOnlyCurves.A[10], + d: dCurve[10], + }; + + // ---- Doc-doc smoke (C6) ---- + const dbSmoke = openVault(VAULT); + const allDocs = getAllDocuments(dbSmoke); + const rng = mulberry32(DOC_DOC_SEED); + const order = shuffledIndices(allDocs.length, rng); + const sample = order + .slice(0, Math.min(DOC_DOC_SAMPLE_SIZE, allDocs.length)) + .map((i) => allDocs[i]); + + await reindexForArm("A"); + const neighborsA = await docDocNeighbors(dbSmoke, getProvider(), sample); + await reindexForArm("B"); + const neighborsB = await docDocNeighbors(dbSmoke, getProvider(), sample); + const docDocOverlap = neighborOverlap(neighborsA, neighborsB); + dbSmoke.close(); + + const docDoc = { + sampleSize: sample.length, + topK: DOC_DOC_TOP_K, + meanTop10Overlap: docDocOverlap, + note: + "This is the tripwire for relatedSearch/vault_themes/edges — all consume the same " + + "doc-embedded vectors. A low overlap means the model-generation jump changes the " + + "document-similarity REGIME, not just query recall; the results doc must state this " + + "residual risk explicitly (final plan, disposition C6) and PR-5's flip review must " + + "cite it. This script does not judge whether the number is acceptable — a human does.", + }; + + const provenance = { + questionsFileSha256: existsSync(QFILE) ? sha256Hex(readFileSync(QFILE)) : null, + vaultListingSha256: sha256Hex( + getAllDocuments(openVault(VAULT)) + .map((d) => d.path) + .sort() + .join("\n"), + ), + ran: { + phase0Spike: false, + realModelDownloads: false, + }, + }; + + const summary = { + smoke: SMOKE, + counts: { total: perQ.length }, + hybridRecall: hybridCurves, + vectorOnlyRecall: vectorOnlyCurves, + armD: { vectorOnlyRecall: dCurve }, + gates, + docDoc, + provenance, + }; + + writeFileSync( + `${OUT}/embedrefresh-perq.json`, + JSON.stringify({ ks: KS, smoke: SMOKE, perQ, dPerQ }, null, 2), + ); + writeFileSync(`${OUT}/embedrefresh-summary.json`, JSON.stringify(summary, null, 2)); + console.log(JSON.stringify(summary, null, 2)); +} + +await main(); diff --git a/integrations/recall-bench/fusion-runner.mjs b/integrations/recall-bench/fusion-runner.mjs new file mode 100644 index 00000000..1aabc6b8 --- /dev/null +++ b/integrations/recall-bench/fusion-runner.mjs @@ -0,0 +1,602 @@ +// Fusion overhaul A/B bench (spec 2026-07-26 fusion overhaul, Decisions 1/2/4 +// as revised by the resolved final plan). Cloned from chunkbm25-runner.mjs's +// pattern: top-of-file constants, dist/ dynamic imports, --smoke, JSON +// outputs. Four arms: +// +// A: hybridSearch(db, q, { limit: 50 }) — weighted fusion, default weights +// B: A + fusion: "rrf" — RRF fusion, same weights +// C: B + weights: routeWeights(classifyQuery(q).class) — RRF + router +// D: C's config on a restricted (RBAC) split vault, post- vs pushed-down +// collection filtering — measures the ACL-pushdown fix (a2ec361) under +// the new fusion. +// +// Question sets, three categories: +// - paraphrase: the machine-local ea-180d-partial-2026-06-21 fixture +// (same convention as chunkbm25-runner.mjs — third-party corpus, +// provenance hash-recorded rather than committed). +// - phrase: synthetic, seeded PRNG, from doc bodies — a corpus-unique +// contiguous 2-3 token run, quoted. Bench mass for the extreme-lexical +// route. +// - identifier: synthetic, seeded PRNG, from doc bodies — a token whose +// stem-aware df === 1. Bench mass for the rare-term signal. +// +// Do NOT run the full (non-smoke) bench without the machine-local QFILE and +// a prepped /tmp/fusion-recall/vault (via `prep-vault.mjs --out +// /tmp/fusion-recall/vault`) — see docs/superpowers/specs/2026-07-26- +// retrieval-fusion-overhaul-design.md and the resolved final plan. + +import { createHash } from "node:crypto"; +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; + +const ROOT = fileURLToPath(new URL("../..", import.meta.url)); +const QFILE = `${ROOT}/integrations/recall-bench/results/ea-180d-partial-2026-06-21/questions.jsonl`; +const DAY_VAULT = "/tmp/fusion-recall/vault"; +const SPLIT_VAULT = "/tmp/fusion-recall/split-vault"; +const OUT = "/tmp/fusion-recall"; +const SMOKE = process.argv.includes("--smoke"); +const SMOKE_CAP = 25; +const KS = [10, 20, 50]; +const LIMIT = 50; +const CATEGORIES = ["paraphrase", "phrase", "identifier"]; + +// Seeded PRNG (committed, deterministic) for the synthetic phrase/identifier +// question sets — same seed always produces the same question set from the +// same day vault. +const PHRASE_SEED = 20260726; +const IDENTIFIER_SEED = 20260727; +const SAMPLE_SIZE = 100; + +// Arm D's restricted-role fixture (spec 2026-07-26 fusion, Decision 3 — +// the ACL-pushdown starvation-bug fix this arm measures under the new +// fusion). 8 collections, ~22-23 docs each; READABLE is the minority-read +// configuration where the starvation bug lived. +const NUM_COLLECTIONS = 8; +const READABLE = ["col-0", "col-1"]; +const ALL_COLLECTIONS = Array.from({ length: NUM_COLLECTIONS }, (_, i) => `col-${i}`); + +// Same K the vector KNN asks sqlite-vec for (src/search/hybrid.ts +// VEC_KNN_K) — used only to LABEL the no-leak comparison's boundary-tie +// heuristic in output; not read from source, so keep in sync by hand. +const VEC_KNN_K = 64; + +const { hybridSearch } = await import(`${ROOT}/dist/search/hybrid.js`); +const { openIndexForActiveProvider } = await import(`${ROOT}/dist/tools/search.js`); +const { getAllDocuments, documentCount } = await import(`${ROOT}/dist/storage/index-db.js`); +const { tokenize } = await import(`${ROOT}/dist/search/bm25.js`); +const { classifyQuery, routeWeights, makeDfLookup } = await import(`${ROOT}/dist/search/router.js`); +const { reindexVault } = await import(`${ROOT}/dist/search/reindex.js`); + +function openVault(path) { + const r = openIndexForActiveProvider(path); + if (!r.ok) { + console.error(`open ${path} failed:`, r.error.message); + process.exit(1); + } + return r.value; +} + +// --------------------------------------------------------------------------- +// Seeded PRNG (mulberry32) + deterministic shuffle-sample +// --------------------------------------------------------------------------- + +function mulberry32(seed) { + let s = seed | 0; + return function rng() { + s = (s + 0x6d2b79f5) | 0; + let t = Math.imul(s ^ (s >>> 15), 1 | s); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +function shuffledIndices(n, rng) { + const idx = Array.from({ length: n }, (_, i) => i); + for (let i = idx.length - 1; i > 0; i--) { + const j = Math.floor(rng() * (i + 1)); + [idx[i], idx[j]] = [idx[j], idx[i]]; + } + return idx; +} + +// --------------------------------------------------------------------------- +// Shared helpers (chunkbm25-runner.mjs convention) +// --------------------------------------------------------------------------- + +const dayOf = (p) => { + const m = /day-(\d+)/.exec(p || ""); + return m ? Number(m[1]) : null; +}; +const recall = (got, rel) => + rel.length ? rel.filter((d) => got.includes(d)).length / rel.length : null; +const daysAtK = (hits, K) => [ + ...new Set( + hits + .slice(0, K) + .map((h) => dayOf(h.path)) + .filter((d) => d !== null), + ), +]; + +// Split-vault collection assignment: day N -> col-{(N-1) % NUM_COLLECTIONS}. +// Shared by the split-vault builder AND the D-arm's readable-day filter so +// the two never disagree about which collection a day landed in. +const collectionOf = (day) => `col-${(day - 1) % NUM_COLLECTIONS}`; + +function sha256Hex(text) { + return createHash("sha256").update(text).digest("hex"); +} + +async function retrieve(db, q, opts) { + const res = await hybridSearch(db, q, opts); + if (!res.ok) throw new Error(`hybridSearch failed for "${q}": ${res.error.message}`); + return res.value; +} + +// Per-arm, expectation-shaped vectorUsed assertion (final plan, C1 revision): +// a routed extreme-lexical query legitimately reports vectorUsed: false +// (weights.vector === 0 skips embedding entirely) — that must not abort the +// run. Any OTHER mismatch between the weights actually passed and what came +// back is a real embedding-path failure and aborts immediately. +function assertVectorUsedExpectation(arm, result, effectiveWeights, question) { + const expected = effectiveWeights.vector > 0; + if (result.vectorUsed !== expected) { + throw new Error( + `[arm ${arm}] vectorUsed=${result.vectorUsed} but weights=${JSON.stringify(effectiveWeights)} ` + + `implies ${expected} for question: ${JSON.stringify(question)}`, + ); + } +} + +// --------------------------------------------------------------------------- +// Question sets +// --------------------------------------------------------------------------- + +function loadParaphraseQuestions() { + if (!existsSync(QFILE)) { + console.error( + `QFILE missing: ${QFILE}\n` + + "(machine-local fixture, same convention as chunkbm25-runner.mjs — " + + "not committed; third-party corpus.)", + ); + process.exit(1); + } + const recs = readFileSync(QFILE, "utf8").split("\n").filter(Boolean).map(JSON.parse); + return recs.map((r) => ({ + category: "paraphrase", + question: r.qa.question, + relevantDays: r.qa.relevantDays || [], + })); +} + +// Synthetic "phrase" category (final plan, disposition C3): from each +// sampled doc's body, a contiguous run of 2-3 tokenize()-valid tokens, +// verified unique in the corpus (FTS5 phrase MATCH count === 1); the query +// is that run in double quotes. Classifies extreme-lexical by construction +// (quoted-phrase signal) — this is the extreme-lexical route's bench mass. +function buildPhraseQuestions(db, docs) { + const rng = mulberry32(PHRASE_SEED); + const order = shuffledIndices(docs.length, rng); + const sample = order.slice(0, Math.min(SAMPLE_SIZE, docs.length)); + const questions = []; + let skipped = 0; + for (const i of sample) { + const doc = docs[i]; + const day = dayOf(doc.path); + const tokens = tokenize(doc.content); + if (day === null || tokens.length < 2) { + skipped++; + continue; + } + let found = null; + for (let attempt = 0; attempt < 20 && !found; attempt++) { + const len = tokens.length >= 3 && rng() < 0.5 ? 3 : 2; + if (tokens.length < len) continue; + const start = Math.floor(rng() * (tokens.length - len + 1)); + const run = tokens.slice(start, start + len).join(" "); + const n = db + .prepare("SELECT count(*) AS n FROM documents_fts WHERE documents_fts MATCH ?") + .get(`"${run}"`).n; + if (n === 1) found = run; + } + if (found) { + questions.push({ category: "phrase", question: `"${found}"`, relevantDays: [day] }); + } else { + skipped++; + } + } + return { questions, skipped }; +} + +// Synthetic "identifier" category (final plan, disposition C3/C8): from each +// sampled doc's body, a token whose stem-aware df === 1 (via the same +// MATCH-count lookup the router uses); the query is that token. Classifies +// lexical (rare-term signal) — relevant = own day is exact because df === 1. +function buildIdentifierQuestions(db, docs) { + const rng = mulberry32(IDENTIFIER_SEED); + const df = makeDfLookup(db); + const order = shuffledIndices(docs.length, rng); + const sample = order.slice(0, Math.min(SAMPLE_SIZE, docs.length)); + const questions = []; + let skipped = 0; + for (const i of sample) { + const doc = docs[i]; + const day = dayOf(doc.path); + if (day === null) { + skipped++; + continue; + } + const unique = [...new Set(tokenize(doc.content))]; + const tokenOrder = shuffledIndices(unique.length, rng).map((k) => unique[k]); + const term = tokenOrder.find((t) => df(t) === 1); + if (term) { + questions.push({ category: "identifier", question: term, relevantDays: [day] }); + } else { + skipped++; + } + } + return { questions, skipped }; +} + +// --------------------------------------------------------------------------- +// Split-vault builder for arm D (final plan, C2 revision) +// --------------------------------------------------------------------------- +// Copies the day vault's SOURCE FILES (not the index — IndexedDocument.content +// is body-only, frontmatter already stripped) to SPLIT_VAULT, rewriting BOTH +// the path (notes/day-NNNN.md -> col-{N mod 8}/day-NNNN.md) AND the +// frontmatter `collection:` line (notes -> col-{N mod 8}) — frontmatter wins +// collection derivation (reindex.ts:311: +// `fm.collection || relPath.split("/")[0]`), so a path-only rewrite would +// leave every doc in "notes" and D would read nothing. +async function buildSplitVault(dayDocPaths) { + const { mkdirSync: mk, writeFileSync: wf, readFileSync: rf, rmSync } = await import("node:fs"); + const { join } = await import("node:path"); + rmSync(SPLIT_VAULT, { recursive: true, force: true }); + for (const path of dayDocPaths) { + const day = dayOf(path); + if (day === null) continue; + const col = collectionOf(day); + const dir = join(SPLIT_VAULT, col); + mk(dir, { recursive: true }); + const raw = rf(join(DAY_VAULT, path), "utf8"); + const rewritten = raw.replace(/^collection: notes$/m, `collection: ${col}`); + wf(join(dir, `day-${String(day).padStart(4, "0")}.md`), rewritten); + } + const reindexed = await reindexVault(SPLIT_VAULT); + if (!reindexed.ok) throw new Error(`split-vault reindex failed: ${reindexed.error.message}`); + return reindexed.value; +} + +// --------------------------------------------------------------------------- +// Aggregation +// --------------------------------------------------------------------------- + +function meanAt(rows, arm, K) { + const v = rows.map((r) => r[arm]?.[K]).filter((x) => x != null); + return v.length ? +(v.reduce((a, b) => a + b, 0) / v.length).toFixed(4) : null; +} + +function curve(rows, arm) { + return Object.fromEntries(KS.map((K) => [K, meanAt(rows, arm, K)])); +} + +function curvesByCategory(rows, arms) { + const out = {}; + for (const cat of CATEGORIES) { + const catRows = rows.filter((r) => r.category === cat); + out[cat] = Object.fromEntries(arms.map((arm) => [arm, curve(catRows, arm)])); + } + return out; +} + +// --------------------------------------------------------------------------- +// Main +// --------------------------------------------------------------------------- + +async function main() { + mkdirSync(OUT, { recursive: true }); + + const DAY = openVault(DAY_VAULT); + const dayDocs = getAllDocuments(DAY); + const dayDf = makeDfLookup(DAY); + const dayDocCount = documentCount(DAY); + console.log(`day vault: ${dayDocs.length} docs`); + + const paraphrase = loadParaphraseQuestions(); + const phrase = buildPhraseQuestions(DAY, dayDocs); + const identifier = buildIdentifierQuestions(DAY, dayDocs); + console.log( + `questions: paraphrase=${paraphrase.length} ` + + `phrase=${phrase.questions.length} (skipped ${phrase.skipped}) ` + + `identifier=${identifier.questions.length} (skipped ${identifier.skipped})`, + ); + + const cap = (arr) => (SMOKE ? arr.slice(0, SMOKE_CAP) : arr); + const allQuestions = [...cap(paraphrase), ...cap(phrase.questions), ...cap(identifier.questions)]; + + // ---- Arms A / B / C over the day vault ---- + const perQ = []; + for (const q of allQuestions) { + const rel = q.relevantDays; + const row = { + id: q.question, + category: q.category, + relLen: rel.length, + rel, + A: {}, + B: {}, + C: {}, + }; + + const a = await retrieve(DAY, q.question, { limit: LIMIT, fusion: "weighted" }); + assertVectorUsedExpectation("A", a, { bm25: 0.5, vector: 0.5 }, q.question); + + const b = await retrieve(DAY, q.question, { limit: LIMIT, fusion: "rrf" }); + assertVectorUsedExpectation("B", b, { bm25: 0.5, vector: 0.5 }, q.question); + + const classified = classifyQuery(q.question, { df: dayDf, docCount: dayDocCount }); + const cWeights = routeWeights(classified.class); + const c = await retrieve(DAY, q.question, { limit: LIMIT, fusion: "rrf", weights: cWeights }); + assertVectorUsedExpectation("C", c, cWeights, q.question); + row.routedClass = classified.class; + row.routedSignals = classified.signals; + + for (const K of KS) { + row.A[K] = recall(daysAtK(a.hits, K), rel); + row.B[K] = recall(daysAtK(b.hits, K), rel); + row.C[K] = recall(daysAtK(c.hits, K), rel); + } + perQ.push(row); + } + + const overall = { A: curve(perQ, "A"), B: curve(perQ, "B"), C: curve(perQ, "C") }; + const byCat = curvesByCategory(perQ, ["A", "B", "C"]); + const byRouteClass = {}; + for (const cls of ["extreme-lexical", "lexical", "balanced"]) { + const rows = perQ.filter((r) => r.routedClass === cls); + byRouteClass[cls] = { + count: rows.length, + A: curve(rows, "A"), + B: curve(rows, "B"), + C: curve(rows, "C"), + }; + } + + function categoryNoRegression(armFrom, armTo, threshold) { + for (const cat of CATEGORIES) { + for (const K of KS) { + const from = byCat[cat][armFrom][K]; + const to = byCat[cat][armTo][K]; + if (from == null || to == null) continue; + if (to - from < threshold) return false; + } + } + return true; + } + + const gates = {}; + gates.rrfFlip = + (overall.B[10] ?? -1) > (overall.A[10] ?? -1) && categoryNoRegression("A", "B", -0.01); + + const noCategoryDrop = (() => { + for (const cat of CATEGORIES) { + for (const K of KS) { + const b = byCat[cat].B[K]; + const c = byCat[cat].C[K]; + if (b == null || c == null) continue; + if (b - c > 0.01) return false; // > 1pp absolute drop + } + } + return true; + })(); + const paraphraseLoss = Math.max(0, (byCat.paraphrase.B[10] ?? 0) - (byCat.paraphrase.C[10] ?? 0)); + const idPhraseGain = + Math.max(0, (byCat.identifier.C[10] ?? 0) - (byCat.identifier.B[10] ?? 0)) + + Math.max(0, (byCat.phrase.C[10] ?? 0) - (byCat.phrase.B[10] ?? 0)); + gates.routingFlip = + (overall.C[10] ?? -1) > (overall.B[10] ?? -1) && + noCategoryDrop && + paraphraseLoss <= idPhraseGain; + + // ---- Arm D: restricted-role split vault ---- + const splitInfo = await buildSplitVault(dayDocs.map((d) => d.path)); + const SPLIT = openVault(SPLIT_VAULT); + const splitCollections = new Map(); + for (const doc of getAllDocuments(SPLIT)) { + splitCollections.set(doc.collection, (splitCollections.get(doc.collection) ?? 0) + 1); + } + const distinctCollections = [...splitCollections.keys()]; + if (distinctCollections.length !== NUM_COLLECTIONS) { + throw new Error( + `split vault: expected ${NUM_COLLECTIONS} collections, got ${distinctCollections.length}`, + ); + } + for (const [col, count] of splitCollections) { + if (count < 22 || count > 23) { + throw new Error(`split vault: collection ${col} has ${count} docs, expected 22-23`); + } + } + console.log( + `split vault: ${splitInfo.documentCount} docs across ${distinctCollections.length} collections`, + ); + + const splitDf = makeDfLookup(SPLIT); + const splitDocCount = documentCount(SPLIT); + + const dPerQ = []; + const mismatches = []; + const boundaryTies = []; + let firstDPushChecked = false; + + for (const q of allQuestions) { + const classified = classifyQuery(q.question, { df: splitDf, docCount: splitDocCount }); + const cWeights = routeWeights(classified.class); + + // D-push: pushdown filter INSIDE the KNN scan (the shipped a2ec361 fix). + const push = await retrieve(SPLIT, q.question, { + limit: LIMIT, + fusion: "rrf", + weights: cWeights, + readableCollections: READABLE, + }); + assertVectorUsedExpectation("D-push", push, cWeights, q.question); + if (!firstDPushChecked) { + if (!push.vectorUsed && cWeights.vector > 0) { + throw new Error("first D-push question expected vectorUsed: true"); + } + firstDPushChecked = true; + } + + // D-post: NO pushdown — over-fetch every ranked candidate (unrestricted + // KNN, exactly like the pre-a2ec361 handler), then post-filter to + // READABLE in the runner, THEN slice — reproducing the starvation bug. + const postRaw = await retrieve(SPLIT, q.question, { + limit: LIMIT, + overFetch: true, + fusion: "rrf", + weights: cWeights, + }); + assertVectorUsedExpectation("D-post", postRaw, cWeights, q.question); + const postHits = postRaw.hits.filter((h) => READABLE.includes(h.collection)).slice(0, LIMIT); + + // No-leak / rank-identity regression: readableCollections = ALL 8 + // collections vs readableCollections: undefined. Compared, collected, + // never aborts mid-run (final plan, C4 revision). + const withAll = await retrieve(SPLIT, q.question, { + limit: LIMIT, + fusion: "rrf", + weights: cWeights, + readableCollections: ALL_COLLECTIONS, + }); + const unfiltered = await retrieve(SPLIT, q.question, { + limit: LIMIT, + fusion: "rrf", + weights: cWeights, + }); + compareRankIdentity(withAll.hits, unfiltered.hits, q.question, mismatches, boundaryTies); + + // Restricted-arm recall excludes unreadable relevant days from the + // denominator (final plan): a question whose relevant day never landed + // in a readable collection cannot be scored under restriction at all. + const readableRel = q.relevantDays.filter((day) => READABLE.includes(collectionOf(day))); + if (readableRel.length === 0) continue; + + const row = { id: q.question, category: q.category, rel: readableRel, DPost: {}, DPush: {} }; + for (const K of KS) { + row.DPost[K] = recall(daysAtK(postHits, K), readableRel); + row.DPush[K] = recall(daysAtK(push.hits, K), readableRel); + } + dPerQ.push(row); + } + + const dOverall = { DPost: curve(dPerQ, "DPost"), DPush: curve(dPerQ, "DPush") }; + const dByCat = curvesByCategory(dPerQ, ["DPost", "DPush"]); + const dPushMinusPost = Object.fromEntries( + KS.map((K) => [K, (dOverall.DPush[K] ?? 0) - (dOverall.DPost[K] ?? 0)]), + ); + gates.noLeak = mismatches.length === 0; + + // ---- Provenance (final plan, C7 revision 2) ---- + const provenance = { + questionsFileSha256: existsSync(QFILE) ? sha256Hex(readFileSync(QFILE)) : null, + // "corpus file listing" here is the day vault's own document-path + // listing (deterministic, sorted) — the runner has no direct view of + // the external Stevenic/recall corpus prep-vault.mjs consumed, so this + // is the closest reproducible provenance signal available to it. + dayVaultListingSha256: sha256Hex( + dayDocs + .map((d) => d.path) + .sort() + .join("\n"), + ), + }; + + const summary = { + smoke: SMOKE, + counts: { + total: perQ.length, + byCategory: Object.fromEntries( + CATEGORIES.map((c) => [c, perQ.filter((r) => r.category === c).length]), + ), + }, + overall, + byCategory: byCat, + byRouteClass, + gates, + arm_d: { + splitVault: { + documentCount: splitInfo.documentCount, + collections: distinctCollections.length, + }, + overall: dOverall, + byCategory: dByCat, + pushMinusPost: dPushMinusPost, + noLeak: { + mismatchCount: mismatches.length, + boundaryTieCount: boundaryTies.length, + vecKnnK: VEC_KNN_K, + }, + }, + provenance, + }; + + writeFileSync( + `${OUT}/fusion-perq.json`, + JSON.stringify({ ks: KS, smoke: SMOKE, perQ, dPerQ }, null, 2), + ); + writeFileSync( + `${OUT}/fusion-mismatches.json`, + JSON.stringify({ mismatches, boundaryTies }, null, 2), + ); + writeFileSync(`${OUT}/fusion-summary.json`, JSON.stringify(summary, null, 2)); + console.log(JSON.stringify(summary, null, 2)); + + DAY.close(); + SPLIT.close(); +} + +// Compares two ordered hit lists path-for-path. Identical (within a 1e-9 +// score epsilon) -> no-op. Otherwise: sqlite-vec gives no ordering guarantee +// among equal-distance neighbours under the KNN `IN (…)` partition +// constraint, so a doc whose vectorScore sits at the tail (minimum) of +// EITHER list's KNN window can legitimately swap in/out between two +// collection-filter shapes even though the fusion math is identical. If +// every differing doc's vectorScore sits at that boundary minimum, this is +// recorded as a "boundary tie" for review, not a gate-failing mismatch. +function compareRankIdentity(a, b, question, mismatches, boundaryTies, epsilon = 1e-9) { + const len = Math.max(a.length, b.length); + let identical = true; + for (let i = 0; i < len; i++) { + const ha = a[i]; + const hb = b[i]; + if (!ha || !hb || ha.path !== hb.path || Math.abs(ha.score - hb.score) > epsilon) { + identical = false; + break; + } + } + if (identical) return; + + const aPaths = new Set(a.map((h) => h.path)); + const bPaths = new Set(b.map((h) => h.path)); + const onlyInA = a.filter((h) => !bPaths.has(h.path)); + const onlyInB = b.filter((h) => !aPaths.has(h.path)); + const reordered = a.filter( + (h, i) => bPaths.has(h.path) && aPaths.has(h.path) && b[i]?.path !== h.path, + ); + const differing = [...onlyInA, ...onlyInB, ...reordered]; + + const allScores = [...a, ...b].map((h) => h.vectorScore); + const boundary = allScores.length ? Math.min(...allScores) : 0; + const allAtBoundary = + differing.length > 0 && differing.every((h) => Math.abs(h.vectorScore - boundary) <= epsilon); + + const record = { + question, + withAllPaths: a.map((h) => h.path), + unfilteredPaths: b.map((h) => h.path), + }; + if (allAtBoundary) boundaryTies.push(record); + else mismatches.push(record); +} + +await main(); diff --git a/integrations/recall-bench/prep-vault.mjs b/integrations/recall-bench/prep-vault.mjs index e603acf0..4b251b71 100644 --- a/integrations/recall-bench/prep-vault.mjs +++ b/integrations/recall-bench/prep-vault.mjs @@ -1,9 +1,21 @@ -import { existsSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { join } from "node:path"; +import { fileURLToPath } from "node:url"; -const ROOT = "/Users/mihirwagle/projects/daftari"; -const CORPUS = "/tmp/recall-review/packages/recall-bench/personas/executive-assistant/memories-180d"; -const VAULT = "/tmp/cov-recall/vault"; +const ROOT = fileURLToPath(new URL("../..", import.meta.url)); +const CORPUS = + "/tmp/recall-review/packages/recall-bench/personas/executive-assistant/memories-180d"; + +// --out : destination vault directory (spec 2026-07-26 fusion overhaul, +// disposition C7 — the fusion bench preps into its own /tmp/fusion-recall/vault +// so it never clobbers the sibling chunkbm25 bench's /tmp/cov-recall/vault +// fixture; no forked prep script). Default unchanged for chunkbm25. +const outFlagIdx = process.argv.indexOf("--out"); +const VAULT = outFlagIdx !== -1 ? process.argv[outFlagIdx + 1] : "/tmp/cov-recall/vault"; +if (outFlagIdx !== -1 && !VAULT) { + console.error("--out requires a directory argument"); + process.exit(1); +} const BASE_DATE = "2026-01-01"; if (!existsSync(CORPUS)) { @@ -31,18 +43,22 @@ const nums = files.map((f) => Number(/day-(\d+)/.exec(f)[1])).sort((a, b) => a - // Invariant assertions (the date-window depends on monotonic, contiguous, one-per-day): if (files.length !== 180) throw new Error(`expected 180 day-files, got ${files.length}`); for (let i = 0; i < nums.length; i++) { - if (nums[i] !== i + 1) throw new Error(`non-contiguous day numbering at index ${i}: got ${nums[i]}`); + if (nums[i] !== i + 1) + throw new Error(`non-contiguous day numbering at index ${i}: got ${nums[i]}`); } // Spot-check ONLY the base offset (NOT per-file in-body dates — body dates are often topic prose): const day1 = readFileSync(join(CORPUS, "day-0001.md"), "utf8"); -if (!day1.includes(BASE_DATE)) console.warn(`warning: day-0001 body does not mention ${BASE_DATE}; confirm BASE_DATE`); +if (!day1.includes(BASE_DATE)) + console.warn(`warning: day-0001 body does not mention ${BASE_DATE}; confirm BASE_DATE`); rmSync(VAULT, { recursive: true, force: true }); mkdirSync(join(VAULT, "notes"), { recursive: true }); for (const n of nums) { const created = dayDate(n); - const body = stripFrontmatter(readFileSync(join(CORPUS, `day-${String(n).padStart(4, "0")}.md`), "utf8")); + const body = stripFrontmatter( + readFileSync(join(CORPUS, `day-${String(n).padStart(4, "0")}.md`), "utf8"), + ); // Inert, question-orthogonal title (NOT the first prose header — that would enter FTS and perturb ranking). const fm = `---\n` + @@ -69,4 +85,5 @@ if (!r.ok) { process.exit(1); } console.log(`prep: indexed ${r.value.documentCount} docs`); -if (r.value.documentCount !== 180) throw new Error(`indexed ${r.value.documentCount}, expected 180`); +if (r.value.documentCount !== 180) + throw new Error(`indexed ${r.value.documentCount}, expected 180`); diff --git a/package-lock.json b/package-lock.json index 9c5b6857..048de5c2 100644 --- a/package-lock.json +++ b/package-lock.json @@ -11,7 +11,8 @@ "dependencies": { "@anthropic-ai/sdk": "^0.110.0", "@huggingface/transformers": "^4.2.0", - "@modelcontextprotocol/sdk": "^1.29.0", + "@modelcontextprotocol/node": "^2.0.0", + "@modelcontextprotocol/server": "^2.0.0", "better-sqlite3": "^12.10.0", "chokidar": "^4.0.3", "glob": "^13.0.6", @@ -25,10 +26,13 @@ }, "devDependencies": { "@biomejs/biome": "^2.4.15", + "@modelcontextprotocol/client": "^2.0.0", + "@modelcontextprotocol/sdk": "^1.29.0", "@types/better-sqlite3": "^7.6.13", "@types/js-yaml": "^4.0.9", "@types/node": "^25.8.0", "@types/pg": "^8.20.0", + "ajv": "^8.20.0", "husky": "^9.1.7", "lint-staged": "^16.4.0", "pg": "^8.22.0", @@ -1236,10 +1240,63 @@ "dev": true, "license": "MIT" }, + "node_modules/@modelcontextprotocol/client": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@modelcontextprotocol/client/-/client-2.0.0.tgz", + "integrity": "sha512-8f1OghQ2rjzIOfqgUCP+8GiUWqRs89njoWLNqAe8kWmDePv3s1fZXseej+QXemssEuuOvLLmLO/kqM3IQHtISw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@modelcontextprotocol/core": "2.0.0", + "cross-spawn": "^7.0.5", + "eventsource": "^3.0.2", + "eventsource-parser": "^3.0.0", + "jose": "^6.1.3", + "pkce-challenge": "^5.0.0", + "zod": "^4.2.0" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/@modelcontextprotocol/core": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@modelcontextprotocol/core/-/core-2.0.0.tgz", + "integrity": "sha512-pJCEwGG7Lfr/+PQp9ZTwKXNeO5wzbfKL7H3MYpCorM4oFBoQrdjnBgEoqG+RjhsvS1FKrDbKux+M1HhlnGWqcA==", + "license": "MIT", + "dependencies": { + "zod": "^4.2.0" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/@modelcontextprotocol/node": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@modelcontextprotocol/node/-/node-2.0.0.tgz", + "integrity": "sha512-Y4hAC2XdGDUdDOCbLDOCA4+aL3NUldjsOWlDL/YwpAxrPhRm1xHd7lZ+mLacvZ9t3PaH28wgNoaLQGrIk1P2pg==", + "license": "MIT", + "dependencies": { + "@hono/node-server": "^1.19.9" + }, + "engines": { + "node": ">=20" + }, + "peerDependencies": { + "@modelcontextprotocol/server": "^2.0.0", + "hono": "^4.11.4" + }, + "peerDependenciesMeta": { + "hono": { + "optional": true + } + } + }, "node_modules/@modelcontextprotocol/sdk": { "version": "1.29.0", "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.29.0.tgz", "integrity": "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ==", + "dev": true, "license": "MIT", "dependencies": { "@hono/node-server": "^1.19.9", @@ -1276,6 +1333,19 @@ } } }, + "node_modules/@modelcontextprotocol/server": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@modelcontextprotocol/server/-/server-2.0.0.tgz", + "integrity": "sha512-YhHWdHfpFMQfd0prsEnxKeS3Qz3ytIGmsS0sth4KDjnacIT7hxk6hXHkJ9KysxlkvTM+WZAtQbbcUhdoP4Hvtw==", + "license": "MIT", + "dependencies": { + "@modelcontextprotocol/core": "2.0.0", + "zod": "^4.2.0" + }, + "engines": { + "node": ">=20" + } + }, "node_modules/@napi-rs/wasm-runtime": { "version": "1.1.6", "resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.6.tgz", @@ -1830,6 +1900,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/accepts/-/accepts-2.0.0.tgz", "integrity": "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==", + "dev": true, "license": "MIT", "dependencies": { "mime-types": "^3.0.0", @@ -1852,6 +1923,7 @@ "version": "8.20.0", "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz", "integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==", + "dev": true, "license": "MIT", "dependencies": { "fast-deep-equal": "^3.1.3", @@ -1868,6 +1940,7 @@ "version": "3.0.1", "resolved": "https://registry.npmjs.org/ajv-formats/-/ajv-formats-3.0.1.tgz", "integrity": "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==", + "dev": true, "license": "MIT", "dependencies": { "ajv": "^8.0.0" @@ -2006,6 +2079,7 @@ "version": "2.3.0", "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz", "integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==", + "dev": true, "license": "MIT", "dependencies": { "bytes": "^3.1.2", @@ -2030,6 +2104,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz", "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==", + "dev": true, "license": "MIT", "engines": { "node": ">=18" @@ -2086,6 +2161,7 @@ "version": "3.1.2", "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz", "integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -2095,6 +2171,7 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==", + "dev": true, "license": "MIT", "dependencies": { "es-errors": "^1.3.0", @@ -2108,6 +2185,7 @@ "version": "1.0.4", "resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz", "integrity": "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg==", + "dev": true, "license": "MIT", "dependencies": { "call-bind-apply-helpers": "^1.0.2", @@ -2205,6 +2283,7 @@ "version": "1.1.0", "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz", "integrity": "sha512-5jRCH9Z/+DRP7rkvY83B+yGIGX96OYdJmzngqnw2SBSxqCFPd0w2km3s5iawpGX8krnwSGmF0FW5Nhr0Hfai3g==", + "dev": true, "license": "MIT", "engines": { "node": ">=18" @@ -2218,6 +2297,7 @@ "version": "1.0.5", "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz", "integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -2234,6 +2314,7 @@ "version": "0.7.2", "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz", "integrity": "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -2243,6 +2324,7 @@ "version": "1.2.2", "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.2.2.tgz", "integrity": "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg==", + "dev": true, "license": "MIT", "engines": { "node": ">=6.6.0" @@ -2252,6 +2334,7 @@ "version": "2.8.6", "resolved": "https://registry.npmjs.org/cors/-/cors-2.8.6.tgz", "integrity": "sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw==", + "dev": true, "license": "MIT", "dependencies": { "object-assign": "^4", @@ -2269,6 +2352,7 @@ "version": "7.0.6", "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz", "integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==", + "dev": true, "license": "MIT", "dependencies": { "path-key": "^3.1.0", @@ -2283,6 +2367,7 @@ "version": "4.4.3", "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, "license": "MIT", "dependencies": { "ms": "^2.1.3" @@ -2358,6 +2443,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz", "integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -2382,6 +2468,7 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==", + "dev": true, "license": "MIT", "dependencies": { "call-bind-apply-helpers": "^1.0.1", @@ -2396,6 +2483,7 @@ "version": "1.1.1", "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", "integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==", + "dev": true, "license": "MIT" }, "node_modules/emoji-regex": { @@ -2409,6 +2497,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz", "integrity": "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -2465,6 +2554,7 @@ "version": "1.1.1", "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz", "integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==", + "dev": true, "license": "MIT", "dependencies": { "es-errors": "^1.3.0" @@ -2525,6 +2615,7 @@ "version": "1.0.3", "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", "integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==", + "dev": true, "license": "MIT" }, "node_modules/escape-string-regexp": { @@ -2566,6 +2657,7 @@ "version": "1.8.1", "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz", "integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -2582,6 +2674,7 @@ "version": "3.0.7", "resolved": "https://registry.npmjs.org/eventsource/-/eventsource-3.0.7.tgz", "integrity": "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA==", + "dev": true, "license": "MIT", "dependencies": { "eventsource-parser": "^3.0.1" @@ -2594,6 +2687,7 @@ "version": "3.0.8", "resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.0.8.tgz", "integrity": "sha512-70QWGkr4snxr0OXLRWsFLeRBIRPuQOvt4s8QYjmUlmlkyTZkRqS7EDVRZtzU3TiyDbXSzaOeF0XUKy8PchzukQ==", + "dev": true, "license": "MIT", "engines": { "node": ">=18.0.0" @@ -2622,6 +2716,7 @@ "version": "5.2.1", "resolved": "https://registry.npmjs.org/express/-/express-5.2.1.tgz", "integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==", + "dev": true, "license": "MIT", "dependencies": { "accepts": "^2.0.0", @@ -2665,6 +2760,7 @@ "version": "8.5.2", "resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.5.2.tgz", "integrity": "sha512-5Kb34ipNX694DH48vN9irak1Qx30nb0PLYHXfJgw4YEjiC3ZEmZJhwOp+VfiCYwFzvFTdB9QkArYS5kXa2cx2A==", + "dev": true, "license": "MIT", "dependencies": { "ip-address": "^10.2.0" @@ -2695,6 +2791,7 @@ "version": "3.1.3", "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", + "dev": true, "license": "MIT" }, "node_modules/fast-sha256": { @@ -2707,6 +2804,7 @@ "version": "3.1.4", "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.4.tgz", "integrity": "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw==", + "dev": true, "funding": [ { "type": "github", @@ -2747,6 +2845,7 @@ "version": "2.1.1", "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-2.1.1.tgz", "integrity": "sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA==", + "dev": true, "license": "MIT", "dependencies": { "debug": "^4.4.0", @@ -2774,6 +2873,7 @@ "version": "0.2.0", "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz", "integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -2783,6 +2883,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/fresh/-/fresh-2.0.0.tgz", "integrity": "sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -2813,6 +2914,7 @@ "version": "1.1.2", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==", + "dev": true, "license": "MIT", "funding": { "url": "https://github.com/sponsors/ljharb" @@ -2835,6 +2937,7 @@ "version": "1.3.0", "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz", "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==", + "dev": true, "license": "MIT", "dependencies": { "call-bind-apply-helpers": "^1.0.2", @@ -2859,6 +2962,7 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz", "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==", + "dev": true, "license": "MIT", "dependencies": { "dunder-proto": "^1.0.1", @@ -2995,6 +3099,7 @@ "version": "1.1.0", "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz", "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.4" @@ -3007,6 +3112,7 @@ "version": "2.0.3", "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz", "integrity": "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg==", + "dev": true, "license": "MIT", "dependencies": { "function-bind": "^1.1.2" @@ -3028,6 +3134,7 @@ "version": "2.0.1", "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz", "integrity": "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==", + "dev": true, "license": "MIT", "dependencies": { "depd": "~2.0.0", @@ -3064,6 +3171,7 @@ "version": "0.7.2", "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.2.tgz", "integrity": "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw==", + "dev": true, "license": "MIT", "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" @@ -3112,6 +3220,7 @@ "version": "10.2.0", "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz", "integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==", + "dev": true, "license": "MIT", "engines": { "node": ">= 12" @@ -3121,6 +3230,7 @@ "version": "1.9.1", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz", "integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.10" @@ -3155,12 +3265,14 @@ "version": "4.0.0", "resolved": "https://registry.npmjs.org/is-promise/-/is-promise-4.0.0.tgz", "integrity": "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==", + "dev": true, "license": "MIT" }, "node_modules/isexe": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz", "integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==", + "dev": true, "license": "ISC" }, "node_modules/jose": { @@ -3211,12 +3323,14 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==", + "dev": true, "license": "MIT" }, "node_modules/json-schema-typed": { "version": "8.0.2", "resolved": "https://registry.npmjs.org/json-schema-typed/-/json-schema-typed-8.0.2.tgz", "integrity": "sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA==", + "dev": true, "license": "BSD-2-Clause" }, "node_modules/json-stringify-safe": { @@ -3615,6 +3729,7 @@ "version": "1.1.0", "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.4" @@ -3624,6 +3739,7 @@ "version": "1.1.0", "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz", "integrity": "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -3633,6 +3749,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-2.0.0.tgz", "integrity": "sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g==", + "dev": true, "license": "MIT", "engines": { "node": ">=18" @@ -3645,6 +3762,7 @@ "version": "1.54.0", "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz", "integrity": "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -3654,6 +3772,7 @@ "version": "3.0.2", "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-3.0.2.tgz", "integrity": "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==", + "dev": true, "license": "MIT", "dependencies": { "mime-db": "^1.54.0" @@ -3734,6 +3853,7 @@ "version": "2.1.3", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, "license": "MIT" }, "node_modules/nanoid": { @@ -3765,6 +3885,7 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-1.0.0.tgz", "integrity": "sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -3786,6 +3907,7 @@ "version": "4.1.1", "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz", "integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==", + "dev": true, "license": "MIT", "engines": { "node": ">=0.10.0" @@ -3795,6 +3917,7 @@ "version": "1.13.4", "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz", "integrity": "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.4" @@ -3827,6 +3950,7 @@ "version": "2.4.1", "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz", "integrity": "sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg==", + "dev": true, "license": "MIT", "dependencies": { "ee-first": "1.1.1" @@ -3907,6 +4031,7 @@ "version": "1.3.3", "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", "integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -3916,6 +4041,7 @@ "version": "3.1.1", "resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz", "integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==", + "dev": true, "license": "MIT", "engines": { "node": ">=8" @@ -3941,6 +4067,7 @@ "version": "8.4.2", "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-8.4.2.tgz", "integrity": "sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==", + "dev": true, "license": "MIT", "funding": { "type": "opencollective", @@ -4075,6 +4202,7 @@ "version": "5.0.1", "resolved": "https://registry.npmjs.org/pkce-challenge/-/pkce-challenge-5.0.1.tgz", "integrity": "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==", + "dev": true, "license": "MIT", "engines": { "node": ">=16.20.0" @@ -4212,6 +4340,7 @@ "version": "2.0.7", "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", "integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==", + "dev": true, "license": "MIT", "dependencies": { "forwarded": "0.2.0", @@ -4235,6 +4364,7 @@ "version": "6.15.2", "resolved": "https://registry.npmjs.org/qs/-/qs-6.15.2.tgz", "integrity": "sha512-Rzq0KEyX/w/tEybncDgdkZrJgVUsUMk3xjh3t5bv3S1HTAtg+uOYt72+ZfwiQwKdysThkTBdL/rTi6HDmX9Ddw==", + "dev": true, "license": "BSD-3-Clause", "dependencies": { "side-channel": "^1.1.0" @@ -4250,6 +4380,7 @@ "version": "1.2.1", "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz", "integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -4259,6 +4390,7 @@ "version": "3.0.2", "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-3.0.2.tgz", "integrity": "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA==", + "dev": true, "license": "MIT", "dependencies": { "bytes": "~3.1.2", @@ -4316,6 +4448,7 @@ "version": "2.0.2", "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", + "dev": true, "license": "MIT", "engines": { "node": ">=0.10.0" @@ -4406,6 +4539,7 @@ "version": "2.2.0", "resolved": "https://registry.npmjs.org/router/-/router-2.2.0.tgz", "integrity": "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==", + "dev": true, "license": "MIT", "dependencies": { "debug": "^4.4.0", @@ -4442,6 +4576,7 @@ "version": "2.1.2", "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", + "dev": true, "license": "MIT" }, "node_modules/section-matter": { @@ -4479,6 +4614,7 @@ "version": "1.2.1", "resolved": "https://registry.npmjs.org/send/-/send-1.2.1.tgz", "integrity": "sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==", + "dev": true, "license": "MIT", "dependencies": { "debug": "^4.4.3", @@ -4520,6 +4656,7 @@ "version": "2.2.1", "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-2.2.1.tgz", "integrity": "sha512-xRXBn0pPqQTVQiC8wyQrKs2MOlX24zQ0POGaj0kultvoOCstBQM5yvOhAVSUwOMjQtTvsPWoNCHfPGwaaQJhTw==", + "dev": true, "license": "MIT", "dependencies": { "encodeurl": "^2.0.0", @@ -4539,6 +4676,7 @@ "version": "1.2.0", "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz", "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==", + "dev": true, "license": "ISC" }, "node_modules/sharp": { @@ -4589,6 +4727,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz", "integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==", + "dev": true, "license": "MIT", "dependencies": { "shebang-regex": "^3.0.0" @@ -4601,6 +4740,7 @@ "version": "3.0.0", "resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz", "integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==", + "dev": true, "license": "MIT", "engines": { "node": ">=8" @@ -4610,6 +4750,7 @@ "version": "1.1.0", "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz", "integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==", + "dev": true, "license": "MIT", "dependencies": { "es-errors": "^1.3.0", @@ -4629,6 +4770,7 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.1.tgz", "integrity": "sha512-mjn/0bi/oUURjc5Xl7IaWi/OJJJumuoJFQJfDDyO46+hBWsfaVM65TBHq2eoZBhzl9EchxOijpkbRC8SVBQU0w==", + "dev": true, "license": "MIT", "dependencies": { "es-errors": "^1.3.0", @@ -4645,6 +4787,7 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz", "integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==", + "dev": true, "license": "MIT", "dependencies": { "call-bound": "^1.0.2", @@ -4663,6 +4806,7 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz", "integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==", + "dev": true, "license": "MIT", "dependencies": { "call-bound": "^1.0.2", @@ -4885,6 +5029,7 @@ "version": "2.0.2", "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz", "integrity": "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -5043,6 +5188,7 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz", "integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==", + "dev": true, "license": "MIT", "engines": { "node": ">=0.6" @@ -5108,6 +5254,7 @@ "version": "2.1.0", "resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz", "integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==", + "dev": true, "license": "MIT", "dependencies": { "content-type": "^2.0.0", @@ -5126,6 +5273,7 @@ "version": "2.0.0", "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz", "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==", + "dev": true, "license": "MIT", "engines": { "node": ">=18" @@ -5159,6 +5307,7 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz", "integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -5174,6 +5323,7 @@ "version": "1.1.2", "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz", "integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==", + "dev": true, "license": "MIT", "engines": { "node": ">= 0.8" @@ -5351,6 +5501,7 @@ "version": "2.0.2", "resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz", "integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==", + "dev": true, "license": "ISC", "dependencies": { "isexe": "^2.0.0" @@ -5460,6 +5611,7 @@ "version": "3.25.2", "resolved": "https://registry.npmjs.org/zod-to-json-schema/-/zod-to-json-schema-3.25.2.tgz", "integrity": "sha512-O/PgfnpT1xKSDeQYSCfRI5Gy3hPf91mKVDuYLUHZJMiDFptvP41MSnWofm8dnCm0256ZNfZIM7DSzuSMAFnjHA==", + "dev": true, "license": "ISC", "peerDependencies": { "zod": "^3.25.28 || ^4" diff --git a/package.json b/package.json index d2d1fa56..30f624a8 100644 --- a/package.json +++ b/package.json @@ -57,7 +57,8 @@ "dependencies": { "@anthropic-ai/sdk": "^0.110.0", "@huggingface/transformers": "^4.2.0", - "@modelcontextprotocol/sdk": "^1.29.0", + "@modelcontextprotocol/node": "^2.0.0", + "@modelcontextprotocol/server": "^2.0.0", "better-sqlite3": "^12.10.0", "chokidar": "^4.0.3", "glob": "^13.0.6", @@ -68,10 +69,13 @@ }, "devDependencies": { "@biomejs/biome": "^2.4.15", + "@modelcontextprotocol/client": "^2.0.0", + "@modelcontextprotocol/sdk": "^1.29.0", "@types/better-sqlite3": "^7.6.13", "@types/js-yaml": "^4.0.9", "@types/node": "^25.8.0", "@types/pg": "^8.20.0", + "ajv": "^8.20.0", "husky": "^9.1.7", "lint-staged": "^16.4.0", "pg": "^8.22.0", diff --git a/packages/router/test/integration.test.ts b/packages/router/test/integration.test.ts index d6483c84..eeff7586 100644 --- a/packages/router/test/integration.test.ts +++ b/packages/router/test/integration.test.ts @@ -274,9 +274,9 @@ describe("router integration", () => { arguments: { path: `vault-a:${TEST_READ_PATH}` }, }); expect(r.isError).toBeFalsy(); - const text = (r.content?.[0] as { text: string }).text; - const payload = JSON.parse(text); - // vault_read returns { content: string (body), frontmatter: {...} } + // vault_read's `content` (body) rides the text channel only (C11); the + // typed fields — frontmatter included — are on structuredContent. + const payload = r.structuredContent as { frontmatter: { title: string } }; expect(payload.frontmatter.title).toBe("Router Read Test"); }); }); diff --git a/src/anchors/classify.ts b/src/anchors/classify.ts new file mode 100644 index 00000000..0e8146b1 --- /dev/null +++ b/src/anchors/classify.ts @@ -0,0 +1,156 @@ +// src/anchors/classify.ts +// The 4-step git-plumbing pin classifier, shared by the read path +// (src/anchors/read.ts), the batch audit (src/audit/checks/pins.ts), and +// vault_lint's Decision-4 softening (src/curation/lint.ts). +// +// Spec: docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification- +// design.md, Decision 2, hardened per the 2026-07-27 plan resolution +// (C5 symlink confinement, C7 CRLF/trivial-content). + +import { realpathSync, statSync } from "node:fs"; +import { isAbsolute, relative as nodeRelative, resolve as nodeResolve, sep } from "node:path"; +import { DEFAULT_MAX_BYTES, readTextFile } from "../audit/readtext.js"; +import { err, ok, type Result } from "../frontmatter/types.js"; +import { blobSize, catBlob, hashObject } from "../utils/git.js"; +import { symlinkSafeExistsWithin } from "../utils/paths.js"; +import type { PinSpec } from "./pin.js"; + +export type AnchorState = "intact" | "moved" | "missing"; + +export interface AnchorVerdict { + state: AnchorState; + relocated?: { start: number; end: number }; +} + +// A pinned target confined to its repo, realpath-resolved. `relPath` is +// relative to the repo's OWN real root — every subsequent git/fs read +// operates on this confirmed real location, never the literal (possibly +// symlinked) `codeRelPath` the doc wrote. +export interface ConfinedFile { + absPath: string; + relPath: string; +} + +// Step 1: realpath-based confinement (C5) — a symlink inside the repo +// pointing outside it resolves to null (the classifier's "missing"), and its +// bytes are never read. Cheap: only fs syscalls (realpath, stat), no +// subprocess, so batch callers may run this per-candidate even at the read +// path's pin cap. +export function resolveConfinedFile(repoAbsPath: string, codeRelPath: string): ConfinedFile | null { + let repoReal: string; + try { + repoReal = realpathSync(repoAbsPath); + } catch { + return null; + } + const targetAbs = nodeResolve(repoAbsPath, codeRelPath); + if (!symlinkSafeExistsWithin(repoReal, targetAbs)) return null; + + let real: string; + try { + real = realpathSync(targetAbs); + } catch { + return null; + } + try { + if (!statSync(real).isFile()) return null; + } catch { + return null; + } + + const rel = nodeRelative(repoReal, real); + if (rel.startsWith("..") || isAbsolute(rel)) return null; // defense in depth + return { absPath: real, relPath: rel.split(sep).join("/") }; +} + +function normalizeNewlines(text: string): string { + return text.replace(/\r\n/g, "\n"); +} + +// Steps 2-4, given the file's CURRENT blob hash (from a batch `hashObjects` +// call, or a single `hashObject` call via classifyPin below). Every failure +// mode this function can hit is an EXPECTED degrade the spec names +// explicitly (blob absent from the odb, size over cap, range past the +// pinned blob's last line, substring not found, current file unreadable) — +// each classifies `moved`, never throws, never returns a Result. The only +// genuinely unexpected failure (the hash-object subprocess itself failing) +// happens one layer up, in the batch/single caller. +export async function classifyAgainstHash( + repoAbsPath: string, + realFilePath: string, + pin: PinSpec, + currentHash: string, +): Promise { + // Step 2: pin sha is a prefix of the current blob id -> intact (blob + // unchanged; every line unchanged). + if (currentHash.startsWith(pin.sha)) { + return { state: "intact" }; + } + + // Whole-file pin with a differing blob -> moved (step 4). + if (pin.start === null || pin.end === null) { + return { state: "moved" }; + } + + // Step 3: range pin, blob differs. Retrieve the pinned content — gated on + // size BEFORE the read, mirroring readtext.ts's stat-before-read guard. + const sizeRes = await blobSize(repoAbsPath, pin.sha); + if (!sizeRes.ok || sizeRes.value > DEFAULT_MAX_BYTES) { + return { state: "moved" }; + } + const blobRes = await catBlob(repoAbsPath, pin.sha); + if (!blobRes.ok) { + return { state: "moved" }; // pinned blob absent from the odb + } + + const pinnedLines = normalizeNewlines(blobRes.value).split("\n"); + if (pin.end > pinnedLines.length) { + return { state: "moved" }; // range past the pinned blob's last line + } + const slice = pinnedLines.slice(pin.start - 1, pin.end).join("\n"); + + // C7: below the threshold the claim is unverifiable — classify `moved` + // (conservative: prompts a re-read rather than asserting freshness). + const nonWhitespace = slice.replace(/\s/g, ""); + if (nonWhitespace.length < 16) { + return { state: "moved" }; + } + + const currentRead = await readTextFile(realFilePath); + if (!currentRead.ok) { + return { state: "moved" }; // current file unreadable (size/binary/encoding) + } + const currentText = normalizeNewlines(currentRead.value.text); + + // C7: CRLF normalization on both sides above; exact-substring search, + // first occurrence wins (no uniqueness requirement — see C7/C11). + const idx = currentText.indexOf(slice); + if (idx === -1) { + return { state: "moved" }; + } + const before = currentText.slice(0, idx); + const startLine = before.split("\n").length; + const endLine = startLine + slice.split("\n").length - 1; + return { state: "intact", relocated: { start: startLine, end: endLine } }; +} + +// The full single-pin classifier: steps 1-4, doing its own confinement and +// hash-object subprocess call. Convenience API for standalone/test use and +// for low-volume callers; batch callers (read.ts, audit/checks/pins.ts, +// lint.ts) call resolveConfinedFile + a shared hashObjects batch + +// classifyAgainstHash directly to pay one subprocess per REPO rather than +// per pin (C1). +export async function classifyPin( + repoAbsPath: string, + codeRelPath: string, + pin: PinSpec, +): Promise> { + const confined = resolveConfinedFile(repoAbsPath, codeRelPath); + if (confined === null) { + return ok({ state: "missing" }); + } + const hashRes = await hashObject(repoAbsPath, confined.relPath); + if (!hashRes.ok) return err(hashRes.error); + const verdict = await classifyAgainstHash(repoAbsPath, confined.absPath, pin, hashRes.value); + return ok(verdict); +} diff --git a/src/anchors/pin.ts b/src/anchors/pin.ts new file mode 100644 index 00000000..29bc3927 --- /dev/null +++ b/src/anchors/pin.ts @@ -0,0 +1,88 @@ +// src/anchors/pin.ts +// Pin grammar: an optional `#L[-]@` suffix on a `describes` +// binding, recording the git blob id (and optionally a line range) the +// author looked at when the binding was written. +// Spec: docs/superpowers/specs/2026-07-26-citation-anchors-jit-verification- +// design.md, Decision 1. +// +// splitPin strips the pin FIRST — end-anchored, so a path containing `@` or +// `#` mid-string is unaffected — then hands the remainder (the bare +// `repo:path::symbol` binding) to the existing describes parser +// (parseDescribesEntry, src/audit/describes.ts) untouched. That parser is +// never modified; this module has no dependency on it, so describes.ts +// depends on pin.ts and not the other way around. + +export interface PinSpec { + start: number | null; + end: number | null; + sha: string; +} + +export interface PinnedEntry { + binding: string; + pin: PinSpec | null; +} + +// End-anchored: an optional `#L[-]` range marker, then a mandatory +// `@<7-40 lowercase hex>` blob id, at the very end of the trimmed entry. A +// path that legitimately contains `@` or `#` mid-string is unaffected (the +// pattern only matches at end-of-string); a path that itself ENDS in text +// matching this shape is a known, accepted ambiguity — the pin wins (spec +// Decision 1). +export const PIN_RE = /(#L(\d+)(?:-(\d+))?)?@([0-9a-f]{7,40})$/; + +// Strips the pin suffix, if present. On no match, `binding` is the entry +// verbatim and `pin` is null — a byte-identical passthrough for every entry +// written before pins existed. An inverted range (`end < start`) degrades +// the WHOLE entry to a bare binding: a malformed range makes the sha claim +// untrustworthy too, so falling back to "no pin" is safer than trusting a +// nonsensical one (surfaced separately by looksLikeMalformedPin, below). +export function splitPin(entry: string): PinnedEntry { + const trimmed = entry.trim(); + const m = trimmed.match(PIN_RE); + if (!m) return { binding: trimmed, pin: null }; + + const startStr = m[2]; + const endStr = m[3]; + const sha = m[4] as string; + const start = startStr !== undefined ? Number.parseInt(startStr, 10) : null; + // A bare "#L40" (no "-end") means the single line 40. + const end = endStr !== undefined ? Number.parseInt(endStr, 10) : start; + + if (start !== null && end !== null && end < start) { + return { binding: trimmed, pin: null }; + } + + const binding = trimmed.slice(0, m.index).trim(); + return { binding, pin: { start, end, sha } }; +} + +// Advisory heuristic for vault_lint's malformedPins check (Phase 8). Never +// blocks a write, never affects splitPin's own parse. Fires ONLY on +// near-misses that a strict PIN_RE match would reject — tightened per the +// 2026-07-27 plan resolution (C11) so ordinary text ending in `@` or `#L`- +// shaped substrings (`::@property`, `::render@v2`, an npm-style scoped +// import) does not false-positive: +// +// (a) a trailing `#L[-]@` — a range marker plus `@` is a +// strong pin-intent signal, so a near-miss sha after it is reported; +// (b) a trailing `@<4-40 hex chars>` that fails strict PIN_RE — too short +// (4-6 chars) or contains uppercase hex; +// (c) a strict structural match with an inverted range (end < start). +const MALFORMED_RANGE_NEAR_MISS = /#L\d+(?:-\d+)?@[0-9a-zA-Z]*$/; +const MALFORMED_HEX_NEAR_MISS = /@[0-9a-fA-F]{4,40}$/; + +export function looksLikeMalformedPin(entry: string): boolean { + const trimmed = entry.trim(); + const strict = trimmed.match(PIN_RE); + if (strict) { + const startStr = strict[2]; + const endStr = strict[3]; + if (startStr !== undefined && endStr !== undefined) { + return Number.parseInt(endStr, 10) < Number.parseInt(startStr, 10); + } + return false; + } + if (MALFORMED_RANGE_NEAR_MISS.test(trimmed)) return true; + return MALFORMED_HEX_NEAR_MISS.test(trimmed); +} diff --git a/src/anchors/read.ts b/src/anchors/read.ts new file mode 100644 index 00000000..98acdbd4 --- /dev/null +++ b/src/anchors/read.ts @@ -0,0 +1,210 @@ +// src/anchors/read.ts +// vault_read integration: repo resolution, batching, the pin cap, and the +// drift banner. Spec: docs/superpowers/specs/2026-07-26-citation-anchors- +// jit-verification-design.md, Decision 2, batched per the 2026-07-27 plan +// resolution (C1). +// +// Cost posture: candidates are grouped by repo; step 1 (existence + +// confinement) is fs-syscall-only per candidate; step 2 (current blob hash) +// is ONE `git hash-object` batch invocation per repo, answering every +// candidate in that repo at once — the all-intact path (the common case) +// therefore costs one subprocess per referenced repo per read, not one per +// pin. Only candidates whose blob differs AND carry a range go on to step 3 +// (git cat-file + a bounded text read), run with a small bounded +// concurrency so a doc with many drifted range pins can't serialize the +// read behind N sequential git spawns. + +import { existsSync, statSync } from "node:fs"; +import { parseDescribesEntry } from "../audit/describes.js"; +import { hashObjects } from "../utils/git.js"; +import { type AnchorState, classifyAgainstHash, resolveConfinedFile } from "./classify.js"; +import { type PinSpec, splitPin } from "./pin.js"; + +export const MAX_PINS_PER_READ = 24; +const STEP3_CONCURRENCY = 4; + +export interface AnchorEntry { + raw: string; + repo: string; + path: string; + symbol: string | null; + pin: { start: number | null; end: number | null; sha: string }; + state: AnchorState; + relocated?: { start: number; end: number }; +} + +export interface AnchorsAnnotation { + entries: AnchorEntry[]; + checked: number; + skipped: number; + // Classifier failures (a repo's whole hashObjects batch call erroring) — + // dropped from `entries`, counted here so the "all intact" softening never + // quantifies over a silently-censored sample (C8). + errored: number; + banner: string | null; +} + +interface Candidate { + idx: number; // original position in `describes`, for stable output order + raw: string; + repo: string; + path: string; + symbol: string | null; + pin: PinSpec; +} + +// A bare (prefix-less) binding resolves to "the doc's own repo" in the +// audit; on the read path that is the vault itself, never a code repo, so it +// is never JIT-checked. Passing this sentinel as parseDescribesEntry's +// `sourceRepo` lets us detect the bare case (parsed.repo === sentinel) without +// duplicating the grammar's `::`/`:` split logic here. +const NO_PREFIX_SENTINEL = ""; + +function selectCandidates(describes: string[], codeRepos: Record): Candidate[] { + const repoDirExists = new Map(); + const out: Candidate[] = []; + describes.forEach((raw, idx) => { + const { binding, pin } = splitPin(raw); + if (!pin) return; + const parsed = parseDescribesEntry(binding, NO_PREFIX_SENTINEL); + if (parsed.repo === NO_PREFIX_SENTINEL) return; // bare binding, not JIT-checked + if (!(parsed.repo in codeRepos)) return; + if (!repoDirExists.has(parsed.repo)) { + const repoPath = codeRepos[parsed.repo] as string; + let exists = false; + try { + exists = existsSync(repoPath) && statSync(repoPath).isDirectory(); + } catch { + exists = false; + } + repoDirExists.set(parsed.repo, exists); + } + if (!repoDirExists.get(parsed.repo)) return; + out.push({ idx, raw, repo: parsed.repo, path: parsed.path, symbol: parsed.symbol, pin }); + }); + return out; +} + +function makeEntry( + c: Candidate, + verdict: { state: AnchorState; relocated?: { start: number; end: number } }, +): AnchorEntry { + return { + raw: c.raw, + repo: c.repo, + path: c.path, + symbol: c.symbol, + pin: { start: c.pin.start, end: c.pin.end, sha: c.pin.sha }, + state: verdict.state, + ...(verdict.relocated ? { relocated: verdict.relocated } : {}), + }; +} + +// Runs `tasks` with at most `limit` in flight at once. Every task here is +// infallible (classifyAgainstHash never throws/rejects — every failure mode +// degrades to a specific AnchorState), so this has no error-collection +// machinery; it exists purely to bound concurrency. +async function runBounded(tasks: Array<() => Promise>, limit: number): Promise { + let next = 0; + async function worker(): Promise { + while (next < tasks.length) { + const task = tasks[next++]; + if (task) await task(); + } + } + const workers = Array.from({ length: Math.max(1, Math.min(limit, tasks.length)) }, () => + worker(), + ); + await Promise.allSettled(workers); +} + +// Candidate selection: valid pin AND explicit repo prefix AND prefix in +// `codeRepos` AND repo dir exists. Zero candidates -> null (byte-identical +// to "nothing to say", the read path's silent-baseline contract). +export async function computeAnchors( + describes: string[], + codeRepos: Record, +): Promise { + const candidates = selectCandidates(describes, codeRepos); + if (candidates.length === 0) return null; + + const capped = candidates.slice(0, MAX_PINS_PER_READ); + const skipped = candidates.length - capped.length; + + const byRepo = new Map(); + for (const c of capped) { + const list = byRepo.get(c.repo) ?? []; + list.push(c); + byRepo.set(c.repo, list); + } + + const out: Array<{ idx: number; entry: AnchorEntry }> = []; + let errored = 0; + + for (const [repoName, repoCandidates] of byRepo) { + const repoAbsPath = codeRepos[repoName] as string; + + // Step 1: cheap, per-candidate, no subprocess. + const resolved = repoCandidates.map((c) => ({ + c, + confined: resolveConfinedFile(repoAbsPath, c.path), + })); + for (const r of resolved) { + if (r.confined === null) { + out.push({ idx: r.c.idx, entry: makeEntry(r.c, { state: "missing" }) }); + } + } + const survivors = resolved.filter( + (r): r is { c: Candidate; confined: NonNullable<(typeof r)["confined"]> } => + r.confined !== null, + ); + if (survivors.length === 0) continue; + + // Step 2: ONE hash-object batch call per repo. + const hashRes = await hashObjects( + repoAbsPath, + survivors.map((s) => s.confined.relPath), + ); + if (!hashRes.ok) { + errored += survivors.length; + continue; + } + const hashes = hashRes.value; + + const step3: Array<() => Promise> = []; + survivors.forEach((s, i) => { + const currentHash = hashes[i] as string; + if (currentHash.startsWith(s.c.pin.sha)) { + out.push({ idx: s.c.idx, entry: makeEntry(s.c, { state: "intact" }) }); + return; + } + if (s.c.pin.start === null || s.c.pin.end === null) { + out.push({ idx: s.c.idx, entry: makeEntry(s.c, { state: "moved" }) }); + return; + } + step3.push(async () => { + const verdict = await classifyAgainstHash( + repoAbsPath, + s.confined.absPath, + s.c.pin, + currentHash, + ); + out.push({ idx: s.c.idx, entry: makeEntry(s.c, verdict) }); + }); + }); + await runBounded(step3, STEP3_CONCURRENCY); + } + + out.sort((a, b) => a.idx - b.idx); + const entries = out.map((o) => o.entry); + + const movedOrMissing = entries.filter((e) => e.state === "moved" || e.state === "missing").length; + const banner = + movedOrMissing > 0 + ? `⚠ CODE DRIFT — ${movedOrMissing} of ${capped.length} code pin(s) on this document report ` + + "moved or missing. The code this document describes has changed since the pins were " + + "written; re-read the code before relying on this document's account of it." + : null; + + return { entries, checked: capped.length, skipped, errored, banner }; +} diff --git a/src/audit/checks/pins.ts b/src/audit/checks/pins.ts new file mode 100644 index 00000000..3efbe96e --- /dev/null +++ b/src/audit/checks/pins.ts @@ -0,0 +1,102 @@ +// src/audit/checks/pins.ts +// Batch pin classification for `daftari audit` (2026-07-26 citation-anchors- +// jit spec, Decision 3). Reuses the read path's classifier +// (src/anchors/classify.ts) and its batching primitive (C1): candidates are +// grouped by target repo so the all-intact case costs one `git hash-object` +// invocation per repo, not one per pinned binding. + +import { classifyAgainstHash, resolveConfinedFile } from "../../anchors/classify.js"; +import type { PinSpec } from "../../anchors/pin.js"; +import { hashObjects } from "../../utils/git.js"; +import type { DescribesEdge, PinFinding, PinState, RepoSnapshot } from "../types.js"; + +function mk( + e: DescribesEdge, + verdict: { state: PinState; relocated?: { start: number; end: number } }, +): PinFinding { + return { + source: { repo: e.sourceRepo, path: e.sourcePath }, + target: { repo: e.targetRepo, path: e.targetPath }, + raw: e.raw, + state: verdict.state, + ...(verdict.relocated ? { relocated: verdict.relocated } : {}), + }; +} + +// Classifies every pinned edge against the audit's own repo snapshots (the +// registry DescribesEdge targets resolve against — unchanged by the +// registry cross-check in index.ts, which only WARNS about a divergence). An +// edge whose target repo isn't in the snapshot set at all (already a +// `broken_describes` finding from checkDescribesRefs) classifies `missing` +// here too, for a consistent pin-totals story. +export async function checkPins( + snapshots: RepoSnapshot[], + edges: DescribesEdge[], +): Promise { + const byRepo = new Map(); + for (const s of snapshots) byRepo.set(s.config.name, s); + + const pinned = edges.filter((e): e is DescribesEdge & { pin: PinSpec } => e.pin !== null); + if (pinned.length === 0) return []; + + const byTargetRepo = new Map>(); + for (const e of pinned) { + const list = byTargetRepo.get(e.targetRepo) ?? []; + list.push(e); + byTargetRepo.set(e.targetRepo, list); + } + + const findings: PinFinding[] = []; + for (const [repoName, repoEdges] of byTargetRepo) { + const snap = byRepo.get(repoName); + if (!snap) { + for (const e of repoEdges) findings.push(mk(e, { state: "missing" })); + continue; + } + const repoAbsPath = snap.config.path; + + const resolved = repoEdges.map((e) => ({ + e, + confined: resolveConfinedFile(repoAbsPath, e.targetPath), + })); + for (const r of resolved) { + if (r.confined === null) findings.push(mk(r.e, { state: "missing" })); + } + const survivors = resolved.filter( + (r): r is { e: (typeof repoEdges)[number]; confined: NonNullable<(typeof r)["confined"]> } => + r.confined !== null, + ); + if (survivors.length === 0) continue; + + const hashRes = await hashObjects( + repoAbsPath, + survivors.map((s) => s.confined.relPath), + ); + if (!hashRes.ok) { + // A whole-batch subprocess failure is rare (missing git binary, a + // corrupted odb) — the audit has no separate "errored" bucket per + // pin, so this degrades conservatively to `moved` (a prompt, not a + // silently-dropped finding) rather than being omitted from the report. + for (const s of survivors) findings.push(mk(s.e, { state: "moved" })); + continue; + } + const hashes = hashRes.value; + + for (let i = 0; i < survivors.length; i++) { + const s = survivors[i] as (typeof survivors)[number]; + const currentHash = hashes[i] as string; + const pin = s.e.pin; + if (currentHash.startsWith(pin.sha)) { + findings.push(mk(s.e, { state: "intact" })); + continue; + } + if (pin.start === null || pin.end === null) { + findings.push(mk(s.e, { state: "moved" })); + continue; + } + const verdict = await classifyAgainstHash(repoAbsPath, s.confined.absPath, pin, currentHash); + findings.push(mk(s.e, verdict)); + } + } + return findings; +} diff --git a/src/audit/collect.ts b/src/audit/collect.ts index f819795b..f96eb34b 100644 --- a/src/audit/collect.ts +++ b/src/audit/collect.ts @@ -4,35 +4,22 @@ // git log to populate mtimes; on any git failure, fall back to fs mtime. import { type ExecFileSyncOptions, execFileSync } from "node:child_process"; -import { realpathSync, statSync } from "node:fs"; +import { statSync } from "node:fs"; import { readFile } from "node:fs/promises"; -import { isAbsolute, relative as nodeRelative, resolve as nodeResolve } from "node:path"; +import { resolve as nodeResolve } from "node:path"; import { glob } from "glob"; import matter from "gray-matter"; import { err, ok, type Result } from "../frontmatter/types.js"; +import { symlinkSafeExistsWithin } from "../utils/paths.js"; import { extractLinksFromBody } from "./links.js"; import type { AuditConfig, AuditError, DocSnapshot, RepoConfig, RepoSnapshot } from "./types.js"; import { runtimeError } from "./types.js"; -// The disk oracle for checkBrokenRefs (#132/#133): true iff targetAbs -// exists AND its REAL location sits under rootAbs. realpathSync resolves -// every component, so a symlink committed inside an audited repo -// (escape -> /) cannot route the probe outside the containment root — a -// lexical check plus a bare existsSync would (security review on #255). -// rootAbs is expected to be already-real: repo roots are realpathSync'd at -// config load, and the parent prefix of a real path is itself real. -// A nonexistent target makes realpathSync throw ENOENT → false, which is -// exactly the "missing" answer. -export function symlinkSafeExistsWithin(rootAbs: string, targetAbs: string): boolean { - let real: string; - try { - real = realpathSync(targetAbs); - } catch { - return false; - } - const rel = nodeRelative(rootAbs, real); - return rel === "" || (!rel.startsWith("..") && !isAbsolute(rel)); -} +// The disk oracle for checkBrokenRefs (#132/#133), now shared with the +// citation-anchor classifier (2026-07-26 spec, C5) — lifted to +// src/utils/paths.ts. Re-exported here for existing callers (index.ts, +// tests) so the import path stays stable. +export { symlinkSafeExistsWithin } from "../utils/paths.js"; function slugify(heading: string): string { // GitHub slug: lowercase, strip non-alphanumeric (keep `-_`), whitespace -> `-`. diff --git a/src/audit/describes.ts b/src/audit/describes.ts index 80110d1b..508eb89d 100644 --- a/src/audit/describes.ts +++ b/src/audit/describes.ts @@ -4,6 +4,7 @@ // that the reference-integrity check (checks/describes_refs.ts) verifies against // the resolved target repo. +import { splitPin } from "../anchors/pin.js"; import type { DescribesEdge, RepoSnapshot } from "./types.js"; export interface ParsedDescribes { @@ -42,7 +43,12 @@ export function classifyDescribesEdges(snapshots: RepoSnapshot[]): DescribesEdge const sourceRepo = snap.config.name; for (const doc of snap.docs.values()) { for (const raw of doc.describes ?? []) { - const parsed = parseDescribesEntry(raw, sourceRepo); + // Pins strip FIRST (2026-07-26 citation-anchors-jit spec, Decision 1) + // — the remainder goes to parseDescribesEntry untouched, so + // targetPath is always pin-stripped and checkDescribesRefs / + // runSemanticCheck are unaffected. + const { binding, pin } = splitPin(raw); + const parsed = parseDescribesEntry(binding, sourceRepo); // A blank or whitespace-only entry resolves to an empty target path — // skip it rather than emit a confusing "missing file: repo/" finding. if (parsed.path.length === 0) continue; @@ -53,6 +59,7 @@ export function classifyDescribesEdges(snapshots: RepoSnapshot[]): DescribesEdge targetPath: parsed.path, symbol: parsed.symbol, raw, + pin, }); } } diff --git a/src/audit/docs-repo.ts b/src/audit/docs-repo.ts new file mode 100644 index 00000000..9bd7e95d --- /dev/null +++ b/src/audit/docs-repo.ts @@ -0,0 +1,21 @@ +// src/audit/docs-repo.ts +// Resolves the single unambiguous docs repo a flag needs to act on — shared +// by --auto-tension (tension entries land in one vault's .daftari/tensions.md) +// and --pin --apply (pins are written to one vault's markdown files). +// Generalized from the original resolveTensionVault per the 2026-07-26 +// citation-anchors-jit plan resolution (C10). + +import type { AuditConfig } from "./types.js"; + +export function resolveSingleDocsRepo( + config: AuditConfig, + flagLabel: string, +): string | { error: string } { + const docsRepos = config.repos.filter((r) => r.type !== "code"); + if (docsRepos.length !== 1) { + return { + error: `${flagLabel} requires exactly one docs repo to act on; found ${docsRepos.length}`, + }; + } + return (docsRepos[0] as { path: string }).path; +} diff --git a/src/audit/index.ts b/src/audit/index.ts index 04bb8c33..dc3a5488 100644 Binary files a/src/audit/index.ts and b/src/audit/index.ts differ diff --git a/src/audit/pin.ts b/src/audit/pin.ts new file mode 100644 index 00000000..d0b4d1df --- /dev/null +++ b/src/audit/pin.ts @@ -0,0 +1,300 @@ +// src/audit/pin.ts +// `daftari audit --pin` / `--pin --apply` — backfills whole-file pins onto +// unpinned `describes` bindings. Spec: docs/superpowers/specs/2026-07-26- +// citation-anchors-jit-verification-design.md, Decision 5, hardened per the +// plan resolution (C6 dirty-tree skip, C10 live-holder refusal). +// +// Follows the `daftari backfill` plan/apply precedent (src/backfill/) with +// one deliberate difference: there is no persisted plan FILE. `--pin` (plan +// mode, the default) recomputes the proposal fresh every run and prints it; +// `--pin --apply` recomputes the SAME proposal and writes it in one pass — +// simpler than backfill's ratify-per-folder flow because a pin backfill is +// either wholly safe (clean tree) or wholly deferred (dirty tree), never +// partially ratified. + +import { existsSync } from "node:fs"; +import { writeFile } from "node:fs/promises"; +import { recordProvenance } from "../curation/provenance.js"; +import { parseDocument } from "../frontmatter/parser.js"; +import { validateFrontmatter } from "../frontmatter/schema.js"; +import { err, ok, type Result } from "../frontmatter/types.js"; +import { isDaftariProcess, readLockfile } from "../lifecycle/lock.js"; +import { readFile, resolveVaultPath } from "../storage/local.js"; +import { serializeDocument } from "../tools/write.js"; +import { loadConfig } from "../utils/config.js"; +import { blobAtHead, commit, hashObjects } from "../utils/git.js"; +import { collectRepos } from "./collect.js"; +import { classifyDescribesEdges } from "./describes.js"; +import { resolveSingleDocsRepo } from "./docs-repo.js"; +import type { AuditConfig, DescribesEdge } from "./types.js"; + +export interface PinPlanEntry { + path: string; // vault-relative doc path + repo: string; // code repo name (matches code_repos) + targetPath: string; // repo-relative code path + oldEntry: string; // the describes entry as written + newEntry: string; // oldEntry + "@" +} + +export interface PinSkip { + path: string; + repo: string; + targetPath: string; + reason: string; +} + +export interface PinPlanResult { + docsRepoPath: string; + docsRepoName: string; + proposals: PinPlanEntry[]; + skipped: PinSkip[]; + // Repo prefixes referenced by an unpinned binding that resolve in the + // AUDIT's own registry but not in the docs vault's own `code_repos` — + // C2's "unpinnable: not in code_repos" bucket, so the CLI-flag workflow + // gets an actionable message instead of dead silence. + unpinnable: string[]; +} + +function liveHolder(vaultRoot: string): { pid: number; mode: string } | null { + const lock = readLockfile(vaultRoot); + if (!lock.ok || lock.value === null) return null; + if (!isDaftariProcess(lock.value.pid, vaultRoot)) return null; // stale, not live + return { pid: lock.value.pid, mode: lock.value.mode ?? "stdio" }; +} + +// Plan mode: read-only. For every unpinned edge whose repo is in the docs +// vault's own `code_repos` and whose target resolves at HEAD, batch-hashes +// the working tree per repo and compares against HEAD's blob. A clean file +// proposes appending `@` (HEAD's blob, so the pin is retrievable from +// the odb and intact-on-arrival by construction); a dirty one is skipped +// with the dirty-skip message (C6) rather than pinning an unretrievable +// working-tree blob. Whole-file pins only — a range is an author's judgment +// call a batch tool never invents (Decision 5). +export async function planPins(config: AuditConfig): Promise> { + const docsRepoPathOrErr = resolveSingleDocsRepo(config, "--pin"); + if (typeof docsRepoPathOrErr !== "string") return err(new Error(docsRepoPathOrErr.error)); + const docsRepoPath = docsRepoPathOrErr; + + const docsConfig = loadConfig(docsRepoPath); + if (!docsConfig.ok) return docsConfig; + const codeRepos = docsConfig.value.codeRepos; + + const collected = await collectRepos(config); + if (!collected.ok) return err(new Error(collected.error.message)); + const snapshots = collected.value; + const docsSnap = snapshots.find((s) => s.config.path === docsRepoPath); + if (!docsSnap) + return err(new Error("internal: docs repo snapshot not found among collected repos")); + + const auditRepoNames = new Set(snapshots.map((s) => s.config.name)); + const edges = classifyDescribesEdges(snapshots); + const unpinned = edges.filter((e) => e.pin === null); + + const unpinnablePrefixes = new Set(); + const candidates: Array<{ edge: DescribesEdge; repoAbsPath: string }> = []; + for (const e of unpinned) { + if (e.targetRepo in codeRepos) { + candidates.push({ edge: e, repoAbsPath: codeRepos[e.targetRepo] as string }); + } else if (auditRepoNames.has(e.targetRepo)) { + unpinnablePrefixes.add(e.targetRepo); + } + } + + // Candidates whose target resolves at HEAD only. + const withHead: Array<{ edge: DescribesEdge; repoAbsPath: string; headSha: string }> = []; + for (const c of candidates) { + const head = await blobAtHead(c.repoAbsPath, c.edge.targetPath); + if (!head.ok) continue; // not resolvable at HEAD -> not plannable + withHead.push({ edge: c.edge, repoAbsPath: c.repoAbsPath, headSha: head.value }); + } + + const byRepo = new Map(); + for (const c of withHead) { + const list = byRepo.get(c.repoAbsPath) ?? []; + list.push(c); + byRepo.set(c.repoAbsPath, list); + } + + const proposals: PinPlanEntry[] = []; + const skipped: PinSkip[] = []; + for (const [repoAbsPath, list] of byRepo) { + // A working-tree deletion must not fail the whole batch hash-object call + // (C1's contract) — split out before hashing, reported as a dirty skip. + const present = list.filter((c) => existsSync(`${repoAbsPath}/${c.edge.targetPath}`)); + const deleted = list.filter((c) => !present.includes(c)); + for (const c of deleted) { + skipped.push({ + path: c.edge.sourcePath, + repo: c.edge.targetRepo, + targetPath: c.edge.targetPath, + reason: "skipped: working tree differs from HEAD (commit first, then re-run)", + }); + } + if (present.length === 0) continue; + + const hashRes = await hashObjects( + repoAbsPath, + present.map((c) => c.edge.targetPath), + ); + if (!hashRes.ok) { + for (const c of present) { + skipped.push({ + path: c.edge.sourcePath, + repo: c.edge.targetRepo, + targetPath: c.edge.targetPath, + reason: `skipped: cannot hash working tree: ${hashRes.error.message}`, + }); + } + continue; + } + present.forEach((c, i) => { + const workingHash = hashRes.value[i] as string; + if (workingHash !== c.headSha) { + skipped.push({ + path: c.edge.sourcePath, + repo: c.edge.targetRepo, + targetPath: c.edge.targetPath, + reason: "skipped: working tree differs from HEAD (commit first, then re-run)", + }); + return; + } + const shortSha = c.headSha.slice(0, 12); + proposals.push({ + path: c.edge.sourcePath, + repo: c.edge.targetRepo, + targetPath: c.edge.targetPath, + oldEntry: c.edge.raw, + newEntry: `${c.edge.raw}@${shortSha}`, + }); + }); + } + + return ok({ + docsRepoPath, + docsRepoName: docsSnap.config.name, + proposals, + skipped, + unpinnable: [...unpinnablePrefixes].sort(), + }); +} + +export interface PinApplyResult { + applied: string[]; + unchanged: string[]; + skipped: PinSkip[]; + commit: string | null; +} + +// Apply: writes the SAME proposals planPins would compute for a clean tree. +// Refuses against a live holder of the docs vault's process.lock (C10) — +// unlike backfill's --scope ratification, there is no override flag in v1: +// the operator's remedy is stopping the server. Idempotent: already-pinned +// (byte-identical serialization) entries produce no write and no commit. +export async function applyPins( + docsRepoPath: string, + proposals: PinPlanEntry[], + agent: string, +): Promise> { + const holder = liveHolder(docsRepoPath); + if (holder) { + return err( + new Error( + `daftari audit --pin --apply refuses: this vault is held by a live daftari ` + + `process (pid=${holder.pid}, mode=${holder.mode}). Stop the server or run ` + + `against an unheld vault.`, + ), + ); + } + + const config = loadConfig(docsRepoPath); + if (!config.ok) return config; + + const byDoc = new Map(); + for (const p of proposals) { + const list = byDoc.get(p.path) ?? []; + list.push(p); + byDoc.set(p.path, list); + } + + const applied: string[] = []; + const unchanged: string[] = []; + const skipped: PinSkip[] = []; + + for (const [docPath, entries] of byDoc) { + const resolved = resolveVaultPath(docsRepoPath, docPath); + if (!resolved.ok) { + skipped.push({ path: docPath, repo: "", targetPath: "", reason: resolved.error.message }); + continue; + } + const existing = await readFile(resolved.value.absPath); + if (!existing.ok) { + skipped.push({ path: docPath, repo: "", targetPath: "", reason: existing.error.message }); + continue; + } + const parsed = parseDocument(existing.value); + if (!parsed.ok) { + skipped.push({ path: docPath, repo: "", targetPath: "", reason: parsed.error.message }); + continue; + } + + const replace = new Map(entries.map((e) => [e.oldEntry, e.newEntry])); + // Already-pinned entries never touched (no auto-repair): only entries + // whose exact `oldEntry` (unpinned, as classified at plan time) still + // appear verbatim are replaced. + const currentDescribes = parsed.value.frontmatter.describes; + const newDescribes = currentDescribes.map((d) => replace.get(d) ?? d); + const proposedFm = { ...parsed.value.frontmatter, describes: newDescribes }; + + const { report } = validateFrontmatter(proposedFm as unknown as Record); + if (!report.valid) { + skipped.push({ + path: docPath, + repo: "", + targetPath: "", + reason: `proposed frontmatter is invalid: ${report.issues.map((i) => `${i.field}: ${i.message}`).join("; ")}`, + }); + continue; + } + + const text = serializeDocument( + proposedFm, + parsed.value.content, + config.value.schemaExtensions, + parsed.value.raw, + ); + if (text === existing.value) { + unchanged.push(docPath); + continue; + } + + try { + await writeFile(resolved.value.absPath, text, "utf-8"); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + skipped.push({ path: docPath, repo: "", targetPath: "", reason: `write failed: ${reason}` }); + continue; + } + applied.push(docPath); + } + + let commitHash: string | null = null; + if (applied.length > 0 && config.value.autoCommit) { + const message = `daftari audit --pin: ${applied.length} doc(s) backfilled with code pins`; + const committed = await commit(docsRepoPath, applied, message, agent, { + gitDir: config.value.gitDir, + }); + if (!committed.ok) return committed; + commitHash = committed.value.hash; + } + + for (const path of applied) { + await recordProvenance(docsRepoPath, { + tool: "daftari-audit", + file: path, + agent, + action: "update", + }); + } + + return ok({ applied, unchanged, skipped, commit: commitHash }); +} diff --git a/src/audit/report.ts b/src/audit/report.ts index fe1c4482..21dfa171 100644 --- a/src/audit/report.ts +++ b/src/audit/report.ts @@ -1,11 +1,14 @@ // src/audit/report.ts // Pure formatters over AuditReport. No IO. +import type { PinApplyResult, PinPlanResult } from "./pin.js"; import type { SemanticFinding } from "./semantic.js"; import type { AuditReport, BrokenRefFinding, DescribesRefFinding, + PinFinding, + RegistryMismatch, StalenessFinding, } from "./types.js"; @@ -44,6 +47,27 @@ function renderDescribesRefs(rows: DescribesRefFinding[]): string { return `${lines.join("\n")}\n`; } +function renderPins(rows: PinFinding[]): string { + if (rows.length === 0) return "_no pinned bindings._\n"; + const lines = ["| state | source | target | relocated |", "|---|---|---|---|"]; + for (const r of rows) { + const relocated = r.relocated ? `L${r.relocated.start}-${r.relocated.end}` : "—"; + lines.push( + `| ${r.state} | ${r.source.repo}/${r.source.path} | ${r.target.repo}/${r.target.path} | ${relocated} |`, + ); + } + return `${lines.join("\n")}\n`; +} + +function renderRegistryMismatches(rows: RegistryMismatch[]): string { + if (rows.length === 0) return ""; + const lines = ["", "## Registry mismatches (read path vs. audit registry)", ""]; + for (const r of rows) { + lines.push(`- '${r.repo}' referenced from ${r.docsRepo}: ${r.detail}`); + } + return `${lines.join("\n")}\n`; +} + function renderSemantic(rows: SemanticFinding[]): string { // Only non-coherent verdicts are worth surfacing (drifted, contradicted, skipped). const notable = rows.filter((r) => r.verdict !== "coherent"); @@ -70,7 +94,10 @@ export function renderMarkdown(report: AuditReport): string { t.directlyStale === 0 && t.transitivelyStale === 0 && t.brokenDescribes === 0 && - report.semantic.length === 0; + t.pinsMoved === 0 && + t.pinsMissing === 0 && + report.semantic.length === 0 && + report.registryMismatches.length === 0; const head = [ "# Coherence Audit Report", "", @@ -86,6 +113,7 @@ export function renderMarkdown(report: AuditReport): string { `- transitively stale docs: **${t.transitivelyStale}**`, `- broken doc-to-code bindings: **${t.brokenDescribes}**`, `- doc-to-code semantic drift: **${t.semanticDrifted}**`, + `- code pins intact / moved / missing: **${t.pinsIntact} / ${t.pinsMoved} / ${t.pinsMissing}**`, "", ]; if (empty) { @@ -103,12 +131,56 @@ export function renderMarkdown(report: AuditReport): string { "## Broken doc-to-code bindings", "", renderDescribesRefs(report.describesRefs), + "## Pin verification", + "", + renderPins(report.pins), ...(report.semantic.length > 0 ? ["## Semantic coherence", "", renderSemantic(report.semantic)] : []), + renderRegistryMismatches(report.registryMismatches), ].join("\n"); } +// --- `daftari audit --pin` / `--pin --apply` output (pure formatters) ----- + +export function renderPinPlan(plan: PinPlanResult): string { + const lines: string[] = [ + `daftari audit --pin: plan for docs repo '${plan.docsRepoName}' (${plan.docsRepoPath})`, + "", + ]; + if (plan.proposals.length === 0) { + lines.push("no unpinned, plannable bindings found."); + } else { + for (const p of plan.proposals) { + lines.push(`${p.path} · ${p.oldEntry} -> ${p.newEntry}`); + } + } + lines.push("", `proposed: ${plan.proposals.length}`); + if (plan.skipped.length > 0) { + lines.push("", "skipped: working tree differs from HEAD (commit first, then re-run):"); + for (const s of plan.skipped) lines.push(` ${s.path} · ${s.repo}:${s.targetPath}`); + lines.push(`skipped: ${plan.skipped.length}`); + } + if (plan.unpinnable.length > 0) { + lines.push( + "", + "unpinnable: not in code_repos (referenced only by the audit's --code-repo/audit.yaml registry):", + ); + for (const prefix of plan.unpinnable) lines.push(` ${prefix}`); + } + return `${lines.join("\n")}\n`; +} + +export function renderPinApplyResult(result: PinApplyResult): string { + const lines: string[] = [ + `daftari audit --pin --apply: ${result.applied.length} doc(s) written, ` + + `${result.unchanged.length} already at their proposed state, ${result.skipped.length} skipped`, + ]; + if (result.commit) lines.push(`commit: ${result.commit}`); + for (const s of result.skipped) lines.push(`skipped: ${s.path} — ${s.reason}`); + return `${lines.join("\n")}\n`; +} + export function renderJson(report: AuditReport): string { return JSON.stringify(report, null, 2); } diff --git a/src/audit/types.ts b/src/audit/types.ts index 4eb2401e..fe32b8f8 100644 --- a/src/audit/types.ts +++ b/src/audit/types.ts @@ -1,6 +1,7 @@ // src/audit/types.ts // Shared types for the coherence audit. Pure data shapes; no logic. +import type { PinSpec } from "../anchors/pin.js"; import type { SemanticFinding } from "./semantic.js"; export type AuditConfig = { @@ -71,7 +72,35 @@ export type DescribesEdge = { targetRepo: string; // resolved repo name (source repo for a bare path) targetPath: string; // repo-relative path of the described code file symbol: string | null; // `::symbol` suffix, retained but unresolved in v1 - raw: string; // the describes entry exactly as written + raw: string; // the describes entry exactly as written, PIN SUFFIX INCLUDED + // Parsed pin suffix (2026-07-26 citation-anchors-jit spec, Decision 1), + // null for an unpinned binding. targetPath is always pin-stripped — + // checkDescribesRefs and runSemanticCheck are unaffected by this field. + pin: PinSpec | null; +}; + +// One pinned binding's classification against the audit's repo snapshots — +// the same 4-step classifier the read path uses (src/anchors/classify.ts), +// batched per repo (src/audit/checks/pins.ts). +export type PinState = "intact" | "moved" | "missing"; + +export type PinFinding = { + source: { repo: string; path: string }; + target: { repo: string; path: string }; + raw: string; + state: PinState; + relocated?: { start: number; end: number }; +}; + +// Registry cross-check (2026-07-26 plan resolution, C2): a repo name +// referenced by a pinned binding that resolves in exactly one of {the +// audit's own repo registry, the docs repo's own `code_repos` config block}, +// or resolves in both to a different realpath. Silent by default — surfaced +// as a stderr warning and a report note, never a fail_on gate. +export type RegistryMismatch = { + repo: string; + docsRepo: string; + detail: string; }; export type DescribesRefFinding = { @@ -115,12 +144,24 @@ export type AuditReport = { // drifted + contradicted bindings from the opt-in --semantic check; 0 when // the check did not run. semanticDrifted: number; + // Pin verification totals (2026-07-26 spec, Decision 3). 0/0/0 when no + // pinned bindings exist — distinct from "not run"; unlike --semantic, + // pin classification always runs when any pin is present. + pinsIntact: number; + pinsMoved: number; + pinsMissing: number; }; brokenRefs: BrokenRefFinding[]; staleness: StalenessFinding[]; describesRefs: DescribesRefFinding[]; // Populated only when --semantic ran; [] otherwise. semantic: SemanticFinding[]; + // Per-pinned-binding classification (2026-07-26 spec, Decision 3). [] when + // no bindings are pinned. + pins: PinFinding[]; + // Registry cross-check notes (C2). [] when no pinned binding's repo name + // diverges between the audit registry and the docs repo's own code_repos. + registryMismatches: RegistryMismatch[]; }; // Tagged error union. runAudit branches on .kind to translate to exit codes diff --git a/src/consolidate/birth.ts b/src/consolidate/birth.ts index d1b760a1..e7d7b2a2 100644 --- a/src/consolidate/birth.ts +++ b/src/consolidate/birth.ts @@ -20,7 +20,11 @@ import { createHash } from "node:crypto"; import { appendFileSync, mkdirSync } from "node:fs"; import { join, posix } from "node:path"; -import type { DerivesFromEdge, ObserveEdgeInput } from "../curation/edges.js"; +import { + computeInputsFingerprint, + type DerivesFromEdge, + type ObserveEdgeInput, +} from "../curation/edges.js"; import type { TensionInput } from "../curation/tension.js"; import type { LlmClient } from "../eval/llm.js"; import { err, ok, type Result } from "../frontmatter/types.js"; @@ -338,6 +342,15 @@ export async function birthOne( axis: "prompt", premiseVote: "to", note: `birth: ${reason}`, + fp: { + inputs: computeInputsFingerprint([ + { path: docPath, text: docContent }, + { path: neighbor, text: neighborContent }, + ]), + principal: CONSOLIDATE_AGENT, + model: opts.model, + prompt: "birth/foundational", + }, }); if (!obs.ok) { verdicts.push({ neighbor, error: `observe failed: ${obs.error.message}` }); @@ -381,6 +394,15 @@ export async function birthOne( axis: "prompt", premiseVote: "symmetric", note: `birth/symmetric(${which}): ${reason}`, + fp: { + inputs: computeInputsFingerprint([ + { path: docPath, text: docContent }, + { path: neighbor, text: neighborContent }, + ]), + principal: CONSOLIDATE_AGENT, + model: opts.model, + prompt: "birth/foundational", + }, }); if (!obs.ok) { verdicts.push({ neighbor, error: `observe failed: ${obs.error.message}` }); diff --git a/src/consolidate/edge-write.ts b/src/consolidate/edge-write.ts index 5bb2ee74..cd634881 100644 --- a/src/consolidate/edge-write.ts +++ b/src/consolidate/edge-write.ts @@ -43,6 +43,8 @@ function stubEdge( toPath, strength: 0, kSurvived: 0, + kEff: 0, + strengthIndependent: 0, firstObserved: at, lastRederived: at, status, diff --git a/src/consolidate/independence.ts b/src/consolidate/independence.ts new file mode 100644 index 00000000..d8e7a7a6 --- /dev/null +++ b/src/consolidate/independence.ts @@ -0,0 +1,190 @@ +// Independence-aware promotion — the would-be verdict, the shadow journal, +// and the needs-review tension body (2026-07-26 spec, Decisions 3-4). +// +// This module owns the SHADOW calibration surface only: `independenceVerdict` +// is pure math over evidence-class counts, `appendIndependenceShadow` / +// `listIndependenceShadow` manage the module's own per-collapse journal +// (`.daftari/independence-shadow.jsonl` — distinct from `shadow-actions.jsonl`, +// which coverage.ts already filters by action kind), and +// `needsReviewTensionInput` renders the class breakdown that surfaces to a +// human via the tension log — the needs-review outcome is NOT a staged +// action (ratifying one would dispatch nothing; see spec Decision 3). +// +// Wiring (src/consolidate/revision.ts, src/consolidate/index.ts) is a +// separate concern: this module has no I/O beyond its own journal file and no +// knowledge of the envelope, the panel loop, or config. + +import { appendFileSync, mkdirSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { + EDGE_INDEPENDENCE_RHO, + EDGE_NEEDS_REVIEW_MIN_GAIN, + FP_SENTINEL, +} from "../curation/edges.js"; +import type { TensionInput } from "../curation/tension.js"; +import { err, ok, type Result } from "../frontmatter/types.js"; +import { CONSOLIDATE_AGENT } from "./constants.js"; + +export function independenceShadowPath(vaultRoot: string): string { + return join(vaultRoot, ".daftari", "independence-shadow.jsonl"); +} + +// One row per revision panel, regardless of decision (including fails / tie / +// no-vote / gated — `wouldDecision: null` for those) so the calibration +// denominator (informative-panel rate, C5) is honest about every panel that +// ran, not just the ones that reached a verdict. `classes` is the PRE-panel +// class state (raw keys — this is a machine surface, unlike the tension +// body's human-readable rendering). +export interface IndependenceShadowRow { + at: string; + fromPath: string; + toPath: string; + kSurvived: number; + kEff: number; + strength: number; + strengthIndependent: number; + classes: Array<{ key: string; count: number }>; + panelClassKeys: string[]; + marginalGain: number; + wouldDecision: "would_accrue" | "would_needs_review" | null; +} + +// Pure: applies the surviving votes' class keys sequentially against a copy +// of `preClasses` — the j-th occurrence within a class adds +// EDGE_INDEPENDENCE_RHO ** (occurrences already counted, pre- or within-panel) +// — and sums the marginal k_eff gain. `would_needs_review` iff that gain is +// STRICTLY below EDGE_NEEDS_REVIEW_MIN_GAIN (the boundary — a second vote in +// a count-1 class, gain exactly 0.5 — accrues). +export function independenceVerdict( + preClasses: Map, + survivingClassKeys: string[], +): { marginalGain: number; wouldDecision: "would_accrue" | "would_needs_review" } { + const working = new Map(preClasses); + let marginalGain = 0; + for (const key of survivingClassKeys) { + const priorCount = working.get(key) ?? 0; + marginalGain += EDGE_INDEPENDENCE_RHO ** priorCount; + working.set(key, priorCount + 1); + } + const wouldDecision = + marginalGain < EDGE_NEEDS_REVIEW_MIN_GAIN ? "would_needs_review" : "would_accrue"; + return { marginalGain, wouldDecision }; +} + +export async function appendIndependenceShadow( + vaultRoot: string, + row: IndependenceShadowRow, +): Promise> { + try { + mkdirSync(join(vaultRoot, ".daftari"), { recursive: true }); + appendFileSync(independenceShadowPath(vaultRoot), `${JSON.stringify(row)}\n`); + return ok(undefined); + } catch (e) { + return err( + new Error( + `cannot record independence shadow row: ${e instanceof Error ? e.message : String(e)}`, + ), + ); + } +} + +export async function listIndependenceShadow( + vaultRoot: string, +): Promise> { + let raw: string; + try { + raw = readFileSync(independenceShadowPath(vaultRoot), "utf-8"); + } catch (e) { + if ((e as NodeJS.ErrnoException).code === "ENOENT") return ok([]); + return err( + new Error( + `cannot read independence shadow log: ${e instanceof Error ? e.message : String(e)}`, + ), + ); + } + const rows: IndependenceShadowRow[] = []; + for (const line of raw.split("\n")) { + const trimmed = line.trim(); + if (!trimmed) continue; + try { + const rec = JSON.parse(trimmed) as IndependenceShadowRow; + if ( + typeof rec.at === "string" && + typeof rec.fromPath === "string" && + typeof rec.toPath === "string" + ) { + rows.push(rec); + } + } catch { + // Skip a corrupt line; the log is append-only and best-effort. + } + } + return ok(rows); +} + +// --- the needs-review tension body (C6) -------------------------------- + +// Decodes one `evidenceClassKey` output (`${inputs}\n${principal}\n${model}`, +// src/curation/edges.ts) back into its three components, `∅` → null. +function decodeClassKey(key: string): { + inputs: string | null; + principal: string | null; + model: string | null; +} { + const [inputs, principal, model] = key.split("\n"); + return { + inputs: inputs === FP_SENTINEL || inputs === undefined ? null : inputs, + principal: principal === FP_SENTINEL || principal === undefined ? null : principal, + model: model === FP_SENTINEL || model === undefined ? null : model, + }; +} + +// Structured class descriptor for the tension body — one per equivalence +// class the edge's CURRENT trail carries, sorted by key for a deterministic, +// reviewable rendering. +export function classesForTension( + classCounts: Map, +): Array<{ inputs: string | null; principal: string | null; model: string | null; count: number }> { + return [...classCounts.entries()] + .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)) + .map(([key, count]) => ({ ...decodeClassKey(key), count })); +} + +// Renders the needs-review interpretive tension (Decision 3). `classes` is a +// STRUCTURED breakdown (never raw `\n`-joined keys, which would break +// tensions.md's line-oriented round-trip — C6) — build it with +// `classesForTension` over the edge's current evidence classes. `claimA` is +// guaranteed single-line: `N classes over M counted votes — class 1 ×c₁: +// model=, principal=, inputs=<12-hex prefix>; …`, with `∅` +// components rendered `unfingerprinted`. +export function needsReviewTensionInput( + fromPath: string, + toPath: string, + classes: Array<{ + inputs: string | null; + principal: string | null; + model: string | null; + count: number; + }>, +): TensionInput { + const totalVotes = classes.reduce((n, c) => n + c.count, 0); + const parts = classes.map((c, i) => { + const model = c.model ?? "unfingerprinted"; + const principal = c.principal ?? "unfingerprinted"; + const inputs = c.inputs ? c.inputs.slice(0, 12) : "unfingerprinted"; + return `class ${i + 1} ×${c.count}: model=${model}, principal=${principal}, inputs=${inputs}`; + }); + const claimA = `${classes.length} classes over ${totalVotes} counted votes — ${parts.join("; ")}`; + return { + title: `correlated-only survival: ${fromPath} derives_from ${toPath}`, + kind: "interpretive", + sourceA: fromPath, + claimA, + sourceB: toPath, + claimB: + "survives re-derivation only on already-counted evidence; supply an independent " + + "re-derivation (vault_edge_observe with a fresh fingerprint) or contest the edge, " + + "then resolve", + loggedBy: CONSOLIDATE_AGENT, + }; +} diff --git a/src/consolidate/index.ts b/src/consolidate/index.ts index dcd262ad..13a03783 100644 --- a/src/consolidate/index.ts +++ b/src/consolidate/index.ts @@ -16,7 +16,7 @@ import { existsSync } from "node:fs"; import { posix, resolve } from "node:path"; -import { type DerivesFromEdge, listEdges } from "../curation/edges.js"; +import { type DerivesFromEdge, edgeEvidenceClasses, listEdges } from "../curation/edges.js"; import { addTension, listTensions } from "../curation/tension.js"; import { loadDocuments } from "../curation/vault-docs.js"; import { createAnthropicClient, type LlmClient } from "../eval/llm.js"; @@ -50,6 +50,7 @@ import { } from "./constants.js"; import { formatDecorrelationReport, loadFixture, runDecorrelation } from "./decorrelation.js"; import { makeContest, makeObserve } from "./edge-write.js"; +import { appendIndependenceShadow } from "./independence.js"; import { prioritize } from "./priority.js"; import { appendRevisionTrace, @@ -417,6 +418,7 @@ export async function runConsolidate(argv: string[]): Promise { vaultRoot, model, stage2, + cfg.value.independenceGraduated, ); } @@ -433,12 +435,25 @@ export async function runConsolidate(argv: string[]): Promise { report += ` est_cost_usd: ${estimateCostUSD(model, stage2.inputTokens, stage2.outputTokens).toFixed(4)} (${model})${ isModelPriced(model) ? "" : " [pricing_fallback: haiku — model unpriced]" }\n`; + // Decision 5's per-session burden honesty: needs_review_emitted counts + // would-be emissions while shadowed (independenceGraduated: false, the + // shipped default) and real emissions once graduated — the suffix + // makes which is which explicit in the report. + report += ` needs_review_emitted: ${stage2.needsReviewEmitted}${ + cfg.value.independenceGraduated ? "" : " (shadowed — would-be)" + }\n`; + if (cfg.value.independenceGraduated) { + report += ` panels_skipped_needs_review: ${stage2.panelsSkippedNeedsReview}\n`; + } if (stage2.traceWriteFailures > 0) { report += ` trace_write_failures: ${stage2.traceWriteFailures} — recall@K evaluator input lost (exit 5)\n`; } if (stage2.journalWriteFailures > 0) { report += ` journal_write_failures: ${stage2.journalWriteFailures} — calibration rows lost\n`; } + if (stage2.independenceJournalWriteFailures > 0) { + report += ` independence_journal_write_failures: ${stage2.independenceJournalWriteFailures} — calibration rows lost\n`; + } // The envelope journals every decision (admit/gate) to shadow-actions.jsonl // regardless of shadow mode (makeAdmit owns this). In shadow mode the edge // STORE write is additionally suppressed (makeObserve); off-shadow, admitted @@ -518,6 +533,17 @@ interface Stage2Result { // returned err). Counted, not thrown — surfaced in the report, never gates. journalWriteFailures: number; shadowMode: boolean; + // Independence-aware promotion (Decision 5): would-be emissions while + // shadowed, real emissions once graduated — see the report suffix. + needsReviewEmitted: number; + // Due edges skipped pre-panel because an open needs-review tension already + // parks them (C1) — zero LLM calls, no trace row. Only nonzero when + // independenceGraduated is true. + panelsSkippedNeedsReview: number; + // Independence shadow journal rows the panel couldn't write (a failed + // pre-panel classes read, or a failed journal append). Counted, not + // thrown — mirrors journalWriteFailures. + independenceJournalWriteFailures: number; } function emptyStage2(): Stage2Result { return { @@ -533,6 +559,9 @@ function emptyStage2(): Stage2Result { traceWriteFailures: 0, journalWriteFailures: 0, shadowMode: false, + needsReviewEmitted: 0, + panelsSkippedNeedsReview: 0, + independenceJournalWriteFailures: 0, }; } @@ -628,6 +657,7 @@ async function runRevisionLoop( vaultRoot: string, model: string, stage2: Stage2Result, + independenceGraduated: boolean, ): Promise { const loadDoc: RevisionDeps["loadDoc"] = async (path) => { const d = docByPath.get(canon(path)); @@ -638,6 +668,20 @@ async function runRevisionLoop( }; const recordRevisionTrace: RevisionDeps["recordRevisionTrace"] = (row) => appendRevisionTrace(vaultRoot, row); + const getEvidenceClasses: RevisionDeps["getEvidenceClasses"] = async (fromPath, toPath) => + edgeEvidenceClasses(vaultRoot, fromPath, toPath); + const recordIndependenceShadow: RevisionDeps["recordIndependenceShadow"] = (row) => + appendIndependenceShadow(vaultRoot, row); + // Needs-review tensions are deduped on open title (mirrors runBirthLoop's + // recordTension) — an edge that keeps failing the panel across sessions + // must not stack a fresh tension on top of an unresolved one. + const recordNeedsReviewTension: RevisionDeps["recordNeedsReviewTension"] = async (input) => { + const existing = await listTensions(vaultRoot); + if (existing.ok && existing.value.some((t) => t.title === input.title && !t.resolved)) { + return ok(undefined); + } + return addTension(vaultRoot, input); + }; const opts: RevisionOpts = { vaultRoot, @@ -645,9 +689,28 @@ async function runRevisionLoop( panelSize: CONSOLIDATE_PANEL_SIZE, budgetRemaining: Number.POSITIVE_INFINITY, model, + independenceGraduated, }; + + // Parking (C1): when graduated, a due edge with an OPEN needs-review + // tension is skipped entirely — zero LLM calls, no trace row. One + // listTensions read for the whole loop (reused by the dedup check above + // too would re-read; this is the loop-level read the parking check needs). + // No skip in shadowed mode: shadow data should stay complete (no tensions + // exist to park against anyway, since needs-review only emits real + // tensions once graduated). + let openNeedsReviewTitles: Set | null = null; + if (independenceGraduated) { + const tensionsRes = await listTensions(vaultRoot); + openNeedsReviewTitles = new Set( + tensionsRes.ok ? tensionsRes.value.filter((t) => !t.resolved).map((t) => t.title) : [], + ); + } + for (const item of edgeItems) { - const key = `${canon(item.fromPath)}\n${canon(item.toPath)}`; + const fromPath = canon(item.fromPath); + const toPath = canon(item.toPath); + const key = `${fromPath}\n${toPath}`; const edge = edgeByKey.get(key); if (!edge) continue; // Defensive: the clocks already exclude direction-symmetric edges from the @@ -656,9 +719,26 @@ async function runRevisionLoop( // so a future due-path (e.g. the deferred TTL clock) can't feed a pending // edge into a directional verdict. if (edge.directionVerdict === "symmetric") continue; + if (openNeedsReviewTitles) { + const title = `correlated-only survival: ${fromPath} derives_from ${toPath}`; + if (openNeedsReviewTitles.has(title)) { + stage2.panelsSkippedNeedsReview++; + continue; + } + } const out = await revisionPanel( edge, - { llm, loadDoc, admit, observe, contest, recordRevisionTrace }, + { + llm, + loadDoc, + admit, + observe, + contest, + recordRevisionTrace, + getEvidenceClasses, + recordIndependenceShadow, + recordNeedsReviewTension, + }, opts, ); if (!out.ok) { @@ -681,6 +761,8 @@ function accumulateRevision(stage2: Stage2Result, out: RevisionOutcome): void { stage2.inputTokens += out.inputTokens; stage2.outputTokens += out.outputTokens; if (!out.traceWritten) stage2.traceWriteFailures++; + if (out.independenceWouldNeedsReview) stage2.needsReviewEmitted++; + if (out.independenceJournalWriteFailure) stage2.independenceJournalWriteFailures++; } // --- --report=decorrelation --------------------------------------------------- diff --git a/src/consolidate/revision.ts b/src/consolidate/revision.ts index c3247354..f7ceb701 100644 --- a/src/consolidate/revision.ts +++ b/src/consolidate/revision.ts @@ -18,15 +18,30 @@ import { appendFileSync, mkdirSync } from "node:fs"; import { join, posix } from "node:path"; import { + agedStrength, type ContestEdgeInput, + computeInputsFingerprint, type DerivesFromEdge, EDGE_AXES, + effectiveK, + evidenceClassKey, type ObserveEdgeInput, } from "../curation/edges.js"; +import type { TensionInput } from "../curation/tension.js"; import type { LlmClient } from "../eval/llm.js"; import { err, ok, type Result } from "../frontmatter/types.js"; -import { CONSOLIDATE_PROMPT_TEMPLATES, type ConsolidatePromptTemplate } from "./constants.js"; +import { + CONSOLIDATE_AGENT, + CONSOLIDATE_PROMPT_TEMPLATES, + type ConsolidatePromptTemplate, +} from "./constants.js"; import type { Admit, EnvelopeVerdict } from "./envelope.js"; +import { + classesForTension, + type IndependenceShadowRow, + independenceVerdict, + needsReviewTensionInput, +} from "./independence.js"; // --- public surface ---------------------------------------------------------- @@ -42,6 +57,16 @@ export interface RevisionDeps { observe: (input: ObserveEdgeInput) => Promise>; contest: (input: ContestEdgeInput) => Promise>; recordRevisionTrace: (row: RevisionTraceRow) => Promise>; + // Independence-aware promotion (Decisions 3-4). The edge's CURRENT + // evidence-class counts, fetched once at panel start (before any observe + // lands). A failure degrades the panel to shadow-off (journal nothing, + // count one journal failure) — it must never change the live decision. + getEvidenceClasses: ( + fromPath: string, + toPath: string, + ) => Promise, Error>>; + recordIndependenceShadow: (row: IndependenceShadowRow) => Promise>; + recordNeedsReviewTension: (input: TensionInput) => Promise>; } export interface RevisionOpts { @@ -50,6 +75,13 @@ export interface RevisionOpts { panelSize: number; budgetRemaining: number; model: string; + // Decision 4 graduation gate — default false (shipped shadowed). When + // false, decision and writes are exactly today's two-way verdict; the + // independence verdict feeds only the shadow journal and the trace. When + // true, a correlated-only survives-majority panel becomes "needs-review" + // instead of "survives": no observes, no envelope consult, a tension + // instead. + independenceGraduated: boolean; } export interface RevisionVote { @@ -67,7 +99,10 @@ export interface RevisionVoteError { // no write — they surface for human attention instead of churning edge state. // "gated" means a majority WAS reached (survives/fails) but the envelope refused // the write — the vote stands in the trace, but nothing was applied. -export type RevisionDecision = "survives" | "fails" | "tie" | "no-vote" | "gated"; +// "needs-review" (Decision 3, graduated only): majority survives but every +// surviving vote landed in an already-present evidence class — no observes, +// a needs-review tension instead. +export type RevisionDecision = "survives" | "fails" | "tie" | "no-vote" | "gated" | "needs-review"; export interface RevisionTraceRow { at: string; @@ -87,6 +122,16 @@ export interface RevisionTraceRow { // EnvelopeVerdict). Absent otherwise. gate?: "invariants" | "budget" | null; gateReason?: string; + // Independence-aware promotion (Decision 3's "last clause": the recall@K + // evaluator and the calibration reads see the class breakdown here too). + // Present only when the panel reached the majority-survives, admitted + // branch AND the pre-panel class read succeeded. + independence?: { + kEff: number; + marginalGain: number; + classKeys: string[]; + wouldDecision: "would_accrue" | "would_needs_review" | null; + }; } export interface RevisionOutcome { @@ -109,6 +154,14 @@ export interface RevisionOutcome { outputTokens: number; traceWritten: boolean; traceError?: string; + // True iff the independence verdict computed to would_needs_review on this + // panel — the Decision-5 burden counter increments on this REGARDLESS of + // independenceGraduated (shadowed: would-be; graduated: the panel actually + // became "needs-review"). + independenceWouldNeedsReview: boolean; + // The pre-panel getEvidenceClasses read failed, OR the shadow journal write + // itself failed. Counted like journalWriteFailures — reported, never gates. + independenceJournalWriteFailure: boolean; } // --- canon ------------------------------------------------------------------- @@ -238,6 +291,24 @@ export async function revisionPanel( const toRes = await deps.loadDoc(toPath); if (!toRes.ok) return toRes; + // Evidence fingerprint (Decision 1): hashed over the EXACT truncated + // strings userBody places in the prompt, computed once per panel — every + // vote in the panel reads the same two endpoint texts, so they share one + // `inputs` hash regardless of which prompt template ran. + const inputsFingerprint = computeInputsFingerprint([ + { path: fromPath, text: truncate(fromRes.value.content) }, + { path: toPath, text: truncate(toRes.value.content) }, + ]); + + // Independence-aware promotion (Decisions 3-4): fetch the edge's CURRENT + // evidence classes BEFORE any observe lands this panel — alongside loadDoc, + // once per panel. A failed read degrades to shadow-off for this panel only + // (no journal row, no independence verdict); the live majority-decides + // path below never depends on this succeeding. + const preClassesRes = await deps.getEvidenceClasses(fromPath, toPath); + const preClasses = preClassesRes.ok ? preClassesRes.value : new Map(); + const classesAvailable = preClassesRes.ok; + const axes = axesForPanel(opts.panelSize); const votes: Array = []; const writeErrors: Array<{ axis: ConsolidatePromptTemplate; error: string }> = []; @@ -287,6 +358,15 @@ export async function revisionPanel( let contestedCount = 0; let gate: "invariants" | "budget" | null | undefined; let gateReason: string | undefined; + // Independence verdict (Decisions 3-4): set ONLY on the majority-survives, + // admitted branch — wouldDecision stays null for fails/tie/no-vote/gated + // panels, matching the spec's "the independence verdict only splits + // survives" scope. survivingClassKeysUsed mirrors it for the journal row. + let indVerdict: { + marginalGain: number; + wouldDecision: "would_accrue" | "would_needs_review"; + } | null = null; + let survivingClassKeysUsed: string[] = []; if (survivesCount === 0 && failsCount === 0) { decision = "no-vote"; // all errored / budget-starved — surface, write nothing @@ -325,23 +405,67 @@ export async function revisionPanel( gateReason = verdict.reason; surviving.length = 0; // ensure the loop below applies nothing } else { - decision = "survives"; - for (let i = 0; i < surviving.length; i++) { - const storeAxis = EDGE_AXES[i % EDGE_AXES.length]; - const obs = await deps.observe({ + // Independence verdict (Decision 3): every surviving vote in THIS + // panel shares one class key — (inputs, principal, model) are + // identical across the panel's votes (only fp.prompt, excluded from + // the key, varies by template) — so the panel's marginal k_eff gain + // against the edge's PRE-panel classes measures whether this panel is + // genuinely fresh evidence or a repeat of an already-present class. + const classKey = evidenceClassKey({ + inputs: inputsFingerprint, + principal: CONSOLIDATE_AGENT, + model: opts.model, + }); + survivingClassKeysUsed = surviving.map(() => classKey); + indVerdict = classesAvailable + ? independenceVerdict(preClasses, survivingClassKeysUsed) + : null; + + if (opts.independenceGraduated && indVerdict?.wouldDecision === "would_needs_review") { + // Correlated-only survival (graduated): no observes, no further + // envelope consult — the tension IS the surface (Decision 3). The + // one admit above already consulted the envelope for this panel + // decision; needs-review writes nothing to the edge store. + decision = "needs-review"; + const tensionInput = needsReviewTensionInput( fromPath, toPath, - observedBy: opts.agent, - blind: true, - axis: storeAxis, - note: `revision/${surviving[i].axis}: ${surviving[i].reason}`, - }); - if (obs.ok) observedCount++; - else + classesForTension(preClasses), + ); + const tensionRes = await deps.recordNeedsReviewTension(tensionInput); + if (!tensionRes.ok) { writeErrors.push({ - axis: surviving[i].axis, - error: `observe failed: ${obs.error.message}`, + axis: axes[0], + error: `needs-review tension failed: ${tensionRes.error.message}`, + }); + } + } else { + // Shipped default (independenceGraduated: false) OR a genuinely + // fresh-class panel: decision and writes are exactly today's. + decision = "survives"; + for (let i = 0; i < surviving.length; i++) { + const storeAxis = EDGE_AXES[i % EDGE_AXES.length]; + const obs = await deps.observe({ + fromPath, + toPath, + observedBy: opts.agent, + blind: true, + axis: storeAxis, + note: `revision/${surviving[i].axis}: ${surviving[i].reason}`, + fp: { + inputs: inputsFingerprint, + principal: CONSOLIDATE_AGENT, + model: opts.model, + prompt: `revision/${surviving[i].axis}`, + }, }); + if (obs.ok) observedCount++; + else + writeErrors.push({ + axis: surviving[i].axis, + error: `observe failed: ${obs.error.message}`, + }); + } } } } else { @@ -349,6 +473,33 @@ export async function revisionPanel( decision = "tie"; } + // Independence shadow journal (Decision 4): one row per panel regardless + // of decision, so the calibration denominator is honest. A failed + // pre-panel classes read degrades to "journal nothing, count one failure" + // — never the live decision above. + let independenceJournalWriteFailure = false; + if (classesAvailable) { + const kEffPre = effectiveK(preClasses.values()); + const now = new Date(); + const shadowRow: IndependenceShadowRow = { + at: now.toISOString().replace(/\.\d{3}Z$/, "Z"), + fromPath, + toPath, + kSurvived: edge.kSurvived, + kEff: kEffPre, + strength: edge.strength, + strengthIndependent: agedStrength(kEffPre, edge.lastRederived, now), + classes: [...preClasses.entries()].map(([key, count]) => ({ key, count })), + panelClassKeys: survivingClassKeysUsed, + marginalGain: indVerdict?.marginalGain ?? 0, + wouldDecision: indVerdict?.wouldDecision ?? null, + }; + const shadowRes = await deps.recordIndependenceShadow(shadowRow); + if (!shadowRes.ok) independenceJournalWriteFailure = true; + } else { + independenceJournalWriteFailure = true; + } + const traceRes = await deps.recordRevisionTrace({ at: new Date().toISOString().replace(/\.\d{3}Z$/, "Z"), fromPath, @@ -362,6 +513,16 @@ export async function revisionPanel( observedCount, contestedCount, ...(decision === "gated" ? { gate, gateReason } : {}), + ...(indVerdict + ? { + independence: { + kEff: effectiveK(preClasses.values()), + marginalGain: indVerdict.marginalGain, + classKeys: survivingClassKeysUsed, + wouldDecision: indVerdict.wouldDecision, + }, + } + : {}), }); return ok({ @@ -378,6 +539,8 @@ export async function revisionPanel( outputTokens, traceWritten: traceRes.ok, ...(traceRes.ok ? {} : { traceError: traceRes.error.message }), + independenceWouldNeedsReview: indVerdict?.wouldDecision === "would_needs_review", + independenceJournalWriteFailure, }); } diff --git a/src/context/assemble.ts b/src/context/assemble.ts new file mode 100644 index 00000000..389fe35e --- /dev/null +++ b/src/context/assemble.ts @@ -0,0 +1,274 @@ +// Context-pack assembly (spec 2026-07-26-context-packs-progressive- +// disclosure-design.md, Decision 2 / Decision 3, final-plan Phase 2.1-2.4). +// +// Pure, deterministic selection + templating over an already-enriched, +// already-RBAC-filtered PackEntry[]. This module knows nothing about the +// index, RBAC, or the ranker — src/tools/context.ts owns retrieval, RBAC +// filtering, supersession dedup, and head-keyed enrichment (C3); this module +// only sorts, greedily cuts to budget, and renders markdown. That split is +// what makes assemble.test.ts able to prove determinism (same PackEntry[] in +// twice ⇒ byte-identical brief out) without a database, a fixture vault, or a +// golden-file pin on retrieval-dependent snippet content (C6). +// +// Decision 3's refusal is enforced HERE, structurally, not by convention: a +// tension flag always renders BOTH claims from `claimSelf`/`claimOther` — a +// blended sentence is not a shape this renderer can produce. Supersession +// prints only the pointer and hop count; the chain head's own snippet is +// what carries content, and it arrives already-resolved on the entry (the +// tool handler set `snippet` to `currentSource.snippet` for a collapsed +// chain). No field here is prose daftari composed *about* the truth of vault +// content — every flag line is a direct rendering of an index fact. + +import type { HiddenDownstream } from "../curation/tension-blast.js"; +import { estimateTokens } from "./estimate.js"; + +// 10% headroom on top of the chars/4 estimate (spec §5): a brief cut at +// budget * 0.9 estimated tokens stays inside the caller's stated budget even +// when the estimator's ~±15% error runs hot. +export const BUDGET_HEADROOM = 0.9; + +export interface PackTensionFlag { + kind: string; + counterpart: string; // vault-relative path of the other side + claimSelf: string; + claimOther: string; +} + +export interface PackDecayFlag { + level: "deprecated" | "warn" | "aging"; + banner: string | null; +} + +export interface PackStructuralFlag { + orphan: boolean; + deprecatedStillLinked: boolean; +} + +export interface PackUpstreamFlag { + pendingBrokenUpstream?: "some" | "many"; + hiddenPendingUpstream?: "some" | "many"; +} + +export interface PackProvenanceFlag { + updatedBy: string; + updated: string; +} + +// One candidate document, already enriched and RBAC-filtered by the tool +// handler. Every flag field below is keyed on THIS entry's own `path` — for +// a collapsed supersession chain, that is the HEAD's path, never the stale +// member's (C3: "all flags describe the entry's path, no exceptions"). A +// stale member that collapsed into a head contributes exactly three things +// to the head entry: `score` (max over the collapsed members), `supersedes` +// (the collapsed count), and `reason` — never its own flags. +export interface PackEntry { + path: string; + title: string; + score: number; + reason: string; + snippet: string; + // Present only on a chain-head entry: the count of stale members collapsed + // into it. + supersedes?: number; + // Present only on a stale hit whose current-source chain hit an unreadable + // hop — the path-free marker (2026-07-14 spec, RBAC omission). Mutually + // exclusive with `supersedes` in practice (a restricted chain never + // resolves to a head), but the renderer does not assume that. + currentSourceRestricted?: boolean; + // Present only on a stale hit whose supersession chain could not be + // followed to a head at all — a broken `superseded_by` pointer or a cycle. + // Kept as itself (never collapsed), same as the restricted case. + supersessionIssue?: "dangling" | "cycle"; + tensions?: PackTensionFlag[]; // open only, capped by the caller (CONTESTED_CAP) + contestedCount?: number; // true total; may exceed tensions.length + decay?: PackDecayFlag | null; + structural?: PackStructuralFlag | null; + upstream?: PackUpstreamFlag; + provenance?: PackProvenanceFlag; +} + +export interface ContextPackManifestEntry { + path: string; + score: number; + reason: string; +} + +export interface ContextPackManifest { + included: ContextPackManifestEntry[]; + omitted_over_budget: number; + // Lower-bound signal over OBSERVABLE withholding, never a completeness + // claim (C4) — see src/tools/context.ts for how the caller computes this. + hidden_remainder: HiddenDownstream; +} + +export interface ContextPack { + task: string; + budget: number; + estimatedTokens: number; + brief: string; + manifest: ContextPackManifest; +} + +function pluralize(n: number, word: string): string { + return `${n} ${word}${n === 1 ? "" : "s"}`; +} + +function renderHeader(task: string, n: number, bodyTokens: number, budget: number): string { + return ( + `# Context brief: ${task}\n\n` + + `_${pluralize(n, "document")}, ~${bodyTokens} tokens (budget ${budget}). ` + + "Selected, not synthesized — drill in with vault_read._" + ); +} + +// Decision 3, structurally enforced: a tension always renders both claims — +// there is no code path here that could merge them into one sentence. +function renderFlagLines(entry: PackEntry): string[] { + const lines: string[] = []; + if (entry.supersedes !== undefined) { + lines.push(`- supersedes ${pluralize(entry.supersedes, "older document")} matching this task`); + } + if (entry.currentSourceRestricted) { + lines.push("- current source: restricted"); + } + if (entry.supersessionIssue === "dangling") { + lines.push("- current source: chain broken (points at a document that no longer exists)"); + } + if (entry.supersessionIssue === "cycle") { + lines.push("- current source: chain forms a cycle"); + } + if (entry.tensions && entry.tensions.length > 0) { + for (const t of entry.tensions) { + lines.push( + `- contested (${t.kind}, open): this doc claims "${t.claimSelf}"; ` + + `${t.counterpart} claims "${t.claimOther}"`, + ); + } + const shown = entry.tensions.length; + const total = entry.contestedCount ?? shown; + if (total > shown) lines.push(`- +${total - shown} more unresolved tension(s)`); + } + if (entry.decay) { + const suffix = entry.decay.banner ? ` — ${entry.decay.banner}` : ""; + lines.push(`- decay: ${entry.decay.level}${suffix}`); + } + if (entry.structural?.orphan) { + lines.push("- structural: orphan — no readable document links here"); + } + if (entry.structural?.deprecatedStillLinked) { + lines.push("- structural: deprecated but still linked from a canonical document"); + } + if (entry.upstream?.pendingBrokenUpstream) { + lines.push( + `- upstream: ${entry.upstream.pendingBrokenUpstream} pending-broken compiled input(s)`, + ); + } + if (entry.upstream?.hiddenPendingUpstream) { + lines.push( + `- upstream: ${entry.upstream.hiddenPendingUpstream} pending change(s) outside your read scope`, + ); + } + if (entry.provenance) { + lines.push( + `- updated ${entry.provenance.updated} by ${entry.provenance.updatedBy || "unknown"}`, + ); + } + return lines; +} + +function renderEntry(entry: PackEntry): string { + const parts = [ + `### ${entry.title}`, + `\`${entry.path}\` — score ${entry.score.toFixed(3)}`, + "", + entry.snippet, + ]; + const flagLines = renderFlagLines(entry); + if (flagLines.length > 0) { + parts.push("", ...flagLines); + } + return parts.join("\n"); +} + +function renderFooter(omitted: number, hiddenRemainder: HiddenDownstream): string { + const lines: string[] = []; + if (omitted > 0) { + lines.push(`_${pluralize(omitted, "more document")} omitted over budget._`); + } + // Absent-is-healthy (spec §"no LLM call" / Decision 3): the scope line + // prints only when there is something to disclose, never affirming + // completeness (C4). + if (hiddenRemainder !== "none") { + lines.push(`_${hiddenRemainder} additional document(s) withheld outside your read scope._`); + } + return lines.length > 0 ? `\n\n${lines.join("\n")}` : ""; +} + +// Assembles a ContextPack from an unsorted candidate pool. Deterministic: +// calling this twice on the same `entries` array produces byte-identical +// output (no Date.now, no randomness, no I/O) — the property assemble.test.ts +// pins directly, instead of a golden-fixture brief (C6). +// +// Selection: sort by score desc (path asc tie-break, matching the ranker's +// own tie-break), then append greedily while the CANDIDATE brief (header + +// body-so-far + the next entry) stays within budget * BUDGET_HEADROOM +// estimated tokens. The first entry that does not fit stops the walk — +// no skip-ahead, so `included` is always a PREFIX of the score-sorted pool +// (final-plan 2.2 step 7). The footer (omitted count, hidden-remainder scope +// line) is appended unconditionally afterward — it is disclosure, never +// budget-gated, so a caller always learns what was left out even when +// nothing fits (C9). +export function assembleContextPack( + task: string, + budget: number, + entries: PackEntry[], + hiddenRemainder: HiddenDownstream, +): ContextPack { + const sorted = [...entries].sort((a, b) => b.score - a.score || a.path.localeCompare(b.path)); + + const included: PackEntry[] = []; + const bodyBlocks: string[] = []; + for (const entry of sorted) { + const candidateBody = [...bodyBlocks, renderEntry(entry)].join("\n\n"); + // The header's own "~N tokens" line reports the CANDIDATE body's size — + // reported metadata, not itself a byte-exact constraint; the 10% + // headroom is precisely what absorbs a chars/4 estimate describing + // itself (see the module comment and estimate.ts). + const candidateHeader = renderHeader( + task, + included.length + 1, + estimateTokens(candidateBody), + budget, + ); + const candidateBrief = `${candidateHeader}\n\n${candidateBody}`; + if (estimateTokens(candidateBrief) <= budget * BUDGET_HEADROOM) { + included.push(entry); + bodyBlocks.push(renderEntry(entry)); + } else { + break; // stop at the first entry that does not fit — no skip-ahead + } + } + + const omitted = sorted.length - included.length; + const body = + sorted.length === 0 + ? "_No matching documents._" + : included.length === 0 + ? "_Nothing fit the requested budget — raise `budget` to include results._" + : bodyBlocks.join("\n\n"); + const header = renderHeader(task, included.length, estimateTokens(body), budget); + const footer = renderFooter(omitted, hiddenRemainder); + const brief = `${header}\n\n${body}${footer}`; + + return { + task, + budget, + estimatedTokens: estimateTokens(brief), + brief, + manifest: { + included: included.map((e) => ({ path: e.path, score: e.score, reason: e.reason })), + omitted_over_budget: omitted, + hidden_remainder: hiddenRemainder, + }, + }; +} diff --git a/src/context/estimate.ts b/src/context/estimate.ts new file mode 100644 index 00000000..7330312f --- /dev/null +++ b/src/context/estimate.ts @@ -0,0 +1,14 @@ +// Token estimation for context-pack budgeting (spec 2026-07-26-context-packs- +// progressive-disclosure-design.md, Decision 2 §5). chars/4, no tokenizer: a +// real tokenizer pins daftari to one model's vocabulary and adds a native or +// WASM dependency to every install, to gain precision this use case does not +// need — a brief cut at 3,900 estimated tokens vs. 4,100 real is not a +// failure mode. Deterministic and model-agnostic; within ~±15% on English +// markdown, which is why the assembler reserves 10% headroom on top of it +// (src/context/assemble.ts's BUDGET_HEADROOM). One function, so swapping in a +// real tokenizer later (should a future embedding provider ship one anyway) +// is a one-line change at every call site. + +export function estimateTokens(text: string): number { + return Math.ceil(text.length / 4); +} diff --git a/src/curation/edges.ts b/src/curation/edges.ts index 519986b7..b75e5fab 100644 --- a/src/curation/edges.ts +++ b/src/curation/edges.ts @@ -46,10 +46,11 @@ // one critical section with no intervening await. The guarantee is // per-process, which suffices under the one-daftari-per-vault process lock. +import { createHash } from "node:crypto"; import { appendFileSync, mkdirSync, readFileSync, statSync } from "node:fs"; import { join } from "node:path"; import { err, ok, type Result } from "../frontmatter/types.js"; -import { getProvider } from "../search/vector.js"; +import { getProvider, getQuantize } from "../search/vector.js"; import { clearDerivesFromEdges, type DerivesFromEdgeRow, @@ -95,6 +96,90 @@ export const EDGE_REPLAY_GAP_DAYS = 1; export const EDGE_AXES = ["prompt", "input-neighborhood", "model"] as const; export type EdgeAxis = (typeof EDGE_AXES)[number]; +// --- independence-aware promotion calibration constants (2026-07-26 spec) -- +// +// Same posture as EDGE_K_CAP above: PROVISIONAL, exported so the shadow +// calibration reads (src/consolidate/independence.ts, +// src/curation/independence-calibration.ts) and the revision loop share the +// exact values the store uses. + +// Correlation discount applied to the j-th vote within an evidence +// equivalence class (Decision 2). k_eff = Σ_classes Σ_{j=1..n} ρ^(j−1). +export const EDGE_INDEPENDENCE_RHO = 0.5; + +// Marginal k_eff gain floor a panel's surviving votes must clear to accrue +// (Decision 3, spec amendment 2026-07-26 PR-2.5): survives-independent iff +// the surviving votes' marginal gain is >= this value; correlated-only +// survival (needs-review) iff strictly below it. A second vote landing in an +// already-count-1 class gains exactly EDGE_INDEPENDENCE_RHO ** 1 = 0.5 — "one +// half-fresh vote" — and so accrues (the boundary is inclusive of accrual). +export const EDGE_NEEDS_REVIEW_MIN_GAIN = 0.5; + +// Sentinel for an absent fingerprint component. Matches only itself: an +// all-legacy (no-fp) trail collapses into a single class, never accidentally +// split from a genuinely fingerprinted one. Exported so consumers that decode +// a class key (e.g. the needs-review tension body, src/consolidate/ +// independence.ts) share this exact literal instead of re-hardcoding it. +export const FP_SENTINEL = "∅"; + +// The loop's authenticated principal — mirrors CONSOLIDATE_AGENT +// (src/consolidate/constants.ts), duplicated here as a literal rather than +// imported: consolidate/ imports curation/ (never the reverse), so importing +// it here would create a module cycle. A cross-check test +// (test/curation/independence-calibration.test.ts) asserts the two stay +// equal. +const LOOP_PRINCIPAL = "agent:curation-loop"; + +// One vote's evidence fingerprint (spec Decision 1). All components are +// optional — an absent component reads as the sentinel class. `prompt` +// travels with the record but is deliberately EXCLUDED from the class key +// (Decision 2: the v1 decorrelation verdict measured prompt-framing lift at +// ~0, so prompt variation alone never buys a fresh class). +export interface EdgeFingerprint { + inputs?: string; + principal?: string; + model?: string; + prompt?: string; +} + +// sha256 hex over the sorted `${path}\0${sha256(text)}` lines, joined by +// "\n". Deterministic and independent of the caller's entry order — two +// votes that read the same (path, bytes) set hash identically. +export function computeInputsFingerprint(entries: Array<{ path: string; text: string }>): string { + const lines = entries + .map((e) => `${e.path}\0${createHash("sha256").update(e.text).digest("hex")}`) + .sort(); + return createHash("sha256").update(lines.join("\n")).digest("hex"); +} + +// The equivalence-class key for one fingerprint (Decision 2): two votes share +// a class iff they agree on ALL of (inputs, principal, model). `prompt` never +// participates. A missing component is the sentinel `∅`, and `∅` matches only +// `∅` — an all-legacy trail collapses to one class (conservative: votes that +// cannot demonstrate independence get no credit for it). +export function evidenceClassKey(fp: EdgeFingerprint | undefined): string { + return `${fp?.inputs ?? FP_SENTINEL}\n${fp?.principal ?? FP_SENTINEL}\n${fp?.model ?? FP_SENTINEL}`; +} + +// k_eff = Σ_classes Σ_{j=1..n} ρ^(j−1) — geometric discount within each class +// (repeated votes in one class are worth geometrically less), full credit +// across classes. Pure; shared by the store, the revision verdict +// (independenceVerdict, src/consolidate/independence.ts), and the +// calibration reads. +export function effectiveK(classCounts: Iterable): number { + let total = 0; + for (const n of classCounts) { + let classSum = 0; + let term = 1; + for (let j = 0; j < n; j++) { + classSum += term; + term *= EDGE_INDEPENDENCE_RHO; + } + total += classSum; + } + return total; +} + export const EDGE_STATUSES = ["candidate", "trigger-bearing", "revoked"] as const; export type EdgeStatus = (typeof EDGE_STATUSES)[number]; @@ -120,6 +205,12 @@ export interface DerivesFromEdge { toPath: string; strength: number; kSurvived: number; + // Independence-aware promotion (Decision 1/2, shadow-only — CLAUDE.md: live + // strength/status keep using raw kSurvived). kEff discounts votes that land + // in an already-present evidence class; strengthIndependent is the aged + // value agedStrength would compute from kEff instead of kSurvived. + kEff: number; + strengthIndependent: number; firstObserved: string; lastRederived: string; status: EdgeStatus; @@ -142,6 +233,10 @@ export interface ObserveEdgeInput { // Which endpoint this observation judged the premise (foundational ordering). // Optional: legacy/unscored observes omit it and don't affect directionVerdict. premiseVote?: PremiseVote; + // Evidence fingerprint (Decision 1). Optional — a missing component (or a + // missing fp entirely) is the sentinel class `∅`. Each present component + // must be a non-empty string with no newline (the class-key separator). + fp?: EdgeFingerprint; // Test-only timestamp override for deterministic aging math. at?: string; } @@ -210,6 +305,46 @@ interface RawEdgeRecord { note?: string; reason?: string; premiseVote?: string; + // Read defensively (Decision 1): a non-string / newline-bearing component + // is treated as absent (∅), never thrown on. + fp?: { inputs?: unknown; principal?: unknown; model?: unknown; prompt?: unknown }; +} + +// A present fp component must be a non-empty string with no newline (the +// class-key line separator) — anything else reads as absent (∅). +function sanitizeFpComponent(v: unknown): string | undefined { + return typeof v === "string" && v.length > 0 && !v.includes("\n") ? v : undefined; +} + +function readFingerprint(raw: RawEdgeRecord["fp"]): EdgeFingerprint | undefined { + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return { + inputs: sanitizeFpComponent(raw.inputs), + principal: sanitizeFpComponent(raw.principal), + model: sanitizeFpComponent(raw.model), + prompt: sanitizeFpComponent(raw.prompt), + }; +} + +// Only defined components are serialized onto the JSONL record — old lines +// stay valid, the log stays append-only, no backfill. +function fpForWrite(fp: EdgeFingerprint): Record | undefined { + const out: Record = {}; + if (fp.inputs !== undefined) out.inputs = fp.inputs; + if (fp.principal !== undefined) out.principal = fp.principal; + if (fp.model !== undefined) out.model = fp.model; + if (fp.prompt !== undefined) out.prompt = fp.prompt; + return Object.keys(out).length > 0 ? out : undefined; +} + +function hasAnyFpComponent(fp: EdgeFingerprint | undefined): boolean { + return ( + fp !== undefined && + (fp.inputs !== undefined || + fp.principal !== undefined || + fp.model !== undefined || + fp.prompt !== undefined) + ); } function readRawRecords(vaultRoot: string): RawEdgeRecord[] { @@ -286,6 +421,17 @@ interface EdgeState { // verdict is derived from this set: unanimous (or empty) ⇒ directed; any // split, or an explicit symmetric ⇒ symmetric. premiseVotes: Set; + // Independence-aware promotion (Decision 1/2): evidence-class key → counted + // -vote count, accumulated ONLY for counted votes (never the seed). Reset + // on re-seed after a contest, like votedPairs. + classCounts: Map; + // Counted votes whose record carried no fp component at all — the + // legacy-∅ lint fraction (Decision 4). + unfingerprintedCountedVotes: number; + // Counted votes whose fp.principal is present and differs from the loop's + // own principal — how much of the class structure is operator-attested + // rather than loop-computed (C3). + nonLoopFingerprintedCountedVotes: number; } // Collapse the cycle's premise votes into a direction verdict (review C1): @@ -383,6 +529,9 @@ function collapse(records: RawEdgeRecord[]): Map { contestReason: null, votedPairs: new Set(seedPair), premiseVotes: new Set(canonVote ? [canonVote] : []), + classCounts: new Map(), + unfingerprintedCountedVotes: 0, + nonLoopFingerprintedCountedVotes: 0, }); continue; } @@ -405,6 +554,17 @@ function collapse(records: RawEdgeRecord[]): Map { // A counted vote at cap still refreshes the clock: it is a real // independent re-test even when k is saturated. existing.lastRederived = at; + + // Independence-aware promotion (Decision 1/2): register this counted + // vote's evidence class. Exactly this branch, matching kSurvived's + // own accrual — the seed observe above never reaches here. + const fp = readFingerprint(rec.fp); + const classKey = evidenceClassKey(fp); + existing.classCounts.set(classKey, (existing.classCounts.get(classKey) ?? 0) + 1); + if (!hasAnyFpComponent(fp)) existing.unfingerprintedCountedVotes += 1; + if (fp?.principal !== undefined && fp.principal !== LOOP_PRINCIPAL) { + existing.nonLoopFingerprintedCountedVotes += 1; + } } } // Non-qualifying and same-sitting-replayed observes move nothing — @@ -415,6 +575,11 @@ function collapse(records: RawEdgeRecord[]): Map { function deriveEdge(state: EdgeState, now: Date): DerivesFromEdge { const strength = state.revoked ? 0 : agedStrength(state.kSurvived, state.lastRederived, now); + // Independence-aware promotion (Decision 2/4, shadow-only): k_eff and its + // aged strength are computed alongside the live values but never gate + // status — status below still derives from raw `strength`. + const kEff = effectiveK(state.classCounts.values()); + const strengthIndependent = state.revoked ? 0 : agedStrength(kEff, state.lastRederived, now); const status: EdgeStatus = state.revoked ? "revoked" : strength >= EDGE_TRIGGER_STRENGTH @@ -445,6 +610,8 @@ function deriveEdge(state: EdgeState, now: Date): DerivesFromEdge { toPath, strength, kSurvived: state.kSurvived, + kEff, + strengthIndependent, firstObserved: state.firstObserved, lastRederived: state.lastRederived, status, @@ -472,7 +639,7 @@ function writeThroughEdgesIndex(vaultRoot: string): Map | nul // Stat BEFORE reading — same marker discipline as rebuildEdgesIndex. const marker = edgesLogStatMarker(vaultRoot); const states = collapse(readRawRecords(vaultRoot)); - const opened = openIndexDb(vaultRoot, getProvider().dim); + const opened = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (opened.ok) { const db = opened.value; try { @@ -518,7 +685,16 @@ export async function observeEdge( ) { return err(new Error(`observeEdge 'premiseVote' must be one of: ${PREMISE_VOTES.join(", ")}`)); } + if (input.fp !== undefined) { + for (const [key, v] of Object.entries(input.fp)) { + if (v === undefined) continue; + if (typeof v !== "string" || v.length === 0 || v.includes("\n")) { + return err(new Error(`observeEdge 'fp.${key}' must be a non-empty string with no newline`)); + } + } + } + const fpOut = input.fp ? fpForWrite(input.fp) : undefined; const record = { kind: "observe", from: input.fromPath.trim(), @@ -529,6 +705,7 @@ export async function observeEdge( axis: input.axis ?? null, ...(input.note ? { note: input.note } : {}), ...(input.premiseVote ? { premiseVote: input.premiseVote } : {}), + ...(fpOut ? { fp: fpOut } : {}), }; try { @@ -609,6 +786,7 @@ export interface ListEdgesFilter { function deriveEdgeFromRow(row: DerivesFromEdgeRow, now: Date): DerivesFromEdge { const revoked = row.status === "revoked"; const strength = revoked ? 0 : agedStrength(row.k_survived, row.last_rederived, now); + const strengthIndependent = revoked ? 0 : agedStrength(row.k_eff, row.last_rederived, now); const status: EdgeStatus = revoked ? "revoked" : strength >= EDGE_TRIGGER_STRENGTH @@ -619,6 +797,8 @@ function deriveEdgeFromRow(row: DerivesFromEdgeRow, now: Date): DerivesFromEdge toPath: row.to_path, strength, kSurvived: row.k_survived, + kEff: row.k_eff, + strengthIndependent, firstObserved: row.first_observed, lastRederived: row.last_rederived, status, @@ -673,7 +853,7 @@ export async function listEdges( filter: ListEdgesFilter = {}, now: Date = new Date(), ): Promise> { - const opened = openIndexDb(vaultRoot, getProvider().dim); + const opened = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (!opened.ok) return listEdgesFromLog(vaultRoot, filter, now); const db = opened.value; try { @@ -716,7 +896,7 @@ export async function getEdge( toPath: string, now: Date = new Date(), ): Promise> { - const opened = openIndexDb(vaultRoot, getProvider().dim); + const opened = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (!opened.ok) { // Same degraded posture as listEdgesFromLog: canonical store, no cache. try { @@ -785,6 +965,7 @@ function rebuildEdgesIndexFromStates( to_path: e.toPath, strength: e.strength, k_survived: e.kSurvived, + k_eff: e.kEff, first_observed: e.firstObserved, last_rederived: e.lastRederived, last_age_decay: at, @@ -811,7 +992,7 @@ function rebuildEdgesIndexFromStates( // derives_from_edges table, and closes. Startup path when no reindex is // otherwise running; the reindex path calls rebuildEdgesIndex directly. export function materializeEdges(vaultRoot: string): Result<{ count: number }, Error> { - const opened = openIndexDb(vaultRoot, getProvider().dim); + const opened = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (!opened.ok) return opened; const db = opened.value; try { @@ -820,3 +1001,80 @@ export function materializeEdges(vaultRoot: string): Result<{ count: number }, E db.close(); } } + +// --- independence-aware promotion: on-demand log collapse (Decision 1-4) ---- + +// The current cycle's evidence-class counts for one edge, collapsed directly +// from the canonical jsonl (no sqlite dependency — class detail is never +// materialized). Empty when the edge is absent or revoked. Consumers: the +// revision loop (src/consolidate/independence.ts) and the needs-review +// tension body. +export function edgeEvidenceClasses( + vaultRoot: string, + fromPath: string, + toPath: string, +): Result, Error> { + try { + const state = collapse(readRawRecords(vaultRoot)).get(edgeKey(...canonPair(fromPath, toPath))); + if (!state || state.revoked) return ok(new Map()); + return ok(new Map(state.classCounts)); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`cannot read edge evidence classes: ${reason}`)); + } +} + +// Per-edge calibration row for the lint surface (src/curation/independence- +// calibration.ts). Counts and aggregates only — class KEYS are not exported +// here, matching the vault-global-counts-only posture the lint section takes. +export interface EdgeIndependenceRow { + fromPath: string; + toPath: string; + kSurvived: number; + kEff: number; + strength: number; + strengthIndependent: number; + classCount: number; + countedVotes: number; + unfingerprintedCountedVotes: number; + nonLoopFingerprintedCountedVotes: number; + status: EdgeStatus; +} + +// One collapse pass over the whole log, live-derived per edge — the +// independenceCalibration lint section's source of truth. +export function independenceCalibrationView( + vaultRoot: string, + now: Date = new Date(), +): Result { + try { + const states = collapse(readRawRecords(vaultRoot)); + const rows: EdgeIndependenceRow[] = []; + for (const state of states.values()) { + const edge = deriveEdge(state, now); + let countedVotes = 0; + for (const n of state.classCounts.values()) countedVotes += n; + rows.push({ + fromPath: edge.fromPath, + toPath: edge.toPath, + kSurvived: edge.kSurvived, + kEff: edge.kEff, + strength: edge.strength, + strengthIndependent: edge.strengthIndependent, + classCount: state.classCounts.size, + countedVotes, + unfingerprintedCountedVotes: state.unfingerprintedCountedVotes, + nonLoopFingerprintedCountedVotes: state.nonLoopFingerprintedCountedVotes, + status: edge.status, + }); + } + return ok(rows); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`cannot compute independence calibration view: ${reason}`)); + } +} + +// Exported so a test can cross-check it against consolidate/constants.ts's +// CONSOLIDATE_AGENT without creating a curation→consolidate import. +export const EDGE_CALIBRATION_LOOP_PRINCIPAL = LOOP_PRINCIPAL; diff --git a/src/curation/independence-calibration.ts b/src/curation/independence-calibration.ts new file mode 100644 index 00000000..1ea0e08a --- /dev/null +++ b/src/curation/independence-calibration.ts @@ -0,0 +1,152 @@ +// Independence-aware promotion — the `vault_lint` calibration section +// (2026-07-26 spec, Decision 4). PURE: callers pass in the already-collapsed +// view and the already-read shadow journal; this module does no I/O. +// +// Cross-layer posture mirrors src/curation/coverage.ts: this is a curation- +// layer module that only TYPE-imports from src/consolidate/independence.js +// (erased at runtime, no runtime coupling) — it does not import consolidate/ +// code, and nothing in consolidate/ imports this module (guard test in +// test/curation/coverage.test.ts covers the sibling coverage.ts invariant; +// this module has no dormant-enact concern of its own to guard, but the +// type-only import keeps the layering the same shape). +// +// Counts and aggregates only — no paths (matches tensionHealth's posture +// under the 2026-07-14 edge-graph existence-disclosure rule). + +// Type-only: erased at runtime. +import type { IndependenceShadowRow } from "../consolidate/independence.js"; +import { EDGE_TRIGGER_STRENGTH, type EdgeIndependenceRow, FP_SENTINEL } from "./edges.js"; + +const ALL_SENTINEL_CLASS_KEY = `${FP_SENTINEL}\n${FP_SENTINEL}\n${FP_SENTINEL}`; + +function mean(values: number[]): number { + if (values.length === 0) return 0; + return values.reduce((a, b) => a + b, 0) / values.length; +} + +function median(values: number[]): number { + if (values.length === 0) return 0; + const sorted = [...values].sort((a, b) => a - b); + const mid = Math.floor(sorted.length / 2); + return sorted.length % 2 === 0 + ? ((sorted[mid - 1] as number) + (sorted[mid] as number)) / 2 + : (sorted[mid] as number); +} + +export interface IndependenceCalibrationSummary { + kVsKEff: { + edgesWithVotes: number; + meanK: number; + meanKEff: number; + medianKEff: number; + // Edges (with >=1 counted vote) whose kEff is strictly below kSurvived — + // the discount actually bit. + kEffBelowKCount: number; + }; + wouldDropBelowTrigger: { + // Edges currently trigger-bearing on raw strength whose strengthIndependent + // would drop below EDGE_TRIGGER_STRENGTH. + count: number; + // The subset of `count` that is all-legacy (every counted vote + // unfingerprinted) — the expected huge all-legacy compression, visibly + // separated from signal (C5). + legacyOnlyCount: number; + }; + wouldNeedsReviewRate: { + // would_needs_review rows / journal rows with a non-null wouldDecision. + rate: number; + needsReviewCount: number; + decidedCount: number; + // C5: the raw rate above is degenerate (~0) until legacy edges carry a + // fingerprinted class, because a fingerprinted class key can never equal + // the all-∅ sentinel — every legacy edge's first fingerprinted panel is + // would_accrue by construction. An "informative" panel is a decided row + // whose PRE-panel classes already include at least one non-∅ key — a + // panel whose verdict could possibly have been would_needs_review. + informativePanels: number; + informativeNeedsReviewCount: number; + rateInformative: number; + }; + // Edges whose counted votes are ALL unfingerprinted, over edges with >=1 + // counted vote — how much of the graph is un-fingerprinted and therefore + // single-class. + legacyUnfingerprintedFraction: number; + // C3: total operator-attested (non-loop-principal) counted votes across + // the view — how much of the class structure is attested, not verified. + nonLoopFingerprintedCountedVotes: number; +} + +export function independenceCalibrationSummaryOf( + view: EdgeIndependenceRow[], + journal: IndependenceShadowRow[], +): IndependenceCalibrationSummary { + const withVotes = view.filter((r) => r.countedVotes > 0); + + const kEffBelowKCount = withVotes.filter((r) => r.kEff < r.kSurvived).length; + + const wouldDropSet = view.filter( + (r) => r.strengthIndependent < EDGE_TRIGGER_STRENGTH && r.strength >= EDGE_TRIGGER_STRENGTH, + ); + const wouldDropLegacyOnly = wouldDropSet.filter( + (r) => r.countedVotes > 0 && r.unfingerprintedCountedVotes === r.countedVotes, + ); + + const decided = journal.filter((r) => r.wouldDecision !== null); + const needsReview = decided.filter((r) => r.wouldDecision === "would_needs_review"); + const informative = decided.filter((r) => + r.classes.some((c) => c.key !== ALL_SENTINEL_CLASS_KEY), + ); + const informativeNeedsReview = informative.filter( + (r) => r.wouldDecision === "would_needs_review", + ); + + const legacyOnlyEdges = withVotes.filter((r) => r.unfingerprintedCountedVotes === r.countedVotes); + + let nonLoopFingerprintedCountedVotes = 0; + for (const r of view) nonLoopFingerprintedCountedVotes += r.nonLoopFingerprintedCountedVotes; + + return { + kVsKEff: { + edgesWithVotes: withVotes.length, + meanK: mean(withVotes.map((r) => r.kSurvived)), + meanKEff: mean(withVotes.map((r) => r.kEff)), + medianKEff: median(withVotes.map((r) => r.kEff)), + kEffBelowKCount, + }, + wouldDropBelowTrigger: { + count: wouldDropSet.length, + legacyOnlyCount: wouldDropLegacyOnly.length, + }, + wouldNeedsReviewRate: { + rate: decided.length > 0 ? needsReview.length / decided.length : 0, + needsReviewCount: needsReview.length, + decidedCount: decided.length, + informativePanels: informative.length, + informativeNeedsReviewCount: informativeNeedsReview.length, + rateInformative: + informative.length > 0 ? informativeNeedsReview.length / informative.length : 0, + }, + legacyUnfingerprintedFraction: + withVotes.length > 0 ? legacyOnlyEdges.length / withVotes.length : 0, + nonLoopFingerprintedCountedVotes, + }; +} + +// Zero summary — for a fresh vault, or when either underlying read fails +// (lint stays advisory and never fails on a calibration read). +export function emptyIndependenceCalibrationSummary(): IndependenceCalibrationSummary { + return { + kVsKEff: { edgesWithVotes: 0, meanK: 0, meanKEff: 0, medianKEff: 0, kEffBelowKCount: 0 }, + wouldDropBelowTrigger: { count: 0, legacyOnlyCount: 0 }, + wouldNeedsReviewRate: { + rate: 0, + needsReviewCount: 0, + decidedCount: 0, + informativePanels: 0, + informativeNeedsReviewCount: 0, + rateInformative: 0, + }, + legacyUnfingerprintedFraction: 0, + nonLoopFingerprintedCountedVotes: 0, + }; +} diff --git a/src/curation/lint.ts b/src/curation/lint.ts index 00e31612..680f5055 100644 --- a/src/curation/lint.ts +++ b/src/curation/lint.ts @@ -7,19 +7,28 @@ // triage. The three tier-0 checks (#232, tier0.ts) are certain rather than // advisory judgments, but the posture is the same: report only. +import { classifyAgainstHash, resolveConfinedFile } from "../anchors/classify.js"; +import { looksLikeMalformedPin, type PinSpec, splitPin } from "../anchors/pin.js"; +import { parseDescribesEntry } from "../audit/describes.js"; +import { listIndependenceShadow } from "../consolidate/independence.js"; import { ok, type Result } from "../frontmatter/types.js"; +import { loadConfig } from "../utils/config.js"; +import { hashObjects } from "../utils/git.js"; import { type CoverageEquitySummary, coverageEquitySummary } from "./coverage.js"; import { DRAFT_MAX_DAYS, LOW_CONFIDENCE_MAX_DAYS } from "./decay.js"; -import { listEdges } from "./edges.js"; +import { independenceCalibrationView, listEdges } from "./edges.js"; +import { + emptyIndependenceCalibrationSummary, + type IndependenceCalibrationSummary, + independenceCalibrationSummaryOf, +} from "./independence-calibration.js"; import { readProvenanceLog } from "./provenance.js"; import { type ReviewThroughputSummary, reviewThroughputSummary } from "./review-throughput.js"; +import { type RankedStagedActionItem, rankPendingActions } from "./risk.js"; import { listShadowActions, type ShadowLintSummary, shadowLintSummaryOf } from "./shadow.js"; -import { - listStagedActions, - pendingLintItems, - type StagedActionLintItem, -} from "./staged-actions.js"; +import { listStagedActions } from "./staged-actions.js"; +export type { RankedStagedActionItem } from "./risk.js"; export type { StagedActionLintItem } from "./staged-actions.js"; import { ageInDays, computeStaleness } from "./staleness.js"; @@ -30,9 +39,15 @@ import { type ResolutionKind, STALE_TIER_LINT_COPY, TENSION_KINDS, + type TensionEntry, type TensionKind, } from "./tension.js"; -import { buildReverseLinkMap, buildReverseSourceMap, computeBlast } from "./tension-blast.js"; +import { + buildReverseLinkMap, + buildReverseSourceMap, + computeBlast, + type HiddenDownstream, +} from "./tension-blast.js"; import { computeTensionClusters } from "./tension-clusters.js"; import { tier0Findings } from "./tier0.js"; import { validityConflicts } from "./validity.js"; @@ -59,6 +74,10 @@ export const LINT_CHECKS = [ // Appended, not inserted: LINT_CHECKS order is presentation order, and new // checks go at the end so an existing reader's mental layout does not shift. "validityConflicts", + // 2026-07-26 citation-anchors-jit spec, Phase 8: a describes entry whose + // pin suffix is near-miss-malformed (tightened heuristic, C11). Pure + // string scan — no git work. + "malformedPins", ] as const; export type LintCheckName = (typeof LINT_CHECKS)[number]; @@ -107,6 +126,13 @@ export interface TensionAging { export const LARGE_CLUSTER_MIN_SIZE = 5; export const AGED_CLUSTER_MIN_DAYS = 90; +// 2026-07-26 citation-anchors-jit spec, Phase 8 / plan resolution C3: caps +// the number of step-3 (drift-path) pin classifications a single lint run +// performs for the Decision-4 softening pass. Beyond the budget, affected +// docs simply don't get the softened copy — the check is advisory +// copy-softening, so dropping it is the correct degradation, not an error. +export const LINT_PIN_STEP3_BUDGET = 200; + export interface TensionClustersHealth { count: number; maxSize: number; @@ -136,11 +162,20 @@ export interface LintReport { checks: Record; totalFindings: number; tensionHealth: TensionHealth; - // Pending staged actions awaiting ratification (spec §11.2), soonest-to- - // expire first. Empty when nothing is staged. Reported, not flagged — like - // the rest of vault_lint. The actual expiry sweep is a side effect of the - // vault_lint tool, not of runLint (which stays read-only). - stagedActions: StagedActionLintItem[]; + // Pending staged actions awaiting ratification (spec §11.2), risk descending + // with soonest-to-expire as the tiebreak (2026-07-26 risk-triaged-ratification + // spec, Decisions 1 + 2 — inverts the prior expiry-only sort). Empty when + // nothing is staged. Reported, not flagged — like the rest of vault_lint. + // The actual expiry sweep is a side effect of the vault_lint tool, not of + // runLint (which stays read-only). Filtered to the caller's vantage + // (Decision 4) — an item whose target the caller cannot read is omitted, + // never named. + stagedActions: RankedStagedActionItem[]; + // Coarsened count of pending actions omitted from `stagedActions` because + // their target is unreadable under the caller's vantage — none/some/many, + // never an exact count (Decision 4). "none" under an operator run (no + // pathVisible) or when nothing is hidden. + hiddenStagedActions: HiddenDownstream; // Shadow-mode summary (spec §11.5): how many writes were shadow-logged and // which would have been gated by the trust budget — the "Would-have-gated // actions" surface Decision 3's calibration reads. Zeroes when the vault has @@ -153,6 +188,19 @@ export interface LintReport { // vs. review throughput over the staged-actions log. Vault-global counts by // design, like tensionHealth — no paths or principals cross here. reviewThroughput: ReviewThroughputSummary; + // Independence-aware promotion shadow calibration (2026-07-26 spec, + // Decision 4): the k vs k_eff distribution, would-drop-below-trigger + // counts, the would-be needs-review rate, and the legacy-∅ fraction. + // Counts and aggregates only — no paths (matches tensionHealth's posture). + // Both underlying reads are error-tolerant: a failure yields the zero + // summary (lint stays advisory, never fails on a calibration read). + independenceCalibration: IndependenceCalibrationSummary; + // 2026-07-26 citation-anchors-jit spec, Phase 8: how many pins the + // Decision-4 softening pass classified via step 3 (git cat-file + a + // bounded text read) this run — the budget-spend counter that makes a + // slowdown attributable. 0 when jit_anchors is off, code_repos is empty, + // or no stale doc carries a pinned range binding. + pinsClassified: number; } export interface LintOptions { @@ -236,8 +284,23 @@ export async function runLint( schemaInvalid: [], domainLeaks: [], validityConflicts: [], + malformedPins: [], }; + // 2026-07-26 citation-anchors-jit spec, Phase 8: malformedPins is a pure + // string scan over every doc's describes entries — no git work, so it + // always runs regardless of jit_anchors/code_repos. + for (const d of docs) { + for (const raw of d.frontmatter.describes ?? []) { + if (looksLikeMalformedPin(raw)) { + checks.malformedPins.push({ + path: d.path, + detail: `malformed pin ignored: ${raw}`, + }); + } + } + } + // 12. Valid-time conflicts. The ONLY surface that reports a malformed or // contradictory interval: the schema layer deliberately declines to, because // `report.valid === false` is a hard write blocker and these fields are @@ -357,12 +420,137 @@ export async function runLint( }); } + // 2026-07-26 citation-anchors-jit spec, Decision 4 (lint side), batched + // and budgeted per the plan resolution (C1/C3): collect candidate pins + // across ALL stale docs first, dedupe the git work — one fs-confinement + // check per unique (repo, path), one hashObjects batch per repo for the + // WHOLE run — and memoise verdicts by full pin identity, so a triple + // recurring across docs is classified once. Soft-fails entirely: any + // config-load or git failure just means no docs get softened this run, + // never a lint failure (lint's own advisory posture). + let pinsClassified = 0; + const lintAnchorsConfig = loadConfig(vaultRoot); + if ( + lintAnchorsConfig.ok && + lintAnchorsConfig.value.jitAnchors && + Object.keys(lintAnchorsConfig.value.codeRepos).length > 0 + ) { + const codeRepos = lintAnchorsConfig.value.codeRepos; + const staleDocPaths = new Set(checks.staleFiles.map((f) => f.path)); + + type PinCandidate = { repo: string; path: string; pin: PinSpec }; + const byDoc = new Map(); + const pathsPerRepo = new Map>(); + + for (const d of docs) { + if (!staleDocPaths.has(d.path)) continue; + const cands: PinCandidate[] = []; + for (const raw of d.frontmatter.describes ?? []) { + const { binding, pin } = splitPin(raw); + if (!pin) continue; + const parsed = parseDescribesEntry(binding, ""); + if (parsed.repo === "" || !(parsed.repo in codeRepos)) continue; + cands.push({ repo: parsed.repo, path: parsed.path, pin }); + const set = pathsPerRepo.get(parsed.repo) ?? new Set(); + set.add(parsed.path); + pathsPerRepo.set(parsed.repo, set); + } + if (cands.length > 0) byDoc.set(d.path, cands); + } + + // Step 1 (confinement, cheap) + step 2 (ONE hashObjects batch per repo + // for the whole run) — independent of how many docs/pins reference the + // same (repo, path). + const currentHash = new Map(); // `${repo}${path}` -> blob id + const confinedOf = new Map(); + for (const [repo, paths] of pathsPerRepo) { + const repoAbsPath = codeRepos[repo] as string; + const survivors: string[] = []; + for (const p of paths) { + const confined = resolveConfinedFile(repoAbsPath, p); + if (confined) { + confinedOf.set(`${repo}${p}`, confined); + survivors.push(p); + } + } + if (survivors.length === 0) continue; + const relPaths = survivors.map( + (p) => (confinedOf.get(`${repo}${p}`) as { relPath: string }).relPath, + ); + const hashRes = await hashObjects(repoAbsPath, relPaths); + if (!hashRes.ok) continue; + survivors.forEach((p, i) => { + currentHash.set(`${repo}${p}`, hashRes.value[i] as string); + }); + } + + const verdictCache = new Map(); + for (const [docPath, cands] of byDoc) { + let allIntact = true; + let droppedForBudget = false; + for (const c of cands) { + const hashKey = `${c.repo}${c.path}`; + const hash = currentHash.get(hashKey); + if (hash === undefined) { + allIntact = false; // repo/path unresolved this run -> not softened + continue; + } + const verdictKey = `${hashKey}${c.pin.sha}${c.pin.start}${c.pin.end}`; + let state = verdictCache.get(verdictKey); + if (state === undefined) { + if (hash.startsWith(c.pin.sha)) { + state = "intact"; + } else if (c.pin.start === null || c.pin.end === null) { + state = "moved"; + } else if (pinsClassified >= LINT_PIN_STEP3_BUDGET) { + droppedForBudget = true; + continue; // over budget: leave uncached, doc goes unsoftened + } else { + pinsClassified += 1; + const confined = confinedOf.get(hashKey); + state = confined + ? ( + await classifyAgainstHash( + codeRepos[c.repo] as string, + confined.absPath, + c.pin, + hash, + ) + ).state + : "missing"; + } + verdictCache.set(verdictKey, state); + } + if (state !== "intact") allIntact = false; + } + if (droppedForBudget || !allIntact) continue; + + const idx = checks.staleFiles.findIndex((f) => f.path === docPath); + if (idx === -1) continue; + const n = cands.length; + const entry = checks.staleFiles[idx] as LintFinding; + checks.staleFiles[idx] = { + ...entry, + detail: + `${entry.detail}; past TTL, but its ${n} code pin${n === 1 ? "" : "s"} are intact — ` + + "the code it describes has not changed since the pins were written", + }; + } + } + const totalFindings = LINT_CHECKS.reduce((n, name) => n + checks[name].length, 0); + // Hoisted (Phase 5, 2026-07-26 risk-triaged-ratification spec): tensions.md + // is read ONCE here and fed into both computeTensionHealth (vault-global, + // unfiltered — #216 rider / #217 decision C) and rankPendingActions (which + // applies pathVisible internally, per-item, for the risk queue). + const tensionsRes = await listTensions(vaultRoot); + if (!tensionsRes.ok) return tensionsRes; + // Vault-global by design (#216 rider / #217 decision C): tension health is // the operator's whole-vault view, so it aggregates over ALL docs and // tensions regardless of pathVisible. Counts only — no paths cross here. - const tensionHealth = await computeTensionHealth(vaultRoot, allDocs, now); + const tensionHealth = computeTensionHealth(tensionsRes.value, allDocs, now); if (!tensionHealth.ok) return tensionHealth; // Each JSONL log is read ONCE; the lint summaries and the coverage view @@ -376,7 +564,18 @@ export async function runLint( const edgesRes = await listEdges(vaultRoot, {}, now); if (!edgesRes.ok) return edgesRes; - const stagedActions = pendingLintItems(stagedRes.value, now); + // Risk-triaged queue (Decisions 1 + 2 + 4): risk descending, expiry + // ascending tiebreak, filtered to the caller's vantage with the hidden + // remainder coarsened. `docs` here is the FULL, unfiltered set — B and T + // read the whole graph and apply pathVisible internally per item, per the + // spec's per-vantage scoring rule (never subtract-the-visible-terms leakage). + const { items: stagedActions, hiddenPending: hiddenStagedActions } = rankPendingActions({ + actions: stagedRes.value, + docs: allDocs, + tensions: tensionsRes.value, + now, + pathVisible, + }); const shadowActions = shadowLintSummaryOf(shadowRecordsRes.value); const coverageEquityRes = coverageEquitySummary({ docs, @@ -387,15 +586,28 @@ export async function runLint( }); if (!coverageEquityRes.ok) return coverageEquityRes; + // Independence-aware promotion calibration (Decision 4). Error-tolerant on + // both reads: a failure here must never fail vault_lint (advisory posture), + // so it degrades to the zero summary rather than propagating the error. + const independenceViewRes = independenceCalibrationView(vaultRoot, now); + const independenceJournalRes = await listIndependenceShadow(vaultRoot); + const independenceCalibration = + independenceViewRes.ok && independenceJournalRes.ok + ? independenceCalibrationSummaryOf(independenceViewRes.value, independenceJournalRes.value) + : emptyIndependenceCalibrationSummary(); + return ok({ generatedAt: now.toISOString(), checks, totalFindings, tensionHealth: tensionHealth.value, stagedActions, + hiddenStagedActions, shadowActions, coverageEquity: coverageEquityRes.value, reviewThroughput: reviewThroughputSummary(stagedRes.value, now), + independenceCalibration, + pinsClassified, }); } @@ -408,14 +620,11 @@ export async function runLint( // entries — including `resolution.kind: accepted` — do not appear in any // aging tier; they show up in the Phase 1 stable-acknowledged and resolved // totals instead. -async function computeTensionHealth( - vaultRoot: string, +function computeTensionHealth( + tensions: TensionEntry[], docs: LoadedDoc[], now: Date, -): Promise> { - const tensions = await listTensions(vaultRoot); - if (!tensions.ok) return tensions; - +): Result { const byKind = Object.fromEntries(TENSION_KINDS.map((k) => [k, 0])) as Record< TensionKind, number @@ -436,7 +645,7 @@ async function computeTensionHealth( let aging = 0; let stale = 0; - for (const t of tensions.value) { + for (const t of tensions) { total += 1; byKind[t.kind] += 1; if (t.kind === "unspecified") unspecifiedLegacy += 1; @@ -469,7 +678,7 @@ async function computeTensionHealth( // Cluster surface (Phase 2). computeTensionClusters applies the same scope // filter the cluster tool does — unresolved AND non-accepted — so the lint // metrics line up exactly with what `vault_tension_clusters` reports. - const clusterResult = computeTensionClusters(tensions.value, now); + const clusterResult = computeTensionClusters(tensions, now); let maxSize = 0; let large = 0; let aged = 0; @@ -495,7 +704,7 @@ async function computeTensionHealth( // source edge), but the published metric is the primary count only — the // top-level lint metric stays disciplined against advisory inflation. const staleSeeds = new Set(); - for (const t of tensions.value) { + for (const t of tensions) { if (t.resolved) continue; if (agingTier(t, now) !== "stale") continue; if (t.sourceA) staleSeeds.add(t.sourceA); diff --git a/src/curation/read-log.ts b/src/curation/read-log.ts index 5c46319a..18a47c55 100644 --- a/src/curation/read-log.ts +++ b/src/curation/read-log.ts @@ -41,6 +41,16 @@ export interface ReadLogEntry { // (or when the classification errored) — consumers must treat absent as // uninstrumented, not as zero. broken_upstream?: number; + // Citation-anchors kill-condition (b) instrumentation (2026-07-26 spec): + // counts from the SAME classification vault_read computed, uncensored by + // the caller's role (the returned `anchors` field may be null for a role + // without code_repo_visibility even though these counts are populated). + // All three are present together or all absent — absent means no anchors + // classification ran (no pins, no code_repos, jit_anchors: false, or a + // config-load failure), not "zero drift". + anchors_moved?: number; + anchors_missing?: number; + anchors_errored?: number; } export function readLogPath(vaultRoot: string): string { @@ -61,6 +71,9 @@ export async function recordRead( ...(entry.run_id ? { run_id: entry.run_id } : {}), ...(entry.principal ? { principal: entry.principal } : {}), ...(entry.broken_upstream !== undefined ? { broken_upstream: entry.broken_upstream } : {}), + ...(entry.anchors_moved !== undefined ? { anchors_moved: entry.anchors_moved } : {}), + ...(entry.anchors_missing !== undefined ? { anchors_missing: entry.anchors_missing } : {}), + ...(entry.anchors_errored !== undefined ? { anchors_errored: entry.anchors_errored } : {}), }; try { mkdirSync(join(vaultRoot, ".daftari"), { recursive: true }); diff --git a/src/curation/risk.ts b/src/curation/risk.ts new file mode 100644 index 00000000..f8b68bb0 Binary files /dev/null and b/src/curation/risk.ts differ diff --git a/src/curation/staged-actions.ts b/src/curation/staged-actions.ts index fc56ec2f..764d7da6 100644 --- a/src/curation/staged-actions.ts +++ b/src/curation/staged-actions.ts @@ -29,7 +29,7 @@ import { appendFileSync, mkdirSync, readFileSync } from "node:fs"; import { join } from "node:path"; import { err, ok, type Result } from "../frontmatter/types.js"; -import { getProvider } from "../search/vector.js"; +import { getProvider, getQuantize } from "../search/vector.js"; import { clearStagedActions, type IndexDb, @@ -58,6 +58,25 @@ export type StagedActionType = (typeof STAGED_ACTION_TYPES)[number]; export const DEFAULT_TTL_DAYS = 14; +// Decision 3 (2026-07-26 risk-triaged-ratification spec): every ratify/reject +// verdict is tagged with a machine-readable decision kind and (usually) a +// correction category, so the outcome data Decision 1's W term and the +// witness's per-principal tallies read is structured, not just free text. +export const DECISION_KINDS = ["approve", "edit-then-approve", "reject"] as const; +export type DecisionKind = (typeof DECISION_KINDS)[number]; + +export const REASON_CATEGORIES = [ + "wrong-conclusion", + "wrong-target", + "overbroad", + "stale-evidence", + "duplicate", + "formatting", + "policy", + "other", +] as const; +export type ReasonCategory = (typeof REASON_CATEGORIES)[number]; + // The lifecycle states a staged action moves through. `ratified-pending-tool` // is a legacy terminal state from v1.17 (before §11.4 wired up the // supersede/merge/confidence-up write tools); it is no longer produced but is @@ -98,6 +117,29 @@ export interface StagedAction { // Caller-supplied trace/run identifier stamped at stage time (#235). Like // decidedByPrincipal, JSONL-only — not in the sqlite staged_actions table. runId: string | null; + // Decision 3 (risk-triaged-ratification spec, 2026-07-26): how the verdict + // was reached and, usually, why. Both null until a decision lands; JSONL + // only, like decidedByPrincipal. + decisionKind: string | null; + reasonCategory: string | null; + // The amendment payload when decisionKind is 'edit-then-approve' — same + // JSON-encode/parse convention as proposedDiff. Null otherwise. + amendedDiff: unknown; + // The authenticated identity (access.user) that STAGED the proposal (C4 + // disposition of the risk-triaged-ratification spec) — distinct from + // proposedBy, which is a caller-claimed, unauthenticated display string. + // Null for legacy records and for proposals staged without an access + // context (operator CLI use). JSONL-only. + stagedByPrincipal: string | null; + // Decision from Mihir (2026-07-27, risk-triaged-ratification spec): a + // NON-AUTHORITATIVE snapshot of the derived risk score at the moment this + // decision was recorded. JSONL-only — never mirrored to sqlite, never read + // for queue ordering (rankPendingActions always recomputes on read). Exists + // solely so the kill-condition-#1 analysis can partition PAST decisions by + // risk quartile without needing to reconstruct blast radius / tension state + // as it was at decision time. Null on proposal rows and on sweep-authored + // expiries (an expiry is not a decision). + riskAtDecision: number | null; } export interface StageActionInput { @@ -113,6 +155,9 @@ export interface StageActionInput { // Override the proposal timestamp — only used by tests for deterministic // expiry math; production callers omit it and get the current clock. proposedAt?: string; + // The authenticated identity staging the proposal (access.user). Optional — + // callers without an AccessContext (operator CLI) omit it. + stagedByPrincipal?: string; } export interface DecisionInput { @@ -123,6 +168,18 @@ export interface DecisionInput { // The authenticated identity that issued the decision (access.user, §11.6). // Optional — omitted when no AccessContext is present. decidedByPrincipal?: string; + // Decision 3 fields (risk-triaged-ratification spec). Validated for enum + // membership here (the durable-boundary re-check, matching + // validateStageInput's precedent) but NOT enforced as required — the tool + // layer (vault_ratify) owns requiredness policy, same split as the + // pending-status check. The sweep never sets any of these three. + decisionKind?: DecisionKind; + reasonCategory?: ReasonCategory; + amendedDiff?: unknown; + // Non-authoritative risk snapshot (Mihir's 2026-07-27 decision). Any finite + // number, including 0, is a valid snapshot — undefined means "not computed + // / not available", never coerced to null silently. + riskAtDecision?: number; } export function stagedActionsPath(vaultRoot: string): string { @@ -173,6 +230,13 @@ interface RawRecord { ratification_reason?: string | null; decided_by_principal?: string | null; run_id?: string | null; + // Proposal branch only (Decision 3, C4 disposition). + staged_by_principal?: string | null; + // Decision branch only (Decision 3 / Mihir's 2026-07-27 addendum). + decision_kind?: string | null; + reason_category?: string | null; + amended_diff?: string | null; + risk_at_decision?: number | null; } function readRawRecords(vaultRoot: string): RawRecord[] { @@ -221,6 +285,11 @@ function collapse(records: RawRecord[]): Map { ratification_reason: null, decided_by_principal: null, run_id: rec.run_id ?? null, + staged_by_principal: rec.staged_by_principal ?? null, + decision_kind: null, + reason_category: null, + amended_diff: null, + risk_at_decision: null, }); } else { // Decision record — only meaningful if the proposal was already seen. @@ -231,6 +300,10 @@ function collapse(records: RawRecord[]): Map { existing.ratified_by = rec.ratified_by ?? null; existing.ratification_reason = rec.ratification_reason ?? null; existing.decided_by_principal = rec.decided_by_principal ?? null; + existing.decision_kind = rec.decision_kind ?? null; + existing.reason_category = rec.reason_category ?? null; + existing.amended_diff = rec.amended_diff ?? null; + existing.risk_at_decision = rec.risk_at_decision ?? null; } } return byId; @@ -243,6 +316,14 @@ function rowToStagedAction(row: StagedActionRow): StagedAction { } catch { proposedDiff = row.proposed_diff; } + let amendedDiff: unknown = null; + if (row.amended_diff) { + try { + amendedDiff = JSON.parse(row.amended_diff); + } catch { + amendedDiff = row.amended_diff; + } + } return { id: row.id, actionType: row.action_type, @@ -256,11 +337,17 @@ function rowToStagedAction(row: StagedActionRow): StagedAction { ratifiedAt: row.ratified_at, ratifiedBy: row.ratified_by, ratificationReason: row.ratification_reason, - // NOTE: decided_by_principal and run_id are JSONL-only — no DDL column in - // staged_actions; SQLite-backed reads (rebuildStagedActionsIndex) always - // yield null here. + // NOTE: decided_by_principal, run_id, staged_by_principal, decision_kind, + // reason_category, amended_diff, and risk_at_decision are all JSONL-only — + // no DDL column in staged_actions; SQLite-backed reads + // (rebuildStagedActionsIndex) always yield null for every field below. decidedByPrincipal: row.decided_by_principal ?? null, runId: row.run_id ?? null, + stagedByPrincipal: row.staged_by_principal ?? null, + decisionKind: row.decision_kind ?? null, + reasonCategory: row.reason_category ?? null, + amendedDiff, + riskAtDecision: row.risk_at_decision ?? null, }; } @@ -338,6 +425,9 @@ function appendProposalRecord( rationale: input.rationale.trim(), proposed_diff: JSON.stringify(input.proposedDiff), ...(input.runId && input.runId.trim().length > 0 ? { run_id: input.runId.trim() } : {}), + ...(input.stagedByPrincipal && input.stagedByPrincipal.trim().length > 0 + ? { staged_by_principal: input.stagedByPrincipal.trim() } + : {}), }; appendFileSync(stagedActionsPath(vaultRoot), `${JSON.stringify(record)}\n`); return { id, expires_at: expiresAt }; @@ -452,6 +542,22 @@ export async function recordDecision( id: string, decision: DecisionInput, ): Promise> { + if ( + decision.decisionKind !== undefined && + !(DECISION_KINDS as readonly string[]).includes(decision.decisionKind) + ) { + return err( + new Error(`recordDecision: decisionKind must be one of: ${DECISION_KINDS.join(", ")}`), + ); + } + if ( + decision.reasonCategory !== undefined && + !(REASON_CATEGORIES as readonly string[]).includes(decision.reasonCategory) + ) { + return err( + new Error(`recordDecision: reasonCategory must be one of: ${REASON_CATEGORIES.join(", ")}`), + ); + } try { // Collapse the pre-existing log ONCE. The decision we are about to write is // fully known, so rather than re-reading + re-collapsing the whole file a @@ -470,6 +576,16 @@ export async function recordDecision( ...(decision.decidedByPrincipal != null ? { decided_by_principal: decision.decidedByPrincipal } : {}), + ...(decision.decisionKind !== undefined ? { decision_kind: decision.decisionKind } : {}), + ...(decision.reasonCategory !== undefined + ? { reason_category: decision.reasonCategory } + : {}), + ...(decision.amendedDiff !== undefined + ? { amended_diff: JSON.stringify(decision.amendedDiff) } + : {}), + ...(decision.riskAtDecision !== undefined + ? { risk_at_decision: decision.riskAtDecision } + : {}), }; mkdirSync(join(vaultRoot, ".daftari"), { recursive: true }); appendFileSync(stagedActionsPath(vaultRoot), `${JSON.stringify(record)}\n`); @@ -481,6 +597,10 @@ export async function recordDecision( existing.ratified_by = record.ratified_by ?? null; existing.ratification_reason = record.ratification_reason ?? null; existing.decided_by_principal = record.decided_by_principal ?? null; + existing.decision_kind = record.decision_kind ?? null; + existing.reason_category = record.reason_category ?? null; + existing.amended_diff = record.amended_diff ?? null; + existing.risk_at_decision = record.risk_at_decision ?? null; return ok(rowToStagedAction(existing)); } catch (e) { const reason = e instanceof Error ? e.message : String(e); @@ -504,7 +624,10 @@ export async function listStagedActions( } // One pending action as the lint surface presents it. `rationale` is trimmed -// to its first sentence; ages are whole days relative to `now`. +// to its first sentence; ages are whole days relative to `now`. Superseded as +// the lint queue's item shape by RankedStagedActionItem (src/curation/risk.ts, +// 2026-07-26 risk-triaged-ratification spec) — kept exported and as the base +// interface so the two can't drift on their shared fields. export interface StagedActionLintItem { id: string; actionType: string; @@ -515,38 +638,62 @@ export interface StagedActionLintItem { } // First sentence of a rationale: up to the first sentence-ending period -// followed by whitespace, else the whole trimmed string. -function firstSentence(text: string): string { +// followed by whitespace, else the whole trimmed string. Exported so +// src/curation/risk.ts's ranked queue item can reuse the same trimming rule. +export function firstSentence(text: string): string { const trimmed = text.trim(); const m = trimmed.match(/^(.*?[.!?])(\s|$)/); return m ? (m[1] as string) : trimmed; } -// Pending actions for the lint "Staged actions" section, soonest-to-expire -// first. Pure — the lint path derives this from an already-read action list -// instead of collapsing the log a second time. -export function pendingLintItems(actions: StagedAction[], now: Date): StagedActionLintItem[] { - return actions - .filter((a) => a.status === "pending") - .sort((a, b) => (a.expiresAt < b.expiresAt ? -1 : a.expiresAt > b.expiresAt ? 1 : 0)) - .map((a) => ({ - id: a.id, - actionType: a.actionType, - targetPath: a.targetPath, - ageDays: daysSince(a.proposedAt, now), - expiresInDays: daysUntil(a.expiresAt, now), - rationale: firstSentence(a.rationale), - })); +// Per-principal proposal outcome tallies — the shared implementation behind +// both the witness's proposal record (src/witness/track-record.ts) and the +// risk scorer's W term (src/curation/risk.ts), so the two can never drift +// apart on what counts as ratified/rejected/expired/edited (2026-07-26 +// risk-triaged-ratification spec, Decision 3 / C4 disposition). +// +// Keyed by `stagedByPrincipal ?? proposedBy`: the authenticated identity that +// staged the proposal when the record has one, else the caller-claimed +// `proposedBy` string (legacy records, or proposals staged without an access +// context). This closes the C4 laundering/poisoning hole — rotating the +// unauthenticated `proposed_by` string under one authenticated stager still +// lands in a single tally bucket, and junk staged by principal X under a +// rival's claimed name counts against X, not the rival. +export interface ProposalTallies { + total: number; + ratified: number; + rejected: number; + expired: number; + pending: number; + // Count of ratified rows whose decisionKind is 'edit-then-approve'. A + // SUBSET of `ratified`, not an addition to it — status stays authoritative. + edited: number; + byCategory: Record; } -// Read-only — the sweep that expires stale actions is a separate step. -export async function listPendingForLint( - vaultRoot: string, - now: Date = new Date(), -): Promise> { - const actions = await listStagedActions(vaultRoot); - if (!actions.ok) return actions; - return ok(pendingLintItems(actions.value, now)); +export function proposalTallies(actions: StagedAction[]): Map { + const tallies = new Map(); + const recordFor = (key: string): ProposalTallies => { + let t = tallies.get(key); + if (!t) { + t = { total: 0, ratified: 0, rejected: 0, expired: 0, pending: 0, edited: 0, byCategory: {} }; + tallies.set(key, t); + } + return t; + }; + for (const a of actions) { + const key = a.stagedByPrincipal ?? a.proposedBy; + const t = recordFor(key); + t.total += 1; + if (a.status === "ratified" || a.status === "ratified-pending-tool") t.ratified += 1; + else if (a.status === "rejected") t.rejected += 1; + else if (a.status === "expired") t.expired += 1; + else t.pending += 1; + if (a.decisionKind === "edit-then-approve") t.edited += 1; + if (a.reasonCategory) + t.byCategory[a.reasonCategory] = (t.byCategory[a.reasonCategory] ?? 0) + 1; + } + return tallies; } export async function getStagedActionById( @@ -626,7 +773,7 @@ export function rebuildStagedActionsIndex( // directly against its already-open handle. The provider dim matters: opening // at the wrong dim would drop and recreate the embeddings_vec mirror. export function materializeStagedActions(vaultRoot: string): Result<{ count: number }, Error> { - const opened = openIndexDb(vaultRoot, getProvider().dim); + const opened = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (!opened.ok) return opened; const db = opened.value; try { diff --git a/src/curation/tier2.ts b/src/curation/tier2.ts index c0fbeba1..8895d5f6 100644 --- a/src/curation/tier2.ts +++ b/src/curation/tier2.ts @@ -174,7 +174,16 @@ export function accumulateFieldChanges( for (const [field, diff] of Object.entries(e.frontmatter_diff ?? {})) { const existing = changes[field]; if (existing) existing.after = diff.after; - else changes[field] = { before: diff.before, after: diff.after }; + // A `create` action's diff has no prior value: frontmatterDiff sets + // `before: undefined` (the field didn't exist on the null "before" + // frontmatter), and JSON serialization of the provenance log then + // drops that key entirely — `diff.before` reads back as `undefined`, + // not `null`. Normalize here so a FieldChange always carries a real + // `null` for "no prior value" instead of an absent key; the schema + // (field_changes' tier2WorkItemSchema) requires both `before` and + // `after` to be present, with `null` as the documented "no value" + // signal — an absent key is a shape the contract never promised. + else changes[field] = { before: diff.before ?? null, after: diff.after }; } if (e.body_changed ?? (e.action === "create" || e.action === "update" || e.action === "append")) bodyChanged = true; diff --git a/src/eval/index.ts b/src/eval/index.ts index a1093369..0c69a20a 100644 --- a/src/eval/index.ts +++ b/src/eval/index.ts @@ -9,6 +9,7 @@ import { err, ok, type Result } from "../frontmatter/types.js"; import { generateQuestions } from "./generate.js"; import { createAnthropicClient, type LlmClient } from "./llm.js"; import { createOpenRouterClient, resolveTransport } from "./llm-openrouter.js"; +import { runPackAnswerer } from "./pack-condition.js"; import { PROMPT_VERSION } from "./prompts.js"; import { type DirPruneResult, type PruneRules, parseOlderThan, prune } from "./prune.js"; import { runAnswerer } from "./run.js"; @@ -37,7 +38,8 @@ const HELP = `daftari eval — cortex quality metric. Usage: daftari eval [--vault ] [--n ] [--k ] [--seed ] [--max-nodes ] [--transport ] daftari eval generate [--vault ] [--n ] [--seed ] [--max-nodes ] [--transport ] - daftari eval run [--questions ] [--vault ] [--model ] [--k ] [--resume ] [--transport ] + daftari eval run [--questions ] [--vault ] [--model ] [--k ] [--resume ] + [--condition tools|pack] [--budget ] [--max-tool-calls ] [--transport ] daftari eval score [--results ] [--vault ] [--grader-model ] [--transport ] daftari eval prune [--vault ] [--keep ] [--older-than ] [--dry-run] @@ -54,6 +56,21 @@ Defaults: --transport anthropic (default, ANTHROPIC_API_KEY) or openrouter (OPENROUTER_API_KEY); env fallback DAFTARI_LLM_TRANSPORT — the same selection rules as daftari sleep/consolidate + --condition tools 'tools' (default, the in-process tool-loop answerer) or + 'pack' (build a vault_context brief and hand it to the + answerer as its ONLY context — no tools). Spec + 2026-07-26-context-packs-progressive-disclosure-design.md, + Decision 4. + --budget 4000 pack condition only: token budget passed to + vault_context (same defaults/clamps). + --max-tool-calls tools condition only: hard cap on REALIZED tool calls + per (question, k) run — parallel calls past the cap + are stubbed, never executed (C5). Default uncapped. + Results ids record a capped run as '-tools-c{N}'; a + pack run as '-pack-b{budget}'. --resume checks the + persisted condition/budget/cap against the flags and + refuses (exit 2) on any mismatch — never a silent + override. Environment: ANTHROPIC_API_KEY required for LLM-mediated stages on the anthropic transport @@ -122,6 +139,36 @@ function maxNodesFlag(argv: string[]): number { return n; } +// --condition tools|pack (spec 2026-07-26-context-packs-progressive- +// disclosure-design.md, final plan Phase 3.4). Default 'tools' — every +// existing invocation keeps its behavior. An invalid value is a config error +// (exit 2 via runEval's catch), same posture as a bad --transport. +function conditionFlag(argv: string[]): "tools" | "pack" { + const raw = flag(argv, "condition"); + if (raw === undefined) return "tools"; + if (raw === "tools" || raw === "pack") return raw; + throw new Error(`--condition must be 'tools' or 'pack' (got ${JSON.stringify(raw)})`); +} + +// --budget for the pack condition (default 4000, matching vault_context's +// own default). vault_context re-validates/clamps on every call (C9); the +// CLI does not duplicate that logic, so a bad value surfaces as a runtime +// error (exit 3) from the first vaultContext call in the run, not here. +function budgetFlag(argv: string[]): number { + return intFlag(argv, "budget", 4000); +} + +// --max-tool-calls (C5): default undefined = uncapped, today's behavior. +function maxToolCallsFlag(argv: string[]): number | undefined { + const raw = flag(argv, "max-tool-calls"); + if (raw === undefined) return undefined; + const n = parseInt(raw, 10); + if (Number.isNaN(n) || n < 0) { + throw new Error("--max-tool-calls must be a non-negative integer"); + } + return n; +} + // Persists an artifact, translating a throw into the RUNTIME exit code. // Storage writes run after config validation succeeded: a failure here is // disk-full/permissions (#102), and letting it bubble to runEval's catch-all @@ -226,6 +273,52 @@ async function runGenerate(argv: string[]): Promise { return 0; } +// Mints the results id (spec 2026-07-26-context-packs-progressive-disclosure- +// design.md, final plan Phase 3.4/C8): `-pack-b{budget}` for a pack run, +// `-tools-c{cap}` for a capped tools run, and the historical shape for an +// uncapped tools run — every existing id keeps its exact form. +function mintRunId( + questionsId: string, + model: string, + timestamp: string, + condition: "tools" | "pack", + budget: number, + maxToolCalls: number | undefined, +): string { + const modelPart = modelIdSlug(model); + if (condition === "pack") return `${questionsId}-${modelPart}-pack-b${budget}-${timestamp}`; + if (maxToolCalls !== undefined) { + return `${questionsId}-${modelPart}-tools-c${maxToolCalls}-${timestamp}`; + } + return `${questionsId}-${modelPart}-${timestamp}`; +} + +// On --resume, the persisted condition/pack_budget/max_tool_calls are +// checked against the CLI flags; any mismatch is a config error (exit 2), +// never a silent override (C8). A legacy artifact with no `condition` field +// loads as an uncapped `tools` run. +function resumeMismatch( + resumeFrom: EvalRun, + condition: "tools" | "pack", + budget: number, + maxToolCalls: number | undefined, +): string | null { + const persistedCondition = resumeFrom.condition ?? "tools"; + if (persistedCondition !== condition) { + return `persisted condition '${persistedCondition}' does not match --condition '${condition}'`; + } + if (condition === "pack" && resumeFrom.pack_budget !== budget) { + return `persisted pack_budget ${resumeFrom.pack_budget} does not match --budget ${budget}`; + } + if (condition === "tools" && resumeFrom.max_tool_calls !== maxToolCalls) { + return ( + `persisted max_tool_calls ${resumeFrom.max_tool_calls} does not match ` + + `--max-tool-calls ${maxToolCalls}` + ); + } + return null; +} + async function runRun(argv: string[]): Promise { const llm = resolveEvalLlm(argv); if (!llm.ok) { @@ -240,6 +333,9 @@ async function runRun(argv: string[]): Promise { } const k = intFlag(argv, "k", 2); const model = flag(argv, "model") ?? llm.value.defaultModel; + const condition = conditionFlag(argv); + const budget = budgetFlag(argv); + const maxToolCalls = maxToolCallsFlag(argv); const qsRead = await readQuestionSet(vault, questionsId); if (!qsRead.ok) { @@ -266,23 +362,39 @@ async function runRun(argv: string[]): Promise { return 3; } resumeFrom = r.value; + const mismatch = resumeMismatch(resumeFrom, condition, budget, maxToolCalls); + if (mismatch) { + process.stderr.write(`--resume ${resumeId}: ${mismatch}\n`); + return 2; + } } // Mint the stable id + timestamp up front so the on-disk file path is stable // across the run and any later --resume; persist incrementally so a mid-run // failure leaves a resumable partial file. const timestamp = new Date().toISOString(); - const runId = resumeFrom - ? resumeFrom.id - : `${qsRead.value.id}-${modelIdSlug(model)}-${timestamp}`; - const run = await runAnswerer(qsRead.value, vault, llm.value.client, { - k, - model, - resumeFrom, - runId, - timestamp, - persist: makeBestEffortPersist(vault), - }); + const runId = + resumeFrom?.id ?? mintRunId(qsRead.value.id, model, timestamp, condition, budget, maxToolCalls); + const run = + condition === "pack" + ? await runPackAnswerer(qsRead.value, vault, llm.value.client, { + k, + model, + budget, + resumeFrom, + runId, + timestamp, + persist: makeBestEffortPersist(vault), + }) + : await runAnswerer(qsRead.value, vault, llm.value.client, { + k, + model, + resumeFrom, + runId, + timestamp, + persist: makeBestEffortPersist(vault), + ...(maxToolCalls !== undefined ? { maxToolCalls } : {}), + }); if (!run.ok) { process.stderr.write(`${run.error.message}\n`); process.stderr.write( @@ -393,6 +505,12 @@ async function runScore(argv: string[]): Promise { score.k = run.k; score.n = qs.questions.length; score.timestamp = new Date().toISOString(); + // Carried from the run (spec 2026-07-26-context-packs-progressive- + // disclosure-design.md, final plan Phase 3.2/C8) so the score artifact is + // self-describing. Absent on a legacy uncapped `tools` run. + if (run.condition !== undefined) score.condition = run.condition; + if (run.pack_budget !== undefined) score.pack_budget = run.pack_budget; + if (run.max_tool_calls !== undefined) score.max_tool_calls = run.max_tool_calls; const wroteScore = await persistOrRuntimeExit("score", () => writeScore(vault, score)); if (wroteScore) return wroteScore; @@ -412,17 +530,28 @@ async function runScore(argv: string[]): Promise { models: score.models, prompt_version: score.prompt_version, spec_version: score.spec_version, + ...(score.condition !== undefined ? { condition: score.condition } : {}), }; const wroteHistory = await persistOrRuntimeExit("history", () => appendHistory(vault, histEntry)); if (wroteHistory) return wroteHistory; // Pretty-print headline + per-tier means, and the coverage line (#102): - // how many answerer runs the score actually stands on. - process.stdout.write(`score: ${score.score.toFixed(3)} ± ${score.score_std.toFixed(3)}\n`); + // how many answerer runs the score actually stands on. The header line is + // self-describing (spec 2026-07-26-context-packs-progressive-disclosure- + // design.md, final plan Phase 3.2/C8): the run's condition parameters + // travel with the printed artifact, not just the JSON file. + const conditionParts = [`condition=${score.condition ?? "tools"}`]; + if (score.pack_budget !== undefined) conditionParts.push(`budget=${score.pack_budget}`); + if (score.max_tool_calls !== undefined) + conditionParts.push(`max-tool-calls=${score.max_tool_calls}`); + process.stdout.write( + `score: ${score.score.toFixed(3)} ± ${score.score_std.toFixed(3)} (${conditionParts.join(", ")})\n`, + ); for (const t of TIERS) { const ts = score.by_tier[t]; process.stdout.write( - ` ${t.padEnd(16)}: ${ts.mean.toFixed(3)} (n=${ts.n}, efficiency=${ts.trace_efficiency.toFixed(1)} calls)\n`, + ` ${t.padEnd(16)}: ${ts.mean.toFixed(3)} (n=${ts.n}, efficiency=${ts.trace_efficiency.toFixed(1)} calls, ` + + `${ts.mean_tokens.toFixed(0)} tokens/correct)\n`, ); } const neverAttempted = Math.max(0, plannedRuns - presentRuns); diff --git a/src/eval/llm-openrouter.ts b/src/eval/llm-openrouter.ts index 2b87ad56..fae85af3 100644 --- a/src/eval/llm-openrouter.ts +++ b/src/eval/llm-openrouter.ts @@ -22,6 +22,7 @@ import { type LlmClient, retry, stripCodeFence, + TOOL_BUDGET_EXHAUSTED_MESSAGE, } from "./llm.js"; import type { CortexEvalError } from "./types.js"; @@ -231,6 +232,10 @@ export function createOpenRouterClient(opts?: { fetchImpl?: typeof fetch }): Llm })); let totalIn = 0; let totalOut = 0; + // Mirrors the anthropic client's exhaustion flag (C5): once the budget + // is spent, the next request omits `tools` entirely, forcing a final + // answer. + let toolsExhausted = false; for (let round = 0; round < maxRounds; round++) { const res = await retry(async () => @@ -238,7 +243,7 @@ export function createOpenRouterClient(opts?: { fetchImpl?: typeof fetch }): Llm model: o.model, max_tokens: o.maxTokens ?? 4096, ...(o.temperature !== undefined ? { temperature: o.temperature } : {}), - tools, + ...(toolsExhausted ? {} : { tools }), messages, }), ); @@ -272,8 +277,26 @@ export function createOpenRouterClient(opts?: { fetchImpl?: typeof fetch }): Llm tool_calls: message?.tool_calls, }); + // C5: cap REALIZED calls, not requested ones — same contract as the + // anthropic client. Every rawCalls[i] past `remaining` still gets a + // tool_result (the wire requires one per tool_call_id) but is + // stubbed, never executed, never counted. + const remaining = + o.maxToolCalls === undefined + ? rawCalls.length + : Math.max(0, o.maxToolCalls - toolCalls.length); + for (let i = 0; i < rawCalls.length; i++) { const tc = rawCalls[i] as OpenRouterToolCall & { function: { name: string } }; + const toolCallId = tc.id ?? `call_${round}_${i}`; + if (i >= remaining) { + messages.push({ + role: "tool", + tool_call_id: toolCallId, + content: JSON.stringify({ tool_error: TOOL_BUDGET_EXHAUSTED_MESSAGE }), + }); + continue; + } const rawArgs = tc.function.arguments ?? ""; let input: unknown; try { @@ -298,10 +321,14 @@ export function createOpenRouterClient(opts?: { fetchImpl?: typeof fetch }): Llm role: "tool", // Some providers omit ids on single calls; synthesize a stable one // so the echo-back stays well-formed. - tool_call_id: tc.id ?? `call_${round}_${i}`, + tool_call_id: toolCallId, content: typeof output === "string" ? output : JSON.stringify(output), }); } + + if (o.maxToolCalls !== undefined && toolCalls.length >= o.maxToolCalls) { + toolsExhausted = true; + } } return err({ kind: "llm", diff --git a/src/eval/llm.ts b/src/eval/llm.ts index 6b0d0f8d..9361b219 100644 --- a/src/eval/llm.ts +++ b/src/eval/llm.ts @@ -30,10 +30,26 @@ export interface ToolDef { input_schema: any; } +// The stub content every excess tool_use id receives once maxToolCalls is +// exhausted mid-round (spec 2026-07-26-context-packs-progressive-disclosure- +// design.md, final plan Phase 3.3 / C5). Exported so the OpenRouter twin and +// tests share the exact string. +export const TOOL_BUDGET_EXHAUSTED_MESSAGE = + "tool-call budget exhausted; answer with the information you already have"; + export interface CompleteWithToolsOpts extends CompleteOpts { tools: ToolDef[]; toolHandler: (name: string, input: unknown) => Promise; maxRounds?: number; // default 12 + // Hard cap on REALIZED tool calls (C5): a round whose parallel tool_use + // blocks would overshoot the remaining budget executes only the first + // `maxToolCalls - used` in block order; every excess id still gets a + // tool_result (the API requires one per id) carrying + // TOOL_BUDGET_EXHAUSTED_MESSAGE, and is never pushed onto `tool_calls` — + // so `tool_calls.length` is a true, enforced upper bound, not a request- + // time hint parallel calls can blow past. Undefined = uncapped (today's + // behavior). + maxToolCalls?: number; } export interface CompleteResult { @@ -125,6 +141,11 @@ export function createAnthropicClient(injected?: Pick): L let totalOut = 0; let lastStop = "unknown"; + // Once the budget is exhausted, the NEXT request omits `tools` entirely + // — the model cannot request another call, so it is forced to a final + // answer (C5). + let toolsExhausted = false; + for (let round = 0; round < maxRounds; round++) { const res = await retry(async () => ok( @@ -133,7 +154,7 @@ export function createAnthropicClient(injected?: Pick): L max_tokens: opts.maxTokens ?? 4096, system: opts.system, // biome-ignore lint/suspicious/noExplicitAny: SDK types - tools: opts.tools as any, + ...(toolsExhausted ? {} : { tools: opts.tools as any }), // biome-ignore lint/suspicious/noExplicitAny: SDK types messages: messages as any, ...(opts.temperature !== undefined ? { temperature: opts.temperature } : {}), @@ -165,8 +186,20 @@ export function createAnthropicClient(injected?: Pick): L messages.push({ role: "assistant", content: blocks }); + // C5: cap REALIZED calls, not requested ones. A round's parallel + // tool_use blocks are executed in order up to the remaining budget; + // every block past that still gets a tool_result (the API requires + // exactly one per tool_use id in the SAME user turn) but is stubbed, + // never executed, never counted. + const remaining = + opts.maxToolCalls === undefined + ? toolUses.length + : Math.max(0, opts.maxToolCalls - toolCalls.length); + const toExecute = toolUses.slice(0, remaining); + const toStub = toolUses.slice(remaining); + const toolResults: unknown[] = []; - for (const tu of toolUses) { + for (const tu of toExecute) { const t0 = Date.now(); let output: unknown; try { @@ -182,7 +215,18 @@ export function createAnthropicClient(injected?: Pick): L content: typeof output === "string" ? output : JSON.stringify(output), }); } + for (const tu of toStub) { + toolResults.push({ + type: "tool_result", + tool_use_id: tu.id, + content: JSON.stringify({ tool_error: TOOL_BUDGET_EXHAUSTED_MESSAGE }), + }); + } messages.push({ role: "user", content: toolResults }); + + if (opts.maxToolCalls !== undefined && toolCalls.length >= opts.maxToolCalls) { + toolsExhausted = true; + } } return err({ kind: "llm", diff --git a/src/eval/pack-condition.ts b/src/eval/pack-condition.ts new file mode 100644 index 00000000..b99a052a --- /dev/null +++ b/src/eval/pack-condition.ts @@ -0,0 +1,126 @@ +// src/eval/pack-condition.ts +// The pack answerer (spec 2026-07-26-context-packs-progressive-disclosure- +// design.md, Decision 4 / final plan Phase 3.1). Per (question, k): build a +// vault_context brief in-process, hand it to the answerer LLM as its ONLY +// context (no tools), and record a normal Trace so gradeAnswer and +// aggregateScore run completely unchanged. Same persist/--resume shape as +// runAnswerer (src/eval/run.ts) — the two conditions are siblings, not one +// extending the other, so a pack run never accidentally inherits a tool-loop +// assumption. +// +// access is passed as `undefined` to vaultContext, mirroring +// tool-surface.ts's established posture: eval runs locally against a +// snapshot, there is no caller identity. vault_context is intentionally NOT +// added to the eval tool-loop surface (src/eval/tool-surface.ts) — the +// baseline `tools` condition stays a clean control, uncontaminated by the +// very capability this condition measures. + +import { err, ok, type Result } from "../frontmatter/types.js"; +import { vaultContext } from "../tools/context.js"; +import type { LlmClient } from "./llm.js"; +import { PACK_ANSWERER_SYSTEM_PROMPT, PROMPT_VERSION } from "./prompts.js"; +import type { CortexEvalError, EvalRun, PerRunResult, QuestionSet, Trace } from "./types.js"; + +export interface PackRunOptions { + k: number; + model: string; + budget: number; + resumeFrom?: EvalRun; + runId?: string; // stable id the caller controls, mirrors RunOptions.runId + timestamp?: string; + persist?: (run: EvalRun) => Promise; +} + +export async function runPackAnswerer( + questions: QuestionSet, + vaultRoot: string, + llm: LlmClient, + opts: PackRunOptions, +): Promise> { + const ts = opts.timestamp ?? "2026-01-01T00:00:00Z"; + const id = + opts.resumeFrom?.id ?? opts.runId ?? `${questions.id}-${opts.model}-pack-b${opts.budget}-${ts}`; + const runs: Record = { ...(opts.resumeFrom?.runs ?? {}) }; + + const snapshot = (): EvalRun => ({ + id, + questions_id: questions.id, + answerer_model: opts.model, + prompt_version: PROMPT_VERSION, + timestamp: ts, + k: opts.k, + runs, + condition: "pack", + pack_budget: opts.budget, + }); + + for (let qi = 0; qi < questions.questions.length; qi++) { + const q = questions.questions[qi]; + for (let k = 0; k < opts.k; k++) { + const key = `${qi}:${k}`; + if (runs[key]?.status === "complete") continue; + + const t0 = Date.now(); + const packResult = await vaultContext( + vaultRoot, + { task: q.question, budget: opts.budget }, + undefined, + ); + if (!packResult.ok) { + runs[key] = { + question_id: q.id, + question_index: qi, + k_index: k, + status: "incomplete", + trace: null, + }; + await opts.persist?.(snapshot()); + return err({ kind: "runtime", message: packResult.error.message }); + } + const pack = packResult.value; + + const r = await llm.complete({ + model: opts.model, + system: PACK_ANSWERER_SYSTEM_PROMPT, + user: pack.brief, + }); + const wall_ms = Date.now() - t0; + if (!r.ok) { + runs[key] = { + question_id: q.id, + question_index: qi, + k_index: k, + status: "incomplete", + trace: null, + }; + await opts.persist?.(snapshot()); + return err(r.error); + } + + const trace: Trace = { + tool_calls: [], + final_answer: r.value.text, + total_tool_calls: 0, + input_tokens: r.value.input_tokens, + output_tokens: r.value.output_tokens, + wall_ms, + stop_reason: r.value.stop_reason, + pack: { + budget: pack.budget, + estimated_tokens: pack.estimatedTokens, + included_paths: pack.manifest.included.map((e) => e.path), + }, + }; + runs[key] = { + question_id: q.id, + question_index: qi, + k_index: k, + status: "complete", + trace, + }; + await opts.persist?.(snapshot()); + } + } + + return ok(snapshot()); +} diff --git a/src/eval/prompts.ts b/src/eval/prompts.ts index 8c2bb55e..35c59700 100644 --- a/src/eval/prompts.ts +++ b/src/eval/prompts.ts @@ -26,6 +26,22 @@ provided Daftari tools. Do not use training knowledge. Do not guess. If the vault does not contain the answer, say "Vault does not contain the answer." Cite source paths in your final answer using the format [path/to/doc.md].`; +// Pack condition (spec 2026-07-26-context-packs-progressive-disclosure- +// design.md, Decision 4 / final plan Phase 3.1). No tools: the answerer's +// ONLY context is the vault_context brief handed to it as the user message. +// A prompt addition does not bump PROMPT_VERSION — the freeze rule covers +// EDITS to an existing prompt an already-scored run depends on; a new, +// separate prompt for a new condition is not an edit. Comparability across +// conditions is carried by EvalRun.condition, not by prompt identity. +export const PACK_ANSWERER_SYSTEM_PROMPT = `You will answer a question about a Markdown knowledge vault using ONLY the +context brief provided below — no tools are available. The brief was +assembled by selecting and annotating the most relevant documents; it is not +synthesized, so treat every line as a direct fact from the vault, never as a +conclusion someone drew for you. Do not use training knowledge. Do not guess. +If the brief does not contain the answer, say "Vault does not contain the +answer." Cite source paths in your final answer using the format +[path/to/doc.md].`; + export const GRADER_PROMPT = `You are grading an answer to a question about a Markdown knowledge vault. Question: {{QUESTION}} diff --git a/src/eval/run.ts b/src/eval/run.ts index b6f5be9d..dbad4546 100644 --- a/src/eval/run.ts +++ b/src/eval/run.ts @@ -17,6 +17,11 @@ export interface RunOptions { runId?: string; // stable id the caller controls (so the on-disk file path is stable across the run and resume) timestamp?: string; // real timestamp for EvalRun metadata persist?: (run: EvalRun) => Promise; // called after every (q,k) status change, enabling --resume + // Hard cap on realized tool calls per (question, k) run (spec 2026-07-26- + // context-packs-progressive-disclosure-design.md, final plan Phase 3.3 / + // C5). Threaded straight through to completeWithTools. Default unset — + // uncapped, today's behavior. + maxToolCalls?: number; } export async function runAnswerer( @@ -40,6 +45,8 @@ export async function runAnswerer( timestamp: ts, k: opts.k, runs, + condition: "tools", + ...(opts.maxToolCalls !== undefined ? { max_tool_calls: opts.maxToolCalls } : {}), }); const tools = buildToolSurface(vaultRoot); @@ -58,6 +65,7 @@ export async function runAnswerer( user: q.question, tools: toolDefs, toolHandler: tools.handler, + ...(opts.maxToolCalls !== undefined ? { maxToolCalls: opts.maxToolCalls } : {}), }); const wall_ms = Date.now() - t0; if (!r.ok) { diff --git a/src/eval/score.ts b/src/eval/score.ts index 393389f0..7c2a7239 100644 --- a/src/eval/score.ts +++ b/src/eval/score.ts @@ -49,6 +49,7 @@ export function aggregateScore( const tierQuestions = questions.filter((q) => q.tier === tier); const perQuestionMeans: number[] = []; const efficiencyHits: number[] = []; + const tokenHits: number[] = []; for (const q of tierQuestions) { // Only include grades with a numeric verdict value (excludes ungraded). @@ -70,7 +71,15 @@ export function aggregateScore( const val = VERDICT_VALUE[grade.verdict]; if (val !== null && val > 0) { const t = opts.traces.get(`${grade.question_id}:${grade.k_index}`); - if (t) efficiencyHits.push(t.total_tool_calls); + if (t) { + efficiencyHits.push(t.total_tool_calls); + // mean_tokens (spec 2026-07-26-context-packs-progressive- + // disclosure-design.md, final plan Phase 3.2): the pack-condition + // twin of trace_efficiency — same "correct-or-partial" population, + // total (input + output) tokens instead of tool-call count, so + // "tokens per correct answer" is computable in either condition. + tokenHits.push(t.input_tokens + t.output_tokens); + } } } } @@ -80,6 +89,7 @@ export function aggregateScore( std: perQuestionMeans.length > 0 ? stddev(perQuestionMeans) : 0, n: perQuestionMeans.length, trace_efficiency: efficiencyHits.length > 0 ? avg(efficiencyHits) : 0, + mean_tokens: tokenHits.length > 0 ? avg(tokenHits) : 0, }; } @@ -123,9 +133,9 @@ export function aggregateScore( function blankByTier(): Record { return { - retrieval: { mean: 0, std: 0, n: 0, trace_efficiency: 0 }, - cross_reference: { mean: 0, std: 0, n: 0, trace_efficiency: 0 }, - contradiction: { mean: 0, std: 0, n: 0, trace_efficiency: 0 }, + retrieval: { mean: 0, std: 0, n: 0, trace_efficiency: 0, mean_tokens: 0 }, + cross_reference: { mean: 0, std: 0, n: 0, trace_efficiency: 0, mean_tokens: 0 }, + contradiction: { mean: 0, std: 0, n: 0, trace_efficiency: 0, mean_tokens: 0 }, }; } diff --git a/src/eval/types.ts b/src/eval/types.ts index 1d60c851..b1bbf2c9 100644 --- a/src/eval/types.ts +++ b/src/eval/types.ts @@ -63,6 +63,11 @@ export interface Trace { output_tokens: number; wall_ms: number; stop_reason: string; + // Present only for a pack-condition trace (spec 2026-07-26-context-packs- + // progressive-disclosure-design.md, final plan Phase 3.2): the pack the + // answerer was handed as its ONLY context — no tools. `included_paths` + // mirrors the pack's own manifest.included paths, in order. + pack?: { budget: number; estimated_tokens: number; included_paths: string[] }; } export interface ToolCall { @@ -96,6 +101,14 @@ export interface EvalRun { k: number; // Keyed by `"${question_index}:${k_index}"`. See spec §6.5 for rationale. runs: Record; + // Run condition parameters (spec 2026-07-26-context-packs-progressive- + // disclosure-design.md, final plan Phase 3.2/C8). All three absent on a + // legacy artifact, which loads and scores as an uncapped `tools` run — + // never inferred, never silently defaulted on `--resume` (C8's config-error + // posture). + condition?: "tools" | "pack"; + pack_budget?: number; + max_tool_calls?: number; } // --- Grade and score shapes --- @@ -116,6 +129,11 @@ export interface TierScore { std: number; n: number; trace_efficiency: number; // mean tool calls per correct-or-partial answer + // Mean total tokens (input + output) per graded run in the tier — the + // pack-condition twin of trace_efficiency's tool-call count, so "tokens + // per correct answer" (final plan Phase 3.2) is computable in EITHER + // condition, tools or pack. + mean_tokens: number; } export interface Score { @@ -131,6 +149,12 @@ export interface Score { k: number; n: number; timestamp: string; + // Carried from the scored EvalRun (spec final plan Phase 3.2/C8) so the + // score artifact is self-describing without cross-referencing the results + // file. Absent for a legacy uncapped `tools` run. + condition?: "tools" | "pack"; + pack_budget?: number; + max_tool_calls?: number; } // --- History --- @@ -147,6 +171,9 @@ export interface HistoryEntry { models: { generator: string; answerer: string; grader: string }; prompt_version: number; spec_version: number; + // Carried from Score (spec 2026-07-26-context-packs-progressive-disclosure- + // design.md, final plan Phase 3.2) — absent for a legacy uncapped run. + condition?: string; } export interface HistoryFile { diff --git a/src/index.ts b/src/index.ts index 250ccb7a..ffdd9d25 100644 --- a/src/index.ts +++ b/src/index.ts @@ -16,7 +16,7 @@ import { resolve } from "node:path"; import { pathToFileURL } from "node:url"; -import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; +import { serveStdio } from "@modelcontextprotocol/server/stdio"; import { GUEST_ROLE, resolveAccess } from "./access/rbac.js"; import { materializeEdges } from "./curation/edges.js"; import { materializeStagedActions } from "./curation/staged-actions.js"; @@ -33,9 +33,16 @@ import { reindexVault, reindexWarnings, } from "./search/reindex.js"; +import { getRerankProvider, setRerankProvider, warmRerankModel } from "./search/rerank-provider.js"; import { setProvider, warmModel } from "./search/vector.js"; import { startWatcher, type VaultWatcher } from "./search/watcher.js"; -import { createServer, resolveToolExposure, SERVER_VERSION } from "./server.js"; +import { + advertisedSurfaceCost, + allRegisteredTools, + createServer, + resolveToolExposure, + SERVER_VERSION, +} from "./server.js"; import { directoryExists } from "./storage/local.js"; import { loadConfig, TOOL_TIERS, type ToolTier } from "./utils/config.js"; @@ -102,9 +109,17 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise exposedNames.has(t.name)); + const cost = advertisedSurfaceCost(exposedTools); + process.stderr.write( + `daftari: advertising ${exposedTools.length} tools (~${cost} tokens of definitions; ` + + `tier=${toolsConfig.tier}). A future major release will default tools.tier to 'core' — ` + + "vault_tools makes every tool discoverable in-band.\n", + ); + } + // The persisted index is a derived cache: if every file on disk matches the // manifest written by the last reindex, the on-disk index already reflects // the vault and we can skip the embedding pass entirely (~25 min on a @@ -173,13 +204,17 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise createServer(vaultRoot, access, toolsConfig)); process.stderr.write( `daftari: serving vault at ${vaultRoot} (stdio) — ` + `user=${access.user} role=${access.roleName}\n`, @@ -299,6 +334,21 @@ async function runBackgroundWarm(): Promise { } else { process.stderr.write(`daftari: warning: embedding warm-up failed: ${result.error.message}\n`); } + // After the embedder warms, warm the reranker too (spec Decision 8): the + // existing warmEmbeddings gate covers both models — "pay model cold-starts + // at startup, not on the first query" applies equally to either. A no-op + // when rerank.provider is "none". Never runs when the search path itself + // would trigger it lazily instead (C5) — this IS the eager path. + if (getRerankProvider() !== null) { + const rerankResult = await warmRerankModel(); + if (rerankResult.ok) { + process.stderr.write(`daftari: reranker model warm — ready for search\n`); + } else { + process.stderr.write( + `daftari: warning: reranker warm-up failed: ${rerankResult.error.message}\n`, + ); + } + } } async function runBackgroundReindex( diff --git a/src/search/bm25.ts b/src/search/bm25.ts index c2d8ecb7..b1714814 100644 --- a/src/search/bm25.ts +++ b/src/search/bm25.ts @@ -65,6 +65,33 @@ export function tokenize(text: string): string[] { .filter((t) => t.length > 1 && !STOPWORDS.has(t)); } +// Extracts quoted spans from the RAW query (before tokenize() strips the +// quotes) and turns each into an FTS5 phrase branch, for buildMatchQuery's +// phrase-emission step (spec 2026-07-26 fusion overhaul, Decision 2 — this +// is what makes the router's "quoted phrase → extreme-lexical" route +// actually true at the FTS5 layer: without it, buildMatchQuery already +// stripped the quotes by the time the extreme route's {bm25:1, vector:0} +// weights disabled the semantic ranker, leaving nothing to distinguish the +// phrase from token-scatter). A span survives only when it tokenizes to >=2 +// usable tokens — a single-token or empty quoted span degrades to today's +// behaviour (the prefix-OR branch already covers it). Deduplicated in +// caller order. +function phraseBranches(rawQuery: string): string[] { + const spans = rawQuery.match(/"[^"]*"/g) ?? []; + const branches: string[] = []; + const seen = new Set(); + for (const span of spans) { + const inner = span.slice(1, -1); + const phraseTokens = tokenize(inner); + if (phraseTokens.length < 2) continue; + const phrase = `"${phraseTokens.join(" ")}"`; + if (seen.has(phrase)) continue; + seen.add(phrase); + branches.push(phrase); + } + return branches; +} + // Builds an FTS5 MATCH query from a free-text user query. // // We tokenize the same way as `tokenize()`, then OR every term together as @@ -73,13 +100,22 @@ export function tokenize(text: string): string[] { // words; FTS5's porter tokenizer already collapses many of these on the // document side, so the prefix is mostly a query-side recall booster. // +// A quoted span of >= 2 usable tokens ALSO emits an FTS5 phrase branch +// (`"tok1 tok2"`) as an extra OR alternative alongside the prefix branches — +// never a replacement. This is strictly recall-non-shrinking (a superset +// query: every prefix-OR match the old query found still matches), but a +// document containing the exact phrase now additionally satisfies the +// phrase branch and BM25 scores it higher than a document where the terms +// merely scatter — which is what quoting a phrase is supposed to mean. +// // FTS5 query syntax is fragile in the face of user input: quotes, hyphens, // the bare words AND / OR / NOT, and the trailing `*` operator all have // meaning to the parser. We strip every character outside [a-zA-Z0-9_] // during tokenization (already done), so the only remaining hazard is the // reserved words. We bypass that by lower-casing every token — FTS5's // reserved words are matched case-sensitively in upper case, so `or` is -// just a search term. +// just a search term. The phrase branch is built from ALREADY-tokenized +// (lowercased, alphanumeric-only) terms, so it inherits the same safety. // // Returns null when the query yields no usable tokens (all-whitespace or // all-stopwords). Callers must treat null as "no lexical match possible" @@ -90,5 +126,6 @@ export function buildMatchQuery(query: string): string | null { // Deduplicate to keep the MATCH string short. Prefix every token with `*` // for partial matches. const unique = [...new Set(tokens)]; - return unique.map((t) => `${t}*`).join(" OR "); + const branches = [...unique.map((t) => `${t}*`), ...phraseBranches(query)]; + return branches.join(" OR "); } diff --git a/src/search/embedding-provider.ts b/src/search/embedding-provider.ts index 8002d214..3d676e98 100644 --- a/src/search/embedding-provider.ts +++ b/src/search/embedding-provider.ts @@ -10,27 +10,58 @@ // anything; the old model's rows stay put and the new model populates its // own row set. // -// Contract: -// - `id` is what gets written to embeddings.model. Two providers with the -// same id would corrupt the cache; treat it as a stable namespace. -// - `dim` is the vector dimension. The cache stores it per row as defense- -// in-depth, but the model id alone scopes the join, so mixed-dim vectors -// for the same model id are a bug — not an expected runtime state. +// Contract (extended 2026-07-26 embedding-refresh-quantization spec, Phase +// 1b — Matryoshka truncation + int8 quantization): +// - `id` is what gets written to embeddings.model AND the durable +// `embeddings` cache key. Two providers with the same id would corrupt +// the cache; treat it as a stable namespace. For a Matryoshka-truncatable +// provider `id` is DIM-FREE (e.g. "local-embeddinggemma#p1") — the cache +// stores the full native-dim vector once, and `dim` below is purely the +// INDEX-time truncation target, not part of the cache identity. The +// trailing `#pN` component is the provider's prompt-format revision: +// bump it whenever the asymmetric prefix strings change so a prefix fix +// behaves like a provider switch (cache miss, re-embed) rather than +// silently leaving stale rows under a still-valid-looking id. +// - `dim` is the CONFIGURED index dimension — the vec table width and the +// length of vectors `embedQuery` returns. The cache stores the native +// dim per row as defense-in-depth (see `nativeDim`), but the model id +// alone scopes the join, so mixed-dim vectors for the same model id are +// a bug — not an expected runtime state. +// - `nativeDim` is the provider's full (untruncated) output width. Absent +// (or equal to `dim`) means the provider has no Matryoshka truncation — +// `embed()`'s output is already at `dim`. When present and greater than +// `dim`, `embed()` returns NATIVE-dim vectors and callers apply +// `toIndexDim` (src/search/vector.ts) at the single choke point where a +// vector meets the index or a query. // - `warm()` is the eager-load entry point. For providers with no warm-up // cost (e.g. a stateless HTTP client), it can be a no-op that returns ok. // - `embed()` returns one Float32Array per input text, in input order, all -// of length `dim`. `onProgress` (if given) fires after each sub-batch. -// Errors are returned as Result.err — the caller (reindex / search) is -// responsible for degrading gracefully to lexical-only ranking. +// of NATIVE dim (== `dim` when `nativeDim` is absent). Doc-side prompt +// prefixing (if any) is applied internally. `onProgress` (if given) +// fires after each sub-batch. Errors are returned as Result.err — the +// caller (reindex / search) is responsible for degrading gracefully to +// lexical-only ranking. +// - `embedQuery()` is the query-side counterpart: applies the provider's +// query prompt prefix (if any) and returns a vector already at +// CONFIGURED `dim` (native output truncated via `toIndexDim` +// internally). Providers with no asymmetric prefix and no truncation +// may omit it — callers fall back to `embed([text])` + `toIndexDim`. +// - `isLoaded()` reports whether the underlying model is resident in +// memory. Absent means "always loaded" (stateless providers, e.g. an +// HTTP client) — `isModelLoaded()` in vector.ts treats a missing +// `isLoaded` as `true`. import type { Result } from "../frontmatter/types.js"; export interface EmbeddingProvider { readonly id: string; readonly dim: number; + readonly nativeDim?: number; warm(): Promise>; embed( texts: string[], onProgress?: (done: number, total: number) => void, ): Promise>; + embedQuery?(text: string): Promise>; + isLoaded?(): boolean; } diff --git a/src/search/hybrid.ts b/src/search/hybrid.ts index 2e964216..10b6f0d0 100644 --- a/src/search/hybrid.ts +++ b/src/search/hybrid.ts @@ -8,9 +8,21 @@ // virtual table, joining back to `chunks` to map content hashes onto // document paths. // -// Each ranker still produces raw scores on its own scale, so both are -// min-normalised to [0, 1] (divide by the top score) before being mixed by -// weight. Default weighting is an even 0.5 / 0.5 split. +// Each ranker still produces raw scores on its own scale. Two fusion modes +// combine them (spec 2026-07-26 fusion overhaul, Decision 1): +// - "weighted" (relatedSearch's default): both halves are min-normalised +// to [0, 1] (divide by the top score) and mixed by weight. +// - "rrf" (hybridSearch's default): each half is converted to a rank list +// and mixed via reciprocal rank fusion at k=60, SCALED by (k+1) so a +// rank-1 contribution is 1.0 rather than textbook RRF's 1/61 — ordering- +// identical to textbook RRF, but keeps fused scores in (0, 1] with a +// top≈1 scale for downstream consumers (summaryLine's toFixed(3), the +// rerank pool, vault hooks) that already calibrate against the weighted +// mode's range. `bm25Score`/`vectorScore` on each hit carry these +// per-ranker contributions (weighted: normalised score; rrf: scaled +// reciprocal rank), unweighted; `score` applies the weights on top. +// +// Default weighting is an even 0.5 / 0.5 split. // // Vector ranking is best-effort. If the query cannot be embedded (model // unavailable) or the index holds no embeddings, the search degrades to @@ -20,17 +32,27 @@ import { computeDecay, type DecayState } from "../curation/decay.js"; import type { ValidityReport } from "../curation/validity.js"; import { err, ok, type Result } from "../frontmatter/types.js"; import { + blobToEmbedding, embeddingToBlob, getChunksForPath, getDocument, getDocumentsByPaths, type IndexDb, + quantizeInt8, + type VecKind, } from "../storage/index-db.js"; import { buildMatchQuery, tokenize } from "./bm25.js"; import type { ContestedTension } from "./contested.js"; import type { CurrentSource } from "./current-source.js"; import type { ValidAtSource } from "./valid-at-source.js"; -import { embedQuery, getProvider, meanEmbedding } from "./vector.js"; +import { + cosineSimilarity, + embedQuery, + getProvider, + getQuantize, + meanEmbedding, + toIndexDim, +} from "./vector.js"; export interface HybridWeights { bm25: number; @@ -39,6 +61,27 @@ export interface HybridWeights { export const DEFAULT_WEIGHTS: HybridWeights = { bm25: 0.5, vector: 0.5 }; +// Fusion mode: how the lexical and vector rank lists are combined into one +// fused score (spec 2026-07-26 fusion overhaul, Decision 1). See the file +// header for the score-scale rationale. +export type FusionMode = "weighted" | "rrf"; + +// hybridSearch's own default. Flipped to "rrf" in PR 3, gated on the +// fusion-runner.mjs bench (docs/superpowers/specs/2026-07-26-retrieval- +// fusion-overhaul-design.md). +export const DEFAULT_FUSION: FusionMode = "weighted"; + +// relatedSearch's own default. Deliberately NOT flipped alongside +// DEFAULT_FUSION: relatedSearch is a materially different fusion problem (up +// to 64 prefix-OR'd source tokens, document granularity) with no bench arm +// exercising it. Callers can still opt in via `fusion: "rrf"`. +const RELATED_DEFAULT_FUSION: FusionMode = "weighted"; + +// RRF's rank-damping constant. Module-private and not configurable — RRF's +// whole appeal is that it needs no tuning; k=60 is the standard literature +// default. +const RRF_K = 60; + export interface HybridHit { path: string; title: string; @@ -108,6 +151,20 @@ export interface HybridSearchResult { // protocol, it never calls a model (the same agent-as-judge division the // tier-2 protocol settled). Tool handler, not ranker. rerank?: { instructions: string; candidates: RerankCandidate[] }; + // Part B (local cross-encoder reranker, spec 2026-07-26-contextual- + // chunking-reranker-design.md Decision 5/7). Set by the TOOL HANDLER, not + // this ranker — hybridSearch itself never reranks. `false` covers every + // degrade path uniformly: provider `none`, not-warm skip, inference + // Result.err, and timeout — the honest twin of `vectorUsed`. Absent from + // relatedSearch's result (no rerank stage there, spec exclusion). + rerankUsed?: boolean; + // Internal transport only (Part B, C2/C4): populated when the caller + // requested `capturePassageRefs`, one entry per hit in `hits`. The tool + // handler consumes this to resolve passage TEXT for exactly the rerank + // pool, then strips it before returning — outputSchema declares + // additionalProperties: false, and leaking synthesized index-layer refs + // would fail client-side validation anyway. + passageRefs?: Record; } const SNIPPET_RADIUS = 140; @@ -121,6 +178,16 @@ const SNIPPET_RADIUS = 140; // counts grow into the millions. const VEC_KNN_K = 64; +// Over-fetch multiplier for the int8 scan-then-rescore path (spec 2026-07-26 +// embedding-refresh-quantization Decision 3, the Sentence Transformers +// rescore-multiplier convention). The KNN scan asks sqlite-vec for +// VEC_KNN_K * RESCORE_MULTIPLIER quantized-distance candidates; every +// candidate is then rescored with exact float32 cosine against the durable +// cache, and the final top-VEC_KNN_K comes from the RESCORED order — the +// quantized distance is used for candidate selection only, never as a score +// (disposition C3). +const RESCORE_MULTIPLIER = 4; + // Pulls a readable excerpt from a document body, centred on the earliest // occurrence of any query term. Falls back to the document head when no term // is found (e.g. a purely semantic match). @@ -158,28 +225,64 @@ function normalize(scores: Map): Map { return new Map([...scores].map(([k, v]) => [k, v / max])); } +// Rank-list construction for RRF. Sorts a score map descending (ties broken +// by path ascending, for deterministic ranks) and maps each entry to its +// scaled reciprocal-rank contribution (RRF_K + 1) / (RRF_K + rank), rank +// 1-based. The (RRF_K + 1) numerator is a constant multiple of textbook +// 1/(k + rank): ordering-identical, but contributions live in (0, 1] with +// rank 1 = 1.0, so fused scores keep the top≈1 scale downstream consumers +// (summaryLine's toFixed(3), the rerank pool, vault hooks) already +// calibrate against. An empty map returns an empty map. +function rrfContributions(scores: Map): Map { + const ranked = [...scores.entries()].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])); + const out = new Map(); + ranked.forEach(([path], i) => { + const rank = i + 1; + out.set(path, (RRF_K + 1) / (RRF_K + rank)); + }); + return out; +} + // Wraps a prefix-OR'd FTS match string in an FTS5 column filter (e.g. "{title tags}"). Null query → null. function columnRestrict(matchQuery: string | null, columns: string): string | null { return matchQuery === null ? null : `${columns} : (${matchQuery})`; } -// Band boundary for the chunk-mode tiered lexical combine. Documents with any -// chunk-body match occupy the upper band (0.5, 1]; documents matched only via -// title/tags occupy the lower band (0, 0.5]. A body match therefore always -// outranks a title-only match by construction — no tunable weight. (Any split -// in (0,1) gives the same strict ordering; 0.5 is the natural midpoint.) +// Band boundary for the chunk-mode tiered lexical combine. +// +// Post-contextual-chunking semantics (spec 2026-07-26-contextual-chunking- +// reranker-design.md, Decision 2 — READ THIS BEFORE CHANGING TIER_SPLIT or +// tieredLexical): every chunk's context column now carries the document's +// title, collection, and tags, so a title- or tag-matching query enters the +// UPPER band via a genuine chunk match (chunkNorm), not just the lower-band +// fallback below. The upper band's meaning has therefore shifted from "any +// BODY match" to "any chunk match, including a context-only match" — the +// strict "body outranks title-only" guarantee now holds only for docs with +// NO context-column match at all. This is the mechanism the spec describes, +// not a bug: bm25(chunks_fts) spans both columns by default weight, which +// *is* contextual BM25. +// +// The `{title tags}` fallback tier below stays — it is spec Decision 2's +// explicitly-kept "strict, harmless fallback" for a doc whose chunks are +// somehow absent from chunks_fts (an index inconsistency, not the common +// case) — largely redundant now that title/tag tokens flow through the +// context column, but its retirement is deferred to the 2026-06-24 +// chunk-BM25 native/title-tag regression suites per the spec's own text. const TIER_SPLIT = 0.5; -// Tiered combine of two normalized lexical signals. chunkNorm (body) is primary: -// its docs land in the upper band, ordered by body score. titleTagNorm docs that -// are NOT already body-matched land in the lower band, ordered by title/tag score -// — a strict fallback that surfaces docs the body ranker missed (the native -// title/tag case) without ever displacing a real body match (the RB case). -// Both inputs are normalized to (0,1] (no zeros) by the callers, so upper band -// is strictly >0.5 and lower band is <=0.5: strict, tie-free separation. The -// `> 0` guards make that precondition self-enforcing rather than relying on the -// upstream invariant — a non-positive score never creates a band entry (so it -// can't floor a body match to exactly 0.5 and tie a title-only match). +// Tiered combine of two normalized lexical signals. chunkNorm is primary — any +// document with a real chunk match (body OR, since contextual chunking, +// title/collection/tag tokens via the context column) lands in the upper band, +// ordered by its chunk bm25 score. titleTagNorm docs that are NOT already +// chunk-matched land in the lower band, ordered by title/tag score — a strict +// fallback that surfaces docs the chunk ranker missed entirely (e.g. its +// chunks are absent from chunks_fts) without ever displacing a real chunk +// match. Both inputs are normalized to (0,1] (no zeros) by the callers, so +// upper band is strictly >0.5 and lower band is <=0.5: strict, tie-free +// separation. The `> 0` guards make that precondition self-enforcing rather +// than relying on the upstream invariant — a non-positive score never creates +// a band entry (so it can't floor a chunk match to exactly 0.5 and tie a +// title-only match). function tieredLexical( chunkNorm: Map, titleTagNorm: Map, @@ -236,39 +339,108 @@ function ftsRanking(db: IndexDb, query: string | null): Map { // candidate set, no remainder is computed or reported, and the result is // shaped exactly as it would be in a vault where those collections do not // exist (2026-07-14 spec). +// `bestHash` (Part B, C2/C4): the content_hash of each path's best-similarity +// chunk — a cheap ref, not the chunk text itself. Resolving passage TEXT for +// the whole over-fetched candidate set would pay per-candidate joins for an +// O(collection)-sized set to use ~50 (C2); the tool handler resolves text +// only for the top RERANK_POOL permitted hits via getChunkByPathAndHash. +// `kind` selects the scoring path (spec Decision 3/C3): +// - "float32": byte-for-byte today's behaviour — sqlite-vec's own distance +// IS the score (1 - distance, clamped to [0, 1]). +// - "int8": scan-then-rescore. The KNN scan over the quantized column +// retrieves VEC_KNN_K * RESCORE_MULTIPLIER candidates by quantized +// distance (candidate SELECTION only); each candidate is then rescored +// with exact float32 cosine against the durable cache, joined in the +// same statement by content_hash + model. A candidate whose cache row +// is missing (a gc race between the vec mirror and the cache) is +// DROPPED from the ranking, not approximated with the quantized +// distance — the over-fetch makes dropping cheap and BM25 still carries +// the document. There is no distance -> score conversion on this path +// at all. function vecRanking( db: IndexDb, queryEmbedding: Float32Array, modelId: string, + kind: VecKind, readableCollections?: string[], -): Map { - const queryBlob = embeddingToBlob(queryEmbedding); - if (readableCollections !== undefined && readableCollections.length === 0) return new Map(); +): { scores: Map; bestHash: Map } { + if (readableCollections !== undefined && readableCollections.length === 0) { + return { scores: new Map(), bestHash: new Map() }; + } const collectionFilter = readableCollections === undefined ? "" : ` AND v.collection IN (${readableCollections.map(() => "?").join(",")})`; + + if (kind === "float32") { + const queryBlob = embeddingToBlob(queryEmbedding); + const rows = db + .prepare( + `SELECT c.path AS path, v.content_hash AS content_hash, v.distance AS distance + FROM embeddings_vec AS v + JOIN chunks AS c ON c.content_hash = v.content_hash + WHERE v.embedding MATCH ? + AND v.model = ? + AND v.k = ?${collectionFilter} + ORDER BY v.distance`, + ) + .all(queryBlob, modelId, VEC_KNN_K, ...(readableCollections ?? [])) as { + path: string; + content_hash: string; + distance: number; + }[]; + const scores = new Map(); + const bestHash = new Map(); + for (const r of rows) { + const sim = Math.max(0, 1 - r.distance); + const prev = scores.get(r.path) ?? -Infinity; + if (sim > prev) { + scores.set(r.path, sim); + bestHash.set(r.path, r.content_hash); + } + } + return { scores, bestHash }; + } + + // int8 path: over-fetch by RESCORE_MULTIPLIER, rescore exact-cosine + // against the durable cache (joined in-statement), drop orphans. + // + // `vec_int8(?)` wraps the MATCH parameter — sqlite-vec infers a bound + // blob's vector type from its byte layout (defaults to float32), so an + // unwrapped int8-byte blob against an int8[] column is rejected ("Query + // vector ... expected to be of type int8, but a float32 vector was + // provided"). Confirmed empirically against the pinned sqlite-vec build. + const queryBlob = quantizeInt8(queryEmbedding); + const overFetchK = VEC_KNN_K * RESCORE_MULTIPLIER; const rows = db .prepare( - `SELECT c.path AS path, v.distance AS distance + `SELECT c.path AS path, v.content_hash AS content_hash, e.embedding AS cache_embedding FROM embeddings_vec AS v JOIN chunks AS c ON c.content_hash = v.content_hash - WHERE v.embedding MATCH ? + LEFT JOIN embeddings AS e + ON e.content_hash = v.content_hash AND e.model = v.model + WHERE v.embedding MATCH vec_int8(?) AND v.model = ? - AND v.k = ?${collectionFilter} - ORDER BY v.distance`, + AND v.k = ?${collectionFilter}`, ) - .all(queryBlob, modelId, VEC_KNN_K, ...(readableCollections ?? [])) as { + .all(queryBlob, modelId, overFetchK, ...(readableCollections ?? [])) as { path: string; - distance: number; + content_hash: string; + cache_embedding: Buffer | null; }[]; - const result = new Map(); + const scores = new Map(); + const bestHash = new Map(); for (const r of rows) { - const sim = Math.max(0, 1 - r.distance); - const prev = result.get(r.path) ?? -Infinity; - if (sim > prev) result.set(r.path, sim); + if (!r.cache_embedding) continue; // orphan candidate — dropped, not approximated (C3) + const cached = toIndexDim(blobToEmbedding(r.cache_embedding), queryEmbedding.length); + const sim = Math.max(0, cosineSimilarity(queryEmbedding, cached)); + const prev = scores.get(r.path) ?? -Infinity; + if (sim > prev) { + scores.set(r.path, sim); + bestHash.set(r.path, r.content_hash); + } } - return result; + return { scores, bestHash }; } // snippet() excerpt budget, in tokens. ~48 stemmed tokens lands near the @@ -294,8 +466,8 @@ const FTS_SNIPPET_TOKENS = 48; function chunkFtsRanking( db: IndexDb, query: string | null, -): { scores: Map; snippets: Map } { - if (query === null) return { scores: new Map(), snippets: new Map() }; +): { scores: Map; snippets: Map; winners: Map } { + if (query === null) return { scores: new Map(), snippets: new Map(), winners: new Map() }; const rows = db .prepare( `SELECT c.path AS path, chunks_fts.rowid AS crowid, -bm25(chunks_fts) AS score @@ -311,7 +483,9 @@ function chunkFtsRanking( // paid once per DOCUMENT, not once per matched chunk. (A ROW_NUMBER() // window subquery would not help here: the projected snippet() is still // evaluated per inner row before the window filter, and FTS5 auxiliary - // functions cannot move outside the MATCH cursor.) + // functions cannot move outside the MATCH cursor.) Also returned to the + // caller (Part B, C2/C4): the reranker's passage resolution reuses these + // same rowids via getChunkTextsByRowids instead of re-deriving a winner. const winners = new Map(); for (const r of rows) { // Some rows may produce a non-positive flipped score if FTS5 returned a @@ -336,9 +510,15 @@ function chunkFtsRanking( if (winners.size > 0) { const ids = [...winners.values()]; const placeholders = ids.map(() => "?").join(","); + // Column 1 (`text`) ONLY — spec Decision 4. chunks_fts is (context, text); + // targeting column 1 means a served snippet can never contain the + // synthesized breadcrumb, even when the query matched ONLY in the context + // column (a title/tag-only match). That case's snippet degrades to the + // chunk's leading body text — acceptable, and strictly better than + // showing invented lines. const snips = db .prepare( - `SELECT c.path AS path, snippet(chunks_fts, 0, '', '', '…', ?) AS snip + `SELECT c.path AS path, snippet(chunks_fts, 1, '', '', '…', ?) AS snip FROM chunks_fts JOIN chunks AS c ON c.rowid = chunks_fts.rowid WHERE chunks_fts MATCH ? AND chunks_fts.rowid IN (${placeholders})`, @@ -349,7 +529,47 @@ function chunkFtsRanking( if (collapsed.length > 0) snippets.set(s.path, collapsed); } } - return { scores, snippets }; + return { scores, snippets, winners }; +} + +// Part B (reranker) passage reference — a cheap POINTER to the chunk that +// carried a hit's ranking, not the chunk text itself (C2: resolving text for +// the whole over-fetched candidate set would pay per-candidate joins for a +// set sized O(collection) to use ~50). The tool handler resolves text for +// exactly the top RERANK_POOL permitted hits via the storage-layer lookups +// (getChunkTextsByRowids / getChunkByPathAndHash / getFirstChunk). +export type PassageRef = + | { kind: "lexical"; rowid: number } + | { kind: "vector"; contentHash: string } + | { kind: "first" }; + +// Provenance choice (C4): a hit with both a lexical winner and a KNN-best +// chunk presents the chunk from whichever signal contributed the higher +// NORMALIZED score for that path — the cross-encoder judges the document on +// the chunk that is the reason it ranked. `lexicalNorm`/`vecNorm` are the +// within-ranker normalized (0,1] maps, independent of fusion mode (weighted +// vs RRF is a downstream combination detail, not a provenance decision). +// Only-one-signal-present → that one. Neither (a title/tag-tier-only hit, +// or document-granularity search) → the terminal `first` fallback. +function choosePassageRef( + path: string, + lexicalNorm: Map, + winners: Map, + vecNorm: Map, + bestHash: Map, +): PassageRef { + const lex = lexicalNorm.get(path) ?? 0; + const vec = vecNorm.get(path) ?? 0; + const hasLex = lex > 0 && winners.has(path); + const hasVec = vec > 0 && bestHash.has(path); + if (hasLex && hasVec) { + return vec > lex + ? { kind: "vector", contentHash: bestHash.get(path) as string } + : { kind: "lexical", rowid: winners.get(path) as number }; + } + if (hasLex) return { kind: "lexical", rowid: winners.get(path) as number }; + if (hasVec) return { kind: "vector", contentHash: bestHash.get(path) as string }; + return { kind: "first" }; } interface RankOptions { @@ -359,6 +579,11 @@ interface RankOptions { lexicalGranularity: "document" | "chunk"; // Readable-collection allow-list pushed into the KNN scan; see vecRanking. readableCollections?: string[]; + fusion: FusionMode; + // Part B: attach a cheap PassageRef per hit (see choosePassageRef) instead + // of resolving passage text here. Off by default — ref capture is wasted + // work when no reranker is configured (C2's "skip ref capture" revision). + capturePassageRefs?: boolean; } // Core ranker shared by query search and related-document search. @@ -373,20 +598,30 @@ function rankDocuments( queryEmbedding: Float32Array | null, queryTokensForSnippet: string[], opts: RankOptions, -): { hits: HybridHit[]; vectorUsed: boolean } { +): { hits: HybridHit[]; vectorUsed: boolean; passageRefs: Map } { let bm25Norm: Map; // Best-chunk excerpts from the lexical pass (#108); empty for the // document-granularity path, whose hits fall back to the JS scan. let lexicalSnippets = new Map(); + // Raw (within-ranker-normalized) lexical/chunk signals, kept around ONLY + // for choosePassageRef — bm25Norm below is the TIERED combine that feeds + // the actual ranking; the passage-provenance choice wants the un-tiered + // chunk signal specifically (C4). + let chunkNormForRefs = new Map(); + let chunkWinners = new Map(); if (opts.lexicalGranularity === "chunk") { // Body granularity (the dilution fix) TIERED with a clean title/tag signal // (the native-shape fix). Each is normalized to its own max to reconcile the - // two FTS score scales; tieredLexical then ranks every body match above every - // title-only match. The title/tag signal reuses ftsRanking with a column- - // restricted query so it scores title+tags only (no body dilution). + // two FTS score scales; tieredLexical then ranks every chunk match (which, + // since contextual chunking, includes title/collection/tag-only matches via + // the context column — see the TIER_SPLIT comment) above every doc the + // chunk ranker missed entirely. The title/tag signal reuses ftsRanking with + // a column-restricted query so it scores title+tags only (no body dilution). const chunkRanked = chunkFtsRanking(db, matchQuery); lexicalSnippets = chunkRanked.snippets; + chunkWinners = chunkRanked.winners; const chunkNorm = normalize(chunkRanked.scores); + chunkNormForRefs = chunkNorm; const titleTagNorm = normalize(ftsRanking(db, columnRestrict(matchQuery, "{title tags}"))); bm25Norm = tieredLexical(chunkNorm, titleTagNorm); } else { @@ -395,17 +630,34 @@ function rankDocuments( let vectorRaw = new Map(); let vectorUsed = false; + let vecBestHash = new Map(); if (queryEmbedding) { const provider = getProvider(); - vectorRaw = vecRanking(db, queryEmbedding, provider.id, opts.readableCollections); + const vecRanked = vecRanking( + db, + queryEmbedding, + provider.id, + getQuantize(), + opts.readableCollections, + ); + vectorRaw = vecRanked.scores; + vecBestHash = vecRanked.bestHash; if (vectorRaw.size > 0) vectorUsed = true; } - const vectorNorm = normalize(vectorRaw); // With no usable vector signal, lexical ranking carries the full weight. const weights: HybridWeights = vectorUsed ? opts.weights : { bm25: 1, vector: 0 }; - const candidates = new Set([...bm25Norm.keys(), ...vectorNorm.keys()]); + // The two fusion modes differ only in how the lexical map (bm25Norm, built + // above — untouched by fusion mode) meets the vector map. "weighted" keeps + // today's cross-ranker normalize(); "rrf" replaces both with rank-based + // scaled-reciprocal-rank contributions and never normalizes across + // rankers. See rrfContributions and the file header. + const lexScores = opts.fusion === "rrf" ? rrfContributions(bm25Norm) : bm25Norm; + const vecNormForScore = normalize(vectorRaw); + const vecScores = opts.fusion === "rrf" ? rrfContributions(vectorRaw) : vecNormForScore; + + const candidates = new Set([...lexScores.keys(), ...vecScores.keys()]); // Fetch full rows for ONLY the candidate paths — not the whole vault. The // FTS + vector rankers above have already collapsed the vault to a small set @@ -417,12 +669,13 @@ function rankDocuments( const byPath = new Map(getDocumentsByPaths(db, fetchPaths).map((d) => [d.path, d])); const hits: HybridHit[] = []; + const passageRefs = new Map(); for (const path of candidates) { if (path === opts.excludePath) continue; const doc = byPath.get(path); if (!doc) continue; - const bm25Score = bm25Norm.get(path) ?? 0; - const vectorScore = vectorNorm.get(path) ?? 0; + const bm25Score = lexScores.get(path) ?? 0; + const vectorScore = vecScores.get(path) ?? 0; const score = weights.bm25 * bm25Score + weights.vector * vectorScore; if (score <= 0) continue; hits.push({ @@ -446,10 +699,31 @@ function rankDocuments( superseded_by: doc.supersededBy, }), }); + if (opts.capturePassageRefs) { + passageRefs.set( + path, + choosePassageRef(path, chunkNormForRefs, chunkWinners, vecNormForScore, vecBestHash), + ); + } } - hits.sort((a, b) => b.score - a.score); - return { hits: hits.slice(0, opts.limit), vectorUsed }; + // Deterministic tie-break in BOTH modes: exact fused-score ties are common + // under RRF (many candidates share the same rank-derived contribution), + // and insertion order otherwise descends from SQL row order with no + // cross-run guarantee. Benign for weighted mode too — it only reorders + // exact ties, which SQL row order previously broke arbitrarily. + hits.sort((a, b) => b.score - a.score || a.path.localeCompare(b.path)); + const sliced = hits.slice(0, opts.limit); + // Trim passageRefs to exactly the paths in the sliced result — the map was + // built over the full candidate set above. + const slicedRefs = new Map(); + if (opts.capturePassageRefs) { + for (const h of sliced) { + const ref = passageRefs.get(h.path); + if (ref) slicedRefs.set(h.path, ref); + } + } + return { hits: sliced, vectorUsed, passageRefs: slicedRefs }; } export interface HybridSearchOptions { @@ -470,6 +744,16 @@ export interface HybridSearchOptions { // the tool handler's post-rank canRead filter remains the authorization // boundary either way, and still covers the lexical half. readableCollections?: string[]; + // Fusion mode (spec 2026-07-26 fusion overhaul, Decision 1). Library-level + // option — no MCP tool argument grows for it. hybridSearch defaults to + // DEFAULT_FUSION; relatedSearch defaults to its own RELATED_DEFAULT_FUSION. + fusion?: FusionMode; + // Part B: attach a PassageRef per hit so the tool handler can resolve + // rerank passage text without rankDocuments paying per-candidate joins for + // the whole over-fetched set (C2). Only vaultSearch sets this, and only + // when a reranker is actually configured — ref capture is wasted work + // otherwise. + capturePassageRefs?: boolean; } // Ranks vault documents against a free-text query. @@ -480,6 +764,7 @@ export async function hybridSearch( ): Promise> { const weights = options.weights ?? DEFAULT_WEIGHTS; const limit = options.limit ?? 10; + const fusion = options.fusion ?? DEFAULT_FUSION; // Default flipped to "chunk" in v1.29.0: chunk-level BM25 recovers the // multi-topic-document dilution gap (RB recall + SQuAD retrieval) and produces // better end-to-end answers where it out-retrieves document, with no regression @@ -500,13 +785,21 @@ export async function hybridSearch( queryEmbedding = embedResult.ok ? embedResult.value : null; } - const { hits, vectorUsed } = rankDocuments(db, matchQuery, queryEmbedding, snippetTokens, { - weights, - limit: rankLimit, - excludePath: undefined, - lexicalGranularity, - readableCollections: options.readableCollections, - }); + const { hits, vectorUsed, passageRefs } = rankDocuments( + db, + matchQuery, + queryEmbedding, + snippetTokens, + { + weights, + limit: rankLimit, + excludePath: undefined, + lexicalGranularity, + readableCollections: options.readableCollections, + fusion, + capturePassageRefs: options.capturePassageRefs, + }, + ); return ok({ query, @@ -514,6 +807,7 @@ export async function hybridSearch( vectorUsed, weights: vectorUsed ? weights : { bm25: 1, vector: 0 }, hits, + ...(options.capturePassageRefs ? { passageRefs: Object.fromEntries(passageRefs) } : {}), }); } @@ -537,6 +831,7 @@ export function relatedSearch( ): Result { const weights = options.weights ?? DEFAULT_WEIGHTS; const limit = options.limit ?? 10; + const fusion = options.fusion ?? RELATED_DEFAULT_FUSION; // See hybridSearch: over-fetch lets the RBAC-filtering tool handler drop // restricted hits before slicing to the user-facing limit. const rankLimit = options.overFetch ? Number.POSITIVE_INFINITY : limit; @@ -547,9 +842,20 @@ export function relatedSearch( } const provider = getProvider(); - const chunkVectors = getChunksForPath(db, path, provider.id, provider.dim) + // getChunksForPath's expectedDim guard reads the DURABLE cache, which + // stores NATIVE-dim vectors (2026-07-26 embedding-refresh-quantization + // spec, disposition C9) — pass nativeDim (falling back to dim for + // providers with no Matryoshka gap), not the configured index dim, or + // every row would fail the dim guard and relatedSearch would silently see + // no vectors at all. Each chunk vector is then truncated to the + // CONFIGURED dim via toIndexDim before meanEmbedding averages them (C9: + // "meanEmbedding inputs pass through toIndexDim") — the mean of unit + // vectors stays in [-1, 1], so the later quantizeInt8 rescore path applies + // unchanged. + const chunkVectors = getChunksForPath(db, path, provider.id, provider.nativeDim ?? provider.dim) .map((c) => c.embedding) - .filter((e): e is Float32Array => e !== null); + .filter((e): e is Float32Array => e !== null) + .map((e) => toIndexDim(e, provider.dim)); const queryEmbedding = meanEmbedding(chunkVectors); // Build the FTS5 match string from the source document's stored token @@ -570,6 +876,7 @@ export function relatedSearch( excludePath: path, lexicalGranularity: "document", readableCollections: options.readableCollections, + fusion, }); return ok({ diff --git a/src/search/providers/local-bge-m3.ts b/src/search/providers/local-bge-m3.ts new file mode 100644 index 00000000..06f56e80 --- /dev/null +++ b/src/search/providers/local-bge-m3.ts @@ -0,0 +1,146 @@ +// local-bge-m3 — BAAI/bge-reranker-v2-m3 cross-encoder, ONNX q8, run locally +// via @huggingface/transformers (onnx-community/bge-reranker-v2-m3-ONNX). +// Zero new dependencies: @huggingface/transformers is already ^4.2.0 +// (package.json), the same runtime local-minilm.ts uses for embeddings. +// +// Score = sigmoid of the single logit per (query, passage) pair, tokenized +// as a (query, passage) text_pair and scored in fixed sub-batches of +// RERANK_BATCH_SIZE — the same peak-memory argument as local-minilm's +// EMBED_BATCH_SIZE: an unbounded batch pads every pair to the batch's +// longest sequence, so peak activation memory would scale with the whole +// rerank pool (bounded at RERANK_POOL, src/tools/search.ts) rather than +// staying flat. +// +// The model loads lazily and is memoised for the process; a warm-up entry +// point exists so the server can pay that cost in the background rather +// than on the first rerank-enabled search — spec Decision 8, and the C5 +// revision that the search path must NEVER trigger a synchronous model load +// inside a tool call (isReady() gates that; see rerank-provider.ts and +// src/tools/search.ts). A load or inference failure returns Result.err so +// the caller degrades to the fused order — reranking, like embedding, is +// never load-bearing for the server staying up. +// +// Deliberately does NOT touch index-state.ts's model markers: those narrate +// the EMBEDDING model's warming lifecycle for tools that distinguish +// "warming embeddings" from "indexing"; a warming reranker is a separate +// concern that never blocks or is blocked by indexing (plan §3.2). + +import { err, ok, type Result } from "../../frontmatter/types.js"; +import type { RerankProvider } from "../rerank-provider.js"; + +export const LOCAL_BGE_M3_ID = "local-bge-m3"; +const HF_MODEL = "onnx-community/bge-reranker-v2-m3-ONNX"; + +// Sub-batch size for (query, passage) pairs scored per model call. See file +// header — mirrors local-minilm.ts's EMBED_BATCH_SIZE rationale exactly. +const RERANK_BATCH_SIZE = 8; + +// Minimal shape of the transformers.js pieces this module actually calls, +// so the rest of the file (and its tests, via the RerankProvider seam) never +// depends on @huggingface/transformers' full type surface. +interface TokenizedInputs { + input_ids: unknown; + attention_mask: unknown; +} +type Tokenizer = ( + queries: string[], + opts: { text_pair: string[]; padding: boolean; truncation: boolean }, +) => TokenizedInputs; +type SequenceClassifier = ( + inputs: TokenizedInputs, +) => Promise<{ logits: { data: ArrayLike } }>; + +interface Model { + tokenizer: Tokenizer; + classify: SequenceClassifier; +} + +let modelPromise: Promise | null = null; + +async function getModel(): Promise { + if (!modelPromise) { + modelPromise = ( + import("@huggingface/transformers") as Promise<{ + AutoTokenizer: { from_pretrained: (id: string) => Promise }; + AutoModelForSequenceClassification: { + from_pretrained: (id: string, opts: { dtype: string }) => Promise; + }; + }> + ) + .then(async ({ AutoTokenizer, AutoModelForSequenceClassification }) => { + const tokenizer = await AutoTokenizer.from_pretrained(HF_MODEL); + const classify = await AutoModelForSequenceClassification.from_pretrained(HF_MODEL, { + dtype: "q8", + }); + return { tokenizer, classify }; + }) + .then( + (model) => model, + (e) => { + // Reset the memoised promise so a later retry (e.g. network came + // back) can succeed — a single transient failure must not poison + // the process for its whole lifetime. Mirrors local-minilm.ts. + modelPromise = null; + throw e; + }, + ); + } + return modelPromise; +} + +// True once the model has been loaded into memory. This IS isReady() on the +// RerankProvider interface — the search path checks it before ever +// attempting a rerank, so reranking never triggers a cold model load inside +// a tool call (C5). +export function isLocalBgeM3Loaded(): boolean { + return modelPromise !== null; +} + +// Test-only: clear the memoised model so a fresh import is forced on the +// next call. Production code must not invoke this. +export function resetLocalBgeM3ForTests(): void { + modelPromise = null; +} + +function sigmoid(x: number): number { + return 1 / (1 + Math.exp(-x)); +} + +async function warm(): Promise> { + try { + await getModel(); + return ok(undefined); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`reranker model warm-up failed: ${reason}`)); + } +} + +async function rerank(query: string, passages: string[]): Promise> { + if (passages.length === 0) return ok([]); + try { + const model = await getModel(); + const scores: number[] = []; + for (let start = 0; start < passages.length; start += RERANK_BATCH_SIZE) { + const batch = passages.slice(start, start + RERANK_BATCH_SIZE); + const inputs = model.tokenizer( + batch.map(() => query), + { text_pair: batch, padding: true, truncation: true }, + ); + const output = await model.classify(inputs); + const logits = Array.from(output.logits.data); + for (const logit of logits) scores.push(sigmoid(logit)); + } + return ok(scores); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`rerank failed: ${reason}`)); + } +} + +export const localBgeM3Provider: RerankProvider = { + id: LOCAL_BGE_M3_ID, + isReady: isLocalBgeM3Loaded, + warm, + rerank, +}; diff --git a/src/search/providers/local-embeddinggemma.ts b/src/search/providers/local-embeddinggemma.ts new file mode 100644 index 00000000..2c6c9c17 --- /dev/null +++ b/src/search/providers/local-embeddinggemma.ts @@ -0,0 +1,86 @@ +// local-embeddinggemma — google/embeddinggemma-300m run locally via +// @huggingface/transformers, using the local-transformers.ts factory (spec +// 2026-07-26-embedding-refresh-quantization Decision 1/2, Phase 1c). +// 768d native, Matryoshka-truncatable to 512 (default) or 768. Mean pooling, +// asymmetric document/query prompt prefixes. + +import type { EmbeddingProvider } from "../embedding-provider.js"; +import { makeLocalTransformersProvider } from "./local-transformers.js"; + +export const LOCAL_EMBEDDINGGEMMA_ID_PREFIX = "local-embeddinggemma"; +const HF_MODEL = "onnx-community/embeddinggemma-300m-ONNX"; +const NATIVE_DIM = 768; +// Matryoshka-trained points this provider exposes. 512 first — the spec's +// default. 384 is deliberately NOT offered: not a trained Matryoshka point +// for this model, so truncating there is off-distribution (spec Decision 2). +export const LOCAL_EMBEDDINGGEMMA_DIMS = [512, 768] as const; + +// [TRAINING] The prefix strings below are working hypotheses from the +// governing spec's Decision 1, pending the Phase 0 spike's confirmation +// against the model card. If the spike corrects them, bump PROMPT_REVISION — +// the cache id carries it (`local-embeddinggemma#p1`), so a prefix fix +// behaves exactly like a provider switch: old rows under `#p1` become cache +// misses under `#p2` and are gc-eligible, never silently reused under a +// stale prefix (disposition C5). Kill condition: if the confirmed prefixes +// differ from these, PROMPT_REVISION must bump in the same change that +// corrects them — a prefix edit that does NOT bump this constant is a bug. +const PROMPT_REVISION = "p1"; +const DOC_PREFIX = "title: none | text: "; +const QUERY_PREFIX = "task: search result | query: "; + +let cached: Map | null = null; + +function providerFor(dim: number): EmbeddingProvider & { resetForTests(): void } { + if (!cached) cached = new Map(); + const existing = cached.get(dim); + if (existing) return existing; + const provider = makeLocalTransformersProvider({ + id: `${LOCAL_EMBEDDINGGEMMA_ID_PREFIX}#${PROMPT_REVISION}`, + hfModel: HF_MODEL, + dtype: "q8", + nativeDim: NATIVE_DIM, + dim, + pooling: "mean", + docPrefix: DOC_PREFIX, + queryPrefix: QUERY_PREFIX, + }); + cached.set(dim, provider); + return provider; +} + +// Constructs (or returns the memoised instance for) the provider at the +// given configured dim. One underlying transformers.js model load is shared +// across every dim requested in a process — the model always outputs +// NATIVE_DIM; only the choke-point truncation (toIndexDim) differs per dim, +// so a second instance at a different dim would otherwise pay a redundant +// model load for zero benefit. +export function makeLocalEmbeddingGemmaProvider(dim: number): EmbeddingProvider { + if (!(LOCAL_EMBEDDINGGEMMA_DIMS as readonly number[]).includes(dim)) { + throw new Error( + `local-embeddinggemma: unsupported dim ${dim} ` + + `(expected one of ${LOCAL_EMBEDDINGGEMMA_DIMS.join(", ")})`, + ); + } + return providerFor(dim); +} + +// True once ANY dim variant's underlying model has been loaded — they share +// one transformers.js extractor per dim, but for the isModelLoaded() +// surface (a process-wide "is embedding warm" signal) any loaded instance +// counts. +export function isLocalEmbeddingGemmaLoaded(): boolean { + if (!cached) return false; + for (const provider of cached.values()) { + if (provider.isLoaded?.()) return true; + } + return false; +} + +// Test-only: clears every memoised dim variant so a fresh import is forced +// on the next call. Production code must not invoke this. +export function resetLocalEmbeddingGemmaForTests(): void { + if (cached) { + for (const provider of cached.values()) provider.resetForTests(); + } + cached = null; +} diff --git a/src/search/providers/local-minilm.ts b/src/search/providers/local-minilm.ts index 3d2e5478..d979883c 100644 --- a/src/search/providers/local-minilm.ts +++ b/src/search/providers/local-minilm.ts @@ -132,4 +132,5 @@ export const localMinilmProvider: EmbeddingProvider = { dim: LOCAL_MINILM_DIM, warm, embed, + isLoaded: isLocalMinilmLoaded, }; diff --git a/src/search/providers/local-qwen3.ts b/src/search/providers/local-qwen3.ts new file mode 100644 index 00000000..50050c01 --- /dev/null +++ b/src/search/providers/local-qwen3.ts @@ -0,0 +1,74 @@ +// local-qwen3 — Qwen/Qwen3-Embedding-0.6B run locally via +// @huggingface/transformers, using the local-transformers.ts factory (spec +// 2026-07-26-embedding-refresh-quantization Decision 1/2, Phase 1c). +// 1024d native — deliberately not offered; this provider exposes at most 768 +// (spec text: "Qwen3's 1024d deliberately not offered yet; 768 is the max +// exposed"). Last-token pooling, instruction-prefixed query / bare document. + +import type { EmbeddingProvider } from "../embedding-provider.js"; +import { makeLocalTransformersProvider } from "./local-transformers.js"; + +export const LOCAL_QWEN3_ID_PREFIX = "local-qwen3-0.6b"; +const HF_MODEL = "onnx-community/Qwen3-Embedding-0.6B-ONNX"; +// The provider's exposed native ceiling — Qwen3-Embedding-0.6B natively +// outputs 1024d, but this provider caps at 768 (the spec's explicit +// deferral). toIndexDim truncates the model's real 1024d output down to +// this ceiling first (a Matryoshka-style truncation the spec treats as the +// provider's own "native" dim for cache/index purposes), then again to the +// caller's configured `dim` when that is smaller still. +const EXPOSED_NATIVE_DIM = 768; +export const LOCAL_QWEN3_DIMS = [512, 768] as const; + +// [TRAINING] Working hypothesis pending the Phase 0 spike (see +// local-embeddinggemma.ts's PROMPT_REVISION comment for the full rationale — +// identical posture here). Qwen3's document side is unprefixed per the +// spec's working hypothesis; the query side carries an instruction prefix. +const PROMPT_REVISION = "p1"; +const DOC_PREFIX = ""; +const QUERY_PREFIX = "Instruct: Given a search query, retrieve relevant passages | Query: "; + +let cached: Map | null = null; + +function providerFor(dim: number): EmbeddingProvider & { resetForTests(): void } { + if (!cached) cached = new Map(); + const existing = cached.get(dim); + if (existing) return existing; + const provider = makeLocalTransformersProvider({ + id: `${LOCAL_QWEN3_ID_PREFIX}#${PROMPT_REVISION}`, + hfModel: HF_MODEL, + dtype: "q8", + nativeDim: EXPOSED_NATIVE_DIM, + dim, + pooling: "last-token", + docPrefix: DOC_PREFIX, + queryPrefix: QUERY_PREFIX, + }); + cached.set(dim, provider); + return provider; +} + +export function makeLocalQwen3Provider(dim: number): EmbeddingProvider { + if (!(LOCAL_QWEN3_DIMS as readonly number[]).includes(dim)) { + throw new Error( + `local-qwen3-0.6b: unsupported dim ${dim} (expected one of ${LOCAL_QWEN3_DIMS.join(", ")})`, + ); + } + return providerFor(dim); +} + +export function isLocalQwen3Loaded(): boolean { + if (!cached) return false; + for (const provider of cached.values()) { + if (provider.isLoaded?.()) return true; + } + return false; +} + +// Test-only: clears every memoised dim variant so a fresh import is forced +// on the next call. Production code must not invoke this. +export function resetLocalQwen3ForTests(): void { + if (cached) { + for (const provider of cached.values()) provider.resetForTests(); + } + cached = null; +} diff --git a/src/search/providers/local-transformers.ts b/src/search/providers/local-transformers.ts new file mode 100644 index 00000000..39079ef9 --- /dev/null +++ b/src/search/providers/local-transformers.ts @@ -0,0 +1,252 @@ +// local-transformers — shared factory for @huggingface/transformers-backed +// local embedding providers (spec 2026-07-26-embedding-refresh-quantization +// Phase 1a). Generalizes local-minilm.ts's shape (memoised lazy extractor, +// markModelWarming/Ready/Error, promise reset on failure, fixed sub-batches, +// Result-typed embed, warm() = load-and-return) to cover the two new +// Matryoshka-truncatable, asymmetric-prompt models: EmbeddingGemma-300M +// (mean pooling) and Qwen3-Embedding-0.6B (last-token pooling). +// +// local-minilm.ts is deliberately NOT refactored onto this factory — its +// exports are load-bearing across the codebase and its tests download the +// real (small) model on every `npm test` run; leaving it untouched keeps +// "vaults that never touch config keep today's behavior exactly" trivially +// true (disposition C8's independence posture applied to this file too). +// +// [HYPOTHESIS] The exact transformers.js call shape for last-token pooling +// (AutoTokenizer + a raw feature-extraction call per single text, no padding, +// so the last row of the returned per-token tensor IS the last real token — +// see poolLastToken below) is UNVERIFIED against a real model load: Phase 0 +// of the governing spec (docs/superpowers/specs/2026-07-26-embedding- +// refresh-quantization-design.md) calls for a smoke spike that loads both +// models and compares against a Python sentence-transformers reference +// before this code path is trusted for a real vault. That spike has not been +// run in this environment (no model download). Kill condition: if the spike +// finds the feature-extraction pipeline cannot express "pooling: none" (raw +// per-token output) for either model, this file's last-token path needs the +// AutoModel/AutoTokenizer low-level API instead (the fallback the spec's +// Decision 1 names explicitly) — a follow-up change, not a silent +// workaround. Every test in this repo exercises this file through a mocked +// `@huggingface/transformers` import (see +// test/search/providers/local-embeddinggemma.test.ts and +// local-qwen3.test.ts); no test here downloads a real model. + +import { err, ok, type Result } from "../../frontmatter/types.js"; +import type { EmbeddingProvider } from "../embedding-provider.js"; +import { markModelError, markModelReady, markModelWarming } from "../index-state.js"; +import { l2Normalize, toIndexDim } from "../vector.js"; + +// See local-minilm.ts's EMBED_BATCH_SIZE comment for the peak-memory +// rationale. 8 is the same starting point; the Phase 0 spike may lower it +// for the larger models. +const DEFAULT_BATCH_SIZE = 8; + +export type Pooling = "mean" | "last-token"; + +// ONNX dtype variants @huggingface/transformers accepts for a model load. +// Mirrored here (rather than imported from the package) so this file's +// public options type doesn't require pulling in the package's full type +// surface just to name "q8". +export type OnnxDtype = + | "auto" + | "fp32" + | "fp16" + | "q8" + | "int8" + | "uint8" + | "q4" + | "bnb4" + | "q4f16"; + +export interface LocalTransformersOptions { + // Cache/model id — dim-free, carries the provider's #pN prompt-revision + // suffix (spec C5/C9). Written to embeddings.model and the durable cache + // key. + id: string; + // Hugging Face repo id for the ONNX community export. + hfModel: string; + // ONNX dtype variant. The spike (Phase 0 gate 4) picks the dtype that + // best balances reindex throughput against query latency; "q8" is this + // file's default pending that measurement. + dtype: OnnxDtype; + // Full model output width. + nativeDim: number; + // Configured index dim (<=nativeDim). Equal to nativeDim means no + // truncation. + dim: number; + pooling: Pooling; + // Asymmetric prompt prefixes (spec Decision 1/C5). Empty string means no + // prefix (Qwen3's document side, per the spec's working hypothesis). + docPrefix: string; + queryPrefix: string; + batchSize?: number; +} + +// Minimal shape of the transformers.js pieces this module calls, mirroring +// local-minilm.ts's `Extractor` type — keeps the rest of the file (and its +// tests, via a mocked "@huggingface/transformers" import) independent of the +// package's full type surface. +type Extractor = ( + texts: string[], + opts: { pooling: "mean" | "none"; normalize: boolean }, +) => Promise<{ data: Float32Array; dims: number[] }>; + +export interface LocalTransformersProvider extends EmbeddingProvider { + // Test-only: clears the memoised extractor promise so a fresh import is + // forced on the next call. Production code must not invoke this. + resetForTests(): void; +} + +// Extracts the last real token's embedding from a [seq_len, hidden] raw +// per-token tensor. Correct WITHOUT an attention mask only when the input +// was NOT padded — which is guaranteed by embedLastToken below calling the +// extractor with a batch of exactly one text at a time, so there is nothing +// to pad against. This trades the sub-batch throughput local-minilm gets +// from mean pooling for correctness simplicity: a padding-aware last-token +// selection would need the tokenizer's attention mask threaded through the +// pipeline call, which the feature-extraction pipeline does not expose +// directly (see the file header's kill condition). +function poolLastToken(data: Float32Array, dims: number[]): Float32Array { + const hidden = dims[dims.length - 1] ?? 0; + const seqLen = dims[dims.length - 2] ?? 1; + const lastTokenStart = (seqLen - 1) * hidden; + return data.slice(lastTokenStart, lastTokenStart + hidden); +} + +export function makeLocalTransformersProvider( + opts: LocalTransformersOptions, +): LocalTransformersProvider { + const batchSize = opts.batchSize ?? DEFAULT_BATCH_SIZE; + let extractorPromise: Promise | null = null; + + async function getExtractor(): Promise { + if (!extractorPromise) { + markModelWarming(); + extractorPromise = ( + import("@huggingface/transformers").then(({ pipeline }) => + pipeline("feature-extraction", opts.hfModel, { dtype: opts.dtype }), + ) as Promise + ).then( + (extractor) => { + markModelReady(); + return extractor; + }, + (e) => { + const reason = e instanceof Error ? e.message : String(e); + markModelError(reason); + extractorPromise = null; + throw e; + }, + ); + } + return extractorPromise; + } + + function isLoaded(): boolean { + return extractorPromise !== null; + } + + function resetForTests(): void { + extractorPromise = null; + } + + async function warm(): Promise> { + try { + await getExtractor(); + return ok(undefined); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`embedding model warm-up failed: ${reason}`)); + } + } + + // Embeds a single prefixed text and returns its NATIVE-dim vector, + // unnormalized-or-not per pooling mode (mean pooling normalizes inside the + // pipeline call; last-token pooling normalizes explicitly afterward, since + // "pooling: none" returns raw hidden states). + async function embedOneNative(prefixedText: string, extractor: Extractor): Promise { + if (opts.pooling === "mean") { + const output = await extractor([prefixedText], { pooling: "mean", normalize: true }); + const dim = output.dims[output.dims.length - 1] ?? opts.nativeDim; + return capToNativeDim(output.data.slice(0, dim)); + } + // last-token: single-item call (no padding), raw per-token output, then + // manual last-token selection + L2 normalize. See poolLastToken's + // correctness note above. + const output = await extractor([prefixedText], { pooling: "none", normalize: false }); + return capToNativeDim(l2Normalize(poolLastToken(output.data, output.dims))); + } + + // Caps a raw pooled vector at `opts.nativeDim`, truncating + re- + // normalizing when the underlying model's real output is wider than the + // dim this provider EXPOSES (local-qwen3.ts sets nativeDim=768 while the + // real model outputs 1024d — spec text: "Qwen3's 1024d deliberately not + // offered yet"). Identity when the raw output already matches (Gemma: + // real 768d == exposed 768d). Reuses toIndexDim so this is the same + // truncate-and-renormalize math the index-facing choke point uses, not a + // second implementation of it. + function capToNativeDim(vec: Float32Array): Float32Array { + return toIndexDim(vec, opts.nativeDim); + } + + async function embed( + texts: string[], + onProgress?: (done: number, total: number) => void, + ): Promise> { + if (texts.length === 0) return ok([]); + try { + const extractor = await getExtractor(); + const vectors: Float32Array[] = []; + for (let start = 0; start < texts.length; start += batchSize) { + const batch = texts.slice(start, start + batchSize); + if (opts.pooling === "mean") { + // Mean pooling batches cleanly (padding does not corrupt a mean + // taken with the pipeline's own internal attention-masked + // average), so this mirrors local-minilm.ts's batched call. + const prefixed = batch.map((t) => `${opts.docPrefix}${t}`); + const output = await extractor(prefixed, { pooling: "mean", normalize: true }); + const dim = output.dims[output.dims.length - 1] ?? opts.nativeDim; + for (let i = 0; i < batch.length; i++) { + vectors.push(capToNativeDim(output.data.slice(i * dim, (i + 1) * dim))); + } + } else { + for (const text of batch) { + vectors.push(await embedOneNative(`${opts.docPrefix}${text}`, extractor)); + } + } + if (onProgress) { + try { + onProgress(vectors.length, texts.length); + } catch { + // ignore — progress reporting is not load-bearing + } + } + } + return ok(vectors); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`embedding failed: ${reason}`)); + } + } + + async function embedQuery(text: string): Promise> { + try { + const extractor = await getExtractor(); + const native = await embedOneNative(`${opts.queryPrefix}${text}`, extractor); + return ok(toIndexDim(native, opts.dim)); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + return err(new Error(`query embedding failed: ${reason}`)); + } + } + + return { + id: opts.id, + dim: opts.dim, + nativeDim: opts.nativeDim, + warm, + embed, + embedQuery, + isLoaded, + resetForTests, + }; +} diff --git a/src/search/providers/openai-3-small.ts b/src/search/providers/openai-3-small.ts index 2473ae94..c25b6831 100644 --- a/src/search/providers/openai-3-small.ts +++ b/src/search/providers/openai-3-small.ts @@ -23,6 +23,7 @@ import { err, ok, type Result } from "../../frontmatter/types.js"; import type { EmbeddingProvider } from "../embedding-provider.js"; +import { l2Normalize } from "../vector.js"; export const OPENAI_3_SMALL_ID = "openai-3-small"; export const OPENAI_3_SMALL_DIM = 1536; @@ -53,25 +54,14 @@ async function delay(ms: number): Promise { return new Promise((resolve) => setTimeout(resolve, ms)); } -// Normalise a vector to unit length in place. OpenAI's text-embedding-3-small -// returns L2-normalized vectors by default, so this is a defense-in-depth -// no-op for the happy path. We still do it because cosine similarity is -// stable iff inputs are normalized, and "the API silently returns un- -// normalized vectors" is exactly the class of upstream change we'd rather -// not be caught by. -function l2Normalize(vec: Float32Array): Float32Array { - let norm = 0; - for (let i = 0; i < vec.length; i++) { - const x = vec[i] as number; - norm += x * x; - } - if (norm === 0) return vec; - const inv = 1 / Math.sqrt(norm); - for (let i = 0; i < vec.length; i++) { - vec[i] = (vec[i] as number) * inv; - } - return vec; -} +// l2Normalize moved to vector.ts (2026-07-26 embedding-refresh-quantization +// spec, Phase 1c) — the new local providers' embedQuery() needs it too, so +// it lives at the shared home the other vector primitives use. OpenAI's +// text-embedding-3-small returns L2-normalized vectors by default, so +// calling it here is a defense-in-depth no-op for the happy path. We still +// do it because cosine similarity is stable iff inputs are normalized, and +// "the API silently returns un-normalized vectors" is exactly the class of +// upstream change we'd rather not be caught by. // One HTTP call to /v1/embeddings for a single batch. Retries on 429/5xx // with exponential backoff; surfaces a definitive failure as Result.err. diff --git a/src/search/reindex.ts b/src/search/reindex.ts index 1f19dc11..2546e7df 100644 --- a/src/search/reindex.ts +++ b/src/search/reindex.ts @@ -31,6 +31,7 @@ import { clearIndex, deleteDocument, documentCount, + embeddingToBlob, existingEmbeddingHashes, gcOrphanedEmbeddings, getEmbeddingBlob, @@ -44,21 +45,29 @@ import { insertEmbeddingVec, openIndexDb, pruneStaleVecRows, + quantizeInt8, replaceDocLinks, setMeta, } from "../storage/index-db.js"; import { listFiles, readFile, resolveVaultPath } from "../storage/local.js"; import { sha256Hex } from "../utils/hash.js"; import { tokenize } from "./bm25.js"; -import { chunkText, embed, getProvider } from "./vector.js"; - -// Opens the index DB with the active embedding provider's dim, so the -// sqlite-vec virtual table is created (or rebuilt) at the right -// dimensionality. Every reindex / index-document path opens the DB this -// way; a caller that doesn't care about vectors (a freshness probe, -// say) can fall back to `openIndexDb(vault)` which uses a default dim. +import { + chunkDocument, + type DocumentChunk, + embed, + embeddingInput, + getProvider, + getQuantize, + toIndexDim, +} from "./vector.js"; + +// Opens the index DB with the active embedding provider's dim + the active +// quantize kind, so the sqlite-vec virtual table is created (or rebuilt) at +// the right dimensionality and representation. Every reindex / index- +// document path opens the DB this way. function openIndexForActiveProvider(vaultRoot: string) { - return openIndexDb(vaultRoot, getProvider().dim); + return openIndexDb(vaultRoot, getProvider().dim, getQuantize()); } // Repopulates the sqlite-vec mirror from the durable `embeddings` cache @@ -66,7 +75,16 @@ function openIndexForActiveProvider(vaultRoot: string) { // table always reflects the current vault. The previous mirror contents // are dropped wholesale — simpler and faster than a diff for the sizes // this index reaches in practice. +// +// The durable cache stores the FULL NATIVE-dim vector (2026-07-26 embedding- +// refresh-quantization spec, disposition C9) — `toIndexDim` truncates (+ +// re-normalizes) to the active provider's CONFIGURED dim here, the single +// choke point where a cached vector meets the index. `quantizeInt8` then +// applies on top when the active quantize kind is "int8" — the vec table +// itself never sees a native-dim or float32-when-int8-configured vector. function rebuildEmbeddingsVec(db: IndexDb, modelId: string): void { + const dim = getProvider().dim; + const kind = getQuantize(); const rebuild = db.transaction(() => { clearEmbeddingsVec(db); // One row per (content_hash, collection): the vec table's `collection` @@ -85,19 +103,37 @@ function rebuildEmbeddingsVec(db: IndexDb, modelId: string): void { WHERE e.model = ?`, ) .all(modelId) as { content_hash: string; collection: string; embedding: Buffer }[]; - const insert = db.prepare( + // sqlite-vec infers a bound blob's vector type from its byte layout + // (defaults to float32); an int8 column needs the value wrapped in + // `vec_int8(?)` — see insertEmbeddingVec's comment in index-db.ts for + // the empirical confirmation against the pinned build. Two prepared + // statements, chosen once outside the loop, keep the per-row cost to a + // single `.run()` regardless of kind. + const insertFloat32 = db.prepare( "INSERT INTO embeddings_vec(content_hash, model, collection, embedding) VALUES (?, ?, ?, ?)", ); + const insertInt8 = db.prepare( + "INSERT INTO embeddings_vec(content_hash, model, collection, embedding) VALUES (?, ?, ?, vec_int8(?))", + ); for (const row of rows) { - insert.run(row.content_hash, modelId, row.collection, row.embedding); + const truncated = toIndexDim(blobToEmbedding(row.embedding), dim); + if (kind === "int8") { + insertInt8.run(row.content_hash, modelId, row.collection, quantizeInt8(truncated)); + } else { + insertFloat32.run(row.content_hash, modelId, row.collection, embeddingToBlob(truncated)); + } } }); rebuild(); } -// The cached vector for a hash, as a Float32Array, or null when the durable -// cache has no row for this model. Used by the incremental path to mirror a -// cache-hit chunk into a collection whose vec row does not exist yet. +// The cached vector for a hash, as a Float32Array AT NATIVE DIM (the +// durable cache's storage width), or null when the cache has no row for +// this model. Used by the incremental path to mirror a cache-hit chunk into +// a collection whose vec row does not exist yet. Callers that write this +// vector into `embeddings_vec` must apply `toIndexDim` first — this +// function itself does no truncation, matching `getEmbeddingBlob`'s raw +// pass-through contract. function readCachedVector(db: IndexDb, hash: string, modelId: string): Float32Array | null { const blob = getEmbeddingBlob(db, hash, modelId); return blob === null ? null : blobToEmbedding(blob); @@ -156,9 +192,30 @@ function manifestsMatch(a: Record, b: Record): b } // Returns true when the persisted index already reflects every markdown file -// on disk: doc count is non-zero, a manifest exists, and every file's mtime -// matches the stored value. Used by `main()` to skip a 20+ minute re-embed -// pass on every restart of a vault that hasn't changed. +// on disk: doc count is non-zero, a manifest exists, every file's mtime +// matches the stored value, AND the vec mirror is coherent with the active +// provider/quantize config (below). Used by `main()` to skip a 20+ minute +// re-embed pass on every restart of a vault that hasn't changed. +// +// Vec-coherence check (2026-07-26 embedding-refresh-quantization spec, +// disposition C1 — this is what makes the "config change + background +// reindex" migration story in the spec's Decision 4 actually real). Without +// it, a config-only change to `provider`, `dim`, or `quantize` on an +// unchanged vault would let `openIndexForActiveProvider`'s drop-recreate +// (openIndexDb) empty `embeddings_vec` and then have THIS function report +// "fresh" anyway — vector search goes silently dark until a manual +// vault_reindex. Two checks, either failing routes through the normal +// reindex: +// (a) the stored `embedding_model` meta must equal the active provider's +// cache id — catches a provider switch AND a `#pN` prompt-revision +// bump (which changes the cache id under a still-same-looking config +// provider name). +// (b) `embeddings_vec` must be non-empty whenever `chunks` is non-empty — +// catches a dim or quantize flip, whose drop-recreate happens under +// an UNCHANGED model id, so (a) alone would miss it. +// For a dim/quantize flip this routes into a reindex that is all cache hits +// (a vec-mirror rebuild in minutes); for a provider switch it is the +// intended cold re-embed. export async function isIndexFresh(vaultRoot: string): Promise { const dbResult = openIndexForActiveProvider(vaultRoot); if (!dbResult.ok) return false; @@ -169,7 +226,17 @@ export async function isIndexFresh(vaultRoot: string): Promise { if (!stored) return false; const current = await buildManifest(vaultRoot); if (!current) return false; - return manifestsMatch(stored, current); + if (!manifestsMatch(stored, current)) return false; + + if (getMeta(db, "embedding_model") !== getProvider().id) return false; + const chunkRows = (db.prepare("SELECT COUNT(*) AS n FROM chunks").get() as { n: number }).n; + if (chunkRows > 0) { + const vecRows = ( + db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number } + ).n; + if (vecRows === 0) return false; + } + return true; } finally { db.close(); } @@ -250,7 +317,7 @@ export function reindexWarnings(result: ReindexResult): string[] { interface StagedDocument { doc: IndexedDocument; - chunks: string[]; + chunks: DocumentChunk[]; hashes: string[]; } @@ -298,8 +365,11 @@ async function stageOne(vaultRoot: string, relPath: string): Promise sha256Hex(t)); + // Hoisted so the document row and the chunk breadcrumbs agree on the same + // resolved collection (spec 2026-07-26 contextual-chunking Decision 2). + const collection = fm.collection || (relPath.split("/")[0] ?? ""); + const chunks = chunkDocument({ title: fm.title, collection, tags: fm.tags, body }); + const hashes = chunks.map((c) => sha256Hex(embeddingInput(c))); return { kind: "staged", @@ -308,7 +378,7 @@ async function stageOne(vaultRoot: string, relPath: string): Promise { + chunks.forEach((chunk, chunkIndex) => { const row: ChunkRowInput = { path: doc.path, chunkIndex, - text, + text: chunk.text, + context: chunk.context, contentHash: hashes[chunkIndex] ?? "", }; insertChunkRow(db, row); @@ -416,9 +488,9 @@ export async function reindexVault( for (const s of staged) { for (let i = 0; i < s.chunks.length; i++) { const h = s.hashes[i] ?? ""; - const t = s.chunks[i] ?? ""; + const chunk = s.chunks[i]; allHashes.push(h); - allTexts.push(t); + allTexts.push(chunk ? embeddingInput(chunk) : ""); } } @@ -467,7 +539,10 @@ export async function reindexVault( const h = sliceHashes[i] ?? ""; const vec = embedResult.value[i]; if (!vec) continue; - insertEmbedding(db, h, provider.id, vec, indexedAt, provider.dim); + // The durable cache stores NATIVE-dim vectors (C9) — embed() + // already returns them at that width; the guard here checks + // against nativeDim, not the (possibly smaller) configured dim. + insertEmbedding(db, h, provider.id, vec, indexedAt, provider.nativeDim ?? provider.dim); } }); writeEmbeds(); @@ -593,7 +668,8 @@ export async function indexDocument( const h = hashes[i] ?? ""; if (cached.has(h)) continue; if (missTextByHash.has(h)) continue; - missTextByHash.set(h, chunks[i] ?? ""); + const chunk = chunks[i]; + missTextByHash.set(h, chunk ? embeddingInput(chunk) : ""); } const missHashes = [...missTextByHash.keys()]; const missTexts = missHashes.map((h) => missTextByHash.get(h) ?? ""); @@ -608,7 +684,9 @@ export async function indexDocument( const h = missHashes[i] ?? ""; const vec = embedResult.value[i]; if (!vec) continue; - insertEmbedding(db, h, provider.id, vec, createdAt, provider.dim); + // Native-dim cache write (C9) — see the reindexVault call site's + // comment for the same rationale. + insertEmbedding(db, h, provider.id, vec, createdAt, provider.nativeDim ?? provider.dim); newlyEmbedded.push({ hash: h, vec }); } }); @@ -649,11 +727,12 @@ export async function indexDocument( deleteDocument(db, doc.path); insertDocument(db, doc); replaceDocLinks(db, doc.path, linkTargets); - chunks.forEach((text, chunkIndex) => { + chunks.forEach((chunk, chunkIndex) => { insertChunkRow(db, { path: doc.path, chunkIndex, - text, + text: chunk.text, + context: chunk.context, contentHash: hashes[chunkIndex] ?? "", }); }); @@ -666,7 +745,15 @@ export async function indexDocument( // the loop runs over every chunk hash and asks the table, rather than // over the newly-embedded list alone. vec0 supports neither // INSERT OR IGNORE nor a unique constraint, so the check is explicit. + // + // `vec` here is NATIVE-dim (freshVecs from the just-embedded batch, or + // readCachedVector from the durable cache — both native-dim per C9); + // `toIndexDim` truncates to the active provider's configured dim + // before the vec-table write, and `insertEmbeddingVec` quantizes on + // top when the active quantize kind is "int8". const freshVecs = new Map(newlyEmbedded.map(({ hash, vec }) => [hash, vec])); + const indexDim = provider.dim; + const quantizeKind = getQuantize(); for (const hash of new Set(hashes)) { if (hash === "") continue; // Drop rows this hash no longer justifies before adding the new one: @@ -678,7 +765,16 @@ export async function indexDocument( pruneStaleVecRows(db, hash, provider.id); if (hasEmbeddingVec(db, hash, provider.id, doc.collection)) continue; const vec = freshVecs.get(hash) ?? readCachedVector(db, hash, provider.id); - if (vec) insertEmbeddingVec(db, hash, provider.id, doc.collection, vec); + if (vec) { + insertEmbeddingVec( + db, + hash, + provider.id, + doc.collection, + toIndexDim(vec, indexDim), + quantizeKind, + ); + } } }); write(); diff --git a/src/search/rerank-provider.ts b/src/search/rerank-provider.ts new file mode 100644 index 00000000..4d751e2e --- /dev/null +++ b/src/search/rerank-provider.ts @@ -0,0 +1,96 @@ +// RerankProvider — pluggable local cross-encoder reranker. +// +// Mirrors EmbeddingProvider (src/search/embedding-provider.ts) and its +// vector.ts selection block: config-selected, instantiated once per process +// (memoised by setRerankProvider/getRerankProvider), warm()/lazy-load, +// Result-typed failures with graceful degradation. Spec 2026-07-26- +// contextual-chunking-reranker-design.md Decision 5. +// +// "none" (the default) maps to null — callers branch on presence, no +// null-object provider. There is exactly one real provider today +// (local-bge-m3); a second would mean a new id in utils/config.ts's +// RERANK_PROVIDERS AND a new branch in instantiateProvider below. +// +// Contract: +// - `id` is a stable namespace, mirrors EmbeddingProvider.id. +// - `isReady()` is true once the model is loaded into memory. The search +// path (src/tools/search.ts) checks this BEFORE ever attempting a +// rerank and never blocks a tool call on a cold model load (spec C5, +// from the plan's challenge resolution): a configured-but-not-warm +// reranker skips reranking for THIS search (rerankUsed stays false) and +// fires a background warmRerankModel() instead. +// - `warm()` is the eager-load entry point, invoked from the same +// background-warm path that warms the embedding model (src/index.ts's +// runBackgroundWarm, guarded by the existing warmEmbeddings config flag +// — no new knob, per spec Decision 8). +// - `rerank(query, passages)` returns one relevance score per passage, in +// input order. Result.err = degrade to the fused order — a missing or +// failing 600MB model must never fail a search. + +import type { Result } from "../frontmatter/types.js"; +import type { RerankProviderId } from "../utils/config.js"; +import { localBgeM3Provider, resetLocalBgeM3ForTests } from "./providers/local-bge-m3.js"; + +export interface RerankProvider { + readonly id: string; + isReady(): boolean; + warm(): Promise>; + rerank(query: string, passages: string[]): Promise>; +} + +// The active provider for this process. null means "none" — no rerank stage. +let activeProvider: RerankProvider | null = null; +// Tracked separately from `activeProvider` purely so setRerankProvider can +// no-op on a repeated call with the same id (mirrors vector.ts's +// `activeProvider.id === id` check, adapted for a nullable active provider). +let activeId: RerankProviderId = "none"; + +function instantiateProvider(id: RerankProviderId): RerankProvider | null { + switch (id) { + case "none": + return null; + case "local-bge-m3": + return localBgeM3Provider; + } +} + +// Called once at server startup (after loadConfig), in the same try/fail- +// loud block as setProvider. Idempotent for the same id. +export function setRerankProvider(id: RerankProviderId): void { + if (activeId === id) return; + activeId = id; + activeProvider = instantiateProvider(id); +} + +// Returns the active provider, or null when rerank.provider is "none" (the +// default). Callers branch on presence — there is no null-object provider. +export function getRerankProvider(): RerankProvider | null { + return activeProvider; +} + +// Eagerly loads the active provider (if any) so the first rerank-enabled +// search does not pay the cold start. A no-op ok() when no provider is +// configured — mirrors warmModel()'s shape but never fails just because +// reranking is off. +export async function warmRerankModel(): Promise> { + if (!activeProvider) return { ok: true, value: undefined }; + return activeProvider.warm(); +} + +// Test-only: install an arbitrary provider (or null) directly, bypassing +// config-driven selection. Used by tests that need a fast, deterministic +// fake reranker without paying the model-load cost. Does NOT touch +// `activeId` — a later real setRerankProvider(id) call still compares +// against the id it was last given production-side, matching +// setProviderForTests' behaviour in vector.ts. +export function setRerankProviderForTests(p: RerankProvider | null): void { + activeProvider = p; +} + +// Test-only: revert to no provider ("none") and clear local-bge-m3's +// memoised model. Production code must not call this. +export function resetRerankProviderForTests(): void { + activeProvider = null; + activeId = "none"; + resetLocalBgeM3ForTests(); +} diff --git a/src/search/router.ts b/src/search/router.ts new file mode 100644 index 00000000..b6d08d65 --- /dev/null +++ b/src/search/router.ts @@ -0,0 +1,170 @@ +// Query router (spec 2026-07-26 fusion overhaul, Decision 2). +// +// Classifies a raw user query into one of three routes and maps the route to +// a static HybridWeights split. Pure functions + types only — the router +// itself does no I/O; `makeDfLookup` is the one place a caller wires it to a +// live index handle, and even that is injected into `classifyQuery` as a +// plain function, not a database handle. +// +// The router is wired into `vault_search` only (src/tools/search.ts), gated +// behind the `search.routing` config switch. CLI/eval/bench callers that use +// `hybridSearch` directly stay on static weights unless they opt in. + +import type { IndexDb } from "../storage/index-db.js"; +import { tokenize } from "./bm25.js"; +import type { HybridWeights } from "./hybrid.js"; +import { DEFAULT_WEIGHTS } from "./hybrid.js"; + +export type RouteClass = "extreme-lexical" | "lexical" | "balanced"; + +export interface RouterOptions { + // Document-frequency lookup, stem-aware (see makeDfLookup). Absent → + // the rare-term signal never fires. + df?: (token: string) => number; + // Vault document count. The rare-term signal is disabled below + // MIN_DOCS_FOR_RARE regardless of df, so this is required alongside df to + // enable the signal at all. + docCount?: number; +} + +export interface ClassifyResult { + class: RouteClass; + signals: string[]; +} + +// A term is "rare" (in the document-frequency sense) when its df sits in +// [1, DF_RARE_FLOOR]. df === 0 means absent-from-corpus and never fires — an +// unknown token is not evidence the query is about something rare, it's +// evidence the query has a typo or the vault doesn't cover it. +const DF_RARE_FLOOR = 2; + +// Below this many vault documents, df <= DF_RARE_FLOOR covers a large +// fraction of the whole vocabulary — the rare-term signal is noise exactly +// where semantic recall matters most, so it is disabled entirely rather than +// floor-adjusted. Kill condition [HYPOTHESIS]: if bench category deltas show +// the signal harming at ~180 docs, this floor is too high and gets +// re-derived before the PR 3 flip. +const MIN_DOCS_FOR_RARE = 100; + +// Path-like extensions. Deliberately a fixed, small, source/config-flavoured +// list — this is a vault of markdown + code/config references, not a +// general-purpose file-type sniffer. +const PATH_LIKE_EXTENSIONS = [ + ".md", + ".ts", + ".js", + ".mjs", + ".json", + ".yaml", + ".yml", + ".py", + ".sql", + ".sh", + ".toml", +]; + +const QUOTED_PHRASE_RE = /"[^"]{2,}"/; +const CAMEL_CASE_RE = /[a-z0-9][A-Z]/; +const SNAKE_CASE_RE = /[A-Za-z0-9]_[A-Za-z0-9]/; + +function isPathLike(token: string): boolean { + if (token.includes("/")) return true; + const lower = token.toLowerCase(); + return PATH_LIKE_EXTENSIONS.some((ext) => lower.endsWith(ext)); +} + +function isDigitHeavy(token: string): boolean { + if (token.length < 3) return false; + const digits = (token.match(/[0-9]/g) ?? []).length; + return digits >= Math.ceil(token.length / 2); +} + +// Classifies a raw query into a route class, plus the full list of signals +// that fired (not just the ones that decided the class — a routed extreme +// query that ALSO carries a rare term should surface both in diagnostics). +// +// Precedence: any extreme-lexical signal wins outright; otherwise any +// lexical signal; otherwise balanced. +export function classifyQuery(rawQuery: string, opts: RouterOptions = {}): ClassifyResult { + const signals: string[] = []; + let sawExtreme = false; + let sawLexical = false; + + if (QUOTED_PHRASE_RE.test(rawQuery)) { + signals.push("quoted-phrase"); + sawExtreme = true; + } + + const rawTokens = rawQuery.split(/\s+/).filter((t) => t.length > 0); + + let sawPathLike = false; + let sawCamelCase = false; + let sawSnakeCase = false; + let sawDigitHeavy = false; + for (const token of rawTokens) { + if (!sawPathLike && isPathLike(token)) sawPathLike = true; + if (!sawCamelCase && CAMEL_CASE_RE.test(token)) sawCamelCase = true; + if (!sawSnakeCase && SNAKE_CASE_RE.test(token)) sawSnakeCase = true; + if (!sawDigitHeavy && isDigitHeavy(token)) sawDigitHeavy = true; + } + if (sawPathLike) { + signals.push("path-like"); + sawExtreme = true; + } + if (sawCamelCase) { + signals.push("camel-case"); + sawLexical = true; + } + if (sawSnakeCase) { + signals.push("snake-case"); + sawLexical = true; + } + if (sawDigitHeavy) { + signals.push("digit-heavy"); + sawLexical = true; + } + + if (opts.df && opts.docCount !== undefined && opts.docCount >= MIN_DOCS_FOR_RARE) { + const df = opts.df; + const isRare = tokenize(rawQuery).some((token) => { + const count = df(token); + return count >= 1 && count <= DF_RARE_FLOOR; + }); + if (isRare) { + signals.push("rare-term"); + sawLexical = true; + } + } + + const cls: RouteClass = sawExtreme ? "extreme-lexical" : sawLexical ? "lexical" : "balanced"; + return { class: cls, signals }; +} + +// Maps a route class to a static weight split. +// +// extreme-lexical skips query embedding entirely (hybridSearch checks +// weights.vector > 0 before calling embedQuery) — the latency win the +// extreme route buys. +export function routeWeights(cls: RouteClass): HybridWeights { + switch (cls) { + case "extreme-lexical": + return { bm25: 1, vector: 0 }; + case "lexical": + return { bm25: 0.8, vector: 0.2 }; + case "balanced": + return { ...DEFAULT_WEIGHTS }; + } +} + +// Document-frequency lookup, stem-aware by construction: FTS5 applies its +// own porter tokenizer to the MATCH query, so "locking" and "locks" both +// count postings stemmed to "lock" — no fts5vocab table, no schema change. +// Document-granularity (not chunk): "rare in the vault" is a whole-document +// notion even though default ranking is chunk-granular. +export function makeDfLookup(db: IndexDb): (token: string) => number { + const stmt = db.prepare("SELECT count(*) AS n FROM documents_fts WHERE documents_fts MATCH ?"); + return (token: string): number => { + const row = stmt.get(`"${token}"`) as { n: number }; + return row.n; + }; +} diff --git a/src/search/vector.ts b/src/search/vector.ts index 23b35175..f4705697 100644 --- a/src/search/vector.ts +++ b/src/search/vector.ts @@ -13,14 +13,27 @@ // is provider-agnostic. import { err, ok, type Result } from "../frontmatter/types.js"; +import type { VecKind } from "../storage/index-db.js"; import type { EmbeddingProviderId } from "../utils/config.js"; import type { EmbeddingProvider } from "./embedding-provider.js"; +import { + isLocalEmbeddingGemmaLoaded, + LOCAL_EMBEDDINGGEMMA_DIMS, + makeLocalEmbeddingGemmaProvider, + resetLocalEmbeddingGemmaForTests, +} from "./providers/local-embeddinggemma.js"; import { isLocalMinilmLoaded, LOCAL_MINILM_DIM, localMinilmProvider, resetLocalMinilmForTests, } from "./providers/local-minilm.js"; +import { + isLocalQwen3Loaded, + LOCAL_QWEN3_DIMS, + makeLocalQwen3Provider, + resetLocalQwen3ForTests, +} from "./providers/local-qwen3.js"; import { makeOpenAi3SmallProvider } from "./providers/openai-3-small.js"; // EMBEDDING_MODEL and EMBEDDING_DIM are retained as deprecated plain @@ -44,11 +57,17 @@ export const EMBED_BATCH_SIZE = 8; const CHUNK_MAX_CHARS = 800; -// Splits a document body into embeddable chunks. Paragraphs (blank-line -// separated) are packed greedily up to CHUNK_MAX_CHARS; a single paragraph -// longer than the cap is hard-split. Always returns at least one chunk so an -// empty body still produces a (possibly empty) vector slot. -export function chunkText(text: string): string[] { +// Paragraph-packing loop, private to this module. Blank-line-separated +// paragraphs are packed greedily up to CHUNK_MAX_CHARS; a single paragraph +// longer than the cap is hard-split. Always returns at least one chunk for +// non-empty input; an all-whitespace input returns []. +// +// Reused verbatim (this WAS chunkText's whole body pre-contextual-chunking) +// as the within-SECTION packer for chunkDocument below — a heading boundary +// always starts a new chunk (spec 2026-07-26 Decision 1), so packing never +// spans two sections. This function itself is section-agnostic; it just packs +// whatever text blob it's given. +function packParagraphs(text: string): string[] { const paragraphs = text .split(/\n\s*\n/) .map((p) => p.trim()) @@ -75,7 +94,238 @@ export function chunkText(text: string): string[] { } } if (current) chunks.push(current); - return chunks.length > 0 ? chunks : [text.trim()]; + return chunks; +} + +// --- Contextual chunking (spec 2026-07-26 contextual-chunking-reranker) --- + +export interface ChunkInput { + title: string; + collection: string; + tags: string[]; + body: string; +} + +export interface DocumentChunk { + text: string; // verbatim body slice, exactly what chunks.text stores + context: string; // one-line breadcrumb, <=160 chars +} + +// Single source of truth for the retrieval identity of a chunk — used by BOTH +// the content_hash and the embedding input so they can never drift. See spec +// Decision 2: the context is part of the chunk's retrieval identity, so it is +// hashed and embedded together with the body text. +export function embeddingInput(c: DocumentChunk): string { + return c.context.length > 0 ? `${c.context}\n\n${c.text}` : c.text; +} + +const CONTEXT_MAX_CHARS = 160; +const CONTEXT_MAX_TAGS = 5; + +// ATX headings, levels 1-4 only (spec: #####/###### and setext headings +// degrade to plain text — the vault house style is ATX). +const ATX_HEADING_RE = /^(#{1,4})\s+(.*)$/; +const FENCE_RE = /^(```|~~~)/; + +function stripHeadingText(raw: string): string { + return raw.replace(/\s+/g, " ").trim(); +} + +// Longest prefix of `text` (<=maxLen chars) such that `render(prefix + "…")` +// (or `render("…")` at zero chars) still fits within CONTEXT_MAX_CHARS. Used +// for both the innermost-heading and title tail-truncation steps below — +// binary search over the truncation point, not a plain slice, because the +// ellipsis and the surrounding breadcrumb literals shift where the cutoff +// needs to land. +function longestFittingPrefix(text: string, render: (candidate: string) => string): string { + let lo = 0; + let hi = text.length; + let best = "…"; + while (lo <= hi) { + const mid = Math.floor((lo + hi) / 2); + const candidate = mid > 0 ? `${text.slice(0, mid)}…` : "…"; + if (render(candidate).length <= CONTEXT_MAX_CHARS) { + best = candidate; + lo = mid + 1; + } else { + hi = mid - 1; + } + } + return best; +} + +// Builds the one-line breadcrumb context for a chunk: +// {collection} › {doc title} › {H1} › {H2} › … · tags: a, b, c +// +// Tags are sorted lexicographically (so tag REORDER never perturbs the hash, +// C7) then capped at CONTEXT_MAX_TAGS; the tag suffix is omitted entirely for +// an untagged doc. The whole line is hard-capped at CONTEXT_MAX_CHARS; +// truncation, in order, is: (1) collapse every heading component except the +// innermost into a single "…", (2) tail-truncate the innermost heading, +// (3) drop the tag suffix, (4) tail-truncate the title. Collection and title +// always survive AS COMPONENTS (never dropped outright) — they are the +// highest-value disambiguators for the vault's short, similar-shaped docs. +function buildContext( + input: { title: string; collection: string; tags: string[] }, + headingPath: string[], +): string { + const sortedTags = [...input.tags].sort((a, b) => a.localeCompare(b)).slice(0, CONTEXT_MAX_TAGS); + const tagsSuffix = sortedTags.length > 0 ? ` · tags: ${sortedTags.join(", ")}` : ""; + + const render = (headings: string[], title: string, tags: string): string => + [input.collection, title, ...headings].join(" › ") + tags; + + let line = render(headingPath, input.title, tagsSuffix); + if (line.length <= CONTEXT_MAX_CHARS) return line; + + // Step 1: collapse every heading but the innermost into a single "…". + let headings = headingPath; + if (headingPath.length > 1) { + headings = ["…", headingPath[headingPath.length - 1] as string]; + line = render(headings, input.title, tagsSuffix); + if (line.length <= CONTEXT_MAX_CHARS) return line; + } + + // Step 2: tail-truncate the innermost heading. + if (headings.length > 0) { + const innermostIdx = headings.length - 1; + const innermost = headings[innermostIdx] as string; + const truncated = longestFittingPrefix(innermost, (candidate) => + render([...headings.slice(0, innermostIdx), candidate], input.title, tagsSuffix), + ); + headings = [...headings.slice(0, innermostIdx), truncated]; + line = render(headings, input.title, tagsSuffix); + if (line.length <= CONTEXT_MAX_CHARS) return line; + } + + // Step 3: drop the tag suffix. + line = render(headings, input.title, ""); + if (line.length <= CONTEXT_MAX_CHARS) return line; + + // Step 4: tail-truncate the title. Collection is never truncated. + const truncatedTitle = longestFittingPrefix(input.title, (candidate) => + render(headings, candidate, ""), + ); + return render(headings, truncatedTitle, ""); +} + +// Splits a document body into heading-aware, breadcrumb-contextualized +// chunks (spec 2026-07-26 Decision 1/2). Line-scans the body tracking fenced +// code blocks (``` / ~~~ toggles — a `#` line inside a fence is never a +// heading) and the open ATX heading stack (levels 1-4). A heading line closes +// the current section and starts a new one; the heading line itself remains +// part of ITS section's text (document content — snippets and FTS body text +// stay real, never synthesized). Within a section, paragraphs are packed +// exactly as before (packParagraphs) — no packing across section boundaries, +// ever, even when two small sections would fit in one chunk together (a +// chunk spanning two headings has no honest breadcrumb). +// +// Always returns >=1 chunk, preserving chunkText's old guarantee: an empty or +// whitespace-only body returns a single chunk with the trimmed (possibly +// empty) body and a heading-path-free breadcrumb. +export function chunkDocument(input: ChunkInput): DocumentChunk[] { + const { title, collection, tags, body } = input; + const lines = body.split("\n"); + + const sections: { headingPath: string[]; text: string }[] = []; + let currentPath: string[] = []; + let currentLines: string[] = []; + let inFence = false; + let fenceMarker = ""; + + const flush = (): void => { + // currentPath can hold sparse holes (e.g. a document that opens directly + // at ## with no preceding #, or after a level drops back below one that + // was never set) — filter them so the heading path is always a dense + // sequence of the headings actually open, never "undefined › Section". + const headingPath = currentPath.filter((h): h is string => typeof h === "string"); + sections.push({ headingPath, text: currentLines.join("\n") }); + currentLines = []; + }; + + for (const line of lines) { + const fenceMatch = FENCE_RE.exec(line); + if (fenceMatch) { + if (!inFence) { + inFence = true; + fenceMarker = fenceMatch[1] as string; + } else if (line.trimStart().startsWith(fenceMarker)) { + inFence = false; + } + currentLines.push(line); + continue; + } + if (!inFence) { + const headingMatch = ATX_HEADING_RE.exec(line); + if (headingMatch) { + flush(); + const level = (headingMatch[1] as string).length; + const text = stripHeadingText(headingMatch[2] as string); + currentPath = currentPath.slice(0, level - 1); + currentPath[level - 1] = text; + currentLines.push(line); + continue; + } + } + currentLines.push(line); + } + flush(); + + const chunks: DocumentChunk[] = []; + for (const section of sections) { + if (section.text.trim().length === 0) continue; // empty preamble before an immediate heading + const context = buildContext({ title, collection, tags }, section.headingPath); + for (const text of packParagraphs(section.text)) { + chunks.push({ text, context }); + } + } + + if (chunks.length === 0) { + chunks.push({ text: body.trim(), context: buildContext({ title, collection, tags }, []) }); + } + return chunks; +} + +// Normalises a vector to unit length IN PLACE and returns it. Moved here +// (2026-07-26 embedding-refresh-quantization spec, Phase 1c) from +// openai-3-small.ts, which was the only caller before this PR — the new +// local providers' embedQuery() (local-transformers.ts) also need it to +// re-normalise after Matryoshka truncation, so it lives at the shared home +// the other vector primitives (cosineSimilarity, meanEmbedding) already use. +export function l2Normalize(vec: Float32Array): Float32Array { + let norm = 0; + for (let i = 0; i < vec.length; i++) { + const x = vec[i] as number; + norm += x * x; + } + if (norm === 0) return vec; + const inv = 1 / Math.sqrt(norm); + for (let i = 0; i < vec.length; i++) { + vec[i] = (vec[i] as number) * inv; + } + return vec; +} + +// The single choke point (spec Decision 9 / disposition C9) where a +// NATIVE-dim vector meets the vec index or a query. Matryoshka-truncatable +// providers (local-embeddinggemma, local-qwen3) cache the FULL native-dim +// vector in the durable `embeddings` table — truncating at write time would +// make a later dim change (512 <-> 768) a full cold re-embed instead of a +// cheap vec-mirror rebuild from cache. This function is where the truncation +// actually happens: slice to `dim` and re-L2-normalize (required — a slice +// of a unit vector is not itself unit length, and cosine similarity is only +// meaningful over normalized vectors). Identity (a fresh copy, not the same +// reference) when `vec.length === dim` already. +// +// Callers, per the spec: `rebuildEmbeddingsVec` and the `indexDocument` +// incremental mirror (both in reindex.ts), `readCachedVector` (reindex.ts), +// the rescore-blob read in `vecRanking` (hybrid.ts), and `relatedSearch`'s +// `meanEmbedding` inputs (hybrid.ts). Providers with no native/configured +// dim gap (local-minilm, openai-3-small) pass through this function too — +// it is a no-op there — so callers never need to branch on provider shape. +export function toIndexDim(vec: Float32Array, dim: number): Float32Array { + if (vec.length === dim) return new Float32Array(vec); + return l2Normalize(vec.slice(0, dim)); } // Cosine similarity in [-1, 1]. Mismatched lengths or a zero vector yield 0. @@ -119,10 +369,24 @@ export function meanEmbedding(vectors: Float32Array[]): Float32Array | null { // switching back). let activeProvider: EmbeddingProvider = localMinilmProvider; -// Resolves the active provider from a config id. The OPENAI_API_KEY presence -// has already been validated by loadConfig; if it's somehow missing here we -// fail loud rather than constructing a broken provider. -function instantiateProvider(id: EmbeddingProviderId): EmbeddingProvider { +// The index-representation quantization for the active provider (spec +// 2026-07-26 embedding-refresh-quantization, Phase 2b/3). Stored as the +// storage-layer VecKind ("float32" | "int8") rather than the config-level +// "int8" | "none" spelling, because every downstream call site +// (openIndexDb's expectedVecKind, createVecTable, insertEmbeddingVec, +// vecRanking's kind param) wants the storage vocabulary directly — the +// "none" -> "float32" translation happens once, here, at the setProvider +// boundary. Defaults to "float32": vaults that never touch config keep +// today's behaviour exactly (local-minilm's implicit quantize: none). +let activeQuantize: VecKind = "float32"; + +// Resolves the active provider from a config id (+ optional dim). The +// OPENAI_API_KEY presence has already been validated by loadConfig; if it's +// somehow missing here we fail loud rather than constructing a broken +// provider. `dim` is required for the two Matryoshka-truncatable providers +// (validated against EMBEDDING_DIMS by loadConfig) and ignored by the +// fixed-dim providers. +function instantiateProvider(id: EmbeddingProviderId, dim?: number): EmbeddingProvider { switch (id) { case "local-minilm": return localMinilmProvider; @@ -133,16 +397,45 @@ function instantiateProvider(id: EmbeddingProviderId): EmbeddingProvider { } return makeOpenAi3SmallProvider(key); } + case "local-embeddinggemma": + return makeLocalEmbeddingGemmaProvider(dim ?? LOCAL_EMBEDDINGGEMMA_DIMS[0]); + case "local-qwen3-0.6b": + return makeLocalQwen3Provider(dim ?? LOCAL_QWEN3_DIMS[0]); } } -// Called once at server startup (after loadConfig). Idempotent for the same -// id — subsequent calls with the same id are no-ops, so test code can call -// it freely without thrashing. A different id replaces the provider; tests -// rely on this. -export function setProvider(id: EmbeddingProviderId): void { - if (activeProvider.id === id) return; - activeProvider = instantiateProvider(id); +export interface SetProviderOptions { + dim?: number; + quantize?: "int8" | "none"; +} + +// Called once at server startup (after loadConfig). Idempotent for the +// resolved (cacheId, dim, quantize) tuple — a repeated call with the same +// effective identity is a no-op, so test code can call it freely without +// thrashing. Any change in id, dim, or quantize replaces the provider (dim +// and quantize alone can change the tuple even when `id` is unchanged — a +// Matryoshka dim flip or a quantize flip on the same provider id must still +// swap the active provider/quantize state, not silently no-op). +export function setProvider(id: EmbeddingProviderId, opts: SetProviderOptions = {}): void { + const quantize: VecKind = opts.quantize === "int8" ? "int8" : "float32"; + const candidate = instantiateProviderCached(id, opts.dim); + if ( + activeProvider.id === candidate.id && + activeProvider.dim === candidate.dim && + activeQuantize === quantize + ) { + return; + } + activeProvider = candidate; + activeQuantize = quantize; +} + +// Avoids constructing a throwaway provider object just to test idempotence — +// instantiateProvider is cheap (no I/O) for every current provider, so this +// is presently a direct pass-through; kept as a seam in case a future +// provider's construction becomes non-trivial. +function instantiateProviderCached(id: EmbeddingProviderId, dim?: number): EmbeddingProvider { + return instantiateProvider(id, dim); } // Returns the active provider. Default is local-minilm; setProvider() (which @@ -151,31 +444,40 @@ export function getProvider(): EmbeddingProvider { return activeProvider; } +// Returns the active vec-index quantization kind. See `activeQuantize` above +// for why this is VecKind, not the config-level "int8" | "none" spelling. +export function getQuantize(): VecKind { + return activeQuantize; +} + // Test-only: install an arbitrary provider object. Used by reindex tests // that need to simulate a provider switch without paying the network or // model-load cost. Resets the local-minilm memoised extractor too so a -// later swap back to local-minilm starts cold. -export function setProviderForTests(provider: EmbeddingProvider): void { +// later swap back to local-minilm starts cold. Does not touch +// `activeQuantize` — tests that care about quantization set it explicitly. +export function setProviderForTests(provider: EmbeddingProvider, quantize?: "int8" | "none"): void { activeProvider = provider; + if (quantize !== undefined) activeQuantize = quantize === "int8" ? "int8" : "float32"; } // Test-only: revert to the default local-minilm provider and clear its // memoised extractor. Production code must not call this. export function resetProviderForTests(): void { activeProvider = localMinilmProvider; + activeQuantize = "float32"; resetLocalMinilmForTests(); + resetLocalEmbeddingGemmaForTests(); + resetLocalQwen3ForTests(); } // --- Provider-delegating surface (kept for back-compat) ------------------- -// Returns true once the active provider's underlying model is loaded. For -// providers with no warm-up cost (e.g. the stateless OpenAI HTTP client) -// this is always true; for local-minilm it tracks the transformers.js -// extractor promise. +// Returns true once the active provider's underlying model is loaded. Reads +// the provider's own `isLoaded()` when present; a provider that omits it +// (a stateless HTTP client, e.g. openai-3-small) is "loaded" by definition — +// see the EmbeddingProvider contract. export function isModelLoaded(): boolean { - if (activeProvider.id === "local-minilm") return isLocalMinilmLoaded(); - // Stateless / always-ready providers are "loaded" by definition. - return true; + return activeProvider.isLoaded?.() ?? true; } // Eagerly loads the active provider so the first user search does not pay @@ -204,11 +506,19 @@ export async function embed( return activeProvider.embed(texts, onProgress); } -// Convenience wrapper for embedding a single query string. +// Convenience wrapper for embedding a single query string. Delegates to the +// active provider's own `embedQuery` when present — that is where a +// provider applies its query-side prompt prefix (spec 2026-07-26 embedding- +// refresh-quantization, Phase 1b/1c) and truncates to configured dim. +// Providers with no asymmetric prefix and no truncation (local-minilm, +// openai-3-small) omit `embedQuery`; the fallback here is byte-identical to +// the pre-PR behaviour: embed([text]) at (what is already) configured dim, +// passed through `toIndexDim` as a defensive no-op. export async function embedQuery(text: string): Promise> { + if (activeProvider.embedQuery) return activeProvider.embedQuery(text); const result = await embed([text]); if (!result.ok) return result; const first = result.value[0]; if (!first) return err(new Error("embedding produced no vector")); - return ok(first); + return ok(toIndexDim(first, activeProvider.dim)); } diff --git a/src/search/watcher.ts b/src/search/watcher.ts index 7ceb2aa7..92fbf2f6 100644 --- a/src/search/watcher.ts +++ b/src/search/watcher.ts @@ -38,7 +38,7 @@ import { resolveVaultPath } from "../storage/local.js"; import { getIndexStatus, markPathIndexing, markPathReady, onceIndexReady } from "./index-state.js"; import { indexDocument, readManifest, writeManifest } from "./reindex.js"; import { consumeSelfWrite } from "./self-write.js"; -import { getProvider } from "./vector.js"; +import { getProvider, getQuantize } from "./vector.js"; // 500ms is the floor the design locks in: short enough for the index to feel // live to a human typing in their editor, long enough to coalesce an @@ -94,7 +94,7 @@ async function defaultDeleteFn( vaultRoot: string, relPath: string, ): Promise> { - const dbResult = openIndexDb(vaultRoot, getProvider().dim); + const dbResult = openIndexDb(vaultRoot, getProvider().dim, getQuantize()); if (!dbResult.ok) return dbResult; const db = dbResult.value; try { diff --git a/src/serve/index.ts b/src/serve/index.ts index aaec23ee..d1c5b231 100644 --- a/src/serve/index.ts +++ b/src/serve/index.ts @@ -1,37 +1,51 @@ -// `daftari serve` (#5, spec 2026-07-20) — server mode over Streamable HTTP. +// `daftari serve` (#5, spec 2026-07-20; stateless per spec 2026-07-26 +// Decision 1) — server mode over the 2026-07-28 MCP revision. // -// One always-on instance, many MCP clients. The mechanical core: createServer -// already parameterizes the access context, so each MCP session gets its own -// Server bound to the identity resolved when the session opened — no tool -// handler changes, and every RBAC/existence-disclosure invariant applies -// per session, transport-independently. +// One always-on instance, many MCP clients, NO sessions: the 2026-07-28 +// revision removed the initialize handshake and the Mcp-Session-Id header, so +// identity is per REQUEST, resolved on every request against the same +// config-declared map. createServer already parameterizes the access context, +// so each request gets an instance bound to the identity its own bearer +// resolved to — no tool handler changes, and every RBAC/existence-disclosure +// invariant applies per request, transport-independently. // -// Fail-loud rules (all from the spec, all startup or session-open errors, +// Serve speaks the 2026-07-28 revision only (`legacy: "reject"`): no +// dual-stacking, the precedent set by refusing the deprecated HTTP+SSE +// transport. Lagging clients use stdio, which serves both eras. +// +// Fail-loud rules (all from the spec, all startup or request-time errors, // never silent downgrades): // - non-loopback bind requires auth configured AND // server.transport_security: external declared; // - a token entry whose env var is unset, or whose role is not declared, // refuses to start; // - once auth is configured, a missing/unmatched bearer token is rejected -// at session open (401) on every bind — never downgraded to guest; +// (401) on EVERY request on every bind — never downgraded to guest; // - the deny-all guest exists only in the no-auth loopback configuration. -import { randomUUID, timingSafeEqual } from "node:crypto"; +import { timingSafeEqual } from "node:crypto"; import { createServer as createHttpServer, type IncomingMessage, type ServerResponse, } from "node:http"; import { resolve } from "node:path"; -import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js"; -import { isInitializeRequest } from "@modelcontextprotocol/sdk/types.js"; +import { toNodeHandler } from "@modelcontextprotocol/node"; +import { type AuthInfo, createMcpHandler } from "@modelcontextprotocol/server"; import { createRemoteJWKSet, jwtVerify } from "jose"; import { type AccessContext, GUEST_ROLE, resolveAccess } from "../access/rbac.js"; import { ok, type Result } from "../frontmatter/types.js"; import { installShutdownHandlers, parseFlag, startVaultServices } from "../index.js"; import { acquireLock } from "../lifecycle/lock.js"; +import { setRerankProvider } from "../search/rerank-provider.js"; import { setProvider } from "../search/vector.js"; -import { createServer, resolveToolExposure, SERVER_VERSION } from "../server.js"; +import { + advertisedSurfaceCost, + allRegisteredTools, + createServer, + resolveToolExposure, + SERVER_VERSION, +} from "../server.js"; import { createBackend, type StorageBackend } from "../storage/backend.js"; import { directoryExists } from "../storage/local.js"; import { syncVault } from "../storage/sync.js"; @@ -53,24 +67,26 @@ Defaults: --takeover deliberately replace a LIVE daftari holding this vault (a plain serve refuses against any live holder) -Endpoint: http://:/mcp (MCP Streamable HTTP) +Endpoint: http://:/mcp (MCP 2026-07-28, stateless — lagging + clients use stdio) -Auth: clients send "Authorization: Bearer ". Two composable schemes: +Auth: clients send "Authorization: Bearer " on EVERY request. Two +composable schemes: server.auth.tokens — static tokens; values come from the env vars named in config, never from config itself. server.auth.oauth — OAuth 2.1 resource-server validation: bearer JWTs are verified against the IdP's JWKS (issuer + audience + expiry) and the subject claim maps through the declared subjects table. A valid token with an unmapped subject is 403 (authenticated, not authorized). -With any auth configured, a missing/invalid credential is a 401 at session -open — never a guest downgrade; with no auth (loopback only), sessions run +With any auth configured, a missing/invalid credential is a 401 on every +request — never a guest downgrade; with no auth (loopback only), requests run as the deny-all guest. Exit codes: 2 config/usage error, 3 runtime error. `; // A resolved phase-1 credential: the secret bytes and the identity a match -// binds the session to. +// binds the request to. interface ResolvedToken { secret: Buffer; user: string; @@ -199,39 +215,12 @@ function writeJson(res: ServerResponse, status: number, body: unknown): void { res.end(JSON.stringify(body)); } -function readBody(req: IncomingMessage): Promise { - return new Promise((resolveBody, rejectBody) => { - const chunks: Buffer[] = []; - req.on("data", (c: Buffer) => chunks.push(c)); - req.on("end", () => { - const raw = Buffer.concat(chunks).toString("utf-8"); - if (raw.length === 0) { - resolveBody(undefined); - return; - } - try { - resolveBody(JSON.parse(raw)); - } catch (e) { - rejectBody(e); - } - }); - req.on("error", rejectBody); - }); -} - -interface LiveSession { - transport: StreamableHTTPServerTransport; - // Identity bound at session open. Later requests must present a credential - // resolving to the SAME user — a session id is not a bearer credential. - user: string; -} - export interface ServeHandle { port: number; close: () => Promise; } -// Starts the HTTP listener and session router. Exported separately from +// Starts the HTTP listener and per-request router. Exported separately from // runServe so tests can drive a live server in-process on an ephemeral port // without argv parsing, lock acquisition, or process-global side effects. // DNS-rebinding guard for LOOPBACK binds (MCP Streamable HTTP security @@ -276,7 +265,6 @@ export function startHttpServer( bind: string, port: number, ): Promise { - const sessions = new Map(); const oauth = config.server.oauth; const authConfigured = tokens.length > 0 || oauth !== undefined; // JWKS key set, created lazily on the first OAuth verification: jose @@ -287,15 +275,16 @@ export function startHttpServer( // port); no request can arrive before listen resolves. let loopbackGuard: LoopbackGuard | null = null; - // Resolves the request's identity under the spec's session rules, or - // writes the rejection and returns null. With auth configured: + // Resolves the request's identity — the first line of EVERY request + // (spec 2026-07-26, Decision 1) — or writes the rejection and returns + // null. With auth configured: // - a static-token match binds its declared identity; // - else, with oauth declared, a bearer that verifies against the IdP's // JWKS (issuer + audience + signature + expiry) maps its subject claim // through the declared table — a valid-but-unmapped subject is 403 // (authenticated, not authorized), NEVER guest; // - anything else is 401. With no auth at all (startup gating - // guarantees loopback) every session is the deny-all guest. + // guarantees loopback) every request runs as the deny-all guest. const authenticate = async ( req: IncomingMessage, res: ServerResponse, @@ -345,6 +334,28 @@ export function startHttpServer( return null; }; + // The MCP handler: per-request, stateless, 2026-07-28 only. The factory + // runs once per request with the identity our authenticate() resolved and + // stashed in the pass-through authInfo — createServer parameterizes the + // access context, which is what makes this migration (like 2026-07-20's) + // cheap. `legacy: "reject"` answers 2025-era traffic with the + // unsupported-protocol-version error: no dual-stacking; lagging clients + // use stdio. + // + // Single-holder stays the process lock's job, not the transport's: two + // daftari processes on one vault is what .daftari/process.lock refuses + // (2026-07-20 Decision 4), stateless wire or not. + const mcpHandler = createMcpHandler( + ({ authInfo }) => { + const access = + (authInfo?.extra as { access?: AccessContext } | undefined)?.access ?? + resolveAccess(config, "guest", GUEST_ROLE); + return createServer(vaultRoot, access, config.tools); + }, + { legacy: "reject" }, + ); + const nodeHandler = toNodeHandler(mcpHandler); + const httpServer = createHttpServer((req, res) => { void handle(req, res).catch((e) => { const reason = e instanceof Error ? e.message : String(e); @@ -381,56 +392,16 @@ export function startHttpServer( const access = await authenticate(req, res); if (access === null) return; - const sessionId = req.headers["mcp-session-id"]; - const existing = typeof sessionId === "string" ? sessions.get(sessionId) : undefined; - - if (existing) { - // A session id is routing state, not a credential: the request's own - // bearer must resolve to the identity the session was opened with. - if (existing.user !== access.user) { - writeJson(res, 401, { - error: "unauthorized", - message: "credential does not match the session's identity", - }); - return; - } - const body = req.method === "POST" ? await readBody(req) : undefined; - await existing.transport.handleRequest(req, res, body); - return; - } - - if (req.method !== "POST") { - writeJson(res, 400, { error: "bad_request", message: "unknown or missing session" }); - return; - } - const body = await readBody(req); - if (!isInitializeRequest(body)) { - writeJson(res, 400, { - error: "bad_request", - message: "expected an initialize request to open a session", - }); - return; - } - - const transport = new StreamableHTTPServerTransport({ - sessionIdGenerator: () => randomUUID(), - onsessioninitialized: (id) => { - sessions.set(id, { transport, user: access.user }); - }, - onsessionclosed: (id) => { - sessions.delete(id); - }, - }); - transport.onclose = () => { - const id = transport.sessionId; - if (id) sessions.delete(id); + // toNodeHandler forwards req.auth as the handler's pass-through authInfo + // (it performs no verification of its own — ours ran above). The bearer + // is the credential; `_meta` client info is diagnostics, never identity. + (req as IncomingMessage & { auth?: AuthInfo }).auth = { + token: bearerFrom(req) ?? "", + clientId: access.user, + scopes: [], + extra: { access }, }; - - // One Server per session, bound to the session's identity — the whole - // point of Decision 2. - const server = createServer(vaultRoot, access, config.tools); - await server.connect(transport); - await transport.handleRequest(req, res, body); + await nodeHandler(req, res); } return new Promise((resolveStart, rejectStart) => { @@ -443,10 +414,10 @@ export function startHttpServer( resolveStart({ port: boundPort, close: async () => { - // Sessions close concurrently — shutdown latency must not scale - // with the number of live clients. - await Promise.all([...sessions.values()].map((s) => s.transport.close().catch(() => {}))); - sessions.clear(); + // close() aborts in-flight exchanges and resolves once every + // per-request instance has terminated — there is no session table + // to drain. + await mcpHandler.close(); await new Promise((r) => httpServer.close(() => r())); }, }); @@ -561,7 +532,11 @@ export async function runServe(argv: string[]): Promise { }); try { - setProvider(config.value.embeddingProvider); + setProvider(config.value.embeddingProvider, { + dim: config.value.embeddingDim ?? undefined, + quantize: config.value.embeddingQuantize, + }); + setRerankProvider(config.value.rerankProvider); } catch (e) { process.stderr.write(`daftari serve: ${e instanceof Error ? e.message : String(e)}\n`); return 3; @@ -574,6 +549,19 @@ export async function runServe(argv: string[]): Promise { ); } + // Startup cost line — the same measurement stdio's main() logs (spec + // 2026-07-26-context-packs-progressive-disclosure-design.md, Phase 1.4). + { + const exposedNames = resolveToolExposure(config.value.tools).exposed; + const exposedTools = allRegisteredTools().filter((t) => exposedNames.has(t.name)); + const cost = advertisedSurfaceCost(exposedTools); + process.stderr.write( + `daftari: advertising ${exposedTools.length} tools (~${cost} tokens of definitions; ` + + `tier=${config.value.tools.tier}). A future major release will default tools.tier to ` + + "'core' — vault_tools makes every tool discoverable in-band.\n", + ); + } + try { handle = await startHttpServer(vaultRoot, config.value, gate.tokens, bind, port); } catch (e) { diff --git a/src/server.ts b/src/server.ts index 0b210c49..edc70b5d 100644 --- a/src/server.ts +++ b/src/server.ts @@ -5,32 +5,33 @@ // additionally guards against unexpected throws at the transport boundary so a // bug cannot take the stdio connection down. +import { randomBytes } from "node:crypto"; import { readFileSync } from "node:fs"; -import { Server } from "@modelcontextprotocol/sdk/server/index.js"; import { - CallToolRequestSchema, - ListResourcesRequestSchema, - ListResourceTemplatesRequestSchema, - ListToolsRequestSchema, - ReadResourceRequestSchema, -} from "@modelcontextprotocol/sdk/types.js"; + acceptedContent, + type CallToolResult, + createRequestStateCodec, + type InputRequiredResult, + inputRequired, + ResourceNotFoundError, + Server, + type ServerContext, + type Tool, +} from "@modelcontextprotocol/server"; import { type AccessContext, guestAccess } from "./access/rbac.js"; import { docUri, listResources, readResource, resourceTemplates } from "./resources.js"; -import { consumesTools } from "./tools/consumes.js"; -import { curationTools } from "./tools/curation.js"; -import { edgeStalenessTools } from "./tools/edge-staleness.js"; -import { edgeTools } from "./tools/edges.js"; -import { readTools, type ToolDefinition } from "./tools/read.js"; -import { receiptTools } from "./tools/receipt.js"; -import { searchTools } from "./tools/search.js"; -import { stagedActionTools } from "./tools/staged-actions.js"; -import { themesTools } from "./tools/themes.js"; -import { tier1Tools } from "./tools/tier1.js"; -import { tier2Tools } from "./tools/tier2.js"; -import { witnessTools } from "./tools/witness.js"; -import { writeTools } from "./tools/write.js"; +import type { ToolDefinition } from "./tools/read.js"; +import { allTools, registeredToolNames, serializeToolDefinition } from "./tools/registry.js"; +import { describeRatifyElicitation } from "./tools/staged-actions.js"; import type { ToolsConfig } from "./utils/config.js"; +// Re-exported so existing importers (tests, docs) keep working unchanged — +// the registry itself moved to tools/registry.ts (spec 2026-07-26 +// context-packs-progressive-disclosure, Phase 1.1) to break an import cycle +// (vault_tools closes over the full registry; a tools file cannot import +// server.ts back). +export { allRegisteredTools, registeredToolNames } from "./tools/registry.js"; + export const SERVER_NAME = "daftari"; // The version is read from the package manifest so it never drifts from the @@ -41,28 +42,6 @@ const manifest = JSON.parse(readFileSync(new URL("../package.json", import.meta. }; export const SERVER_VERSION = manifest.version; -// The full registry. Static — assembled once at module load, shared by every -// server instance and by the tier-exposure helpers below. -const allTools: ToolDefinition[] = [ - ...readTools, - ...receiptTools, - ...witnessTools, - ...searchTools, - ...themesTools, - ...writeTools, - ...curationTools, - ...stagedActionTools, - ...edgeTools, - ...consumesTools, - ...tier1Tools, - ...tier2Tools, - ...edgeStalenessTools, -]; - -export function registeredToolNames(): string[] { - return allTools.map((t) => t.name); -} - // Tool-exposure tiers (#103). Tiers are additive: standard = core + its own // list; full = the whole registry (never enumerated, so a new tool is // full-tier by default and only joins a leaner tier deliberately). @@ -72,6 +51,12 @@ export function registeredToolNames(): string[] { // RBAC vaults with propose-only roles — plus index diagnostics. Everything // else (tensions, themes, witness/receipt epistemics, the edge graph, // tier-1/tier-2 dispatch, staleness) is specialist curation surface: full. +// +// vault_tools and vault_context join core in the same wave that ships them +// (spec 2026-07-26 context-packs-progressive-disclosure, Phase 1.4): the +// long-tail-behind-an-index tool and the task-brief assembler are exactly +// what makes a core-tier session no longer an amputation — every other tool +// stays discoverable in-band via vault_tools even when not advertised. export const CORE_TOOLS: readonly string[] = [ "vault_search", "vault_read", @@ -79,6 +64,8 @@ export const CORE_TOOLS: readonly string[] = [ "vault_index", "vault_lint", "vault_status", + "vault_tools", + "vault_context", ]; export const STANDARD_TOOLS: readonly string[] = [ @@ -130,6 +117,208 @@ export function resolveToolExposure(tools: ToolsConfig): ToolExposure { return { exposed, unknown: [...unknown] }; } +// Measures what ListTools actually pushes onto the wire for a resolved +// exposure set: the same serialization ListTools ships (serializeToolDefinition), +// JSON.stringify'd, chars/4 — the same estimator vault_context uses (spec +// 2026-07-26 context-packs-progressive-disclosure, Phase 1.4), so the +// startup log and a pack's budget accounting speak the same units. Not the +// registry's cost (registeredToolNames().length worth) — the tier-resolved +// EXPOSED set, which is the question "what does THIS session's ListTools +// actually cost", the thing the whole spec is about. +export function advertisedSurfaceCost(tools: ToolDefinition[]): number { + const serialized = tools.map(serializeToolDefinition); + return Math.ceil(JSON.stringify(serialized).length / 4); +} + +// MCP content block shapes this module emits. Kept local (not imported from +// the SDK) so this file's return types stay self-describing; the SDK +// accepts a wider shape, this is the subset we ever construct. +interface TextBlock { + type: "text"; + text: string; +} +interface ResourceLinkBlock { + type: "resource_link"; + uri: string; + name: string; + mimeType: string; +} + +// The CallTool bridge's presentation step (spec 2026-07-26, Decision 3), +// pulled out of the request handler as its own pure-ish function so it can +// be unit-tested directly against ANY ToolDefinition — including a +// hand-built stub with no `summarize`, or one that throws — without needing +// a live Server/transport (test/server.test.ts, C5). Takes the tool's +// already-successful ok-value; the caller (createServer's CallTool handler) +// owns the RBAC/dispatch/error-branch decisions around it. +// +// Three channels: +// content — a compact, model-facing summary, plus resource_link +// entries for the docs the result references; +// structuredContent — the full typed result, matching outputSchema (or a +// tool-projected subset — see wireValue). +// +// A tool with no `summarize` falls back to the pretty-printed value, so this +// is backward compatible for any tool that has not opted in. +// +// Presentation hardening (C5): `summarize`/`docLinks` are pure functions +// over an already-successful result, but a summarizer bug must never turn a +// correct tool call into an error response — that would be worse than the +// JSON.stringify fallback it was meant to improve on. Each runs in its own +// try/catch with the same fallback the tool would get by not opting in. +export function formatSuccessResult( + tool: ToolDefinition, + value: unknown, +): { content: (TextBlock | ResourceLinkBlock)[]; structuredContent: Record } { + let summary: string; + try { + summary = tool.summarize ? tool.summarize(value) : JSON.stringify(value, null, 2); + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + process.stderr.write(`daftari: warning: summarize threw for ${tool.name}: ${reason}\n`); + summary = JSON.stringify(value, null, 2); + } + let links: string[] = []; + try { + links = tool.docLinks ? tool.docLinks(value) : []; + } catch (e) { + const reason = e instanceof Error ? e.message : String(e); + process.stderr.write(`daftari: warning: docLinks threw for ${tool.name}: ${reason}\n`); + links = []; + } + const wireValue = tool.wireValue ? tool.wireValue(value) : (value as Record); + return { + content: [ + { type: "text", text: summary }, + // Links inherit read-gating: a handler only ever names docs the caller + // may read, so every link emitted here is readable by construction + // (Decision 3). Filtered to non-empty strings so a summarizer bug (or + // a legitimate empty-string edge case) never mints a resource_link + // with no uri. + ...links + .filter((path) => typeof path === "string" && path.length > 0) + .map( + (path): ResourceLinkBlock => ({ + type: "resource_link", + uri: docUri(path), + name: path, + mimeType: "text/markdown", + }), + ), + ], + structuredContent: wireValue, + }; +} + +// vault_ratify form-mode elicitation (spec 2026-07-26, Decision 5): called +// without a decision, the tool answers with an input_required form instead of +// an error — the server proposes, the human disposes, and the wire format +// itself says so. The opaque request state carries the action id, the vault +// HEAD at proposal time, and the deciding user, HMAC-signed so it round-trips +// untampered; the server remembers nothing between the two requests. +// +// A per-process random key is sound here because the process lock guarantees +// exactly one daftari process serves every round of a flow (2026-07-20 +// Decision 4); a restart invalidates in-flight forms, and the client simply +// re-calls. TTL matches the codec default posture: a stale form must not +// ratify. +// +// Single-`id` calls only — a batch `ids` call always requires an explicit +// decision (elicitation shows one action's rationale; there is no form for +// "decide N actions at once"). createServer's CallTool handler only enters +// this path when `ids` is absent. +interface RatifyElicitState { + action: string; + head: string | null; + user: string; +} + +const ratifyStateCodec = createRequestStateCodec({ + key: randomBytes(32), + ttlSeconds: 600, +}); + +// One round of the vault_ratify elicitation flow, entered only when the call +// carries no decision. Returns a `reply` to short-circuit with (the form, a +// declined-form acknowledgement, or a gate error), or the args augmented with +// the elicited decision to fall through to the normal dispatch — which +// re-validates pending/conflict-free exactly as a direct call would. +async function ratifyElicitationRound( + vaultRoot: string, + args: Record, + access: AccessContext, + ctx: ServerContext, +): Promise< + | { reply: CallToolResult | InputRequiredResult; args?: undefined } + | { reply?: undefined; args: Record } +> { + // Resubmit path: a verified state minted by THIS process, bound to the + // same user and the same action. The state is bearer proof of nothing more + // than "this identity was shown this form" — the ratify grant and the + // action's pending/conflict-free status are re-checked downstream. + const state = ctx.mcpReq.requestState(); + if (state && state.user === access.user && state.action === args.id) { + const answer = acceptedContent<{ decision?: unknown }>(ctx.mcpReq.inputResponses, "decision"); + const decision = answer?.decision; + if (decision === "approve" || decision === "reject") { + return { args: { ...args, decision } }; + } + // Declined or cancelled: apply nothing, record nothing. The action + // stays pending — the safe answer is the one that changes nothing. + return { + reply: { + content: [ + { + type: "text" as const, + text: + `vault_ratify: elicitation declined — staged action ` + + `${state.action} remains pending`, + }, + ], + }, + }; + } + + // First round: run the same gates vaultRatify would (grant, propose-only, + // unknown/decided action) so a caller that could not ratify never sees a + // form, then hand the client the form plus the signed state. + const form = await describeRatifyElicitation(vaultRoot, args, access); + if (!form.ok) { + return { + reply: { + isError: true, + content: [{ type: "text" as const, text: `Error: ${form.error.message}` }], + }, + }; + } + return { + reply: inputRequired({ + inputRequests: { + decision: inputRequired.elicit({ + message: form.value.message, + requestedSchema: { + type: "object", + required: ["decision"], + properties: { + decision: { + type: "string", + enum: ["approve", "reject"], + default: "reject", + description: "Approve applies the staged action; reject records the refusal.", + }, + }, + }, + }), + }, + requestState: await ratifyStateCodec.mint({ + action: form.value.actionId, + head: form.value.head, + user: access.user, + }), + }), + }; +} + // The server runs as one access identity for its whole lifetime — the // --user / --role it was started with. Every tool call is enforced against it. // Absent an explicit context the server falls back to the deny-all guest. @@ -143,46 +332,57 @@ export function createServer( ): Server { const server = new Server( { name: SERVER_NAME, version: SERVER_VERSION }, - { capabilities: { tools: {}, resources: {} } }, + { + capabilities: { tools: {}, resources: {} }, + // Decision 5: echoed request state is attacker-controlled input; the + // codec's verify hook is what makes ctx.mcpReq.requestState() trusted. + requestState: { verify: ratifyStateCodec.verify }, + }, ); const byName = new Map(allTools.map((t) => [t.name, t])); const exposedNames = toolsConfig ? resolveToolExposure(toolsConfig).exposed : null; const exposed = exposedNames ? allTools.filter((t) => exposedNames.has(t.name)) : allTools; - server.setRequestHandler(ListToolsRequestSchema, async () => ({ - tools: exposed.map((t) => ({ - name: t.name, - ...(t.title ? { title: t.title } : {}), - description: t.description, - inputSchema: t.inputSchema, - outputSchema: t.outputSchema, - ...(t.annotations ? { annotations: t.annotations } : {}), - })), + server.setRequestHandler("tools/list", async () => ({ + tools: exposed.map((t) => { + const s = serializeToolDefinition(t); + // ToolDefinition keeps schemas as plain Record JSON Schema (2020-12); + // the wire type narrows the root to `type: "object"`, which every + // registered schema satisfies by construction. + return { + ...s, + inputSchema: s.inputSchema as Tool["inputSchema"], + outputSchema: s.outputSchema as Tool["outputSchema"], + }; + }), })); // Resources (spec 2026-07-26, Decision 2). Every listing and read resolves // against this server's access context, exactly as tools do — a resource is // not a back door around RBAC. src/resources.ts holds the disclosure rules. - server.setRequestHandler(ListResourceTemplatesRequestSchema, async () => ({ + server.setRequestHandler("resources/templates/list", async () => ({ resourceTemplates: resourceTemplates(), })); - server.setRequestHandler(ListResourcesRequestSchema, async () => { + server.setRequestHandler("resources/list", async () => { const result = await listResources(vaultRoot, access); // A listing failure yields an empty list rather than an error: a doc list // that fails loudly for some callers and not others is itself a signal. return { resources: result.ok ? result.value : [] }; }); - server.setRequestHandler(ReadResourceRequestSchema, async (request) => { + server.setRequestHandler("resources/read", async (request) => { const uri = request.params.uri; const result = await readResource(vaultRoot, uri, access); - if (!result.ok) throw new Error(result.error.message); + // One error for "no such document" and "you may not read it" alike — + // resources.ts keeps the messages byte-identical (omission over + // redaction), and this single throw site keeps the wire code identical. + if (!result.ok) throw new ResourceNotFoundError(uri, result.error.message); return { contents: [result.value] }; }); - server.setRequestHandler(CallToolRequestSchema, async (request) => { + server.setRequestHandler("tools/call", async (request, ctx) => { const name = request.params.name; const tool = byName.get(name); if (!tool) { @@ -192,7 +392,23 @@ export function createServer( }; } try { - const args = (request.params.arguments ?? {}) as Record; + let args = (request.params.arguments ?? {}) as Record; + // vault_ratify without a decision speaks form-mode elicitation + // (Decision 5) — single-`id` calls only; a batch `ids` call always + // requires an explicit decision, so it falls straight through to the + // tool handler's own validation. A direct call with the decision + // inline never enters this branch — it keeps working for clients that + // don't do elicitation, and on 2025-era connections the SDK's legacy + // shim fulfils the form over the session. + if ( + name === "vault_ratify" && + args.ids === undefined && + typeof args.decision !== "string" + ) { + const round = await ratifyElicitationRound(vaultRoot, args, access, ctx); + if (round.reply !== undefined) return round.reply; + args = round.args; + } const result = await tool.handler(vaultRoot, args, access); if (!result.ok) { return { @@ -200,35 +416,9 @@ export function createServer( content: [{ type: "text" as const, text: `Error: ${result.error.message}` }], }; } - // Three channels (spec 2026-07-26, Decision 3): - // structuredContent — the full typed result, matching outputSchema; - // content — a compact, model-facing summary; - // resource_link — handles for the docs the result references, - // so the agent reads the two it needs at full - // fidelity instead of receiving twenty bodies it - // will truncate in context anyway. - // - // A tool with no `summarize` falls back to the pretty-printed value, so - // this is backward compatible for any tool that has not opted in. - const summary = tool.summarize - ? tool.summarize(result.value) - : JSON.stringify(result.value, null, 2); - const links = tool.docLinks ? tool.docLinks(result.value) : []; - return { - content: [ - { type: "text" as const, text: summary }, - // Links inherit read-gating: a handler only ever names docs the - // caller may read, so every link emitted here is readable by - // construction (Decision 3). - ...links.map((path) => ({ - type: "resource_link" as const, - uri: docUri(path), - name: path, - mimeType: "text/markdown", - })), - ], - structuredContent: result.value as Record, - }; + // Presentation (spec 2026-07-26, Decision 3) is a pure function of the + // tool and its ok-value — see formatSuccessResult above. + return formatSuccessResult(tool, result.value); } catch (e) { const reason = e instanceof Error ? e.message : String(e); return { diff --git a/src/storage/index-db.ts b/src/storage/index-db.ts index 365acb41..c90f7c7a 100644 --- a/src/storage/index-db.ts +++ b/src/storage/index-db.ts @@ -56,21 +56,38 @@ export type IndexDb = Database.Database; // answered with one indexed query at read/search time. // 10 added the `collection` partition key to `embeddings_vec` (2026-07-26 // retrieval-fusion spec, Decision 3), in #303. -// 11 covers the valid-time columns (valid_from, valid_until) and the index on -// superseded_by that the bi-temporal walk queries in reverse. Those columns -// arrived in #305, which added them to the `documents` DDL and to the upsert -// but left this constant at "10" — the comment there claimed the bump covered -// them, and no bump had happened. `CREATE TABLE IF NOT EXISTS` is a no-op -// against an existing table, so every index built between #303 and #305 stored -// version 10 with no valid_from column, skipped the rebuild on the version -// check, and then failed the first upsert with `no such column: valid_from`. -// CI never caught it because `.daftari/index.db` is gitignored and every run -// builds a fresh index; only an upgrade in place reaches that state. -// -// The bump is cheap now precisely because #305 also removed `embeddings` from -// the drop list below: derived tables are rebuilt from the markdown, and the -// durable vector cache survives. -const SCHEMA_VERSION = "11"; +// 11 on main covers the valid-time columns (valid_from, valid_until) and the +// index on superseded_by that the bi-temporal walk queries in reverse. Those +// columns arrived in #305, which added them to the `documents` DDL and to the +// upsert but left this constant at "10" — every index built between #303 and +// #305 stored version 10 with no valid_from column, skipped the rebuild on +// the version check, and then failed the first upsert with `no such column: +// valid_from`. #309 fixed that by claiming "11". +// The enhancement-wave branch, cut before #309, had independently claimed +// 11-13 for its own chain; the merge renumbers that chain 12-14 so no value +// is claimed twice and every pre-merge index — either lineage — rebuilds: +// 11 -> 12: chunks.context + two-column chunks_fts(context, text) — +// contextual chunking, spec 2026-07-26-contextual-chunking-reranker-design.md +// Decision 2. Every chunk's hash input changes (context is now hashed WITH +// the text — see embeddingInput in search/vector.ts), so this bump forces a +// full re-embed by design; the release notes carry the cost. +// 12 -> 13: derives_from_edges.k_eff — the independence-aware-promotion +// spec's shadow-only effective-k column (2026-07-26-independence-aware- +// promotion-design.md, Decisions 1-2/4). A column addition to a jsonl-derived +// table, so the version-mismatch path's existing drop-and-rebuild of +// derives_from_edges covers it; no migration to write. +// 13 -> 14: documents.updated_by — the context-packs spec's provenance line +// (2026-07-26-context-packs-progressive-disclosure-design.md, final plan +// 2.5): vault_context's per-entry "updated N by M" flag reads this column +// instead of re-parsing frontmatter. Populated from `frontmatter.updated_by` +// on every write path (stageOne, src/search/reindex.ts); NOT NULL DEFAULT '' +// so the version-mismatch path's full rebuild backfills every existing row +// and no migration statement is needed for a mid-version upgrade — the index +// is an ephemeral cache, rebuilt wholesale on a version bump. +// The bumps stay cheap because #305 removed `embeddings` from the drop list +// below: derived tables are rebuilt from the markdown, and the durable +// vector cache survives. +const SCHEMA_VERSION = "14"; // Meta key that records the dim at which `embeddings_vec` was created. Used // on every open to decide whether to rebuild the virtual table (provider @@ -79,6 +96,23 @@ const SCHEMA_VERSION = "11"; // cache hits. const VEC_DIM_META_KEY = "embeddings_vec_dim"; +// Meta key that records the sqlite-vec column TYPE `embeddings_vec` was +// created at ("float32" | "int8") — twin of VEC_DIM_META_KEY (2026-07-26 +// embedding-refresh-quantization spec, Phase 3a). A quantize flip +// (embeddings.quantize: none -> int8, or back) on an otherwise-unchanged +// vault must trigger the same drop-and-recreate a dim mismatch does — the +// vec0 column type is fixed at CREATE TABLE time, same as its width. +const VEC_KIND_META_KEY = "embeddings_vec_kind"; + +// The sqlite-vec column representation. "float32" is today's exact behaviour +// (byte-for-byte); "int8" is the quantized-index path (spec Decision 3) — +// components are int8-scaled ([-127, 127]) under the calibration-free +// assumption that every provider L2-normalizes (components live in +// [-1, 1]). The durable `embeddings` cache is ALWAYS float32, native-dim, +// regardless of this setting — quantization is an index-representation +// choice only, droppable and rebuildable from the cache at any time. +export type VecKind = "float32" | "int8"; + export interface IndexedDocument { path: string; title: string; @@ -95,12 +129,18 @@ export interface IndexedDocument { supersededBy: string | null; validFrom: string | null; validUntil: string | null; + // Frontmatter `updated_by` (spec 2026-07-26-context-packs-progressive- + // disclosure-design.md, final plan 2.5) — vault_context's provenance flag + // reads this instead of a second frontmatter parse. '' when the document + // authors no updated_by (schema default). + updatedBy: string; } export interface IndexedChunk { path: string; chunkIndex: number; text: string; + context: string; contentHash: string; embedding: Float32Array | null; } @@ -129,6 +169,11 @@ CREATE TABLE IF NOT EXISTS documents ( ttl_days INTEGER, created TEXT NOT NULL DEFAULT '', superseded_by TEXT, + -- Frontmatter updated_by (spec 2026-07-26-context-packs-progressive- + -- disclosure-design.md, final plan 2.5). NOT NULL DEFAULT '' matching the + -- created column's convention: an "undateable"-shaped sibling for + -- "unattributed". + updated_by TEXT NOT NULL DEFAULT '', -- Valid time. Nullable with no default: NULL is the unknown sentinel, and -- unlike created (required, so "" means undateable) these fields are -- optional, so NULL carries the meaning directly. @@ -143,6 +188,13 @@ CREATE TABLE IF NOT EXISTS chunks ( path TEXT NOT NULL, chunk_index INTEGER NOT NULL, text TEXT NOT NULL, + -- One-line breadcrumb context ({collection} > {title} > {headings}, tags), + -- spec 2026-07-26 Decision 2. Part of the chunk's retrieval identity: it is + -- hashed and embedded together with the chunk text (see embeddingInput, + -- search/vector.ts) and is the second chunks_fts column. DEFAULT '' only + -- matters for direct low-level inserts (tests); every real write path + -- (reindex.ts) always supplies a real breadcrumb. + context TEXT NOT NULL DEFAULT '', content_hash TEXT NOT NULL, PRIMARY KEY (path, chunk_index) ); @@ -188,6 +240,7 @@ CREATE TABLE IF NOT EXISTS derives_from_edges ( to_path TEXT NOT NULL, strength REAL NOT NULL, k_survived INTEGER NOT NULL, + k_eff REAL NOT NULL DEFAULT 0, first_observed TEXT NOT NULL, last_rederived TEXT NOT NULL, last_age_decay TEXT NOT NULL, @@ -212,15 +265,22 @@ CREATE INDEX IF NOT EXISTS idx_edges_status ON derives_from_edges(status); // stock English stemming pipeline; it lowercases, strips diacritics, and // folds plurals / -ing forms. // -// Also contains chunks_fts: an FTS5 external-content table over `chunks` -// using the same pattern. FTS sync relies on delete-before-insert: every -// write path deletes a path's chunk rows before inserting new ones, so -// the triggers fire in the right order. chunks_au is defensive — no current -// write path UPDATEs a chunk row in place — but is included for correctness. -// Note: recursive_triggers is OFF in this project, so INSERT OR REPLACE -// conflict triggers do NOT fire both DELETE+INSERT; that is why the -// documents write path was migrated off INSERT OR REPLACE. chunks follows -// the same pattern. +// Also contains chunks_fts: a TWO-COLUMN FTS5 external-content table over +// `chunks` (context, text) — spec 2026-07-26 Decision 2. bm25(chunks_fts) +// scores both columns at default weight, spanning the breadcrumb context +// (title/collection/tags/headings) AND the body text: this IS contextual +// BM25. Column order matters for two SQL-level reasons: snippet()'s column +// index argument (hybrid.ts's chunkFtsRanking targets column 1, `text`, so a +// served snippet can never contain synthesized breadcrumb prose — spec +// Decision 4), and the `chunks_fts : (…)` prefix-restrict syntax elsewhere +// keys off column NAME, not position, so it is unaffected either way. FTS +// sync relies on delete-before-insert: every write path deletes a path's +// chunk rows before inserting new ones, so the triggers fire in the right +// order. chunks_au is defensive — no current write path UPDATEs a chunk row +// in place — but is included for correctness. Note: recursive_triggers is +// OFF in this project, so INSERT OR REPLACE conflict triggers do NOT fire +// both DELETE+INSERT; that is why the documents write path was migrated off +// INSERT OR REPLACE. chunks follows the same pattern. const FTS_SCHEMA = ` CREATE VIRTUAL TABLE IF NOT EXISTS documents_fts USING fts5( title, tags, content_body, @@ -243,20 +303,22 @@ CREATE TRIGGER IF NOT EXISTS documents_au AFTER UPDATE ON documents BEGIN VALUES (new.rowid, new.title, new.tags, new.content); END; CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5( - text, + context, text, content='chunks', content_rowid='rowid', tokenize='porter unicode61' ); CREATE TRIGGER IF NOT EXISTS chunks_ai AFTER INSERT ON chunks BEGIN - INSERT INTO chunks_fts(rowid, text) VALUES (new.rowid, new.text); + INSERT INTO chunks_fts(rowid, context, text) VALUES (new.rowid, new.context, new.text); END; CREATE TRIGGER IF NOT EXISTS chunks_ad AFTER DELETE ON chunks BEGIN - INSERT INTO chunks_fts(chunks_fts, rowid, text) VALUES('delete', old.rowid, old.text); + INSERT INTO chunks_fts(chunks_fts, rowid, context, text) + VALUES('delete', old.rowid, old.context, old.text); END; CREATE TRIGGER IF NOT EXISTS chunks_au AFTER UPDATE ON chunks BEGIN - INSERT INTO chunks_fts(chunks_fts, rowid, text) VALUES('delete', old.rowid, old.text); - INSERT INTO chunks_fts(rowid, text) VALUES (new.rowid, new.text); + INSERT INTO chunks_fts(chunks_fts, rowid, context, text) + VALUES('delete', old.rowid, old.context, old.text); + INSERT INTO chunks_fts(rowid, context, text) VALUES (new.rowid, new.context, new.text); END; `; @@ -343,12 +405,23 @@ function loadVecExtension(db: IndexDb): Result { return ok(undefined); } -// Creates the sqlite-vec virtual table at the given dim, dropping any -// existing copy first. `dim` is fixed at CREATE TABLE time for vec0, so a -// provider switch (which changes the active dim) means dropping and -// recreating; the durable `embeddings` cache survives, and the next reindex -// repopulates `embeddings_vec` from it. -function createVecTable(db: IndexDb, dim: number): void { +// Creates the sqlite-vec virtual table at the given dim + kind, dropping any +// existing copy first. Both `dim` and the column type are fixed at CREATE +// TABLE time for vec0, so a provider switch OR a quantize flip (which +// changes the active dim/kind) means dropping and recreating; the durable +// `embeddings` cache survives, and the next reindex repopulates +// `embeddings_vec` from it. +// +// `distance_metric=cosine` is used for BOTH kinds unless the Phase 0 spike +// (docs/superpowers/specs/2026-07-26-embedding-refresh-quantization- +// design.md, gate 3) finds cosine unsupported on int8 in the pinned +// sqlite-vec build, in which case the fallback is L2 — equivalent candidate +// ORDERING for unit vectors up to quantization error, never used as a score +// on the int8 path regardless (see hybrid.ts's vecRanking). That spike has +// not been run in this environment; cosine is used for both kinds here as +// the working assumption, with this comment as the pointer to revisit if a +// real int8 table creation fails. +function createVecTable(db: IndexDb, dim: number, kind: VecKind): void { if (!Number.isInteger(dim) || dim <= 0) { throw new Error( `cannot create embeddings_vec at non-positive dim ${dim} — ` + @@ -367,15 +440,33 @@ function createVecTable(db: IndexDb, dim: number): void { // from two collections gets one vec row per collection. That is the only // honest shape — a single row cannot carry two ACL labels — and the durable // `embeddings` cache stays content-addressed and deduped regardless. + const columnType = kind === "int8" ? `int8[${dim}]` : `FLOAT[${dim}]`; db.exec( `CREATE VIRTUAL TABLE embeddings_vec USING vec0( content_hash TEXT NOT NULL, model TEXT NOT NULL, collection TEXT NOT NULL, - embedding FLOAT[${dim}] distance_metric=cosine + embedding ${columnType} distance_metric=cosine );`, ); setMeta(db, VEC_DIM_META_KEY, String(dim)); + setMeta(db, VEC_KIND_META_KEY, kind); +} + +// Quantizes an L2-normalized Float32Array to int8: round(x * 127) clamped to +// [-127, 127]. Deterministic, provider-agnostic — calibration-free because +// every EmbeddingProvider normalizes its output (components live in +// [-1, 1]), so a fixed unit-range scale needs no per-model calibration step +// (spec Decision 3). JS-side rather than sqlite-vec's own quantize helper — +// keeps the same rounding rule visible and testable independent of the +// sqlite-vec build. +export function quantizeInt8(vec: Float32Array): Buffer { + const out = new Int8Array(vec.length); + for (let i = 0; i < vec.length; i++) { + const scaled = Math.round((vec[i] as number) * 127); + out[i] = Math.max(-127, Math.min(127, scaled)); + } + return Buffer.from(out.buffer, out.byteOffset, out.byteLength); } // Drops every row from `embeddings_vec`. Called by the reindex path when @@ -388,13 +479,34 @@ export function clearEmbeddingsVec(db: IndexDb): void { // Inserts a vector row into the sqlite-vec mirror. Separate from // `insertEmbedding` because the durable cache and the vec index are two // stores — the cache survives a vec-table rebuild on provider switch. +// `kind` selects the on-disk representation: "float32" (default — byte-for- +// byte today's behaviour) writes the vector as-is; "int8" quantizes it first +// (spec Decision 3) via `quantizeInt8`. `embedding` must already be at the +// table's configured dim (callers apply `toIndexDim`, src/search/vector.ts, +// before calling this — this function does no truncation of its own). export function insertEmbeddingVec( db: IndexDb, contentHash: string, model: string, collection: string, embedding: Float32Array, + kind: VecKind = "float32", ): void { + // sqlite-vec infers a bound blob's vector TYPE from its byte layout, which + // defaults to float32 — a raw int8-byte blob bound directly into an + // int8[] column is rejected ("expected type int8, but a float32 vector + // was provided"). The `vec_int8(?)` SQL function is the documented way to + // tell sqlite-vec the bound blob is already int8-encoded; confirmed + // empirically against the pinned sqlite-vec build in this repo (this is + // the "working insert/query binding form" the governing spec's Phase 0 + // gate 3 asks the (unrun) spike to record — recorded here instead, since + // the spike itself has not been run in this environment). + if (kind === "int8") { + db.prepare( + "INSERT INTO embeddings_vec(content_hash, model, collection, embedding) VALUES (?, ?, ?, vec_int8(?))", + ).run(contentHash, model, collection, quantizeInt8(embedding)); + return; + } db.prepare( "INSERT INTO embeddings_vec(content_hash, model, collection, embedding) VALUES (?, ?, ?, ?)", ).run(contentHash, model, collection, embeddingToBlob(embedding)); @@ -421,14 +533,27 @@ export function hasEmbeddingVec( return row !== undefined; } -// `expectedVecDim` is the active embedding provider's dim. If the persisted -// `embeddings_vec` was created at a different dim (or doesn't exist yet), -// it is dropped and recreated at the expected dim — the durable `embeddings` -// cache is untouched, so a switch back to the previous provider is all -// cache hits. `expectedVecDim` is required — pass the active provider's dim -// (e.g. `getProvider().dim`). Tests that don't exercise vector queries should -// use `LOCAL_MINILM_DIM` from `src/search/providers/local-minilm.ts`. -export function openIndexDb(vaultRoot: string, expectedVecDim: number): Result { +// `expectedVecDim` is the active embedding provider's dim; `expectedVecKind` +// is the active quantization kind ("float32" | "int8", `getQuantize()` in +// src/search/vector.ts). If the persisted `embeddings_vec` was created at a +// different dim OR kind (or doesn't exist yet), it is dropped and recreated +// at the expected dim/kind — the durable `embeddings` cache is untouched, so +// a switch back to the previous provider/quantize setting is all cache hits. +// +// `expectedVecKind` is REQUIRED (2026-07-26 embedding-refresh-quantization +// spec, disposition C2) for the same reason `expectedVecDim` already was +// (see the historic precedent this comment used to cite): a default here +// would turn every caller that doesn't thread the active quantize setting +// into a silent destructive drop the moment quantization is used anywhere +// in the vault. Making it required moves that hazard from a runtime bug +// class to a compile error — every production call site passes +// `getQuantize()`; test callers that don't exercise quantization pass +// `"float32"` explicitly. +export function openIndexDb( + vaultRoot: string, + expectedVecDim: number, + expectedVecKind: VecKind, +): Result { try { mkdirSync(join(vaultRoot, ".daftari"), { recursive: true }); const db = new Database(indexDbPath(vaultRoot)); @@ -488,23 +613,28 @@ export function openIndexDb(vaultRoot: string, expectedVecDim: number): Result 0; - if (!vecTableExists || persistedDim !== targetDim) { - createVecTable(db, targetDim); + if (!vecTableExists || persistedDim !== targetDim || persistedKind !== expectedVecKind) { + createVecTable(db, targetDim, expectedVecKind); } setMeta(db, "schema_version", SCHEMA_VERSION); @@ -613,6 +743,13 @@ export function insertDocument(db: IndexDb, doc: IndexedDocument): void { // authored value stays on disk untouched for vault_lint to name. const validFrom = doc.validFrom === null ? null : (normalizeIsoDate(doc.validFrom) ?? null); const validUntil = doc.validUntil === null ? null : (normalizeIsoDate(doc.validUntil) ?? null); + // Defensive default, not just a type-contract convenience: better-sqlite3 + // throws on an `undefined` bind, and IndexedDocument is a plain interface — + // a construction site outside this file's own writers (test fixtures built + // by hand before this field existed) can supply an object shaped without + // `updatedBy` and TypeScript's structural check never runs on it at + // runtime. `?? ""` matches the column's own NOT NULL DEFAULT ''. + const updatedBy = doc.updatedBy ?? ""; // ON CONFLICT(path) DO UPDATE (rather than INSERT OR REPLACE) is required // so the AFTER UPDATE trigger on `documents` fires and keeps // `documents_fts` in sync. SQLite's OR REPLACE conflict resolution does @@ -622,8 +759,8 @@ export function insertDocument(db: IndexDb, doc: IndexedDocument): void { db.prepare( `INSERT INTO documents (path, title, collection, domain, status, confidence, updated, tags, content, tokens, - ttl_days, created, superseded_by, valid_from, valid_until) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ttl_days, created, superseded_by, valid_from, valid_until, updated_by) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(path) DO UPDATE SET title = excluded.title, collection = excluded.collection, @@ -638,7 +775,8 @@ export function insertDocument(db: IndexDb, doc: IndexedDocument): void { created = excluded.created, superseded_by = excluded.superseded_by, valid_from = excluded.valid_from, - valid_until = excluded.valid_until`, + valid_until = excluded.valid_until, + updated_by = excluded.updated_by`, ).run( doc.path, doc.title, @@ -655,6 +793,7 @@ export function insertDocument(db: IndexDb, doc: IndexedDocument): void { doc.supersededBy, validFrom, validUntil, + updatedBy, ); } @@ -662,6 +801,12 @@ export interface ChunkRowInput { path: string; chunkIndex: number; text: string; + // Breadcrumb context (spec 2026-07-26 Decision 2). Optional so direct + // low-level callers (tests exercising unrelated behavior) don't all need a + // real breadcrumb; defaults to '' — the same default the column carries. + // Every real write path (reindex.ts) supplies the real chunkDocument() + // context. + context?: string; contentHash: string; } @@ -676,9 +821,9 @@ export interface ChunkRowInput { // instead, surfacing caller bugs loudly rather than drifting the FTS index. export function insertChunkRow(db: IndexDb, chunk: ChunkRowInput): void { db.prepare( - `INSERT INTO chunks (path, chunk_index, text, content_hash) - VALUES (?, ?, ?, ?)`, - ).run(chunk.path, chunk.chunkIndex, chunk.text, chunk.contentHash); + `INSERT INTO chunks (path, chunk_index, text, context, content_hash) + VALUES (?, ?, ?, ?, ?)`, + ).run(chunk.path, chunk.chunkIndex, chunk.text, chunk.context ?? "", chunk.contentHash); } // Returns the set of content_hash values that already have a row for `model` @@ -848,6 +993,7 @@ interface DocumentRow { ttl_days: number | null; created: string; superseded_by: string | null; + updated_by: string; valid_from: string | null; valid_until: string | null; } @@ -869,6 +1015,7 @@ function rowToDocument(row: DocumentRow): IndexedDocument { supersededBy: row.superseded_by, validFrom: row.valid_from, validUntil: row.valid_until, + updatedBy: row.updated_by, }; } @@ -959,6 +1106,7 @@ interface ChunkJoinRow { path: string; chunk_index: number; text: string; + context: string; content_hash: string; embedding: Buffer | null; dim: number | null; @@ -985,6 +1133,7 @@ function rowToChunk(row: ChunkJoinRow, expectedDim: number): IndexedChunk { path: row.path, chunkIndex: row.chunk_index, text: row.text, + context: row.context, contentHash: row.content_hash, embedding, }; @@ -1000,7 +1149,7 @@ function rowToChunk(row: ChunkJoinRow, expectedDim: number): IndexedChunk { export function getAllChunks(db: IndexDb, model: string, expectedDim = 0): IndexedChunk[] { const rows = db .prepare( - `SELECT c.path, c.chunk_index, c.text, c.content_hash, e.embedding, e.dim + `SELECT c.path, c.chunk_index, c.text, c.context, c.content_hash, e.embedding, e.dim FROM chunks c LEFT JOIN embeddings e ON e.content_hash = c.content_hash AND e.model = ? @@ -1018,7 +1167,7 @@ export function getChunksForPath( ): IndexedChunk[] { const rows = db .prepare( - `SELECT c.path, c.chunk_index, c.text, c.content_hash, e.embedding, e.dim + `SELECT c.path, c.chunk_index, c.text, c.context, c.content_hash, e.embedding, e.dim FROM chunks c LEFT JOIN embeddings e ON e.content_hash = c.content_hash AND e.model = ? @@ -1029,6 +1178,65 @@ export function getChunksForPath( return rows.map((row) => rowToChunk(row, expectedDim)); } +// --- Passage lookups for the reranker (Part B, §4.2) ------------------------ +// +// The rerank stage needs the exact (context, text) of the chunk that WON a +// hit's ranking — the same shape the embedding pipeline hashed +// (embeddingInput = context + "\n\n" + text). These three helpers cover the +// three passage-reference kinds a rank-time hit can carry (PassageRef in +// tools/search.ts): a lexical winner by rowid, a vector winner by +// (path, content_hash), or the terminal `first` fallback. None of them join +// against `embeddings` — the reranker never needs the vector, only the text. + +export interface ChunkPassage { + context: string; + text: string; +} + +// Batched lookup by chunks.rowid — the lexical winner's rowid, as tracked by +// chunkFtsRanking. Chunked under SQLite's bound-variable ceiling like every +// other batched IN() lookup in this file. +export function getChunkTextsByRowids(db: IndexDb, rowids: number[]): Map { + const out = new Map(); + if (rowids.length === 0) return out; + const BATCH = 500; + for (let start = 0; start < rowids.length; start += BATCH) { + const slice = rowids.slice(start, start + BATCH); + const placeholders = slice.map(() => "?").join(","); + const rows = db + .prepare(`SELECT rowid AS rowid, context, text FROM chunks WHERE rowid IN (${placeholders})`) + .all(...slice) as { rowid: number; context: string; text: string }[]; + for (const r of rows) out.set(r.rowid, { context: r.context, text: r.text }); + } + return out; +} + +// The chunk at `path` whose content_hash matches — the vector winner's best +// KNN chunk. A hash can repeat within a path (rare, but content-addressing +// allows it); the first match is as good as any since they're byte-identical. +export function getChunkByPathAndHash( + db: IndexDb, + path: string, + contentHash: string, +): ChunkPassage | null { + const row = db + .prepare("SELECT context, text FROM chunks WHERE path = ? AND content_hash = ? LIMIT 1") + .get(path, contentHash) as { context: string; text: string } | undefined; + return row ?? null; +} + +// The terminal passage fallback for a hit with no query-matched chunk +// (a title/tag-tier-only hit): the document's first chunk by chunk_index. +// Total, not partial — chunkDocument() guarantees every indexed doc has +// >=1 chunk, so this never returns null for a document that made it into +// `chunks` at all. +export function getFirstChunk(db: IndexDb, path: string): ChunkPassage | null { + const row = db + .prepare("SELECT context, text FROM chunks WHERE path = ? ORDER BY chunk_index ASC LIMIT 1") + .get(path) as { context: string; text: string } | undefined; + return row ?? null; +} + export function getMeta(db: IndexDb, key: string): string | null { const row = db.prepare("SELECT value FROM meta WHERE key = ?").get(key) as | { value: string } @@ -1086,6 +1294,16 @@ export interface StagedActionRow { // Trace/run id from the proposal record (#235). JSONL-only, like // decided_by_principal — no sqlite column. run_id?: string | null; + // 2026-07-26 risk-triaged-ratification spec, Decision 3. All four follow the + // decided_by_principal / run_id precedent exactly: JSONL + row type only, + // no DDL column, no upsert change — SQLite-backed reads always yield null. + staged_by_principal?: string | null; // proposal branch (C4) + decision_kind?: string | null; // decision branch + reason_category?: string | null; // decision branch + amended_diff?: string | null; // decision branch, JSON-encoded + // Non-authoritative risk snapshot (Mihir's 2026-07-27 decision) — decision + // branch only, never read for ordering. + risk_at_decision?: number | null; } // Inserts or replaces a staged-action row by id. Used by the jsonl→sqlite @@ -1166,6 +1384,7 @@ export interface DerivesFromEdgeRow { to_path: string; strength: number; k_survived: number; + k_eff: number; first_observed: string; last_rederived: string; last_age_decay: string; @@ -1181,13 +1400,14 @@ export interface DerivesFromEdgeRow { export function upsertDerivesFromEdge(db: IndexDb, row: DerivesFromEdgeRow): void { db.prepare( `INSERT INTO derives_from_edges - (from_path, to_path, strength, k_survived, first_observed, last_rederived, + (from_path, to_path, strength, k_survived, k_eff, first_observed, last_rederived, last_age_decay, status, direction_verdict, observations, contested_at, contest_reason) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(from_path, to_path) DO UPDATE SET strength = excluded.strength, k_survived = excluded.k_survived, + k_eff = excluded.k_eff, first_observed = excluded.first_observed, last_rederived = excluded.last_rederived, last_age_decay = excluded.last_age_decay, @@ -1201,6 +1421,7 @@ export function upsertDerivesFromEdge(db: IndexDb, row: DerivesFromEdgeRow): voi row.to_path, row.strength, row.k_survived, + row.k_eff, row.first_observed, row.last_rederived, row.last_age_decay, diff --git a/src/tools/consumes.ts b/src/tools/consumes.ts index 0038a939..45f01884 100644 --- a/src/tools/consumes.ts +++ b/src/tools/consumes.ts @@ -22,6 +22,7 @@ import { err, ok, type Result } from "../frontmatter/types.js"; import { canonicalVaultRelPath } from "../storage/local.js"; import type { ToolDefinition } from "./read.js"; import { openIndexForAccessOrNull } from "./search.js"; +import { SUMMARY_MAX_ROWS } from "./summary.js"; export interface ConsumesResult { direction: "forward" | "reverse"; @@ -118,10 +119,44 @@ const consumesEdgeSchema: Record = { required: ["artifact", "unit", "edge_type", "fields", "run_id", "compile_ts"], }; +// --------------------------------------------------------------------------- +// Compact `content` summary + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +function summarizeConsumes(value: unknown): string { + const r = value as ConsumesResult; + if (r.total === 0) return `0 ${r.direction} edges for ${r.anchor}.`; + const shown = r.edges.slice(0, SUMMARY_MAX_ROWS); + const lines = [ + `${r.total} ${r.direction} edge(s) for ${r.anchor}` + + (r.include_history ? " (history included):" : ":"), + ...shown.map((e) => ` ${e.artifact} ← ${e.unit} (${e.edge_type})`), + ]; + const rest = r.total - shown.length; + if (rest > 0) lines.push(` … ${rest} more in structuredContent`); + return lines.join("\n"); +} + +function docLinksConsumes(value: unknown): string[] { + const r = value as ConsumesResult; + const seen = new Set(); + const paths: string[] = []; + for (const e of r.edges.slice(0, SUMMARY_MAX_ROWS)) { + for (const p of [e.artifact, e.unit]) { + if (seen.has(p)) continue; + seen.add(p); + paths.push(p); + } + } + return paths; +} + export const consumesTools: ToolDefinition[] = [ { name: "vault_consumes", title: "Query the compiled dependency graph", + oneLine: "Query the compiled consumes graph: what a document read, or what reads it.", annotations: { readOnlyHint: true }, description: "Query the compiled consumes graph (#233): edges minted mechanically " + @@ -175,6 +210,8 @@ export const consumesTools: ToolDefinition[] = [ }, required: ["direction", "anchor", "edges", "total", "include_history"], }, + summarize: summarizeConsumes, + docLinks: docLinksConsumes, handler: (vaultRoot, args, access) => vaultConsumes(vaultRoot, args, access), }, ]; diff --git a/src/tools/context.ts b/src/tools/context.ts new file mode 100644 index 00000000..8afefe17 --- /dev/null +++ b/src/tools/context.ts @@ -0,0 +1,463 @@ +// vault_context — task-shaped context briefs (spec 2026-07-26-context-packs- +// progressive-disclosure-design.md, Decision 2/3, final-plan Phase 2). +// +// Mirrors vaultSearch's structure (retrieve, RBAC-filter, enrich, log) but +// assembles a single token-budgeted markdown brief instead of a ranked hit +// list. No LLM call anywhere in this file — hard line, pure selection and +// templating over the index (Decision 2's "no LLM call" section). +// +// RBAC-first, omission over redaction, no existence leak (CLAUDE.md): the +// caller's readable set is applied before any budgeting, exactly as the +// tension surfaces do; a pack never names, quotes, or counts a document in +// an unreadable collection. `hidden_remainder` is a LOWER-BOUND signal over +// OBSERVABLE withholding (C4) — see the comment on `computeHiddenRemainder`. + +import { type AccessContext, canRead, readableCollections } from "../access/rbac.js"; +import { assembleContextPack, type ContextPack, type PackEntry } from "../context/assemble.js"; +import { computeDecay } from "../curation/decay.js"; +import { structuralDecay } from "../curation/structural.js"; +import { bucketHiddenDownstream, type HiddenDownstream } from "../curation/tension-blast.js"; +import { err, ok, type Result } from "../frontmatter/types.js"; +import { CONTESTED_CAP, contestedFor } from "../search/contested.js"; +import { applyCoveragePass, DEFAULT_COVERAGE_OPTIONS } from "../search/coverage.js"; +import { resolveCurrentSource } from "../search/current-source.js"; +import { type HybridHit, hybridSearch } from "../search/hybrid.js"; +import { getDocument, type IndexDb } from "../storage/index-db.js"; +import type { ToolDefinition } from "./read.js"; +import { + annotateUpstreamHits, + ensureIndexReady, + logServedHits, + openIndexForActiveProvider, + type PendingLogEntry, +} from "./search.js"; + +// --------------------------------------------------------------------------- +// Budget parsing (C9) +// --------------------------------------------------------------------------- + +export const DEFAULT_BUDGET = 4000; +export const MIN_BUDGET = 500; +export const MAX_BUDGET = 20000; + +// Non-numeric / absent / non-finite -> default (the clampPositiveInt +// silent-fallback posture, src/tools/search.ts). Below MIN_BUDGET -> error: +// silently delivering up to ~450 tokens against a stated 300 would break the +// contract the 10% headroom exists to keep. Above MAX_BUDGET -> clamp down, +// silently — the safe direction (C9). +export function parseBudget(raw: unknown): Result { + if (typeof raw !== "number" || !Number.isFinite(raw)) return ok(DEFAULT_BUDGET); + if (raw < MIN_BUDGET) { + return err(new Error(`vault_context: budget must be >= ${MIN_BUDGET}`)); + } + if (raw > MAX_BUDGET) return ok(MAX_BUDGET); + return ok(raw); +} + +// --------------------------------------------------------------------------- +// Retrieval pool +// --------------------------------------------------------------------------- + +// Candidate pool cap before ranking/rendering — mirrors search.ts's +// RERANK_POOL sizing rationale: bounds worst-case enrichment cost regardless +// of vault size. +const POOL_MAX = 50; +// How many of the RBAC-filtered ranked hits seed the coverage pass +// (final-plan 2.2 step 4) — deliberately wider than DEFAULT_COVERAGE_OPTIONS' +// own seedK (3): a brief's candidate pool is bigger than a search page, so +// the seed window widens to match. +const COVERAGE_SEED_K = 10; + +// --------------------------------------------------------------------------- +// Pipeline +// --------------------------------------------------------------------------- + +// One candidate after supersession dedup (final-plan 2.2 step 5), before +// head-keyed enrichment. `path` is always the entry's OWN final identity — +// the chain head for a resolved supersession, the stale doc itself for +// restricted/dangling/cycle, or the hit's own path otherwise (C3). +interface ResolvedCandidate { + path: string; + score: number; + reason: string; + snippet: string; + supersedes?: number; + currentSourceRestricted?: boolean; + supersessionIssue?: "dangling" | "cycle"; +} + +// Supersession dedup + collapse (final-plan 2.2 step 5). A stale hit whose +// `currentSource` resolves collapses into (or joins) its head's entry — +// score = max over collapsed members, `supersedes` = collapsed count. A +// `restricted`/`dangling`/`cycle` outcome keeps the stale hit AS ITSELF, +// flagged, never substituted (the head is either unreadable or unknown). +// Returns the resolved candidates plus the count of restricted hops observed +// (C4's third hidden_remainder component). +function resolveSupersession( + db: IndexDb, + candidates: HybridHit[], + access: AccessContext | undefined, +): { resolved: ResolvedCandidate[]; restrictedHops: number } { + const heads = new Map(); + const standalone: ResolvedCandidate[] = []; + let restrictedHops = 0; + + for (const hit of candidates) { + const cs = resolveCurrentSource(db, hit.path, access); + if (!cs) { + standalone.push({ + path: hit.path, + score: hit.score, + reason: "matches task", + snippet: hit.snippet, + }); + continue; + } + if (cs.kind === "resolved") { + const existing = heads.get(cs.path); + if (existing) { + existing.score = Math.max(existing.score, hit.score); + existing.supersedes += 1; + } else { + // `cs.snippet` is the head's own verbatim leading content + // (previewSnippet, src/search/current-source.ts) — daftari authors + // the relation, never the value (Decision 3). + heads.set(cs.path, { score: hit.score, supersedes: 1, snippet: cs.snippet }); + } + continue; + } + if (cs.kind === "restricted") { + restrictedHops += 1; + standalone.push({ + path: hit.path, + score: hit.score, + reason: "matches task", + snippet: hit.snippet, + currentSourceRestricted: true, + }); + continue; + } + // dangling | cycle — kept as itself, flagged. + standalone.push({ + path: hit.path, + score: hit.score, + reason: "matches task", + snippet: hit.snippet, + supersessionIssue: cs.kind, + }); + } + + const headEntries: ResolvedCandidate[] = []; + for (const [path, { score, supersedes, snippet }] of heads) { + headEntries.push({ + path, + score, + reason: `supersedes ${supersedes} older document${supersedes === 1 ? "" : "s"} matching this task`, + snippet, + supersedes, + }); + } + + return { resolved: [...headEntries, ...standalone], restrictedHops }; +} + +// Head-keyed enrichment (final-plan 2.2 step 6 / C3): every flag is computed +// from the ENTRY'S OWN path's index row — for a collapsed chain that is the +// head, never the stale member. A missing index row (shouldn't happen for a +// `resolved` outcome; defensive) renders with no flag lines, never borrowed +// ones — the invariant assemble.ts's module comment documents. +function enrichCandidate( + vaultRoot: string, + db: IndexDb, + resolved: ResolvedCandidate, + access: AccessContext | undefined, +): PackEntry { + const doc = getDocument(db, resolved.path); + if (!doc) { + return { + path: resolved.path, + title: resolved.path, + score: resolved.score, + reason: resolved.reason, + snippet: resolved.snippet, + ...(resolved.supersedes !== undefined ? { supersedes: resolved.supersedes } : {}), + ...(resolved.currentSourceRestricted ? { currentSourceRestricted: true } : {}), + ...(resolved.supersessionIssue ? { supersessionIssue: resolved.supersessionIssue } : {}), + }; + } + + const decay = computeDecay({ + status: doc.status, + confidence: doc.confidence, + updated: doc.updated, + created: doc.created, + ttl_days: doc.ttlDays, + superseded_by: doc.supersededBy, + }); + const structuralRaw = structuralDecay({ db, path: resolved.path, status: doc.status, access }); + const structural = structuralRaw + ? { + orphan: structuralRaw.orphan, + deprecatedStillLinked: structuralRaw.deprecated_still_linked !== null, + } + : null; + const contested = contestedFor(vaultRoot, db, resolved.path, access); + + return { + path: resolved.path, + title: doc.title, + score: resolved.score, + reason: resolved.reason, + snippet: resolved.snippet, + ...(resolved.supersedes !== undefined ? { supersedes: resolved.supersedes } : {}), + ...(resolved.currentSourceRestricted ? { currentSourceRestricted: true } : {}), + ...(resolved.supersessionIssue ? { supersessionIssue: resolved.supersessionIssue } : {}), + decay, + structural, + ...(contested + ? { + tensions: contested.contested.slice(0, CONTESTED_CAP).map((t) => ({ + kind: t.kind, + counterpart: t.counterpart, + claimSelf: t.claimSelf, + claimOther: t.claimOther, + })), + contestedCount: contested.contestedCount, + } + : {}), + provenance: { updatedBy: doc.updatedBy, updated: doc.updated }, + }; +} + +// C4: hidden_remainder is a LOWER-BOUND signal over OBSERVABLE withholding, +// never a completeness claim. The vector half of retrieval is KNN-pushdown +// scrubbed of unreadable collections (2026-07-26 fusion spec, Decision 3), so +// it can never produce an RBAC-droppable candidate — a semantically-relevant, +// lexically-quiet hidden document is structurally invisible to this count. +// "none" means "no withholding observed", never "nothing withheld". Counted: +// (a) RBAC-dropped BM25-side pool candidates, (b) RBAC-dropped coverage-pass +// additions, (c) restricted supersession hops. +function computeHiddenRemainder(hiddenDropped: number): HiddenDownstream { + return bucketHiddenDownstream(hiddenDropped); +} + +export async function vaultContext( + vaultRoot: string, + args: Record, + access?: AccessContext, +): Promise> { + const task = args.task; + if (typeof task !== "string" || task.trim().length === 0) { + return err(new Error("vault_context requires a non-empty 'task' argument")); + } + const budgetResult = parseBudget(args.budget); + if (!budgetResult.ok) return budgetResult; + const budget = budgetResult.value; + + const ready = await ensureIndexReady(vaultRoot); + if (!ready.ok) return ready; + + const dbResult = openIndexForActiveProvider(vaultRoot); + if (!dbResult.ok) return dbResult; + const db = dbResult.value; + try { + // 3. Retrieve, over-fetch, RBAC-filter (before any budgeting — CLAUDE.md). + const searchResult = await hybridSearch(db, task, { + limit: POOL_MAX, + overFetch: true, + readableCollections: access ? readableCollections(access.role) : undefined, + }); + if (!searchResult.ok) return searchResult; + + let hiddenDropped = 0; + const permitted = access + ? searchResult.value.hits.filter((h) => { + const readable = canRead(access.role, h.collection); + if (!readable) hiddenDropped += 1; + return readable; + }) + : searchResult.value.hits; + const pool = permitted.slice(0, POOL_MAX); + + // 4. Coverage pass, seeded from the top COVERAGE_SEED_K permitted hits. + // RBAC-filter the additions the same way; a dropped addition is the B′ + // edge-attached case (2026-07-14 spec). + const seeds = pool.slice(0, COVERAGE_SEED_K); + const covered = applyCoveragePass(db, seeds, DEFAULT_COVERAGE_OPTIONS); + const rawAdds = covered.slice(seeds.length); + const poolPaths = new Set(pool.map((h) => h.path)); + const coverageAdds = rawAdds.filter((h) => !poolPaths.has(h.path)); + const permittedAdds = access + ? coverageAdds.filter((h) => { + const readable = canRead(access.role, h.collection); + if (!readable) hiddenDropped += 1; + return readable; + }) + : coverageAdds; + const candidates = [...pool, ...permittedAdds]; + + // 5. Supersession dedup — chain heads collapse, restricted/dangling/cycle + // hops stay as themselves and flag (C4's restricted-hop tally folds in). + const { resolved, restrictedHops } = resolveSupersession(db, candidates, access); + hiddenDropped += restrictedHops; + + // 6. Head-keyed enrichment (C3). No logging in this step. + const entries: PackEntry[] = resolved.map((r) => enrichCandidate(vaultRoot, db, r, access)); + + // Upstream buckets (#234), batched over one synthetic hit per final + // entry — annotateUpstreamHits mutates pendingBrokenUpstream / + // hiddenPendingUpstream on the hit it's given, keyed on `path`, which is + // already the entry's own (head, for a collapsed chain) path. + const synthHits: HybridHit[] = entries.map((e) => ({ + path: e.path, + title: e.title, + collection: "", + status: "", + score: 0, + bm25Score: 0, + vectorScore: 0, + snippet: "", + decay: null, + })); + const pendingEntries: PendingLogEntry[] = await annotateUpstreamHits( + vaultRoot, + db, + synthHits, + access, + ); + const upstreamByPath = new Map(synthHits.map((h) => [h.path, h])); + for (const entry of entries) { + const h = upstreamByPath.get(entry.path); + if (h?.pendingBrokenUpstream || h?.hiddenPendingUpstream) { + entry.upstream = { + ...(h.pendingBrokenUpstream ? { pendingBrokenUpstream: h.pendingBrokenUpstream } : {}), + ...(h.hiddenPendingUpstream ? { hiddenPendingUpstream: h.hiddenPendingUpstream } : {}), + }; + } + } + + // 7. Rank, render, cut to budget — pure, deterministic (src/context/assemble.ts). + const hiddenRemainder = computeHiddenRemainder(hiddenDropped); + const pack = assembleContextPack(task, budget, entries, hiddenRemainder); + + // 8. Log served hits — ONLY the entries that survived the budget cut + // (C1), keyed on the rendered (head) paths. + const includedPaths = new Set(pack.manifest.included.map((e) => e.path)); + const pendingByPath = new Map(pendingEntries.map((e) => [e.file, e])); + const toLog: PendingLogEntry[] = []; + for (const path of includedPaths) { + const pending = pendingByPath.get(path); + toLog.push( + pending ?? { file: path, ...(access?.user != null ? { principal: access.user } : {}) }, + ); + } + await logServedHits(vaultRoot, "vault_context", toLog); + + return ok(pack); + } finally { + db.close(); + } +} + +// --------------------------------------------------------------------------- +// MCP tool definition +// --------------------------------------------------------------------------- + +const PACK_ENTRY_SCHEMA: Record = { + type: "object", + properties: { + path: { type: "string" }, + score: { type: "number" }, + reason: { type: "string" }, + }, + required: ["path", "score", "reason"], + additionalProperties: false, +}; + +const CONTEXT_OUTPUT_SCHEMA: Record = { + type: "object", + properties: { + task: { type: "string" }, + budget: { type: "integer" }, + estimatedTokens: { + type: "integer", + description: "chars/4 estimate of the whole brief, including header and footer.", + }, + brief: { + type: "string", + description: + "The model-facing markdown brief. Selected, not synthesized — every " + + "line is an index fact (score, snippet, decay, tensions with both " + + "claims, staleness, provenance); never a sentence daftari composed " + + "about the truth of vault content.", + }, + manifest: { + type: "object", + properties: { + included: { type: "array", items: PACK_ENTRY_SCHEMA }, + omitted_over_budget: { type: "integer", minimum: 0 }, + hidden_remainder: { + type: "string", + enum: ["none", "some", "many"], + description: + "A LOWER-BOUND signal over observable withholding (RBAC-dropped " + + "BM25-side candidates, dropped coverage additions, restricted " + + "supersession hops) — never a completeness claim. The vector " + + "half of retrieval is RBAC-pushdown-scrubbed, so a semantically- " + + "relevant but lexically-quiet hidden document is structurally " + + "invisible to this count; 'none' means 'no withholding " + + "observed', not 'nothing withheld'.", + }, + }, + required: ["included", "omitted_over_budget", "hidden_remainder"], + additionalProperties: false, + }, + }, + required: ["task", "budget", "estimatedTokens", "brief", "manifest"], + additionalProperties: false, +}; + +function summarizeContext(value: unknown): string { + const r = value as ContextPack; + return r.brief; +} + +function docLinksContext(value: unknown): string[] { + const r = value as ContextPack; + return r.manifest.included.map((e) => e.path); +} + +export const contextTools: ToolDefinition[] = [ + { + name: "vault_context", + title: "Assemble a task context brief", + oneLine: "Assemble a token-budgeted brief of the most relevant documents for a task.", + annotations: { readOnlyHint: true }, + description: + "Assemble a token-budgeted brief for a task: the most relevant current " + + "documents, with open tensions, staleness, and provenance flagged " + + "inline. Selects and annotates only — never synthesizes. A tension " + + "shows both claims and status:open, never a blended verdict; " + + "supersession points at the current source, never paraphrases it. " + + "Cites paths; drill in with vault_read.", + inputSchema: { + type: "object", + properties: { + task: { type: "string", description: "Free-text description of the task at hand." }, + budget: { + type: "number", + description: + `Token budget for the brief. Default ${DEFAULT_BUDGET}. Must be >= ` + + `${MIN_BUDGET} (an error otherwise — a stated budget is a contract); ` + + `values above ${MAX_BUDGET} clamp down silently.`, + }, + }, + required: ["task"], + additionalProperties: false, + }, + outputSchema: CONTEXT_OUTPUT_SCHEMA, + summarize: summarizeContext, + docLinks: docLinksContext, + handler: (vaultRoot, args, access) => vaultContext(vaultRoot, args, access), + }, +]; diff --git a/src/tools/curation.ts b/src/tools/curation.ts index 71487045..7c9523b5 100644 --- a/src/tools/curation.ts +++ b/src/tools/curation.ts @@ -12,12 +12,13 @@ import { type AccessContext, canRatify, canRead, hasAnyRead } from "../access/rbac.js"; import { CONSOLIDATE_AGENT } from "../consolidate/constants.js"; import type { CoverageEquitySummary } from "../curation/coverage.js"; +import type { IndependenceCalibrationSummary } from "../curation/independence-calibration.js"; import { LINT_CHECKS, type LintCheckName, type LintFinding, + type RankedStagedActionItem, runLint, - type StagedActionLintItem, type TensionHealth, } from "../curation/lint.js"; import { type ProvenanceEntry, readProvenanceLog } from "../curation/provenance.js"; @@ -39,6 +40,7 @@ import { canSeeTension, sourceReadable, visibleTensions } from "../curation/tens import { bucketHiddenDownstream, computeTensionBlast, + type HiddenDownstream, type TensionBlastResult, } from "../curation/tension-blast.js"; import { loadTensionClusters, type TensionClustersResult } from "../curation/tension-clusters.js"; @@ -47,6 +49,7 @@ import { err, ok, type Result } from "../frontmatter/types.js"; import { readFile, resolveVaultPath } from "../storage/local.js"; import type { ToolDefinition } from "./read.js"; import { openIndexForAccessOrNull } from "./search.js"; +import { clip, SUMMARY_DETAIL_CHARS } from "./summary.js"; // Curation tools are open to any role with at least one read grant. A guest // (or any role with no read access) is denied. @@ -350,10 +353,15 @@ export interface VaultLintResult { checks: Partial>; totalFindings: number; tensionHealth: TensionHealth; - stagedActions: StagedActionLintItem[]; + stagedActions: RankedStagedActionItem[]; + hiddenStagedActions: HiddenDownstream; shadowActions: ShadowLintSummary; coverageEquity: CoverageEquitySummary; reviewThroughput: ReviewThroughputSummary; + independenceCalibration: IndependenceCalibrationSummary; + // 2026-07-26 citation-anchors-jit spec, Phase 8: step-3 pin classifications + // spent this run by the Decision-4 softening pass (budget-spend counter). + pinsClassified: number; } export async function vaultLint( @@ -410,9 +418,12 @@ export async function vaultLint( totalFindings: findings.length, tensionHealth: report.value.tensionHealth, stagedActions: report.value.stagedActions, + hiddenStagedActions: report.value.hiddenStagedActions, shadowActions: report.value.shadowActions, coverageEquity: report.value.coverageEquity, reviewThroughput: report.value.reviewThroughput, + independenceCalibration: report.value.independenceCalibration, + pinsClassified: report.value.pinsClassified, }); } @@ -423,9 +434,12 @@ export async function vaultLint( totalFindings: report.value.totalFindings, tensionHealth: report.value.tensionHealth, stagedActions: report.value.stagedActions, + hiddenStagedActions: report.value.hiddenStagedActions, shadowActions: report.value.shadowActions, coverageEquity: report.value.coverageEquity, reviewThroughput: report.value.reviewThroughput, + independenceCalibration: report.value.independenceCalibration, + pinsClassified: report.value.pinsClassified, }); } @@ -756,7 +770,10 @@ const lintOutputSchema: Record = { }, stagedActions: { type: "array", - description: "Pending staged actions awaiting ratification, soonest-to-expire first", + description: + "Pending staged actions awaiting ratification, risk descending, soonest-to-expire " + + "tiebreak (2026-07-26 risk-triaged-ratification spec). Filtered to the caller's " + + "vantage — see hiddenStagedActions for the coarsened remainder.", items: { type: "object", properties: { @@ -766,11 +783,49 @@ const lintOutputSchema: Record = { ageDays: { type: "integer" }, expiresInDays: { type: "integer" }, rationale: { type: "string", description: "First sentence of the staged rationale" }, + risk: { type: "number", description: "Ordinal risk score in [0,1], recomputed on read" }, + proposedBy: { type: "string" }, + proposerTrackRecord: { + type: "number", + description: "The risk score's W term: proposer's Laplace-smoothed correction rate", + }, + diffBucket: { type: "string", enum: ["small", "medium", "large"] }, + blast: { + type: "object", + properties: { + primary: { type: "integer" }, + advisory: { type: "integer" }, + hidden: { type: "string", enum: ["none", "some", "many"] }, + }, + required: ["primary", "advisory", "hidden"], + additionalProperties: false, + }, + openTension: { type: "boolean" }, + conflict: { type: "boolean" }, }, - required: ["id", "actionType", "targetPath", "ageDays", "expiresInDays", "rationale"], + required: [ + "id", + "actionType", + "targetPath", + "ageDays", + "expiresInDays", + "rationale", + "risk", + "proposedBy", + "proposerTrackRecord", + "diffBucket", + "blast", + "openTension", + "conflict", + ], additionalProperties: false, }, }, + hiddenStagedActions: { + type: "string", + enum: ["none", "some", "many"], + description: "Coarsened count of pending actions omitted from stagedActions — never exact", + }, shadowActions: { type: "object", properties: { @@ -884,6 +939,68 @@ const lintOutputSchema: Record = { required: ["lifetime", "last7d", "last30d", "timeToDecisionDays", "oldestPendingDays"], additionalProperties: false, }, + // Independence-aware promotion shadow calibration (2026-07-26 spec, + // Decision 4). Vault-global counts/aggregates only — no paths, matching + // tensionHealth's posture. + independenceCalibration: { + type: "object", + properties: { + kVsKEff: { + type: "object", + properties: { + edgesWithVotes: { type: "integer" }, + meanK: { type: "number" }, + meanKEff: { type: "number" }, + medianKEff: { type: "number" }, + kEffBelowKCount: { type: "integer" }, + }, + required: ["edgesWithVotes", "meanK", "meanKEff", "medianKEff", "kEffBelowKCount"], + additionalProperties: false, + }, + wouldDropBelowTrigger: { + type: "object", + properties: { + count: { type: "integer" }, + legacyOnlyCount: { type: "integer" }, + }, + required: ["count", "legacyOnlyCount"], + additionalProperties: false, + }, + wouldNeedsReviewRate: { + type: "object", + properties: { + rate: { type: "number" }, + needsReviewCount: { type: "integer" }, + decidedCount: { type: "integer" }, + informativePanels: { type: "integer" }, + informativeNeedsReviewCount: { type: "integer" }, + rateInformative: { type: "number" }, + }, + required: [ + "rate", + "needsReviewCount", + "decidedCount", + "informativePanels", + "informativeNeedsReviewCount", + "rateInformative", + ], + additionalProperties: false, + }, + legacyUnfingerprintedFraction: { type: "number" }, + nonLoopFingerprintedCountedVotes: { type: "integer" }, + }, + required: [ + "kVsKEff", + "wouldDropBelowTrigger", + "wouldNeedsReviewRate", + "legacyUnfingerprintedFraction", + "nonLoopFingerprintedCountedVotes", + ], + additionalProperties: false, + }, + // 2026-07-26 citation-anchors-jit spec, Phase 8: step-3 pin + // classifications spent this run by the Decision-4 softening pass. + pinsClassified: { type: "integer", minimum: 0 }, }, required: [ "generatedAt", @@ -892,9 +1009,12 @@ const lintOutputSchema: Record = { "totalFindings", "tensionHealth", "stagedActions", + "hiddenStagedActions", "shadowActions", "coverageEquity", "reviewThroughput", + "independenceCalibration", + "pinsClassified", ], additionalProperties: false, }; @@ -950,12 +1070,6 @@ const TIER0_LINT_CHECKS: readonly LintCheckName[] = [ ]; const LINT_SUMMARY_TOP_FINDINGS = 6; -const LINT_SUMMARY_DETAIL_CHARS = 110; - -function clip(text: string, max: number): string { - const flat = text.replace(/\s+/g, " ").trim(); - return flat.length > max ? `${flat.slice(0, max - 1)}…` : flat; -} function summarizeLint(value: unknown): string { const report = value as VaultLintResult; @@ -989,8 +1103,10 @@ function summarizeLint(value: unknown): string { `${health.aging.stale} fresh/aging/stale; ${health.clusters.count} cluster(s) ` + `(${health.clusters.large} large, ${health.clusters.aged} aged); ` + `stale blast ${health.blastRadiusOfStaleTensions}`, - `staged: ${report.stagedActions.length} pending, ` + - `${report.reviewThroughput.lifetime.expired} expired lifetime; ` + + `staged: ${report.stagedActions.length} pending` + + (report.stagedActions[0] ? ` (top risk ${report.stagedActions[0].risk.toFixed(2)})` : "") + + (report.hiddenStagedActions !== "none" ? `, ${report.hiddenStagedActions} hidden` : "") + + `, ${report.reviewThroughput.lifetime.expired} expired lifetime; ` + `shadow: ${report.shadowActions.total} logged, ${report.shadowActions.gated} would-gate; ` + `coverage: ${report.coverageEquity.backstopOverdue.count} backstop-overdue edge(s)`, ]; @@ -999,14 +1115,72 @@ function summarizeLint(value: unknown): string { if (top.length > 0) { lines.push(`top ${top.length} of ${flat.length} finding(s):`); for (const { check, finding } of top) { - lines.push( - ` [${check}] ${finding.path} — ${clip(finding.detail, LINT_SUMMARY_DETAIL_CHARS)}`, - ); + lines.push(` [${check}] ${finding.path} — ${clip(finding.detail, SUMMARY_DETAIL_CHARS)}`); } } return lines.join("\n"); } +// --------------------------------------------------------------------------- +// Compact `content` summaries + resource links for the remaining curation +// tools (spec 2026-07-26, Decision 3, PR 1 gap closure) +// --------------------------------------------------------------------------- + +function summarizeTensionEntry(value: unknown): string { + const t = value as TensionEntry; + return `${t.id ?? "(legacy, no id)"} [${t.kind}] ${t.status}`; +} + +// sourceA/sourceB are already present in the (already visible) result value +// — the hard rule that docLinks never re-derives or re-queries anything. +function docLinksTensionEntry(value: unknown): string[] { + const t = value as TensionEntry; + return [t.sourceA, t.sourceB]; +} + +function summarizeTensionClusters(value: unknown): string { + const r = value as TensionClustersResult; + if (r.cluster_count === 0) return "0 tension clusters."; + const lines = [`${r.cluster_count} tension cluster(s):`]; + for (const c of r.clusters) { + lines.push( + ` ${c.id}: ${c.size} doc(s), ${c.tension_count} tension(s), ` + + `${c.oldest_tension_age_days}-${c.newest_tension_age_days}d old`, + ); + } + return lines.join("\n"); +} + +function summarizeTensionBlast(value: unknown): string { + const r = value as TensionBlastResult; + const anchor = r.contested_document ?? r.cluster_id ?? "(unknown anchor)"; + return ( + `blast from ${anchor}: ${r.downstream.length} downstream ` + + `(${r.primary_blast} primary / ${r.advisory_blast} advisory), ` + + `depth ${r.max_depth}, hidden: ${r.hidden_downstream}` + ); +} + +// Every path the (already RBAC-filtered) value names: the contested anchor +// (when a single document was queried), the cluster's members, and the +// visible downstream set. +function docLinksTensionBlast(value: unknown): string[] { + const r = value as TensionBlastResult; + const paths: string[] = []; + if (r.contested_document) paths.push(r.contested_document); + paths.push(...r.cluster_documents); + paths.push(...r.downstream.map((d) => d.path)); + return paths; +} + +function summarizeProvenance(value: unknown): string { + const r = value as VaultProvenanceResult; + if (r.count === 0) return `${r.path}: no write history.`; + const lines = [`${r.path}: ${r.count} entry(ies)`]; + for (const e of r.history.slice(-5)) lines.push(` ${e.timestamp} ${e.agent} ${e.action}`); + return lines.join("\n"); +} + // --------------------------------------------------------------------------- // MCP tool definitions // --------------------------------------------------------------------------- @@ -1015,6 +1189,7 @@ export const curationTools: ToolDefinition[] = [ { name: "vault_tension_log", title: "Log a contradiction", + oneLine: "Log a contradiction between two documents' claims.", annotations: { destructiveHint: true }, description: "Record a tension — a contradiction or unresolved pull between two " + @@ -1064,11 +1239,14 @@ export const curationTools: ToolDefinition[] = [ }, // The entry as logged: id assigned, status 'unresolved', no resolution. outputSchema: tensionEntrySchema(LOGGABLE_TENSION_KINDS), + summarize: summarizeTensionEntry, + docLinks: docLinksTensionEntry, handler: (vaultRoot, args, access) => vaultTensionLog(vaultRoot, args, access), }, { name: "vault_tension_resolve", title: "Resolve a logged tension", + oneLine: "Resolve a logged tension with an outcome and rationale.", annotations: { destructiveHint: true }, description: "Record the closure of a previously logged tension. The 'kind' parameter " + @@ -1109,11 +1287,14 @@ export const curationTools: ToolDefinition[] = [ }, // The updated entry: status 'resolved' with the resolution block attached. outputSchema: tensionEntrySchema(TENSION_KINDS), + summarize: summarizeTensionEntry, + docLinks: docLinksTensionEntry, handler: (vaultRoot, args, access) => vaultTensionResolve(vaultRoot, args, access), }, { name: "vault_tension_clusters", title: "Compute tension clusters", + oneLine: "Compute connected components of the tension graph.", annotations: { readOnlyHint: true }, description: "Compute connected components of the tension graph: groups of vault " + @@ -1130,11 +1311,13 @@ export const curationTools: ToolDefinition[] = [ additionalProperties: false, }, outputSchema: tensionClustersOutputSchema, + summarize: summarizeTensionClusters, handler: (vaultRoot, args, access) => vaultTensionClusters(vaultRoot, args, access), }, { name: "vault_tension_blast", title: "Compute tension blast radius", + oneLine: "Compute the downstream blast radius of a contested document or cluster.", annotations: { readOnlyHint: true }, description: "Compute the transitive closure of downstream documents that cite or " + @@ -1170,10 +1353,13 @@ export const curationTools: ToolDefinition[] = [ additionalProperties: false, }, outputSchema: tensionBlastOutputSchema, + summarize: summarizeTensionBlast, + docLinks: docLinksTensionBlast, handler: (vaultRoot, args, access) => vaultTensionBlast(vaultRoot, args, access), }, { name: "vault_lint", + oneLine: "Run advisory curation checks: staleness, orphans, drafts, tensions, and more.", // Not read-only: the staged-action sweep (§11.2) expires actions past // their TTL, appending expiry records to .daftari/staged-actions.jsonl. // It never edits vault content — only the staging queue's own lifecycle. @@ -1189,7 +1375,15 @@ export const curationTools: ToolDefinition[] = [ "stable acknowledged persistent disagreements, and legacy unspecified " + "entries) — tension-health counts are deliberately VAULT-GLOBAL, not " + "RBAC-filtered: counts only, no paths, so vault health reads the same " + - "for every role. Lists pending staged actions awaiting ratification, and — " + + "for every role. Lists pending staged actions awaiting ratification, ordered " + + "risk descending with soonest-to-expire as the tiebreak (2026-07-26 " + + "risk-triaged-ratification spec) — each item carries an ordinal risk " + + "score in [0,1] (recomputed on read, never stored), the diff-size " + + "bucket, the proposer's smoothed track record, visible blast counts, " + + "and open-tension / conflict flags. Filtered to the caller's vantage: " + + "an item whose target is unreadable is omitted, and the hidden " + + "remainder is reported coarsened (hiddenStagedActions: none/some/many), " + + "never as an exact count. And — " + "when the vault has run shadow_mode — summarizes shadow-logged writes " + "with the ones the trust budget would have gated. " + "Never auto-fixes vault content; it does, as housekeeping, expire " + @@ -1212,6 +1406,7 @@ export const curationTools: ToolDefinition[] = [ { name: "vault_provenance", title: "View document write history", + oneLine: "View a document's write history.", annotations: { readOnlyHint: true }, description: "Return the write history of a single document from the provenance " + @@ -1229,6 +1424,8 @@ export const curationTools: ToolDefinition[] = [ additionalProperties: false, }, outputSchema: provenanceOutputSchema, + summarize: summarizeProvenance, + docLinks: (value) => [(value as VaultProvenanceResult).path], handler: (vaultRoot, args, access) => vaultProvenance(vaultRoot, args, access), }, ]; diff --git a/src/tools/edge-staleness.ts b/src/tools/edge-staleness.ts index e05db0b3..1d213d9e 100644 --- a/src/tools/edge-staleness.ts +++ b/src/tools/edge-staleness.ts @@ -341,10 +341,45 @@ const brokenReadReportSchema: Record = { ], }; +// --------------------------------------------------------------------------- +// Compact `content` summary + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure). Two modes, discriminated by `mode` — see +// ArtifactStalenessResult / BrokenReadReport above. No new prose is written +// over what summarizeUpstream already computed for the artifact mode; this +// just renders its counts plus the coarsened hidden-pending bucket verbatim +// (#217 — never sharpened into a number). +// --------------------------------------------------------------------------- + +function summarizeStaleness(value: unknown): string { + const r = value as ArtifactStalenessResult | BrokenReadReport; + if (r.mode === "artifact") { + const s = r.summary; + return ( + `${r.artifact}: ${s.current} current, ${s.pending_unchecked} pending-unchecked, ` + + `${s.pending_compatible} pending-compatible, ${s.pending_broken} pending-broken ` + + `(hidden_pending: ${r.hidden_pending})` + ); + } + const rate = r.broken_read_rate === null ? "n/a" : `${(r.broken_read_rate * 100).toFixed(1)}%`; + return ( + `broken-read rate over ${r.window_days}d: ${rate} (${r.broken_serves}/${r.serves} serves) ` + + `— ${r.uninstrumented} uninstrumented` + ); +} + +// Artifact mode: the anchor plus every visible upstream unit. Report mode +// names no document — by_tool is keyed by tool name, not a path. +function docLinksStaleness(value: unknown): string[] { + const r = value as ArtifactStalenessResult | BrokenReadReport; + if (r.mode !== "artifact") return []; + return [r.artifact, ...r.edges.map((e) => e.unit)]; +} + export const edgeStalenessTools: ToolDefinition[] = [ { name: "vault_staleness", title: "Edge staleness — pending upstream changes and the broken-read rate", + oneLine: "Report pending upstream changes and the vault's broken-read rate.", annotations: { readOnlyHint: true }, description: "Edge staleness (#234): is a document stale WITH RESPECT TO its " + @@ -381,6 +416,8 @@ export const edgeStalenessTools: ToolDefinition[] = [ type: "object", oneOf: [artifactStalenessSchema, brokenReadReportSchema], }, + summarize: summarizeStaleness, + docLinks: docLinksStaleness, handler: (vaultRoot, args, access) => vaultStaleness(vaultRoot, args, access), }, ]; diff --git a/src/tools/edges.ts b/src/tools/edges.ts index d02f6147..93d0dbcf 100644 --- a/src/tools/edges.ts +++ b/src/tools/edges.ts @@ -20,11 +20,13 @@ import { type AccessContext, canRatify, hasAnyRead } from "../access/rbac.js"; import { + computeInputsFingerprint, contestEdge, type DerivesFromEdge, EDGE_AXES, EDGE_STATUSES, type EdgeAxis, + type EdgeFingerprint, type EdgeStatus, getEdge, listEdges, @@ -36,6 +38,7 @@ import { err, ok, type Result } from "../frontmatter/types.js"; import { canonicalVaultRelPath, readFile, resolveVaultPath } from "../storage/local.js"; import type { ToolDefinition } from "./read.js"; import { openIndexForAccessOrNull } from "./search.js"; +import { SUMMARY_MAX_ROWS } from "./summary.js"; function requireReadAccess(tool: string, access?: AccessContext): Result { if (access && !hasAnyRead(access.role)) { @@ -73,6 +76,30 @@ async function requireDocument( return ok(undefined); } +// Reads the current full bytes of each canonicalized evidence path — the +// server-computed half of the fp trust split (C3): `fp.inputs` is a hash +// over what the server itself read, never a caller-supplied hash. Each path +// must name a real document (mirrors requireDocument's stage-time check). +async function loadEvidenceBytes( + vaultRoot: string, + relPaths: string[], + tool: string, +): Promise, Error>> { + const out: Array<{ path: string; text: string }> = []; + for (const raw of relPaths) { + const canonPath = canonicalVaultRelPath(vaultRoot, raw); + if (!canonPath.ok) return canonPath; + const resolved = resolveVaultPath(vaultRoot, canonPath.value); + if (!resolved.ok) return resolved; + const content = await readFile(resolved.value.absPath); + if (!content.ok) { + return err(new Error(`${tool}: evidence path not found: ${canonPath.value}`)); + } + out.push({ path: canonPath.value, text: content.value }); + } + return ok(out); +} + // --------------------------------------------------------------------------- // vault_edge_observe // --------------------------------------------------------------------------- @@ -116,6 +143,40 @@ export async function vaultEdgeObserve( if (trimmed.length > 0) note = trimmed; } + // Evidence fingerprint (spec Decision 1, PR-3): `evidence_paths` names the + // vault-relative paths the caller's derivation actually read; the server + // hashes their CURRENT full bytes — the caller cannot mint fp.inputs + // without naming real, presently-readable documents. `model` / `prompt_id` + // are recorded verbatim (caller-attested, same trust class as + // blind/varied_axis today). `fp.principal` comes ONLY from the access + // context — never from args, never from the free-text `observed_by`. + let evidencePaths: string[] | undefined; + if (args.evidence_paths !== undefined && args.evidence_paths !== null) { + if ( + !Array.isArray(args.evidence_paths) || + !args.evidence_paths.every((p) => typeof p === "string" && p.trim().length > 0) + ) { + return err( + new Error("vault_edge_observe 'evidence_paths' must be a list of non-empty strings"), + ); + } + evidencePaths = args.evidence_paths as string[]; + } + let model: string | undefined; + if (args.model !== undefined && args.model !== null) { + if (typeof args.model !== "string" || args.model.includes("\n")) { + return err(new Error("vault_edge_observe 'model' must be a string with no newline")); + } + if (args.model.trim().length > 0) model = args.model.trim(); + } + let promptId: string | undefined; + if (args.prompt_id !== undefined && args.prompt_id !== null) { + if (typeof args.prompt_id !== "string" || args.prompt_id.includes("\n")) { + return err(new Error("vault_edge_observe 'prompt_id' must be a string with no newline")); + } + if (args.prompt_id.trim().length > 0) promptId = args.prompt_id.trim(); + } + const canonFrom = canonicalVaultRelPath(vaultRoot, fromPath.value); if (!canonFrom.ok) return canonFrom; const canonTo = canonicalVaultRelPath(vaultRoot, toPath.value); @@ -132,6 +193,23 @@ export async function vaultEdgeObserve( const toExists = await requireDocument(vaultRoot, canonTo.value, "vault_edge_observe"); if (!toExists.ok) return toExists; + let inputsFingerprint: string | undefined; + if (evidencePaths !== undefined) { + const bytes = await loadEvidenceBytes(vaultRoot, evidencePaths, "vault_edge_observe"); + if (!bytes.ok) return bytes; + inputsFingerprint = computeInputsFingerprint(bytes.value); + } + const principal = access?.user; + const fp: EdgeFingerprint | undefined = + inputsFingerprint !== undefined || model !== undefined || promptId !== undefined || principal + ? { + ...(inputsFingerprint !== undefined ? { inputs: inputsFingerprint } : {}), + ...(principal ? { principal } : {}), + ...(model !== undefined ? { model } : {}), + ...(promptId !== undefined ? { prompt: promptId } : {}), + } + : undefined; + return observeEdge(vaultRoot, { fromPath: canonFrom.value, toPath: canonTo.value, @@ -139,6 +217,7 @@ export async function vaultEdgeObserve( blind: args.blind, ...(axis !== undefined ? { axis } : {}), ...(note !== undefined ? { note } : {}), + ...(fp !== undefined ? { fp } : {}), }); } @@ -326,6 +405,19 @@ const derivesFromEdgeSchema: Record = { type: "integer", description: "Raw independent-vote count the aging applies to (capped at EDGE_K_CAP)", }, + kEff: { + type: "number", + description: + "Independence-aware promotion shadow calibration value (2026-07-26 spec): kSurvived " + + "discounted for votes sharing an evidence-fingerprint class. Not yet the live status " + + "input — status still derives from strength/kSurvived.", + }, + strengthIndependent: { + type: "number", + description: + "Shadow calibration value: the aged strength agedStrength would compute from kEff " + + "instead of kSurvived. Not yet the live status input.", + }, firstObserved: { type: "string", description: "ISO 8601 timestamp of the observation that seeded the current cycle", @@ -364,6 +456,8 @@ const derivesFromEdgeSchema: Record = { "toPath", "strength", "kSurvived", + "kEff", + "strengthIndependent", "firstObserved", "lastRederived", "status", @@ -374,17 +468,80 @@ const derivesFromEdgeSchema: Record = { ], }; +// --------------------------------------------------------------------------- +// Compact `content` summaries + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +function edgeLine(e: DerivesFromEdge): string { + return `${e.fromPath} → ${e.toPath} (k=${e.kSurvived}, strength=${e.strength.toFixed(2)}, ${e.status})`; +} + +function summarizeEdgeObserve(value: unknown): string { + return `observed: ${edgeLine(value as DerivesFromEdge)}`; +} + +function docLinksEdge(value: unknown): string[] { + const e = value as DerivesFromEdge; + return [e.fromPath, e.toPath]; +} + +function summarizeEdgeContest(value: unknown): string { + const r = value as ContestResult; + const tension = r.tension_id ? ` — tension ${r.tension_id}` : ""; + return `contested (revoked): ${edgeLine(r.edge)}${tension}`; +} + +function docLinksEdgeContest(value: unknown): string[] { + return docLinksEdge((value as ContestResult).edge); +} + +function summarizeEdges(value: unknown): string { + const r = value as EdgesResult; + if (r.total === 0) return "0 edges match."; + const shown = r.edges.slice(0, SUMMARY_MAX_ROWS); + const lines = [`${r.total} edge(s):`, ...shown.map((e) => ` ${edgeLine(e)}`)]; + const rest = r.total - shown.length; + if (rest > 0) lines.push(` … ${rest} more in structuredContent`); + return lines.join("\n"); +} + +function docLinksEdges(value: unknown): string[] { + const r = value as EdgesResult; + const seen = new Set(); + const paths: string[] = []; + for (const e of r.edges.slice(0, SUMMARY_MAX_ROWS)) { + for (const p of [e.fromPath, e.toPath]) { + if (seen.has(p)) continue; + seen.add(p); + paths.push(p); + } + } + return paths; +} + export const edgeTools: ToolDefinition[] = [ { name: "vault_edge_observe", title: "Record a derives_from observation", + oneLine: "Record a derives_from observation between two documents.", annotations: { destructiveHint: false }, description: "Record that a (re-)derivation observed a derives_from edge between two " + "documents. The first observation seeds the edge as a zero-strength " + "candidate; an edge earns strength only through later blind observations " + "that vary at least one axis (prompt | input-neighborhood | model). " + - "Normally called by the consolidation loop, not by a human directly.", + "Optionally carries an evidence fingerprint (2026-07-26 independence-aware-" + + "promotion spec, Decision 1) that feeds the SHADOW k_eff calibration — " + + "'evidence_paths' is server-computed (hashed from the named paths' current " + + "bytes), 'principal' is server-derived from the access context (never from " + + "args), and 'model'/'prompt_id' are caller-attested, the same trust class " + + "as 'blind'/'varied_axis' today ('attestation, not verification' — RBAC is " + + "the control on who may write, not fingerprint validation). This is also " + + "the human resolution path for a needs-review tension (Decision 3): supply " + + "a genuinely independent re-derivation — different model, principal, or " + + "inputs — with a fresh fingerprint. Normally called by the consolidation " + + "loop, not by a human directly.", inputSchema: { type: "object", properties: { @@ -417,17 +574,38 @@ export const edgeTools: ToolDefinition[] = [ type: "string", description: "Optional free-text context recorded with the observation", }, + evidence_paths: { + type: "array", + items: { type: "string" }, + description: + "Vault-relative paths this re-derivation actually read. The server " + + "computes fp.inputs as a hash over their CURRENT full bytes — each " + + "path must exist. Omit to leave fp.inputs unset (∅, the sentinel class).", + }, + model: { + type: "string", + description: + "The model id this re-derivation ran on (caller-attested). Recorded as fp.model.", + }, + prompt_id: { + type: "string", + description: + "The prompt-template id this re-derivation used (caller-attested). Recorded as fp.prompt.", + }, }, required: ["from_path", "to_path", "observed_by", "blind"], additionalProperties: false, }, // The edge's state AFTER this observation collapsed into it. outputSchema: derivesFromEdgeSchema, + summarize: summarizeEdgeObserve, + docLinks: docLinksEdge, handler: (vaultRoot, args, access) => vaultEdgeObserve(vaultRoot, args, access), }, { name: "vault_edge_contest", title: "Contest and revoke a derives_from edge", + oneLine: "Contest and revoke a derives_from edge.", annotations: { destructiveHint: true }, description: "Record a case-2 contradiction: a re-derivation failed with no upstream " + @@ -471,11 +649,14 @@ export const edgeTools: ToolDefinition[] = [ }, required: ["edge"], }, + summarize: summarizeEdgeContest, + docLinks: docLinksEdgeContest, handler: (vaultRoot, args, access) => vaultEdgeContest(vaultRoot, args, access), }, { name: "vault_edges", title: "List derives_from edges", + oneLine: "List derives_from edges, optionally filtered by document or status.", annotations: { readOnlyHint: true }, description: "List derives_from edges with their live aged strength, strongest " + @@ -521,6 +702,8 @@ export const edgeTools: ToolDefinition[] = [ }, required: ["edges", "total"], }, + summarize: summarizeEdges, + docLinks: docLinksEdges, handler: (vaultRoot, args, access) => vaultEdges(vaultRoot, args, access), }, ]; diff --git a/src/tools/read.ts b/src/tools/read.ts index 71d35fb7..2ebc45d4 100644 --- a/src/tools/read.ts +++ b/src/tools/read.ts @@ -5,6 +5,7 @@ // definitions; tests call the logic functions directly. import { type AccessContext, canRead, filterByReadPermission } from "../access/rbac.js"; +import { type AnchorsAnnotation, computeAnchors } from "../anchors/read.js"; import { computeDecay, type DecayState } from "../curation/decay.js"; import { compiledUpstreamStaleness, @@ -34,12 +35,14 @@ import { type ValidationReport, } from "../frontmatter/types.js"; import { type ContestedTension, contestedFor } from "../search/contested.js"; -import { getProvider } from "../search/vector.js"; +import { getProvider, getQuantize } from "../search/vector.js"; import { countDimMismatches, openIndexDb } from "../storage/index-db.js"; import { listFiles, readFile, resolveVaultPath } from "../storage/local.js"; +import { loadConfig } from "../utils/config.js"; import { sha256Hex } from "../utils/hash.js"; import { readRunId } from "../utils/run-id.js"; import { openIndexForAccessOrNull } from "./search.js"; +import { SUMMARY_MAX_ROWS } from "./summary.js"; // Tool-annotation hints surfaced to MCP clients. The MCP spec treats these as // *hints* — clients must not gate behavior on them — but directory reviewers @@ -57,6 +60,14 @@ export interface ToolDefinition { // Human-readable title surfaced in UIs (Claude Desktop, the connectors // directory). `name` stays machine-style; `title` is for humans. title?: string; + // One-line index entry for vault_tools' index mode (spec 2026-07-26 + // context-packs-progressive-disclosure, Decision 1 / Phase 1.2): imperative, + // no schema talk, capped at 120 chars (enforced by a test, not a runtime + // truncation — a tool whose one-liner needs more than 120 chars needs a + // shorter one-liner, not a truncated one). Required so every tool the + // registry ever gains is forced to declare one; there is no back-compat + // fallback because vault_tools ships alongside this field, not before it. + oneLine: string; description: string; inputSchema: Record; // JSON Schema (2020-12) for the handler's ok-value. Required: handlers @@ -70,6 +81,14 @@ export interface ToolDefinition { // daftari://doc/{path} resource_link per entry. Paths must already be // read-gated by the handler (links inherit read-gating by construction). docLinks?: (value: unknown) => string[]; + // Projects the full ok-value down to what rides `structuredContent` + // (spec 2026-07-26, Decision 3 / C11). Absent, the bridge ships the value + // verbatim. Exists so a tool whose body-shaped payload already rides + // `content` in full (vault_read) does not ship it a second time on + // structuredContent — the wire's worst token offender, doubled, in the PR + // whose purpose is cutting waste. `summarize`/`docLinks` still see the + // FULL value; only the wire projection is narrowed. + wireValue?: (value: unknown) => Record; annotations?: ToolAnnotations; // `access` is supplied by the server transport on every call. When omitted // (a direct in-process call, e.g. from a test) RBAC is not enforced. @@ -125,6 +144,15 @@ export interface VaultReadResult { // none are visible. contested?: ContestedTension[]; contestedCount?: number; + // Citation-anchors JIT verification (2026-07-26 spec, Decisions 1-2): per + // pinned `describes` binding, whether the code the doc cites still matches + // what the pin recorded. Null when there is nothing to say — no pinned + // bindings, no resolvable code_repos, jit_anchors: false, OR (2026-07-27 + // resolution) the caller's role lacks the code_repo_visibility grant even + // though the underlying check ran (see the recordRead call below — the + // read-log counts are unfiltered local telemetry; this field is the + // caller-facing, role-gated surface). + anchors: AnchorsAnnotation | null; // SHA-256 (hex) of the raw file bytes, frontmatter included. A caller passes // this back as a write tool's `base_version` to detect a stale write. version: string; @@ -159,6 +187,30 @@ export async function vaultRead( } } + // Citation anchors (2026-07-26 spec, Decisions 1-2): JIT-verify this doc's + // pinned `describes` bindings against the locally checked-out code_repos. + // Placed AFTER the RBAC gate; independent of the index-db handle below. + // Computed regardless of the caller's role — recordRead's anchors_* counts + // are local operator telemetry (kill-condition (b) instrumentation), + // unfiltered by design, the same posture broken_upstream already takes. + // The RETURNED `anchors` field is separately role-gated just before the + // final return (2026-07-27 resolution): the check runs either way, but a + // role without code_repo_visibility never sees its result. Config-load + // failure, jit_anchors: false, an empty code_repos map, or a doc with no + // describes entries all short-circuit before any git work. + const config = loadConfig(vaultRoot); + let rawAnchors: AnchorsAnnotation | null = null; + if ( + config.ok && + config.value.jitAnchors && + Object.keys(config.value.codeRepos).length > 0 && + parsed.value.frontmatter.describes.length > 0 + ) { + rawAnchors = await computeAnchors(parsed.value.frontmatter.describes, config.value.codeRepos); + } + const anchorsVisible = !access || access.role?.codeRepoVisibility === true; + const anchors = anchorsVisible ? rawAnchors : null; + // #234: classify this document's compiled upstream edges as of the serve. // Best-effort — the read never fails on telemetry; on a log-read error the // serve is still recorded, just uninstrumented (broken_upstream absent). @@ -193,6 +245,13 @@ export async function vaultRead( ...(rows ? { broken_upstream: rows.filter((r) => r.staleness === "pending-broken").length } : {}), + ...(rawAnchors + ? { + anchors_moved: rawAnchors.entries.filter((e) => e.state === "moved").length, + anchors_missing: rawAnchors.entries.filter((e) => e.state === "missing").length, + anchors_errored: rawAnchors.errored, + } + : {}), }); // One index handle serves every graph-backed enrichment below: the #234 @@ -251,6 +310,35 @@ export async function vaultRead( db?.close(); } + // Decision 4: an intact pin is evidence of freshness — annotate, never + // extend. computeDecay stays pure and byte-identical (vault_status's + // distribution is unaffected); this only appends to the ALREADY-non-null + // banner a past-TTL doc gets, and only when every classified pin is + // intact with nothing dropped (skipped/errored) — a censored or partial + // sample must never license an "unchanged" claim (C8). + let decay = computeDecay(parsed.value.frontmatter); + if (decay?.banner) { + const staleness = computeStaleness( + { updated: parsed.value.frontmatter.updated, ttl_days: parsed.value.frontmatter.ttl_days }, + new Date(), + ); + const allIntact = + anchors !== null && + anchors.entries.length >= 1 && + anchors.skipped === 0 && + anchors.errored === 0 && + anchors.entries.every((e) => e.state === "intact"); + if (staleness.expired && allIntact) { + const n = (anchors as AnchorsAnnotation).entries.length; + decay = { + ...decay, + banner: + `${decay.banner}\n— past TTL, but its ${n} code pin${n === 1 ? "" : "s"} are intact: ` + + "the code it describes has not changed since the pins were written.", + }; + } + } + return ok({ path, content: parsed.value.content, @@ -258,7 +346,7 @@ export async function vaultRead( raw: parsed.value.raw, validation: parsed.value.validation, hasFrontmatter: parsed.value.hasFrontmatter, - decay: computeDecay(parsed.value.frontmatter), + decay, // Evaluated against today. No index access and no RBAC branch — these // fields belong to a document the caller has already been permitted to // read. @@ -268,6 +356,7 @@ export async function vaultRead( ...(contestedResult ? { contested: contestedResult.contested, contestedCount: contestedResult.contestedCount } : {}), + anchors, version: sha256Hex(file.value), }); } @@ -540,13 +629,20 @@ export async function vaultStatus( // Dim-mismatch counter. A non-zero value means some embedding cache rows // for the active model have the wrong dim and are being silently skipped // by vector ranking. We open the DB defensively — if sqlite-vec isn't - // installed or the index hasn't been built yet, the field is 0. + // installed or the index hasn't been built yet, the field is 0. The + // durable cache stores NATIVE-dim vectors (C9), so the expected dim here + // is nativeDim (falling back to dim for providers with no Matryoshka gap), + // not the configured index dim. const provider = getProvider(); let embeddingDimMismatches = 0; - const dbResult = openIndexDb(vaultRoot, provider.dim); + const dbResult = openIndexDb(vaultRoot, provider.dim, getQuantize()); if (dbResult.ok) { try { - embeddingDimMismatches = countDimMismatches(dbResult.value, provider.id, provider.dim); + embeddingDimMismatches = countDimMismatches( + dbResult.value, + provider.id, + provider.nativeDim ?? provider.dim, + ); } finally { dbResult.value.close(); } @@ -664,6 +760,55 @@ export const DECAY_SCHEMA: Record = { required: ["level", "reasons", "banner"], }; +// One classified citation-anchor pin (2026-07-26 spec, Decisions 1-2). +// `relocated` is present only for a range pin that classified `intact` via +// step 3's substring search. +const ANCHOR_ENTRY_SCHEMA: Record = { + type: "object", + properties: { + raw: { type: "string", description: "The describes entry exactly as written" }, + repo: { type: "string" }, + path: { type: "string" }, + symbol: { type: ["string", "null"] }, + pin: { + type: "object", + properties: { + start: { type: ["integer", "null"] }, + end: { type: ["integer", "null"] }, + sha: { type: "string" }, + }, + required: ["start", "end", "sha"], + }, + state: { type: "string", enum: ["intact", "moved", "missing"] }, + relocated: { + type: "object", + properties: { + start: { type: "integer" }, + end: { type: "integer" }, + }, + required: ["start", "end"], + }, + }, + required: ["raw", "repo", "path", "symbol", "pin", "state"], +}; + +// Null when there is nothing to say — no pinned bindings, no resolvable +// code_repos, jit_anchors: false, or (2026-07-27 resolution) the caller's +// role lacks code_repo_visibility. `errored` (C8) counts classifier +// failures dropped from `entries`, so the Decision-4 "all intact" softening +// never quantifies over a silently-censored sample. +export const ANCHORS_SCHEMA: Record = { + type: ["object", "null"], + properties: { + entries: { type: "array", items: ANCHOR_ENTRY_SCHEMA }, + checked: { type: "integer", minimum: 0 }, + skipped: { type: "integer", minimum: 0, description: "Over-cap remainder (MAX_PINS_PER_READ)" }, + errored: { type: "integer", minimum: 0 }, + banner: { type: ["string", "null"] }, + }, + required: ["entries", "checked", "skipped", "errored", "banner"], +}; + // One classified upstream edge (#234). Only compiled edges can reach // pending-broken; the other classes park in pending-unchecked. const UPSTREAM_EDGE_SCHEMA: Record = { @@ -758,10 +903,78 @@ function asStringArray(v: unknown): string[] | undefined { return out.length > 0 ? out : undefined; } +// --------------------------------------------------------------------------- +// Compact `content` summaries + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +// vault_read: header line, then every advisory banner that is non-null, then +// the contested count, then the body VERBATIM — this is the one and only +// channel the body crosses the wire on (see `wireValue` below / C11). +function summarizeRead(value: unknown): string { + const r = value as VaultReadResult; + const fm = r.frontmatter; + const lines = [`${r.path} — ${fm.status} / ${fm.confidence} confidence / ${fm.collection}`]; + if (r.decay?.banner) lines.push(`decay: ${r.decay.banner}`); + if (r.validity?.banner) lines.push(`validity: ${r.validity.banner}`); + if (r.upstream_staleness?.banner) lines.push(`upstream: ${r.upstream_staleness.banner}`); + if (r.structural?.banner) lines.push(`structural: ${r.structural.banner}`); + if (r.anchors?.banner) lines.push(`anchors: ${r.anchors.banner}`); + if (r.contestedCount !== undefined && r.contestedCount > 0) { + lines.push(`contested: ${r.contestedCount} unresolved tension(s)`); + } + lines.push("", r.content); + return lines.join("\n"); +} + +// Every upstream unit the caller can see, plus the document itself — all +// already RBAC-filtered by vaultRead (omission, #217), so every path here is +// readable by construction. +function docLinksRead(value: unknown): string[] { + const r = value as VaultReadResult; + const paths = [r.path]; + for (const edge of r.upstream_staleness?.edges ?? []) paths.push(edge.unit); + return paths; +} + +// C11: the body ships once, in `content` (summarizeRead, above) — never +// doubled onto `structuredContent`. Delivered via the doc resource too +// (Decision 2), for a programmatic consumer that wants it without the +// summary text around it. +function wireValueRead(value: unknown): Record { + const { content: _content, ...rest } = value as VaultReadResult; + return rest; +} + +function summarizeIndex(value: unknown): string { + const r = value as VaultIndexResult; + if (r.count === 0) return "0 documents match."; + const shown = r.entries.slice(0, SUMMARY_MAX_ROWS); + const lines = [`${r.count} document(s):`, ...shown.map((e) => ` ${e.path} (${e.status})`)]; + const rest = r.count - shown.length; + if (rest > 0) lines.push(` … ${rest} more in structuredContent`); + return lines.join("\n"); +} + +function summarizeStatus(value: unknown): string { + const r = value as VaultStatusResult; + const sd = r.stalenessDistribution; + const vc = r.validityCoverage; + return [ + `${r.vault}: ${r.fileCount} doc(s), ${r.invalidCount} invalid — ${r.generatedAt}`, + `index health: ${r.embeddingDimMismatches} embedding dim mismatch(es)`, + `staleness: ${sd.fresh} fresh / ${sd.aging} aging / ${sd.stale} stale (of ${sd.total})`, + `validity: ${vc.authored} authored / ${vc.unknown} unknown (of ${vc.total})`, + `tensions: ${r.unresolvedTensions.count} unresolved`, + `recent writes: ${r.recentWrites.count}`, + ].join("\n"); +} + export const readTools: ToolDefinition[] = [ { name: "vault_read", title: "Read a vault document", + oneLine: "Read a single vault document, with decay, validity, and staleness annotations.", annotations: { readOnlyHint: true }, description: "Read a single vault document. Returns its markdown body, parsed " + @@ -779,9 +992,14 @@ export const readTools: ToolDefinition[] = [ "orphan: nothing you can read links here; deprecated_still_linked: " + "canonical docs still lean on this deprecated one; null when healthy), " + "any unresolved tensions involving the document (contested, same " + - "shape as search hits), and a 'version' token (SHA-256 of the file) " + - "that can be passed back to a write tool as 'base_version' for " + - "optimistic-concurrency checking. Path is relative to the vault root.", + "shape as search hits), an anchors report (citation-anchors JIT " + + "verification — per pinned `describes` binding, whether the code it " + + "cites still matches what was pinned: intact/moved/missing; null when " + + "there are no pins, no configured code_repos, jit_anchors is off, or " + + "the caller's role lacks code-repo visibility), and a 'version' token " + + "(SHA-256 of the file) that can be passed back to a write tool as " + + "'base_version' for optimistic-concurrency checking. Path is relative " + + "to the vault root.", inputSchema: { type: "object", properties: { @@ -804,7 +1022,14 @@ export const readTools: ToolDefinition[] = [ type: "object", properties: { path: { type: "string", description: "The path as requested by the caller" }, - content: { type: "string", description: "Markdown body, frontmatter block stripped" }, + content: { + type: "string", + description: + "Markdown body, frontmatter block stripped. Delivered in the `content` " + + "channel (verbatim, alongside the header/banners) and via the doc " + + "resource (daftari://doc/{path}) — never duplicated onto " + + "structuredContent, so this field is absent there (C11).", + }, frontmatter: FRONTMATTER_SCHEMA, raw: { type: "object", @@ -879,11 +1104,11 @@ export const readTools: ToolDefinition[] = [ }, }, contestedCount: { type: "integer", minimum: 0 }, + anchors: ANCHORS_SCHEMA, version: { type: "string", description: "SHA-256 (hex) of the raw file bytes" }, }, required: [ "path", - "content", "frontmatter", "raw", "validation", @@ -891,9 +1116,13 @@ export const readTools: ToolDefinition[] = [ "decay", "upstream_staleness", "structural", + "anchors", "version", ], }, + summarize: summarizeRead, + docLinks: docLinksRead, + wireValue: wireValueRead, handler: (vaultRoot, args, access) => { const runId = readRunId(args, "vault_read"); if (!runId.ok) return Promise.resolve(runId); @@ -903,6 +1132,7 @@ export const readTools: ToolDefinition[] = [ { name: "vault_index", title: "List vault documents", + oneLine: "List vault documents with metadata, optionally filtered.", annotations: { readOnlyHint: true }, description: "List vault documents with their metadata, including each document's " + @@ -948,6 +1178,7 @@ export const readTools: ToolDefinition[] = [ }, required: ["count", "entries"], }, + summarize: summarizeIndex, handler: (vaultRoot, args, access) => vaultIndex( vaultRoot, @@ -964,6 +1195,7 @@ export const readTools: ToolDefinition[] = [ { name: "vault_status", title: "Vault health dashboard", + oneLine: "Vault health dashboard: staleness, tensions, and recent writes.", annotations: { readOnlyHint: true }, description: "Vault health dashboard: total file count, per-collection counts, " + @@ -1059,6 +1291,7 @@ export const readTools: ToolDefinition[] = [ "embeddingDimMismatches", ], }, + summarize: summarizeStatus, handler: (vaultRoot, _args, access) => vaultStatus(vaultRoot, access), }, ]; diff --git a/src/tools/receipt.ts b/src/tools/receipt.ts index f16103e9..52110d63 100644 --- a/src/tools/receipt.ts +++ b/src/tools/receipt.ts @@ -309,10 +309,32 @@ export async function vaultReceipt( // MCP tool definition // --------------------------------------------------------------------------- +// --------------------------------------------------------------------------- +// Compact `content` summary + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +function summarizeReceipt(value: unknown): string { + const r = value as VaultReceiptResult; + const s = r.summary; + const verdict = s.flags.length === 0 ? "clean" : s.flags.join(", "); + const lines = [ + `receipt: ${s.sourceCount} source(s) — ${verdict}` + + (s.openTensions > 0 ? ` (${s.openTensions} open tension(s))` : ""), + ]; + for (const src of r.sources) lines.push(` ${src.path} (${src.status}/${src.confidence})`); + return lines.join("\n"); +} + +function docLinksReceipt(value: unknown): string[] { + return (value as VaultReceiptResult).sources.map((s) => s.path); +} + export const receiptTools: ToolDefinition[] = [ { name: "vault_receipt", title: "Compile an epistemic receipt", + oneLine: "Compile a signed receipt over cited sources: status, decay, and tensions.", annotations: { readOnlyHint: true }, description: "Compile an epistemic receipt for the vault documents an answer relies " + @@ -494,6 +516,8 @@ export const receiptTools: ToolDefinition[] = [ }, required: ["claim", "sources", "summary", "vaultHead", "generatedAt", "receiptHash"], }, + summarize: summarizeReceipt, + docLinks: docLinksReceipt, handler: (vaultRoot, args, access) => vaultReceipt( vaultRoot, diff --git a/src/tools/registry.ts b/src/tools/registry.ts new file mode 100644 index 00000000..5b0f4d6d --- /dev/null +++ b/src/tools/registry.ts @@ -0,0 +1,258 @@ +// The full tool registry, assembled once at module load — moved out of +// server.ts (spec 2026-07-26-context-packs-progressive-disclosure-design.md, +// Phase 1.1) to break an import cycle: `vault_tools` needs to close over the +// full `allTools` array, and it needs to live somewhere every tools/*.ts file +// can be assembled without server.ts importing back into a tools file (or a +// tools file importing server.ts). +// +// `serializeToolDefinition` is the SINGLE wire-shape serializer, used by both +// server.ts's ListTools handler and vault_tools' expand mode, so the two can +// never drift (spec Phase 1.1's stated purpose for extracting it). + +import type { AccessContext } from "../access/rbac.js"; +import { err, ok, type Result } from "../frontmatter/types.js"; +import { loadConfig } from "../utils/config.js"; +import { consumesTools } from "./consumes.js"; +import { contextTools } from "./context.js"; +import { curationTools } from "./curation.js"; +import { edgeStalenessTools } from "./edge-staleness.js"; +import { edgeTools } from "./edges.js"; +import { readTools, type ToolAnnotations, type ToolDefinition } from "./read.js"; +import { receiptTools } from "./receipt.js"; +import { searchTools } from "./search.js"; +import { stagedActionTools } from "./staged-actions.js"; +import { themesTools } from "./themes.js"; +import { tier1Tools } from "./tier1.js"; +import { tier2Tools } from "./tier2.js"; +import { witnessTools } from "./witness.js"; +import { writeTools } from "./write.js"; + +// Every tool EXCEPT vault_tools itself — vault_tools is appended below, once +// defined, so its own entry appears in its own index (a caller asking "what +// tools exist" should see vault_tools listed, not have to already know it +// exists to call it). +const registeredTools: ToolDefinition[] = [ + ...readTools, + ...receiptTools, + ...witnessTools, + ...searchTools, + ...themesTools, + ...writeTools, + ...curationTools, + ...stagedActionTools, + ...edgeTools, + ...consumesTools, + ...tier1Tools, + ...tier2Tools, + ...edgeStalenessTools, + ...contextTools, +]; + +// The wire shape ListTools ships (name, title?, description, inputSchema, +// outputSchema, annotations?) — the SAME projection vault_tools' expand mode +// returns, so the two surfaces can never describe a tool differently. +export interface SerializedToolDefinition { + name: string; + title?: string; + description: string; + inputSchema: Record; + outputSchema: Record; + annotations?: ToolAnnotations; +} + +export function serializeToolDefinition(t: ToolDefinition): SerializedToolDefinition { + return { + name: t.name, + ...(t.title ? { title: t.title } : {}), + description: t.description, + inputSchema: t.inputSchema, + outputSchema: t.outputSchema, + ...(t.annotations ? { annotations: t.annotations } : {}), + }; +} + +// --------------------------------------------------------------------------- +// vault_tools (spec Decision 1 / Phase 1.3) +// --------------------------------------------------------------------------- + +export interface VaultToolsIndexEntry { + name: string; + oneLine: string; +} + +export type VaultToolsResult = + | { mode: "index"; count: number; tools: VaultToolsIndexEntry[] } + | { mode: "expand"; tools: SerializedToolDefinition[]; unknown: string[] }; + +// The vault's `exclude` list applies to vault_tools too — "exclude always +// wins" (#104) extends to the in-band catalog (C2). Tier and `include` do +// NOT affect vault_tools: making tiered-out tools discoverable is the whole +// point of this tool. A config-load failure degrades to "nothing excluded" +// (the same posture #104's own exclude filtering takes when it cannot read +// config — a missing/malformed config yields the empty ToolsConfig default +// upstream, never a hard failure here). +function loadExcludeSet(vaultRoot: string): Set { + const cfg = loadConfig(vaultRoot); + return new Set(cfg.ok ? cfg.value.tools.exclude : []); +} + +export async function vaultTools( + vaultRoot: string, + args: Record, + _access?: AccessContext, +): Promise> { + const excluded = loadExcludeSet(vaultRoot); + + // Index mode: `expand` omitted entirely. An explicit empty array is still + // an expand request (of zero names) — distinct from "browse everything" — + // so the branch keys on `undefined`, not falsiness or emptiness. + if (args.expand === undefined) { + const tools = allTools + .filter((t) => !excluded.has(t.name)) + .map((t): VaultToolsIndexEntry => ({ name: t.name, oneLine: t.oneLine })) + .sort((a, b) => a.name.localeCompare(b.name)); + return ok({ mode: "index", count: tools.length, tools }); + } + + if (!Array.isArray(args.expand) || !args.expand.every((n) => typeof n === "string")) { + return err(new Error("vault_tools: 'expand' must be an array of tool name strings")); + } + + const byName = new Map(allTools.map((t) => [t.name, t])); + const unknown: string[] = []; + const tools: SerializedToolDefinition[] = []; + // Excluded names in an expand request land in `unknown`, identical to + // unregistered names — omission, shaped as if the tool does not exist + // (spec C2: definitions are not documents, so this is a consistency + // choice, not an existence-leak requirement). + for (const name of args.expand as string[]) { + const tool = byName.get(name); + if (!tool || excluded.has(name)) { + unknown.push(name); + continue; + } + tools.push(serializeToolDefinition(tool)); + } + return ok({ mode: "expand", tools, unknown }); +} + +const VAULT_TOOLS_INDEX_ENTRY_SCHEMA: Record = { + type: "object", + properties: { + name: { type: "string" }, + oneLine: { type: "string" }, + }, + required: ["name", "oneLine"], + additionalProperties: false, +}; + +// Expanded entries carry each tool's own inputSchema/outputSchema — nested, +// arbitrary-shaped JSON Schema fragments. A permissive object schema here is +// deliberate (spec Phase 1.3): the only alternative is describing "any JSON +// Schema" recursively, which strict ajv compilation does not make cheap and +// this contract does not need — the wire shape is already pinned by +// SerializedToolDefinition and the ListTools-drift test. +const VAULT_TOOLS_EXPANDED_ENTRY_SCHEMA: Record = { + type: "object", + properties: { + name: { type: "string" }, + title: { type: "string" }, + description: { type: "string" }, + inputSchema: { type: "object" }, + outputSchema: { type: "object" }, + annotations: { type: "object" }, + }, + required: ["name", "description", "inputSchema", "outputSchema"], +}; + +function summarizeVaultTools(value: unknown): string { + const r = value as VaultToolsResult; + if (r.mode === "index") { + const lines = [`${r.count} tool(s):`, ...r.tools.map((t) => `${t.name} — ${t.oneLine}`)]; + return lines.join("\n"); + } + const lines = [`expanded ${r.tools.length} tool(s)`]; + for (const name of r.unknown) lines.push(`unknown tool: ${name}`); + return lines.join("\n"); +} + +const vaultToolsDefinition: ToolDefinition = { + name: "vault_tools", + title: "Browse or expand the tool catalog", + oneLine: "List every vault tool (one line each), or expand named tools to full schemas.", + annotations: { readOnlyHint: true }, + description: + "List every tool this vault offers, one line each, or expand named tools " + + "to their full schemas (description, inputSchema, outputSchema). Call " + + "with no arguments to browse; call with 'expand' before first use of a " + + "non-core tool. Every registered tool remains callable regardless of " + + "what this index shows — this is advertisement only, never a gate. " + + "Names excluded by the vault's config are omitted from the index and " + + "reported in 'unknown' on expand, the same as an unregistered name.", + inputSchema: { + type: "object", + properties: { + expand: { + type: "array", + items: { type: "string" }, + description: + "Tool names to expand to full schemas. Omit entirely to get the " + + "one-line index instead.", + }, + }, + additionalProperties: false, + }, + outputSchema: { + // The MCP Tool schema requires outputSchema.type === "object" at the + // root (the SDK's own client-side Zod validator rejects a bare `oneOf` + // with no sibling `type` — caught by test/e2e/server.e2e.test.ts against + // the built artifact). `type: "object"` here is the root-level contract; + // `oneOf` still discriminates on `mode` underneath it. + type: "object", + oneOf: [ + { + type: "object", + properties: { + mode: { const: "index" }, + count: { type: "integer", minimum: 0 }, + tools: { type: "array", items: VAULT_TOOLS_INDEX_ENTRY_SCHEMA }, + }, + required: ["mode", "count", "tools"], + additionalProperties: false, + }, + { + type: "object", + properties: { + mode: { const: "expand" }, + tools: { type: "array", items: VAULT_TOOLS_EXPANDED_ENTRY_SCHEMA }, + unknown: { + type: "array", + items: { type: "string" }, + description: "Requested names that are unregistered or excluded by config.", + }, + }, + required: ["mode", "tools", "unknown"], + additionalProperties: false, + }, + ], + }, + summarize: summarizeVaultTools, + handler: (vaultRoot, args, access) => vaultTools(vaultRoot, args, access), +}; + +// The full registry. Static — assembled once at module load, shared by every +// server instance and by the tier-exposure helpers in server.ts. +// `vaultToolsDefinition`'s handler closes over this binding by name (see +// `vaultTools` above); that is a plain forward reference, not a cycle — the +// handler only runs after this module has finished evaluating. +export const allTools: ToolDefinition[] = [...registeredTools, vaultToolsDefinition]; + +export function registeredToolNames(): string[] { + return allTools.map((t) => t.name); +} + +// The full ToolDefinition registry, for tests that need more than the name +// (output-schema compilation, summarize/docLinks presence checks). +export function allRegisteredTools(): ToolDefinition[] { + return allTools; +} diff --git a/src/tools/search.ts b/src/tools/search.ts index 7e9c0d2e..d90d74ca 100644 --- a/src/tools/search.ts +++ b/src/tools/search.ts @@ -33,6 +33,7 @@ import { type HybridSearchResult, type HybridWeights, hybridSearch, + type PassageRef, type RelatedSearchResult, relatedSearch, } from "../search/hybrid.js"; @@ -45,11 +46,24 @@ import { onceIndexReady, } from "../search/index-state.js"; import { type ReindexResult, reindexVault } from "../search/reindex.js"; +import { getRerankProvider, warmRerankModel } from "../search/rerank-provider.js"; +import { classifyQuery, makeDfLookup, type RouteClass, routeWeights } from "../search/router.js"; import { resolveValidAtSource } from "../search/valid-at-source.js"; -import { getProvider } from "../search/vector.js"; -import { documentCount, getDocument, type IndexDb, openIndexDb } from "../storage/index-db.js"; +import { embeddingInput, getProvider, getQuantize } from "../search/vector.js"; +import { + type ChunkPassage, + documentCount, + getChunkByPathAndHash, + getChunkTextsByRowids, + getDocument, + getFirstChunk, + type IndexDb, + openIndexDb, +} from "../storage/index-db.js"; +import { loadConfig } from "../utils/config.js"; import { normalizeIsoDate } from "../utils/dates.js"; import type { ToolDefinition } from "./read.js"; +import { clip } from "./summary.js"; // All tool-side opens pass the active provider's dim so the sqlite-vec // table matches the embeddings the search will query. A read-only tool @@ -59,7 +73,7 @@ import type { ToolDefinition } from "./read.js"; // Exported so other index-backed tools (vault_themes) reuse the same // dim-aware open path. export function openIndexForActiveProvider(vaultRoot: string): Result { - return openIndexDb(vaultRoot, getProvider().dim); + return openIndexDb(vaultRoot, getProvider().dim, getQuantize()); } // Read-only index handle for RBAC collection lookups. openIndexForActiveProvider @@ -110,8 +124,14 @@ export async function ensureIndexReady(vaultRoot: string): Promise; const bm25 = obj.bm25; const vector = obj.vector; @@ -125,7 +145,14 @@ function parseWeights(raw: unknown): HybridWeights { return { bm25, vector }; } } - return DEFAULT_WEIGHTS; + return "invalid"; +} + +// vault_search_related has no user query to classify — it is never routed, +// so an absent or invalid `weights` arg both fall back to the static +// default (today's exact parseWeights behaviour, unchanged). +function staticWeightsFallback(explicit: HybridWeights | "invalid" | null): HybridWeights { + return explicit !== null && explicit !== "invalid" ? explicit : DEFAULT_WEIGHTS; } // Shared numeric-arg posture: a positive finite number floors and clamps to @@ -165,12 +192,112 @@ const RERANK_INSTRUCTIONS = "candidates carry no enrichment (tensions, staleness, structural flags); " + "the served hits and vault_read do."; +// --------------------------------------------------------------------------- +// Part B: local cross-encoder reranker (spec 2026-07-26-contextual-chunking- +// reranker-design.md Decisions 5-8). Unrelated to the #3 agent-as-judge pool +// above (RERANK_CANDIDATES_MAX / RERANK_INSTRUCTIONS) despite the shared +// vocabulary — that pool hands compact judging records to the CALLING agent; +// this stage runs a local ONNX model, INSIDE the server, and reorders the +// hits themselves before they are ever returned. +// --------------------------------------------------------------------------- + +// How many of the fused, RBAC-and-validity-filtered hits get scored by the +// cross-encoder. A fixed pool bounds worst-case latency regardless of vault +// size — the reranker cannot move recall by construction (it only reorders +// within the pool), so widening it trades latency for no recall gain past +// what the fused order already surfaced. +const RERANK_POOL = 50; + +// Wall-clock budget for one rerank call (spec C5). The in-flight ONNX +// inference is not cancellable, but the search must never hang on it: on +// timeout the fused order stands and `rerankUsed` stays false. One stderr +// warning per PROCESS (not per call) — a slow model is an operational fact +// worth one log line, not a warning storm on every subsequent search. +const RERANK_TIMEOUT_MS = 1500; + +let rerankDegradeWarned = false; + +// Races `promise` against a timeout that resolves to Result.err — never +// rejects, so the caller's `.ok` branch handles both a real provider error +// and a timeout identically (fused order stands either way). +async function withTimeout( + promise: Promise>, + ms: number, +): Promise> { + let timer: ReturnType | undefined; + const timeout = new Promise>((resolve) => { + timer = setTimeout(() => resolve(err(new Error(`rerank timed out after ${ms}ms`))), ms); + }); + try { + return await Promise.race([promise, timeout]); + } finally { + clearTimeout(timer); + } +} + +// Resolves passage TEXT for exactly the top RERANK_POOL permitted hits (C2) — +// never for the whole over-fetched candidate set. Lexical refs are batch- +// resolved in one call via getChunkTextsByRowids; vector and `first` refs are +// resolved per-hit (bounded at RERANK_POOL, so this is cheap). The passage +// string is embeddingInput(chunk) — the same context+text concatenation the +// embedding pipeline hashed, so the cross-encoder sees the exact retrieval +// unit that earned the hit its rank (spec §4.2). Missing refs, or a resolved +// ref whose chunk row is somehow gone, fall back to getFirstChunk and then, +// only as a last-resort defensive guard (an index inconsistency, not a +// passage strategy — chunkDocument guarantees >=1 chunk per indexed doc), to +// the hit's own served snippet, logging once so the inconsistency is visible. +function resolvePassages( + db: IndexDb, + pool: HybridHit[], + passageRefs: Record | undefined, +): string[] { + const refs = passageRefs ?? {}; + const lexicalRowids = pool + .map((h) => refs[h.path]) + .filter((r): r is Extract => r?.kind === "lexical") + .map((r) => r.rowid); + const lexicalTexts = getChunkTextsByRowids(db, lexicalRowids); + + return pool.map((h) => { + const ref = refs[h.path]; + let passage: ChunkPassage | null = null; + if (ref?.kind === "lexical") passage = lexicalTexts.get(ref.rowid) ?? null; + else if (ref?.kind === "vector") passage = getChunkByPathAndHash(db, h.path, ref.contentHash); + if (!passage) passage = getFirstChunk(db, h.path); + if (!passage) { + process.stderr.write( + `daftari: warning: no chunk row found for reranked hit ${h.path} — index inconsistency, ` + + "falling back to the served snippet\n", + ); + return h.snippet; + } + return embeddingInput(passage); + }); +} + // #234 serve instrumentation, shared by every snippet-serving tool -// (vault_search AND vault_search_related — the broken-read rate's -// denominator counts serves, whichever tool served them). Each SERVED hit -// becomes one read-log entry carrying its pending-broken upstream count — -// the TRUE count, unfiltered, because the log is local operator telemetry — -// batched into a single append so N hits do not pay N fs writes. +// (vault_search, vault_search_related, and vault_context — the broken-read +// rate's denominator counts serves, whichever tool served them). +// +// Split in two (spec 2026-07-26-context-packs-progressive-disclosure-design.md +// final plan, C1): the original `annotateAndLogServedHits` both computed the +// upstream buckets AND appended the read-log entries in one pass, but +// vault_context needs the buckets during enrichment (before the budget cut) +// and the log write only for entries that SURVIVE the cut — logging a serve +// that was never actually served would corrupt the read log's own broken-read +// denominator. `annotateUpstreamHits` writes nothing; `logServedHits` is the +// batch append. vault_search / vault_search_related call both back-to-back, +// unchanged behavior. +export interface PendingLogEntry { + file: string; + principal?: string; + broken_upstream?: number; +} + +// Computes and attaches `pendingBrokenUpstream`/`hiddenPendingUpstream` to +// each hit (mutated in place, same as before the split) and returns the +// per-hit pending log entry — the TRUE broken count, unfiltered, because the +// log is local operator telemetry. Never writes. // // The caller-facing hit uses the shared #217 split (splitUpstreamVisibility): // the "broken" (incident) classification is disclosed only for upstream @@ -180,15 +307,14 @@ const RERANK_INSTRUCTIONS = // verdict derived from a hidden unit would leak that unit's change activity // across the ACL boundary. The visible count is bucketed for hit-payload // compactness, not disclosure — vault_read's exact pending_broken is the -// drill-down. Best-effort: a telemetry failure never fails the search. -async function annotateAndLogServedHits( +// drill-down. +export async function annotateUpstreamHits( vaultRoot: string, db: IndexDb, - tool: string, hits: HybridHit[], access?: AccessContext, -): Promise { - if (hits.length === 0) return; +): Promise { + if (hits.length === 0) return []; // The newest-compile-group collapse is O(total edges); do it ONCE per // call, not per hit. Passing the pre-collapsed set through is sound // because currentConsumesEdges is idempotent. An empty consumes log @@ -198,7 +324,7 @@ async function annotateAndLogServedHits( const staleCtx = loaded ? { consumes: currentConsumesEdges(loaded.consumes), provenance: loaded.provenance } : null; - const entries: Parameters[1] = []; + const entries: PendingLogEntry[] = []; for (const hit of hits) { let broken: number | undefined; if (staleCtx) { @@ -215,13 +341,40 @@ async function annotateAndLogServedHits( if (hiddenPending !== "none") hit.hiddenPendingUpstream = hiddenPending; } entries.push({ - tool, file: hit.path, ...(access?.user != null ? { principal: access.user } : {}), ...(broken !== undefined ? { broken_upstream: broken } : {}), }); } - await recordReads(vaultRoot, entries); + return entries; +} + +// The batch append `annotateUpstreamHits` no longer performs. Best-effort: +// a telemetry failure never fails the calling tool. +export async function logServedHits( + vaultRoot: string, + tool: string, + entries: PendingLogEntry[], +): Promise { + if (entries.length === 0) return; + await recordReads( + vaultRoot, + entries.map((e) => ({ tool, ...e })), + ); +} + +// vault_search / vault_search_related's shared call shape: annotate then log +// every served hit, unconditionally — behavior byte-identical to the +// pre-split `annotateAndLogServedHits`. +async function annotateAndLogServedHits( + vaultRoot: string, + db: IndexDb, + tool: string, + hits: HybridHit[], + access?: AccessContext, +): Promise { + const entries = await annotateUpstreamHits(vaultRoot, db, hits, access); + await logServedHits(vaultRoot, tool, entries); } // --------------------------------------------------------------------------- @@ -237,11 +390,18 @@ function validityForPath(db: IndexDb, path: string, at: string): ValidityReport return computeValidity({ valid_from: doc.validFrom, valid_until: doc.validUntil }, at); } +// vault_search's result shape: HybridSearchResult plus the optional `routed` +// diagnostic the tool handler attaches when (and only when) the query router +// chose the weights (spec 2026-07-26 fusion overhaul, Decision 2). +export interface VaultSearchResult extends HybridSearchResult { + routed?: { class: RouteClass; signals: string[] }; +} + export async function vaultSearch( vaultRoot: string, args: Record, access?: AccessContext, -): Promise> { +): Promise> { const query = args.query; if (typeof query !== "string" || query.trim().length === 0) { return { @@ -287,13 +447,52 @@ export async function vaultSearch( const db = dbResult.value; try { const limit = parseLimit(args.limit); + + // Weight resolution precedence (spec 2026-07-26 fusion overhaul, + // Decision 2): an explicit VALID `weights` arg always wins. An explicit + // INVALID `weights` arg gets the static default — a caller who expressed + // intent to control weights and got the shape wrong must never silently + // fall through to router-driven ranking. Only a genuinely ABSENT + // `weights` arg considers the router, and only when `search.routing` is + // on; a config LOAD failure degrades to the static default rather than + // failing the search. `routed` stays undefined unless the router + // actually chose the weights — it is absent for explicit weights, + // routing-off, and config-load degrade alike, so its presence + // distinguishes "the router picked lexical-only" from "embeddings + // degraded" even though both can report vectorUsed: false. + const explicitWeights = parseExplicitWeights(args.weights); + const cfg = loadConfig(vaultRoot); + const routingOn = cfg.ok && cfg.value.search.routing; + let routed: { class: RouteClass; signals: string[] } | undefined; + let weights: HybridWeights; + if (explicitWeights !== null && explicitWeights !== "invalid") { + weights = explicitWeights; + } else if (explicitWeights === "invalid") { + weights = DEFAULT_WEIGHTS; + } else if (routingOn) { + const classified = classifyQuery(query, { + df: makeDfLookup(db), + docCount: documentCount(db), + }); + routed = classified; + weights = routeWeights(classified.class); + } else { + weights = DEFAULT_WEIGHTS; + } + + // Part B: resolve the reranker ONCE, before hybridSearch, so ref capture + // (cheap) can be requested only when a reranker is actually configured — + // capturing refs for a "none" search would be wasted work (C2's "skip ref + // capture" revision). + const reranker = getRerankProvider(); + // Over-fetch every ranked candidate so RBAC filtering happens BEFORE the // user-facing slice. If we sliced to `limit` first (the old behaviour), // restricted docs occupying the top-`limit` slots would be dropped by // canRead below and shrink the permitted page below `limit`, even though // more readable docs ranked just past the cut. const result = await hybridSearch(db, query, { - weights: parseWeights(args.weights), + weights, limit, overFetch: true, // Push the readable-collection allow-list into the vector KNN so a @@ -301,6 +500,7 @@ export async function vaultSearch( // (2026-07-26 fusion spec, Decision 3). The canRead filter below stays: // pushdown is a recall fix, not the authorization boundary. readableCollections: access ? readableCollections(access.role) : undefined, + capturePassageRefs: reranker !== null, }); if (!result.ok) return result; @@ -338,7 +538,47 @@ export async function vaultSearch( } } - const ranked = permittedRanked.slice(0, limit); + // Part B rerank stage (spec Decision 7): between the RBAC/validity filter + // and the slice. `permittedRanked` is the RBAC-and-validity-filtered fused + // order; reranking it before the slice lets a fused-#12 hit with the top + // rerank score land #1 in a limit-10 page. Coverage/current-source/ + // contested/structural and the token cap all run AFTER, over the + // reranked page, unchanged — they are additive recall levers, not ranking + // levers, and reranking after them would let a relevance model evict + // recall insurance. + let rerankUsed = false; + let finalRanked = permittedRanked; + if (reranker && !reranker.isReady()) { + // Never block a tool call on a cold model load (C5): fire the warm in + // the background and serve the fused order for THIS search. + void warmRerankModel(); + } + if (reranker?.isReady()) { + const pool = permittedRanked.slice(0, RERANK_POOL); + if (pool.length > 0) { + const passages = resolvePassages(db, pool, result.value.passageRefs); + const scored = await withTimeout(reranker.rerank(query, passages), RERANK_TIMEOUT_MS); + if (scored.ok) { + const order = scored.value + .map((s, i) => ({ s, i })) + .sort((a, b) => b.s - a.s) + .map(({ i }) => pool[i]) + .filter((h): h is HybridHit => h !== undefined); + finalRanked = [...order, ...permittedRanked.slice(RERANK_POOL)]; + rerankUsed = true; + } else if (!rerankDegradeWarned) { + // Fires for BOTH a provider Result.err and a timeout — either way + // the fused order stands and rerankUsed stays false (Decision 8 + + // C5). One line per process, not per call. + rerankDegradeWarned = true; + process.stderr.write( + `daftari: warning: rerank degraded to fused order: ${scored.error.message}\n`, + ); + } + } + } + + const ranked = finalRanked.slice(0, limit); // Coverage pass: conditionally widen the ranked set with same-entity docs in // the seeds' date window. Quiet (returns `ranked` unchanged) when no signal @@ -397,16 +637,18 @@ export async function vaultSearch( await annotateAndLogServedHits(vaultRoot, db, "vault_search", capped, access); - // #3: opt-in agent-as-judge rerank pool — the top-K of the SAME - // RBAC-filtered fused ranking the hits were sliced from (never coverage - // additions; those are recall, not ranking). Compact judging records - // only: no enrichment joins, per the protocol text. + // #3: opt-in agent-as-judge rerank pool — the top-K of the SAME fused + // ranking the hits were sliced from (never coverage additions; those are + // recall, not ranking). Drawn from `finalRanked` (spec Decision 7): when + // Part B's cross-encoder is on, the agent judges the ALREADY-reranked + // pool, not the pre-rerank fused order. Compact judging records only: no + // enrichment joins, per the protocol text. const rerankK = parseRerankCandidates(args.rerank_candidates); const rerank = rerankK > 0 ? { instructions: RERANK_INSTRUCTIONS, - candidates: permittedRanked.slice(0, rerankK).map((h, i) => ({ + candidates: finalRanked.slice(0, rerankK).map((h, i) => ({ rank: i + 1, path: h.path, title: h.title, @@ -420,10 +662,17 @@ export async function vaultSearch( } : undefined; + // passageRefs is internal transport (Part B) — never serialized. The + // outputSchema declares additionalProperties: false, and it would fail + // client-side validation anyway; strip it explicitly rather than rely on + // that alone. + const { passageRefs: _passageRefs, ...resultRest } = result.value; return ok({ - ...result.value, + ...resultRest, count: capped.length, hits: capped, + rerankUsed, + ...(routed ? { routed } : {}), ...(rerank ? { rerank } : {}), }); } finally { @@ -459,7 +708,7 @@ export async function vaultSearchRelated( // Over-fetch, then RBAC-filter, then slice — same ordering as vaultSearch so // restricted docs in the top-`limit` slots can't shrink the permitted page. const result = relatedSearch(db, path, { - weights: parseWeights(args.weights), + weights: staticWeightsFallback(parseExplicitWeights(args.weights)), limit, overFetch: true, readableCollections: access ? readableCollections(access.role) : undefined, @@ -632,8 +881,8 @@ const hybridHitSchema = { collection: { type: "string" }, status: { type: "string" }, score: { type: "number", description: "Fused bm25/vector score; larger is better." }, - bm25Score: { type: "number", description: "Normalised lexical component." }, - vectorScore: { type: "number", description: "Normalised semantic component." }, + bm25Score: { type: "number", description: "Lexical component of the fused score." }, + vectorScore: { type: "number", description: "Semantic component of the fused score." }, snippet: { type: "string" }, decay: decaySchema, currentSource: currentSourceSchema, @@ -740,8 +989,7 @@ function summaryLine(rank: number, hit: HybridHit): string { // Snippets arrive whitespace-collapsed, so this is normally the whole // snippet; the split keeps the line single-line regardless. const head = (hit.snippet.split("\n", 1)[0] ?? "").trim(); - const snippet = - head.length > SUMMARY_SNIPPET_MAX ? `${head.slice(0, SUMMARY_SNIPPET_MAX)}…` : head; + const snippet = clip(head, SUMMARY_SNIPPET_MAX); const tail = snippet.length > 0 ? ` — ${snippet}` : ""; return `${rank}. ${hit.path} (${hit.score.toFixed(3)})${tail}`; } @@ -783,10 +1031,23 @@ function hitDocLinks(hits: HybridHit[]): string[] { return paths; } +function summarizeReindex(value: unknown): string { + const r = value as VaultReindexResult; + const warnings = r.skipped.length + r.invalidFrontmatter.length; + const lines = [ + `Reindexed ${r.vault}: ${r.documentCount} doc(s), ${r.chunkCount} chunk(s), ` + + `vectors ${r.vectorEnabled ? "on" : "off"} — ${warnings} warning(s)`, + ]; + const top = [...r.skipped, ...r.invalidFrontmatter].slice(0, 5); + for (const f of top) lines.push(` ${f.path} — ${f.reason}`); + return lines.join("\n"); +} + export const searchTools: ToolDefinition[] = [ { name: "vault_search", title: "Search the vault", + oneLine: "Hybrid BM25 + vector search across the vault, with inline tension/decay flags.", annotations: { readOnlyHint: true }, description: "Hybrid search across the vault: BM25 lexical ranking combined with " + @@ -799,7 +1060,15 @@ export const searchTools: ToolDefinition[] = [ "of the fused ranking as compact judging records plus instructions — " + "and act as the reranker yourself: fusion scores measure retrieval " + "proximity, not answer quality, so judging the pool against the query " + - "can surface candidates ranked past the returned hits.", + "can surface candidates ranked past the returned hits. When the " + + "vault's `search.routing` config is on and no explicit `weights` was " + + "passed, a `routed` field reports the class the query router picked " + + "and which signals fired — present only when the router chose the " + + "weights, distinguishing a routed lexical-only result from one where " + + "embeddings degraded. When `rerank.provider` is configured, hits are " + + "additionally reordered by a local cross-encoder before slicing; " + + "`rerankUsed` reports whether that actually happened (false covers " + + "provider none, a cold model, an inference error, and a timeout alike).", inputSchema: { type: "object", properties: { @@ -852,6 +1121,31 @@ export const searchTools: ToolDefinition[] = [ }, weights: weightsResultSchema, hits: { type: "array", items: hybridHitSchema }, + rerankUsed: { + type: "boolean", + description: + "True iff the local cross-encoder reranker (rerank.provider config) actually " + + "reordered the pool. False covers every degrade path uniformly: provider " + + "'none', not-warm (a background warm was fired for next time), inference " + + "error, timeout, and an empty pool.", + }, + routed: { + type: "object", + description: + "Present only when the query router chose the weights (search.routing " + + "on, no explicit `weights` arg). Absent for explicit weights, " + + "routing-off, and embedding-degrade alike.", + properties: { + class: { type: "string", enum: ["extreme-lexical", "lexical", "balanced"] }, + signals: { + type: "array", + description: "Every signal that fired, not just the ones that decided `class`.", + items: { type: "string" }, + }, + }, + required: ["class", "signals"], + additionalProperties: false, + }, rerank: { type: "object", description: "Present only when rerank_candidates was passed.", @@ -863,30 +1157,37 @@ export const searchTools: ToolDefinition[] = [ additionalProperties: false, }, }, - required: ["query", "count", "vectorUsed", "weights", "hits"], + required: ["query", "count", "vectorUsed", "weights", "hits", "rerankUsed"], additionalProperties: false, }, summarize: (value) => { - const result = value as HybridSearchResult; + const result = value as VaultSearchResult; const n = result.hits.length; const mode = result.vectorUsed ? "bm25+vector" : "bm25 only"; const header = n === 0 ? `No hits for "${result.query}" (${mode}).` : `${n} hit${n === 1 ? "" : "s"} for "${result.query}" (${mode}).`; - const summary = summarizeHits(header, result.hits); + let summary = summarizeHits(header, result.hits); + if (result.routed) { + summary += `\nRouted: ${result.routed.class}${ + result.routed.signals.length > 0 ? ` (${result.routed.signals.join(", ")})` : "" + }`; + } + if (result.rerankUsed) summary += "\nReranked: local cross-encoder reordered this page."; // The rerank pool and its protocol text live on structuredContent; a // caller reading only `content` would otherwise never learn it opted in. return result.rerank ? `${summary}\nRerank pool: ${result.rerank.candidates.length} candidate(s) — you are the reranker; see structuredContent.rerank.` : summary; }, - docLinks: (value) => hitDocLinks((value as HybridSearchResult).hits), + docLinks: (value) => hitDocLinks((value as VaultSearchResult).hits), handler: (vaultRoot, args, access) => vaultSearch(vaultRoot, args, access), }, { name: "vault_search_related", title: "Find related documents", + oneLine: "Find documents related to a given vault document.", annotations: { readOnlyHint: true }, description: "Find documents related to a given vault document. Uses that " + @@ -939,6 +1240,7 @@ export const searchTools: ToolDefinition[] = [ { name: "vault_reindex", title: "Rebuild search index", + oneLine: "Rebuild the search index from the markdown files on disk.", // Not read-only — it writes the SQLite index. But it operates on a // rebuildable derived cache, not the markdown source of truth, so // destructiveHint is false. @@ -990,6 +1292,7 @@ export const searchTools: ToolDefinition[] = [ ], additionalProperties: false, }, + summarize: summarizeReindex, handler: (vaultRoot) => vaultReindex(vaultRoot), }, ]; diff --git a/src/tools/staged-actions.ts b/src/tools/staged-actions.ts index 963abc3e..a44fa22c 100644 --- a/src/tools/staged-actions.ts +++ b/src/tools/staged-actions.ts @@ -12,16 +12,35 @@ // vault_supersede, confidence-up → vault_set_confidence, merge → vault_merge // (the §11.4 write tools). A dispatch failure (including a malformed // proposed_diff) leaves the action pending so it can be retried. +// +// 2026-07-26 risk-triaged-ratification spec (Decisions 2 + 3) extended +// vault_ratify with: a batch `ids` alternative to `id` (Decision 2); a +// required-on-reject `reason_category` and an optional `amended_diff` that +// dispatches an edit-then-approve instead of the staged diff (Decision 3); +// and — per Mihir's 2026-07-27 decision resolving the spec's Decision-1 / +// kill-condition-#1 contradiction — a non-authoritative `risk_at_decision` +// snapshot on every decision record. The single-action approve/reject path is +// extracted into approveOneAction/rejectOneAction so the batch path and the +// single-`id` path share one implementation; a batch is N independent verdicts +// processed sequentially, never a transactional compound one. import { type AccessContext, canRatify, canRead, canWrite, isProposeOnly } from "../access/rbac.js"; +import { BATCH_RATIFY_MAX, rankPendingActions } from "../curation/risk.js"; import { + DECISION_KINDS, + type DecisionKind, getStagedActionById, + listStagedActions, nowISO, + REASON_CATEGORIES, + type ReasonCategory, recordDecision, STAGED_ACTION_TYPES, + type StagedAction, type StagedActionType, stageActionWithConflictCheck, } from "../curation/staged-actions.js"; +import { listTensions } from "../curation/tension.js"; import { bucketHiddenDownstream } from "../curation/tension-blast.js"; import { tier0DeprecateGate, tier0PromoteGate } from "../curation/tier0.js"; import { type LoadedDoc, loadDocuments } from "../curation/vault-docs.js"; @@ -30,6 +49,7 @@ import { validateFrontmatter } from "../frontmatter/schema.js"; import { err, ok, type Result } from "../frontmatter/types.js"; import { readFile, resolveVaultPath } from "../storage/local.js"; import { loadConfig } from "../utils/config.js"; +import { log as gitLog } from "../utils/git.js"; import { readRunId } from "../utils/run-id.js"; import type { ToolDefinition } from "./read.js"; import { @@ -182,6 +202,11 @@ export async function vaultStageAction( proposedDiff: args.proposed_diff, ...(runId.value !== undefined ? { runId: runId.value } : {}), ...(ttlDays !== undefined ? { ttlDays } : {}), + // C4 disposition (risk-triaged-ratification spec): the authenticated + // identity, when present, is the tally key the witness and the risk + // scorer's W term read — `proposed_by` remains claimed-agent display + // metadata only. Absent under operator context (no AccessContext). + ...(access?.user != null ? { stagedByPrincipal: access.user } : {}), }); } @@ -189,6 +214,67 @@ export async function vaultStageAction( // vault_ratify // --------------------------------------------------------------------------- +// --------------------------------------------------------------------------- +// vault_ratify form-mode elicitation (spec 2026-07-26, Decision 5) +// --------------------------------------------------------------------------- + +// What one elicitation round needs: the form prompt, the action it decides, +// and the vault HEAD at proposal time — the payload server.ts seals into the +// signed opaque request state. The MCP wiring (inputRequired, the state +// codec) stays in server.ts; this function is the tool layer's share: the +// same gates vaultRatify runs, so a role that could not ratify never sees a +// form, and an unknown/decided action errors before any round-trip starts. +// Single-id only — server.ts only enters this path when the call has no +// `ids` (see createServer's CallTool handler). +export interface RatifyElicitationSpec { + actionId: string; + message: string; + head: string | null; +} + +export async function describeRatifyElicitation( + vaultRoot: string, + args: Record, + access?: AccessContext, +): Promise> { + if (access && !canRatify(access.role)) { + return err(new Error(`access denied: role '${access.roleName}' cannot ratify staged actions`)); + } + if (access && isProposeOnly(access.role)) { + return err( + new Error( + `access denied: role '${access.roleName}' is propose-only — it cannot ` + + `ratify staged actions`, + ), + ); + } + const id = requireString(args, "id", "vault_ratify"); + if (!id.ok) return id; + const found = await getStagedActionById(vaultRoot, id.value); + if (!found.ok) return found; + const action = found.value; + if (!action) return err(new Error(`vault_ratify: unknown staged action: ${id.value}`)); + if (action.status !== "pending") { + return err( + new Error( + `vault_ratify: staged action ${id.value} is '${action.status}', not 'pending' — ` + + "it cannot be ratified", + ), + ); + } + // HEAD at proposal time rides the signed state for the audit trail; the + // dispatch path re-validates pending/conflict-free on resubmit regardless, + // so a missing HEAD (no git yet) degrades to null rather than blocking. + const head = await gitLog(vaultRoot, { limit: 1 }); + return ok({ + actionId: id.value, + message: + `Ratify staged action ${id.value}: ${action.actionType} ${action.targetPath}?` + + (action.rationale ? ` Rationale: ${action.rationale}` : ""), + head: head.ok ? (head.value[0]?.hash ?? null) : null, + }); +} + export interface RatifyResult { action_id: string; decision: "approve" | "reject"; @@ -197,6 +283,31 @@ export interface RatifyResult { // True when the vault runs shadow_mode (§11.5): the dispatch was computed // and shadow-logged but nothing was written, so the action stays pending. shadow?: boolean; + // Derived server-side, never an input: 'reject' on reject; 'approve' for a + // plain approval; 'edit-then-approve' when the caller supplied amended_diff. + // Absent on a shadow-mode approve (no decision was recorded). + decision_kind?: DecisionKind; +} + +// One id's outcome within a batch `ids` call. `ok` is false for anything that +// kept the action pending: unknown id, not-pending, a blocked tier-0 gate, or +// a dispatch failure — the same per-id failure modes the single-id path +// returns as an err, just captured instead of short-circuiting the batch. +export interface BatchRatifyOutcome { + action_id: string; + ok: boolean; + applied: boolean; + shadow?: boolean; + commit?: string; + decision_kind?: DecisionKind; + error?: string; +} + +export interface BatchRatifyResult { + decision: "approve" | "reject"; + results: BatchRatifyOutcome[]; + succeeded: number; + failed: number; } // The two tier-0 gate problem assemblies, shared by the four ratify gate @@ -223,108 +334,33 @@ function deprecateGateProblems(gate: ReturnType): str return problems.length > 0 ? problems.join("; ") : null; } -export async function vaultRatify( +// Approves ONE action: validates the (possibly amended) payload shape, runs +// the tier-0 gates against it, and dispatches to the matching write tool. No +// decision record is written here — the caller (vaultRatify) records the +// decision only after a live (non-shadow) dispatch succeeds, exactly the +// original single-action contract. `docs` is the caller's already-loaded doc +// set — batch callers pass a hoisted, invalidate-on-write snapshot (2026-07-26 +// spec, C2 disposition); the single-id path (docs === undefined) loads fresh, +// matching pre-refactor behavior exactly. +async function approveOneAction( vaultRoot: string, - args: Record, - access?: AccessContext, -): Promise> { - // Ratifying is the curation-verdict tier (§11.6): it needs the explicit - // `ratify` grant, not merely any read grant. The inner write tools still - // re-check their own canWrite/canPromote on dispatch. - if (access && !canRatify(access.role)) { - return err(new Error(`access denied: role '${access.roleName}' cannot ratify staged actions`)); - } - // A propose-only role must never ratify, even if a hand-built role grants - // both (config load rejects the combination, but AccessContexts constructed - // in code bypass that). Without this, approving a `write` action would be - // coerced by vaultWrite's propose-only path into staging a NEW proposal - // while the original got marked ratified/applied — a silent no-op. - if (access && isProposeOnly(access.role)) { - return err( - new Error( - `access denied: role '${access.roleName}' is propose-only — it cannot ` + - `ratify staged actions`, - ), - ); - } - - const id = requireString(args, "id", "vault_ratify"); - if (!id.ok) return id; - const decisionRaw = requireString(args, "decision", "vault_ratify"); - if (!decisionRaw.ok) return decisionRaw; - if (decisionRaw.value !== "approve" && decisionRaw.value !== "reject") { - return err(new Error("vault_ratify 'decision' must be 'approve' or 'reject'")); - } - const decision = decisionRaw.value; - const principal = requireString(args, "principal", "vault_ratify"); - if (!principal.ok) return principal; + action: StagedAction, + diffRaw: unknown, + principal: string, + access: AccessContext | undefined, + docs: LoadedDoc[] | undefined, +): Promise> { + const diff = diffRaw && typeof diffRaw === "object" ? (diffRaw as Record) : {}; - let reason: string | undefined; - if (args.reason !== undefined && args.reason !== null) { - if (typeof args.reason !== "string") { - return err(new Error("vault_ratify 'reason' must be a string")); - } - const trimmed = args.reason.trim(); - if (trimmed.length > 0) reason = trimmed; - } - - // Validate the action exists and is still open. - const found = await getStagedActionById(vaultRoot, id.value); - if (!found.ok) return found; - const action = found.value; - if (!action) return err(new Error(`vault_ratify: unknown staged action: ${id.value}`)); - if (action.status !== "pending") { - return err( - new Error( - `vault_ratify: staged action ${id.value} is '${action.status}', not 'pending' — ` + - "it cannot be ratified", - ), - ); - } - - const decidedAt = nowISO(); - - // --- reject: record and apply nothing --- - if (decision === "reject") { - const recorded = await recordDecision(vaultRoot, id.value, { - status: "rejected", - ratifiedAt: decidedAt, - ratifiedBy: principal.value, - ...(reason ? { reason } : {}), - ...(access?.user != null ? { decidedByPrincipal: access.user } : {}), - }); - if (!recorded.ok) return recorded; - return ok({ action_id: id.value, decision, applied: false }); - } - - // --- approve: dispatch by action type to the matching write tool --- - // A dispatch failure — including a malformed proposed_diff — leaves the action - // pending so it can be retried; no decision record is written until a write - // lands. The proposed_diff carries the per-action payload set at stage time. - const diff = - action.proposedDiff && typeof action.proposedDiff === "object" - ? (action.proposedDiff as Record) - : {}; - - // Tier 0 ratify gate (#232; quick win 1 of #236). Ratification is already a - // gate, so blocking here does not violate the advisory-curation rule — the - // direct write tools stay unblocked. A blocked approval is an error, which - // leaves the action pending (same contract as a dispatch failure): fix the - // underlying state and re-approve, or reject. Under RBAC the error names - // only docs the ratifier can read; a hidden remainder is coarsened - // (#217 B′), never reported as an exact count. - // A `write` proposal's payload shape is needed by both the gate and the - // dispatch — validate it once, up front. A malformed payload errors and - // leaves the action pending, same contract as a malformed supersede diff. let writePayload: { frontmatter: Record; body: string } | null = null; if (action.actionType === "write") { if (diff.frontmatter === null || typeof diff.frontmatter !== "object") { return err( - new Error(`vault_ratify: write action ${id.value} needs proposed_diff.frontmatter`), + new Error(`vault_ratify: write action ${action.id} needs proposed_diff.frontmatter`), ); } if (typeof diff.body !== "string") { - return err(new Error(`vault_ratify: write action ${id.value} needs proposed_diff.body`)); + return err(new Error(`vault_ratify: write action ${action.id} needs proposed_diff.body`)); } writePayload = { frontmatter: diff.frontmatter as Record, body: diff.body }; } @@ -334,9 +370,15 @@ export async function vaultRatify( action.actionType === "deprecate" || action.actionType === "write" ) { - const loaded = await loadDocuments(vaultRoot); - // Fail closed: without the doc set there is no gate, so no dispatch. - if (!loaded.ok) return loaded; + let loadedDocs: LoadedDoc[]; + if (docs) { + loadedDocs = docs; + } else { + const loaded = await loadDocuments(vaultRoot); + // Fail closed: without the doc set there is no gate, so no dispatch. + if (!loaded.ok) return loaded; + loadedDocs = loaded.value; + } const visible = access ? (d: LoadedDoc) => canRead(access.role, d.frontmatter.collection) : undefined; @@ -350,7 +392,7 @@ export async function vaultRatify( // alone has two bypasses: omitted `sources` inherit the on-disk value // unseen, and an omitted `status` on an already-canonical doc keeps it // canonical while dodging a payload-declared-status check. - const existing = loaded.value.find((d) => d.path === action.targetPath); + const existing = loadedDocs.find((d) => d.path === action.targetPath); const mergedRaw: Record = existing ? { ...(existing.frontmatter as Record) } : {}; @@ -377,8 +419,8 @@ export async function vaultRatify( content: writePayload.body, validation: report, }; - const docs = [...loaded.value.filter((d) => d.path !== action.targetPath), synthetic]; - const problems = promoteGateProblems(tier0PromoteGate(docs, action.targetPath, visible)); + const spliced = [...loadedDocs.filter((d) => d.path !== action.targetPath), synthetic]; + const problems = promoteGateProblems(tier0PromoteGate(spliced, action.targetPath, visible)); if (problems !== null) { return err( new Error( @@ -398,7 +440,7 @@ export async function vaultRatify( // gets the same gate. A merged superseded_by provides the // resolution path and passes, same as a forwarded deprecate. const problems = deprecateGateProblems( - tier0DeprecateGate(loaded.value, action.targetPath, visible), + tier0DeprecateGate(loadedDocs, action.targetPath, visible), ); if (problems !== null) { return err( @@ -412,7 +454,7 @@ export async function vaultRatify( } } else if (action.actionType === "promote") { const problems = promoteGateProblems( - tier0PromoteGate(loaded.value, action.targetPath, visible), + tier0PromoteGate(loadedDocs, action.targetPath, visible), ); if (problems !== null) { return err( @@ -427,7 +469,7 @@ export async function vaultRatify( // successor (same as supersede) — only an unforwarded deprecate can // strand canonical dependents on a retired source. const problems = deprecateGateProblems( - tier0DeprecateGate(loaded.value, action.targetPath, visible), + tier0DeprecateGate(loadedDocs, action.targetPath, visible), ); if (problems !== null) { return err( @@ -441,7 +483,6 @@ export async function vaultRatify( } } - let dispatched: Result; switch (action.actionType as StagedActionType) { case "write": { // Payload validated above (writePayload is always set for this type). @@ -449,59 +490,53 @@ export async function vaultRatify( // write so provenance correlates the landed content with the run that // proposed it (#235 → #233). if (!writePayload) { - return err(new Error(`vault_ratify: write action ${id.value} lost its payload`)); + return err(new Error(`vault_ratify: write action ${action.id} lost its payload`)); } - dispatched = await vaultWrite( + return vaultWrite( vaultRoot, { path: action.targetPath, frontmatter: writePayload.frontmatter, body: writePayload.body, - agent: principal.value, + agent: principal, ...(action.runId ? { run_id: action.runId } : {}), }, access, ); - break; } case "promote": - dispatched = await vaultPromote( - vaultRoot, - { path: action.targetPath, agent: principal.value }, - access, - ); - break; + return vaultPromote(vaultRoot, { path: action.targetPath, agent: principal }, access); case "deprecate": { const deprecateArgs: Record = { path: action.targetPath, - agent: principal.value, + agent: principal, reason: action.rationale, }; // Carry through a superseded_by hint from the proposed diff if present. if (typeof diff.superseded_by === "string") { deprecateArgs.superseded_by = diff.superseded_by; } - dispatched = await vaultDeprecate(vaultRoot, deprecateArgs, access); - break; + return vaultDeprecate(vaultRoot, deprecateArgs, access); } case "supersede": { // proposed_diff = { superseded_by: "" } if (typeof diff.superseded_by !== "string" || diff.superseded_by.trim().length === 0) { return err( - new Error(`vault_ratify: supersede action ${id.value} needs proposed_diff.superseded_by`), + new Error( + `vault_ratify: supersede action ${action.id} needs proposed_diff.superseded_by`, + ), ); } - dispatched = await vaultSupersede( + return vaultSupersede( vaultRoot, { old_path: action.targetPath, new_path: diff.superseded_by, reason: action.rationale, - agent: principal.value, + agent: principal, }, access, ); - break; } case "confidence-up": { // proposed_diff = { confidence: "" }. The enum name is @@ -509,21 +544,20 @@ export async function vaultRatify( if (typeof diff.confidence !== "string") { return err( new Error( - `vault_ratify: confidence-up action ${id.value} needs proposed_diff.confidence`, + `vault_ratify: confidence-up action ${action.id} needs proposed_diff.confidence`, ), ); } - dispatched = await vaultSetConfidence( + return vaultSetConfidence( vaultRoot, { path: action.targetPath, confidence: diff.confidence, reason: action.rationale, - agent: principal.value, + agent: principal, }, access, ); - break; } case "merge": { // proposed_diff = { merge_from: [path_a, path_b], body, frontmatter? }; @@ -537,7 +571,7 @@ export async function vaultRatify( ) { return err( new Error( - `vault_ratify: merge action ${id.value} needs proposed_diff.merge_from ` + + `vault_ratify: merge action ${action.id} needs proposed_diff.merge_from ` + "(two paths) and proposed_diff.body", ), ); @@ -547,43 +581,370 @@ export async function vaultRatify( path_b: mergeFrom[1], target_path: action.targetPath, body: diff.body, - agent: principal.value, + agent: principal, }; if (diff.frontmatter && typeof diff.frontmatter === "object") { mergeArgs.frontmatter = diff.frontmatter; } - dispatched = await vaultMerge(vaultRoot, mergeArgs, access); - break; + return vaultMerge(vaultRoot, mergeArgs, access); } default: return err(new Error(`vault_ratify: no dispatch for action type '${action.actionType}'`)); } +} - if (!dispatched.ok) return dispatched; - - // Shadow mode (§11.5): the dispatch computed and shadow-logged the write but - // applied nothing. Recording a `ratified` decision over a write that never - // landed would be false history — leave the action pending so a live-mode - // ratification can really apply it later. - if (dispatched.value.shadow) { - return ok({ action_id: id.value, decision, applied: false, shadow: true }); - } - - const recorded = await recordDecision(vaultRoot, id.value, { - status: "ratified", +// Rejects ONE action: records the decision, applies nothing. Thin wrapper +// kept separate from approveOneAction per the plan's refactor (C2) — the +// reject path has no gate/dispatch, only bookkeeping. +async function rejectOneAction( + vaultRoot: string, + actionId: string, + principal: string, + reason: string | undefined, + reasonCategory: ReasonCategory | undefined, + riskAtDecision: number | null, + decidedAt: string, + access: AccessContext | undefined, +): Promise> { + return recordDecision(vaultRoot, actionId, { + status: "rejected", ratifiedAt: decidedAt, - ratifiedBy: principal.value, + ratifiedBy: principal, + decisionKind: "reject", ...(reason ? { reason } : {}), + ...(reasonCategory ? { reasonCategory } : {}), + ...(riskAtDecision !== null ? { riskAtDecision } : {}), ...(access?.user != null ? { decidedByPrincipal: access.user } : {}), }); - if (!recorded.ok) return recorded; +} - return ok({ - action_id: id.value, - decision, - applied: true, - ...(dispatched.value.commit ? { commit: dispatched.value.commit } : {}), +export async function vaultRatify( + vaultRoot: string, + args: Record, + access?: AccessContext, +): Promise> { + // Ratifying is the curation-verdict tier (§11.6): it needs the explicit + // `ratify` grant, not merely any read grant. The inner write tools still + // re-check their own canWrite/canPromote on dispatch. + if (access && !canRatify(access.role)) { + return err(new Error(`access denied: role '${access.roleName}' cannot ratify staged actions`)); + } + // A propose-only role must never ratify, even if a hand-built role grants + // both (config load rejects the combination, but AccessContexts constructed + // in code bypass that). Without this, approving a `write` action would be + // coerced by vaultWrite's propose-only path into staging a NEW proposal + // while the original got marked ratified/applied — a silent no-op. + if (access && isProposeOnly(access.role)) { + return err( + new Error( + `access denied: role '${access.roleName}' is propose-only — it cannot ` + + `ratify staged actions`, + ), + ); + } + + // --- id / ids: exactly one, batch shape validated up front (Decision 2) --- + const hasId = typeof args.id === "string" && args.id.trim().length > 0; + const hasIds = args.ids !== undefined && args.ids !== null; + if (hasId && hasIds) { + return err(new Error("vault_ratify accepts exactly one of 'id' or 'ids', not both")); + } + if (!hasId && !hasIds) { + return err(new Error("vault_ratify requires exactly one of 'id' or 'ids'")); + } + + let ids: string[]; + if (hasIds) { + if (!Array.isArray(args.ids)) { + return err(new Error("vault_ratify 'ids' must be an array of strings")); + } + if (args.ids.length === 0) { + return err(new Error("vault_ratify 'ids' must not be empty")); + } + if (args.ids.length > BATCH_RATIFY_MAX) { + return err( + new Error(`vault_ratify 'ids' must not exceed ${BATCH_RATIFY_MAX} — the batch cap`), + ); + } + const seen = new Set(); + const cleaned: string[] = []; + for (const raw of args.ids) { + if (typeof raw !== "string" || raw.trim().length === 0) { + return err(new Error("vault_ratify 'ids' must be an array of non-empty strings")); + } + const v = raw.trim(); + if (seen.has(v)) { + return err(new Error(`vault_ratify 'ids' contains a duplicate id: ${v}`)); + } + seen.add(v); + cleaned.push(v); + } + ids = cleaned; + } else { + ids = [(args.id as string).trim()]; + } + + const decisionRaw = requireString(args, "decision", "vault_ratify"); + if (!decisionRaw.ok) return decisionRaw; + if (decisionRaw.value !== "approve" && decisionRaw.value !== "reject") { + return err(new Error("vault_ratify 'decision' must be 'approve' or 'reject'")); + } + const decision = decisionRaw.value; + const principal = requireString(args, "principal", "vault_ratify"); + if (!principal.ok) return principal; + + let reason: string | undefined; + if (args.reason !== undefined && args.reason !== null) { + if (typeof args.reason !== "string") { + return err(new Error("vault_ratify 'reason' must be a string")); + } + const trimmed = args.reason.trim(); + if (trimmed.length > 0) reason = trimmed; + } + + // --- reason_category (Decision 3 / C6): required on reject, optional on + // plain approve, required alongside amended_diff. No silent default — an + // un-chosen 'other' would gut the calibration signal the category exists + // to collect. --- + let reasonCategory: ReasonCategory | undefined; + if (args.reason_category !== undefined && args.reason_category !== null) { + if ( + typeof args.reason_category !== "string" || + !(REASON_CATEGORIES as readonly string[]).includes(args.reason_category) + ) { + return err( + new Error(`vault_ratify 'reason_category' must be one of: ${REASON_CATEGORIES.join(", ")}`), + ); + } + reasonCategory = args.reason_category as ReasonCategory; + } + + // --- amended_diff (Decision 3): single-id, approve-only --- + const hasAmendedDiff = args.amended_diff !== undefined && args.amended_diff !== null; + let amendedDiff: unknown; + if (hasAmendedDiff) { + if (hasIds) { + return err( + new Error( + "vault_ratify 'amended_diff' is single-id only — an amendment is per-action " + + "deliberation, incompatible with 'ids'", + ), + ); + } + if (decision !== "approve") { + return err(new Error("vault_ratify 'amended_diff' is only valid with decision 'approve'")); + } + if (typeof args.amended_diff !== "object") { + return err(new Error("vault_ratify 'amended_diff' must be an object")); + } + amendedDiff = args.amended_diff; + } + + // This is an INTENTIONAL, spec-mandated contract break for reject callers + // (C6 disposition): the `decision` enum and every approve-path caller are + // untouched; reject now requires reason_category. The error enumerates the + // categories so an agent caller self-corrects in one round trip. + if (decision === "reject" && reasonCategory === undefined) { + return err( + new Error( + `vault_ratify: 'reason_category' is required on reject — one of: ` + + `${REASON_CATEGORIES.join(", ")}`, + ), + ); + } + if (hasAmendedDiff && reasonCategory === undefined) { + return err( + new Error( + `vault_ratify: 'reason_category' is required with 'amended_diff' — one of: ` + + `${REASON_CATEGORIES.join(", ")}`, + ), + ); + } + + // Shadow mode + amended_diff (C7): silently discarding an operator-authored + // amendment is the one option this must never do. Shadow mode records no + // decisions of any kind today (that recording surface belongs to the + // shadow-mode graduation story, out of scope here per the spec's own + // boundaries) — so an amendment under shadow is an explicit error instead. + if (hasAmendedDiff) { + const shadowConfig = loadConfig(vaultRoot); + if (!shadowConfig.ok) return shadowConfig; + if (shadowConfig.value.shadowMode) { + return err( + new Error( + "vault_ratify: shadow mode is active for this action type; the amendment would be " + + "discarded — re-issue without amended_diff, or ratify after shadow mode is lifted", + ), + ); + } + } + + // --- hoist: collapse the log once, load tensions and docs once (C2) --- + const actionsRes = await listStagedActions(vaultRoot); + if (!actionsRes.ok) return actionsRes; + const tensionsRes = await listTensions(vaultRoot); + if (!tensionsRes.ok) return tensionsRes; + const docsRes = await loadDocuments(vaultRoot); + if (!docsRes.ok) return docsRes; + let docs = docsRes.value; + + const actionsById = new Map(actionsRes.value.map((a) => [a.id, a] as const)); + const now = new Date(); + + // The non-authoritative risk_at_decision snapshot (Mihir's 2026-07-27 + // decision): computed ONCE from this hoisted, pre-decision snapshot and + // reused for every id in the call (single or batch) — consistent with the + // batch's cost discipline and with the field's own "frozen observation, + // never re-read for ordering" framing. Full-graph (no pathVisible). + const { items: rankedAtStart } = rankPendingActions({ + actions: actionsRes.value, + docs, + tensions: tensionsRes.value, + now, }); + const riskById = new Map(rankedAtStart.map((i) => [i.id, i.risk] as const)); + + const decidedAt = nowISO(); + const results: BatchRatifyOutcome[] = []; + + for (const id of ids) { + const action = actionsById.get(id); + if (!action) { + results.push({ + action_id: id, + ok: false, + applied: false, + error: `vault_ratify: unknown staged action: ${id}`, + }); + continue; + } + if (action.status !== "pending") { + results.push({ + action_id: id, + ok: false, + applied: false, + error: + `vault_ratify: staged action ${id} is '${action.status}', not 'pending' — ` + + "it cannot be ratified", + }); + continue; + } + + // action.status === "pending" here (checked above), so it was necessarily + // scored into rankedAtStart / riskById above — the fallback is defensive + // only (e.g. a future refactor that filters riskById). + const riskAtDecision = riskById.get(id) ?? null; + + if (decision === "reject") { + const recorded = await rejectOneAction( + vaultRoot, + id, + principal.value, + reason, + reasonCategory, + riskAtDecision, + decidedAt, + access, + ); + if (!recorded.ok) { + results.push({ action_id: id, ok: false, applied: false, error: recorded.error.message }); + continue; + } + results.push({ action_id: id, ok: true, applied: false, decision_kind: "reject" }); + continue; + } + + // --- approve --- + const isAmending = !hasIds && hasAmendedDiff; + const effectiveDiffRaw = isAmending ? amendedDiff : action.proposedDiff; + const decisionKind: DecisionKind = isAmending ? "edit-then-approve" : "approve"; + + const dispatched = await approveOneAction( + vaultRoot, + action, + effectiveDiffRaw, + principal.value, + access, + docs, + ); + if (!dispatched.ok) { + results.push({ action_id: id, ok: false, applied: false, error: dispatched.error.message }); + continue; + } + + // Shadow mode (§11.5): the dispatch computed and shadow-logged the write + // but applied nothing. Recording a `ratified` decision over a write that + // never landed would be false history — leave the action pending so a + // live-mode ratification can really apply it later. + if (dispatched.value.shadow) { + results.push({ action_id: id, ok: true, applied: false, shadow: true }); + continue; + } + + const recorded = await recordDecision(vaultRoot, id, { + status: "ratified", + ratifiedAt: decidedAt, + ratifiedBy: principal.value, + decisionKind, + ...(reason ? { reason } : {}), + ...(reasonCategory ? { reasonCategory } : {}), + ...(isAmending ? { amendedDiff: effectiveDiffRaw } : {}), + ...(riskAtDecision !== null ? { riskAtDecision } : {}), + ...(access?.user != null ? { decidedByPrincipal: access.user } : {}), + }); + if (!recorded.ok) { + // The write LANDED but the decision record failed to append — surface + // loudly rather than silently losing the outcome; `applied` is still + // true because the mutation and its commit are real. + results.push({ + action_id: id, + ok: false, + applied: true, + ...(dispatched.value.commit ? { commit: dispatched.value.commit } : {}), + error: recorded.error.message, + }); + continue; + } + + results.push({ + action_id: id, + ok: true, + applied: true, + decision_kind: decisionKind, + ...(dispatched.value.commit ? { commit: dispatched.value.commit } : {}), + }); + + // Invalidate-on-write (C2): this dispatch mutated the vault, so later + // ids' tier-0 gates must see the mutated state. Reloading only here keeps + // the common cases (batch reject, gate-blocked batches, shadow) to one + // load total. A reload failure means the vault is now in an unknown + // state relative to what later gates would check — fail the whole call + // rather than gate the rest against stale docs; every id decided so far + // is already durably recorded (recordDecision/rejectOneAction already + // landed), so this is safe to surface as an error and re-issue. + const reloaded = await loadDocuments(vaultRoot); + if (!reloaded.ok) return reloaded; + docs = reloaded.value; + } + + if (!hasIds) { + const single = results[0]; + if (!single) return err(new Error(`vault_ratify: no outcome recorded for ${ids[0]}`)); + if (!single.ok) + return err(new Error(single.error ?? `vault_ratify: ${single.action_id} failed`)); + return ok({ + action_id: single.action_id, + decision, + applied: single.applied, + ...(single.commit ? { commit: single.commit } : {}), + ...(single.shadow ? { shadow: true } : {}), + ...(single.decision_kind ? { decision_kind: single.decision_kind } : {}), + }); + } + + const succeeded = results.filter((r) => r.ok).length; + const failed = results.length - succeeded; + return ok({ decision, results, succeeded, failed }); } // --------------------------------------------------------------------------- @@ -614,7 +975,7 @@ const stageActionOutputSchema: Record = { additionalProperties: false, }; -const ratifyOutputSchema: Record = { +const ratifySingleOutputSchema: Record = { type: "object", properties: { action_id: { type: "string" }, @@ -627,11 +988,103 @@ const ratifyOutputSchema: Record = { // §11.5: computed and shadow-logged but not written — the action stays // pending for a live ratification later. shadow: { type: "boolean" }, + decision_kind: { + type: "string", + enum: [...DECISION_KINDS], + description: + "Present on approve: 'approve', or 'edit-then-approve' when amended_diff was used", + }, }, required: ["action_id", "decision", "applied"], additionalProperties: false, }; +const ratifyBatchOutcomeSchema: Record = { + type: "object", + properties: { + action_id: { type: "string" }, + ok: { type: "boolean", description: "False for anything that left the action pending" }, + applied: { type: "boolean" }, + shadow: { type: "boolean" }, + commit: { type: "string" }, + decision_kind: { type: "string", enum: [...DECISION_KINDS] }, + error: { type: "string" }, + }, + required: ["action_id", "ok", "applied"], + additionalProperties: false, +}; + +const ratifyBatchOutputSchema: Record = { + type: "object", + properties: { + decision: { type: "string", enum: ["approve", "reject"] }, + results: { type: "array", items: ratifyBatchOutcomeSchema }, + succeeded: { type: "integer" }, + failed: { type: "integer" }, + }, + required: ["decision", "results", "succeeded", "failed"], + additionalProperties: false, +}; + +// Decision 2: a single-`id` call keeps today's RatifyResult shape (plus the +// optional decision_kind); a batch `ids` call returns the aggregate shape. +// The two are structurally disjoint (results/succeeded/failed vs +// action_id/applied), so anyOf is unambiguous for a validator. MCP requires +// `type: 'object'` at the outputSchema root (Tool.outputSchema in the SDK's +// types.ts) — harmless here since both anyOf branches are themselves object +// schemas, so the extra top-level constraint is redundant, never conflicting. +const ratifyOutputSchema: Record = { + type: "object", + anyOf: [ratifySingleOutputSchema, ratifyBatchOutputSchema], +}; + +// --------------------------------------------------------------------------- +// Compact `content` summaries (spec 2026-07-26, Decision 3, PR 1 gap +// closure). Neither tool's docLinks names a path: the target document +// itself is not part of either result value (StageActionResult carries only +// the proposal's own id/expiry; RatifyResult carries only the decision), +// and the docLinks hard rule forbids inventing one from args. +// --------------------------------------------------------------------------- + +function summarizeStageAction(value: unknown): string { + const r = value as StageActionResult; + const conflicts = + r.conflicts_with.length > 0 + ? `conflicts with ${r.conflicts_with.length}: ${r.conflicts_with.join(", ")}` + : "uncontested"; + const lines = [`staged ${r.id}, expires ${r.expires_at} — ${conflicts}`]; + if (r.tension_id) lines.push(`tension: ${r.tension_id}`); + if (r.tension_error) lines.push(`tension error: ${r.tension_error}`); + return lines.join("\n"); +} + +function isBatchRatifyResult(value: unknown): value is BatchRatifyResult { + return typeof value === "object" && value !== null && "results" in value; +} + +function summarizeRatify(value: unknown): string { + if (isBatchRatifyResult(value)) { + const lines = [ + `batch ${value.decision}: ${value.succeeded} succeeded, ${value.failed} failed ` + + `(${value.results.length} total)`, + ]; + for (const r of value.results) { + const outcome = !r.ok + ? `error: ${r.error ?? "unknown"}` + : r.shadow + ? "shadow" + : r.applied + ? (r.commit ?? "applied") + : "not applied"; + lines.push(` ${r.action_id} — ${outcome}`); + } + return lines.join("\n"); + } + const r = value as RatifyResult; + const outcome = r.shadow ? "shadow" : r.applied ? (r.commit ?? "applied") : "not applied"; + return `${r.action_id} ${r.decision} — ${outcome}`; +} + // --------------------------------------------------------------------------- // MCP tool definitions // --------------------------------------------------------------------------- @@ -640,6 +1093,7 @@ export const stagedActionTools: ToolDefinition[] = [ { name: "vault_stage_action", title: "Stage an action for ratification", + oneLine: "Stage a proposed action for later ratification.", annotations: { destructiveHint: false }, description: "Record a proposed change to the vault for later human ratification via " + @@ -697,41 +1151,83 @@ export const stagedActionTools: ToolDefinition[] = [ additionalProperties: false, }, outputSchema: stageActionOutputSchema, + summarize: summarizeStageAction, handler: (vaultRoot, args, access) => vaultStageAction(vaultRoot, args, access), }, { name: "vault_ratify", - title: "Approve or reject a staged action", + title: "Approve or reject staged action(s)", + oneLine: "Approve or reject one or more staged actions.", annotations: { destructiveHint: true }, description: - "Approve or reject a single pending staged action. On approve, dispatches " + - "to the matching write tool (promote → vault_promote, deprecate → " + - "vault_deprecate, supersede → vault_supersede, confidence-up → " + - "vault_set_confidence, merge → vault_merge, write → vault_write) and " + - "auto-commits. On reject, " + - "records the rejection and applies nothing. A dispatch failure leaves the " + - "action pending. Approving a promote, an unforwarded deprecate, or a " + - "write that declares status canonical runs the " + - "tier-0 gate first (#232): if applying would create a certain structural " + - "violation (broken source refs, canonical citing draft/deprecated/archived, " + - "schema-invalid frontmatter, stranded canonical dependents), the approval " + - "errors and the action stays pending — fix the state and re-approve, or " + - "reject. Errors if the id is unknown or the action is not pending " + - "(already decided or expired). Requires the role's 'ratify' grant. If " + - "the vault runs shadow_mode, an approved dispatch is computed and " + - "shadow-logged but NOT applied — the result carries shadow: true and " + - "the action stays pending for a live ratification later.", + "Approve or reject one pending staged action ('id'), or up to " + + `${BATCH_RATIFY_MAX} at once ('ids', an explicit list — never a threshold ` + + "or an 'all pending' sentinel: the parameter shape cannot express one). " + + "With a single 'id', 'decision' may be omitted to have the server elicit " + + "it from the human as a form (spec 2026-07-26, Decision 5) — the server " + + "proposes, the human disposes; a batch 'ids' call always requires an " + + "explicit 'decision'. Each id is processed independently in caller order " + + "— RBAC, the " + + "pending-status check, and the tier-0 gates run per action exactly as a " + + "single call would; one gate-blocked or failing id leaves THAT action " + + "pending and the batch continues with per-id outcomes, never a rollback " + + "of the rest. Durability: each id's decision record and git commit land " + + "before the next id is processed, so an interrupted batch leaves a " + + "complete record of what landed — re-issuing the same batch is the " + + "recovery path (already-decided ids report a 'not pending' outcome; the " + + "remainder applies). On approve, dispatches to the matching write tool " + + "(promote → vault_promote, deprecate → vault_deprecate, supersede → " + + "vault_supersede, confidence-up → vault_set_confidence, merge → " + + "vault_merge, write → vault_write) and auto-commits. On reject, records " + + "the rejection and applies nothing; 'reason_category' is REQUIRED on " + + "reject — a spec-mandated, intentional break from the prior optional " + + "contract — one of: wrong-conclusion, wrong-target, overbroad, " + + "stale-evidence, duplicate, formatting, policy, other. Approve-path " + + "callers are unaffected: 'reason_category' stays optional there, unless " + + "'amended_diff' is present, where it is also required. 'amended_diff' " + + "(single-'id' + approve only) dispatches an edited payload instead of " + + "the staged one — the tier-0 gates run against the amendment too — and " + + "the decision record keeps both what was proposed and what actually " + + "landed (decision_kind: 'edit-then-approve'). If shadow_mode is active, " + + "'amended_diff' errors rather than silently discarding the amendment: " + + "re-issue without it, or ratify after shadow mode is lifted. Approving " + + "a promote, an unforwarded deprecate, or a write that declares status " + + "canonical runs the tier-0 gate first (#232): a certain structural " + + "violation (broken source refs, canonical citing draft/deprecated/" + + "archived, schema-invalid frontmatter, stranded canonical dependents) " + + "errors and the action stays pending. Errors if an id is unknown or not " + + "pending. Requires the role's 'ratify' grant. If the vault runs " + + "shadow_mode, an approved dispatch is computed and shadow-logged but NOT " + + "applied — the outcome carries shadow: true and the action stays " + + "pending for a live ratification later.", inputSchema: { type: "object", properties: { id: { type: "string", - description: "Id of the staged action to decide, e.g. 'stage-042'", + description: + "Id of a single staged action to decide, e.g. 'stage-042'. Exactly one of id/ids.", + }, + ids: { + type: "array", + items: { type: "string" }, + minItems: 1, + maxItems: BATCH_RATIFY_MAX, + description: + `Explicit list of staged-action ids to decide together, 1-${BATCH_RATIFY_MAX}, ` + + "no duplicates. Exactly one of id/ids. One shared decision/reason/" + + "reason_category applies to every id.", }, decision: { type: "string", enum: ["approve", "reject"], - description: "Whether to approve (apply) or reject the action", + description: + "Whether to approve (apply) or reject the action(s). With a single " + + "'id' (no 'ids'), omit it to have the server elicit the decision " + + "from the human as a form (spec 2026-07-26, Decision 5): the server " + + "proposes, the human disposes, and the form's preselected answer is " + + "the safe 'reject'. Required when 'ids' is present — a batch has no " + + "single-action form to elicit.", }, principal: { type: "string", @@ -741,11 +1237,27 @@ export const stagedActionTools: ToolDefinition[] = [ type: "string", description: "Optional free-text reason recorded with the decision", }, + reason_category: { + type: "string", + enum: [...REASON_CATEGORIES], + description: + "Machine-readable correction category. Required on reject and when " + + "amended_diff is present; optional on a plain approve.", + }, + amended_diff: { + type: "object", + description: + "Single-'id' + decision:'approve' only. An edited payload dispatched " + + "instead of the staged proposed_diff; same shape rules as proposed_diff " + + "for the action's type. Errors under shadow_mode instead of discarding it.", + additionalProperties: true, + }, }, - required: ["id", "decision", "principal"], + required: ["principal"], additionalProperties: false, }, outputSchema: ratifyOutputSchema, + summarize: summarizeRatify, handler: (vaultRoot, args, access) => vaultRatify(vaultRoot, args, access), }, ]; diff --git a/src/tools/summary.ts b/src/tools/summary.ts new file mode 100644 index 00000000..01f6fc60 --- /dev/null +++ b/src/tools/summary.ts @@ -0,0 +1,23 @@ +// Shared helpers for the `content`-channel summarizers (spec 2026-07-26, +// Decision 3). Extracted from vault_lint's summarizer so every tool's +// compact summary clips detail text the same way, instead of each +// summarizer inventing its own truncation rule. + +// Default cap on how much of a free-text field (a finding detail, a +// rationale, ...) a summary line shows before eliding the rest. The full +// text always still rides `structuredContent`; this only bounds what the +// MODEL-FACING text block spends tokens on. +export const SUMMARY_DETAIL_CHARS = 110; + +// Default cap on how many rows a listing-shaped summary enumerates (index +// entries, edges, queue items, ...) before switching to a "N more in +// structuredContent" trailer. +export const SUMMARY_MAX_ROWS = 20; + +// Collapses internal whitespace (so a multi-line detail renders as one +// summary line) and truncates to `max` chars with an ellipsis. Idempotent on +// already-short, already-flat text. +export function clip(text: string, max: number): string { + const flat = text.replace(/\s+/g, " ").trim(); + return flat.length > max ? `${flat.slice(0, max - 1)}…` : flat; +} diff --git a/src/tools/themes.ts b/src/tools/themes.ts index 9106a6b4..3d4a8375 100644 --- a/src/tools/themes.ts +++ b/src/tools/themes.ts @@ -26,7 +26,7 @@ import { type AccessContext, canRead } from "../access/rbac.js"; import { err, ok, type Result } from "../frontmatter/types.js"; -import { getProvider } from "../search/vector.js"; +import { getProvider, toIndexDim } from "../search/vector.js"; import { blobToEmbedding, type IndexDb, type IndexedDocument } from "../storage/index-db.js"; import { clusterCoherence, @@ -229,16 +229,23 @@ function loadEmbeddingsByPath( collection ? db.prepare(sql).all(model, collection) : db.prepare(sql).all(model) ) as Row[]; const provider = getProvider(); - const expectedDim = provider.dim; + // The durable cache stores NATIVE-dim vectors (2026-07-26 embedding- + // refresh-quantization spec, disposition C9) — the dim guard below must + // compare against nativeDim (falling back to dim for providers with no + // Matryoshka gap), not the configured index dim, or every row would fail + // the guard. Each surviving vector is truncated to the CONFIGURED dim via + // toIndexDim before being handed to the caller's clustering math — same + // pattern as relatedSearch's meanEmbedding inputs (hybrid.ts). + const expectedDim = provider.nativeDim ?? provider.dim; const out = new Map(); for (const row of rows) { if (!row.embedding) continue; // Defense-in-depth: skip rows whose stored dim disagrees with the - // provider's expected dim — same guard `getAllChunks` applies. + // provider's expected (native) dim — same guard `getAllChunks` applies. const blobOk = row.embedding.length === expectedDim * 4; const dimOk = row.dim === expectedDim; if (!blobOk || !dimOk) continue; - const vec = blobToEmbedding(row.embedding); + const vec = toIndexDim(blobToEmbedding(row.embedding), provider.dim); const list = out.get(row.path); if (list) list.push(vec); else out.set(row.path, [vec]); @@ -584,6 +591,7 @@ export async function vaultThemes( supersededBy: null, validFrom: null, validUntil: null, + updatedBy: "", }); } @@ -833,10 +841,42 @@ const themeSchema = { additionalProperties: false, }; +// --------------------------------------------------------------------------- +// Compact `content` summary + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +function summarizeThemes(value: unknown): string { + const r = value as VaultThemesResult; + if (r.themes.length === 0) return `0 themes over ${r.totalDocuments} document(s).`; + const lines = [ + `${r.themes.length} theme(s) over ${r.totalDocuments} document(s) (k=${r.selectedK}):`, + ]; + for (const t of r.themes) { + const exemplar = t.representativeDocs[0]; + lines.push(` ${t.label} — ${t.documentCount} doc(s)` + (exemplar ? `, e.g. ${exemplar}` : "")); + } + return lines.join("\n"); +} + +// One exemplar per theme — the primary member ranked first by membership +// weight. A theme with no retained primary member (all visitors) contributes +// nothing rather than guessing. +function docLinksThemes(value: unknown): string[] { + const r = value as VaultThemesResult; + const paths: string[] = []; + for (const t of r.themes) { + const exemplar = t.representativeDocs[0]; + if (exemplar) paths.push(exemplar); + } + return paths; +} + export const themesTools: ToolDefinition[] = [ { name: "vault_themes", title: "Cluster vault themes", + oneLine: "Cluster vault documents into thematic groups via k-means over embeddings.", annotations: { readOnlyHint: true }, description: "Surface thematic clusters across the vault using k-means over CHUNK " + @@ -935,6 +975,8 @@ export const themesTools: ToolDefinition[] = [ ], additionalProperties: false, }, + summarize: summarizeThemes, + docLinks: docLinksThemes, handler: (vaultRoot, args, access) => vaultThemes(vaultRoot, args, access), }, ]; diff --git a/src/tools/tier1.ts b/src/tools/tier1.ts index f7b5aa4c..ad7df984 100644 --- a/src/tools/tier1.ts +++ b/src/tools/tier1.ts @@ -164,10 +164,27 @@ export async function vaultTier1( } } +// --------------------------------------------------------------------------- +// Compact `content` summary + resource link (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +function summarizeTier1(value: unknown): string { + const r = value as Tier1Result; + const s = r.summary; + return ( + `${r.unit} (${r.change_source}, fields: ${r.changed_fields.join(", ") || "none"}) — ` + + `${r.verdicts.length} dependent(s): ${s.unaffected} unaffected, ${s.affected} affected, ` + + `${s.possibly_affected} possibly-affected, ${s.semantic_review} semantic-review ` + + `(resolved_at_tier1: ${s.resolved_at_tier1})` + ); +} + export const tier1Tools: ToolDefinition[] = [ { name: "vault_tier1", title: "Type-directed change dispatch (tier 1)", + oneLine: "Classify a change's effect on dependents by type (tier 1).", annotations: { readOnlyHint: true }, description: "Deterministic, LLM-free compatibility dispatch for a changed document " + @@ -258,6 +275,8 @@ export const tier1Tools: ToolDefinition[] = [ }, required: ["unit", "changed_fields", "change_source", "verdicts", "summary"], }, + summarize: summarizeTier1, + docLinks: (value) => [(value as Tier1Result).unit], handler: (vaultRoot, args, access) => vaultTier1(vaultRoot, args, access), }, ]; diff --git a/src/tools/tier2.ts b/src/tools/tier2.ts index 73d27470..7289d2a7 100644 --- a/src/tools/tier2.ts +++ b/src/tools/tier2.ts @@ -426,10 +426,61 @@ const tier2VerdictSchema: Record = { ], }; +// --------------------------------------------------------------------------- +// Compact `content` summaries + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure) +// --------------------------------------------------------------------------- + +// Row cap for the queue summary — a tool-specific value, distinct from the +// shared SUMMARY_MAX_ROWS default: a tier-2 item carries a usage span and a +// full question, so 10 rows is already a denser line budget than a bare path +// listing. +const TIER2_QUEUE_SUMMARY_ROWS = 10; + +function summarizeTier2Queue(value: unknown): string { + const r = value as Tier2QueueResult; + if (r.total === 0) return "0 pending tier-2 judgments."; + const shown = r.items.slice(0, TIER2_QUEUE_SUMMARY_ROWS); + const lines = [ + `${r.total} pending tier-2 judgment(s):`, + ...shown.map((i) => ` ${i.artifact} vs ${i.unit} (${i.edge_class})`), + ]; + const rest = r.total - shown.length; + if (rest > 0) lines.push(` … ${rest} more in structuredContent`); + return lines.join("\n"); +} + +function docLinksTier2Queue(value: unknown): string[] { + const r = value as Tier2QueueResult; + const seen = new Set(); + const paths: string[] = []; + for (const i of r.items.slice(0, TIER2_QUEUE_SUMMARY_ROWS)) { + for (const p of [i.artifact, i.unit]) { + if (seen.has(p)) continue; + seen.add(p); + paths.push(p); + } + } + return paths; +} + +function summarizeTier2Verdict(value: unknown): string { + const r = value as Tier2VerdictResult; + const v = r.recorded; + const tension = r.tension_id ? ` — tension ${r.tension_id}` : ""; + return `${v.artifact} vs ${v.unit}: ${v.verdict}${tension}`; +} + +function docLinksTier2Verdict(value: unknown): string[] { + const v = (value as Tier2VerdictResult).recorded; + return [v.artifact, v.unit]; +} + export const tier2Tools: ToolDefinition[] = [ { name: "vault_tier2_queue", title: "Semantic-review queue (tier 2)", + oneLine: "List pairs awaiting semantic review (tier 2).", annotations: { readOnlyHint: true }, description: "The tier-2 semantic-review queue (#232): every declared/earned " + @@ -467,11 +518,14 @@ export const tier2Tools: ToolDefinition[] = [ }, required: ["items", "total"], }, + summarize: summarizeTier2Queue, + docLinks: docLinksTier2Queue, handler: (vaultRoot, args, access) => vaultTier2Queue(vaultRoot, args, access), }, { name: "vault_tier2_verdict", title: "Record a tier-2 semantic verdict", + oneLine: "Record a tier-2 semantic-review verdict for a dependent pair.", annotations: { readOnlyHint: false, idempotentHint: false }, description: "Record the answer to a vault_tier2_queue item (#232 tier 2). " + @@ -529,6 +583,8 @@ export const tier2Tools: ToolDefinition[] = [ }, required: ["recorded", "tension_id"], }, + summarize: summarizeTier2Verdict, + docLinks: docLinksTier2Verdict, handler: (vaultRoot, args, access) => vaultTier2Verdict(vaultRoot, args, access), }, ]; diff --git a/src/tools/witness.ts b/src/tools/witness.ts index c262b27e..0b9d04db 100644 --- a/src/tools/witness.ts +++ b/src/tools/witness.ts @@ -77,8 +77,18 @@ const principalRecordSchema: Record = { rejected: { type: "integer" }, expired: { type: "integer" }, pending: { type: "integer" }, + edited: { + type: "integer", + description: + "Ratified via edit-then-approve — a subset of 'ratified', not additional to it", + }, + byCategory: { + type: "object", + additionalProperties: { type: "integer" }, + description: "Decided (ratified/rejected) proposals by reason_category", + }, }, - required: ["total", "ratified", "rejected", "expired", "pending"], + required: ["total", "ratified", "rejected", "expired", "pending", "edited", "byCategory"], additionalProperties: false, }, tensionsLogged: { type: "integer" }, @@ -135,10 +145,35 @@ const witnessOutputSchema: Record = { additionalProperties: false, }; +// --------------------------------------------------------------------------- +// Compact `content` summary (spec 2026-07-26, Decision 3, PR 1 gap closure). +// No docLinks: a principal is an identity string, not a vault document path. +// --------------------------------------------------------------------------- + +interface WitnessSingle { + principal: WitnessResult["principals"][number]; + concentration: WitnessResult["concentration"]; + flatCurveWarning: boolean; +} + +function summarizeWitness(value: unknown): string { + const v = value as WitnessResult | WitnessSingle; + if ("principal" in v) { + const p = v.principal; + return ( + `${p.principal}: ${p.writes} write(s), ${p.liveClaims} live claim(s), ` + + `balance ${p.balance.toFixed(1)}` + ); + } + const flat = v.flatCurveWarning ? " — flat-curve warning: one principal dominates" : ""; + return `${v.principals.length} principal(s), ${v.unattributedDocs} unattributed doc(s)${flat}`; +} + export const witnessTools: ToolDefinition[] = [ { name: "vault_witness", title: "Per-principal track records", + oneLine: "Report per-principal track records of proposals and writes.", annotations: { readOnlyHint: true }, description: "Per-principal track records aggregated from the vault's own ledgers " + @@ -168,6 +203,7 @@ export const witnessTools: ToolDefinition[] = [ additionalProperties: false, }, outputSchema: witnessOutputSchema, + summarize: summarizeWitness, handler: (vaultRoot, args, access) => vaultWitness(vaultRoot, args, access), }, ]; diff --git a/src/tools/write.ts b/src/tools/write.ts index 407d5062..a0eeb342 100644 --- a/src/tools/write.ts +++ b/src/tools/write.ts @@ -330,6 +330,11 @@ export interface WriteResult { conflicts_with?: string[]; tension_id?: string | null; tension_error?: string; + // vault_merge only: the two source documents folded into `path`, canonical + // relPaths. Exists so vault_merge's docLinks summarizer can name the + // sources without re-deriving them from anything outside the result value + // (the docLinks hard rule: only paths literally present in the value). + sources?: string[]; } // First 12 chars of a 64-char SHA-256, for human-readable provenance reasons. @@ -1898,6 +1903,7 @@ export async function vaultMerge( validation: targetReport, indexUpdated: false, shadow: true, + sources: [resolvedA.value.relPath, resolvedB.value.relPath], }); } @@ -1975,6 +1981,7 @@ export async function vaultMerge( updated: stampedTarget.updated, validation: targetReport, indexUpdated: allIndexed, + sources: [resolvedA.value.relPath, resolvedB.value.relPath], }); } catch (e) { const reason = e instanceof Error ? e.message : String(e); @@ -2261,10 +2268,52 @@ function writeResultSchema(opts: { }; } +// --------------------------------------------------------------------------- +// Compact `content` summaries + resource links (spec 2026-07-26, Decision 3, +// PR 1 gap closure). One shared pair — every write tool's ok-value is a +// WriteResult, and the shape they need to report is the same: what +// happened, to which document, and any advisory note that rode along. +// --------------------------------------------------------------------------- + +function summarizeWrite(value: unknown): string { + const r = value as WriteResult; + const commitPart = r.commit + ? ` (${r.commit})` + : r.shadow + ? " (shadow — nothing written)" + : r.committed + ? "" + : " (not committed)"; + const lines = [`${r.action}: ${r.path}${commitPart}`]; + if (r.action === "staged") { + const conflicts = + r.conflicts_with && r.conflicts_with.length > 0 + ? `; conflicts with ${r.conflicts_with.length}: ${r.conflicts_with.join(", ")}` + : "; uncontested"; + lines.push(`staged as ${r.staged_id}, expires ${r.expires_at}${conflicts}`); + if (r.tension_id) lines.push(`tension: ${r.tension_id}`); + if (r.tension_error) lines.push(`tension error: ${r.tension_error}`); + } + if (r.hint) lines.push(`hint: ${r.hint}`); + if (r.supersede_hint) lines.push(`hint: ${r.supersede_hint}`); + if (r.domain_warnings && r.domain_warnings.length > 0) { + lines.push(`domain warnings: ${r.domain_warnings.join("; ")}`); + } + return lines.join("\n"); +} + +// The written path, plus (vault_merge only) the two sources it folded in — +// both already RBAC-gated by the handler before the write landed. +function docLinksWrite(value: unknown): string[] { + const r = value as WriteResult; + return r.sources && r.sources.length > 0 ? [r.path, ...r.sources] : [r.path]; +} + export const writeTools: ToolDefinition[] = [ { name: "vault_write", title: "Create or update a document", + oneLine: "Create or update a vault document.", annotations: { destructiveHint: true }, description: "Create a new vault document or overwrite an existing one. Supply the " + @@ -2353,11 +2402,14 @@ export const writeTools: ToolDefinition[] = [ }, }, }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultWrite(vaultRoot, args, access), }, { name: "vault_append", title: "Append to a document", + oneLine: "Append markdown text to a document's body.", annotations: { destructiveHint: true }, description: "Append a markdown section to an existing vault document. Frontmatter " + @@ -2391,11 +2443,14 @@ export const writeTools: ToolDefinition[] = [ actions: ["append"], properties: { domain_warnings: domainWarningsProperty }, }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultAppend(vaultRoot, args, access), }, { name: "vault_promote", title: "Promote draft to canonical", + oneLine: "Promote a draft document to canonical.", annotations: { destructiveHint: true }, description: "Promote a draft document to canonical status. Refuses unless the " + @@ -2420,11 +2475,14 @@ export const writeTools: ToolDefinition[] = [ actions: ["promote"], statuses: ["canonical"], }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultPromote(vaultRoot, args, access), }, { name: "vault_deprecate", title: "Deprecate a document", + oneLine: "Deprecate a document, optionally pointing at its replacement.", annotations: { destructiveHint: true }, description: "Mark a document deprecated. A reason is required; optionally record " + @@ -2457,11 +2515,14 @@ export const writeTools: ToolDefinition[] = [ actions: ["deprecate"], statuses: ["deprecated"], }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultDeprecate(vaultRoot, args, access), }, { name: "vault_set_confidence", title: "Set a document's confidence", + oneLine: "Set a document's confidence level.", annotations: { destructiveHint: true }, description: "Change only a document's confidence level (low | medium | high), leaving " + @@ -2496,11 +2557,14 @@ export const writeTools: ToolDefinition[] = [ "document's confidence and the 'updated' / 'updated_by' stamps moved.", actions: ["confidence-set"], }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultSetConfidence(vaultRoot, args, access), }, { name: "vault_set_tier", title: "Set a document's write-protection tier", + oneLine: "Set a document's write-protection tier.", annotations: { destructiveHint: true }, description: "Change only a document's write-protection tier (source | compiled | " + @@ -2540,11 +2604,14 @@ export const writeTools: ToolDefinition[] = [ "write-protection tier and the 'updated' / 'updated_by' stamps moved.", actions: ["tier-set"], }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultSetTier(vaultRoot, args, access), }, { name: "vault_supersede", title: "Supersede a document", + oneLine: "Mark a document as superseded by another.", annotations: { destructiveHint: true }, description: "Mark a document superseded by a named successor. Sets status=superseded " + @@ -2581,11 +2648,14 @@ export const writeTools: ToolDefinition[] = [ actions: ["supersede"], statuses: ["superseded"], }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultSupersede(vaultRoot, args, access), }, { name: "vault_merge", title: "Merge two documents into one", + oneLine: "Merge two documents into one, pointing both at a successor.", annotations: { destructiveHint: true }, description: "Combine two source documents into a target and supersede both sources to " + @@ -2629,11 +2699,21 @@ export const writeTools: ToolDefinition[] = [ outputSchema: writeResultSchema({ description: "The applied merge, reported for the TARGET document (path is " + - "target_path); the superseded sources are written under the same " + - "commit and are not enumerated here. 'commit' covers all three files, " + - "and 'indexUpdated' is true only when every written file reindexed.", + "target_path). 'commit' covers all three files, and 'indexUpdated' " + + "is true only when every written file reindexed.", actions: ["merge"], + properties: { + sources: { + type: "array", + items: { type: "string" }, + description: + "Canonical vault-relative paths of the two source documents " + + "superseded into the target, written under the same commit.", + }, + }, }), + summarize: summarizeWrite, + docLinks: docLinksWrite, handler: (vaultRoot, args, access) => vaultMerge(vaultRoot, args, access), }, ]; diff --git a/src/utils/config.ts b/src/utils/config.ts index d734dcfc..86931698 100644 --- a/src/utils/config.ts +++ b/src/utils/config.ts @@ -39,6 +39,17 @@ export interface RoleConfig { // permission layer, not convention. YAML key: propose_only. Optional so // existing configs (and role literals) are unchanged; absent means false. proposeOnly?: boolean; + // Citation-anchors gate (2026-07-26 spec, "Decision from Mihir 2026-07-27"): + // a role sees vault_read's `anchors` annotation only when it ALSO carries + // this grant — read access to the pinned doc's collection is necessary but + // not sufficient. Distinct from `read` because the annotation discloses + // facts about a code repo (existence, blob-match, relocated line numbers) + // that RBAC's collection model has no vocabulary for. YAML key: + // code_repo_visibility. Optional; absent means false — off by default for + // every non-operator role. A caller with no AccessContext at all (direct + // in-process call, stdio without --role) is unaffected by this gate, the + // same posture every other RBAC check in this file takes. + codeRepoVisibility?: boolean; } // The primitive types a schema-extension field may declare. `array` is v1 @@ -62,9 +73,51 @@ export interface SchemaExtension { // the runtime instantiates the matching backend (see search/vector.ts // getProvider). Adding a third provider would mean a new id here AND a new // branch in getProvider AND a config-load check if it needs env vars. -export const EMBEDDING_PROVIDERS = ["local-minilm", "openai-3-small"] as const; +// +// local-embeddinggemma / local-qwen3-0.6b added by the 2026-07-26 embedding- +// refresh-quantization spec (Decision 1): Matryoshka-truncatable, fully +// local providers run via @huggingface/transformers, same posture as +// local-minilm. Neither is the programmatic default yet (loadConfig's +// fallback stays local-minilm — the flip is gated on the spec's Phase 5 +// recall-bench, not landed with this PR). +export const EMBEDDING_PROVIDERS = [ + "local-minilm", + "openai-3-small", + "local-embeddinggemma", + "local-qwen3-0.6b", +] as const; export type EmbeddingProviderId = (typeof EMBEDDING_PROVIDERS)[number]; +// Allowed `embeddings.dim` values per provider. `null` means the provider is +// fixed-dim and `dim` must not be set for it (a hard error, same posture as +// an unknown provider id) — local-minilm (384) and openai-3-small (1536) +// have no Matryoshka truncation to configure. The two new providers expose +// 512 (default) and 768 (full, untruncated). +export const EMBEDDING_DIMS: Record = { + "local-minilm": null, + "openai-3-small": null, + "local-embeddinggemma": [512, 768], + "local-qwen3-0.6b": [512, 768], +}; + +// Recognised values of `embeddings.quantize`. "int8" is the vec-INDEX +// representation only — the durable `embeddings` cache always stays +// float32, native-dim (spec Decision 3/9). Default is "int8" for the two new +// providers and "none" for local-minilm/openai-3-small (existing vaults stay +// bit-identical unless the operator opts in). "int8" is accepted for ANY +// provider — every provider L2-normalizes, so quantization is calibration- +// free unit-range scaling regardless of which model produced the vector. +export const EMBEDDING_QUANTIZE_VALUES = ["int8", "none"] as const; +export type EmbeddingQuantize = (typeof EMBEDDING_QUANTIZE_VALUES)[number]; + +// Recognised values of `rerank.provider` (spec 2026-07-26-contextual- +// chunking-reranker-design.md Decision 5). "none" (the default) means no +// rerank stage at all — getRerankProvider() returns null, callers branch on +// presence rather than a null-object provider. Adding a third provider means +// a new id here AND a new branch in rerank-provider.ts's instantiateProvider. +export const RERANK_PROVIDERS = ["none", "local-bge-m3"] as const; +export type RerankProviderId = (typeof RERANK_PROVIDERS)[number]; + // Budgets and attribution for the sleep tension-scan dream (`daftari sleep // --dream tension-scan`). All values are HARD requirements on the pass: // `maxLlmCalls` caps pairwise judgments per pass (the real spend bound), @@ -103,6 +156,19 @@ export const TOOLS_DEFAULTS: ToolsConfig = { exclude: [], }; +// `search` block (spec 2026-07-26 fusion overhaul, Decision 2). Library- +// level fusion options (RRF vs weighted) never get a config knob — only the +// query router's opt-in does, because unlike fusion mode, routing changes +// what weights a query gets based on its own shape, which an operator may +// reasonably want to keep off even after the fusion default flips. +export interface SearchConfig { + routing: boolean; +} + +// Off by default; flipped to true only after the fusion-runner.mjs bench's +// gates.routingFlip passes (PR 3 of the fusion overhaul). +export const SEARCH_DEFAULTS: SearchConfig = { routing: false }; + // `server` block (#5, spec 2026-07-20): configuration for `daftari serve`. // Token VALUES never live here — .daftari/config.yaml sits inside the vault's // git repo, so each entry names the ENV VAR that carries the secret. An @@ -208,6 +274,21 @@ export interface DaftariConfig { // providers preserves both side's rows — the new provider populates a // fresh row set on first reindex, and switching back reuses the old. embeddingProvider: EmbeddingProviderId; + // `embeddings.dim` — optional Matryoshka truncation target. null when + // absent (the provider's own default applies); validated against + // EMBEDDING_DIMS for the active provider (spec Decision 2). + embeddingDim: number | null; + // `embeddings.quantize` — vec-index representation (spec Decision 3). + // Defaults to "int8" for the two new local providers, "none" otherwise; + // see EMBEDDING_QUANTIZE_VALUES. + embeddingQuantize: EmbeddingQuantize; + // Local cross-encoder reranker selection (`rerank` block, spec 2026-07-26- + // contextual-chunking-reranker-design.md Decision 5). Defaults to "none" — + // opt-in, unlike embeddings: the rerank stage's q8 weights are a much + // heavier default-install cost, and the project's own playbook (chunk-level + // BM25) is to ship a lever behind an option, measure it, then flip the + // default on evidence. + rerankProvider: RerankProviderId; // Optional git-author → identity mapping consumed by `daftari backfill` // (§11.1) when deriving the `updated_by` frontmatter field from a doc's git // history. Keys are raw git author names (`%aN`); values are Daftari @@ -225,6 +306,13 @@ export interface DaftariConfig { // The consolidate loop refuses live writes (mode != scan) unless the operator // has made an explicit choice, so a surprising default can't spend or mutate. shadowModeSet: boolean; + // Independence-aware promotion graduation gate (2026-07-26 spec, Decision + // 4). Defaults to false — the shipped shadow posture: the revision loop + // computes and journals the would-be verdict but the live two-way decision + // is untouched. Opt-in only, unlike shadow_mode: absence is a valid false, + // no explicit-declaration tripwire, because the ungraduated behavior IS + // today's behavior (nothing new to silently start doing). + independenceGraduated: boolean; // Absolute path to an external git directory (git's --separate-git-dir), or // undefined for a normal in-vault .git. Lets a cloud-synced vault hold only a // static `.git` file while git's churn lives off-cloud. Always resolved @@ -246,6 +334,21 @@ export interface DaftariConfig { // with no backing configured; `daftari sync` then refuses with a pointer // to the config block. storage?: StorageConfig; + // Query router opt-in (`search` block, spec 2026-07-26 fusion overhaul + // Decision 2). Always populated — routing off when the block is absent. + search: SearchConfig; + // `code_repos` (2026-07-26 citation-anchors-jit spec, Decision 2): name -> + // absolute path, ~ expanded and resolved against the vault root. This is + // the READ-PATH repo registry — distinct from the audit's own `--code-repo` + // / audit.yaml `repos:` declarations, which are per-invocation and not + // visible here. Path EXISTENCE is deliberately not checked at load + // (config travels with a synced vault onto machines where the code repo + // may simply not be checked out); an absent repo degrades silently to + // `anchors: null` at read time. Empty when the block is absent. + codeRepos: Record; + // `jit_anchors` (same spec): kill-switch for the read-path pin check. + // Defaults to true; false removes the entire code path (zero git work). + jitAnchors: boolean; } // A config with no roles and no extensions. Returned for a missing or empty @@ -259,14 +362,21 @@ function emptyConfig(): DaftariConfig { watch: true, warmEmbeddings: true, embeddingProvider: "local-minilm", + embeddingDim: null, + embeddingQuantize: "none", + rerankProvider: "none", backfillIdentityMap: {}, shadowMode: false, shadowModeSet: false, + independenceGraduated: false, gitDir: undefined, tensionScan: { ...TENSION_SCAN_DEFAULTS }, tools: { ...TOOLS_DEFAULTS, include: [], exclude: [] }, server: { tokens: [] }, storage: undefined, + search: { ...SEARCH_DEFAULTS }, + codeRepos: {}, + jitAnchors: true, }; } @@ -322,6 +432,14 @@ function validateRole(name: string, raw: unknown): Result { proposeOnly = obj.propose_only; } + let codeRepoVisibility = false; + if (obj.code_repo_visibility !== undefined) { + if (typeof obj.code_repo_visibility !== "boolean") { + return err(new Error(`role '${name}' code_repo_visibility must be true or false`)); + } + codeRepoVisibility = obj.code_repo_visibility; + } + // Contradictory grants fail loud at load: a propose-only role proposes, it // does not decide. Allowing both would let vault_ratify's write dispatch be // coerced back into a NEW proposal while marking the original ratified. @@ -348,6 +466,7 @@ function validateRole(name: string, raw: unknown): Result { promote, ratify, ...(proposeOnly ? { proposeOnly } : {}), + ...(codeRepoVisibility ? { codeRepoVisibility } : {}), }); } @@ -898,6 +1017,69 @@ function validateTools(raw: unknown): Result { return ok({ tier, include: include.value, exclude: exclude.value }); } +const RECOGNISED_SEARCH_KEYS = ["routing"] as const; + +// `routing` accepts booleans AND the strings "on"/"off" — js-yaml 4's +// YAML-1.2 core schema loads a bare `off`/`on` as the STRING "off"/"on", not +// a boolean, so a config author writing the natural `search.routing: off` +// (the spec's own example) must not hit a type error. +function validateSearch(raw: unknown): Result { + if (raw === undefined) return ok({ ...SEARCH_DEFAULTS }); + const mapping = requireMapping(raw, "'search'"); + if (!mapping.ok) return mapping; + const obj = mapping.value; + const known = rejectUnknownKeys(obj, RECOGNISED_SEARCH_KEYS, "search"); + if (!known.ok) return known; + + let routing: boolean = SEARCH_DEFAULTS.routing; + if (obj.routing !== undefined) { + if (typeof obj.routing === "boolean") { + routing = obj.routing; + } else if (obj.routing === "on" || obj.routing === "off") { + routing = obj.routing === "on"; + } else { + return err( + new Error( + `'search.routing' must be on/off or true/false (got ${JSON.stringify(obj.routing)})`, + ), + ); + } + } + return ok({ routing }); +} + +// `code_repos` (2026-07-26 citation-anchors-jit spec, Decision 2). A mapping +// of non-empty name -> non-empty path string. Names containing `:` are +// rejected at load — the read path resolves a `describes` binding's `repo:` +// prefix by exact-name lookup into this map, and a colon in the name itself +// would make that lookup ambiguous against the grammar's own delimiter. +// Shape-only, like every block: path EXISTENCE is intentionally unchecked +// here (see the field's doc comment on DaftariConfig). +function validateCodeRepos(raw: unknown, vaultRoot: string): Result, Error> { + if (raw === undefined) return ok({}); + const mapping = requireMapping(raw, "'code_repos'"); + if (!mapping.ok) return mapping; + const out: Record = Object.create(null); + for (const [name, value] of Object.entries(mapping.value)) { + if (name.length === 0) { + return err(new Error("'code_repos' keys must be non-empty")); + } + if (name.includes(":")) { + return err( + new Error( + `'code_repos.${name}': repo names must not contain ':' ` + + "(it collides with the describes grammar's own repo-prefix delimiter)", + ), + ); + } + if (typeof value !== "string" || value.trim().length === 0) { + return err(new Error(`'code_repos.${name}' must be a non-empty string`)); + } + out[name] = resolve(vaultRoot, expandTilde(value.trim())); + } + return ok(out); +} + function dataHome(): string { const xdg = process.env.XDG_DATA_HOME; return xdg && xdg.length > 0 ? xdg : join(homedir(), ".local", "share"); @@ -1105,6 +1287,22 @@ function loadConfigUncached(vaultRoot: string): Result { const storageConfig = validateStorage(root.storage); if (!storageConfig.ok) return err(new Error(`malformed config: ${storageConfig.error.message}`)); + const searchConfig = validateSearch(root.search); + if (!searchConfig.ok) return err(new Error(`malformed config: ${searchConfig.error.message}`)); + + const codeReposConfig = validateCodeRepos(root.code_repos, vaultRoot); + if (!codeReposConfig.ok) { + return err(new Error(`malformed config: ${codeReposConfig.error.message}`)); + } + + let jitAnchors = true; + if (root.jit_anchors !== undefined) { + if (typeof root.jit_anchors !== "boolean") { + return err(new Error("malformed config: 'jit_anchors' must be true or false")); + } + jitAnchors = root.jit_anchors; + } + let watch = true; if (root.watch !== undefined) { if (typeof root.watch !== "boolean") { @@ -1130,13 +1328,30 @@ function loadConfigUncached(vaultRoot: string): Result { shadowMode = root.shadow_mode; } + // Independence-aware promotion graduation gate (2026-07-26 spec, Decision + // 4). Opt-in; absence is a valid false (unlike shadow_mode, no explicit- + // declaration requirement — the default behavior is today's behavior). + let independenceGraduated = false; + if (root.independence_graduated !== undefined) { + if (typeof root.independence_graduated !== "boolean") { + return err(new Error("malformed config: 'independence_graduated' must be true or false")); + } + independenceGraduated = root.independence_graduated; + } + // Embedding provider selection. Defaults to local-minilm. Unknown ids fail // loud — the trust model is "vault owner configures the server" so a typo // is a config error, not a fall-through to default. The OPENAI_API_KEY // check happens here too: a paid provider with no key in env can't quietly // degrade to lexical-only after every search; the vault owner needs to // know at startup that the key is missing. + const RECOGNISED_EMBEDDINGS_KEYS = ["provider", "dim", "quantize"] as const; let embeddingProvider: EmbeddingProviderId = "local-minilm"; + let embeddingDim: number | null = null; + // Default quantize is per-provider (int8 for the two new local providers, + // none for local-minilm/openai-3-small) — resolved AFTER the provider is + // known, below. + let embeddingQuantizeRaw: EmbeddingQuantize | undefined; if (root.embeddings !== undefined) { if ( root.embeddings === null || @@ -1146,6 +1361,8 @@ function loadConfigUncached(vaultRoot: string): Result { return err(new Error("malformed config: 'embeddings' must be a mapping")); } const block = root.embeddings as Record; + const known = rejectUnknownKeys(block, RECOGNISED_EMBEDDINGS_KEYS, "embeddings"); + if (!known.ok) return err(new Error(`malformed config: ${known.error.message}`)); if (block.provider !== undefined) { if (typeof block.provider !== "string") { return err(new Error("malformed config: 'embeddings.provider' must be a string")); @@ -1160,6 +1377,26 @@ function loadConfigUncached(vaultRoot: string): Result { } embeddingProvider = block.provider as EmbeddingProviderId; } + if (block.dim !== undefined) { + if (typeof block.dim !== "number" || !Number.isInteger(block.dim) || block.dim <= 0) { + return err(new Error("malformed config: 'embeddings.dim' must be a positive integer")); + } + embeddingDim = block.dim; + } + if (block.quantize !== undefined) { + if ( + typeof block.quantize !== "string" || + !(EMBEDDING_QUANTIZE_VALUES as readonly string[]).includes(block.quantize) + ) { + return err( + new Error( + `malformed config: 'embeddings.quantize' must be one of ` + + `${EMBEDDING_QUANTIZE_VALUES.join(", ")} (got ${JSON.stringify(block.quantize)})`, + ), + ); + } + embeddingQuantizeRaw = block.quantize as EmbeddingQuantize; + } } if (embeddingProvider === "openai-3-small" && !process.env.OPENAI_API_KEY) { return err( @@ -1169,6 +1406,73 @@ function loadConfigUncached(vaultRoot: string): Result { ); } + // `dim` validation against the active provider's allowed set (spec + // Decision 2). A fixed-dim provider (EMBEDDING_DIMS[id] === null) rejects + // `dim` outright — setting it would silently disagree with the id's real + // (only) dimension. A Matryoshka provider with `dim` unset falls back to + // its first (default) allowed value. + const allowedDims = EMBEDDING_DIMS[embeddingProvider]; + if (allowedDims === null) { + if (embeddingDim !== null) { + return err( + new Error( + `malformed config: 'embeddings.dim' is not accepted for provider '${embeddingProvider}' ` + + "(it has no configurable dimension)", + ), + ); + } + } else { + if (embeddingDim === null) { + embeddingDim = allowedDims[0] as number; + } else if (!(allowedDims as readonly number[]).includes(embeddingDim)) { + return err( + new Error( + `malformed config: 'embeddings.dim' ${embeddingDim} is not valid for provider ` + + `'${embeddingProvider}' (expected one of ${allowedDims.join(", ")})`, + ), + ); + } + } + + // `quantize` default: "int8" for the two new local providers, "none" + // otherwise (existing vaults stay bit-identical unless the operator opts + // in). Explicit config value always wins. + const embeddingQuantize: EmbeddingQuantize = + embeddingQuantizeRaw ?? + (embeddingProvider === "local-embeddinggemma" || embeddingProvider === "local-qwen3-0.6b" + ? "int8" + : "none"); + + // Rerank provider selection (spec 2026-07-26-contextual-chunking-reranker- + // design.md Decision 5). Defaults to "none" — opt-in, since local-bge-m3's + // q8 weights are an order of magnitude past local-minilm's on-disk/RAM + // footprint and the default install must stay light. Same posture as + // embeddings.provider above: an unknown id fails loud rather than silently + // falling back to "none" — a typo that meant to enable reranking must never + // silently no-op. No env-var check: local-bge-m3 is a fully local model, + // no API key to validate. + let rerankProvider: RerankProviderId = "none"; + if (root.rerank !== undefined) { + if (root.rerank === null || typeof root.rerank !== "object" || Array.isArray(root.rerank)) { + return err(new Error("malformed config: 'rerank' must be a mapping")); + } + const block = root.rerank as Record; + if (block.provider !== undefined) { + if (typeof block.provider !== "string") { + return err(new Error("malformed config: 'rerank.provider' must be a string")); + } + if (!(RERANK_PROVIDERS as readonly string[]).includes(block.provider)) { + return err( + new Error( + `malformed config: unknown rerank.provider ${JSON.stringify(block.provider)} ` + + `(expected one of ${RERANK_PROVIDERS.join(", ")})`, + ), + ); + } + rerankProvider = block.provider as RerankProviderId; + } + } + return ok({ roles, schemaExtensions: extensions.value, @@ -1177,13 +1481,20 @@ function loadConfigUncached(vaultRoot: string): Result { watch, warmEmbeddings, embeddingProvider, + embeddingDim, + embeddingQuantize, + rerankProvider, backfillIdentityMap: backfillIdentityMap.value, shadowMode, shadowModeSet, + independenceGraduated, gitDir: gitDir.value, tensionScan: tensionScan.value, tools: toolsConfig.value, server: serverConfig.value, storage: storageConfig.value, + search: searchConfig.value, + codeRepos: codeReposConfig.value, + jitAnchors, }); } diff --git a/src/utils/git.ts b/src/utils/git.ts index 39b872c2..d17b772c 100644 --- a/src/utils/git.ts +++ b/src/utils/git.ts @@ -221,6 +221,77 @@ export async function log( return ok(commits); } +// --- citation-anchor plumbing (2026-07-26 spec, Decisions 1-2, C1) -------- +// +// The read-path latency budget (kill condition: <=50ms p95 even at the pin +// cap) rules out one `execFile` per pin. Batching `hash-object` into ONE +// invocation per repo — regardless of how many pins reference it — is what +// keeps the all-intact (common) case to one subprocess per referenced repo +// per read/lint-run/audit, instead of one per pin. blobAtHead/blobSize/ +// catBlob stay per-call: they only run on the drift (cold) path, where the +// spec knowingly exceeds its own "two invocations per pin" budget by one +// `cat-file -s` size gate ahead of the (bounded) blob read. + +// Batches `git hash-object` for many working-tree files in ONE invocation, +// mapped back to `relPaths` by position. Callers `fs.stat` (or otherwise +// confirm existence of) every path first — hash-object fails the WHOLE batch +// on a single missing file, so a missing candidate must never reach here. +export async function hashObjects( + repoRoot: string, + relPaths: string[], +): Promise> { + if (relPaths.length === 0) return ok([]); + const result = await git(repoRoot, ["hash-object", "--", ...relPaths]); + if (!result.ok) return result; + const lines = result.value.split("\n").filter((l) => l.length > 0); + if (lines.length !== relPaths.length) { + return err( + new Error(`git hash-object: expected ${relPaths.length} blob id(s), got ${lines.length}`), + ); + } + return ok(lines); +} + +// Single-path wrapper over the batch primitive above. +export async function hashObject( + repoRoot: string, + relPath: string, +): Promise> { + const batch = await hashObjects(repoRoot, [relPath]); + if (!batch.ok) return batch; + return ok(batch.value[0] as string); +} + +// The blob id `relPath` had at HEAD — the committed blob `daftari audit +// --pin` always pins (never the working tree), because a committed blob +// stays retrievable from the object database. +export async function blobAtHead( + repoRoot: string, + relPath: string, +): Promise> { + const result = await git(repoRoot, ["rev-parse", `HEAD:${relPath}`]); + if (!result.ok) return result; + return ok(result.value.trim()); +} + +// Size (bytes) of a blob, checked BEFORE catBlob so a huge pinned blob is +// never pulled into memory — the same stat-before-read guard readtext.ts +// uses for working-tree files. +export async function blobSize(repoRoot: string, sha: string): Promise> { + const result = await git(repoRoot, ["cat-file", "-s", sha]); + if (!result.ok) return result; + const n = Number.parseInt(result.value.trim(), 10); + if (!Number.isFinite(n)) return err(new Error(`git cat-file -s ${sha}: unparseable size`)); + return ok(n); +} + +// Retrieves a blob's raw content. Callers gate on blobSize first (the size +// cap is a caller concern, not enforced here); git()'s 16 MiB maxBuffer is +// the backstop. +export async function catBlob(repoRoot: string, sha: string): Promise> { + return git(repoRoot, ["cat-file", "blob", sha]); +} + // Vault-relative .md paths changed between `sinceCommit` and HEAD. Used by the // consolidate event clock (spec §3.1). A bad/unknown commit is an error, not [] — // the caller treats that as the nil baseline path (skip the event clock), so the diff --git a/src/utils/paths.ts b/src/utils/paths.ts index b598bd6e..6c28e7ba 100644 --- a/src/utils/paths.ts +++ b/src/utils/paths.ts @@ -1,7 +1,9 @@ // paths — lexical vault-relative path canonicalization shared by the search -// and curation layers. +// and curation layers, plus realpath-based filesystem confinement shared by +// the audit and citation-anchor classifiers. -import { posix } from "node:path"; +import { realpathSync } from "node:fs"; +import { isAbsolute, relative as nodeRelative, posix } from "node:path"; // Lexical, IO-free canonicalization of a vault-relative path: aliasing // (`pricing/../pricing/a.md`) must join its canonical hit (#127/#128 class). @@ -14,3 +16,29 @@ export function canonicalRel(p: string): string { const n = posix.normalize(p.trim().replace(/\\/g, "/")); return n === "." ? "" : n.replace(/^\.\//, ""); } + +// Realpath-based filesystem confinement: true iff targetAbs exists AND its +// REAL location sits under rootAbs. realpathSync resolves every path +// component, so a symlink committed inside an audited/referenced tree +// (escape -> /) cannot route the probe outside the containment root — a +// lexical check plus a bare existsSync would (security review on #255). +// rootAbs is expected to be already-real: repo roots are realpathSync'd at +// config load, and the parent prefix of a real path is itself real. +// A nonexistent target makes realpathSync throw ENOENT -> false, which is +// exactly the "missing" answer. +// +// Originally `src/audit/collect.ts#symlinkSafeExistsWithin`; lifted here +// (2026-07-26 citation-anchors-jit spec, C5) so the anchor classifier +// (src/anchors/classify.ts) shares the same confinement primitive instead of +// growing a second, lexical-only implementation. Re-exported from +// collect.ts for existing callers. +export function symlinkSafeExistsWithin(rootAbs: string, targetAbs: string): boolean { + let real: string; + try { + real = realpathSync(targetAbs); + } catch { + return false; + } + const rel = nodeRelative(rootAbs, real); + return rel === "" || (!rel.startsWith("..") && !isAbsolute(rel)); +} diff --git a/src/utils/vault-gitignore.ts b/src/utils/vault-gitignore.ts index 7d00fc06..3613bbb0 100644 --- a/src/utils/vault-gitignore.ts +++ b/src/utils/vault-gitignore.ts @@ -29,6 +29,11 @@ export const VAULT_GITIGNORE = `# Daftari rebuilds these from the markdown files .daftari/tier2-verdicts.jsonl .daftari/shadow-actions.jsonl .daftari/consolidate-state.json +# Cortex loop traces + the independence-aware-promotion shadow calibration +# journal (2026-07-26 spec) — advisory calibration data, same posture as +# shadow-actions.jsonl above. +.daftari/revision-trace.jsonl +.daftari/independence-shadow.jsonl # Transient backfill staging surface (daftari backfill --plan). The apply # commit is the durable audit trail — the plan itself is never committed. .daftari/backfill-plan.jsonl @@ -42,8 +47,11 @@ const MARKER = ".daftari/index.db"; // Ensures the vault's .gitignore carries the Daftari ignore block. Idempotent: // "created" — no .gitignore existed; wrote VAULT_GITIGNORE. -// "appended" — a .gitignore existed without the block; appended it. -// "present" — the block was already there; left the file untouched. +// "appended" — a .gitignore existed without the block, OR carried the block +// but was missing pattern lines added to the constant since +// (C7, 2026-07-26 spec): appended what was missing. +// "present" — every `.daftari/` pattern line was already there; left the +// file untouched. export async function ensureVaultGitignore( vaultRoot: string, ): Promise<"created" | "appended" | "present"> { @@ -60,11 +68,25 @@ export async function ensureVaultGitignore( throw e; } - if (existing.includes(MARKER)) return "present"; + if (!existing.includes(MARKER)) { + // No Daftari block at all yet — separate the user's content from ours so + // it doesn't glue onto their last line. A leading "\n" guarantees a clean + // break whether or not the file ends in a newline. + await appendFile(path, `\n${VAULT_GITIGNORE}`); + return "appended"; + } + + // The block's marker is present, but the constant may have grown new + // `.daftari/` pattern lines since this vault's .gitignore was first + // scaffolded (or last reconciled) — a retrofit gap the block-marker check + // alone can't see (C7). Per-line reconciliation: diff the constant's + // pattern lines against what the file already has, and append only what's + // missing. + const existingLines = new Set(existing.split(/\r?\n/).map((l) => l.trim())); + const patternLines = VAULT_GITIGNORE.split("\n").filter((l) => l.startsWith(".daftari/")); + const missing = patternLines.filter((l) => !existingLines.has(l)); + if (missing.length === 0) return "present"; - // Separate the user's content from our block so it doesn't glue onto their - // last line. A leading "\n" guarantees a clean break whether or not the file - // ends in a newline. - await appendFile(path, `\n${VAULT_GITIGNORE}`); + await appendFile(path, `\n${missing.join("\n")}\n`); return "appended"; } diff --git a/src/witness/track-record.ts b/src/witness/track-record.ts index 5b6cf5d9..4aa0a3f0 100644 --- a/src/witness/track-record.ts +++ b/src/witness/track-record.ts @@ -22,7 +22,7 @@ import { type AccessContext, canRead } from "../access/rbac.js"; import { readProvenanceLog } from "../curation/provenance.js"; -import { listStagedActions } from "../curation/staged-actions.js"; +import { listStagedActions, proposalTallies } from "../curation/staged-actions.js"; import { ageInDays, computeStaleness } from "../curation/staleness.js"; import { listTensions } from "../curation/tension.js"; import { loadDocuments } from "../curation/vault-docs.js"; @@ -63,13 +63,19 @@ export interface PrincipalRecord { survived: number; // authored docs maintained through ≥1 full TTL cycle, still canonical creditEarned: number; balance: number; // creditEarned − burnedStake (advisory; provisional constants) - // Proposal record (staged actions). + // Proposal record (staged actions). `edited` and `byCategory` are the + // 2026-07-26 risk-triaged-ratification spec's Decision 3 additions — sourced + // from the same proposalTallies implementation the risk scorer's W term + // reads, keyed on the authenticated stager (C4 disposition) so the tallies + // cannot be laundered by rotating the unauthenticated proposed_by string. proposals: { total: number; ratified: number; rejected: number; expired: number; pending: number; + edited: number; + byCategory: Record; }; tensionsLogged: number; } @@ -151,7 +157,15 @@ export async function buildWitness( survived: 0, creditEarned: 0, balance: 0, - proposals: { total: 0, ratified: 0, rejected: 0, expired: 0, pending: 0 }, + proposals: { + total: 0, + ratified: 0, + rejected: 0, + expired: 0, + pending: 0, + edited: 0, + byCategory: {}, + }, tensionsLogged: 0, }; records.set(principal, r); @@ -203,13 +217,22 @@ export async function buildWitness( } } - for (const a of visibleActions) { - const r = recordFor(a.proposedBy); - r.proposals.total += 1; - if (a.status === "ratified" || a.status === "ratified-pending-tool") r.proposals.ratified += 1; - else if (a.status === "rejected") r.proposals.rejected += 1; - else if (a.status === "expired") r.proposals.expired += 1; - else r.proposals.pending += 1; + // proposalTallies keys on stagedByPrincipal ?? proposedBy (C4) — the + // authenticated identity that staged the proposal when the record has one, + // the caller-claimed proposedBy string otherwise. One shared implementation + // with the risk scorer's W term (src/curation/risk.ts) so the two numbers + // can never drift apart. + for (const [principal, tallies] of proposalTallies(visibleActions)) { + const r = recordFor(principal); + r.proposals = { + total: tallies.total, + ratified: tallies.ratified, + rejected: tallies.rejected, + expired: tallies.expired, + pending: tallies.pending, + edited: tallies.edited, + byCategory: tallies.byCategory, + }; } for (const t of visibleTensions) { diff --git a/test/anchors/classify.test.ts b/test/anchors/classify.test.ts new file mode 100644 index 00000000..5e98e422 --- /dev/null +++ b/test/anchors/classify.test.ts @@ -0,0 +1,360 @@ +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + classifyAgainstHash, + classifyPin, + resolveConfinedFile, +} from "../../src/anchors/classify.js"; +import type { PinSpec } from "../../src/anchors/pin.js"; +import { hashObjects } from "../../src/utils/git.js"; + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): string { + return execFileSync("git", args, { cwd, env: GIT_ENV }).toString(); +} + +function initRepo(): string { + const dir = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-"))); + git(dir, ["init", "-q"]); + return dir; +} + +function commitAll(dir: string, message: string): void { + git(dir, ["add", "-A"]); + git(dir, ["commit", "-q", "-m", message]); +} + +function wholeFilePin(sha: string): PinSpec { + return { start: null, end: null, sha }; +} + +describe("resolveConfinedFile", () => { + let repo: string; + beforeEach(() => { + repo = initRepo(); + }); + afterEach(() => { + rmSync(repo, { recursive: true, force: true }); + }); + + it("resolves a plain file inside the repo", () => { + writeFileSync(join(repo, "a.ts"), "hello\n"); + const c = resolveConfinedFile(repo, "a.ts"); + expect(c).not.toBeNull(); + expect(c?.relPath).toBe("a.ts"); + }); + + it("returns null for a missing file", () => { + expect(resolveConfinedFile(repo, "nope.ts")).toBeNull(); + }); + + it("returns null for a symlink escaping the repo (never reads its bytes)", () => { + // `outside` is a SIBLING of the repo root under its own isolated temp + // dir, not nested inside the repo — the symlink genuinely escapes. + const outer = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-escape-"))); + try { + const nestedRepo = join(outer, "repo"); + mkdirSync(nestedRepo); + mkdirSync(join(outer, "outside")); + writeFileSync(join(outer, "outside", "secret.ts"), "secret\n"); + symlinkSync(join(outer, "outside"), join(nestedRepo, "escape")); + expect(resolveConfinedFile(nestedRepo, "escape/secret.ts")).toBeNull(); + } finally { + rmSync(outer, { recursive: true, force: true }); + } + }); + + it("returns null for a directory (must be a regular file)", () => { + mkdirSync(join(repo, "dir")); + expect(resolveConfinedFile(repo, "dir")).toBeNull(); + }); +}); + +describe("classifyPin — whole-file pins", () => { + let repo: string; + beforeEach(() => { + repo = initRepo(); + }); + afterEach(() => { + rmSync(repo, { recursive: true, force: true }); + }); + + it("intact: current hash matches the pinned sha", async () => { + writeFileSync(join(repo, "retry.ts"), "export function retry() {}\n"); + commitAll(repo, "init"); + const hashes = await hashObjects(repo, ["retry.ts"]); + expect(hashes.ok).toBe(true); + if (!hashes.ok) return; + const sha = hashes.value[0] as string; + + const result = await classifyPin(repo, "retry.ts", wholeFilePin(sha)); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toEqual({ state: "intact" }); + }); + + it("intact via a short (7-char) sha prefix", async () => { + writeFileSync(join(repo, "retry.ts"), "export function retry() {}\n"); + commitAll(repo, "init"); + const hashes = await hashObjects(repo, ["retry.ts"]); + if (!hashes.ok) throw hashes.error; + const shortSha = (hashes.value[0] as string).slice(0, 7); + + const result = await classifyPin(repo, "retry.ts", wholeFilePin(shortSha)); + expect(result.ok && result.value.state).toBe("intact"); + }); + + it("moved: file content changed since the pin", async () => { + writeFileSync(join(repo, "retry.ts"), "export function retry() {}\n"); + commitAll(repo, "init"); + const hashes = await hashObjects(repo, ["retry.ts"]); + if (!hashes.ok) throw hashes.error; + const sha = hashes.value[0] as string; + + writeFileSync(join(repo, "retry.ts"), "export function retry(n) { return n; }\n"); + const result = await classifyPin(repo, "retry.ts", wholeFilePin(sha)); + expect(result.ok && result.value.state).toBe("moved"); + }); + + it("missing: target file absent from the working tree", async () => { + const result = await classifyPin(repo, "gone.ts", wholeFilePin("abcdef1")); + expect(result.ok && result.value.state).toBe("missing"); + }); + + it("missing: a symlink escaping the repo, bytes never read", async () => { + const outer = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-escape2-"))); + try { + const nestedRepo = join(outer, "repo"); + mkdirSync(nestedRepo); + git(nestedRepo, ["init", "-q"]); + mkdirSync(join(outer, "outside")); + writeFileSync(join(outer, "outside", "secret.ts"), "TOP SECRET CONTENT\n"); + symlinkSync(join(outer, "outside"), join(nestedRepo, "escape")); + commitAll(nestedRepo, "init"); + const result = await classifyPin(nestedRepo, "escape/secret.ts", wholeFilePin("abcdef1")); + expect(result.ok && result.value.state).toBe("missing"); + } finally { + rmSync(outer, { recursive: true, force: true }); + } + }); +}); + +describe("classifyAgainstHash — range pins (step 3)", () => { + let repo: string; + beforeEach(() => { + repo = initRepo(); + }); + afterEach(() => { + rmSync(repo, { recursive: true, force: true }); + }); + + it("intact via relocation: exact text found at a new line range", async () => { + const original = + [ + "line1", + "line2", + "TARGET BLOCK START", + "some meaningful content here", + "TARGET BLOCK END", + "line6", + ].join("\n") + "\n"; + writeFileSync(join(repo, "f.ts"), original); + commitAll(repo, "init"); + const pinnedHashes = await hashObjects(repo, ["f.ts"]); + if (!pinnedHashes.ok) throw pinnedHashes.error; + const pinnedSha = pinnedHashes.value[0] as string; + const pin: PinSpec = { start: 3, end: 5, sha: pinnedSha }; + + // Move the pinned block down by adding lines above it — text unchanged, + // location changed. + const relocated = + [ + "padding1", + "padding2", + "line1", + "line2", + "TARGET BLOCK START", + "some meaningful content here", + "TARGET BLOCK END", + "line6", + ].join("\n") + "\n"; + writeFileSync(join(repo, "f.ts"), relocated); + + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + const currentHash = currentHashes.value[0] as string; + + const verdict = await classifyAgainstHash(repo, join(repo, "f.ts"), pin, currentHash); + expect(verdict.state).toBe("intact"); + expect(verdict.relocated).toEqual({ start: 5, end: 7 }); + }); + + it("moved: the pinned range's text is gone", async () => { + const original = ["line1", "TARGET BLOCK marker text here", "line3"].join("\n") + "\n"; + writeFileSync(join(repo, "f.ts"), original); + commitAll(repo, "init"); + const pinnedHashes = await hashObjects(repo, ["f.ts"]); + if (!pinnedHashes.ok) throw pinnedHashes.error; + const pin: PinSpec = { start: 2, end: 2, sha: pinnedHashes.value[0] as string }; + + writeFileSync( + join(repo, "f.ts"), + ["line1", "COMPLETELY DIFFERENT rewritten text", "line3"].join("\n") + "\n", + ); + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + + const verdict = await classifyAgainstHash( + repo, + join(repo, "f.ts"), + pin, + currentHashes.value[0] as string, + ); + expect(verdict.state).toBe("moved"); + }); + + it("trivial pinned content (a single `}`) never classifies intact — moved instead (C7)", async () => { + const original = ["function f() {", " return 1;", "}"].join("\n") + "\n"; + writeFileSync(join(repo, "f.ts"), original); + commitAll(repo, "init"); + const pinnedHashes = await hashObjects(repo, ["f.ts"]); + if (!pinnedHashes.ok) throw pinnedHashes.error; + const pin: PinSpec = { start: 3, end: 3, sha: pinnedHashes.value[0] as string }; + + writeFileSync(join(repo, "f.ts"), ["function f() {", " return 2;", "}"].join("\n") + "\n"); + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + + const verdict = await classifyAgainstHash( + repo, + join(repo, "f.ts"), + pin, + currentHashes.value[0] as string, + ); + expect(verdict.state).toBe("moved"); + }); + + it("CRLF working file vs LF-pinned blob still finds the match — intact with relocated", async () => { + const original = + ["line1", "MEANINGFUL TARGET LINE for the test case", "line3"].join("\n") + "\n"; + writeFileSync(join(repo, "f.ts"), original); + commitAll(repo, "init"); + const pinnedHashes = await hashObjects(repo, ["f.ts"]); + if (!pinnedHashes.ok) throw pinnedHashes.error; + const pin: PinSpec = { start: 2, end: 2, sha: pinnedHashes.value[0] as string }; + + // Current working file uses CRLF line endings and has an extra leading + // line, so the blob differs but the target text is present, relocated. + const crlf = + ["header", "line1", "MEANINGFUL TARGET LINE for the test case", "line3"].join("\r\n") + + "\r\n"; + writeFileSync(join(repo, "f.ts"), crlf); + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + + const verdict = await classifyAgainstHash( + repo, + join(repo, "f.ts"), + pin, + currentHashes.value[0] as string, + ); + expect(verdict.state).toBe("intact"); + expect(verdict.relocated).toEqual({ start: 3, end: 3 }); + }); + + it("range past the pinned blob's last line -> moved", async () => { + writeFileSync(join(repo, "f.ts"), "one line only\n"); + commitAll(repo, "init"); + const pinnedHashes = await hashObjects(repo, ["f.ts"]); + if (!pinnedHashes.ok) throw pinnedHashes.error; + const pin: PinSpec = { start: 1, end: 50, sha: pinnedHashes.value[0] as string }; + + writeFileSync(join(repo, "f.ts"), "changed\n"); + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + const verdict = await classifyAgainstHash( + repo, + join(repo, "f.ts"), + pin, + currentHashes.value[0] as string, + ); + expect(verdict.state).toBe("moved"); + }); + + it("a pinned sha absent from the odb -> moved (not an error)", async () => { + writeFileSync(join(repo, "f.ts"), "current content\n"); + commitAll(repo, "init"); + const pin: PinSpec = { start: 1, end: 1, sha: "0000000000000000000000000000000000dead" }; + const currentHashes = await hashObjects(repo, ["f.ts"]); + if (!currentHashes.ok) throw currentHashes.error; + const verdict = await classifyAgainstHash( + repo, + join(repo, "f.ts"), + pin, + currentHashes.value[0] as string, + ); + expect(verdict.state).toBe("moved"); + }); +}); + +describe("classifyPin — non-git / traversal edge cases", () => { + it("../ traversal outside a real repo -> missing", async () => { + const outer = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-outer-"))); + const repo = join(outer, "repo"); + mkdirSync(repo); + writeFileSync(join(outer, "secret.ts"), "top secret\n"); + try { + const result = await classifyPin(repo, "../secret.ts", wholeFilePin("abcdef1")); + expect(result.ok && result.value.state).toBe("missing"); + } finally { + rmSync(outer, { recursive: true, force: true }); + } + }); +}); + +describe("hashObjects batching", () => { + let repo: string; + beforeEach(() => { + repo = initRepo(); + }); + afterEach(() => { + rmSync(repo, { recursive: true, force: true }); + }); + + it("hashes many files in one call, mapped back by position", async () => { + writeFileSync(join(repo, "a.ts"), "aaa\n"); + writeFileSync(join(repo, "b.ts"), "bbb\n"); + writeFileSync(join(repo, "c.ts"), "ccc\n"); + commitAll(repo, "init"); + const result = await hashObjects(repo, ["a.ts", "b.ts", "c.ts"]); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toHaveLength(3); + // Each hash should match a single-file hash-object call individually. + const a = await hashObjects(repo, ["a.ts"]); + if (!a.ok) throw a.error; + expect(result.value[0]).toBe(a.value[0]); + }); + + it("errors the whole batch when one candidate file is missing", async () => { + writeFileSync(join(repo, "a.ts"), "aaa\n"); + commitAll(repo, "init"); + const result = await hashObjects(repo, ["a.ts", "missing.ts"]); + expect(result.ok).toBe(false); + }); + + it("returns [] for an empty path list without spawning git", async () => { + const result = await hashObjects(repo, []); + expect(result).toEqual({ ok: true, value: [] }); + }); +}); diff --git a/test/anchors/perf.test.ts b/test/anchors/perf.test.ts new file mode 100644 index 00000000..d6019384 --- /dev/null +++ b/test/anchors/perf.test.ts @@ -0,0 +1,80 @@ +// test/anchors/perf.test.ts +// CI tripwire (2026-07-26 spec, C1): 24 intact pins across 2 repos must +// classify in well under the batched budget. This is NOT the authoritative +// check — the spec's 50ms p95 live-vault measurement is — but it is fast and +// deterministic enough to run on every CI pass, catching a regression back +// toward one git subprocess per pin (24 spawns) before it ships. + +import { execFileSync } from "node:child_process"; +import { mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { computeAnchors } from "../../src/anchors/read.js"; + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): void { + execFileSync("git", args, { cwd, env: GIT_ENV, stdio: "ignore" }); +} + +function hashOf(repo: string, relPath: string): string { + return execFileSync("git", ["-C", repo, "hash-object", relPath], { env: GIT_ENV }) + .toString() + .trim(); +} + +describe("computeAnchors perf tripwire — 24 intact pins across 2 repos", () => { + let repoA: string; + let repoB: string; + let describes: string[]; + let codeRepos: Record; + + beforeAll(() => { + repoA = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-perf-a-"))); + repoB = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-perf-b-"))); + git(repoA, ["init", "-q"]); + git(repoB, ["init", "-q"]); + + describes = []; + for (const [name, repo] of [["a", repoA] as const, ["b", repoB] as const]) { + for (let i = 0; i < 12; i++) { + writeFileSync(join(repo, `f${i}.ts`), `export const v${i} = ${i};\n`); + } + git(repo, ["add", "."]); + git(repo, ["commit", "-q", "-m", "init"]); + for (let i = 0; i < 12; i++) { + describes.push(`${name}:f${i}.ts@${hashOf(repo, `f${i}.ts`)}`); + } + } + codeRepos = { a: repoA, b: repoB }; + }); + + afterAll(() => { + rmSync(repoA, { recursive: true, force: true }); + rmSync(repoB, { recursive: true, force: true }); + }); + + it("classifies all 24 intact pins in well under 150ms", async () => { + // One warmup call so the timed run isn't paying for a cold `git` + // process-spawn cache the OS builds on first exec. + await computeAnchors(describes, codeRepos); + + const t0 = performance.now(); + const result = await computeAnchors(describes, codeRepos); + const elapsed = performance.now() - t0; + + expect(result).not.toBeNull(); + expect(result?.checked).toBe(24); + expect(result?.skipped).toBe(0); + expect(result?.errored).toBe(0); + expect(result?.entries.every((e) => e.state === "intact")).toBe(true); + expect(elapsed).toBeLessThan(150); + }); +}); diff --git a/test/anchors/pin.test.ts b/test/anchors/pin.test.ts new file mode 100644 index 00000000..e70f1606 --- /dev/null +++ b/test/anchors/pin.test.ts @@ -0,0 +1,135 @@ +import { describe, expect, it } from "vitest"; +import { looksLikeMalformedPin, PIN_RE, splitPin } from "../../src/anchors/pin.js"; + +describe("splitPin", () => { + it("passes a bare binding through byte-identical", () => { + expect(splitPin("api:src/retry.ts")).toEqual({ binding: "api:src/retry.ts", pin: null }); + }); + + it("passes a ::symbol binding through byte-identical", () => { + expect(splitPin("api:src/retry.ts::withRetry")).toEqual({ + binding: "api:src/retry.ts::withRetry", + pin: null, + }); + }); + + it("parses a whole-file pin", () => { + expect(splitPin("api:src/retry.ts@9f3c2ab")).toEqual({ + binding: "api:src/retry.ts", + pin: { start: null, end: null, sha: "9f3c2ab" }, + }); + }); + + it("parses a range pin", () => { + expect(splitPin("api:src/retry.ts#L40-58@9f3c2ab")).toEqual({ + binding: "api:src/retry.ts", + pin: { start: 40, end: 58, sha: "9f3c2ab" }, + }); + }); + + it("parses a symbol + range pin", () => { + expect(splitPin("api:src/retry.ts::withRetry#L40-58@9f3c2ab")).toEqual({ + binding: "api:src/retry.ts::withRetry", + pin: { start: 40, end: 58, sha: "9f3c2ab" }, + }); + }); + + it("a bare #L40 means the single line 40", () => { + expect(splitPin("api:src/retry.ts#L40@9f3c2ab")).toEqual({ + binding: "api:src/retry.ts", + pin: { start: 40, end: 40, sha: "9f3c2ab" }, + }); + }); + + it("accepts a full 40-char sha", () => { + const sha = "a".repeat(40); + const result = splitPin(`api:src/retry.ts@${sha}`); + expect(result.pin).toEqual({ start: null, end: null, sha }); + }); + + it("rejects a sha shorter than 7 chars — degrades to bare binding", () => { + expect(splitPin("api:src/retry.ts@abc12")).toEqual({ + binding: "api:src/retry.ts@abc12", + pin: null, + }); + }); + + it("is end-anchored: an @ mid-path is unaffected", () => { + expect(splitPin("api:src/@scope/pkg.ts")).toEqual({ + binding: "api:src/@scope/pkg.ts", + pin: null, + }); + }); + + it("bundle@.js is NOT reinterpreted as a pin (trailing .js defeats the end anchor)", () => { + expect(splitPin("api:dist/bundle@9f3c2ab1.js")).toEqual({ + binding: "api:dist/bundle@9f3c2ab1.js", + pin: null, + }); + }); + + it("a path that itself ends in pin-shaped text: the pin wins (accepted ambiguity)", () => { + // Pathological but explicitly accepted by the spec: a literal path + // component that happens to look like a pin suffix parses as a pin. + const result = splitPin("api:weird/file#L1-2@abcdef0"); + expect(result.pin).toEqual({ start: 1, end: 2, sha: "abcdef0" }); + expect(result.binding).toBe("api:weird/file"); + }); + + it("an inverted range degrades the WHOLE entry to a bare binding", () => { + expect(splitPin("api:src/retry.ts#L58-40@9f3c2ab")).toEqual({ + binding: "api:src/retry.ts#L58-40@9f3c2ab", + pin: null, + }); + }); + + it("trims surrounding whitespace", () => { + expect(splitPin(" api:src/retry.ts@9f3c2ab ")).toEqual({ + binding: "api:src/retry.ts", + pin: { start: null, end: null, sha: "9f3c2ab" }, + }); + }); +}); + +describe("PIN_RE", () => { + it("matches the grammar's own examples", () => { + expect(PIN_RE.test("api:src/retry.ts@9f3c2ab")).toBe(true); + expect(PIN_RE.test("api:src/retry.ts#L40-58@9f3c2ab")).toBe(true); + expect(PIN_RE.test("api:src/retry.ts::withRetry#L40-58@9f3c2ab")).toBe(true); + expect(PIN_RE.test("api:src/retry.ts")).toBe(false); + }); +}); + +describe("looksLikeMalformedPin", () => { + it("flags a range marker with a non-hex/uppercase sha near-miss", () => { + expect(looksLikeMalformedPin("api:src/retry.ts#L40-58@ZZZZZZZ")).toBe(true); + expect(looksLikeMalformedPin("api:src/retry.ts#L40@notahash")).toBe(true); + }); + + it("flags a trailing @ that is too short or uppercase", () => { + expect(looksLikeMalformedPin("api:src/retry.ts@abcd")).toBe(true); // 4 chars, below 7 + expect(looksLikeMalformedPin("api:src/retry.ts@ABCDEF1")).toBe(true); // uppercase + }); + + it("flags a structural match with an inverted range", () => { + expect(looksLikeMalformedPin("api:src/retry.ts#L58-40@9f3c2ab")).toBe(true); + }); + + it("does NOT flag ::@property (non-hex letters defeat the near-miss)", () => { + expect(looksLikeMalformedPin("component.css::@property")).toBe(false); + }); + + it("does NOT flag ::render@v2 ('v' is not hex)", () => { + expect(looksLikeMalformedPin("api:src/view.ts::render@v2")).toBe(false); + }); + + it("does NOT flag bundle@.js (trailing .js defeats the end anchor)", () => { + expect(looksLikeMalformedPin("api:dist/bundle@9f3c2ab1.js")).toBe(false); + }); + + it("does NOT flag a well-formed pin", () => { + expect(looksLikeMalformedPin("api:src/retry.ts#L40-58@9f3c2ab")).toBe(false); + expect(looksLikeMalformedPin("api:src/retry.ts@9f3c2ab")).toBe(false); + expect(looksLikeMalformedPin("api:src/retry.ts")).toBe(false); + }); +}); diff --git a/test/audit/describes.test.ts b/test/audit/describes.test.ts index b135a02d..a3ff35c1 100644 --- a/test/audit/describes.test.ts +++ b/test/audit/describes.test.ts @@ -71,6 +71,7 @@ describe("classifyDescribesEdges", () => { targetPath: "src/login.ts", symbol: null, raw: "svc:src/login.ts", + pin: null, }, { sourceRepo: "docs", @@ -79,6 +80,7 @@ describe("classifyDescribesEdges", () => { targetPath: "guide.md", symbol: null, raw: "guide.md", + pin: null, }, ]); }); diff --git a/test/audit/pin-cli.test.ts b/test/audit/pin-cli.test.ts new file mode 100644 index 00000000..363f865d --- /dev/null +++ b/test/audit/pin-cli.test.ts @@ -0,0 +1,509 @@ +// test/audit/pin-cli.test.ts +// `daftari audit --pin` / `--pin --apply` end to end, plus the missing-pin +// auto-tension dedupe and moved-first semantic ordering (2026-07-26 spec, +// Decisions 3 and 5). + +import { execFileSync, spawn } from "node:child_process"; +import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { runAudit } from "../../src/audit/index.js"; +import { listTensions } from "../../src/curation/tension.js"; +import type { LlmClient } from "../../src/eval/llm.js"; +import { ok } from "../../src/frontmatter/types.js"; +import { writeLockfile } from "../../src/lifecycle/lock.js"; + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): void { + execFileSync("git", args, { cwd, env: GIT_ENV, stdio: "ignore" }); +} + +function hashOf(repo: string, relPath: string): string { + return execFileSync("git", ["-C", repo, "hash-object", relPath], { env: GIT_ENV }) + .toString() + .trim(); +} + +function docFrontmatter(describes: string[]): string { + return `--- +title: "Retry logic" +domain: accumulation +collection: engineering +status: canonical +confidence: high +created: "2026-01-05" +updated: "2026-01-05" +updated_by: agent:test +provenance: direct +tags: [] +describes: +${describes.map((d) => ` - "${d}"`).join("\n")} +--- + +The retry loop. +`; +} + +// Names the code repo "svc" in the AUDIT's own registry, matching what +// code_repos and the describes prefix both use — the aligned case. (A +// misaligned case, where --code-repo gets an anonymous code-N name instead, +// is exactly what the "registry mismatch" / "unpinnable" tests exercise.) +function writeAuditYaml(tmp: string, docsRepo: string, codeRepo: string): string { + const path = join(tmp, "audit.yaml"); + writeFileSync( + path, + `repos:\n - name: docs\n path: ${docsRepo}\n - name: svc\n path: ${codeRepo}\n type: code\n`, + ); + return path; +} + +const stubLlm = (parsed: unknown): LlmClient => + ({ + completeJson: vi.fn(async () => + ok({ text: "", input_tokens: 1, output_tokens: 1, stop_reason: "end_turn", parsed }), + ), + }) as unknown as LlmClient; + +describe("daftari audit --pin", () => { + let tmp: string; + let docsRepo: string; + let codeRepo: string; + + beforeEach(() => { + tmp = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-pin-"))); + docsRepo = join(tmp, "docs"); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-pin-code-"))); + + mkdirSync(join(docsRepo, "engineering"), { recursive: true }); + mkdirSync(join(docsRepo, ".daftari"), { recursive: true }); + writeFileSync(join(docsRepo, ".daftari", "config.yaml"), `code_repos:\n svc: ${codeRepo}\n`); + + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + + git(docsRepo, ["init", "-q"]); + }); + + afterEach(() => { + rmSync(tmp, { recursive: true, force: true }); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + it("plan mode prints proposals and writes nothing", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + const before = readFileSync(join(docsRepo, "engineering/retry.md"), "utf-8"); + + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const code = await runAudit(["--repo", docsRepo, "--code-repo", codeRepo, "--pin"]); + stdout.mockRestore(); + + expect(code).toBe(0); + const after = readFileSync(join(docsRepo, "engineering/retry.md"), "utf-8"); + expect(after).toBe(before); // nothing written + }); + + it("plan mode's printed output names the proposed pin", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + let printed = ""; + const stdout = vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + printed += String(chunk); + return true; + }); + await runAudit(["--repo", docsRepo, "--code-repo", codeRepo, "--pin"]); + stdout.mockRestore(); + + const sha = hashOf(codeRepo, "retry.ts").slice(0, 12); + expect(printed).toContain("engineering/retry.md"); + expect(printed).toContain(`svc:retry.ts -> svc:retry.ts@${sha}`); + expect(printed).toContain("proposed: 1"); + }); + + it("skips a dirty working-tree file with the dirty-skip message", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + // Dirty the code repo's working tree without committing. + writeFileSync(join(codeRepo, "retry.ts"), "export function retry(n) { return n; }\n"); + + let printed = ""; + const stdout = vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + printed += String(chunk); + return true; + }); + const code = await runAudit(["--repo", docsRepo, "--code-repo", codeRepo, "--pin"]); + stdout.mockRestore(); + + expect(code).toBe(0); + expect(printed).toContain("skipped: working tree differs from HEAD"); + expect(printed).toContain("proposed: 0"); + }); + + it("--pin --apply writes a whole-file 12-char pin, commits once, is idempotent, and the pin classifies intact", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + const yamlPath = writeAuditYaml(tmp, docsRepo, codeRepo); + + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const code = await runAudit(["--config", yamlPath, "--pin", "--apply"]); + stdout.mockRestore(); + expect(code).toBe(0); + + const written = readFileSync(join(docsRepo, "engineering/retry.md"), "utf-8"); + const sha = hashOf(codeRepo, "retry.ts"); + expect(written).toContain(`svc:retry.ts@${sha.slice(0, 12)}`); + expect(written).not.toContain('"svc:retry.ts"'); // old unpinned entry replaced + + const log = execFileSync("git", ["-C", docsRepo, "log", "--oneline"], { + env: GIT_ENV, + }).toString(); + expect(log.trim().split("\n")).toHaveLength(1); // one commit + + // Re-running --apply against the now-pinned doc is a no-op: nothing left + // to propose, so nothing to write, and no second commit. + const stdout2 = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath, "--pin", "--apply"]); + stdout2.mockRestore(); + const log2 = execFileSync("git", ["-C", docsRepo, "log", "--oneline"], { + env: GIT_ENV, + }).toString(); + expect(log2.trim().split("\n")).toHaveLength(1); // still one commit + + // The applied pin, read back via the audit's own pin classifier (against + // a registry that names the code repo the same as code_repos does), is + // intact-on-arrival by construction. + let output = ""; + const stdout3 = vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + output += String(chunk); + return true; + }); + await runAudit(["--config", yamlPath]); + stdout3.mockRestore(); + expect(output).toContain("code pins intact / moved / missing: **1 / 0 / 0**"); + }); + + it("--pin --apply refuses with two docs repos", async () => { + const docsRepo2 = join(tmp, "docs2"); + mkdirSync(docsRepo2, { recursive: true }); + git(docsRepo2, ["init", "-q"]); + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + + const code = await runAudit([ + "--repo", + docsRepo, + "--repo", + docsRepo2, + "--code-repo", + codeRepo, + "--pin", + ]); + expect(code).toBe(2); + }); + + it("--pin --apply refuses against a live process.lock holder, naming its pid and mode", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts"])); + + // A real, live child process whose OWN argv contains the vault path — + // isDaftariProcess matches on that substring via `ps`, so a synthetic + // pid (or this test's own pid, whose argv is vitest's own command line) + // would not exercise the real liveness check. + const child = spawn(process.execPath, ["-e", "setTimeout(() => {}, 30000)", docsRepo], { + stdio: "ignore", + }); + await new Promise((resolve) => setTimeout(resolve, 250)); // let it actually start + try { + writeLockfile(docsRepo, { + daftari: true, + pid: child.pid as number, + vaultRoot: docsRepo, + startedAt: new Date().toISOString(), + version: "test", + mode: "serve", + }); + + const before = readFileSync(join(docsRepo, "engineering/retry.md"), "utf-8"); + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const code = await runAudit([ + "--repo", + docsRepo, + "--code-repo", + codeRepo, + "--pin", + "--apply", + ]); + const messages = stderr.mock.calls.map((c) => String(c[0])).join(""); + stderr.mockRestore(); + stdout.mockRestore(); + + expect(code).toBe(2); + expect(messages).toContain(String(child.pid)); + expect(messages).toContain("serve"); + const after = readFileSync(join(docsRepo, "engineering/retry.md"), "utf-8"); + expect(after).toBe(before); // refused before any write + } finally { + child.kill("SIGKILL"); + } + }); + + it("lists a prefix as unpinnable when it's only in the audit's --code-repo registry, not code_repos", async () => { + // A SECOND anonymous code repo the audit knows about (code-0/code-1) but + // the docs vault's own code_repos does not declare. + const otherCode = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-pin-other-"))); + try { + git(otherCode, ["init", "-q"]); + writeFileSync(join(otherCode, "x.ts"), "export const x = 1;\n"); + git(otherCode, ["add", "."]); + git(otherCode, ["commit", "-q", "-m", "init"]); + + writeFileSync(join(docsRepo, "engineering/other.md"), docFrontmatter(["code-1:x.ts"])); + + let printed = ""; + const stdout = vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + printed += String(chunk); + return true; + }); + await runAudit([ + "--repo", + docsRepo, + "--code-repo", + codeRepo, + "--code-repo", + otherCode, + "--pin", + ]); + stdout.mockRestore(); + expect(printed).toContain("unpinnable"); + expect(printed).toContain("code-1"); + } finally { + rmSync(otherCode, { recursive: true, force: true }); + } + }); +}); + +describe("daftari audit — registry cross-check (C2)", () => { + let tmp: string; + let docsRepo: string; + let codeRepo: string; + + beforeEach(() => { + tmp = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-registry-"))); + docsRepo = join(tmp, "docs"); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-registry-code-"))); + mkdirSync(join(docsRepo, "engineering"), { recursive: true }); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + git(docsRepo, ["init", "-q"]); + }); + + afterEach(() => { + rmSync(tmp, { recursive: true, force: true }); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + it("warns when a pinned repo name is in the audit registry but not code_repos", async () => { + // No .daftari/config.yaml written for docsRepo -> code_repos is empty. + writeFileSync( + join(docsRepo, "engineering/retry.md"), + docFrontmatter([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + const yamlPath = writeAuditYaml(tmp, docsRepo, codeRepo); + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath]); + const calls = stderr.mock.calls.map((c) => String(c[0])); + stderr.mockRestore(); + stdout.mockRestore(); + expect(calls.some((c) => c.includes("registry mismatch") && c.includes("svc"))).toBe(true); + }); + + it("warns when a pinned repo name is in code_repos but not the audit registry", async () => { + mkdirSync(join(docsRepo, ".daftari"), { recursive: true }); + writeFileSync(join(docsRepo, ".daftari", "config.yaml"), `code_repos:\n svc: ${codeRepo}\n`); + writeFileSync( + join(docsRepo, "engineering/retry.md"), + docFrontmatter([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + // Audit registry names the SAME repo "code-0" (anonymous), not "svc". + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--repo", docsRepo, "--code-repo", codeRepo]); + const calls = stderr.mock.calls.map((c) => String(c[0])); + stderr.mockRestore(); + stdout.mockRestore(); + expect(calls.some((c) => c.includes("registry mismatch") && c.includes("svc"))).toBe(true); + }); + + it("warns when both registries know the name but resolve it to different paths", async () => { + const otherCode = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-registry-other-"))); + try { + git(otherCode, ["init", "-q"]); + writeFileSync(join(otherCode, "retry.ts"), "export function retry() { /* different */ }\n"); + git(otherCode, ["add", "."]); + git(otherCode, ["commit", "-q", "-m", "init"]); + + mkdirSync(join(docsRepo, ".daftari"), { recursive: true }); + // code_repos points 'svc' at otherCode; the audit.yaml below points + // its own 'svc' entry at codeRepo — same name, different real paths. + writeFileSync( + join(docsRepo, ".daftari", "config.yaml"), + `code_repos:\n svc: ${otherCode}\n`, + ); + writeFileSync( + join(docsRepo, "engineering/retry.md"), + docFrontmatter([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + const yamlPath = writeAuditYaml(tmp, docsRepo, codeRepo); + + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath]); + const calls = stderr.mock.calls.map((c) => String(c[0])); + stderr.mockRestore(); + stdout.mockRestore(); + expect( + calls.some((c) => c.includes("registry mismatch") && c.includes("different paths")), + ).toBe(true); + } finally { + rmSync(otherCode, { recursive: true, force: true }); + } + }); +}); + +describe("daftari audit — missing-pin auto-tension (Decision 3, deduplicated)", () => { + let tmp: string; + let docsRepo: string; + let codeRepo: string; + let yamlPath: string; + + beforeEach(() => { + tmp = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-autotension-"))); + docsRepo = join(tmp, "docs"); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-autotension-code-"))); + mkdirSync(join(docsRepo, "engineering"), { recursive: true }); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + git(docsRepo, ["init", "-q"]); + mkdirSync(join(docsRepo, ".daftari"), { recursive: true }); + writeFileSync(join(docsRepo, ".daftari", "config.yaml"), `code_repos:\n svc: ${codeRepo}\n`); + yamlPath = writeAuditYaml(tmp, docsRepo, codeRepo); + }); + + afterEach(() => { + rmSync(tmp, { recursive: true, force: true }); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + it("logs a missing-pin tension without an LLM, and dedupes on a second run", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:gone.ts@0000000"])); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath, "--auto-tension"]); + await runAudit(["--config", yamlPath, "--auto-tension"]); + stdout.mockRestore(); + + const tensions = await listTensions(docsRepo); + expect(tensions.ok).toBe(true); + if (!tensions.ok) return; + const missing = tensions.value.filter((t) => t.title.startsWith("Doc-code missing:")); + expect(missing).toHaveLength(1); // deduped, not doubled on the second run + }); + + it("never auto-logs a bare 'moved' pin as a tension", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:retry.ts@0000000"])); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath, "--auto-tension"]); + stdout.mockRestore(); + + const tensions = await listTensions(docsRepo); + expect(tensions.ok && tensions.value.length).toBe(0); + }); + + it("--auto-tension without --semantic still warns when there are zero pinned bindings", async () => { + writeFileSync(join(docsRepo, "engineering/plain.md"), docFrontmatter([])); + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath, "--auto-tension"]); + stdout.mockRestore(); + const calls = stderr.mock.calls.map((c) => String(c[0])); + stderr.mockRestore(); + expect(calls.some((c) => c.includes("has no effect without --semantic"))).toBe(true); + }); + + it("--auto-tension without --semantic does NOT warn when pinned bindings exist (missing-pin logging is useful work)", async () => { + writeFileSync(join(docsRepo, "engineering/retry.md"), docFrontmatter(["svc:gone.ts@0000000"])); + const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); + const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true); + await runAudit(["--config", yamlPath, "--auto-tension"]); + stdout.mockRestore(); + const calls = stderr.mock.calls.map((c) => String(c[0])); + stderr.mockRestore(); + expect(calls.some((c) => c.includes("has no effect without --semantic"))).toBe(false); + }); +}); + +describe("daftari audit — moved-first semantic ordering (Decision 3)", () => { + let tmp: string; + let docsRepo: string; + let codeRepo: string; + let yamlPath: string; + + beforeEach(() => { + tmp = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-order-"))); + docsRepo = join(tmp, "docs"); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-order-code-"))); + mkdirSync(join(docsRepo, "engineering"), { recursive: true }); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "fresh.ts"), "export const fresh = 1;\n"); + writeFileSync(join(codeRepo, "stale.ts"), "export const stale = 1;\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + git(docsRepo, ["init", "-q"]); + mkdirSync(join(docsRepo, ".daftari"), { recursive: true }); + writeFileSync(join(docsRepo, ".daftari", "config.yaml"), `code_repos:\n svc: ${codeRepo}\n`); + yamlPath = writeAuditYaml(tmp, docsRepo, codeRepo); + }); + + afterEach(() => { + rmSync(tmp, { recursive: true, force: true }); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + it("classifies the moved-pin binding first under a --max-semantic cap of 1", async () => { + const freshSha = hashOf(codeRepo, "fresh.ts"); + // fresh.md's pin is intact; stale.md's pin is moved. fresh.md sorts + // alphabetically first, so WITHOUT reordering the semantic pass (capped + // at 1) would pick fresh.md; WITH moved-first reordering it picks + // stale.md instead. + writeFileSync( + join(docsRepo, "engineering/fresh.md"), + docFrontmatter([`svc:fresh.ts@${freshSha}`]), + ); + writeFileSync(join(docsRepo, "engineering/stale.md"), docFrontmatter(["svc:stale.ts@0000000"])); + + const llm = stubLlm({ verdict: "coherent", contradictions: [] }); + let output = ""; + const stdout = vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + output += String(chunk); + return true; + }); + await runAudit(["--config", yamlPath, "--semantic", "--max-semantic", "1"], { llm }); + stdout.mockRestore(); + + expect(llm.completeJson).toHaveBeenCalledTimes(1); + const call = (llm.completeJson as ReturnType).mock.calls[0]?.[0] as { + user: string; + }; + expect(call.user).toContain("stale.md"); + expect(output).toContain("stale.md"); + }); +}); diff --git a/test/audit/pins.test.ts b/test/audit/pins.test.ts new file mode 100644 index 00000000..3e2493f0 --- /dev/null +++ b/test/audit/pins.test.ts @@ -0,0 +1,157 @@ +// test/audit/pins.test.ts +// checkPins: batch pin classification for `daftari audit` (2026-07-26 spec, +// Decision 3). + +import { execFileSync } from "node:child_process"; +import { mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { checkPins } from "../../src/audit/checks/pins.js"; +import { classifyDescribesEdges } from "../../src/audit/describes.js"; +import type { DocSnapshot, RepoSnapshot } from "../../src/audit/types.js"; + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): void { + execFileSync("git", args, { cwd, env: GIT_ENV, stdio: "ignore" }); +} + +function doc(relPath: string, describes: string[]): DocSnapshot { + return { + relPath, + absPath: `/x/${relPath}`, + mtime: "2026-01-01T00:00:00.000Z", + mtimeSource: "git", + headings: new Set(), + links: [], + describes, + }; +} + +function docsRepo(docs: DocSnapshot[]): RepoSnapshot { + return { + config: { name: "docs", path: "/docs", docsGlob: "**/*.md", urls: [], type: "docs" }, + docs: new Map(docs.map((d) => [d.relPath, d])), + }; +} + +describe("checkPins", () => { + let codeRepo: string; + + beforeEach(() => { + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-audit-pins-"))); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + }); + + afterEach(() => { + rmSync(codeRepo, { recursive: true, force: true }); + }); + + function codeSnap(): RepoSnapshot { + return { + config: { name: "svc", path: codeRepo, docsGlob: "**/*", urls: [], type: "code" }, + docs: new Map(), + }; + } + + function sha(): string { + return execFileSync("git", ["-C", codeRepo, "hash-object", "retry.ts"], { env: GIT_ENV }) + .toString() + .trim(); + } + + it("returns [] when no bindings are pinned", async () => { + const snaps = [docsRepo([doc("a.md", ["svc:retry.ts"])]), codeSnap()]; + const edges = classifyDescribesEdges(snaps); + expect(await checkPins(snaps, edges)).toEqual([]); + }); + + it("classifies an intact whole-file pin", async () => { + const snaps = [docsRepo([doc("a.md", [`svc:retry.ts@${sha()}`])]), codeSnap()]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings).toHaveLength(1); + expect(findings[0]?.state).toBe("intact"); + expect(findings[0]?.source).toEqual({ repo: "docs", path: "a.md" }); + expect(findings[0]?.target).toEqual({ repo: "svc", path: "retry.ts" }); + }); + + it("classifies a moved pin (blob changed)", async () => { + const snaps = [docsRepo([doc("a.md", ["svc:retry.ts@0000000"])]), codeSnap()]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings[0]?.state).toBe("moved"); + }); + + it("classifies missing when the target file is absent", async () => { + const snaps = [docsRepo([doc("a.md", ["svc:gone.ts@0000000"])]), codeSnap()]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings[0]?.state).toBe("missing"); + }); + + it("classifies missing when the target repo isn't in the snapshot set", async () => { + const snaps = [docsRepo([doc("a.md", ["ghost:retry.ts@0000000"])])]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings[0]?.state).toBe("missing"); + }); + + it("batches multiple pins against the same repo into one classification pass", async () => { + writeFileSync(join(codeRepo, "second.ts"), "export const y = 2;\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "second"]); + const secondSha = execFileSync("git", ["-C", codeRepo, "hash-object", "second.ts"], { + env: GIT_ENV, + }) + .toString() + .trim(); + + const snaps = [ + docsRepo([doc("a.md", [`svc:retry.ts@${sha()}`, `svc:second.ts@${secondSha}`])]), + codeSnap(), + ]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings).toHaveLength(2); + expect(findings.every((f) => f.state === "intact")).toBe(true); + }); + + it("computes a relocated range for an intact range pin", async () => { + const content = + ["a", "TARGET LINE with enough content to pass the trivial-content floor", "c"].join("\n") + + "\n"; + writeFileSync(join(codeRepo, "range.ts"), content); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "range"]); + const rangeSha = execFileSync("git", ["-C", codeRepo, "hash-object", "range.ts"], { + env: GIT_ENV, + }) + .toString() + .trim(); + + // Move the line down by prepending content — blob differs, text intact. + writeFileSync( + join(codeRepo, "range.ts"), + ["pad", "a", "TARGET LINE with enough content to pass the trivial-content floor", "c"].join( + "\n", + ) + "\n", + ); + + const snaps = [docsRepo([doc("a.md", [`svc:range.ts#L2-2@${rangeSha}`])]), codeSnap()]; + const edges = classifyDescribesEdges(snaps); + const findings = await checkPins(snaps, edges); + expect(findings[0]?.state).toBe("intact"); + expect(findings[0]?.relocated).toEqual({ start: 3, end: 3 }); + }); +}); diff --git a/test/audit/report.test.ts b/test/audit/report.test.ts index 81121c74..48f27511 100644 --- a/test/audit/report.test.ts +++ b/test/audit/report.test.ts @@ -20,6 +20,9 @@ const REPORT: AuditReport = { transitivelyStale: 1, brokenDescribes: 1, semanticDrifted: 0, + pinsIntact: 0, + pinsMoved: 0, + pinsMissing: 0, }, brokenRefs: [ { @@ -50,6 +53,8 @@ const REPORT: AuditReport = { }, ], semantic: [], + pins: [], + registryMismatches: [], }; describe("renderMarkdown", () => { @@ -85,6 +90,43 @@ describe("renderMarkdown", () => { }); }); +describe("renderMarkdown — pin verification", () => { + it("renders the pin verification table and totals line", () => { + const md = renderMarkdown({ + ...REPORT, + totals: { ...REPORT.totals, pinsIntact: 1, pinsMoved: 1, pinsMissing: 0 }, + pins: [ + { + source: { repo: "a", path: "auth.md" }, + target: { repo: "svc", path: "src/login.ts" }, + raw: "svc:src/login.ts@abc1234", + state: "intact", + }, + { + source: { repo: "a", path: "retry.md" }, + target: { repo: "svc", path: "src/retry.ts" }, + raw: "svc:src/retry.ts#L1-5@abc1234", + state: "moved", + }, + ], + }); + expect(md).toContain("code pins intact / moved / missing: **1 / 1 / 0**"); + expect(md).toContain("## Pin verification"); + expect(md).toContain("svc/src/login.ts"); + expect(md).toContain("intact"); + expect(md).toContain("moved"); + }); + + it("renders registry mismatch notes", () => { + const md = renderMarkdown({ + ...REPORT, + registryMismatches: [{ repo: "svc", docsRepo: "a", detail: "resolves to different paths" }], + }); + expect(md).toContain("## Registry mismatches"); + expect(md).toContain("'svc' referenced from a: resolves to different paths"); + }); +}); + describe("renderJson", () => { it("emits round-trippable JSON of the AuditReport", () => { const json = renderJson(REPORT); diff --git a/test/consolidate/birth.test.ts b/test/consolidate/birth.test.ts index 09046495..8d8f0ec8 100644 --- a/test/consolidate/birth.test.ts +++ b/test/consolidate/birth.test.ts @@ -18,6 +18,7 @@ import { reconcileDirection, } from "../../src/consolidate/birth.js"; import type { DerivationVerdict } from "../../src/consolidate/derivation-prompt.js"; +import { computeInputsFingerprint } from "../../src/curation/edges.js"; import type { LlmClient } from "../../src/eval/llm.js"; import { ok } from "../../src/frontmatter/types.js"; @@ -202,6 +203,38 @@ describe("birthOne — directed", () => { } }); + it("observe carries an fp whose inputs hash the (doc, neighbor) bytes and prompt id 'birth/foundational'", async () => { + const root = tmpVault(); + try { + const observedFp: unknown[] = []; + const { deps } = makeDeps({ + llm: mockLlm(V.docPremise()), + observe: async (input) => { + observedFp.push(input.fp); + return ok(stubEdge(input.fromPath, input.toPath)); + }, + }); + const r = await birthOne({ relPath: "a.md", content: "claim A" }, deps, { + ...baseOpts, + vaultRoot: root, + }); + expect(r.ok).toBe(true); + expect(observedFp.length).toBe(1); + const expectedInputs = computeInputsFingerprint([ + { path: "a.md", text: "claim A" }, + { path: "b.md", text: "neighbor content body" }, + ]); + expect(observedFp[0]).toMatchObject({ + inputs: expectedInputs, + principal: "agent:curation-loop", + model: baseOpts.model, + prompt: "birth/foundational", + }); + } finally { + cleanup(root); + } + }); + it("neighbor-premise → observe(from=doc, to=neighbor) with premiseVote 'to'", async () => { const root = tmpVault(); try { diff --git a/test/consolidate/independence.test.ts b/test/consolidate/independence.test.ts new file mode 100644 index 00000000..1670f293 --- /dev/null +++ b/test/consolidate/independence.test.ts @@ -0,0 +1,188 @@ +// Independence-aware promotion: the would-be verdict, the shadow journal, +// and the needs-review tension body (2026-07-26 spec, Decisions 3-4, PR-2). + +import { appendFileSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + appendIndependenceShadow, + classesForTension, + type IndependenceShadowRow, + independenceShadowPath, + independenceVerdict, + listIndependenceShadow, + needsReviewTensionInput, +} from "../../src/consolidate/independence.js"; +import { addTension, parseTensionLog } from "../../src/curation/tension.js"; + +function tmpVault(): string { + const root = join( + tmpdir(), + `daftari-independence-${process.pid}-${Math.random().toString(36).slice(2, 8)}`, + ); + mkdirSync(root, { recursive: true }); + mkdirSync(join(root, ".daftari"), { recursive: true }); + return root; +} + +function cleanup(root: string): void { + try { + rmSync(root, { recursive: true, force: true }); + } catch {} +} + +const baseRow: IndependenceShadowRow = { + at: "2026-07-27T00:00:00Z", + fromPath: "a.md", + toPath: "b.md", + kSurvived: 3, + kEff: 1.9375, + strength: 3, + strengthIndependent: 1.9375, + classes: [{ key: "h1\np1\nm1", count: 3 }], + panelClassKeys: ["h1\np1\nm1"], + marginalGain: 0.125, + wouldDecision: "would_needs_review", +}; + +describe("independenceVerdict — boundary math", () => { + it("empty preClasses: each fresh-class vote gains 1.0 → accrues", () => { + const r = independenceVerdict(new Map(), ["classA"]); + expect(r.marginalGain).toBeCloseTo(1, 6); + expect(r.wouldDecision).toBe("would_accrue"); + }); + + it("a second vote in a count-1 class gains exactly 0.5 → accrues (boundary inclusive)", () => { + const r = independenceVerdict(new Map([["classA", 1]]), ["classA"]); + expect(r.marginalGain).toBeCloseTo(0.5, 6); + expect(r.wouldDecision).toBe("would_accrue"); + }); + + it("a third vote in a count-2 class gains 0.25 → needs-review (strictly below 0.5)", () => { + const r = independenceVerdict(new Map([["classA", 2]]), ["classA"]); + expect(r.marginalGain).toBeCloseTo(0.25, 6); + expect(r.wouldDecision).toBe("would_needs_review"); + }); + + it("sequential same-panel votes discount against each other, not just against preClasses", () => { + // Two panel votes into an empty class: 1.0 (fresh) + 0.5 (second in-panel) = 1.5. + const r = independenceVerdict(new Map(), ["classA", "classA"]); + expect(r.marginalGain).toBeCloseTo(1.5, 6); + expect(r.wouldDecision).toBe("would_accrue"); + }); + + it("a fresh class always accrues regardless of other classes' saturation", () => { + const r = independenceVerdict(new Map([["classA", 5]]), ["classB"]); + expect(r.marginalGain).toBeCloseTo(1, 6); + expect(r.wouldDecision).toBe("would_accrue"); + }); +}); + +describe("independence shadow journal", () => { + let vault: string; + beforeEach(() => { + vault = tmpVault(); + }); + afterEach(() => { + cleanup(vault); + }); + + it("append/read round-trips a row", async () => { + const res = await appendIndependenceShadow(vault, baseRow); + expect(res.ok).toBe(true); + const listed = await listIndependenceShadow(vault); + expect(listed.ok).toBe(true); + if (!listed.ok) return; + expect(listed.value).toHaveLength(1); + expect(listed.value[0]).toEqual(baseRow); + }); + + it("a missing log reads as empty, not an error", async () => { + const listed = await listIndependenceShadow(vault); + expect(listed.ok).toBe(true); + if (listed.ok) expect(listed.value).toEqual([]); + }); + + it("skips a corrupt line without failing the whole read", async () => { + await appendIndependenceShadow(vault, baseRow); + appendFileSync(independenceShadowPath(vault), "not json at all\n"); + await appendIndependenceShadow(vault, { ...baseRow, toPath: "c.md" }); + const listed = await listIndependenceShadow(vault); + expect(listed.ok).toBe(true); + if (!listed.ok) return; + expect(listed.value).toHaveLength(2); + expect(listed.value.map((r) => r.toPath)).toEqual(["b.md", "c.md"]); + }); + + it("a wholly malformed jsonl file yields an empty list, not a thrown error", async () => { + writeFileSync(independenceShadowPath(vault), "{{{not json\n\n \n"); + const listed = await listIndependenceShadow(vault); + expect(listed.ok).toBe(true); + if (listed.ok) expect(listed.value).toEqual([]); + }); +}); + +describe("needsReviewTensionInput — the class breakdown (C6)", () => { + it("claimA is a single line, never a raw newline-joined class key", () => { + const classes = classesForTension(new Map([["h1234567890abcdef\np1\nm1", 3]])); + const input = needsReviewTensionInput("a.md", "b.md", classes); + expect(input.claimA).not.toContain("\n"); + expect(input.title).toBe("correlated-only survival: a.md derives_from b.md"); + expect(input.kind).toBe("interpretive"); + expect(input.sourceA).toBe("a.md"); + expect(input.sourceB).toBe("b.md"); + }); + + it("renders ∅ components as 'unfingerprinted' and truncates inputs to a 12-hex prefix", () => { + // classesForTension decodes the sentinel "∅" (via evidenceClassKey) to null. + const classes = classesForTension(new Map([["∅\n∅\n∅", 5]])); + const input = needsReviewTensionInput("a.md", "b.md", classes); + expect(input.claimA).toContain("unfingerprinted"); + expect(input.claimA).not.toContain("∅"); + expect(input.claimA).toBe( + "1 classes over 5 counted votes — class 1 ×5: model=unfingerprinted, principal=unfingerprinted, inputs=unfingerprinted", + ); + }); + + it("truncates a real sha256 inputs hash to a 12-char prefix and reports N classes over M votes", () => { + const hash = "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567"; + const classes = classesForTension( + new Map([ + [`${hash}\np1\nm1`, 2], + ["∅\np2\nm2", 1], + ]), + ); + const input = needsReviewTensionInput("a.md", "b.md", classes); + expect(input.claimA).toContain("2 classes over 3 counted votes"); + expect(input.claimA).toContain(`inputs=${hash.slice(0, 12)}`); + expect(input.claimA).not.toContain(hash.slice(0, 13)); + }); + + it("the emitted tension round-trips byte-stable through addTension render + parse", async () => { + const vault = tmpVault(); + try { + const hash = "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567"; + const classes = classesForTension( + new Map([ + [`${hash}\nagent:curation-loop\nclaude-haiku`, 3], + ["∅\n∅\n∅", 2], + ]), + ); + const input = needsReviewTensionInput("pricing/plan.md", "research/basis.md", classes); + const added = await addTension(vault, input); + expect(added.ok).toBe(true); + if (!added.ok) return; + const raw = readFileSync(join(vault, ".daftari", "tensions.md"), "utf-8"); + const parsed = parseTensionLog(raw); + expect(parsed).toHaveLength(1); + expect(parsed[0]?.claimA).toBe(added.value.claimA); + expect(parsed[0]?.title).toBe(input.title); + expect(parsed[0]?.kind).toBe("interpretive"); + expect(parsed[0]?.sourceA).toBe("pricing/plan.md"); + expect(parsed[0]?.sourceB).toBe("research/basis.md"); + } finally { + cleanup(vault); + } + }); +}); diff --git a/test/consolidate/index-stage2.test.ts b/test/consolidate/index-stage2.test.ts index 600477fc..bf607309 100644 --- a/test/consolidate/index-stage2.test.ts +++ b/test/consolidate/index-stage2.test.ts @@ -356,6 +356,90 @@ describe("--report=decorrelation", () => { }); }); +describe("Stage 2 dispatch — revision, independence-aware promotion (Decisions 3-4)", () => { + it("shadowed default (independence_graduated unset): a due edge still panels and accrues, needs_review_emitted reported as would-be", async () => { + process.env.ANTHROPIC_API_KEY = "test-key"; + writeFileSync(join(dir, ".daftari", "config.yaml"), "version: 1\nshadow_mode: true\n"); + const { observeEdge } = await import("../../src/curation/edges.js"); + const past = new Date(Date.now() - 100 * 86_400_000).toISOString(); + await observeEdge(dir, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: false, + at: past, + }); + await observeEdge(dir, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: true, + axis: "prompt", + at: past, + }); + + const { out } = captureStdout(); + const code = await runConsolidate(["--vault", dir, "--mode", "revision"]); + expect([0, 4]).toContain(code); + const text = out.join(""); + expect(text).toMatch(/panels_cast: 1/); + // Not graduated: the report suffixes needs_review_emitted as would-be and + // never reports panels_skipped_needs_review (that line is graduated-only). + expect(text).toMatch(/needs_review_emitted: \d+ \(shadowed — would-be\)/); + expect(text).not.toMatch(/panels_skipped_needs_review/); + }); + + it("graduated + a due edge parked by an open needs-review tension: zero LLM calls, panel skipped", async () => { + process.env.ANTHROPIC_API_KEY = "test-key"; + writeFileSync( + join(dir, ".daftari", "config.yaml"), + "version: 1\nshadow_mode: true\nindependence_graduated: true\n", + ); + const { observeEdge } = await import("../../src/curation/edges.js"); + const past = new Date(Date.now() - 100 * 86_400_000).toISOString(); + // Seed a due edge a.md -> b.md (no premiseVote given, so the seed's own + // orientation — fromPath=a.md (dependent), toPath=b.md (premise) — is the + // output orientation the title must match). + await observeEdge(dir, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: false, + at: past, + }); + await observeEdge(dir, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: true, + axis: "prompt", + at: past, + }); + // An OPEN needs-review tension already parks this exact edge. + const t = await addTension(dir, { + title: "correlated-only survival: a.md derives_from b.md", + kind: "interpretive", + sourceA: "a.md", + claimA: + "1 classes over 1 counted votes — class 1 x1: model=unfingerprinted, principal=unfingerprinted, inputs=unfingerprinted", + sourceB: "b.md", + claimB: "survives re-derivation only on already-counted evidence", + loggedBy: "agent:curation-loop", + }); + expect(t.ok).toBe(true); + + const { out } = captureStdout(); + const code = await runConsolidate(["--vault", dir, "--mode", "revision"]); + expect([0, 4]).toContain(code); + const text = out.join(""); + expect(text).toMatch(/panels_cast: 0/); + expect(text).toMatch(/llm_calls: 0/); + expect(text).toMatch(/panels_skipped_needs_review: 1/); + // Would-be/real emissions counter stays at 0 — the panel never ran. + expect(text).toMatch(/needs_review_emitted: 0/); + }); +}); + describe("Stage 2 — trace failure → exit 5", () => { it("if the trace cannot be written (read-only .daftari), exit code is 5", async () => { process.env.ANTHROPIC_API_KEY = "test-key"; diff --git a/test/consolidate/revision.test.ts b/test/consolidate/revision.test.ts index e9cfb783..d3e4bbb3 100644 --- a/test/consolidate/revision.test.ts +++ b/test/consolidate/revision.test.ts @@ -12,6 +12,7 @@ import { type RevisionOpts, revisionPanel, } from "../../src/consolidate/revision.js"; +import { computeInputsFingerprint, evidenceClassKey } from "../../src/curation/edges.js"; import type { LlmClient } from "../../src/eval/llm.js"; import { ok } from "../../src/frontmatter/types.js"; @@ -55,6 +56,7 @@ const baseOpts: RevisionOpts = { panelSize: 2, budgetRemaining: 100, model: "claude-haiku-test", + independenceGraduated: false, }; // Default: the envelope always admits. Refusing tests pass their own `admit`. @@ -65,6 +67,20 @@ const ADMIT_OK: RevisionDeps["admit"] = async () => ({ impact: 0, }); +// Independence-aware promotion (Decisions 3-4) default stubs: an empty +// pre-panel class map (so the panel behaves as before PR-2 unless a test +// overrides `independenceGraduated` or the class read) and no-op journal / +// tension recorders. Spread first in each `RevisionDeps` literal so a test +// can still override any of the three. +const DEFAULT_INDEPENDENCE_DEPS: Pick< + RevisionDeps, + "getEvidenceClasses" | "recordIndependenceShadow" | "recordNeedsReviewTension" +> = { + getEvidenceClasses: async () => ok(new Map()), + recordIndependenceShadow: async () => ok(undefined), + recordNeedsReviewTension: async () => ok(undefined), +}; + const dueEdge = { fromPath: "a.md", toPath: "b.md", @@ -102,6 +118,7 @@ describe("revisionPanel — majority decides, once", () => { const observed: Array<{ axis?: string }> = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "ok" }, @@ -132,12 +149,55 @@ describe("revisionPanel — majority decides, once", () => { } }); + it("each surviving observe carries an fp whose inputs hash the truncated endpoint bodies", async () => { + const root = tmpVault(); + try { + const observed: Array<{ + fp?: { inputs?: string; principal?: string; model?: string; prompt?: string }; + }> = []; + const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `[content of ${p}]` }), + observe: async (input) => { + observed.push({ fp: input.fp }); + return ok({ ...dueEdge }); + }, + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + }; + const r = await revisionPanel(dueEdge, deps, { ...baseOpts, vaultRoot: root }); + if (!r.ok) throw r.error; + expect(observed.length).toBe(2); + const expectedInputs = computeInputsFingerprint([ + { path: "a.md", text: "[content of a.md]" }, + { path: "b.md", text: "[content of b.md]" }, + ]); + for (const o of observed) { + expect(o.fp?.inputs).toBe(expectedInputs); + expect(o.fp?.principal).toBe("agent:curation-loop"); + expect(o.fp?.model).toBe(baseOpts.model); + } + // Same inputs hash across both votes (same panel, same docs); the + // template id is what makes the two fp.prompt values distinct. + expect(observed[0]?.fp?.prompt).toMatch(/^revision\//); + expect(observed[1]?.fp?.prompt).not.toBe(observed[0]?.fp?.prompt); + } finally { + cleanup(root); + } + }); + it("M=2 split (1 survive, 1 fail) is a TIE → no write, no revoke/reseed churn", async () => { const root = tmpVault(); try { const observed: unknown[] = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "still ok" }, @@ -170,6 +230,7 @@ describe("revisionPanel — majority decides, once", () => { const observed: unknown[] = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "fails", reason: "no link" }, @@ -204,6 +265,7 @@ describe("revisionPanel — majority decides, once", () => { const observed: unknown[] = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "ok" }, @@ -239,6 +301,7 @@ describe("revisionPanel — envelope admit (gate consulted once per panel decisi const observed: unknown[] = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: async () => ({ admit: false, gate: "budget" as const, @@ -277,6 +340,7 @@ describe("revisionPanel — envelope admit (gate consulted once per panel decisi const observed: unknown[] = []; const contests: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: async () => ({ admit: false, gate: "invariants" as const, @@ -315,6 +379,7 @@ describe("revisionPanel — envelope admit (gate consulted once per panel decisi try { const observed: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: async () => { throw new Error("makeAdmit fs error"); }, @@ -347,6 +412,7 @@ describe("revisionPanel — envelope admit (gate consulted once per panel decisi try { let admitCalls = 0; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: async () => { admitCalls++; return ADMIT_OK({ action: "edge-observe", fromPath: "", toPath: "" }); @@ -397,6 +463,7 @@ describe("revisionPanel — independence by axis (§11.3 replay-gap)", () => { completeWithTools: vi.fn(), }; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm, loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), @@ -419,6 +486,7 @@ describe("revisionPanel — budget + stop", () => { const root = tmpVault(); try { const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "1" }, @@ -447,6 +515,7 @@ describe("revisionPanel — budget + stop", () => { const root = tmpVault(); try { const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "1" }, @@ -478,6 +547,7 @@ describe("revisionPanel — trace", () => { try { const rows: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([ { verdict: "survives", reason: "1" }, @@ -512,6 +582,7 @@ describe("revisionPanel — trace", () => { try { const observed: unknown[] = []; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([{ verdict: "survives", reason: "x" }]), loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), @@ -545,6 +616,7 @@ describe("revisionPanel — path canonicalization", () => { toPath: "research/./b.md", }; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([{ verdict: "survives", reason: "x" }]), loadDoc: async (p) => { @@ -572,6 +644,7 @@ describe("revisionPanel — write failure post-vote (observe/contest disk error) const root = tmpVault(); try { const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm: mockLlm([{ verdict: "survives", reason: "ok" }]), loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), @@ -620,6 +693,7 @@ describe("revisionPanel — LLM failures", () => { completeWithTools: vi.fn(), }; const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, admit: ADMIT_OK, llm, loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), @@ -641,3 +715,260 @@ describe("revisionPanel — LLM failures", () => { } }); }); + +describe("revisionPanel — independence-aware promotion (Decisions 3-4)", () => { + const panelClassKey = evidenceClassKey({ + inputs: computeInputsFingerprint([ + { path: "a.md", text: "[content of a.md]" }, + { path: "b.md", text: "[content of b.md]" }, + ]), + principal: "agent:curation-loop", + model: "claude-haiku-test", + }); + + it("graduated + correlated-only panel → needs-review, zero observes, one tension, trace carries the decision", async () => { + const root = tmpVault(); + try { + const observed: unknown[] = []; + const tensions: Array<{ title: string; kind: string }> = []; + const journalRows: Array<{ wouldDecision: string | null; marginalGain: number }> = []; + const traceRows: Array<{ + decision: string; + independence?: { wouldDecision: string | null }; + }> = []; + const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `[content of ${p}]` }), + observe: async (input) => { + observed.push(input); + return ok({ ...dueEdge }); + }, + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async (r) => { + traceRows.push(r); + return ok(undefined); + }, + // Two prior counted votes already in this exact class: the panel's + // survivors land at priorCount 2 and 3 → gains 0.25 + 0.125, both + // sub-boundary. + getEvidenceClasses: async () => ok(new Map([[panelClassKey, 2]])), + recordIndependenceShadow: async (r) => { + journalRows.push(r); + return ok(undefined); + }, + recordNeedsReviewTension: async (input) => { + tensions.push({ title: input.title, kind: input.kind }); + return ok(undefined); + }, + }; + const r = await revisionPanel(dueEdge, deps, { + ...baseOpts, + vaultRoot: root, + independenceGraduated: true, + }); + if (!r.ok) throw r.error; + expect(r.value.decision).toBe("needs-review"); + expect(r.value.observedCount).toBe(0); + expect(observed.length).toBe(0); + expect(tensions).toEqual([ + { title: "correlated-only survival: a.md derives_from b.md", kind: "interpretive" }, + ]); + expect(journalRows.length).toBe(1); + expect(journalRows[0]?.wouldDecision).toBe("would_needs_review"); + expect(traceRows.length).toBe(1); + expect(traceRows[0]?.decision).toBe("needs-review"); + expect(traceRows[0]?.independence?.wouldDecision).toBe("would_needs_review"); + expect(r.value.independenceWouldNeedsReview).toBe(true); + } finally { + cleanup(root); + } + }); + + it("graduated + fresh-class panel (no pre-existing classes) → accrues exactly as today", async () => { + const root = tmpVault(); + try { + const observed: unknown[] = []; + const tensions: unknown[] = []; + const journalRows: Array<{ wouldDecision: string | null }> = []; + const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `[content of ${p}]` }), + observe: async (input) => { + observed.push(input); + return ok({ ...dueEdge }); + }, + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + getEvidenceClasses: async () => ok(new Map()), + recordIndependenceShadow: async (r) => { + journalRows.push(r); + return ok(undefined); + }, + recordNeedsReviewTension: async (input) => { + tensions.push(input); + return ok(undefined); + }, + }; + const r = await revisionPanel(dueEdge, deps, { + ...baseOpts, + vaultRoot: root, + independenceGraduated: true, + }); + if (!r.ok) throw r.error; + expect(r.value.decision).toBe("survives"); + expect(observed.length).toBe(2); + expect(tensions.length).toBe(0); + expect(journalRows.length).toBe(1); + expect(journalRows[0]?.wouldDecision).toBe("would_accrue"); + expect(r.value.independenceWouldNeedsReview).toBe(false); + } finally { + cleanup(root); + } + }); + + it("shadowed default: a correlated-only panel still accrues (writes are exactly today's), but the journal + would-be flag see the correlation", async () => { + const root = tmpVault(); + try { + const observed: unknown[] = []; + const journalRows: Array<{ wouldDecision: string | null }> = []; + const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `[content of ${p}]` }), + observe: async (input) => { + observed.push(input); + return ok({ ...dueEdge }); + }, + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + getEvidenceClasses: async () => ok(new Map([[panelClassKey, 2]])), + recordIndependenceShadow: async (r) => { + journalRows.push(r); + return ok(undefined); + }, + }; + // independenceGraduated: false (baseOpts default) — decision and writes + // are exactly today's two-way verdict. + const r = await revisionPanel(dueEdge, deps, { ...baseOpts, vaultRoot: root }); + if (!r.ok) throw r.error; + expect(r.value.decision).toBe("survives"); + expect(observed.length).toBe(2); + expect(r.value.independenceWouldNeedsReview).toBe(true); // would-be, reported not acted on + expect(journalRows[0]?.wouldDecision).toBe("would_needs_review"); + } finally { + cleanup(root); + } + }); + + it("one independence-shadow journal row per panel, including tie and gated (wouldDecision null)", async () => { + const root = tmpVault(); + try { + const journalRows: Array<{ wouldDecision: string | null }> = []; + const recordIndependenceShadow: RevisionDeps["recordIndependenceShadow"] = async (r) => { + journalRows.push(r); + return ok(undefined); + }; + + // Tie: no majority, nothing written. + const tieDeps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "fails", reason: "no" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), + observe: async () => ok({ ...dueEdge }), + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + recordIndependenceShadow, + }; + const tieRes = await revisionPanel(dueEdge, tieDeps, { ...baseOpts, vaultRoot: root }); + if (!tieRes.ok) throw tieRes.error; + expect(tieRes.value.decision).toBe("tie"); + + // Gated: majority survives, envelope refuses. + const gatedDeps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: async () => ({ + admit: false, + gate: "budget" as const, + reason: "trust-budget exhausted", + impact: 0.05, + }), + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), + observe: async () => ok({ ...dueEdge }), + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + recordIndependenceShadow, + }; + const gatedRes = await revisionPanel(dueEdge, gatedDeps, { ...baseOpts, vaultRoot: root }); + if (!gatedRes.ok) throw gatedRes.error; + expect(gatedRes.value.decision).toBe("gated"); + + expect(journalRows.length).toBe(2); + for (const row of journalRows) expect(row.wouldDecision).toBeNull(); + } finally { + cleanup(root); + } + }); + + it("a failed pre-panel classes read degrades to shadow-off: journal nothing, decision unaffected", async () => { + const root = tmpVault(); + try { + const observed: unknown[] = []; + let journalCalls = 0; + const deps: RevisionDeps = { + ...DEFAULT_INDEPENDENCE_DEPS, + admit: ADMIT_OK, + llm: mockLlm([ + { verdict: "survives", reason: "ok" }, + { verdict: "survives", reason: "still" }, + ]), + loadDoc: async (p) => ok({ path: p, content: `c-${p}` }), + observe: async (input) => { + observed.push(input); + return ok({ ...dueEdge }); + }, + contest: async () => ok({ ...dueEdge }), + recordRevisionTrace: async () => ok(undefined), + getEvidenceClasses: async () => ({ ok: false, error: new Error("edges.jsonl unreadable") }), + recordIndependenceShadow: async () => { + journalCalls++; + return ok(undefined); + }, + }; + const r = await revisionPanel(dueEdge, deps, { + ...baseOpts, + vaultRoot: root, + independenceGraduated: true, + }); + if (!r.ok) throw r.error; + // Live decision unaffected: still accrues normally. + expect(r.value.decision).toBe("survives"); + expect(observed.length).toBe(2); + expect(r.value.independenceJournalWriteFailure).toBe(true); + expect(journalCalls).toBe(0); // nothing journaled + } finally { + cleanup(root); + } + }); +}); diff --git a/test/context/assemble.test.ts b/test/context/assemble.test.ts new file mode 100644 index 00000000..8d3e7b1d --- /dev/null +++ b/test/context/assemble.test.ts @@ -0,0 +1,153 @@ +// Pure-function tests for src/context/assemble.ts (spec 2026-07-26-context- +// packs-progressive-disclosure-design.md, final plan Phase 2.7 / C6). +// +// No database, no fixture vault, no golden-brief byte pin — determinism is +// proven by calling the assembler twice on the same in-memory PackEntry[] +// and comparing outputs to each other (C6). + +import { describe, expect, it } from "vitest"; +import { assembleContextPack, type PackEntry } from "../../src/context/assemble.js"; + +function entry(overrides: Partial & Pick): PackEntry { + return { + title: overrides.path, + reason: "matches task", + snippet: "a short snippet", + ...overrides, + }; +} + +describe("assembleContextPack — determinism (C6)", () => { + it("calling twice on the same input produces byte-identical output", () => { + const entries: PackEntry[] = [ + entry({ path: "a.md", score: 0.9, snippet: "alpha" }), + entry({ path: "b.md", score: 0.5, snippet: "beta" }), + ]; + const a = assembleContextPack("do the thing", 4000, entries, "none"); + const b = assembleContextPack("do the thing", 4000, entries, "none"); + expect(a).toEqual(b); + expect(a.brief).toBe(b.brief); + }); +}); + +describe("assembleContextPack — budget cut", () => { + it("never returns a brief whose estimated tokens exceed the stated budget", () => { + const entries: PackEntry[] = Array.from({ length: 8 }, (_, i) => + entry({ path: `doc-${i}.md`, score: 8 - i, snippet: "x".repeat(300) }), + ); + for (const budget of [500, 1000, 2000, 4000]) { + const pack = assembleContextPack("task", budget, entries, "none"); + expect(pack.estimatedTokens, `budget=${budget}`).toBeLessThanOrEqual(budget); + } + }); + + it("a larger budget never includes fewer entries than a smaller one", () => { + const entries: PackEntry[] = Array.from({ length: 8 }, (_, i) => + entry({ path: `doc-${i}.md`, score: 8 - i, snippet: "x".repeat(300) }), + ); + const small = assembleContextPack("task", 500, entries, "none"); + const large = assembleContextPack("task", 20000, entries, "none"); + expect(large.manifest.included.length).toBeGreaterThanOrEqual(small.manifest.included.length); + }); + + it("omitted_over_budget accounts for every entry not included", () => { + const entries: PackEntry[] = Array.from({ length: 5 }, (_, i) => + entry({ path: `doc-${i}.md`, score: 5 - i, snippet: "x".repeat(500) }), + ); + const pack = assembleContextPack("task", 500, entries, "none"); + expect(pack.manifest.included.length + pack.manifest.omitted_over_budget).toBe(entries.length); + }); + + it("stops at the first entry that does not fit — no skip-ahead", () => { + const entries: PackEntry[] = [ + entry({ path: "fits.md", score: 3, snippet: "tiny" }), + // Score 2: ranks second, and its huge snippet blows the budget. + entry({ path: "huge.md", score: 2, snippet: "x".repeat(200_000) }), + // Score 1: ranks last, but its snippet is tiny too — would fit on its + // own if the walk skipped ahead past `huge.md`. It must not be + // included: the walk stops at the FIRST non-fitting entry. + entry({ path: "also-fits.md", score: 1, snippet: "tiny" }), + ]; + const pack = assembleContextPack("task", 500, entries, "none"); + expect(pack.manifest.included.map((e) => e.path)).toEqual(["fits.md"]); + expect(pack.manifest.omitted_over_budget).toBe(2); + }); +}); + +describe("assembleContextPack — degenerate outcomes (C9)", () => { + it("zero candidates (zero-hit task): body reads 'no matching documents'", () => { + const pack = assembleContextPack("task", 4000, [], "none"); + expect(pack.manifest.included).toEqual([]); + expect(pack.manifest.omitted_over_budget).toBe(0); + expect(pack.brief).toContain("No matching documents"); + }); + + it("candidates exist but none fit the budget: distinct body from the zero-hit case", () => { + const entries: PackEntry[] = [entry({ path: "a.md", score: 1, snippet: "x".repeat(200_000) })]; + const pack = assembleContextPack("task", 500, entries, "none"); + expect(pack.manifest.included).toEqual([]); + expect(pack.manifest.omitted_over_budget).toBe(1); + expect(pack.brief).toContain("Nothing fit the requested budget"); + expect(pack.brief).not.toContain("No matching documents"); + }); +}); + +describe("assembleContextPack — hidden_remainder disclosure", () => { + it("the scope line is absent when hidden_remainder is 'none'", () => { + const pack = assembleContextPack("task", 4000, [entry({ path: "a.md", score: 1 })], "none"); + expect(pack.brief).not.toContain("withheld outside your read scope"); + }); + + it("the scope line appears when hidden_remainder is non-'none', never an exact count", () => { + const pack = assembleContextPack("task", 4000, [entry({ path: "a.md", score: 1 })], "some"); + expect(pack.brief).toContain("some additional document(s) withheld outside your read scope"); + }); +}); + +describe("assembleContextPack — Decision 3 refusal, structurally enforced", () => { + it("a tension flag renders BOTH claims verbatim, never a blended sentence", () => { + const entries: PackEntry[] = [ + entry({ + path: "a.md", + score: 1, + tensions: [ + { + kind: "factual", + counterpart: "b.md", + claimSelf: "the deploy target is X", + claimOther: "the deploy target is Y", + }, + ], + contestedCount: 1, + }), + ]; + const pack = assembleContextPack("task", 4000, entries, "none"); + expect(pack.brief).toContain('this doc claims "the deploy target is X"'); + expect(pack.brief).toContain('b.md claims "the deploy target is Y"'); + // Never a resolved/composed verdict line. + expect(pack.brief.toLowerCase()).not.toContain("resolv"); + }); + + it("supersession prints only the pointer and hop count, not paraphrased content", () => { + const entries: PackEntry[] = [ + entry({ + path: "head.md", + score: 1, + snippet: "the head's own verbatim content", + supersedes: 3, + }), + ]; + const pack = assembleContextPack("task", 4000, entries, "none"); + expect(pack.brief).toContain("supersedes 3 older documents matching this task"); + expect(pack.brief).toContain("the head's own verbatim content"); + }); + + it("flag lines are absent-is-healthy: no flags means no flag bullet lines", () => { + const entries: PackEntry[] = [entry({ path: "a.md", score: 1 })]; + const pack = assembleContextPack("task", 4000, entries, "none"); + expect(pack.brief).not.toContain("- decay:"); + expect(pack.brief).not.toContain("- contested"); + expect(pack.brief).not.toContain("- structural:"); + expect(pack.brief).not.toContain("- upstream:"); + }); +}); diff --git a/test/context/estimate.test.ts b/test/context/estimate.test.ts new file mode 100644 index 00000000..fbf6ddac --- /dev/null +++ b/test/context/estimate.test.ts @@ -0,0 +1,21 @@ +import { describe, expect, it } from "vitest"; +import { estimateTokens } from "../../src/context/estimate.js"; + +describe("estimateTokens — chars/4, ceil", () => { + it("empty string is 0 tokens", () => { + expect(estimateTokens("")).toBe(0); + }); + + it("divides length by 4 and rounds up", () => { + expect(estimateTokens("a")).toBe(1); // 1/4 -> ceil -> 1 + expect(estimateTokens("abcd")).toBe(1); // 4/4 -> 1 + expect(estimateTokens("abcde")).toBe(2); // 5/4 -> ceil -> 2 + expect(estimateTokens("a".repeat(400))).toBe(100); + expect(estimateTokens("a".repeat(401))).toBe(101); + }); + + it("is deterministic across repeated calls", () => { + const s = "the quick brown fox jumps over the lazy dog"; + expect(estimateTokens(s)).toBe(estimateTokens(s)); + }); +}); diff --git a/test/context/no-court-import.test.ts b/test/context/no-court-import.test.ts new file mode 100644 index 00000000..cd2d1f1d --- /dev/null +++ b/test/context/no-court-import.test.ts @@ -0,0 +1,32 @@ +// Tripwire: src/context/ and src/tools/context.ts must never import from +// src/court/ (CLAUDE.md: "the Tension Court is an operator-only surface; +// court/docket code never takes an access context. Exposing any court +// surface via MCP requires revisiting the 2026-07-14 edge-graph spec +// first." vault_context IS an MCP-exposed, access-context-carrying read +// path — a static import check, not a convention, keeps that boundary +// honest as the module grows.) + +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; + +const REPO_ROOT = join(__dirname, "..", ".."); +const GUARDED_FILES = [ + join(REPO_ROOT, "src", "context", "estimate.ts"), + join(REPO_ROOT, "src", "context", "assemble.ts"), + join(REPO_ROOT, "src", "tools", "context.ts"), +]; + +describe("court import tripwire", () => { + for (const file of GUARDED_FILES) { + it(`${file.replace(REPO_ROOT, "")} imports nothing from src/court/`, () => { + const text = readFileSync(file, "utf-8"); + const importLines = text + .split("\n") + .filter((line) => /^\s*import\b/.test(line) || /^\s*export\s+.*\bfrom\b/.test(line)); + for (const line of importLines) { + expect(line, `${file} imports from src/court/: ${line}`).not.toMatch(/["'].*\/court\//); + } + }); + } +}); diff --git a/test/curation/coverage.test.ts b/test/curation/coverage.test.ts index ac19a9df..4ab8d4a4 100644 --- a/test/curation/coverage.test.ts +++ b/test/curation/coverage.test.ts @@ -342,4 +342,14 @@ describe("monitor-never-target invariant", () => { } expect(offenders).toEqual([]); }); + + // 2026-07-26 independence-aware-promotion spec, PR-3 2.3: the coverage + // monitor stays decoupled from the independence-aware-promotion loop's + // action-mix specifics — it does not need src/consolidate/independence.js + // to do its job, and importing it would blur the monitor-not-target line + // this describe block is named for. + it("coverage.ts does not import src/consolidate/independence.ts", () => { + const src = readFileSync(join(process.cwd(), "src", "curation", "coverage.ts"), "utf8"); + expect(/from\s+["'][^"']*consolidate\/independence(\.js)?["']/.test(src)).toBe(false); + }); }); diff --git a/test/curation/edges.test.ts b/test/curation/edges.test.ts index 066e7f05..84bb97c9 100644 --- a/test/curation/edges.test.ts +++ b/test/curation/edges.test.ts @@ -3,10 +3,16 @@ import { join } from "node:path"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { agedStrength, + computeInputsFingerprint, contestEdge, + EDGE_CALIBRATION_LOOP_PRINCIPAL, EDGE_K_CAP, + edgeEvidenceClasses, edgesPath, + effectiveK, + evidenceClassKey, getEdge, + independenceCalibrationView, listEdges, observeEdge, rebuildEdgesIndex, @@ -260,7 +266,7 @@ describe("derives_from edge store", () => { }); mkdirSync(join(vault, ".daftari"), { recursive: true }); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -573,7 +579,7 @@ describe("derives_from direction verdict", () => { at: T0, }); mkdirSync(join(vault, ".daftari"), { recursive: true }); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -601,7 +607,7 @@ describe("sql-authoritative edge reads", () => { }); function withDb(fn: (db: IndexDb) => T): T { - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -742,3 +748,284 @@ describe("sql-authoritative edge reads", () => { expect(got.value?.contestReason).toBe("case-2 contradiction"); }); }); + +describe("independence-aware promotion: fingerprint + effective k (Decisions 1-2)", () => { + let vault: string; + beforeEach(() => { + vault = makeTempVault(); + }); + afterEach(() => { + cleanupVault(vault); + }); + + it("effectiveK: geometric discount within a class, full credit across classes", () => { + // Single class of 5 (all-legacy): 1 + .5 + .25 + .125 + .0625 = 1.9375. + expect(effectiveK([5])).toBeCloseTo(1.9375, 6); + // Five distinct classes: full credit each, kEff = 5. + expect(effectiveK([1, 1, 1, 1, 1])).toBeCloseTo(5, 6); + // Empty trail. + expect(effectiveK([])).toBe(0); + }); + + it("evidenceClassKey: prompt is excluded; a missing component is the ∅ sentinel", () => { + const a = evidenceClassKey({ inputs: "h1", principal: "p1", model: "m1", prompt: "forward" }); + const b = evidenceClassKey({ inputs: "h1", principal: "p1", model: "m1", prompt: "reverse" }); + expect(a).toBe(b); // prompt-only variation never opens a class + const legacy1 = evidenceClassKey(undefined); + const legacy2 = evidenceClassKey({}); + expect(legacy1).toBe(legacy2); + expect(legacy1).not.toBe(a); + }); + + it("computeInputsFingerprint is order-independent over the (path, content) set", () => { + const h1 = computeInputsFingerprint([ + { path: "a.md", text: "alpha" }, + { path: "b.md", text: "beta" }, + ]); + const h2 = computeInputsFingerprint([ + { path: "b.md", text: "beta" }, + { path: "a.md", text: "alpha" }, + ]); + expect(h1).toBe(h2); + const h3 = computeInputsFingerprint([ + { path: "a.md", text: "ALPHA" }, + { path: "b.md", text: "beta" }, + ]); + expect(h3).not.toBe(h1); + }); + + it("seeds no evidence class (birth is not a survival)", async () => { + const seeded = await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: T0, + fp: { inputs: "h1", principal: "human:x", model: "m1" }, + }); + expect(seeded.ok).toBe(true); + if (!seeded.ok) return; + expect(seeded.value.kEff).toBe(0); + const classes = edgeEvidenceClasses(vault, "a.md", "b.md"); + expect(classes.ok && classes.value.size).toBe(0); + }); + + it("legacy no-fp votes collapse into a single ∅ class (kEff of an all-legacy k=5 trail)", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + let lastAt = T0; + for (let i = 0; i < 5; i++) { + lastAt = `2026-01-0${i + 2}T00:00:00Z`; + const r = await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: lastAt, + // No fp: legacy vote. + }); + if (!r.ok) throw r.error; + } + const edge = await getEdge(vault, "a.md", "b.md", new Date(lastAt)); + expect(edge.ok).toBe(true); + if (!edge.ok || !edge.value) return; + expect(edge.value.kSurvived).toBe(5); + expect(edge.value.kEff).toBeCloseTo(1.9375, 6); + const view = independenceCalibrationView(vault, new Date(lastAt)); + expect(view.ok).toBe(true); + if (!view.ok) return; + const row = view.value.find((r) => r.fromPath === "a.md" && r.toPath === "b.md"); + expect(row?.classCount).toBe(1); + expect(row?.unfingerprintedCountedVotes).toBe(5); + expect(row?.nonLoopFingerprintedCountedVotes).toBe(0); + }); + + it("five distinct-class votes give kEff = 5 (full credit across classes)", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + for (let i = 0; i < 5; i++) { + const r = await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: `2026-01-0${i + 2}T00:00:00Z`, + fp: { inputs: `h${i}`, principal: "agent:curation-loop", model: "m1" }, + }); + if (!r.ok) throw r.error; + } + const edge = await getEdge(vault, "a.md", "b.md", new Date("2026-01-06T00:00:00Z")); + expect(edge.ok).toBe(true); + if (!edge.ok || !edge.value) return; + expect(edge.value.kEff).toBeCloseTo(5, 6); + }); + + it("class map resets on re-seed after a contest", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: T1, + fp: { inputs: "h1", principal: "human:x", model: "m1" }, + }); + await contestEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + contestedBy: BY, + reason: "no upstream change", + at: T2, + }); + const reseeded = await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: "2026-01-04T00:00:00Z", + }); + expect(reseeded.ok).toBe(true); + if (!reseeded.ok) return; + expect(reseeded.value.kEff).toBe(0); + const classes = edgeEvidenceClasses(vault, "a.md", "b.md"); + expect(classes.ok && classes.value.size).toBe(0); + }); + + it("rejects an fp component containing a newline", async () => { + const r = await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + fp: { model: "line1\nline2" }, + }); + expect(r.ok).toBe(false); + if (!r.ok) expect(r.error.message).toMatch(/fp\.model/); + }); + + it("nonLoopFingerprintedCountedVotes counts only non-loop fp principals", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: T1, + fp: { inputs: "h1", principal: EDGE_CALIBRATION_LOOP_PRINCIPAL, model: "m1" }, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "model", + at: T2, + fp: { inputs: "h2", principal: "human:mihir", model: "m1" }, + }); + const view = independenceCalibrationView(vault, new Date(T2)); + expect(view.ok).toBe(true); + if (!view.ok) return; + const row = view.value.find((r) => r.fromPath === "a.md" && r.toPath === "b.md"); + expect(row?.countedVotes).toBe(2); + expect(row?.nonLoopFingerprintedCountedVotes).toBe(1); + expect(row?.unfingerprintedCountedVotes).toBe(0); + }); + + it("k_eff survives the sqlite round trip and matches the degraded log path", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: T1, + fp: { inputs: "h1", principal: "human:x", model: "m1" }, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "model", + at: T2, + fp: { inputs: "h1", principal: "human:x", model: "m1" }, // same class, second vote + }); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); + expect(opened.ok).toBe(true); + if (!opened.ok) return; + const db: IndexDb = opened.value; + try { + const rebuilt = rebuildEdgesIndex(db, vault, new Date(T2)); + expect(rebuilt.ok).toBe(true); + const rows = getAllDerivesFromEdges(db); + const row = rows.find((r) => r.from_path === "a.md" && r.to_path === "b.md"); + expect(row).toBeDefined(); + expect(row?.k_eff).toBeCloseTo(1.5, 6); // 1 + 0.5 within one class of 2 + } finally { + db.close(); + } + const viaLog = independenceCalibrationView(vault, new Date(T2)); + expect(viaLog.ok).toBe(true); + if (!viaLog.ok) return; + const row = viaLog.value.find((r) => r.fromPath === "a.md" && r.toPath === "b.md"); + expect(row?.kEff).toBeCloseTo(1.5, 6); + }); + + it("strengthIndependent ages on the same clock as strength", async () => { + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: false, + at: T0, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: BY, + blind: true, + axis: "prompt", + at: T1, + fp: { inputs: "h1", principal: "human:x", model: "m1" }, + }); + const fresh = await getEdge(vault, "a.md", "b.md", new Date(T1)); + expect(fresh.ok && fresh.value?.strengthIndependent).toBeCloseTo(1, 5); + const atHalf = await getEdge(vault, "a.md", "b.md", DAYS_90); + expect(atHalf.ok && atHalf.value?.strengthIndependent).toBeCloseTo(0.5, 5); + }); +}); diff --git a/test/curation/independence-calibration.test.ts b/test/curation/independence-calibration.test.ts new file mode 100644 index 00000000..b5047099 --- /dev/null +++ b/test/curation/independence-calibration.test.ts @@ -0,0 +1,215 @@ +// Independence-aware promotion — the vault_lint calibration section +// (2026-07-26 spec, Decision 4, PR-3). + +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { IndependenceShadowRow } from "../../src/consolidate/independence.js"; +import { appendIndependenceShadow } from "../../src/consolidate/independence.js"; +import type { EdgeIndependenceRow } from "../../src/curation/edges.js"; +import { observeEdge } from "../../src/curation/edges.js"; +import { + emptyIndependenceCalibrationSummary, + independenceCalibrationSummaryOf, +} from "../../src/curation/independence-calibration.js"; +import { runLint } from "../../src/curation/lint.js"; +import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; + +function row(overrides: Partial = {}): EdgeIndependenceRow { + return { + fromPath: "a.md", + toPath: "b.md", + kSurvived: 0, + kEff: 0, + strength: 0, + strengthIndependent: 0, + classCount: 0, + countedVotes: 0, + unfingerprintedCountedVotes: 0, + nonLoopFingerprintedCountedVotes: 0, + status: "candidate", + ...overrides, + }; +} + +function shadowRow(overrides: Partial = {}): IndependenceShadowRow { + return { + at: "2026-07-27T00:00:00Z", + fromPath: "a.md", + toPath: "b.md", + kSurvived: 0, + kEff: 0, + strength: 0, + strengthIndependent: 0, + classes: [], + panelClassKeys: [], + marginalGain: 0, + wouldDecision: null, + ...overrides, + }; +} + +describe("independenceCalibrationSummaryOf — pure aggregation", () => { + it("zeroes on empty inputs", () => { + const summary = independenceCalibrationSummaryOf([], []); + expect(summary).toEqual(emptyIndependenceCalibrationSummary()); + }); + + it("k vs k_eff distribution: scoped to edges WITH counted votes; edges with none are excluded", () => { + const view = [ + row({ kSurvived: 3, kEff: 1.9375, countedVotes: 3, unfingerprintedCountedVotes: 3 }), + row({ fromPath: "x.md", toPath: "y.md", kSurvived: 0, kEff: 0, countedVotes: 0 }), // no votes + ]; + const summary = independenceCalibrationSummaryOf(view, []); + expect(summary.kVsKEff.edgesWithVotes).toBe(1); + expect(summary.kVsKEff.meanK).toBeCloseTo(3, 6); + expect(summary.kVsKEff.meanKEff).toBeCloseTo(1.9375, 6); + expect(summary.kVsKEff.medianKEff).toBeCloseTo(1.9375, 6); + expect(summary.kVsKEff.kEffBelowKCount).toBe(1); // 1.9375 < 3 + }); + + it("wouldDropBelowTrigger: counts trigger-bearing-on-strength edges that drop under strengthIndependent, splitting legacy-only", () => { + const view = [ + // All-legacy: every counted vote unfingerprinted → legacy-only. + row({ + fromPath: "a.md", + toPath: "b.md", + strength: 0.6, + strengthIndependent: 0.3, + countedVotes: 3, + unfingerprintedCountedVotes: 3, + }), + // Partially fingerprinted: drops too, but NOT legacy-only. + row({ + fromPath: "c.md", + toPath: "d.md", + strength: 0.7, + strengthIndependent: 0.2, + countedVotes: 3, + unfingerprintedCountedVotes: 1, + }), + // Stays trigger-bearing under strengthIndependent too — not counted. + row({ + fromPath: "e.md", + toPath: "f.md", + strength: 2, + strengthIndependent: 1, + countedVotes: 2, + unfingerprintedCountedVotes: 0, + }), + // Was never trigger-bearing on raw strength — not counted even though + // strengthIndependent is also below the floor. + row({ + fromPath: "g.md", + toPath: "h.md", + strength: 0.2, + strengthIndependent: 0.1, + countedVotes: 1, + unfingerprintedCountedVotes: 1, + }), + ]; + const summary = independenceCalibrationSummaryOf(view, []); + expect(summary.wouldDropBelowTrigger.count).toBe(2); + expect(summary.wouldDropBelowTrigger.legacyOnlyCount).toBe(1); + }); + + it("wouldNeedsReviewRate: raw rate over decided rows; informative rate excludes rows whose pre-panel classes are all-∅ (C5)", () => { + const journal: IndependenceShadowRow[] = [ + // A legacy edge's FIRST fingerprinted panel: pre-panel classes are + // all-∅ (never fingerprinted before) — not informative, even though + // decided would_accrue (a fresh class always accrues). + shadowRow({ + classes: [{ key: "∅\n∅\n∅", count: 3 }], + wouldDecision: "would_accrue", + }), + // An edge with a genuinely fingerprinted pre-panel class → informative. + shadowRow({ + fromPath: "c.md", + toPath: "d.md", + classes: [{ key: "hash123\nagent:curation-loop\nclaude-haiku", count: 2 }], + wouldDecision: "would_needs_review", + }), + // Not decided (tie/gated) — excluded from both denominators. + shadowRow({ fromPath: "e.md", toPath: "f.md", classes: [], wouldDecision: null }), + ]; + const summary = independenceCalibrationSummaryOf([], journal); + expect(summary.wouldNeedsReviewRate.decidedCount).toBe(2); + expect(summary.wouldNeedsReviewRate.needsReviewCount).toBe(1); + expect(summary.wouldNeedsReviewRate.rate).toBeCloseTo(0.5, 6); + // Informative excludes the all-∅ row: denominator 1, not 2. + expect(summary.wouldNeedsReviewRate.informativePanels).toBe(1); + expect(summary.wouldNeedsReviewRate.informativeNeedsReviewCount).toBe(1); + expect(summary.wouldNeedsReviewRate.rateInformative).toBeCloseTo(1, 6); + }); + + it("legacyUnfingerprintedFraction: edges with all-unfingerprinted counted votes / edges with any counted votes", () => { + const view = [ + row({ fromPath: "a.md", toPath: "b.md", countedVotes: 3, unfingerprintedCountedVotes: 3 }), + row({ fromPath: "c.md", toPath: "d.md", countedVotes: 2, unfingerprintedCountedVotes: 0 }), + row({ fromPath: "e.md", toPath: "f.md", countedVotes: 0, unfingerprintedCountedVotes: 0 }), + ]; + const summary = independenceCalibrationSummaryOf(view, []); + expect(summary.legacyUnfingerprintedFraction).toBeCloseTo(0.5, 6); // 1 of 2 edges-with-votes + }); + + it("nonLoopFingerprintedCountedVotes sums across the view", () => { + const view = [ + row({ nonLoopFingerprintedCountedVotes: 2 }), + row({ fromPath: "c.md", toPath: "d.md", nonLoopFingerprintedCountedVotes: 3 }), + ]; + const summary = independenceCalibrationSummaryOf(view, []); + expect(summary.nonLoopFingerprintedCountedVotes).toBe(5); + }); +}); + +describe("vault_lint independenceCalibration section — wiring", () => { + let vault: string; + beforeEach(() => { + vault = makeTempVault(); + }); + afterEach(() => { + cleanupVault(vault); + }); + + it("is present and zeroed on a fresh vault", async () => { + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.independenceCalibration).toEqual(emptyIndependenceCalibrationSummary()); + }); + + it("reflects a real fixture trail: edges.jsonl + independence-shadow.jsonl feed the same summary the pure function computes", async () => { + // A due edge whose one counted vote is unfingerprinted (legacy). + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: false, + }); + await observeEdge(vault, { + fromPath: "a.md", + toPath: "b.md", + observedBy: "agent:curation-loop", + blind: true, + axis: "prompt", + }); + // One journaled panel row: an informative, decided would_needs_review. + await appendIndependenceShadow( + vault, + shadowRow({ + fromPath: "a.md", + toPath: "b.md", + classes: [{ key: "hash\nagent:curation-loop\nclaude-haiku", count: 2 }], + wouldDecision: "would_needs_review", + }), + ); + + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + const ic = report.value.independenceCalibration; + expect(ic.kVsKEff.edgesWithVotes).toBe(1); + expect(ic.legacyUnfingerprintedFraction).toBeCloseTo(1, 6); // the one edge is all-legacy + expect(ic.wouldNeedsReviewRate.decidedCount).toBe(1); + expect(ic.wouldNeedsReviewRate.informativePanels).toBe(1); + expect(ic.wouldNeedsReviewRate.rateInformative).toBeCloseTo(1, 6); + }); +}); diff --git a/test/curation/lint-anchors.test.ts b/test/curation/lint-anchors.test.ts new file mode 100644 index 00000000..58439106 --- /dev/null +++ b/test/curation/lint-anchors.test.ts @@ -0,0 +1,292 @@ +// test/curation/lint-anchors.test.ts +// vault_lint's citation-anchors surfaces (2026-07-26 spec, Phase 8): +// malformedPins and the Decision-4 softened stale copy, budgeted. + +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { LINT_PIN_STEP3_BUDGET, runLint } from "../../src/curation/lint.js"; + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): void { + execFileSync("git", args, { cwd, env: GIT_ENV, stdio: "ignore" }); +} + +function hashOf(repo: string, relPath: string): string { + return execFileSync("git", ["-C", repo, "hash-object", relPath], { env: GIT_ENV }) + .toString() + .trim(); +} + +function staleDoc(describes: string[], extraDescribes: string[] = []): string { + const all = [...describes, ...extraDescribes]; + return `--- +title: "Stale doc with pins" +domain: accumulation +collection: engineering +status: canonical +confidence: high +created: "2020-01-01" +updated: "2020-01-01" +updated_by: agent:test +provenance: direct +ttl_days: 30 +tags: [] +describes: +${all.map((d) => ` - "${d}"`).join("\n")} +--- + +Body. +`; +} + +describe("vault_lint — citation anchors", () => { + let vault: string; + let codeRepo: string; + + beforeEach(() => { + vault = realpathSync(mkdtempSync(join(tmpdir(), "daftari-lint-anchors-"))); + mkdirSync(join(vault, "engineering"), { recursive: true }); + mkdirSync(join(vault, ".daftari"), { recursive: true }); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-lint-anchors-code-"))); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + }); + + afterEach(() => { + rmSync(vault, { recursive: true, force: true }); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + function writeConfig(extra = ""): void { + writeFileSync( + join(vault, ".daftari", "config.yaml"), + `code_repos:\n svc: ${codeRepo}\n${extra}`, + ); + } + + describe("malformedPins", () => { + it("flags a near-miss malformed pin as advisory, never blocking", async () => { + writeConfig(); + writeFileSync( + join(vault, "engineering/malformed.md"), + staleDoc(["svc:retry.ts#L1-2@ZZZZZZZ"]), + ); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.checks.malformedPins).toHaveLength(1); + expect(report.value.checks.malformedPins[0]?.path).toBe("engineering/malformed.md"); + expect(report.value.checks.malformedPins[0]?.detail).toContain("svc:retry.ts#L1-2@ZZZZZZZ"); + }); + + it("does not flag a well-formed pin", async () => { + writeConfig(); + writeFileSync( + join(vault, "engineering/fine.md"), + staleDoc([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.checks.malformedPins).toHaveLength(0); + }); + }); + + describe("Decision 4 softened stale copy", () => { + it("appends softened copy when every pin on a stale doc is intact", async () => { + writeConfig(); + writeFileSync( + join(vault, "engineering/stale-intact.md"), + staleDoc([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + const finding = report.value.checks.staleFiles.find( + (f) => f.path === "engineering/stale-intact.md", + ); + expect(finding).toBeDefined(); + expect(finding?.detail).toContain("past TTL, but its 1 code pin"); + expect(finding?.detail).toContain("has not changed since the pins were written"); + expect(report.value.pinsClassified).toBe(0); // whole-file pin never hits step 3 + }); + + it("does NOT soften when a pin is moved", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/stale-moved.md"), staleDoc(["svc:retry.ts@0000000"])); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + const finding = report.value.checks.staleFiles.find( + (f) => f.path === "engineering/stale-moved.md", + ); + expect(finding?.detail).not.toContain("code pin"); + }); + + it("does NOT soften a fresh (non-stale) doc even with intact pins", async () => { + writeConfig(); + writeFileSync( + join(vault, "engineering/fresh.md"), + `--- +title: "Fresh doc" +domain: accumulation +collection: engineering +status: canonical +confidence: high +created: "2026-01-05" +updated: "2026-01-05" +updated_by: agent:test +provenance: direct +tags: [] +describes: + - "svc:retry.ts@${hashOf(codeRepo, "retry.ts")}" +--- + +Body. +`, + ); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.checks.staleFiles.some((f) => f.path === "engineering/fresh.md")).toBe( + false, + ); + }); + + it("does not soften when jit_anchors is false", async () => { + writeConfig("jit_anchors: false\n"); + writeFileSync( + join(vault, "engineering/stale-off.md"), + staleDoc([`svc:retry.ts@${hashOf(codeRepo, "retry.ts")}`]), + ); + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + const finding = report.value.checks.staleFiles.find( + (f) => f.path === "engineering/stale-off.md", + ); + expect(finding?.detail).not.toContain("code pin"); + expect(report.value.pinsClassified).toBe(0); + }); + + it("softens using a range pin (exercises step 3, counted in pinsClassified)", async () => { + const content = + ["a", "TARGET LINE with enough content for the trivial-content floor", "c"].join("\n") + + "\n"; + writeFileSync(join(codeRepo, "range.ts"), content); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "range"]); + const rangeSha = hashOf(codeRepo, "range.ts"); + // Move the line down — blob differs, text intact (step 3 required). + writeFileSync( + join(codeRepo, "range.ts"), + ["pad", "a", "TARGET LINE with enough content for the trivial-content floor", "c"].join( + "\n", + ) + "\n", + ); + writeConfig(); + writeFileSync( + join(vault, "engineering/stale-range.md"), + staleDoc([`svc:range.ts#L2-2@${rangeSha}`]), + ); + + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + const finding = report.value.checks.staleFiles.find( + (f) => f.path === "engineering/stale-range.md", + ); + expect(finding?.detail).toContain("code pin"); + expect(report.value.pinsClassified).toBe(1); + }); + + it("memoises a verdict recurring across multiple stale docs (classified once)", async () => { + writeConfig(); + const rangeSha = (() => { + writeFileSync( + join(codeRepo, "shared.ts"), + ["a", "SHARED TARGET LINE with enough content here", "c"].join("\n") + "\n", + ); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "shared"]); + const sha = hashOf(codeRepo, "shared.ts"); + writeFileSync( + join(codeRepo, "shared.ts"), + ["pad", "a", "SHARED TARGET LINE with enough content here", "c"].join("\n") + "\n", + ); + return sha; + })(); + writeFileSync( + join(vault, "engineering/doc-a.md"), + staleDoc([`svc:shared.ts#L2-2@${rangeSha}`]), + ); + writeFileSync( + join(vault, "engineering/doc-b.md"), + staleDoc([`svc:shared.ts#L2-2@${rangeSha}`]), + ); + + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + // Two docs, same (repo, path, sha, range) triple -> classified ONCE. + expect(report.value.pinsClassified).toBe(1); + for (const path of ["engineering/doc-a.md", "engineering/doc-b.md"]) { + const finding = report.value.checks.staleFiles.find((f) => f.path === path); + expect(finding?.detail).toContain("code pin"); + } + }); + + it("respects the step-3 budget: docs beyond it are not softened", async () => { + writeConfig(); + // Create (LINT_PIN_STEP3_BUDGET + 1) distinct stale docs, each with a + // DISTINCT range pin requiring its own step-3 classification (distinct + // target files so the batch hash can't short-circuit them to intact). + const n = LINT_PIN_STEP3_BUDGET + 1; + for (let i = 0; i < n; i++) { + const fname = `f${i}.ts`; + writeFileSync( + join(codeRepo, fname), + ["a", `TARGET LINE number ${i} with enough content here`, "c"].join("\n") + "\n", + ); + } + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "many"]); + const shas: string[] = []; + for (let i = 0; i < n; i++) shas.push(hashOf(codeRepo, `f${i}.ts`)); + // Now change every file so the blob differs (forcing step 3). + for (let i = 0; i < n; i++) { + writeFileSync( + join(codeRepo, `f${i}.ts`), + ["pad", "a", `TARGET LINE number ${i} with enough content here`, "c"].join("\n") + "\n", + ); + } + for (let i = 0; i < n; i++) { + writeFileSync( + join(vault, `engineering/doc-${i}.md`), + staleDoc([`svc:f${i}.ts#L2-2@${shas[i]}`]), + ); + } + + const report = await runLint(vault); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.pinsClassified).toBe(LINT_PIN_STEP3_BUDGET); + // At least one doc's finding must NOT carry the softened copy — the + // budget was exceeded before every doc could be classified. + const softened = report.value.checks.staleFiles.filter((f) => f.detail.includes("code pin")); + expect(softened.length).toBeLessThan(n); + }, 20000); + }); +}); diff --git a/test/curation/lint.test.ts b/test/curation/lint.test.ts index bf1e4556..831046b8 100644 --- a/test/curation/lint.test.ts +++ b/test/curation/lint.test.ts @@ -1021,6 +1021,111 @@ ${body} const expired = await listStagedActions(dir, "expired"); expect(expired.ok && expired.value.map((a) => a.id)).toEqual(["stage-001"]); }); + + // 2026-07-26 risk-triaged-ratification spec, Phase 5: risk-descending + // ordering (Decisions 1 + 2) and the pathVisible vantage filter closing + // the disclosure gap named by Decision 4 (staged-action targets used to + // bypass pathVisible entirely — src/curation/lint.ts:379-era). + describe("risk-triaged ordering and vantage filtering (2026-07-26 spec)", () => { + it("orders risk descending, inverting the prior expiry-only sort", async () => { + dir = mkdtempSync(join(tmpdir(), "daftari-lint-staged-")); + // Higher-risk supersede expires LATER; lower-risk confidence-up + // expires SOONER. An expiry-only sort would put the confidence-up + // first; risk-triaged ordering must not. + await stageAction(dir, { + actionType: "confidence-up", + targetPath: "pricing/typo-fix.md", + proposedBy: "agent:x", + rationale: "Typo fix.", + proposedDiff: { confidence: "high" }, + proposedAt: "2026-06-01T00:00:00Z", + ttlDays: 5, // expires soonest + }); + await stageAction(dir, { + actionType: "supersede", + targetPath: "pricing/big-doc.md", + proposedBy: "agent:x", + rationale: "Replaced by new analysis.", + proposedDiff: { superseded_by: "pricing/big-doc-v2.md" }, + proposedAt: "2026-06-01T00:00:00Z", + ttlDays: 30, // expires later + }); + + const report = await runLint(dir, { now: new Date("2026-06-02T00:00:00Z") }); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.stagedActions.map((s) => s.actionType)).toEqual([ + "supersede", + "confidence-up", + ]); + expect(report.value.stagedActions[0]?.risk).toBeGreaterThan( + report.value.stagedActions[1]?.risk ?? 1, + ); + }); + + it("keeps the expiry tiebreak among equally-risky items", async () => { + dir = mkdtempSync(join(tmpdir(), "daftari-lint-staged-")); + await stageAction(dir, { + ...base, + targetPath: "pricing/x.md", + proposedAt: "2026-06-01T00:00:00Z", + ttlDays: 20, + }); + await stageAction(dir, { + ...base, + targetPath: "pricing/y.md", + proposedAt: "2026-06-01T00:00:00Z", + ttlDays: 5, + }); + + const report = await runLint(dir, { now: new Date("2026-06-02T00:00:00Z") }); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.stagedActions[0]?.risk).toBeCloseTo( + report.value.stagedActions[1]?.risk ?? -1, + 6, + ); + // Soonest-to-expire (stage-002, ttl 5) sorts first among the tie. + expect(report.value.stagedActions.map((s) => s.id)).toEqual(["stage-002", "stage-001"]); + }); + + it("omits a pending item whose target is unreadable and buckets the remainder (Decision 4)", async () => { + dir = mkdtempSync(join(tmpdir(), "daftari-lint-staged-")); + await stageAction(dir, { ...base, targetPath: "pricing/visible.md" }); + await stageAction(dir, { ...base, targetPath: "secret/hidden.md" }); + + const report = await runLint(dir, { pathVisible: (p) => !p.startsWith("secret/") }); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.stagedActions.map((s) => s.targetPath)).toEqual(["pricing/visible.md"]); + expect(report.value.hiddenStagedActions).toBe("some"); + }); + + it("an operator run (no pathVisible) shows every pending action, hiddenStagedActions 'none'", async () => { + dir = mkdtempSync(join(tmpdir(), "daftari-lint-staged-")); + await stageAction(dir, { ...base, targetPath: "pricing/visible.md" }); + await stageAction(dir, { ...base, targetPath: "secret/hidden.md" }); + + const report = await runLint(dir); + expect(report.ok).toBe(true); + if (!report.ok) return; + expect(report.value.stagedActions).toHaveLength(2); + expect(report.value.hiddenStagedActions).toBe("none"); + }); + + it("two vantages can produce different orderings — accepted consequence, pinned deliberately", async () => { + dir = mkdtempSync(join(tmpdir(), "daftari-lint-staged-")); + await stageAction(dir, { ...base, targetPath: "pricing/a.md" }); + await stageAction(dir, { ...base, targetPath: "intel/b.md" }); + + const pricingOnly = await runLint(dir, { pathVisible: (p) => p.startsWith("pricing/") }); + const intelOnly = await runLint(dir, { pathVisible: (p) => p.startsWith("intel/") }); + expect(pricingOnly.ok && intelOnly.ok).toBe(true); + if (!pricingOnly.ok || !intelOnly.ok) return; + expect(pricingOnly.value.stagedActions.map((s) => s.targetPath)).toEqual(["pricing/a.md"]); + expect(intelOnly.value.stagedActions.map((s) => s.targetPath)).toEqual(["intel/b.md"]); + }); + }); }); // ----- Task 7: coverageEquity in runLint ----------------------------------- diff --git a/test/curation/risk.test.ts b/test/curation/risk.test.ts new file mode 100644 index 00000000..19fcf347 --- /dev/null +++ b/test/curation/risk.test.ts @@ -0,0 +1,612 @@ +// Risk-triaged ratification — the risk scorer (2026-07-26 spec, Decision 1; +// final plan Phase 4). Pure-function tests: each term isolated with +// hand-built fixtures, plus the C1/C4/C5 challenge dispositions and the +// spec's own motivating example. + +import { describe, expect, it } from "vitest"; +import { + DIFF_BUCKET_THRESHOLDS, + HIDDEN_BLAST_BUMP, + RISK_KIND_WEIGHTS, + RISK_TERM_WEIGHTS, + rankPendingActions, +} from "../../src/curation/risk.js"; +import type { StagedAction } from "../../src/curation/staged-actions.js"; +import type { TensionEntry } from "../../src/curation/tension.js"; +import type { LoadedDoc } from "../../src/curation/vault-docs.js"; +import type { Frontmatter } from "../../src/frontmatter/types.js"; + +const NOW = new Date("2026-06-10T00:00:00Z"); + +function mkAction(overrides: Partial = {}): StagedAction { + return { + id: "stage-001", + actionType: "confidence-up", + targetPath: "pricing/a.md", + proposedBy: "agent:proposer", + proposedAt: "2026-06-01T00:00:00Z", + expiresAt: "2026-06-15T00:00:00Z", + status: "pending", + rationale: "Rationale.", + proposedDiff: {}, + ratifiedAt: null, + ratifiedBy: null, + ratificationReason: null, + decidedByPrincipal: null, + runId: null, + decisionKind: null, + reasonCategory: null, + amendedDiff: null, + stagedByPrincipal: null, + riskAtDecision: null, + ...overrides, + }; +} + +function mkDoc(path: string, overrides: Partial = {}, content = ""): LoadedDoc { + const fm: Frontmatter = { + title: path, + domain: "accumulation", + collection: path.split("/")[0] ?? "", + status: "canonical", + confidence: "medium", + created: "2026-01-01", + updated: "2026-01-01", + updated_by: "agent:test", + provenance: "direct", + tier: null, + sources: [], + superseded_by: null, + ttl_days: null, + valid_from: null, + valid_until: null, + tags: [], + describes: [], + questions_answered: [], + questions_raised: [], + ...overrides, + }; + return { path, frontmatter: fm, content, validation: { valid: true, issues: [] } }; +} + +function mkTension(overrides: Partial = {}): TensionEntry { + return { + id: "tension-001", + date: "2026-06-01", + title: "T", + kind: "factual", + sourceA: "pricing/a.md", + claimA: "x", + sourceB: "pricing/b.md", + claimB: "y", + status: "unresolved", + loggedBy: "agent:x", + resolved: false, + ...overrides, + }; +} + +// D term, verbatim from the spec: min(1, log10(1+bytes)/4). +function expectedD(bytes: number): number { + return Math.min(1, Math.log10(1 + bytes) / 4); +} + +// Pads a JSON object to an EXACT serialized byte count using plain ASCII +// (no escaping edge cases), so diff-bucket boundary tests land exactly on +// 255/256/4095/4096. +function diffOfBytes(targetBytes: number): unknown { + let note = ""; + while (Buffer.byteLength(JSON.stringify({ note }), "utf-8") < targetBytes) note += "x"; + return { note }; +} + +describe("risk.ts — rankPendingActions", () => { + describe("K — action-kind weight", () => { + it("orders a supersede above a confidence-up, all else equal", () => { + const confidenceUp = mkAction({ id: "stage-001", actionType: "confidence-up" }); + const supersede = mkAction({ + id: "stage-002", + actionType: "supersede", + targetPath: "pricing/b.md", + }); + const { items } = rankPendingActions({ + actions: [confidenceUp, supersede], + docs: [], + tensions: [], + now: NOW, + }); + const a = items.find((i) => i.id === "stage-001"); + const b = items.find((i) => i.id === "stage-002"); + expect(a).toBeDefined(); + expect(b).toBeDefined(); + expect(RISK_KIND_WEIGHTS["confidence-up"]).toBe(0.2); + expect(RISK_KIND_WEIGHTS.supersede).toBe(1.0); + expect(b?.risk).toBeGreaterThan(a?.risk ?? 0); + // K's contribution alone: 0.3 * (1.0 - 0.2) = 0.24 — larger than every + // other term's max possible swing except B, so it should dominate here + // since B/T/C are both 0 for these targets (empty docs/tensions). + expect((b?.risk ?? 0) - (a?.risk ?? 0)).toBeCloseTo(RISK_TERM_WEIGHTS.K * 0.8, 3); + }); + + it("defaults an unknown action kind to write's weight (fail toward scrutiny)", () => { + const unknown = mkAction({ actionType: "frobnicate" as unknown as string }); + const { items } = rankPendingActions({ + actions: [unknown], + docs: [], + tensions: [], + now: NOW, + }); + const write = mkAction({ id: "stage-002", actionType: "write", targetPath: "pricing/b.md" }); + const { items: writeItems } = rankPendingActions({ + actions: [write], + docs: [], + tensions: [], + now: NOW, + }); + expect(items[0]?.risk).toBeCloseTo(writeItems[0]?.risk ?? -1, 3); + }); + }); + + describe("D — proposed-diff size and the byte-bucket (C3)", () => { + it("matches the spec's log-scaled formula", () => { + const action = mkAction({ proposedDiff: diffOfBytes(1000) }); + const { items } = rankPendingActions({ actions: [action], docs: [], tensions: [], now: NOW }); + const bytes = Buffer.byteLength(JSON.stringify(action.proposedDiff), "utf-8"); + const K = RISK_KIND_WEIGHTS["confidence-up"] ?? 0; + const W = 0.5; // fresh proposer, Laplace midpoint + const expected = + RISK_TERM_WEIGHTS.K * K + RISK_TERM_WEIGHTS.D * expectedD(bytes) + RISK_TERM_WEIGHTS.W * W; + expect(items[0]?.risk).toBeCloseTo(expected, 3); + }); + + it("buckets small < 256 bytes, medium in [256, 4096), large >= 4096 bytes (C3 disposition)", () => { + const cases: Array<[number, "small" | "medium" | "large"]> = [ + [DIFF_BUCKET_THRESHOLDS.mediumBytes - 1, "small"], + [DIFF_BUCKET_THRESHOLDS.mediumBytes, "medium"], + [DIFF_BUCKET_THRESHOLDS.largeBytes - 1, "medium"], + [DIFF_BUCKET_THRESHOLDS.largeBytes, "large"], + ]; + for (const [bytes, bucket] of cases) { + const diff = diffOfBytes(bytes); + expect(Buffer.byteLength(JSON.stringify(diff), "utf-8")).toBe(bytes); + const action = mkAction({ proposedDiff: diff }); + const { items } = rankPendingActions({ + actions: [action], + docs: [], + tensions: [], + now: NOW, + }); + expect(items[0]?.diffBucket).toBe(bucket); + } + }); + + it("a constant-size lifecycle pointer (empty proposedDiff) reads small", () => { + const action = mkAction({ + actionType: "supersede", + proposedDiff: { superseded_by: "pricing/b.md" }, + }); + const { items } = rankPendingActions({ actions: [action], docs: [], tensions: [], now: NOW }); + expect(items[0]?.diffBucket).toBe("small"); + }); + }); + + describe("B — blast radius, direct inbound only", () => { + it("counts direct source and link inbound, source wins on overlap", () => { + const target = mkDoc("pricing/target.md"); + const viaSource = mkDoc("pricing/s1.md", { sources: ["pricing/target.md"] }); + const viaLink = mkDoc("pricing/l1.md", {}, "See [target](target.md)."); + const viaBoth = mkDoc( + "pricing/both.md", + { sources: ["pricing/target.md"] }, + "See [t](target.md).", + ); + const docs = [target, viaSource, viaLink, viaBoth]; + const action = mkAction({ targetPath: "pricing/target.md" }); + const { items } = rankPendingActions({ actions: [action], docs, tensions: [], now: NOW }); + // primary: s1 + both (both wins primary over link on overlap); advisory: l1 only. + expect(items[0]?.blast.primary).toBe(2); + expect(items[0]?.blast.advisory).toBe(1); + expect(items[0]?.blast.hidden).toBe("none"); + }); + + it("coarsens hidden inbound to none/some/many, never an exact count (Decision 4)", () => { + const target = mkDoc("pricing/target.md"); + const visible = mkDoc("pricing/visible.md", { sources: ["pricing/target.md"] }); + const hidden1 = mkDoc("secret/hidden1.md", { + collection: "secret", + sources: ["pricing/target.md"], + }); + const docs = [target, visible, hidden1]; + const action = mkAction({ targetPath: "pricing/target.md" }); + const pathVisible = (p: string) => !p.startsWith("secret/"); + + const { items } = rankPendingActions({ + actions: [action], + docs, + tensions: [], + now: NOW, + pathVisible, + }); + expect(items[0]?.blast.primary).toBe(1); // only 'visible' counted + expect(items[0]?.blast.hidden).toBe("some"); // coarsened bucket, not the exact hidden count (1) + }); + + it("bumps B by the hidden-bucket amount when part of the inbound is hidden", () => { + const target = mkDoc("pricing/target.md"); + const visible = mkDoc("pricing/visible.md", { sources: ["pricing/target.md"] }); + const hidden = mkDoc("secret/hidden.md", { + collection: "secret", + sources: ["pricing/target.md"], + }); + const docs = [target, visible, hidden]; + const action = mkAction({ targetPath: "pricing/target.md" }); + const pathVisible = (p: string) => !p.startsWith("secret/"); + + const { items } = rankPendingActions({ + actions: [action], + docs, + tensions: [], + now: NOW, + pathVisible, + }); + const B = Math.min(1, 1 / 10 + 0 / 40 + HIDDEN_BLAST_BUMP.some); + const K = RISK_KIND_WEIGHTS["confidence-up"] ?? 0; + const W = 0.5; + const expected = + RISK_TERM_WEIGHTS.K * K + + RISK_TERM_WEIGHTS.D * expectedD(2) + + RISK_TERM_WEIGHTS.B * B + + RISK_TERM_WEIGHTS.W * W; + expect(items[0]?.risk).toBeCloseTo(expected, 3); + }); + }); + + describe("T — open tension, endpoints canonicalized (C5)", () => { + it("an interpretive tension flips T (independence-spec compatibility, kind-blind)", () => { + const action = mkAction({ targetPath: "pricing/a.md" }); + const tension = mkTension({ + kind: "interpretive", + sourceA: "pricing/a.md", + sourceB: "pricing/b.md", + }); + const { items } = rankPendingActions({ + actions: [action], + docs: [], + tensions: [tension], + now: NOW, + }); + expect(items[0]?.openTension).toBe(true); + }); + + it("a resolved tension does not flip T", () => { + const action = mkAction({ targetPath: "pricing/a.md" }); + const tension = mkTension({ resolved: true }); + const { items } = rankPendingActions({ + actions: [action], + docs: [], + tensions: [tension], + now: NOW, + }); + expect(items[0]?.openTension).toBe(false); + }); + + it("a basename-spelled tension endpoint flips T on a canonical-relPath target (C5)", () => { + const doc = mkDoc("competitive-intel/pricing-model.md"); + const action = mkAction({ targetPath: "competitive-intel/pricing-model.md" }); + const tension = mkTension({ + sourceA: "pricing-model", // basename, no extension, no directory + sourceB: "unrelated.md", + }); + const { items } = rankPendingActions({ + actions: [action], + docs: [doc], + tensions: [tension], + now: NOW, + }); + expect(items[0]?.openTension).toBe(true); + }); + + it("an unresolvable endpoint falls back to the raw string, never false-matching a live target", () => { + const action = mkAction({ targetPath: "pricing/a.md" }); + const tension = mkTension({ + sourceA: "deleted-doc-that-never-existed.md", + sourceB: "also-gone.md", + }); + const { items } = rankPendingActions({ + actions: [action], + docs: [], + tensions: [tension], + now: NOW, + }); + expect(items[0]?.openTension).toBe(false); + }); + }); + + describe("C — conflict / retry markers (C1 disposition)", () => { + it("clause (a): another pending action sharing the target sets C for both", () => { + const a = mkAction({ id: "stage-001", targetPath: "pricing/a.md" }); + const b = mkAction({ id: "stage-002", targetPath: "pricing/a.md" }); + const { items } = rankPendingActions({ actions: [a, b], docs: [], tensions: [], now: NOW }); + expect(items.every((i) => i.conflict)).toBe(true); + }); + + it("clause (b): a prior rejected action with no later ratify sets C on a fresh retry", () => { + const rejected = mkAction({ + id: "stage-001", + actionType: "promote", + targetPath: "pricing/a.md", + status: "rejected", + ratifiedAt: "2026-06-02T00:00:00Z", + }); + const retry = mkAction({ + id: "stage-002", + actionType: "promote", + targetPath: "pricing/a.md", + status: "pending", + }); + const { items } = rankPendingActions({ + actions: [rejected, retry], + docs: [], + tensions: [], + now: NOW, + }); + const item = items.find((i) => i.id === "stage-002"); + expect(item?.conflict).toBe(true); + }); + + it("an EXPIRED same-pair record does not set C — expiry cost lives only in W", () => { + const expired = mkAction({ + id: "stage-001", + actionType: "promote", + targetPath: "pricing/a.md", + status: "expired", + ratifiedAt: "2026-06-02T00:00:00Z", + ratifiedBy: "system:lint-sweep", + }); + const retry = mkAction({ + id: "stage-002", + actionType: "promote", + targetPath: "pricing/a.md", + status: "pending", + }); + const { items } = rankPendingActions({ + actions: [expired, retry], + docs: [], + tensions: [], + now: NOW, + }); + const item = items.find((i) => i.id === "stage-002"); + expect(item?.conflict).toBe(false); + }); + + it("a later ratified same-pair record clears the retry mark (C1)", () => { + const rejected = mkAction({ + id: "stage-001", + actionType: "promote", + targetPath: "pricing/a.md", + status: "rejected", + ratifiedAt: "2026-06-01T00:00:00Z", + }); + const laterRatified = mkAction({ + id: "stage-002", + actionType: "promote", + targetPath: "pricing/a.md", + status: "ratified", + ratifiedAt: "2026-06-03T00:00:00Z", + }); + const pending = mkAction({ + id: "stage-003", + actionType: "promote", + targetPath: "pricing/a.md", + status: "pending", + }); + const { items } = rankPendingActions({ + actions: [rejected, laterRatified, pending], + docs: [], + tensions: [], + now: NOW, + }); + const item = items.find((i) => i.id === "stage-003"); + expect(item?.conflict).toBe(false); + }); + + it("a retry restaged under a DIFFERENT action kind does not trip C", () => { + const rejected = mkAction({ + id: "stage-001", + actionType: "promote", + targetPath: "pricing/a.md", + status: "rejected", + ratifiedAt: "2026-06-01T00:00:00Z", + }); + const differentKind = mkAction({ + id: "stage-002", + actionType: "confidence-up", + targetPath: "pricing/a.md", + status: "pending", + }); + const { items } = rankPendingActions({ + actions: [rejected, differentKind], + docs: [], + tensions: [], + now: NOW, + }); + expect(items[0]?.conflict).toBe(false); + }); + }); + + describe("W — proposer track record, Laplace-smoothed", () => { + it("defaults an unseen principal to the Laplace midpoint 0.5", () => { + const action = mkAction({ proposedBy: "agent:brand-new" }); + const { items } = rankPendingActions({ actions: [action], docs: [], tensions: [], now: NOW }); + expect(items[0]?.proposerTrackRecord).toBeCloseTo(0.5, 3); + }); + + it("is unchanged by proposed_by rotation under one authenticated stagedByPrincipal (C4)", () => { + const decided = [ + mkAction({ + id: "stage-001", + proposedBy: "agent:rotating-alpha", + stagedByPrincipal: "human:mihir", + status: "rejected", + ratifiedAt: "2026-06-02T00:00:00Z", + }), + mkAction({ + id: "stage-002", + proposedBy: "agent:rotating-beta", + stagedByPrincipal: "human:mihir", + status: "rejected", + ratifiedAt: "2026-06-03T00:00:00Z", + }), + ]; + const pendingRotated = mkAction({ + id: "stage-003", + proposedBy: "agent:rotating-gamma", + stagedByPrincipal: "human:mihir", + targetPath: "pricing/rotated.md", + }); + const rotated = rankPendingActions({ + actions: [...decided, pendingRotated], + docs: [], + tensions: [], + now: NOW, + }); + + const decidedStatic = [ + mkAction({ + id: "stage-001", + proposedBy: "human:mihir", + stagedByPrincipal: "human:mihir", + status: "rejected", + ratifiedAt: "2026-06-02T00:00:00Z", + }), + mkAction({ + id: "stage-002", + proposedBy: "human:mihir", + stagedByPrincipal: "human:mihir", + status: "rejected", + ratifiedAt: "2026-06-03T00:00:00Z", + }), + ]; + const pendingStatic = mkAction({ + id: "stage-003", + proposedBy: "human:mihir", + stagedByPrincipal: "human:mihir", + targetPath: "pricing/rotated.md", + }); + const staticResult = rankPendingActions({ + actions: [...decidedStatic, pendingStatic], + docs: [], + tensions: [], + now: NOW, + }); + + expect(rotated.items[0]?.proposerTrackRecord).toBeCloseTo( + staticResult.items[0]?.proposerTrackRecord ?? -1, + 6, + ); + // Two rejections, no ratifies/edits/expiries: (2 + 1) / (0 + 2 + 2) = 0.75. + expect(rotated.items[0]?.proposerTrackRecord).toBeCloseTo(0.75, 3); + }); + + it("an edited approval is not double-counted (plainRatified = ratified - edited)", () => { + const edited = mkAction({ + id: "stage-001", + proposedBy: "agent:editor", + status: "ratified", + decisionKind: "edit-then-approve", + ratifiedAt: "2026-06-02T00:00:00Z", + }); + const pending = mkAction({ + id: "stage-002", + proposedBy: "agent:editor", + targetPath: "pricing/other.md", + }); + const { items } = rankPendingActions({ + actions: [edited, pending], + docs: [], + tensions: [], + now: NOW, + }); + // plainRatified = 1 - 1 = 0; rejected=0, edited=1, expired=0: + // (0 + 1 + 0 + 1) / (0 + 0 + 1 + 0 + 2) = 2/3. + expect(items[0]?.proposerTrackRecord).toBeCloseTo(2 / 3, 3); + }); + }); + + describe("sort order: risk descending, expiry-ascending tiebreak, id tiebreak", () => { + it("the spec's motivating example: a heavily-cited supersede outranks a typo-fix confidence-up despite expiring later", () => { + const heavilyCited = mkDoc("pricing/hot.md"); + const inbound = Array.from({ length: 10 }, (_, i) => + mkDoc(`pricing/dep-${i}.md`, { sources: ["pricing/hot.md"] }), + ); + const docs = [heavilyCited, ...inbound]; + + const typoFix = mkAction({ + id: "stage-001", + actionType: "confidence-up", + targetPath: "pricing/cold.md", + expiresAt: "2026-06-11T00:00:00Z", // Tuesday — expires sooner + }); + const supersede = mkAction({ + id: "stage-002", + actionType: "supersede", + targetPath: "pricing/hot.md", + expiresAt: "2026-06-13T00:00:00Z", // Thursday — expires later + proposedDiff: { superseded_by: "pricing/hot-v2.md" }, + }); + + const { items } = rankPendingActions({ + actions: [typoFix, supersede], + docs, + tensions: [], + now: NOW, + }); + expect(items[0]?.id).toBe("stage-002"); + expect(items[1]?.id).toBe("stage-001"); + }); + + it("breaks a risk tie by soonest-to-expire, then by id", () => { + const a = mkAction({ + id: "stage-002", + expiresAt: "2026-06-20T00:00:00Z", + targetPath: "pricing/a.md", + }); + const b = mkAction({ + id: "stage-001", + expiresAt: "2026-06-12T00:00:00Z", + targetPath: "pricing/b.md", + }); + const { items } = rankPendingActions({ actions: [a, b], docs: [], tensions: [], now: NOW }); + expect(items[0]?.risk).toBeCloseTo(items[1]?.risk ?? -1, 6); + expect(items.map((i) => i.id)).toEqual(["stage-001", "stage-002"]); + }); + }); + + describe("vantage filtering (Decision 4)", () => { + it("omits a pending item whose target is unreadable and buckets the remainder", () => { + const visible = mkAction({ id: "stage-001", targetPath: "pricing/open.md" }); + const hidden = mkAction({ id: "stage-002", targetPath: "secret/hidden.md" }); + const pathVisible = (p: string) => !p.startsWith("secret/"); + const { items, hiddenPending } = rankPendingActions({ + actions: [visible, hidden], + docs: [], + tensions: [], + now: NOW, + pathVisible, + }); + expect(items.map((i) => i.id)).toEqual(["stage-001"]); + expect(hiddenPending).toBe("some"); + }); + + it("reports 'none' hidden under an operator run (no pathVisible)", () => { + const a = mkAction({ id: "stage-001" }); + const { hiddenPending } = rankPendingActions({ + actions: [a], + docs: [], + tensions: [], + now: NOW, + }); + expect(hiddenPending).toBe("none"); + }); + }); +}); diff --git a/test/curation/staged-actions.test.ts b/test/curation/staged-actions.test.ts index 41cce6b6..8cf7370b 100644 --- a/test/curation/staged-actions.test.ts +++ b/test/curation/staged-actions.test.ts @@ -7,9 +7,11 @@ import { listStagedActions, materializeStagedActions, nowISO, + proposalTallies, rebuildStagedActionsIndex, recordDecision, type StageActionInput, + type StagedAction, stageAction, stageActionWithConflictCheck, stagedActionsPath, @@ -190,7 +192,7 @@ describe("staged-actions", () => { ratifiedBy: "human:mihir", }); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -213,7 +215,7 @@ describe("staged-actions", () => { if (!result.ok) return; expect(result.value.count).toBe(1); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; try { expect(getAllStagedActions(opened.value)).toHaveLength(1); @@ -339,4 +341,199 @@ describe("staged-actions", () => { expect(staged.ok && staged.value?.status).toBe("pending"); }); }); + + // 2026-07-26 risk-triaged-ratification spec, Decision 3 (Phase 1): the + // decision-record extensions (decision_kind, reason_category, amended_diff, + // staged_by_principal) and Mihir's 2026-07-27 risk_at_decision addendum. + describe("Decision 3 fields (decision_kind, reason_category, amended_diff, staged_by_principal)", () => { + it("round-trips decision_kind, reason_category, amended_diff, and risk_at_decision through collapse", async () => { + const staged = await stageAction(vault, sampleInput); + if (!staged.ok) return; + const decided = await recordDecision(vault, staged.value.id, { + status: "ratified", + ratifiedAt: nowISO(), + ratifiedBy: "human:mihir", + decisionKind: "edit-then-approve", + reasonCategory: "overbroad", + amendedDiff: { status: { from: "draft", to: "canonical" }, note: "edited" }, + riskAtDecision: 0.42, + }); + expect(decided.ok).toBe(true); + if (!decided.ok) return; + expect(decided.value.decisionKind).toBe("edit-then-approve"); + expect(decided.value.reasonCategory).toBe("overbroad"); + expect(decided.value.amendedDiff).toEqual({ + status: { from: "draft", to: "canonical" }, + note: "edited", + }); + expect(decided.value.riskAtDecision).toBe(0.42); + + // Re-read from a fresh collapse (not the in-memory mirror) — the two + // sites the module warns about must not drift. + const reread = await getStagedActionById(vault, staged.value.id); + expect(reread.ok).toBe(true); + if (!reread.ok || !reread.value) return; + expect(reread.value.decisionKind).toBe("edit-then-approve"); + expect(reread.value.reasonCategory).toBe("overbroad"); + expect(reread.value.amendedDiff).toEqual({ + status: { from: "draft", to: "canonical" }, + note: "edited", + }); + expect(reread.value.riskAtDecision).toBe(0.42); + }); + + it("an old-shaped decision record (no Decision-3 fields) still collapses, yielding nulls", async () => { + const staged = await stageAction(vault, sampleInput); + if (!staged.ok) return; + const decided = await recordDecision(vault, staged.value.id, { + status: "ratified", + ratifiedAt: nowISO(), + ratifiedBy: "human:mihir", + }); + expect(decided.ok).toBe(true); + if (!decided.ok) return; + expect(decided.value.decisionKind).toBeNull(); + expect(decided.value.reasonCategory).toBeNull(); + expect(decided.value.amendedDiff).toBeNull(); + expect(decided.value.riskAtDecision).toBeNull(); + expect(decided.value.stagedByPrincipal).toBeNull(); + }); + + it("records staged_by_principal on the proposal and round-trips it", async () => { + const staged = await stageAction(vault, { ...sampleInput, stagedByPrincipal: "human:mihir" }); + if (!staged.ok) return; + const fetched = await getStagedActionById(vault, staged.value.id); + expect(fetched.ok).toBe(true); + if (!fetched.ok || !fetched.value) return; + expect(fetched.value.stagedByPrincipal).toBe("human:mihir"); + }); + + it("recordDecision validates decisionKind and reasonCategory enum membership", async () => { + const staged = await stageAction(vault, sampleInput); + if (!staged.ok) return; + const badKind = await recordDecision(vault, staged.value.id, { + status: "ratified", + ratifiedAt: nowISO(), + ratifiedBy: "human:mihir", + decisionKind: "not-a-real-kind" as never, + }); + expect(badKind.ok).toBe(false); + const badCategory = await recordDecision(vault, staged.value.id, { + status: "rejected", + ratifiedAt: nowISO(), + ratifiedBy: "human:mihir", + reasonCategory: "not-a-real-category" as never, + }); + expect(badCategory.ok).toBe(false); + }); + + it("the sweep's expiry decisions stay bare — no Decision-3 fields, no risk_at_decision", async () => { + await stageAction(vault, { ...sampleInput, proposedAt: "2026-01-01T00:00:00Z" }); + const swept = await sweepExpiredActions(vault, new Date("2026-06-01T00:00:00Z")); + expect(swept.ok).toBe(true); + const expired = await getStagedActionById(vault, "stage-001"); + expect(expired.ok).toBe(true); + if (!expired.ok || !expired.value) return; + expect(expired.value.status).toBe("expired"); + expect(expired.value.decisionKind).toBeNull(); + expect(expired.value.reasonCategory).toBeNull(); + expect(expired.value.riskAtDecision).toBeNull(); + }); + }); + + describe("proposalTallies", () => { + function action(overrides: Partial = {}): StagedAction { + return { + id: "stage-001", + actionType: "promote", + targetPath: "a.md", + proposedBy: "agent:x", + proposedAt: "2026-06-01T00:00:00Z", + expiresAt: "2026-06-15T00:00:00Z", + status: "pending", + rationale: "r", + proposedDiff: {}, + ratifiedAt: null, + ratifiedBy: null, + ratificationReason: null, + decidedByPrincipal: null, + runId: null, + decisionKind: null, + reasonCategory: null, + amendedDiff: null, + stagedByPrincipal: null, + riskAtDecision: null, + ...overrides, + }; + } + + it("counts edited (a subset of ratified) and byCategory over decided rows", () => { + const actions: StagedAction[] = [ + action({ + id: "1", + status: "ratified", + decisionKind: "edit-then-approve", + reasonCategory: "overbroad", + }), + action({ id: "2", status: "ratified" }), + action({ id: "3", status: "rejected", reasonCategory: "duplicate" }), + action({ id: "4", status: "pending" }), + ]; + const tallies = proposalTallies(actions); + const t = tallies.get("agent:x"); + expect(t).toEqual({ + total: 4, + ratified: 2, + rejected: 1, + expired: 0, + pending: 1, + edited: 1, + byCategory: { overbroad: 1, duplicate: 1 }, + }); + }); + + it("keys by stagedByPrincipal, falling back to proposedBy (anti-laundering, C4)", () => { + const actions: StagedAction[] = [ + action({ + id: "1", + proposedBy: "agent:rival-name-1", + stagedByPrincipal: "human:mihir", + status: "rejected", + }), + action({ + id: "2", + proposedBy: "agent:rival-name-2", + stagedByPrincipal: "human:mihir", + status: "rejected", + }), + ]; + const tallies = proposalTallies(actions); + // Rotating the unauthenticated proposed_by string does NOT fragment the + // tally — both land under the one authenticated stager. + expect(tallies.size).toBe(1); + expect(tallies.get("human:mihir")?.rejected).toBe(2); + expect(tallies.has("agent:rival-name-1")).toBe(false); + expect(tallies.has("agent:rival-name-2")).toBe(false); + }); + + it("junk staged under a rival's claimed name counts against the actual stager (anti-poisoning, C4)", () => { + const actions: StagedAction[] = [ + action({ + id: "1", + proposedBy: "agent:rival", + stagedByPrincipal: "human:attacker", + status: "rejected", + }), + ]; + const tallies = proposalTallies(actions); + expect(tallies.get("human:attacker")?.rejected).toBe(1); + expect(tallies.has("agent:rival")).toBe(false); + }); + + it("falls back to proposedBy for legacy records with no stagedByPrincipal", () => { + const actions: StagedAction[] = [action({ id: "1", proposedBy: "agent:legacy" })]; + const tallies = proposalTallies(actions); + expect(tallies.get("agent:legacy")?.total).toBe(1); + }); + }); }); diff --git a/test/eval/index.test.ts b/test/eval/index.test.ts index 4834ece8..a9a29ef8 100644 --- a/test/eval/index.test.ts +++ b/test/eval/index.test.ts @@ -54,7 +54,7 @@ vi.mock("../../src/eval/subgraph.js", () => ({ })); import { runEval } from "../../src/eval/index.js"; -import { writeQuestionSet, writeResults } from "../../src/eval/storage.js"; +import { readResults, writeQuestionSet, writeResults } from "../../src/eval/storage.js"; import type { EvalRun, QuestionSet } from "../../src/eval/types.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; @@ -255,6 +255,287 @@ describe("daftari eval --transport", () => { }); }); +// spec 2026-07-26-context-packs-progressive-disclosure-design.md, final plan +// Phase 3.4/3.5/C8: --condition/--budget/--max-tool-calls CLI wiring, run-id +// minting, and the --resume mismatch guard. Every run here uses an empty +// question set, so runAnswerer/runPackAnswerer's loop bodies never execute — +// these tests exercise only the CLI's flag parsing, id minting, and metadata +// persistence, not the answerer loops themselves (covered in +// test/eval/{run,pack-condition}.test.ts). +describe("daftari eval run --condition / --max-tool-calls (Phase 3.4)", () => { + let errSpy: ReturnType; + let outSpy: ReturnType; + let dir: string; + + beforeEach(async () => { + process.env.ANTHROPIC_API_KEY = "test-key"; + errSpy = vi.spyOn(process.stderr, "write").mockReturnValue(true); + outSpy = vi.spyOn(process.stdout, "write").mockReturnValue(true); + dir = mkdtempSync(join(tmpdir(), "daftari-eval-condition-")); + await writeQuestionSet(dir, minimalQuestionSet("qs-cond")); + }); + + afterEach(() => { + delete process.env.ANTHROPIC_API_KEY; + errSpy.mockRestore(); + outSpy.mockRestore(); + rmSync(dir, { recursive: true, force: true }); + }); + + function stdoutText(): string { + return outSpy.mock.calls.map((c) => String(c[0])).join(""); + } + function stderrText(): string { + return errSpy.mock.calls.map((c) => String(c[0])).join(""); + } + + it("rejects an invalid --condition value (exit 2)", async () => { + const code = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "bogus", + ]); + expect(code).toBe(2); + expect(stderrText()).toContain("--condition must be 'tools' or 'pack'"); + }); + + it("default (uncapped tools) mints the historical id shape and persists condition: 'tools'", async () => { + const code = await runEval(["run", "--vault", dir, "--questions", "qs-cond"]); + expect(code).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1]; + expect(id).toBeTruthy(); + expect(id).not.toContain("-tools-c"); + expect(id).not.toContain("-pack-b"); + const read = await readResults(dir, id as string); + expect(read.ok).toBe(true); + if (!read.ok) return; + expect(read.value.condition).toBe("tools"); + expect(read.value.max_tool_calls).toBeUndefined(); + }); + + it("--max-tool-calls mints a '-tools-c{N}' id and persists max_tool_calls", async () => { + const code = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--max-tool-calls", + "6", + ]); + expect(code).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1]; + expect(id).toContain("-tools-c6-"); + const read = await readResults(dir, id as string); + expect(read.ok).toBe(true); + if (!read.ok) return; + expect(read.value.condition).toBe("tools"); + expect(read.value.max_tool_calls).toBe(6); + }); + + it("--condition pack mints a '-pack-b{budget}' id and persists condition/pack_budget", async () => { + const code = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "pack", + "--budget", + "3000", + ]); + expect(code).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1]; + expect(id).toContain("-pack-b3000-"); + const read = await readResults(dir, id as string); + expect(read.ok).toBe(true); + if (!read.ok) return; + expect(read.value.condition).toBe("pack"); + expect(read.value.pack_budget).toBe(3000); + }); + + it("--resume refuses (exit 2) when the persisted condition does not match --condition", async () => { + const first = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "pack", + ]); + expect(first).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1] as string; + errSpy.mockClear(); + outSpy.mockClear(); + const second = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "tools", + "--resume", + id, + ]); + expect(second).toBe(2); + expect(stderrText()).toContain("persisted condition 'pack' does not match --condition 'tools'"); + }); + + it("--resume refuses (exit 2) when the persisted --max-tool-calls does not match", async () => { + const first = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--max-tool-calls", + "6", + ]); + expect(first).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1] as string; + errSpy.mockClear(); + outSpy.mockClear(); + const second = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--max-tool-calls", + "3", + "--resume", + id, + ]); + expect(second).toBe(2); + expect(stderrText()).toContain("does not match --max-tool-calls 3"); + }); + + it("--resume refuses (exit 2) when the persisted pack budget does not match", async () => { + const first = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "pack", + "--budget", + "3000", + ]); + expect(first).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1] as string; + errSpy.mockClear(); + outSpy.mockClear(); + const second = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--condition", + "pack", + "--budget", + "5000", + "--resume", + id, + ]); + expect(second).toBe(2); + expect(stderrText()).toContain("does not match --budget 5000"); + }); + + it("--resume with matching condition/budget/cap proceeds (no mismatch error)", async () => { + const first = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--max-tool-calls", + "6", + ]); + expect(first).toBe(0); + const id = stdoutText().match(/wrote results (\S+)/)?.[1] as string; + errSpy.mockClear(); + outSpy.mockClear(); + const second = await runEval([ + "run", + "--vault", + dir, + "--questions", + "qs-cond", + "--max-tool-calls", + "6", + "--resume", + id, + ]); + expect(second).toBe(0); + expect(stderrText()).not.toContain("does not match"); + }); +}); + +// C8: a legacy artifact minted before condition/pack_budget/max_tool_calls +// existed carries none of the three fields — it must still load and score +// as an uncapped `tools` run, never fail to parse. +describe("daftari eval score — legacy artifacts (C8)", () => { + let errSpy: ReturnType; + let outSpy: ReturnType; + + beforeEach(() => { + process.env.ANTHROPIC_API_KEY = "test-key"; + errSpy = vi.spyOn(process.stderr, "write").mockReturnValue(true); + outSpy = vi.spyOn(process.stdout, "write").mockReturnValue(true); + }); + + afterEach(() => { + delete process.env.ANTHROPIC_API_KEY; + errSpy.mockRestore(); + outSpy.mockRestore(); + }); + + it("a results file with no condition/pack_budget/max_tool_calls fields still scores", async () => { + const dir = mkdtempSync(join(tmpdir(), "daftari-eval-legacy-")); + try { + const qs = minimalQuestionSet("qs-legacy"); + qs.questions = [ + { + id: "q1", + tier: "retrieval", + question: "how many tiers?", + expected_answer: "3", + expected_sources: ["a.md"], + origin: "generated", + }, + ]; + await writeQuestionSet(dir, qs); + // Deliberately no `condition` / `pack_budget` / `max_tool_calls` keys — + // the exact shape a pre-this-wave artifact has. + const legacyRun: EvalRun = { + id: "legacy-run-1", + questions_id: "qs-legacy", + answerer_model: "m", + prompt_version: 1, + timestamp: "2026-01-01T00:00:00Z", + k: 1, + runs: {}, + }; + await writeResults(dir, legacyRun); + const code = await runEval(["score", "--vault", dir, "--results", "legacy-run-1"]); + expect(code).toBe(0); + const out = outSpy.mock.calls.map((c) => String(c[0])).join(""); + expect(out).toContain("condition=tools"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + // Artifact ids double as filenames: an OpenRouter model slug's "/" must never // reach the results/scores paths. describe("modelIdSlug", () => { diff --git a/test/eval/llm-openrouter.test.ts b/test/eval/llm-openrouter.test.ts index 7e54e2c0..d6f29185 100644 --- a/test/eval/llm-openrouter.test.ts +++ b/test/eval/llm-openrouter.test.ts @@ -410,4 +410,68 @@ describe("createOpenRouterClient — completeWithTools", () => { expect(r.value.text).toBe("recovered"); expect(r.value.tool_calls[0].output).toEqual({ tool_error: "boom" }); }); + + // C5 (spec 2026-07-26-context-packs-progressive-disclosure-design.md, + // final plan Phase 3.3): maxToolCalls caps REALIZED calls even when a + // round's parallel tool_calls would overshoot it — the OpenRouter twin of + // the anthropic-client test in test/eval/llm.test.ts. + it("maxToolCalls stubs excess calls within an overshooting round and forces a final answer", async () => { + function multiCallBody(n: number, prefix: string) { + return { + choices: [ + { + message: { + content: null, + tool_calls: Array.from({ length: n }, (_, i) => ({ + id: `${prefix}-${i}`, + type: "function", + function: { name: "vault_read", arguments: "{}" }, + })), + }, + finish_reason: "tool_calls", + }, + ], + usage: { prompt_tokens: 10, completion_tokens: 10 }, + }; + } + const bodies: any[] = []; + const fetchImpl = vi + .fn() + .mockImplementationOnce(async (_url: string, init: any) => { + bodies.push(JSON.parse(init.body)); + return fakeRes(200, multiCallBody(5, "r1")); + }) + .mockImplementationOnce(async (_url: string, init: any) => { + bodies.push(JSON.parse(init.body)); + return fakeRes(200, multiCallBody(3, "r2")); + }) + .mockImplementationOnce(async (_url: string, init: any) => { + bodies.push(JSON.parse(init.body)); + return fakeRes(200, okBody("final answer")); + }); + const handler = vi.fn().mockResolvedValue("ok"); + const client = createOpenRouterClient({ fetchImpl: fetchImpl as any }); + + const r = await client.completeWithTools({ + ...TOOL_OPTS, + toolHandler: handler, + maxToolCalls: 6, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.text).toBe("final answer"); + expect(r.value.tool_calls).toHaveLength(6); + expect(handler).toHaveBeenCalledTimes(6); + + // Round 3's request omitted `tools` — the cap forces a final answer. + expect(bodies[2]).not.toHaveProperty("tools"); + + // 2 of round 2's 3 calls got stubbed, never executed. + const round2Messages = bodies[2].messages; + const stubbed = round2Messages.filter( + (m: { content?: string }) => + typeof m.content === "string" && m.content.includes("tool-call budget exhausted"), + ); + expect(stubbed).toHaveLength(2); + }); }); diff --git a/test/eval/llm.test.ts b/test/eval/llm.test.ts index 734abdfb..7c09e632 100644 --- a/test/eval/llm.test.ts +++ b/test/eval/llm.test.ts @@ -68,6 +68,110 @@ describe("temperature passthrough", () => { }); }); +// C5 (spec 2026-07-26-context-packs-progressive-disclosure-design.md, final +// plan Phase 3.3): maxToolCalls caps REALIZED tool calls, not requested +// ones — a round's parallel tool_use blocks can overshoot a naive +// "check-then-execute" cap, so the loop must enforce it call-by-call. +describe("completeWithTools — maxToolCalls cap (C5)", () => { + function makeClientWith(create: ReturnType) { + const prev = process.env.ANTHROPIC_API_KEY; + process.env.ANTHROPIC_API_KEY = "test-key"; + // biome-ignore lint/suspicious/noExplicitAny: minimal SDK stand-in + const client = createAnthropicClient({ messages: { create } } as any); + if (prev) process.env.ANTHROPIC_API_KEY = prev; + else delete process.env.ANTHROPIC_API_KEY; + return client; + } + + function toolUseBlock(id: string) { + return { type: "tool_use", id, name: "probe", input: {} }; + } + + it("a round whose parallel calls would overshoot the cap executes only the remaining slots, stubs the rest, and still reaches a final answer", async () => { + // Round 1: 5 parallel calls, cap=6 (0 used so far) — all 5 execute. + // Round 2: 3 MORE parallel calls, remaining = 6-5 = 1 — only 1 executes, + // 2 are stubbed. Realized total hits the cap (6). + // Round 3: tools omitted (budget exhausted) — the mock still answers + // with plain text, proving the loop forces a final answer rather than + // looping forever or erroring. + const create = vi + .fn() + .mockResolvedValueOnce({ + content: Array.from({ length: 5 }, (_, i) => toolUseBlock(`r1-${i}`)), + usage: { input_tokens: 1, output_tokens: 1 }, + stop_reason: "tool_use", + }) + .mockResolvedValueOnce({ + content: Array.from({ length: 3 }, (_, i) => toolUseBlock(`r2-${i}`)), + usage: { input_tokens: 1, output_tokens: 1 }, + stop_reason: "tool_use", + }) + .mockResolvedValueOnce({ + content: [{ type: "text", text: "final answer", citations: null }], + usage: { input_tokens: 1, output_tokens: 1 }, + stop_reason: "end_turn", + }); + const client = makeClientWith(create); + + const toolHandler = vi.fn(async () => "ok"); + const r = await client.completeWithTools({ + model: "m", + system: "s", + user: "u", + tools: [{ name: "probe", description: "d", input_schema: { type: "object" } }], + toolHandler, + maxToolCalls: 6, + }); + + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.tool_calls).toHaveLength(6); // realized calls, never more than the cap + expect(r.value.text).toBe("final answer"); + expect(toolHandler).toHaveBeenCalledTimes(6); + + // Round 3's request must have omitted `tools` — the cap forces the + // final answer rather than merely hoping the model stops asking. + expect(create.mock.calls[2][0]).not.toHaveProperty("tools"); + + // The 2 stubbed calls from round 2 got a tool_result each (the API + // requires one per tool_use id) carrying the budget-exhausted marker, + // and were never counted. + const round2ToolResults = create.mock.calls[2][0].messages.at(-1).content; + const stubbed = round2ToolResults.filter((m: { content: string }) => + m.content.includes("tool-call budget exhausted"), + ); + expect(stubbed).toHaveLength(2); + }); + + it("uncapped (maxToolCalls unset) behaves exactly as before — every requested call executes", async () => { + const create = vi + .fn() + .mockResolvedValueOnce({ + content: [toolUseBlock("a"), toolUseBlock("b")], + usage: { input_tokens: 1, output_tokens: 1 }, + stop_reason: "tool_use", + }) + .mockResolvedValueOnce({ + content: [{ type: "text", text: "done", citations: null }], + usage: { input_tokens: 1, output_tokens: 1 }, + stop_reason: "end_turn", + }); + const client = makeClientWith(create); + const toolHandler = vi.fn(async () => "ok"); + const r = await client.completeWithTools({ + model: "m", + system: "s", + user: "u", + tools: [{ name: "probe", description: "d", input_schema: { type: "object" } }], + toolHandler, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.tool_calls).toHaveLength(2); + expect(toolHandler).toHaveBeenCalledTimes(2); + }); +}); + describe("stripCodeFence", () => { it("strips a ```json fenced block", () => { expect(stripCodeFence('```json\n{"a":1}\n```')).toBe('{"a":1}'); diff --git a/test/eval/pack-condition.test.ts b/test/eval/pack-condition.test.ts new file mode 100644 index 00000000..f1b955ca --- /dev/null +++ b/test/eval/pack-condition.test.ts @@ -0,0 +1,149 @@ +import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import type { CompleteOpts, LlmClient } from "../../src/eval/llm.js"; +import { runPackAnswerer } from "../../src/eval/pack-condition.js"; +import type { EvalRun, Question, QuestionSet } from "../../src/eval/types.js"; +import { reindexVault } from "../../src/search/reindex.js"; +import { vaultContext } from "../../src/tools/context.js"; + +const sampleQs: QuestionSet = { + id: "qs-pack-1", + vault_hash: "h", + seed: "s", + timestamp: "t", + subgraph: { seed_doc: "a.md", nodes: ["a.md"], edges: [] }, + questions: [ + { + id: "q1", + tier: "retrieval", + question: "widget launch plan", + expected_answer: "the widget launches in Q1", + expected_sources: ["notes/a.md"], + origin: "generated", + }, + ] as Question[], + generator_model: "g", + prompt_version: 1, + tier_counts_requested: { retrieval: 1, cross_reference: 0, contradiction: 0 }, + tier_counts_produced: { retrieval: 1, cross_reference: 0, contradiction: 0 }, +}; + +function mockLlm(onComplete?: (opts: CompleteOpts) => void): LlmClient { + return { + complete: async (opts) => { + onComplete?.(opts); + return { + ok: true, + value: { + text: "the widget launches in Q1 [notes/a.md]", + input_tokens: 7, + output_tokens: 3, + stop_reason: "end_turn", + }, + }; + }, + completeJson: async () => ({ + ok: false, + error: { kind: "llm", message: "not used", retryable: false }, + }), + completeWithTools: async () => ({ + ok: false, + error: { kind: "llm", message: "the pack condition must never call this", retryable: false }, + }), + }; +} + +describe("runPackAnswerer", () => { + let vault: string; + + beforeAll(async () => { + vault = mkdtempSync(join(tmpdir(), "daftari-pack-condition-")); + mkdirSync(join(vault, "notes"), { recursive: true }); + writeFileSync( + join(vault, "notes", "a.md"), + "---\ntitle: A\ncollection: notes\ndomain: product\nstatus: canonical\n" + + "confidence: high\ncreated: 2026-01-01\nupdated: 2026-01-01\ntags: []\n---\n\n" + + "widget launch plan ".repeat(20), + ); + const reindexed = await reindexVault(vault); + if (!reindexed.ok) throw reindexed.error; + }, 60_000); + + afterAll(() => {}); + + it("produces a zero-tool-call Trace carrying pack metadata", async () => { + const r = await runPackAnswerer(sampleQs, vault, mockLlm(), { + k: 1, + model: "fake", + budget: 4000, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.condition).toBe("pack"); + expect(r.value.pack_budget).toBe(4000); + const pr = r.value.runs["0:0"]; + expect(pr.status).toBe("complete"); + if (pr.status !== "complete") return; + expect(pr.trace.tool_calls).toEqual([]); + expect(pr.trace.total_tool_calls).toBe(0); + expect(pr.trace.pack).toBeTruthy(); + expect(typeof pr.trace.pack?.estimated_tokens).toBe("number"); + expect(Array.isArray(pr.trace.pack?.included_paths)).toBe(true); + }); + + it("the brief handed to the LLM is byte-identical to vaultContext's own return", async () => { + let seenUser = ""; + const r = await runPackAnswerer( + sampleQs, + vault, + mockLlm((opts) => { + seenUser = opts.user; + }), + { k: 1, model: "fake", budget: 4000 }, + ); + expect(r.ok).toBe(true); + const direct = await vaultContext( + vault, + { task: sampleQs.questions[0].question, budget: 4000 }, + undefined, + ); + expect(direct.ok).toBe(true); + if (!direct.ok) return; + expect(seenUser).toBe(direct.value.brief); + }); + + it("resume skips already-complete (q,k) pairs", async () => { + const seeded = await runPackAnswerer(sampleQs, vault, mockLlm(), { + k: 2, + model: "fake", + budget: 4000, + }); + if (!seeded.ok) throw new Error("seed failed"); + const partial: EvalRun = { + ...seeded.value, + runs: { + "0:0": seeded.value.runs["0:0"], + "0:1": { + question_id: "q1", + question_index: 0, + k_index: 1, + status: "incomplete", + trace: null, + }, + }, + }; + let calls = 0; + const r = await runPackAnswerer( + sampleQs, + vault, + mockLlm(() => { + calls++; + }), + { k: 2, model: "fake", budget: 4000, resumeFrom: partial }, + ); + expect(r.ok).toBe(true); + expect(calls).toBe(1); // only the incomplete pair re-ran + }); +}); diff --git a/test/eval/run.test.ts b/test/eval/run.test.ts index 8e71a411..f4ed7645 100644 --- a/test/eval/run.test.ts +++ b/test/eval/run.test.ts @@ -191,4 +191,47 @@ describe("runAnswerer", () => { expect(last.runs["0:0"].status).toBe("complete"); expect(last.runs["0:1"].status).toBe("incomplete"); }); + + // spec 2026-07-26-context-packs-progressive-disclosure-design.md, final + // plan Phase 3.3/C8. + it("stamps condition: 'tools' and threads maxToolCalls through to completeWithTools", async () => { + let seenMaxToolCalls: number | undefined; + const client: LlmClient = { + ...mockClient(), + completeWithTools: async (opts) => { + seenMaxToolCalls = opts.maxToolCalls; + return { + ok: true, + value: { + text: "X is foo [a.md]", + input_tokens: 1, + output_tokens: 1, + stop_reason: "end_turn", + tool_calls: [], + }, + }; + }, + }; + const r = await runAnswerer(sampleQs, "/tmp/fake-vault", client, { + k: 1, + model: "claude-sonnet-fake", + maxToolCalls: 6, + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.condition).toBe("tools"); + expect(r.value.max_tool_calls).toBe(6); + expect(seenMaxToolCalls).toBe(6); + }); + + it("max_tool_calls is absent (not just undefined-valued) when unset", async () => { + const r = await runAnswerer(sampleQs, "/tmp/fake-vault", mockClient(), { + k: 1, + model: "claude-sonnet-fake", + }); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect("max_tool_calls" in r.value).toBe(false); + expect(r.value.condition).toBe("tools"); + }); }); diff --git a/test/eval/score.test.ts b/test/eval/score.test.ts index 0faba981..f51b6c92 100644 --- a/test/eval/score.test.ts +++ b/test/eval/score.test.ts @@ -23,13 +23,13 @@ function g(question: Question, k: number, v: "yes" | "partial" | "no" | "ungrade grader_model: "claude-sonnet-fake", }; } -function tr(totalToolCalls: number): Trace { +function tr(totalToolCalls: number, inputTokens = 0, outputTokens = 0): Trace { return { tool_calls: [], final_answer: "", total_tool_calls: totalToolCalls, - input_tokens: 0, - output_tokens: 0, + input_tokens: inputTokens, + output_tokens: outputTokens, wall_ms: 0, stop_reason: "end_turn", }; @@ -105,6 +105,22 @@ describe("aggregateScore", () => { const s = aggregateScore(grades, qs, { traces }); expect(s.by_tier.retrieval.trace_efficiency).toBeCloseTo(3); }); + + // mean_tokens (spec 2026-07-26-context-packs-progressive-disclosure- + // design.md, final plan Phase 3.2): the pack-condition twin of + // trace_efficiency — same correct/partial-only population, total + // (input+output) tokens instead of tool-call count. + it("mean_tokens averages total tokens over correct/partial runs only", () => { + const qs = [q("retrieval", 0)]; + const grades = [g(qs[0], 0, "yes"), g(qs[0], 1, "partial"), g(qs[0], 2, "no")]; + const traces = new Map([ + [`${qs[0].id}:0`, tr(0, 100, 50)], // 150 total + [`${qs[0].id}:1`, tr(0, 40, 10)], // 50 total + [`${qs[0].id}:2`, tr(0, 9999, 9999)], // a 'no' run — excluded + ]); + const s = aggregateScore(grades, qs, { traces }); + expect(s.by_tier.retrieval.mean_tokens).toBeCloseTo((150 + 50) / 2); + }); }); function graderClient(verdict: "yes" | "partial" | "no"): LlmClient { diff --git a/test/helpers/output-schema.ts b/test/helpers/output-schema.ts new file mode 100644 index 00000000..ee4eafa7 --- /dev/null +++ b/test/helpers/output-schema.ts @@ -0,0 +1,76 @@ +// Test-only ajv compilation/validation of tool `outputSchema`s (spec +// 2026-07-26, Decision 3, PR 1 gap closure / jugalbandi challenge C6). +// +// Production code never validates outputs at runtime — outputSchema is a +// contract on the wire shape, and this file is how the contract is +// enforced: every registered tool's schema must compile under strict +// JSON Schema 2020-12, and every value a handler test asserts must +// validate against its own tool's schema. +// +// `strict: true` is deliberate, not incidental (C6): `strict: false` (ajv's +// default when unset) SILENTLY ACCEPTS a misspelled keyword — `eunm` +// instead of `enum` compiles and matches everything, which would let this +// helper certify a typo'd schema as correct. Strict mode makes a misspelled +// keyword a compile-time failure. Where a schema genuinely needs a +// non-standard keyword, it goes in ALLOWED_VOCABULARY below — an explicit, +// reviewable relaxation, never a blanket one. + +import type { ErrorObject, ValidateFunction } from "ajv"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect } from "vitest"; +import type { ToolDefinition } from "../../src/tools/read.js"; + +// No non-standard keywords are in use today. A tool that legitimately needs +// one adds it here, by name, with a one-line justification — never a +// blanket `strict: false`. +const ALLOWED_VOCABULARY: Record = {}; + +function makeAjv(): Ajv2020 { + const ajv = new Ajv2020({ + strict: true, + // The registry's schemas use JSON Schema union types throughout + // (`type: ["string", "null"]` for every "absent means nothing to say" + // field — the decay/validity/structural contract). That is standard + // 2020-12, not a laxness; ajv's strict mode requires this opt-in + // separately from `strict: true` itself. + allowUnionTypes: true, + }); + if (Object.keys(ALLOWED_VOCABULARY).length > 0) { + ajv.addVocabulary(Object.keys(ALLOWED_VOCABULARY)); + } + return ajv; +} + +// One ajv instance for the whole test run — schemas are static, compiling +// per-call would just be slower for no benefit. +const ajv = makeAjv(); +const compiled = new Map(); + +// Compiles (and caches) a tool's outputSchema. Throws ajv's own compile +// error on a genuinely invalid or misspelled schema — callers that just want +// "does this compile" should wrap this in `expect(() => ...).not.toThrow()`. +export function compileToolSchema(tool: ToolDefinition): ValidateFunction { + const cached = compiled.get(tool.name); + if (cached) return cached; + // ajv keys its internal schema cache by $id; two tools' schemas are + // structurally independent (they're plain object literals with no shared + // $id), so compiling per tool name is correct and collision-free. + const validate = ajv.compile(tool.outputSchema); + compiled.set(tool.name, validate); + return validate; +} + +function formatErrors(errors: ErrorObject[] | null | undefined): string { + return (errors ?? []).map((e) => `${e.instancePath || "(root)"} ${e.message}`).join("; "); +} + +// Asserts `value` validates against `tool`'s own outputSchema. Failure +// message includes ajv's error paths so a broken assertion points straight +// at the offending field instead of a bare "expected true, got false". +export function expectMatchesOutputSchema(tool: ToolDefinition, value: unknown): void { + const validate = compileToolSchema(tool); + const ok = validate(value); + expect(ok, `${tool.name} output failed schema validation: ${formatErrors(validate.errors)}`).toBe( + true, + ); +} diff --git a/test/ratify-elicitation.test.ts b/test/ratify-elicitation.test.ts new file mode 100644 index 00000000..8aaef393 --- /dev/null +++ b/test/ratify-elicitation.test.ts @@ -0,0 +1,229 @@ +// vault_ratify form-mode elicitation (spec 2026-07-26, Decision 5), driven +// end-to-end over the 2026-07-28 wire: the server answers a decision-less +// vault_ratify with an input_required form plus HMAC-signed opaque state, the +// client fulfils it through its elicitation/create handler, and the SDK +// retries the call with the answer and the echoed state — the server +// remembers nothing in between. The default (and only safe preselection) is +// reject; a declined form applies nothing and leaves the action pending. + +import { Client } from "@modelcontextprotocol/client"; +import { InMemoryTransport } from "@modelcontextprotocol/server"; +import { type StdioServerHandle, serveStdio } from "@modelcontextprotocol/server/stdio"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { AccessContext } from "../src/access/rbac.js"; +import { getStagedActionById } from "../src/curation/staged-actions.js"; +import { createServer } from "../src/server.js"; +import { vaultStageAction } from "../src/tools/staged-actions.js"; +import { cleanupVault, makeTempVault } from "./helpers/temp-vault.js"; + +const RATIFIER: AccessContext = { + user: "human:mihir", + roleName: "ratifier", + role: { read: ["*"], write: ["*"], promote: true, ratify: true }, +}; + +const HUMAN = "human:mihir"; + +// A `write` proposal is the cleanest approve fixture: dispatch creates a new +// draft document, so no tier-0 gate participates in the assertion. +async function stageWrite(vault: string): Promise { + const staged = await vaultStageAction(vault, { + action_type: "write", + target_path: "pricing/elicited.md", + proposed_by: "agent:loop", + rationale: "Synthesized from run traces.", + proposed_diff: { + frontmatter: { + title: "Elicited", + domain: "accumulation", + collection: "pricing", + status: "draft", + confidence: "medium", + created: "2026-07-28", + provenance: "direct", + sources: [], + superseded_by: null, + ttl_days: 90, + tags: ["spec"], + }, + body: "# Elicited\n\nProposed content.\n", + }, + }); + if (!staged.ok) throw staged.error; + return staged.value.id; +} + +type ElicitAnswer = + | { action: "accept"; content: Record } + | { action: "decline" } + | { action: "cancel" }; + +interface Harness { + client: Client; + // Every elicitation request the client saw, for asserting the form shape. + seen: Array<{ message?: string; requestedSchema?: Record }>; + close: () => Promise; +} + +async function connectHarness( + vault: string, + access: AccessContext | undefined, + answer: ElicitAnswer, +): Promise { + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); + // serveStdio owns the era decision (a bare Server.connect speaks legacy + // only); the injected in-memory transport keeps the harness in-process. + const handle: StdioServerHandle = serveStdio( + () => (access ? createServer(vault, access) : createServer(vault)), + { transport: serverTransport }, + ); + + const seen: Harness["seen"] = []; + const client = new Client( + { name: "ratify-elicitation-test", version: "0.0.0" }, + { + capabilities: { elicitation: {} }, + versionNegotiation: { mode: { pin: "2026-07-28" } }, + }, + ); + client.setRequestHandler("elicitation/create", async (request) => { + const params = request.params as { + message?: string; + requestedSchema?: Record; + }; + seen.push({ message: params.message, requestedSchema: params.requestedSchema }); + return answer; + }); + await client.connect(clientTransport); + return { + client, + seen, + close: async () => { + await client.close(); + await handle.close(); + }, + }; +} + +describe("vault_ratify form-mode elicitation (Decision 5)", () => { + let vault: string; + beforeEach(() => { + vault = makeTempVault(); + }); + afterEach(() => { + cleanupVault(vault); + }); + + it("a decision-less call elicits a form (default reject) and an accepted approve applies", async () => { + const id = await stageWrite(vault); + const h = await connectHarness(vault, RATIFIER, { + action: "accept", + content: { decision: "approve" }, + }); + try { + const res = await h.client.callTool({ + name: "vault_ratify", + arguments: { id, principal: HUMAN }, + }); + expect(res.isError).toBeFalsy(); + const payload = res.structuredContent as { + action_id: string; + decision: string; + applied: boolean; + }; + expect(payload.action_id).toBe(id); + expect(payload.decision).toBe("approve"); + expect(payload.applied).toBe(true); + + // The form the human saw: the action named in the prompt, and reject + // preselected — the safe answer is the default one. + expect(h.seen).toHaveLength(1); + expect(h.seen[0]?.message).toContain(`Ratify staged action ${id}`); + const schema = h.seen[0]?.requestedSchema as { + properties?: { decision?: { enum?: string[]; default?: string } }; + }; + expect(schema?.properties?.decision?.enum).toEqual(["approve", "reject"]); + expect(schema?.properties?.decision?.default).toBe("reject"); + } finally { + await h.close(); + } + }, 60_000); + + it("a declined form applies nothing and leaves the action pending", async () => { + const id = await stageWrite(vault); + const h = await connectHarness(vault, RATIFIER, { action: "decline" }); + try { + const res = await h.client.callTool({ + name: "vault_ratify", + arguments: { id, principal: HUMAN }, + }); + expect(res.isError).toBeFalsy(); + const text = (res.content as Array<{ text?: string }>)[0]?.text ?? ""; + expect(text).toContain("remains pending"); + + const action = await getStagedActionById(vault, id); + expect(action.ok && action.value?.status).toBe("pending"); + expect(h.seen).toHaveLength(1); + } finally { + await h.close(); + } + }, 60_000); + + it("a direct call with the decision inline never elicits", async () => { + const id = await stageWrite(vault); + const h = await connectHarness(vault, RATIFIER, { + action: "accept", + content: { decision: "approve" }, + }); + try { + const res = await h.client.callTool({ + name: "vault_ratify", + arguments: { id, decision: "reject", principal: HUMAN }, + }); + expect(res.isError).toBeFalsy(); + const payload = res.structuredContent as { decision: string; applied: boolean }; + expect(payload.decision).toBe("reject"); + expect(payload.applied).toBe(false); + expect(h.seen).toHaveLength(0); + } finally { + await h.close(); + } + }, 60_000); + + it("the gates run before any form: an unknown action errors, a role without the grant is denied", async () => { + const id = await stageWrite(vault); + + const unknown = await connectHarness(vault, RATIFIER, { + action: "accept", + content: { decision: "approve" }, + }); + try { + const res = await unknown.client.callTool({ + name: "vault_ratify", + arguments: { id: "stage-nope", principal: HUMAN }, + }); + expect(res.isError).toBe(true); + expect((res.content as Array<{ text?: string }>)[0]?.text).toContain("unknown staged action"); + expect(unknown.seen).toHaveLength(0); + } finally { + await unknown.close(); + } + + // The deny-all guest never sees a form — access denied before the round. + const guest = await connectHarness(vault, undefined, { + action: "accept", + content: { decision: "approve" }, + }); + try { + const res = await guest.client.callTool({ + name: "vault_ratify", + arguments: { id, principal: HUMAN }, + }); + expect(res.isError).toBe(true); + expect((res.content as Array<{ text?: string }>)[0]?.text).toContain("access denied"); + expect(guest.seen).toHaveLength(0); + } finally { + await guest.close(); + } + }, 60_000); +}); diff --git a/test/regression/retrieval/retrieval.test.ts b/test/regression/retrieval/retrieval.test.ts index f62bfbdf..4b282486 100644 --- a/test/regression/retrieval/retrieval.test.ts +++ b/test/regression/retrieval/retrieval.test.ts @@ -63,7 +63,7 @@ describe("retrieval regression (lexical BM25, native-shape vault)", () => { expect(reindexed.value.skipped).toEqual([]); expect(reindexed.value.invalidFrontmatter).toEqual([]); expect(reindexed.value.documentCount).toBe(100); - const opened = openIndexDb(vault, STUB_DIM); + const opened = openIndexDb(vault, STUB_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; diff --git a/test/search/acl-pushdown.test.ts b/test/search/acl-pushdown.test.ts index 99deffad..3c383feb 100644 --- a/test/search/acl-pushdown.test.ts +++ b/test/search/acl-pushdown.test.ts @@ -87,7 +87,7 @@ let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, DIM); + const opened = openIndexDb(vault, DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); @@ -242,7 +242,7 @@ describe("reindex writes one vec row per (hash, collection)", () => { const result = await reindexVault(writeVault); expect(result.ok).toBe(true); - const opened = openIndexDb(writeVault, DIM); + const opened = openIndexDb(writeVault, DIM, "float32"); if (!opened.ok) throw opened.error; try { const rows = opened.value diff --git a/test/search/bm25.test.ts b/test/search/bm25.test.ts index cce640a8..81a08cc9 100644 --- a/test/search/bm25.test.ts +++ b/test/search/bm25.test.ts @@ -51,3 +51,54 @@ describe("buildMatchQuery", () => { expect(buildMatchQuery(`"cirrus" AND "pricing"`)).toBe("cirrus* OR pricing*"); }); }); + +describe("buildMatchQuery — phrase emission (Decision 2)", () => { + it("adds a phrase branch for a quoted span of >= 2 usable tokens", () => { + // The prefix-OR branches for the individual tokens survive UNCHANGED — + // the phrase branch is an ADDITION, not a replacement (recall-non-shrinking). + expect(buildMatchQuery(`"cirrus pricing"`)).toBe('cirrus* OR pricing* OR "cirrus pricing"'); + }); + + it("tokenizes the phrase's contents the same way as the rest of the query", () => { + // Stopwords/punctuation inside the quotes are dropped before the phrase + // is assembled, exactly like the prefix-token path. + expect(buildMatchQuery(`"the Cirrus-Pricing, model!"`)).toBe( + 'cirrus* OR pricing* OR model* OR "cirrus pricing model"', + ); + }); + + it("degrades to today's behaviour for a single-token quoted span", () => { + expect(buildMatchQuery(`"cirrus"`)).toBe("cirrus*"); + }); + + it("degrades to today's behaviour for an empty or stopword-only quoted span", () => { + expect(buildMatchQuery(`""`)).toBe(null); + expect(buildMatchQuery(`"the of"`)).toBe(null); + }); + + it("degrades to today's behaviour for a stray unmatched quote", () => { + expect(buildMatchQuery(`cirrus "pricing`)).toBe("cirrus* OR pricing*"); + }); + + it("handles multiple quoted phrases in one query, each its own branch", () => { + expect(buildMatchQuery(`"cirrus pricing" and "capacity tiers"`)).toBe( + 'cirrus* OR pricing* OR capacity* OR tiers* OR "cirrus pricing" OR "capacity tiers"', + ); + }); + + it("deduplicates an identical phrase branch", () => { + expect(buildMatchQuery(`"cirrus pricing" "cirrus pricing"`)).toBe( + 'cirrus* OR pricing* OR "cirrus pricing"', + ); + }); + + it("recall superset property: every document matching the old prefix-only query still matches", () => { + // The phrase branch is OR'd in alongside every prefix branch the + // pre-Decision-2 query produced — so the new MATCH string is a strict + // superset match, never a subset. + const withPhrase = buildMatchQuery(`"cirrus pricing"`); + const prefixOnly = "cirrus* OR pricing*"; + expect(withPhrase).not.toBeNull(); + expect(withPhrase?.startsWith(prefixOnly)).toBe(true); + }); +}); diff --git a/test/search/coverage.test.ts b/test/search/coverage.test.ts index 5a81a0fc..d08eb80c 100644 --- a/test/search/coverage.test.ts +++ b/test/search/coverage.test.ts @@ -53,7 +53,7 @@ describe("detectSharedEntity", () => { let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const o = openIndexDb(vault, LOCAL_MINILM_DIM); + const o = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!o.ok) throw o.error; db = o.value; }); @@ -107,7 +107,7 @@ describe("computeWindow", () => { let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const o = openIndexDb(vault, LOCAL_MINILM_DIM); + const o = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!o.ok) throw o.error; db = o.value; }); @@ -180,7 +180,7 @@ describe("applyCoveragePass", () => { let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const o = openIndexDb(vault, LOCAL_MINILM_DIM); + const o = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!o.ok) throw o.error; db = o.value; }); diff --git a/test/search/current-source.test.ts b/test/search/current-source.test.ts index b88473c7..90d1554d 100644 --- a/test/search/current-source.test.ts +++ b/test/search/current-source.test.ts @@ -44,7 +44,7 @@ describe("resolveCurrentSource", () => { beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); diff --git a/test/search/hybrid.test.ts b/test/search/hybrid.test.ts index f1f5574d..897b8ae7 100644 --- a/test/search/hybrid.test.ts +++ b/test/search/hybrid.test.ts @@ -1,10 +1,12 @@ import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { err } from "../../src/frontmatter/types.js"; import { DEFAULT_WEIGHTS, hybridSearch, relatedSearch } from "../../src/search/hybrid.js"; import { LOCAL_MINILM_DIM } from "../../src/search/providers/local-minilm.js"; import { reindexVault } from "../../src/search/reindex.js"; +import * as vectorMod from "../../src/search/vector.js"; import * as indexDb from "../../src/storage/index-db.js"; import { type IndexDb, openIndexDb } from "../../src/storage/index-db.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; @@ -22,7 +24,7 @@ describe("hybrid search", () => { vault = makeTempVault(); const reindexed = await reindexVault(vault); if (!reindexed.ok) throw reindexed.error; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }, 60_000); @@ -307,7 +309,7 @@ Reference material for general use across the vault. const reindexed = await reindexVault(decayVault); if (!reindexed.ok) throw reindexed.error; - const opened = openIndexDb(decayVault, LOCAL_MINILM_DIM); + const opened = openIndexDb(decayVault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; decayDb = opened.value; }, 60_000); @@ -435,7 +437,7 @@ The zephyr system was briefly mentioned in a prior report and has no further det const reindexed = await reindexVault(chunkVault); if (!reindexed.ok) throw reindexed.error; - const opened = openIndexDb(chunkVault, LOCAL_MINILM_DIM); + const opened = openIndexDb(chunkVault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; chunkDb = opened.value; }, 60_000); @@ -678,7 +680,7 @@ Additional filler content for padding the vault retrieval set. const reindexed = await reindexVault(ttVault); if (!reindexed.ok) throw reindexed.error; - const opened = openIndexDb(ttVault, LOCAL_MINILM_DIM); + const opened = openIndexDb(ttVault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; ttDb = opened.value; }, 60_000); @@ -708,10 +710,16 @@ Additional filler content for padding the vault retrieval set. expect(res.value.hits[0]?.path).toBe("tagdoc.md"); }); - // bodywin.md has "tiebreak" in its body → upper band (>0.5). titlecoincidence.md - // has it title-only → lower band (<=0.5). The tier boundary guarantees body wins; - // titlecoincidence is still retrieved, just ranked below. - it("body match wins a tie against a coincidental title-only match", async () => { + // Post-contextual-chunking (spec 2026-07-26 Decision 2, plan C1): + // titlecoincidence.md's title token now flows into EVERY chunk's context + // column, so it is no longer confined to the lower title/tag-fallback band + // — it enters the upper band via a genuine (context-column) chunk match, + // same as bodywin.md's real body match. bodywin.md still ranks first here, + // but that is BM25's own length-normalization math (bodywin's chunk row is + // much shorter than titlecoincidence's context+body row), not an a-priori + // tier guarantee — see the two new tests below, which pin the actual + // guarantee directly instead of relying on this ordering. + it("body match still ranks first against a coincidental title-only match", async () => { const res = await hybridSearch(ttDb, "tiebreak", { weights: { bm25: 1, vector: 0 }, lexicalGranularity: "chunk", @@ -721,4 +729,624 @@ Additional filler content for padding the vault retrieval set. expect(res.value.hits[0]?.path).toBe("bodywin.md"); expect(res.value.hits.some((h) => h.path === "titlecoincidence.md")).toBe(true); }); + + // C1: the tier invariant this test used to pin ("title-only stays below + // 0.5") no longer holds — asserted directly here rather than inferred from + // ranking order. + it("a title-only match now enters the upper band via its context chunk (C1)", async () => { + const res = await hybridSearch(ttDb, "tiebreak", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + const titleHit = res.value.hits.find((h) => h.path === "titlecoincidence.md"); + expect(titleHit).toBeDefined(); + // TIER_SPLIT (hybrid.ts) is 0.5 — the upper-band boundary. + expect(titleHit?.bm25Score).toBeGreaterThan(0.5); + }); + + // C1: a pure-body match and a context-only match now co-rank by bm25 score + // WITHIN the same upper band, not by the old strict tier rule — both docs + // clear TIER_SPLIT, and their relative order is bm25's business, not the + // tier's. + it("a pure-body match and a context-only match co-rank by bm25, not the old tier rule (C1)", async () => { + const res = await hybridSearch(ttDb, "tiebreak", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + const bodyHit = res.value.hits.find((h) => h.path === "bodywin.md"); + const titleHit = res.value.hits.find((h) => h.path === "titlecoincidence.md"); + expect(bodyHit?.bm25Score).toBeGreaterThan(0.5); + expect(titleHit?.bm25Score).toBeGreaterThan(0.5); + }); + + // The {title tags} fallback tier (spec Decision 2's "stays — a strict, + // harmless fallback") still fires for a document whose CHUNKS are absent + // from chunks_fts — the index-inconsistency case the fallback exists for. + // Simulated directly: insert a document row (with its own documents_fts + // trigger firing normally) but never insert a chunk row for it, so + // chunkFtsRanking has nothing to find and only the column-restricted + // title/tag ftsRanking can surface it. + it("the title/tag fallback still fires when a document has no chunks at all (C1)", async () => { + indexDb.insertDocument(ttDb, { + path: "no-chunks-doc.md", + title: "Zzznoveltermxyz Reference", + collection: "general", + domain: "product", + status: "canonical", + confidence: "high", + updated: "2026-01-01", + tags: [], + content: "irrelevant — no chunk row is ever written for this path", + tokens: ["zzznoveltermxyz", "reference"], + ttlDays: null, + created: "2026-01-01", + supersededBy: null, + validFrom: null, + validUntil: null, + }); + // No insertChunkRow call — this path has zero rows in `chunks`/`chunks_fts`. + try { + const res = await hybridSearch(ttDb, "zzznoveltermxyz", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + const hit = res.value.hits.find((h) => h.path === "no-chunks-doc.md"); + expect(hit).toBeDefined(); + // Lower band: the chunk ranker never saw this doc at all. + expect(hit?.bm25Score).toBeLessThanOrEqual(0.5); + } finally { + indexDb.deleteDocument(ttDb, "no-chunks-doc.md"); + } + }); + + // Under RRF, the tier invariant (every body match precedes every + // title-only match) is preserved at the lexical-LIST layer by + // construction: tieredLexical's output is strict and tie-free, so its + // sorted order feeds rrfContributions unchanged. This pins the ORDERING + // consequence of bodywin's higher bm25Score, not a tier guarantee (C1). + it("preserves bodywin's bm25 lead over titlecoincidence under RRF", async () => { + const res = await hybridSearch(ttDb, "tiebreak", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "chunk", + fusion: "rrf", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.hits[0]?.path).toBe("bodywin.md"); + const bodyIdx = res.value.hits.findIndex((h) => h.path === "bodywin.md"); + const titleIdx = res.value.hits.findIndex((h) => h.path === "titlecoincidence.md"); + expect(bodyIdx).toBeGreaterThanOrEqual(0); + expect(titleIdx).toBeGreaterThan(bodyIdx); + }); +}); + +// --------------------------------------------------------------------------- +// RRF fusion (spec 2026-07-26 fusion overhaul, Decision 1) +// --------------------------------------------------------------------------- +// A small, hand-built fixture with three documents whose ONLY difference is +// how many times "widget" appears (5 / 3 / 1), padded with distinct filler +// tokens so every document is the same length — length normalization cannot +// confound the term-frequency ordering, so document-granularity BM25 ranks +// them top/second/third strictly by widget count, deterministically. +describe("hybrid search — RRF fusion (Decision 1)", () => { + let rrfVault: string; + let rrfDb: IndexDb; + + function widgetDoc(name: string, title: string, widgetCount: number, fillerPrefix: string) { + const widget = Array.from({ length: widgetCount }, () => "widget").join(" "); + const filler = Array.from({ length: 5 - widgetCount }, (_, i) => `${fillerPrefix}${i}`).join( + " ", + ); + return `--- +title: "${title}" +domain: product +collection: general +status: canonical +confidence: high +created: 2026-01-01 +updated: 2026-01-01 +updated_by: human:test +provenance: direct +sources: + - test-source +superseded_by: null +tags: [test] +--- + +# ${title} + +${widget} ${filler} +`; + } + + beforeAll(async () => { + rrfVault = mkdtempSync(join(tmpdir(), "daftari-rrf-")); + writeFileSync(join(rrfVault, "top.md"), widgetDoc("top", "Top Doc", 5, "topfiller")); + writeFileSync(join(rrfVault, "second.md"), widgetDoc("second", "Second Doc", 3, "secfiller")); + writeFileSync(join(rrfVault, "third.md"), widgetDoc("third", "Third Doc", 1, "thirdfiller")); + + const reindexed = await reindexVault(rrfVault); + if (!reindexed.ok) throw reindexed.error; + const opened = openIndexDb(rrfVault, LOCAL_MINILM_DIM, "float32"); + if (!opened.ok) throw opened.error; + rrfDb = opened.value; + }, 60_000); + + afterAll(() => { + rrfDb.close(); + rmSync(rrfVault, { recursive: true, force: true }); + }); + + afterEach(() => vi.restoreAllMocks()); + + it("scores rank 1 at exactly 1.0 and rank r at (k+1)/(k+r) under pure-lexical RRF", async () => { + const res = await hybridSearch(rrfDb, "widget", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "document", + fusion: "rrf", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.hits.map((h) => h.path)).toEqual(["top.md", "second.md", "third.md"]); + expect(res.value.hits[0]?.score).toBeCloseTo(1.0, 12); + expect(res.value.hits[0]?.bm25Score).toBeCloseTo(1.0, 12); + expect(res.value.hits[1]?.score).toBeCloseTo(61 / 62, 12); + expect(res.value.hits[2]?.score).toBeCloseTo(61 / 63, 12); + // vector:0 → the vector list is never even queried, so every hit's + // vectorScore contributes 0. + for (const hit of res.value.hits) expect(hit.vectorScore).toBe(0); + }); + + it("weights: {bm25: 1, vector: 0} + fusion: rrf reproduces the pure tiered lexical ordering", async () => { + const weighted = await hybridSearch(rrfDb, "widget", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "document", + fusion: "weighted", + }); + const rrf = await hybridSearch(rrfDb, "widget", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "document", + fusion: "rrf", + }); + expect(weighted.ok && rrf.ok).toBe(true); + if (!weighted.ok || !rrf.ok) return; + expect(rrf.value.hits.map((h) => h.path)).toEqual(weighted.value.hits.map((h) => h.path)); + }); + + it("defaults to weighted fusion when `fusion` is omitted (regression)", async () => { + const implicit = await hybridSearch(rrfDb, "widget", { lexicalGranularity: "document" }); + const explicit = await hybridSearch(rrfDb, "widget", { + lexicalGranularity: "document", + fusion: "weighted", + }); + expect(implicit.ok && explicit.ok).toBe(true); + if (!implicit.ok || !explicit.ok) return; + expect(implicit.value.hits.map((h) => h.path)).toEqual(explicit.value.hits.map((h) => h.path)); + expect(implicit.value.hits.map((h) => h.score)).toEqual( + explicit.value.hits.map((h) => h.score), + ); + }); + + it("degrades to lexical-only RRF when the embedding provider fails", async () => { + const spy = vi + .spyOn(vectorMod, "embedQuery") + .mockResolvedValue(err(new Error("embedding provider unavailable"))); + + const res = await hybridSearch(rrfDb, "widget", { + lexicalGranularity: "document", + fusion: "rrf", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(spy).toHaveBeenCalled(); + expect(res.value.vectorUsed).toBe(false); + expect(res.value.weights).toEqual({ bm25: 1, vector: 0 }); + expect(res.value.hits.map((h) => h.path)).toEqual(["top.md", "second.md", "third.md"]); + expect(res.value.hits[0]?.score).toBeCloseTo(1.0, 12); + }); + + it("ties break deterministically by path ascending, twice in a row", async () => { + // second.md and third.md have distinct widget counts, so force an exact + // tie instead by querying a term that hits none of them and only + // "second"/"third" via title/tag path — simplest reliable tie: two docs + // scoring identically under document-granularity BM25 for a shared term. + // Reuse "widget" but restrict candidates via bm25:1 so ties are visible + // only among docs with IDENTICAL term frequency: top.md vs a clone. + const cloneVault = mkdtempSync(join(tmpdir(), "daftari-rrf-tie-")); + try { + writeFileSync(join(cloneVault, "alpha.md"), widgetDoc("alpha", "Alpha", 3, "af")); + writeFileSync(join(cloneVault, "beta.md"), widgetDoc("beta", "Beta", 3, "bf")); + const reindexed = await reindexVault(cloneVault); + expect(reindexed.ok).toBe(true); + if (!reindexed.ok) return; + const opened = openIndexDb(cloneVault, LOCAL_MINILM_DIM, "float32"); + expect(opened.ok).toBe(true); + if (!opened.ok) return; + const db = opened.value; + try { + const run = async () => + hybridSearch(db, "widget", { + weights: { bm25: 1, vector: 0 }, + lexicalGranularity: "document", + fusion: "rrf", + }); + const first = await run(); + const second = await run(); + expect(first.ok && second.ok).toBe(true); + if (!first.ok || !second.ok) return; + // Identical term frequency + identical length → tied bm25 → tied + // fused score → deterministic path-ascending tie-break both times. + expect(first.value.hits.map((h) => h.path)).toEqual(["alpha.md", "beta.md"]); + expect(second.value.hits.map((h) => h.path)).toEqual(["alpha.md", "beta.md"]); + } finally { + db.close(); + } + } finally { + rmSync(cloneVault, { recursive: true, force: true }); + } + }, 60_000); + + it("surfaces a semantic-only doc via its vector-list RRF contribution (cross-list fusion)", async () => { + // None of these query words appear anywhere in the vault, so the entire + // lexical contribution is 0 for every hit; any surfaced hit is driven + // purely by the vector list's RRF contribution. + const res = await hybridSearch(rrfDb, "gadgets and gizmos for everyday tasks", { + fusion: "rrf", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.vectorUsed).toBe(true); + expect(res.value.hits.length).toBeGreaterThan(0); + // The top vector-rank doc contributes the full (k+1)/(k+1)=1.0 from its + // vector half; at default 0.5/0.5 weights that alone should out-score + // every doc's zero lexical contribution. + expect(res.value.hits[0]?.vectorScore).toBeCloseTo(1.0, 6); + }); +}); + +describe("relatedSearch — fusion default (Decision 1)", () => { + let relVault: string; + let relDb: IndexDb; + + beforeAll(async () => { + relVault = makeTempVault(); + const reindexed = await reindexVault(relVault); + if (!reindexed.ok) throw reindexed.error; + const opened = openIndexDb(relVault, LOCAL_MINILM_DIM, "float32"); + if (!opened.ok) throw opened.error; + relDb = opened.value; + }, 60_000); + + afterAll(() => { + relDb.close(); + cleanupVault(relVault); + }); + + it("defaults to weighted fusion (independent of hybridSearch's DEFAULT_FUSION)", () => { + const path = "pricing/helios-consumption-pricing.md"; + const implicit = relatedSearch(relDb, path, { limit: 4 }); + const explicitWeighted = relatedSearch(relDb, path, { limit: 4, fusion: "weighted" }); + expect(implicit.ok && explicitWeighted.ok).toBe(true); + if (!implicit.ok || !explicitWeighted.ok) return; + expect(implicit.value.hits.map((h) => h.path)).toEqual( + explicitWeighted.value.hits.map((h) => h.path), + ); + expect(implicit.value.hits.map((h) => h.score)).toEqual( + explicitWeighted.value.hits.map((h) => h.score), + ); + }); + + it("accepts fusion: rrf as an explicit opt-in", () => { + const path = "pricing/helios-consumption-pricing.md"; + const res = relatedSearch(relDb, path, { limit: 4, fusion: "rrf" }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.hits.length).toBeGreaterThan(0); + }); +}); + +// --------------------------------------------------------------------------- +// Part B passage-ref provenance (spec 2026-07-26-contextual-chunking- +// reranker-design.md, plan C4). Hand-built rows (no reindex, no real +// embedding model) so the lexical and vector signals for one document are +// independently controllable — a real reindex could never guarantee one +// path's vector similarity beats its own lexical score deterministically. +// --------------------------------------------------------------------------- +describe("hybridSearch — passage ref provenance (capturePassageRefs, C4)", () => { + const DIM = 4; + + function vec(axis: number): Float32Array { + const v = new Float32Array(DIM); + v[axis % DIM] = 1; + return v; + } + + function doc(path: string, collection: string): indexDb.IndexedDocument { + return { + path, + title: path, + collection, + domain: "accumulation", + status: "canonical", + confidence: "high", + updated: "2026-05-01", + tags: [], + content: `body of ${path}`, + tokens: ["body"], + ttlDays: null, + created: "2026-01-01", + supersededBy: null, + validFrom: null, + validUntil: null, + }; + } + + let vault: string; + let db: IndexDb; + + beforeEach(() => { + vault = makeTempVault(); + const opened = openIndexDb(vault, DIM, "float32"); + if (!opened.ok) throw opened.error; + db = opened.value; + + // A fake embedding provider so the QUERY embeds to a fixed, known vector + // (vec(1)) without loading a real model — the chunk-side vectors are + // written directly into embeddings_vec below, never through embed(). + vectorMod.setProviderForTests({ + id: "fake-provenance", + dim: DIM, + warm: async () => ({ ok: true, value: undefined }) as const, + embed: async (texts) => ({ ok: true, value: texts.map(() => vec(1)) }) as const, + }); + + const model = vectorMod.getProvider().id; + + // strong-lexical.md: the query term repeated many times (strong bm25), + // but its vector is orthogonal to the query embedding (weak similarity). + indexDb.insertDocument(db, doc("strong-lexical.md", "general")); + indexDb.insertChunkRow(db, { + path: "strong-lexical.md", + chunkIndex: 0, + text: "provenance provenance provenance provenance provenance provenance", + context: "general › strong-lexical.md", + contentHash: "hash-strong-lexical", + }); + indexDb.insertEmbedding(db, "hash-strong-lexical", model, vec(1), "2026-05-01", DIM); + indexDb.insertEmbeddingVec(db, "hash-strong-lexical", model, "general", vec(2)); // orthogonal + + // target.md: the query term appears once, diluted by filler (weak bm25 + // relative to strong-lexical.md's chunk) in chunk 0; chunk 1 carries no + // lexical signal at all but an embeddings_vec row at vec(1) — a PERFECT + // match to the query embedding (the best, and only meaningful, vector hit). + indexDb.insertDocument(db, doc("target.md", "general")); + indexDb.insertChunkRow(db, { + path: "target.md", + chunkIndex: 0, + text: "filler filler filler filler filler filler filler filler provenance filler filler filler filler", + context: "general › target.md", + contentHash: "hash-target-lexical", + }); + indexDb.insertEmbedding(db, "hash-target-lexical", model, vec(3), "2026-05-01", DIM); + indexDb.insertEmbeddingVec(db, "hash-target-lexical", model, "general", vec(3)); // far from query + indexDb.insertChunkRow(db, { + path: "target.md", + chunkIndex: 1, + text: "no lexical signal here whatsoever", + context: "general › target.md", + contentHash: "hash-target-vector", + }); + indexDb.insertEmbedding(db, "hash-target-vector", model, vec(1), "2026-05-01", DIM); + indexDb.insertEmbeddingVec(db, "hash-target-vector", model, "general", vec(1)); // perfect match + }); + + afterEach(() => { + db.close(); + cleanupVault(vault); + vectorMod.resetProviderForTests(); + }); + + it("a hit whose vector signal outscores its lexical signal presents its KNN chunk (C4)", async () => { + const res = await hybridSearch(db, "provenance", { + weights: { bm25: 0.5, vector: 0.5 }, + lexicalGranularity: "chunk", + capturePassageRefs: true, + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + // Sanity: target.md's lexical score is real but weak relative to + // strong-lexical.md's — and its vector score is the strongest in the + // corpus (a perfect match), so vector must win the provenance choice. + const targetHit = res.value.hits.find((h) => h.path === "target.md"); + expect(targetHit).toBeDefined(); + const ref = res.value.passageRefs?.["target.md"]; + expect(ref).toEqual({ kind: "vector", contentHash: "hash-target-vector" }); + }); + + it("passageRefs is absent unless capturePassageRefs was requested", async () => { + const res = await hybridSearch(db, "provenance", { + weights: { bm25: 0.5, vector: 0.5 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.passageRefs).toBeUndefined(); + }); +}); + +// --------------------------------------------------------------------------- +// int8 scan-then-rescore (spec 2026-07-26-embedding-refresh-quantization, +// Phase 3c / disposition C3). Hand-built rows, same convention as the +// passage-ref provenance block above: no reindex, no real embedding model, +// full control over which vectors go into the durable cache vs the vec +// mirror so the rescore/orphan behavior is deterministic. +// --------------------------------------------------------------------------- +describe("hybridSearch — int8 scan-then-rescore (C3)", () => { + const DIM = 8; + + function doc(path: string): indexDb.IndexedDocument { + return { + path, + title: path, + collection: "general", + domain: "accumulation", + status: "canonical", + confidence: "high", + updated: "2026-05-01", + tags: [], + content: `body of ${path}`, + tokens: ["body"], + ttlDays: null, + created: "2026-01-01", + supersededBy: null, + validFrom: null, + validUntil: null, + }; + } + + function normalize(v: number[]): Float32Array { + const norm = Math.sqrt(v.reduce((s, x) => s + x * x, 0)); + return new Float32Array(v.map((x) => x / norm)); + } + + let vault: string; + let db: IndexDb; + + beforeEach(() => { + vault = makeTempVault(); + const opened = openIndexDb(vault, DIM, "int8"); // vec mirror created at kind=int8 + if (!opened.ok) throw opened.error; + db = opened.value; + }); + + afterEach(() => { + db.close(); + cleanupVault(vault); + vectorMod.resetProviderForTests(); + }); + + it("rescored ordering matches exact float32 cosine, not raw quantized distance", async () => { + // A close pair (docA, docB) where a single-component int8 rounding + // difference (~1/127) is comparable to their true cosine gap — if the + // rescore ever regressed to scoring off the quantized distance instead + // of the durable-cache float32 vector, this pairing is the one likely + // to flip. docC is far from both, a sanity anchor. + const query = normalize([10, 9, 9, 9, 0, 0, 0, 0]); + const vecA = query; // identical to the query — cos = 1 exactly + const vecB = normalize([10, 9, 9, 8, 0, 0, 0, 0]); // very close, not identical + const vecC = normalize([0, 0, 0, 0, 10, 9, 9, 9]); // far + + vectorMod.setProviderForTests( + { + id: "fake-int8-rescore", + dim: DIM, + warm: async () => ({ ok: true, value: undefined }) as const, + embed: async (texts) => ({ ok: true, value: texts.map(() => query) }) as const, + }, + "int8", + ); + const model = vectorMod.getProvider().id; + + for (const [path, vec] of [ + ["docA.md", vecA], + ["docB.md", vecB], + ["docC.md", vecC], + ] as const) { + indexDb.insertDocument(db, doc(path)); + indexDb.insertChunkRow(db, { + path, + chunkIndex: 0, + text: `irrelevant filler for ${path}`, + context: `general › ${path}`, + contentHash: `hash-${path}`, + }); + // Durable cache: exact float32 (what rescoring reads). + indexDb.insertEmbedding(db, `hash-${path}`, model, vec, "2026-05-01", DIM); + // Vec mirror: int8-quantized (what candidate SELECTION reads). + indexDb.insertEmbeddingVec(db, `hash-${path}`, model, "general", vec, "int8"); + } + + const res = await hybridSearch(db, "irrelevant filler", { + weights: { bm25: 0, vector: 1 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + expect(res.value.vectorUsed).toBe(true); + const order = res.value.hits.map((h) => h.path); + // docC is orthogonal to the query (cosine 0), so a normalized score of 0 + // is legitimately filtered out of hits entirely (score <= 0) — the + // load-bearing assertion is docA before docB, both present. + expect(order.slice(0, 2)).toEqual(["docA.md", "docB.md"]); + + // The exact rescore's own reported vectorScore for docA is the true + // cosine (1.0), not a quantized approximation. + const scoreA = res.value.hits.find((h) => h.path === "docA.md")?.vectorScore; + expect(scoreA).toBeCloseTo(1, 3); + }); + + it("a candidate with a vec-mirror row but no durable cache row (orphan) is dropped, and every score is in [0, 1]", async () => { + const query = normalize([1, 1, 1, 1, 0, 0, 0, 0]); + + vectorMod.setProviderForTests( + { + id: "fake-int8-orphan", + dim: DIM, + warm: async () => ({ ok: true, value: undefined }) as const, + embed: async (texts) => ({ ok: true, value: texts.map(() => query) }) as const, + }, + "int8", + ); + const model = vectorMod.getProvider().id; + + // orphan.md: a PERFECT vector match (would rank #1 if not dropped), and + // NO lexical signal at all — so if it survives the rescore it is the + // ONLY thing that could put it in the candidate set (bm25 weight is 0). + indexDb.insertDocument(db, doc("orphan.md")); + indexDb.insertChunkRow(db, { + path: "orphan.md", + chunkIndex: 0, + text: "nothing lexically relevant here", + context: "general › orphan.md", + contentHash: "hash-orphan", + }); + // Vec mirror row exists (a gc race left it behind)... + indexDb.insertEmbeddingVec(db, "hash-orphan", model, "general", query, "int8"); + // ...but the durable cache row does NOT — orphan. + + // A normal, non-orphaned document for scale/comparison. + indexDb.insertDocument(db, doc("normal.md")); + indexDb.insertChunkRow(db, { + path: "normal.md", + chunkIndex: 0, + text: "also nothing lexically relevant", + context: "general › normal.md", + contentHash: "hash-normal", + }); + const normalVec = normalize([1, 1, 0, 0, 0, 0, 0, 0]); + indexDb.insertEmbedding(db, "hash-normal", model, normalVec, "2026-05-01", DIM); + indexDb.insertEmbeddingVec(db, "hash-normal", model, "general", normalVec, "int8"); + + const res = await hybridSearch(db, "irrelevant query text", { + weights: { bm25: 0, vector: 1 }, + lexicalGranularity: "chunk", + }); + expect(res.ok).toBe(true); + if (!res.ok) return; + const paths = res.value.hits.map((h) => h.path); + expect(paths).not.toContain("orphan.md"); // dropped, not approximated + expect(paths).toContain("normal.md"); + for (const hit of res.value.hits) { + expect(hit.vectorScore).toBeGreaterThanOrEqual(0); + expect(hit.vectorScore).toBeLessThanOrEqual(1); + expect(hit.score).toBeGreaterThanOrEqual(0); + expect(hit.score).toBeLessThanOrEqual(1); + } + }); }); diff --git a/test/search/local-bge-m3-smoke.test.ts b/test/search/local-bge-m3-smoke.test.ts new file mode 100644 index 00000000..bfd8a5a9 --- /dev/null +++ b/test/search/local-bge-m3-smoke.test.ts @@ -0,0 +1,40 @@ +// Real-model smoke test for local-bge-m3 (spec 2026-07-26-contextual- +// chunking-reranker-design.md §3.2). Downloads the ~600MB q8 ONNX weights on +// first run — a 600MB download has no business in default `npm test`, so +// this whole file is skipped unless DAFTARI_BGE_SMOKE is set in env: +// +// DAFTARI_BGE_SMOKE=1 npx vitest run test/search/local-bge-m3-smoke.test.ts +// +// This is the §3.2 latency spike's SANITY half (ordering makes sense over a +// fixture query + 5 passages). The MEASURED half (published per-50-pair CPU +// latency, the actual merge precondition for Part B) is a throwaway script +// run from the scratchpad per the plan, not this committed test. + +import { describe, expect, it } from "vitest"; +import { + localBgeM3Provider, + resetLocalBgeM3ForTests, +} from "../../src/search/providers/local-bge-m3.js"; + +describe.skipIf(!process.env.DAFTARI_BGE_SMOKE)("local-bge-m3 (real model smoke)", () => { + it("scores an obviously-relevant passage above an obviously-irrelevant one", async () => { + resetLocalBgeM3ForTests(); + const warm = await localBgeM3Provider.warm(); + expect(warm.ok).toBe(true); + expect(localBgeM3Provider.isReady()).toBe(true); + + const query = "What is the capital of France?"; + const passages = [ + "Paris is the capital and most populous city of France.", + "The mitochondria is the powerhouse of the cell.", + "Quarterly revenue grew 12% driven by cloud infrastructure spend.", + ]; + const result = await localBgeM3Provider.rerank(query, passages); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toHaveLength(passages.length); + const [parisScore, mitoScore, revenueScore] = result.value; + expect(parisScore).toBeGreaterThan(mitoScore as number); + expect(parisScore).toBeGreaterThan(revenueScore as number); + }, 120_000); +}); diff --git a/test/search/local-embeddinggemma-smoke.test.ts b/test/search/local-embeddinggemma-smoke.test.ts new file mode 100644 index 00000000..a04426f8 --- /dev/null +++ b/test/search/local-embeddinggemma-smoke.test.ts @@ -0,0 +1,59 @@ +// Real-model smoke test for local-embeddinggemma (spec 2026-07-26-embedding- +// refresh-quantization). Downloads the ~600MB q8 ONNX weights on first run — +// no business in default `npm test` — so this whole file is skipped unless +// DAFTARI_EMBEDDINGGEMMA_SMOKE is set in env: +// +// DAFTARI_EMBEDDINGGEMMA_SMOKE=1 npx vitest run test/search/local-embeddinggemma-smoke.test.ts +// +// This is a SANITY check only (semantically-similar text embeds closer than +// dissimilar text, at both offered dims). It is NOT the governing spec's +// Phase 0 spike — that spike additionally requires comparing against a +// Python sentence-transformers reference at cosine >= 0.999 (per-vector) and +// confirming the exact asymmetric prompt-prefix strings against the model +// card; neither of those has been run in this environment. Do not treat a +// green run of this file as spike completion. + +import { describe, expect, it } from "vitest"; +import { + isLocalEmbeddingGemmaLoaded, + makeLocalEmbeddingGemmaProvider, + resetLocalEmbeddingGemmaForTests, +} from "../../src/search/providers/local-embeddinggemma.js"; +import { cosineSimilarity } from "../../src/search/vector.js"; + +describe.skipIf(!process.env.DAFTARI_EMBEDDINGGEMMA_SMOKE)( + "local-embeddinggemma (real model smoke)", + () => { + it("places semantically similar sentences closer than dissimilar ones, at dim=512", async () => { + resetLocalEmbeddingGemmaForTests(); + const provider = makeLocalEmbeddingGemmaProvider(512); + const warm = await provider.warm(); + expect(warm.ok).toBe(true); + expect(isLocalEmbeddingGemmaLoaded()).toBe(true); + + const result = await provider.embed([ + "a cat sat on the mat", + "a kitten rested on the rug", + "quarterly cloud infrastructure budget forecast", + ]); + expect(result.ok).toBe(true); + if (!result.ok) return; + const [catA, catB, budget] = result.value; + if (!catA || !catB || !budget) throw new Error("expected three embeddings"); + expect(catA.length).toBe(768); // embed() returns NATIVE dim + expect(cosineSimilarity(catA, catB)).toBeGreaterThan(cosineSimilarity(catA, budget)); + }, 180_000); + + it("embedQuery() at dim=512 outperforms bare embed() on a query/document pair", async () => { + resetLocalEmbeddingGemmaForTests(); + const provider = makeLocalEmbeddingGemmaProvider(512); + const docs = await provider.embed(["Paris is the capital of France."]); + expect(docs.ok).toBe(true); + if (!docs.ok) return; + const queryResult = await provider.embedQuery?.("What is the capital of France?"); + expect(queryResult?.ok).toBe(true); + if (!queryResult?.ok) return; + expect(queryResult.value.length).toBe(512); + }, 180_000); + }, +); diff --git a/test/search/local-qwen3-smoke.test.ts b/test/search/local-qwen3-smoke.test.ts new file mode 100644 index 00000000..36d3f083 --- /dev/null +++ b/test/search/local-qwen3-smoke.test.ts @@ -0,0 +1,58 @@ +// Real-model smoke test for local-qwen3 (spec 2026-07-26-embedding-refresh- +// quantization). Downloads the ~1.5GB q8 ONNX weights on first run — no +// business in default `npm test` — so this whole file is skipped unless +// DAFTARI_QWEN3_SMOKE is set in env: +// +// DAFTARI_QWEN3_SMOKE=1 npx vitest run test/search/local-qwen3-smoke.test.ts +// +// This is a SANITY check only, and it is the file most likely to fail +// against the real model: local-transformers.ts's last-token pooling path +// (see its file header) is an UNVERIFIED [HYPOTHESIS] pending the governing +// spec's Phase 0 spike, which has not been run in this environment. If this +// file fails, the failure mode to check first is whether +// "pooling: none" on the feature-extraction pipeline actually returns a raw +// per-token [seq_len, hidden] tensor for this model — if not, local- +// transformers.ts needs the AutoModel/AutoTokenizer low-level API instead +// (the spec's named fallback), not a workaround here. + +import { describe, expect, it } from "vitest"; +import { + isLocalQwen3Loaded, + makeLocalQwen3Provider, + resetLocalQwen3ForTests, +} from "../../src/search/providers/local-qwen3.js"; +import { cosineSimilarity } from "../../src/search/vector.js"; + +describe.skipIf(!process.env.DAFTARI_QWEN3_SMOKE)("local-qwen3 (real model smoke)", () => { + it("places semantically similar sentences closer than dissimilar ones, at dim=512", async () => { + resetLocalQwen3ForTests(); + const provider = makeLocalQwen3Provider(512); + const warm = await provider.warm(); + expect(warm.ok).toBe(true); + expect(isLocalQwen3Loaded()).toBe(true); + + const result = await provider.embed([ + "a cat sat on the mat", + "a kitten rested on the rug", + "quarterly cloud infrastructure budget forecast", + ]); + expect(result.ok).toBe(true); + if (!result.ok) return; + const [catA, catB, budget] = result.value; + if (!catA || !catB || !budget) throw new Error("expected three embeddings"); + expect(catA.length).toBe(768); // embed() returns the exposed native dim + expect(cosineSimilarity(catA, catB)).toBeGreaterThan(cosineSimilarity(catA, budget)); + }, 300_000); + + it("embedQuery() returns a configured-dim, unit-norm vector", async () => { + resetLocalQwen3ForTests(); + const provider = makeLocalQwen3Provider(512); + const result = await provider.embedQuery?.("What is the capital of France?"); + expect(result?.ok).toBe(true); + if (!result?.ok) return; + expect(result.value.length).toBe(512); + let norm = 0; + for (const x of result.value) norm += x * x; + expect(Math.sqrt(norm)).toBeCloseTo(1, 3); + }, 300_000); +}); diff --git a/test/search/providers/local-embeddinggemma.test.ts b/test/search/providers/local-embeddinggemma.test.ts new file mode 100644 index 00000000..264f89ea --- /dev/null +++ b/test/search/providers/local-embeddinggemma.test.ts @@ -0,0 +1,136 @@ +// local-embeddinggemma provider coverage (spec 2026-07-26-embedding-refresh- +// quantization, Phase 1c). Mirrors local-minilm.test.ts's shape (id, dim, +// embed shape) but mocks @huggingface/transformers entirely — this provider +// downloads a ~600MB q8 ONNX model on first real use, which has no business +// in default `npm test` (see the file header of local-transformers.ts and +// the governing spec's Phase 0, which has NOT been run against a real model +// in this environment). A real-model smoke test exists separately, gated +// behind DAFTARI_EMBEDDINGGEMMA_SMOKE (not run here). + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +// Deterministic fake "mean-pooled, normalized" extractor: each input text +// maps to a fixed-length vector derived from its char codes, so different +// texts (e.g. prefixed vs unprefixed) produce distinguishably different +// output — enough to assert the provider applied its prompt prefix without +// needing real semantics. +const FAKE_NATIVE_DIM = 768; +function fakeVectorFor(text: string): Float32Array { + const v = new Float32Array(FAKE_NATIVE_DIM); + for (let i = 0; i < FAKE_NATIVE_DIM; i++) { + v[i] = Math.sin(i + 1) * ((text.charCodeAt(i % text.length) ?? 1) + 1); + } + // L2-normalize, matching what pooling:"mean", normalize:true would return. + let norm = 0; + for (let i = 0; i < v.length; i++) norm += (v[i] as number) * (v[i] as number); + const inv = 1 / Math.sqrt(norm || 1); + for (let i = 0; i < v.length; i++) v[i] = (v[i] as number) * inv; + return v; +} + +const pipelineCalls: Array<{ texts: string[]; opts: unknown }> = []; +let failNext = false; + +vi.mock("@huggingface/transformers", () => ({ + pipeline: vi.fn(async () => { + if (failNext) { + failNext = false; + throw new Error("simulated model load failure"); + } + return async (texts: string[], opts: { pooling: "mean" | "none"; normalize: boolean }) => { + pipelineCalls.push({ texts, opts }); + // Mean-pooling batched shape: [batch, dim] flattened. + const data = new Float32Array(texts.length * FAKE_NATIVE_DIM); + texts.forEach((t, i) => { + data.set(fakeVectorFor(t), i * FAKE_NATIVE_DIM); + }); + return { data, dims: [texts.length, FAKE_NATIVE_DIM] }; + }; + }), +})); + +// Imported AFTER the mock is registered (vitest hoists vi.mock calls above +// imports automatically). +const { + makeLocalEmbeddingGemmaProvider, + isLocalEmbeddingGemmaLoaded, + resetLocalEmbeddingGemmaForTests, +} = await import("../../../src/search/providers/local-embeddinggemma.js"); + +describe("local-embeddinggemma provider", () => { + beforeEach(() => { + pipelineCalls.length = 0; + failNext = false; + resetLocalEmbeddingGemmaForTests(); + }); + afterEach(() => { + resetLocalEmbeddingGemmaForTests(); + }); + + it("exposes an id carrying #p1 and no dim, and the configured dim", () => { + const provider = makeLocalEmbeddingGemmaProvider(512); + expect(provider.id).toBe("local-embeddinggemma#p1"); + expect(provider.id).not.toContain("512"); + expect(provider.dim).toBe(512); + expect(provider.nativeDim).toBe(768); + }); + + it("rejects an unsupported dim (not a trained Matryoshka point)", () => { + expect(() => makeLocalEmbeddingGemmaProvider(384)).toThrow(/unsupported dim/); + }); + + it("embed() returns NATIVE-dim vectors with the document prefix applied", async () => { + const provider = makeLocalEmbeddingGemmaProvider(512); + const result = await provider.embed(["hello world"]); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toHaveLength(1); + expect(result.value[0]?.length).toBe(768); + expect(pipelineCalls[0]?.texts[0]).toBe("title: none | text: hello world"); + expect(pipelineCalls[0]?.opts).toMatchObject({ pooling: "mean", normalize: true }); + }); + + it("embedQuery() returns CONFIGURED-dim vectors with the query prefix applied", async () => { + const provider = makeLocalEmbeddingGemmaProvider(512); + expect(provider.embedQuery).toBeTypeOf("function"); + const result = await provider.embedQuery?.("what is the capital"); + expect(result?.ok).toBe(true); + if (!result?.ok) return; + expect(result.value.length).toBe(512); + expect(pipelineCalls[0]?.texts[0]).toBe("task: search result | query: what is the capital"); + // Truncated + renormalized: unit length. + let norm = 0; + for (const x of result.value) norm += x * x; + expect(Math.sqrt(norm)).toBeCloseTo(1, 4); + }); + + it("embedQuery() at dim=768 (no truncation) returns the untruncated vector", async () => { + const provider = makeLocalEmbeddingGemmaProvider(768); + const result = await provider.embedQuery?.("q"); + expect(result?.ok).toBe(true); + if (!result?.ok) return; + expect(result.value.length).toBe(768); + }); + + it("isLoaded() reflects the memoised extractor state", async () => { + const provider = makeLocalEmbeddingGemmaProvider(512); + expect(isLocalEmbeddingGemmaLoaded()).toBe(false); + expect(provider.isLoaded?.()).toBe(false); + await provider.warm(); + expect(isLocalEmbeddingGemmaLoaded()).toBe(true); + expect(provider.isLoaded?.()).toBe(true); + }); + + it("a load failure returns Result.err and resets the memo so a retry can succeed", async () => { + const provider = makeLocalEmbeddingGemmaProvider(512); + failNext = true; + const result = await provider.embed(["x"]); + expect(result.ok).toBe(false); + expect(provider.isLoaded?.()).toBe(false); + + // Retry succeeds — the memo was reset, not poisoned for the process. + const retry = await provider.embed(["x"]); + expect(retry.ok).toBe(true); + expect(provider.isLoaded?.()).toBe(true); + }); +}); diff --git a/test/search/providers/local-qwen3.test.ts b/test/search/providers/local-qwen3.test.ts new file mode 100644 index 00000000..07bcf314 --- /dev/null +++ b/test/search/providers/local-qwen3.test.ts @@ -0,0 +1,123 @@ +// local-qwen3 provider coverage (spec 2026-07-26-embedding-refresh- +// quantization, Phase 1c). Mirrors local-embeddinggemma.test.ts's shape but +// exercises the LAST-TOKEN pooling path (opts.pooling: "none", raw per-token +// output) instead of mean pooling. Mocks @huggingface/transformers entirely +// — see local-embeddinggemma.test.ts's header for why. A real-model smoke +// test exists separately, gated behind DAFTARI_QWEN3_SMOKE (not run here). + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +// The real Qwen3-Embedding-0.6B outputs 1024d; this provider exposes at +// most 768d (local-qwen3.ts's EXPOSED_NATIVE_DIM) — the mock's raw output is +// deliberately WIDER than that so the tests exercise the capToNativeDim +// truncation path, not just the identity case. +const FAKE_RAW_DIM = 1024; +const SEQ_LEN = 5; + +// Builds a fake [1, SEQ_LEN, FAKE_RAW_DIM] raw per-token tensor (flattened +// row-major) where token i's row is filled with the constant (i + 1) so the +// LAST row is trivially distinguishable from any earlier row. +function fakeRawTokens(): Float32Array { + const data = new Float32Array(SEQ_LEN * FAKE_RAW_DIM); + for (let tok = 0; tok < SEQ_LEN; tok++) { + for (let d = 0; d < FAKE_RAW_DIM; d++) { + data[tok * FAKE_RAW_DIM + d] = tok + 1; + } + } + return data; +} + +const pipelineCalls: Array<{ texts: string[]; opts: unknown }> = []; +let failNext = false; + +vi.mock("@huggingface/transformers", () => ({ + pipeline: vi.fn(async () => { + if (failNext) { + failNext = false; + throw new Error("simulated model load failure"); + } + return async (texts: string[], opts: { pooling: "mean" | "none"; normalize: boolean }) => { + pipelineCalls.push({ texts, opts }); + return { data: fakeRawTokens(), dims: [1, SEQ_LEN, FAKE_RAW_DIM] }; + }; + }), +})); + +const { makeLocalQwen3Provider, isLocalQwen3Loaded, resetLocalQwen3ForTests } = await import( + "../../../src/search/providers/local-qwen3.js" +); + +describe("local-qwen3 provider", () => { + beforeEach(() => { + pipelineCalls.length = 0; + failNext = false; + resetLocalQwen3ForTests(); + }); + afterEach(() => { + resetLocalQwen3ForTests(); + }); + + it("exposes an id carrying #p1 and no dim, and caps nativeDim at the exposed ceiling", () => { + const provider = makeLocalQwen3Provider(512); + expect(provider.id).toBe("local-qwen3-0.6b#p1"); + expect(provider.id).not.toContain("512"); + expect(provider.dim).toBe(512); + // Exposed ceiling (768), NOT the real model's native 1024 — the spec + // text: "Qwen3's 1024d deliberately not offered yet". + expect(provider.nativeDim).toBe(768); + }); + + it("rejects an unsupported dim", () => { + expect(() => makeLocalQwen3Provider(1024)).toThrow(/unsupported dim/); + }); + + it("embed() last-token-pools the LAST row, caps to the exposed native dim, and normalizes", async () => { + const provider = makeLocalQwen3Provider(512); + const result = await provider.embed(["some document text"]); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toHaveLength(1); + const vec = result.value[0]; + if (!vec) throw new Error("expected a vector"); + expect(vec.length).toBe(768); // capped, not the raw 1024 + // The last raw token row was constant (SEQ_LEN), so after L2-normalize + // every component of the pooled (pre-truncation) vector is equal — + // truncating a subset of equal components and renormalizing keeps them + // equal, positive, and unit-norm. + let norm = 0; + for (const x of vec) norm += x * x; + expect(Math.sqrt(norm)).toBeCloseTo(1, 4); + expect(vec.every((x) => x > 0)).toBe(true); + // Document side is unprefixed (bare document, per the spec hypothesis). + expect(pipelineCalls[0]?.texts[0]).toBe("some document text"); + expect(pipelineCalls[0]?.opts).toMatchObject({ pooling: "none", normalize: false }); + }); + + it("embedQuery() applies the instruction prefix and returns configured dim", async () => { + const provider = makeLocalQwen3Provider(512); + const result = await provider.embedQuery?.("capital of France"); + expect(result?.ok).toBe(true); + if (!result?.ok) return; + expect(result.value.length).toBe(512); + expect(pipelineCalls[0]?.texts[0]).toBe( + "Instruct: Given a search query, retrieve relevant passages | Query: capital of France", + ); + }); + + it("isLoaded() reflects the memoised extractor state", async () => { + const provider = makeLocalQwen3Provider(512); + expect(isLocalQwen3Loaded()).toBe(false); + await provider.warm(); + expect(isLocalQwen3Loaded()).toBe(true); + }); + + it("a load failure returns Result.err and resets the memo so a retry can succeed", async () => { + const provider = makeLocalQwen3Provider(512); + failNext = true; + const result = await provider.embed(["x"]); + expect(result.ok).toBe(false); + expect(provider.isLoaded?.()).toBe(false); + const retry = await provider.embed(["x"]); + expect(retry.ok).toBe(true); + }); +}); diff --git a/test/search/reindex-resume.test.ts b/test/search/reindex-resume.test.ts index 4511e02f..c549fc0f 100644 --- a/test/search/reindex-resume.test.ts +++ b/test/search/reindex-resume.test.ts @@ -38,7 +38,7 @@ function fakeProvider(dieAfterCalls = Number.POSITIVE_INFINITY): { } function embeddingRowCount(vault: string): number { - const db = openIndexDb(vault, DIM); + const db = openIndexDb(vault, DIM, "float32"); if (!db.ok) throw db.error; try { const row = db.value diff --git a/test/search/reindex.test.ts b/test/search/reindex.test.ts index a75890b0..c007b597 100644 --- a/test/search/reindex.test.ts +++ b/test/search/reindex.test.ts @@ -46,7 +46,7 @@ describe("reindexVault", () => { expect(result.value.skipped).toEqual([]); expect(result.value.vectorEnabled).toBe(true); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -93,7 +93,7 @@ describe("reindexVault", () => { // Still indexed and searchable — advisory, not rejected (matches the // _drafts/incomplete-note.md fixture's documented intent). - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -195,7 +195,7 @@ describe("reindexVault", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -261,7 +261,7 @@ describe("reindexVault", () => { expect(first.ok).toBe(true); if (!first.ok) return; const cacheSizeBefore = (() => { - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; try { return embeddingCount(opened.value); @@ -282,7 +282,7 @@ describe("reindexVault", () => { // Cache size unchanged — every old hash is still referenced (by the // renamed file) so no orphans were reaped. const cacheSizeAfter = (() => { - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; try { return embeddingCount(opened.value); @@ -293,39 +293,70 @@ describe("reindexVault", () => { expect(cacheSizeAfter).toBe(cacheSizeBefore); }, 120_000); - it("moved paragraph re-embeds zero: identical chunk text in a different file is a cache hit", async () => { + // Contextual chunking (spec 2026-07-26-contextual-chunking-reranker-design.md + // Decision 2 / plan C7): the chunk hash covers the breadcrumb context + // (collection > title > headings > tags) AS WELL AS the body text — the + // context is part of the chunk's retrieval identity now, not a + // pre-contextual-chunking cache key of text alone. So identical body text + // is a cache hit ONLY when the surrounding metadata (title/collection/tags) + // also matches; a differently-titled document with the same paragraph is + // an intentional cache MISS (a stale-vector hit would silently serve + // pre-edit semantics, which the spec judges worse than the recompute). + it("identical body text under IDENTICAL title/collection/tags shares one embedding row (same-pass dedupe)", async () => { const first = await reindexVault(vault); expect(first.ok).toBe(true); if (!first.ok) return; - // Grab an actual chunk's text from the index and use it verbatim as - // the body of a brand-new file. Because chunkText is deterministic - // and the body equals exactly one chunk's worth of text, the new - // file's chunker round-trips to the same content_hash — which the - // cache already holds. - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); - if (!opened.ok) throw opened.error; - const chunkText = (() => { - try { - const all = getAllChunks(opened.value, EMBEDDING_MODEL); - return all[0]?.text ?? ""; - } finally { - opened.value.close(); - } - })(); - expect(chunkText.length).toBeGreaterThan(0); + const uniqueParagraph = + "orthogonal quokka fandango cache-identity probe text that appears nowhere else."; + const frontmatter = + "---\ntitle: Twin Doc\ndomain: positioning\nstatus: draft\nconfidence: low\n" + + "updated: 2026-05-20\ntags: [probe]\n---\n\n"; + await writeFile( + join(vault, "competitive-intel/twin-a.md"), + `${frontmatter}${uniqueParagraph}\n`, + ); + await writeFile( + join(vault, "competitive-intel/twin-b.md"), + `${frontmatter}${uniqueParagraph}\n`, + ); + const second = await reindexVault(vault); + expect(second.ok).toBe(true); + if (!second.ok) return; + // Both new files hash identically (same context AND same text) — the + // in-pass miss-dedupe (missTextByHash) embeds the shared hash exactly + // once, not twice. + expect(second.value.embeddedCount).toBe(1); + expect(second.value.documentCount).toBe(first.value.documentCount + 2); + }, 120_000); + + it("identical body text under DIFFERENT titles produces two embedding rows, not a cache hit (C7)", async () => { + const first = await reindexVault(vault); + expect(first.ok).toBe(true); + if (!first.ok) return; + + const uniqueParagraph = + "cerulean marmoset syzygy cache-identity probe text that appears nowhere else."; + const frontmatterFor = (title: string): string => + `---\ntitle: ${title}\ndomain: positioning\nstatus: draft\nconfidence: low\n` + + "updated: 2026-05-20\ntags: [probe]\n---\n\n"; + await writeFile( + join(vault, "competitive-intel/distinct-a.md"), + `${frontmatterFor("Distinct Title A")}${uniqueParagraph}\n`, + ); await writeFile( - join(vault, "competitive-intel/clone-paragraph.md"), - `---\ntitle: Clone\ndomain: positioning\nstatus: draft\nconfidence: low\nupdated: 2026-05-20\ntags: []\n---\n\n${chunkText}\n`, + join(vault, "competitive-intel/distinct-b.md"), + `${frontmatterFor("Distinct Title B")}${uniqueParagraph}\n`, ); const second = await reindexVault(vault); expect(second.ok).toBe(true); if (!second.ok) return; - // No new embedding work — the cloned chunk hashes to a cached row. - expect(second.value.embeddedCount).toBe(0); - expect(second.value.documentCount).toBe(first.value.documentCount + 1); + // Same body text, different titles => different breadcrumb context => + // different content_hash => NOT deduped. Two embedding rows, not one. + expect(second.value.embeddedCount).toBe(2); + expect(second.value.documentCount).toBe(first.value.documentCount + 2); }, 120_000); it("vault_gc reaps embeddings whose chunks no longer reference them", async () => { @@ -333,7 +364,7 @@ describe("reindexVault", () => { expect(first.ok).toBe(true); if (!first.ok) return; const cacheBefore = (() => { - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; try { return embeddingCount(opened.value); @@ -359,7 +390,7 @@ describe("reindexVault", () => { // After the reindex, every surviving embeddings row must be referenced // by at least one chunk row. - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -460,6 +491,12 @@ describe("reindexVault", () => { }; setProviderForTests(altProvider); + // Vec-coherence check (C1): a provider switch on an otherwise + // unchanged vault must be detected as stale BEFORE the reindex that + // fixes it — this is what makes "config change + background reindex" + // an actually-triggered migration rather than a silent no-op. + expect(await isIndexFresh(vault)).toBe(false); + const second = await reindexVault(vault); expect(second.ok).toBe(true); if (!second.ok) return; @@ -472,7 +509,7 @@ describe("reindexVault", () => { // Both providers' rows coexist in the cache (the composite PK lets // them) — a switch-back to the original id would be all cache hits. - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -489,7 +526,7 @@ describe("reindexVault", () => { } // The current provider's id is what gets written to meta. - const dbMeta = openIndexDb(vault, LOCAL_MINILM_DIM); + const dbMeta = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!dbMeta.ok) throw dbMeta.error; try { expect(getMeta(dbMeta.value, "embedding_model")).toBe("alt-minilm"); @@ -506,7 +543,7 @@ describe("reindexVault", () => { const result = await reindexVault(vault); expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -523,6 +560,131 @@ describe("reindexVault", () => { }, 120_000); }); + // 2026-07-26 embedding-refresh-quantization spec, Phase 3d / dispositions + // C1 (freshness coherence) and C9 (native-dim cache, dim-free id). A fake + // Matryoshka-style provider — same cache id, varying `dim`/`nativeDim` — + // keeps these tests fast and network-free instead of paying local-minilm's + // real embed cost like the "provider switch" tests above. + describe("dim / quantize coherence (C1, C9)", () => { + const FAKE_ID = "fake-truncatable"; + const NATIVE_DIM = 8; + + function fakeVectorFor(text: string): Float32Array { + const v = new Float32Array(NATIVE_DIM); + for (let i = 0; i < NATIVE_DIM; i++) v[i] = ((text.charCodeAt(i % text.length) ?? 1) % 7) + 1; + let norm = 0; + for (const x of v) norm += x * x; + const inv = 1 / Math.sqrt(norm); + for (let i = 0; i < v.length; i++) v[i] = (v[i] as number) * inv; + return v; + } + + function fakeTruncatableProvider(dim: number, counter: { calls: number }): EmbeddingProvider { + return { + id: FAKE_ID, + dim, + nativeDim: NATIVE_DIM, + async warm(): Promise> { + return ok(undefined); + }, + async embed(texts) { + counter.calls += texts.length; + return ok(texts.map((t) => fakeVectorFor(t))); + }, + }; + } + + afterEach(() => { + resetProviderForTests(); + }); + + it("a dim flip on the same provider id is all cache hits — zero embed calls, mirror rebuilt truncated", async () => { + const counter = { calls: 0 }; + setProviderForTests(fakeTruncatableProvider(NATIVE_DIM, counter), "float32"); + const first = await reindexVault(vault); + expect(first.ok).toBe(true); + if (!first.ok) return; + expect(first.value.embeddedCount).toBeGreaterThan(0); + const embedsAtNativeDim = counter.calls; + expect(embedsAtNativeDim).toBeGreaterThan(0); + + // Flip dim 8 -> 4 on the SAME provider id. isIndexFresh must catch + // this (embeddings_vec gets drop-recreated at the new dim, emptying + // it, under an unchanged model id — the case check (a) alone misses). + counter.calls = 0; + setProviderForTests(fakeTruncatableProvider(4, counter), "float32"); + expect(await isIndexFresh(vault)).toBe(false); + + const second = await reindexVault(vault); + expect(second.ok).toBe(true); + if (!second.ok) return; + expect(second.value.embeddedCount).toBe(0); // zero embed() calls — pure cache hits + expect(second.value.cacheHits).toBeGreaterThan(0); + expect(counter.calls).toBe(0); // the provider's own embed() was never invoked + + const opened = openIndexDb(vault, 4, "float32"); + if (!opened.ok) throw opened.error; + try { + expect(getMeta(opened.value, "embeddings_vec_dim")).toBe("4"); + const row = opened.value.prepare("SELECT embedding FROM embeddings_vec LIMIT 1").get() as + | { embedding: Buffer } + | undefined; + expect(row).toBeDefined(); + expect(row?.embedding.byteLength).toBe(4 * 4); // 4 float32 components, truncated + const vecCount = opened.value.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { + n: number; + }; + expect(vecCount.n).toBeGreaterThan(0); + } finally { + opened.value.close(); + } + + // Switch back to dim 8 — also all cache hits. + counter.calls = 0; + setProviderForTests(fakeTruncatableProvider(NATIVE_DIM, counter), "float32"); + expect(await isIndexFresh(vault)).toBe(false); + const third = await reindexVault(vault); + expect(third.ok).toBe(true); + if (!third.ok) return; + expect(third.value.embeddedCount).toBe(0); + expect(counter.calls).toBe(0); + }, 60_000); + + it("a quantize flip alone (provider/dim unchanged) is caught by isIndexFresh and repopulates via cache hits", async () => { + const counter = { calls: 0 }; + setProviderForTests(fakeTruncatableProvider(NATIVE_DIM, counter), "float32"); + const first = await reindexVault(vault); + expect(first.ok).toBe(true); + expect(await isIndexFresh(vault)).toBe(true); + + // Same provider object (same id, same dim) — only the quantize STATE + // flips. This is exactly the case check (a) (embedding_model meta) + // cannot see: the model id is unchanged. + counter.calls = 0; + setProviderForTests(fakeTruncatableProvider(NATIVE_DIM, counter), "int8"); + expect(await isIndexFresh(vault)).toBe(false); + + const second = await reindexVault(vault); + expect(second.ok).toBe(true); + if (!second.ok) return; + expect(second.value.embeddedCount).toBe(0); // all cache hits + expect(counter.calls).toBe(0); + + const opened = openIndexDb(vault, NATIVE_DIM, "int8"); + if (!opened.ok) throw opened.error; + try { + expect(getMeta(opened.value, "embeddings_vec_kind")).toBe("int8"); + const vecCount = opened.value.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { + n: number; + }; + expect(vecCount.n).toBeGreaterThan(0); // KNN-non-empty after the repopulating reindex + } finally { + opened.value.close(); + } + expect(await isIndexFresh(vault)).toBe(true); // now coherent again + }, 60_000); + }); + describe("valid-time columns", () => { it("carries authored validity endpoints from frontmatter into the index", async () => { await writeFile( @@ -538,7 +700,7 @@ describe("reindexVault", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -555,7 +717,7 @@ describe("reindexVault", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { diff --git a/test/search/rerank-provider.test.ts b/test/search/rerank-provider.test.ts new file mode 100644 index 00000000..47bc1cc0 --- /dev/null +++ b/test/search/rerank-provider.test.ts @@ -0,0 +1,103 @@ +// RerankProvider selection (spec 2026-07-26-contextual-chunking-reranker- +// design.md Decision 5). Mirrors the embedding-provider selection tests in +// spirit: memoisation, "none" -> null, isReady semantics, test seams — all +// exercised with a fake provider so this file never touches the real +// local-bge-m3 model or the network. + +import { afterEach, describe, expect, it } from "vitest"; +import { ok } from "../../src/frontmatter/types.js"; +import { + getRerankProvider, + type RerankProvider, + resetRerankProviderForTests, + setRerankProvider, + setRerankProviderForTests, + warmRerankModel, +} from "../../src/search/rerank-provider.js"; + +function fakeProvider(overrides: Partial = {}): RerankProvider { + return { + id: "fake-rerank", + isReady: () => true, + warm: async () => ok(undefined), + rerank: async (_query, passages) => ok(passages.map(() => 1)), + ...overrides, + }; +} + +describe("rerank provider selection", () => { + afterEach(() => { + resetRerankProviderForTests(); + }); + + it("defaults to no provider ('none' maps to null)", () => { + expect(getRerankProvider()).toBeNull(); + }); + + it("setRerankProvider('none') keeps it null", () => { + setRerankProvider("none"); + expect(getRerankProvider()).toBeNull(); + }); + + it("setRerankProvider('local-bge-m3') installs the real provider object", () => { + setRerankProvider("local-bge-m3"); + const provider = getRerankProvider(); + expect(provider).not.toBeNull(); + expect(provider?.id).toBe("local-bge-m3"); + }); + + it("setRerankProvider is idempotent for a repeated id", () => { + setRerankProvider("local-bge-m3"); + const first = getRerankProvider(); + setRerankProvider("local-bge-m3"); + const second = getRerankProvider(); + expect(second).toBe(first); // same object reference — no reinstantiation + }); + + it("setRerankProviderForTests installs an arbitrary provider, bypassing config selection", () => { + const fake = fakeProvider(); + setRerankProviderForTests(fake); + expect(getRerankProvider()).toBe(fake); + }); + + it("setRerankProviderForTests(null) simulates 'none' without touching setRerankProvider's memoisation", () => { + setRerankProviderForTests(fakeProvider()); + setRerankProviderForTests(null); + expect(getRerankProvider()).toBeNull(); + }); + + it("resetRerankProviderForTests reverts to no provider", () => { + setRerankProviderForTests(fakeProvider()); + resetRerankProviderForTests(); + expect(getRerankProvider()).toBeNull(); + }); + + it("isReady() reflects the installed fake provider's own semantics", () => { + const notReady = fakeProvider({ isReady: () => false }); + setRerankProviderForTests(notReady); + expect(getRerankProvider()?.isReady()).toBe(false); + + setRerankProviderForTests(fakeProvider({ isReady: () => true })); + expect(getRerankProvider()?.isReady()).toBe(true); + }); + + it("warmRerankModel() is a no-op ok() when no provider is configured", async () => { + const result = await warmRerankModel(); + expect(result.ok).toBe(true); + }); + + it("warmRerankModel() delegates to the active provider's warm()", async () => { + let warmed = false; + setRerankProviderForTests( + fakeProvider({ + warm: async () => { + warmed = true; + return ok(undefined); + }, + }), + ); + const result = await warmRerankModel(); + expect(result.ok).toBe(true); + expect(warmed).toBe(true); + }); +}); diff --git a/test/search/router.test.ts b/test/search/router.test.ts new file mode 100644 index 00000000..e798e115 --- /dev/null +++ b/test/search/router.test.ts @@ -0,0 +1,157 @@ +// Query router tests (spec 2026-07-26 fusion overhaul, Decision 2). + +import { describe, expect, it } from "vitest"; +import { DEFAULT_WEIGHTS } from "../../src/search/hybrid.js"; +import { LOCAL_MINILM_DIM } from "../../src/search/providers/local-minilm.js"; +import { reindexVault } from "../../src/search/reindex.js"; +import { + classifyQuery, + makeDfLookup, + type RouteClass, + routeWeights, +} from "../../src/search/router.js"; +import { type IndexDb, openIndexDb } from "../../src/storage/index-db.js"; +import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; + +describe("classifyQuery — fixture table, one query per signal", () => { + const cases: [query: string, expected: RouteClass, signal: string][] = [ + [`"exact phrase"`, "extreme-lexical", "quoted-phrase"], + ["src/search/hybrid.ts", "extreme-lexical", "path-like"], + ["config.yaml", "extreme-lexical", "path-like"], + ["processTensionDocket", "lexical", "camel-case"], + ["tension_scan", "lexical", "snake-case"], + ["PR 303", "lexical", "digit-heavy"], + ["2026-07-26", "lexical", "digit-heavy"], + ["how do write locks expire", "balanced", ""], + ]; + + for (const [query, expected, signal] of cases) { + it(`classifies "${query}" as ${expected}`, () => { + const result = classifyQuery(query); + expect(result.class).toBe(expected); + if (signal) expect(result.signals).toContain(signal); + else expect(result.signals).toEqual([]); + }); + } +}); + +describe("classifyQuery — rare-term signal (injected stub df)", () => { + const RICH_DOC_COUNT = 100; + + it("fires lexical when df === 1", () => { + const result = classifyQuery("zephyr", { + df: () => 1, + docCount: RICH_DOC_COUNT, + }); + expect(result.class).toBe("lexical"); + expect(result.signals).toContain("rare-term"); + }); + + it("fires lexical when df === DF_RARE_FLOOR (2)", () => { + const result = classifyQuery("zephyr", { + df: () => 2, + docCount: RICH_DOC_COUNT, + }); + expect(result.class).toBe("lexical"); + expect(result.signals).toContain("rare-term"); + }); + + it("does not fire when df === 3 (above the floor)", () => { + const result = classifyQuery("zephyr", { + df: () => 3, + docCount: RICH_DOC_COUNT, + }); + expect(result.class).toBe("balanced"); + expect(result.signals).not.toContain("rare-term"); + }); + + it("never fires when df === 0 (absent from corpus)", () => { + const result = classifyQuery("zephyr", { + df: () => 0, + docCount: RICH_DOC_COUNT, + }); + expect(result.class).toBe("balanced"); + expect(result.signals).not.toContain("rare-term"); + }); + + it("never fires when the vault holds fewer than MIN_DOCS_FOR_RARE (100) documents", () => { + const result = classifyQuery("zephyr", { + df: () => 1, + docCount: 99, + }); + expect(result.class).toBe("balanced"); + expect(result.signals).not.toContain("rare-term"); + }); + + it("never fires when docCount is absent, even with a df function", () => { + const result = classifyQuery("zephyr", { df: () => 1 }); + expect(result.class).toBe("balanced"); + expect(result.signals).not.toContain("rare-term"); + }); + + it("never fires when df is absent, even with a rich docCount", () => { + const result = classifyQuery("zephyr", { docCount: RICH_DOC_COUNT }); + expect(result.class).toBe("balanced"); + expect(result.signals).not.toContain("rare-term"); + }); +}); + +describe("classifyQuery — precedence and signals array", () => { + it("an extreme signal wins even when a lexical signal also fires", () => { + const result = classifyQuery(`"exact phrase" tension_scan`); + expect(result.class).toBe("extreme-lexical"); + // Both signals are reported, even though only the extreme one decided the class. + expect(result.signals).toContain("quoted-phrase"); + expect(result.signals).toContain("snake-case"); + }); + + it("is deterministic across repeated calls", () => { + const first = classifyQuery("src/search/hybrid.ts and tension_scan"); + const second = classifyQuery("src/search/hybrid.ts and tension_scan"); + expect(second).toEqual(first); + }); +}); + +describe("routeWeights", () => { + it("maps extreme-lexical to pure lexical weights", () => { + expect(routeWeights("extreme-lexical")).toEqual({ bm25: 1, vector: 0 }); + }); + + it("maps lexical to a lexical-leaning split", () => { + expect(routeWeights("lexical")).toEqual({ bm25: 0.8, vector: 0.2 }); + }); + + it("maps balanced to the library default weights", () => { + expect(routeWeights("balanced")).toEqual(DEFAULT_WEIGHTS); + }); +}); + +describe("makeDfLookup — real indexed handle (stem-aware df)", () => { + let vault: string; + let db: IndexDb; + + it("counts stemmed postings and returns 0 for an absent token", async () => { + vault = makeTempVault(); + try { + const reindexed = await reindexVault(vault); + expect(reindexed.ok).toBe(true); + if (!reindexed.ok) return; + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); + expect(opened.ok).toBe(true); + if (!opened.ok) return; + db = opened.value; + try { + const df = makeDfLookup(db); + // "pricing" appears in the sample vault (see hybrid.test.ts's + // CREDIT_DOC fixture); its stemmed form should count "pricing" and + // any morphological variant FTS5's porter tokenizer folds to it. + expect(df("pricing")).toBeGreaterThanOrEqual(1); + expect(df("zzzqx")).toBe(0); + } finally { + db.close(); + } + } finally { + cleanupVault(vault); + } + }, 60_000); +}); diff --git a/test/search/sql-native.test.ts b/test/search/sql-native.test.ts index 9a28d053..1b74dea3 100644 --- a/test/search/sql-native.test.ts +++ b/test/search/sql-native.test.ts @@ -43,7 +43,7 @@ describe("reindex populates both virtual tables", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, EMBEDDING_DIM); + const opened = openIndexDb(vault, EMBEDDING_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const db = opened.value; @@ -99,7 +99,7 @@ describe("reindex populates both virtual tables", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, EMBEDDING_DIM); + const opened = openIndexDb(vault, EMBEDDING_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -122,7 +122,7 @@ describe("reindex populates both virtual tables", () => { expect(result.ok).toBe(true); if (!result.ok) return; - const opened = openIndexDb(vault, EMBEDDING_DIM); + const opened = openIndexDb(vault, EMBEDDING_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -155,7 +155,7 @@ describe("provider switch rebuilds embeddings_vec at the new dim", () => { if (!first.ok) return; { - const opened = openIndexDb(vault, 384); + const opened = openIndexDb(vault, 384, "float32"); if (!opened.ok) throw opened.error; try { expect(getMeta(opened.value, "embeddings_vec_dim")).toBe("384"); @@ -179,7 +179,7 @@ describe("provider switch rebuilds embeddings_vec at the new dim", () => { }; setProviderForTests(fakeProvider); - const opened = openIndexDb(vault, 1024); + const opened = openIndexDb(vault, 1024, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; try { @@ -214,7 +214,7 @@ describe("provider switch rebuilds embeddings_vec at the new dim", () => { let vecBefore = 0; { - const opened = openIndexDb(vault, 384); + const opened = openIndexDb(vault, 384, "float32"); if (!opened.ok) throw opened.error; try { vecBefore = ( @@ -237,7 +237,7 @@ describe("provider switch rebuilds embeddings_vec at the new dim", () => { }; setProviderForTests(altProvider); - const opened = openIndexDb(vault, altProvider.dim); + const opened = openIndexDb(vault, altProvider.dim, "float32"); if (!opened.ok) throw opened.error; try { const vecAfter = ( diff --git a/test/search/valid-at-source.test.ts b/test/search/valid-at-source.test.ts index 60577168..2224de0f 100644 --- a/test/search/valid-at-source.test.ts +++ b/test/search/valid-at-source.test.ts @@ -60,7 +60,7 @@ describe("resolveValidAtSource", () => { beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); diff --git a/test/search/vector.test.ts b/test/search/vector.test.ts index 7b8129f6..aea8e1f4 100644 --- a/test/search/vector.test.ts +++ b/test/search/vector.test.ts @@ -1,12 +1,23 @@ -import { describe, expect, it } from "vitest"; +import { afterEach, describe, expect, it } from "vitest"; +import { ok } from "../../src/frontmatter/types.js"; +import type { EmbeddingProvider } from "../../src/search/embedding-provider.js"; import { - chunkText, + type ChunkInput, + chunkDocument, cosineSimilarity, EMBED_BATCH_SIZE, EMBEDDING_DIM, embed, + embeddingInput, + embedQuery, + getProvider, + getQuantize, meanEmbedding, + resetProviderForTests, + setProviderForTests, + toIndexDim, } from "../../src/search/vector.js"; +import { sha256Hex } from "../../src/utils/hash.js"; describe("cosineSimilarity", () => { it("is 1 for identical vectors", () => { @@ -28,20 +39,200 @@ describe("cosineSimilarity", () => { }); }); -describe("chunkText", () => { - it("returns a single chunk for short text", () => { - expect(chunkText("a short paragraph")).toEqual(["a short paragraph"]); +// Default input for chunkDocument tests; individual tests override fields. +function baseInput(overrides: Partial = {}): ChunkInput { + return { + title: "Doc Title", + collection: "notes", + tags: [], + body: "", + ...overrides, + }; +} + +describe("chunkDocument", () => { + it("returns a single chunk for short text with no headings", () => { + const chunks = chunkDocument(baseInput({ body: "a short paragraph" })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.text).toBe("a short paragraph"); }); it("splits long text into multiple chunks under the size cap", () => { const para = "word ".repeat(400); // ~2000 chars in one paragraph - const chunks = chunkText(para); + const chunks = chunkDocument(baseInput({ body: para })); expect(chunks.length).toBeGreaterThan(1); - expect(chunks.every((c) => c.length <= 800)).toBe(true); + expect(chunks.every((c) => c.text.length <= 800)).toBe(true); + }); + + it("packs separate paragraphs together when they fit, within a section", () => { + const chunks = chunkDocument(baseInput({ body: "first para\n\nsecond para" })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.text).toBe("first para\n\nsecond para"); + }); + + it("always returns >=1 chunk for an empty or whitespace-only body", () => { + expect(chunkDocument(baseInput({ body: "" }))).toHaveLength(1); + expect(chunkDocument(baseInput({ body: " \n\n " }))).toHaveLength(1); + const chunks = chunkDocument(baseInput({ body: "" })); + expect(chunks[0]?.text).toBe(""); + expect(chunks[0]?.context).toBe("notes › Doc Title"); + }); + + it("splits at ATX headings — a heading boundary always starts a new chunk", () => { + const body = "# H1\n\nIntro text.\n\n## H2\n\nSection two text."; + const chunks = chunkDocument(baseInput({ body })); + // "# H1" and "Intro text." pack together (same section, both short); + // "## H2" and "Section two text." pack together in the NEXT section. + expect(chunks).toHaveLength(2); + expect(chunks[0]?.text).toBe("# H1\n\nIntro text."); + expect(chunks[1]?.text).toBe("## H2\n\nSection two text."); + }); + + it("never packs across a section boundary, even when both sections are tiny", () => { + const body = "## A\n\nx\n\n## B\n\ny"; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks).toHaveLength(2); + expect(chunks[0]?.text).toBe("## A\n\nx"); + expect(chunks[1]?.text).toBe("## B\n\ny"); + }); + + it("preamble before the first heading gets its own heading-free chunk", () => { + const body = "Some intro paragraph.\n\n## Section\n\nBody text."; + const chunks = chunkDocument(baseInput({ title: "T", collection: "c", body })); + expect(chunks).toHaveLength(2); + expect(chunks[0]?.text).toBe("Some intro paragraph."); + expect(chunks[0]?.context).toBe("c › T"); // heading-path-free + expect(chunks[1]?.context).toBe("c › T › Section"); + }); + + it("a document that starts with a heading produces no empty preamble chunk", () => { + const body = "## Section\n\nBody text."; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.text).toBe("## Section\n\nBody text."); + }); + + it("tracks the open heading stack across levels: a same/shallower heading replaces deeper ones", () => { + const body = "# H1\n\n## H2a\n\ntext a\n\n### H3\n\ntext b\n\n## H2b\n\ntext c"; + const chunks = chunkDocument(baseInput({ title: "T", collection: "c", body })); + const contexts = chunks.map((c) => c.context); + expect(contexts).toContain("c › T › H1"); + expect(contexts).toContain("c › T › H1 › H2a"); + expect(contexts).toContain("c › T › H1 › H2a › H3"); + // H2b closes the open H3 (and H2a) — its path is H1 › H2b, not + // H1 › H2a › H3 › H2b. + expect(contexts).toContain("c › T › H1 › H2b"); + }); + + it("a heading line inside a fenced code block is not a heading", () => { + const body = "intro\n\n```\n# not a heading\n```\n\nafter fence"; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.text).toContain("# not a heading"); + expect(chunks[0]?.context).toBe("notes › Doc Title"); // no heading path + }); + + it("a heading line inside a ~~~ fence is not a heading", () => { + const body = "intro\n\n~~~\n## also not a heading\n~~~\n\nmore"; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.context).toBe("notes › Doc Title"); + }); + + it("H5/H6 and setext headings degrade to plain text (not section boundaries)", () => { + const body = "intro\n\n##### H5 not real\n\nSetext Title\n===\n\nmore text"; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks).toHaveLength(1); + expect(chunks[0]?.context).toBe("notes › Doc Title"); + expect(chunks[0]?.text).toContain("##### H5 not real"); + }); + + it("oversized single paragraph hard-splits even within a section", () => { + const para = "word ".repeat(400); + const body = `## Big Section\n\n${para}`; + const chunks = chunkDocument(baseInput({ body })); + expect(chunks.length).toBeGreaterThan(1); + expect(chunks.every((c) => c.context === "notes › Doc Title › Big Section")).toBe(true); + }); + + describe("breadcrumb context", () => { + it("shape: {collection} › {title} › {headings} · tags: a, b, c", () => { + const chunks = chunkDocument( + baseInput({ + title: "My Doc", + collection: "pricing", + tags: ["zeta", "alpha"], + body: "# Heading One\n\nbody", + }), + ); + expect(chunks[0]?.context).toBe("pricing › My Doc › Heading One · tags: alpha, zeta"); + }); + + it("omits the tag suffix entirely for an untagged doc", () => { + const chunks = chunkDocument(baseInput({ tags: [], body: "text" })); + expect(chunks[0]?.context).not.toContain("tags:"); + }); + + it("sorts tags lexicographically before capping at 5", () => { + const chunks = chunkDocument( + baseInput({ tags: ["z", "y", "x", "w", "v", "u"], body: "text" }), + ); + expect(chunks[0]?.context).toContain("tags: u, v, w, x, y"); + expect(chunks[0]?.context).not.toContain(", z"); + }); + + it("tag reorder produces an identical breadcrumb (and therefore an identical hash)", () => { + const a = chunkDocument(baseInput({ tags: ["b", "a", "c"], body: "text" }))[0]; + const b = chunkDocument(baseInput({ tags: ["c", "b", "a"], body: "text" }))[0]; + if (!a || !b) throw new Error("expected a chunk"); + expect(a.context).toBe(b.context); + expect(sha256Hex(embeddingInput(a))).toBe(sha256Hex(embeddingInput(b))); + }); + + it("caps the whole line at 160 chars, collapsing middle headings first", () => { + const body = + "# " + + "A".repeat(50) + + "\n\n## " + + "B".repeat(50) + + "\n\n### " + + "C".repeat(50) + + "\n\ntext"; + const chunks = chunkDocument(baseInput({ title: "Title", collection: "col", body })); + const ctx = chunks[chunks.length - 1]?.context ?? ""; + expect(ctx.length).toBeLessThanOrEqual(160); + // Innermost heading (C...) survives; the outer ones collapse to "…". + expect(ctx).toContain("…"); + expect(ctx).toContain("col"); + expect(ctx).toContain("Title"); + }); + + it("collection and title always survive as components even under extreme truncation", () => { + const chunks = chunkDocument( + baseInput({ + title: "T", + collection: "c", + tags: Array.from({ length: 5 }, (_, i) => `tag-${i}-${"x".repeat(30)}`), + body: `# ${"H".repeat(200)}\n\ntext`, + }), + ); + const ctx = chunks[chunks.length - 1]?.context ?? ""; + expect(ctx.length).toBeLessThanOrEqual(160); + expect(ctx.startsWith("c › T")).toBe(true); + }); }); - it("packs separate paragraphs together when they fit", () => { - expect(chunkText("first para\n\nsecond para")).toEqual(["first para\n\nsecond para"]); + describe("embeddingInput", () => { + it("concatenates context and text with a blank line", () => { + const chunks = chunkDocument(baseInput({ body: "hello world" })); + const chunk = chunks[0]; + if (!chunk) throw new Error("expected a chunk"); + expect(embeddingInput(chunk)).toBe(`${chunk.context}\n\n${chunk.text}`); + }); + + it("falls back to bare text when context is empty", () => { + expect(embeddingInput({ context: "", text: "just text" })).toBe("just text"); + }); }); }); @@ -56,6 +247,128 @@ describe("meanEmbedding", () => { }); }); +describe("toIndexDim", () => { + it("is identity (a fresh copy) when the vector is already at the target dim", () => { + const v = new Float32Array([0.6, 0.8]); + const out = toIndexDim(v, 2); + expect(out[0]).toBeCloseTo(0.6, 5); + expect(out[1]).toBeCloseTo(0.8, 5); + expect(out).not.toBe(v); // fresh array, not the same reference + }); + + it("slices and re-L2-normalizes when truncating", () => { + // A unit vector in 4d; truncating to 2d and renormalizing must still be + // unit length, and the truncated components must be proportional to the + // original's. + const v = new Float32Array([0.5, 0.5, 0.5, 0.5]); + const out = toIndexDim(v, 2); + expect(out.length).toBe(2); + let norm = 0; + for (const x of out) norm += x * x; + expect(Math.sqrt(norm)).toBeCloseTo(1, 6); + expect(out[0]).toBeCloseTo(out[1] as number, 6); // proportionality preserved + }); +}); + +describe("provider selection", () => { + afterEach(() => { + resetProviderForTests(); + }); + + it("getQuantize() defaults to float32 after a reset", () => { + expect(getQuantize()).toBe("float32"); + }); + + it("swaps the active provider when dim changes on an unchanged provider id", async () => { + resetProviderForTests(); + const { setProvider } = await import("../../src/search/vector.js"); + setProvider("local-embeddinggemma", { dim: 512 }); + const first = getProvider(); + expect(first.dim).toBe(512); + setProvider("local-embeddinggemma", { dim: 512 }); // repeated tuple: no-op + expect(getProvider()).toBe(first); + setProvider("local-embeddinggemma", { dim: 768 }); // dim flip: must swap + const second = getProvider(); + expect(second.dim).toBe(768); + expect(second).not.toBe(first); + }); + + it("swaps activeQuantize when quantize changes even though (id, dim) is unchanged", async () => { + resetProviderForTests(); + const { setProvider } = await import("../../src/search/vector.js"); + setProvider("local-embeddinggemma", { dim: 512, quantize: "none" }); + const providerA = getProvider(); + expect(getQuantize()).toBe("float32"); + setProvider("local-embeddinggemma", { dim: 512, quantize: "int8" }); + expect(getProvider()).toBe(providerA); // same cached instance — id/dim unchanged + expect(getQuantize()).toBe("int8"); // but the quantize STATE must have swapped + }); + + it("resetProviderForTests reverts to local-minilm and quantize=float32", async () => { + const { setProvider } = await import("../../src/search/vector.js"); + setProvider("local-embeddinggemma", { dim: 512, quantize: "int8" }); + resetProviderForTests(); + expect(getProvider().id).toBe("local-minilm"); + expect(getQuantize()).toBe("float32"); + }); +}); + +function fakeProvider(overrides: Partial = {}): EmbeddingProvider { + return { + id: "fake-provider", + dim: 4, + async embed(texts) { + return ok(texts.map(() => new Float32Array([1, 0, 0, 0]))); + }, + async warm() { + return ok(undefined); + }, + ...overrides, + }; +} + +describe("embedQuery (module-level delegation)", () => { + afterEach(() => { + resetProviderForTests(); + }); + + it("falls back to embed([text]) + toIndexDim when the provider has no embedQuery", async () => { + setProviderForTests(fakeProvider()); + const result = await embedQuery("anything"); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect([...result.value]).toEqual([1, 0, 0, 0]); + }); + + it("delegates directly to the provider's own embedQuery when present", async () => { + const queryVec = new Float32Array([0, 1, 0, 0]); + setProviderForTests( + fakeProvider({ + async embedQuery() { + return ok(queryVec); + }, + }), + ); + const result = await embedQuery("anything"); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value).toBe(queryVec); // exactly the provider's own return, untouched + }); + + it("propagates a provider embed() failure as Result.err", async () => { + const { err } = await import("../../src/frontmatter/types.js"); + setProviderForTests( + fakeProvider({ + async embed() { + return err(new Error("boom")); + }, + }), + ); + const result = await embedQuery("anything"); + expect(result.ok).toBe(false); + }); +}); + describe("embed", () => { it("returns an empty array for empty input without loading the model", async () => { const result = await embed([]); diff --git a/test/search/watcher-integration.test.ts b/test/search/watcher-integration.test.ts index 5b8f44c1..3bcbaf64 100644 --- a/test/search/watcher-integration.test.ts +++ b/test/search/watcher-integration.test.ts @@ -72,7 +72,7 @@ describe("watcher integration with index db", () => { // Sanity: the doc and its manifest entry are present before the unlink. const target = "pricing/helios-consumption-pricing.md"; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; expect(getDocument(opened.value, target)).not.toBeNull(); @@ -83,7 +83,7 @@ describe("watcher integration with index db", () => { fake.emit("unlink", osPath(join(vault, target))); await sleep(80); - const opened2 = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened2 = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened2.ok).toBe(true); if (!opened2.ok) return; expect(getDocument(opened2.value, target)).toBeNull(); @@ -171,7 +171,7 @@ ${marker} body content. // and SQLite write). await sleep(800); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (!opened.ok) return; const doc = getDocument(opened.value, target); diff --git a/test/serve/oauth.test.ts b/test/serve/oauth.test.ts index 25f0e32d..283e812b 100644 --- a/test/serve/oauth.test.ts +++ b/test/serve/oauth.test.ts @@ -7,8 +7,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { createServer as createHttpServer, type Server } from "node:http"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { Client } from "@modelcontextprotocol/sdk/client/index.js"; -import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"; +import { Client, StreamableHTTPClientTransport } from "@modelcontextprotocol/client"; import { exportJWK, generateKeyPair, SignJWT } from "jose"; import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { type ServeHandle, startHttpServer, validateServeStartup } from "../../src/serve/index.js"; @@ -88,7 +87,11 @@ describe("serve OAuth resource-server auth (#7)", () => { new URL(`http://127.0.0.1:${handle.port}/mcp`), { requestInit: token ? { headers: { Authorization: `Bearer ${token}` } } : {} }, ); - const client = new Client({ name: "oauth-test", version: "0.0.0" }); + // Serve is 2026-07-28-only (Decision 1) — pin the modern revision. + const client = new Client( + { name: "oauth-test", version: "0.0.0" }, + { versionNegotiation: { mode: { pin: "2026-07-28" } } }, + ); await client.connect(transport); return client; } @@ -151,26 +154,28 @@ describe("serve OAuth resource-server auth (#7)", () => { }, 30_000); it("a valid JWT with an UNMAPPED subject is 403 — never guest, never a default role", async () => { - await expect(connect(await signJwt("mallory@example.com"))).rejects.toThrow(/forbidden/); + await expect(connect(await signJwt("mallory@example.com"))).rejects.toThrow( + /forbidden|HTTP 403/i, + ); }); it("subjects colliding with Object.prototype members are still 403", async () => { // A plain-object lookup would resolve these to inherited members and // skip the unmapped-subject rejection. for (const sub of ["constructor", "toString", "hasOwnProperty", "__proto__"]) { - await expect(connect(await signJwt(sub))).rejects.toThrow(/forbidden/); + await expect(connect(await signJwt(sub))).rejects.toThrow(/forbidden|HTTP 403/i); } }, 30_000); it("wrong audience, wrong issuer, and expired tokens are 401", async () => { await expect( connect(await signJwt("alice@example.com", { audience: "someone-else" })), - ).rejects.toThrow(/unauthorized/); + ).rejects.toThrow(/unauthorized|HTTP 401/i); await expect( connect(await signJwt("alice@example.com", { issuer: "https://evil.example" })), - ).rejects.toThrow(/unauthorized/); + ).rejects.toThrow(/unauthorized|HTTP 401/i); await expect(connect(await signJwt("alice@example.com", { expiresIn: "-5m" }))).rejects.toThrow( - /unauthorized/, + /unauthorized|HTTP 401/i, ); }); @@ -265,6 +270,6 @@ describe("serve OAuth resource-server auth (#7)", () => { }); it("oauth alone counts as auth configured: no token is 401, not guest", async () => { - await expect(connect()).rejects.toThrow(/unauthorized/); + await expect(connect()).rejects.toThrow(/unauthorized|HTTP 401/i); }); }); diff --git a/test/serve/serve.test.ts b/test/serve/serve.test.ts index 2a447a43..4c04ae54 100644 --- a/test/serve/serve.test.ts +++ b/test/serve/serve.test.ts @@ -1,14 +1,14 @@ -// `daftari serve` (#5): startup gating, token auth, and per-session RBAC -// vantage over Streamable HTTP. The server runs IN-PROCESS on an ephemeral -// loopback port and is driven by the SDK's own client transport — no spawn, -// no network flake surface (spec 2026-07-20, test posture). +// `daftari serve` (#5): startup gating, token auth, and per-request RBAC +// vantage over the stateless 2026-07-28 revision (spec 2026-07-26, +// Decision 1). The server runs IN-PROCESS on an ephemeral loopback port and +// is driven by the SDK's own client transport — no spawn, no network flake +// surface (spec 2026-07-20, test posture). import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { request as httpRequest } from "node:http"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { Client } from "@modelcontextprotocol/sdk/client/index.js"; -import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"; +import { Client, StreamableHTTPClientTransport } from "@modelcontextprotocol/client"; import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; import { matchToken, @@ -80,7 +80,12 @@ async function connect(port: number, token?: string): Promise { const transport = new StreamableHTTPClientTransport(new URL(`http://127.0.0.1:${port}/mcp`), { requestInit: token ? { headers: { Authorization: `Bearer ${token}` } } : {}, }); - const client = new Client({ name: "serve-test", version: "0.0.0" }); + // The client SDK still defaults to legacy negotiation; serve is + // 2026-07-28-only (Decision 1), so pin the modern revision. + const client = new Client( + { name: "serve-test", version: "0.0.0" }, + { versionNegotiation: { mode: { pin: "2026-07-28" } } }, + ); await client.connect(transport); return client; } @@ -185,7 +190,7 @@ describe("serve over Streamable HTTP (in-process, loopback)", () => { rmSync(vault, { recursive: true, force: true }); }); - it("two sessions with different tokens see different RBAC vantages", async () => { + it("two clients with different tokens see different RBAC vantages", async () => { const analyst = await connect(handle.port, "analyst-secret"); const admin = await connect(handle.port, "admin-secret"); try { @@ -201,14 +206,16 @@ describe("serve over Streamable HTTP (in-process, loopback)", () => { } }, 30_000); - it("rejects a missing or unmatched token at session open (401)", async () => { - await expect(connect(handle.port)).rejects.toThrow(/unauthorized/); - await expect(connect(handle.port, "wrong-secret")).rejects.toThrow(/unauthorized/); + it("rejects a missing or unmatched token on every request (401)", async () => { + await expect(connect(handle.port)).rejects.toThrow(/unauthorized|HTTP 401/i); + await expect(connect(handle.port, "wrong-secret")).rejects.toThrow(/unauthorized|HTTP 401/i); }); - it("a session id is not a credential — mismatched identity is 401", async () => { - // Open a session as the analyst via raw fetch so the session id header - // is observable, then replay it with the admin's token. + it("speaks 2026-07-28 only — a 2025-era initialize is refused, and no session id is ever issued", async () => { + // Decision 1: no dual-stacking. A legacy `initialize` (the session-open + // ceremony the 2026-07-28 revision deleted) gets the + // unsupported-protocol-version rejection, not a session. Lagging clients + // use stdio, which serves both eras. const init = await fetch(`http://127.0.0.1:${handle.port}/mcp`, { method: "POST", headers: { @@ -227,22 +234,10 @@ describe("serve over Streamable HTTP (in-process, loopback)", () => { }, }), }); - expect(init.status).toBe(200); - const sessionId = init.headers.get("mcp-session-id"); - expect(sessionId).toBeTruthy(); - await init.body?.cancel(); - - const hijack = await fetch(`http://127.0.0.1:${handle.port}/mcp`, { - method: "POST", - headers: { - authorization: "Bearer admin-secret", - "content-type": "application/json", - accept: "application/json, text/event-stream", - "mcp-session-id": sessionId as string, - }, - body: JSON.stringify({ jsonrpc: "2.0", id: 2, method: "tools/list" }), - }); - expect(hijack.status).toBe(401); + expect(init.headers.get("mcp-session-id")).toBeNull(); + const body = await init.text(); + expect(body).toMatch(/protocol/i); + expect(body).not.toMatch(/"result"\s*:\s*\{[^}]*serverInfo/); }); it("non-/mcp paths are 404", async () => { @@ -294,9 +289,9 @@ describe("serve over Streamable HTTP (in-process, loopback)", () => { }); expect(evil.status).toBe(403); - // A loopback Origin for the bound port passes the guard (the request - // then fails further in as an unknown session, which is the point — - // it got past the rebinding gate, not the session gate). + // A loopback Origin for the bound port passes the guard (the bare + // legacy-shaped request is then refused further in by the protocol + // router, which is the point — it got past the rebinding gate). const okOrigin = await fetch(`http://127.0.0.1:${handle.port}/mcp`, { method: "POST", headers: { @@ -330,7 +325,7 @@ describe("serve with no auth declared (loopback guest mode)", () => { rmSync(vault, { recursive: true, force: true }); }); - it("sessions open without a token and run as the deny-all guest", async () => { + it("requests without a token run as the deny-all guest", async () => { const guest = await connect(handle.port); try { const paths = await searchPaths(guest, "zephyr protocol calibration"); diff --git a/test/server.test.ts b/test/server.test.ts index 4a41b04c..4ab94616 100644 --- a/test/server.test.ts +++ b/test/server.test.ts @@ -1,11 +1,18 @@ import { describe, expect, it } from "vitest"; import { + advertisedSurfaceCost, + allRegisteredTools, CORE_TOOLS, + formatSuccessResult, registeredToolNames, resolveToolExposure, STANDARD_TOOLS, } from "../src/server.js"; +import type { ToolDefinition } from "../src/tools/read.js"; +import { serializeToolDefinition } from "../src/tools/registry.js"; import type { ToolsConfig } from "../src/utils/config.js"; +import { compileToolSchema } from "./helpers/output-schema.js"; +import { cleanupVault, makeTempVault } from "./helpers/temp-vault.js"; function exposure(overrides: Partial): ReturnType { return resolveToolExposure({ tier: "full", include: [], exclude: [], ...overrides }); @@ -66,3 +73,209 @@ describe("tool exposure tiers (#103/#104)", () => { expect(exposed.size).toBe(CORE_TOOLS.length); }); }); + +// spec 2026-07-26, Decision 3 / jugalbandi challenge C6: every registered +// tool's outputSchema must compile under STRICT JSON Schema 2020-12 — a +// misspelled keyword must fail this test, not silently validate everything. +describe("outputSchema — registry-wide strict compile (Decision 3, C6)", () => { + it("every registered tool's outputSchema compiles under strict 2020-12", () => { + for (const tool of allRegisteredTools()) { + expect( + () => compileToolSchema(tool), + `${tool.name}'s outputSchema failed strict compilation`, + ).not.toThrow(); + } + }); + + it("outputSchema is present on every registered tool (Decision 3: required, not optional)", () => { + for (const tool of allRegisteredTools()) { + expect(tool.outputSchema, `${tool.name} has no outputSchema`).toBeTruthy(); + } + }); +}); + +// spec 2026-07-26, Decision 3, PR 1 gap closure. formatSuccessResult is the +// CallTool bridge's presentation step, extracted so it can be driven +// directly against hand-built stub tools — including ones a real registered +// tool can never be (no summarize; a throwing summarize) — without a live +// Server/transport. +describe("formatSuccessResult — CallTool bridge presentation (Decision 3, C5)", () => { + const stubTool = (overrides: Partial): ToolDefinition => ({ + name: "stub_tool", + oneLine: "test stub", + description: "test stub", + inputSchema: { type: "object", properties: {} }, + outputSchema: { type: "object", properties: { n: { type: "number" } }, required: ["n"] }, + handler: async () => ({ ok: true, value: { n: 1 } }), + ...overrides, + }); + + it("a tool with no summarize falls back to pretty-printed JSON (back-compat pin)", () => { + const tool = stubTool({}); + const value = { n: 1, label: "x" }; + const out = formatSuccessResult(tool, value); + expect(out.content[0]).toEqual({ type: "text", text: JSON.stringify(value, null, 2) }); + expect(out.structuredContent).toBe(value); + }); + + it("a tool with summarize ships the summary text instead of JSON", () => { + const tool = stubTool({ summarize: (v) => `summary of ${(v as { n: number }).n}` }); + const out = formatSuccessResult(tool, { n: 42 }); + expect(out.content[0]).toEqual({ type: "text", text: "summary of 42" }); + }); + + it("a throwing summarize still returns the JSON fallback, never an error", () => { + const tool = stubTool({ + summarize: () => { + throw new Error("boom"); + }, + }); + const value = { n: 7 }; + const out = formatSuccessResult(tool, value); + expect(out.content[0]).toEqual({ type: "text", text: JSON.stringify(value, null, 2) }); + // formatSuccessResult never sets isError — the caller (CallTool handler) + // decides that, and a presentation failure must never make it decide + // "error" over a successful handler result. + expect((out as { isError?: boolean }).isError).toBeUndefined(); + }); + + it("a throwing docLinks still returns the summary, with no resource_link entries", () => { + const tool = stubTool({ + summarize: () => "ok", + docLinks: () => { + throw new Error("boom"); + }, + }); + const out = formatSuccessResult(tool, { n: 1 }); + expect(out.content).toEqual([{ type: "text", text: "ok" }]); + }); + + it("docLinks round-trip through docUri as resource_link entries", () => { + const tool = stubTool({ + summarize: () => "ok", + docLinks: () => ["a/b.md", "c.md"], + }); + const out = formatSuccessResult(tool, { n: 1 }); + const links = out.content.slice(1); + expect(links).toEqual([ + { + type: "resource_link", + uri: "daftari://doc/a/b.md", + name: "a/b.md", + mimeType: "text/markdown", + }, + { type: "resource_link", uri: "daftari://doc/c.md", name: "c.md", mimeType: "text/markdown" }, + ]); + }); + + it("docLinks entries are filtered to non-empty strings before becoming links", () => { + const tool = stubTool({ + summarize: () => "ok", + // biome-ignore lint/suspicious/noExplicitAny: exercising a malformed docLinks return + docLinks: () => ["", "real.md", null as any, undefined as any], + }); + const out = formatSuccessResult(tool, { n: 1 }); + expect(out.content).toHaveLength(2); // text + one real link + expect(out.content[1]).toMatchObject({ uri: "daftari://doc/real.md" }); + }); + + it("wireValue projects structuredContent while summarize/docLinks still see the full value", () => { + const seenBySummarize: unknown[] = []; + const tool = stubTool({ + summarize: (v) => { + seenBySummarize.push(v); + return "ok"; + }, + wireValue: (v) => { + const { secret: _secret, ...rest } = v as { secret: string; n: number }; + return rest; + }, + }); + const value = { n: 1, secret: "full-value" }; + const out = formatSuccessResult(tool, value); + expect(out.structuredContent).toEqual({ n: 1 }); + expect(seenBySummarize).toEqual([value]); + }); + + it("no wireValue ships the value verbatim on structuredContent", () => { + const tool = stubTool({ summarize: () => "ok" }); + const value = { n: 1 }; + const out = formatSuccessResult(tool, value); + expect(out.structuredContent).toBe(value); + }); +}); + +// vault_read (C11): the body ships exactly once, on the `content` channel — +// never doubled onto structuredContent. +describe("vault_read wire projection (Decision 3, C11)", () => { + it("structuredContent carries no `content` field, while content[0].text carries it verbatim", () => { + const tool = allRegisteredTools().find((t) => t.name === "vault_read"); + expect(tool).toBeTruthy(); + const value = { + path: "a.md", + content: "the body text", + frontmatter: { title: "t", status: "draft", confidence: "low", collection: "c" }, + raw: {}, + validation: { valid: true, issues: [] }, + hasFrontmatter: true, + decay: null, + validity: null, + upstream_staleness: null, + structural: null, + version: "deadbeef", + }; + const out = formatSuccessResult(tool as ToolDefinition, value); + expect(out.structuredContent).not.toHaveProperty("content"); + expect(out.structuredContent).toMatchObject({ path: "a.md", version: "deadbeef" }); + const text = (out.content[0] as { text: string }).text; + expect(text).toContain("the body text"); + }); +}); + +// spec 2026-07-26-context-packs-progressive-disclosure-design.md, final plan +// Phase 1.5. +describe("vault_tools / vault_context tier membership (Phase 1.4)", () => { + it("both new tools are advertised under tier: core", () => { + const { exposed } = exposure({ tier: "core" }); + expect(exposed.has("vault_tools")).toBe(true); + expect(exposed.has("vault_context")).toBe(true); + }); +}); + +describe("advertisedSurfaceCost", () => { + it("counts only the exposed set, not the whole registry", () => { + const core = allRegisteredTools().filter((t) => CORE_TOOLS.includes(t.name)); + const full = allRegisteredTools(); + expect(core.length).toBeLessThan(full.length); + expect(advertisedSurfaceCost(core)).toBeLessThan(advertisedSurfaceCost(full)); + }); + + it("is the chars/4 estimate of the exact ListTools serialization", () => { + const core = allRegisteredTools().filter((t) => CORE_TOOLS.includes(t.name)); + const serialized = core.map(serializeToolDefinition); + expect(advertisedSurfaceCost(core)).toBe(Math.ceil(JSON.stringify(serialized).length / 4)); + }); +}); + +describe("vault_tools expand output matches the ListTools wire shape (drift test)", () => { + it("serializeToolDefinition is what both vault_tools' expand mode and ListTools ship", async () => { + const searchDef = allRegisteredTools().find((t) => t.name === "vault_search"); + expect(searchDef).toBeTruthy(); + if (!searchDef) return; + const viaListTools = serializeToolDefinition(searchDef); + + const vaultToolsDef = allRegisteredTools().find((t) => t.name === "vault_tools"); + expect(vaultToolsDef).toBeTruthy(); + if (!vaultToolsDef) return; + const vault = makeTempVault(); + try { + const result = await vaultToolsDef.handler(vault, { expand: ["vault_search"] }, undefined); + expect(result.ok).toBe(true); + if (!result.ok) return; + const expanded = (result.value as { tools: unknown[] }).tools[0]; + expect(expanded).toEqual(viaListTools); + } finally { + cleanupVault(vault); + } + }); +}); diff --git a/test/storage/index-db-quantize.test.ts b/test/storage/index-db-quantize.test.ts new file mode 100644 index 00000000..25542799 --- /dev/null +++ b/test/storage/index-db-quantize.test.ts @@ -0,0 +1,178 @@ +// int8 vec-index quantization coverage (spec 2026-07-26-embedding-refresh- +// quantization, Phase 3a). Separate file from index-db.test.ts (that file's +// shared beforeEach opens at "float32" — these tests need their own +// kind-varying opens). + +import { afterEach, describe, expect, it } from "vitest"; +import { + getMeta, + type IndexDb, + insertEmbedding, + insertEmbeddingVec, + openIndexDb, + quantizeInt8, +} from "../../src/storage/index-db.js"; +import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; + +const MODEL = "test-model-v1"; +const COLLECTION = "notes"; + +describe("quantizeInt8", () => { + it("rounds unit-range components to the nearest int8, clamped to [-127, 127]", () => { + const buf = quantizeInt8(new Float32Array([1, -1, 0, 0.5, -0.5])); + const view = new Int8Array(buf.buffer, buf.byteOffset, buf.byteLength); + // Math.round rounds .5 toward +Infinity (JS semantics), so 0.5*127=63.5 + // rounds to 64 but -0.5*127=-63.5 rounds to -63, not -64. + expect([...view]).toEqual([127, -127, 0, 64, -63]); + }); + + it("clamps a value whose scaled magnitude would exceed the int8 range", () => { + // L2-normalized vectors never exceed [-1, 1] per-component, but the + // function itself is defense-in-depth clamped regardless of input. + const buf = quantizeInt8(new Float32Array([2, -2])); + const view = new Int8Array(buf.buffer, buf.byteOffset, buf.byteLength); + expect([...view]).toEqual([127, -127]); + }); + + it("produces a buffer of exactly vec.length bytes", () => { + const buf = quantizeInt8(new Float32Array(512)); + expect(buf.byteLength).toBe(512); + }); +}); + +describe("openIndexDb — kind coherence", () => { + let vault: string; + let db: IndexDb | null = null; + + afterEach(() => { + db?.close(); + db = null; + if (vault) cleanupVault(vault); + }); + + it("createVecTable at kind='int8' persists VEC_KIND_META_KEY and accepts int8-width inserts", () => { + vault = makeTempVault(); + const opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + expect(getMeta(db, "embeddings_vec_kind")).toBe("int8"); + + const vec = new Float32Array([1, 0, 0, 0]); + insertEmbeddingVec(db, "h1", MODEL, COLLECTION, vec, "int8"); + const rows = db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }; + expect(rows.n).toBe(1); + }); + + it("a kind flip (float32 -> int8) on an unchanged dim drops and recreates the vec table", () => { + vault = makeTempVault(); + let opened = openIndexDb(vault, 4, "float32"); + if (!opened.ok) throw opened.error; + db = opened.value; + expect(getMeta(db, "embeddings_vec_kind")).toBe("float32"); + insertEmbeddingVec(db, "h1", MODEL, COLLECTION, new Float32Array([1, 0, 0, 0]), "float32"); + expect((db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }).n).toBe( + 1, + ); + + db.close(); + opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + expect(getMeta(db, "embeddings_vec_kind")).toBe("int8"); + // The vec mirror is dropped and recreated — its rows are gone. + expect((db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }).n).toBe( + 0, + ); + + // A new insert at the now-active int8 kind round-trips correctly. + insertEmbeddingVec(db, "h2", MODEL, COLLECTION, new Float32Array([1, 0, 0, 0]), "int8"); + expect((db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }).n).toBe( + 1, + ); + }); + + it("the durable `embeddings` cache survives a kind flip (only the vec mirror is dropped)", () => { + vault = makeTempVault(); + let opened = openIndexDb(vault, 4, "float32"); + if (!opened.ok) throw opened.error; + db = opened.value; + insertEmbedding(db, "h1", MODEL, new Float32Array([1, 0, 0, 0]), "2026-01-01", 4); + + db.close(); + opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + const row = db + .prepare("SELECT COUNT(*) AS n FROM embeddings WHERE content_hash = ?") + .get("h1") as { n: number }; + expect(row.n).toBe(1); // durable cache untouched by the vec-mirror drop + }); + + it("leaves the vec table alone when both dim and kind persist unchanged", () => { + vault = makeTempVault(); + let opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + insertEmbeddingVec(db, "h1", MODEL, COLLECTION, new Float32Array([1, 0, 0, 0]), "int8"); + + db.close(); + opened = openIndexDb(vault, 4, "int8"); // same dim, same kind + if (!opened.ok) throw opened.error; + db = opened.value; + expect((db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }).n).toBe( + 1, + ); // untouched + }); + + it("a dim flip alone (kind unchanged) still drops and recreates, same as before this PR", () => { + vault = makeTempVault(); + let opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + insertEmbeddingVec(db, "h1", MODEL, COLLECTION, new Float32Array([1, 0, 0, 0]), "int8"); + + db.close(); + opened = openIndexDb(vault, 8, "int8"); // dim changes, kind stays + if (!opened.ok) throw opened.error; + db = opened.value; + expect((db.prepare("SELECT COUNT(*) AS n FROM embeddings_vec").get() as { n: number }).n).toBe( + 0, + ); + }); +}); + +describe("int8 vec table — KNN ordering matches float32 up to quantization error", () => { + let vault: string; + let db: IndexDb; + + afterEach(() => { + db.close(); + cleanupVault(vault); + }); + + it("ranks int8-quantized vectors by cosine distance consistently with their float32 originals", () => { + vault = makeTempVault(); + const opened = openIndexDb(vault, 4, "int8"); + if (!opened.ok) throw opened.error; + db = opened.value; + + const v1 = new Float32Array([1, 0, 0, 0]); + const v2 = new Float32Array([0, 1, 0, 0]); + const v3 = new Float32Array([0.9, Math.sqrt(1 - 0.81), 0, 0]); // unit, near v1 + + insertEmbeddingVec(db, "h1", MODEL, COLLECTION, v1, "int8"); + insertEmbeddingVec(db, "h2", MODEL, COLLECTION, v2, "int8"); + insertEmbeddingVec(db, "h3", MODEL, COLLECTION, v3, "int8"); + + const queryBlob = quantizeInt8(v1); + const rows = db + .prepare( + `SELECT content_hash, distance + FROM embeddings_vec + WHERE embedding MATCH vec_int8(?) AND model = ? AND k = ? + ORDER BY distance`, + ) + .all(queryBlob, MODEL, 3) as { content_hash: string; distance: number }[]; + expect(rows.map((r) => r.content_hash)).toEqual(["h1", "h3", "h2"]); + }); +}); diff --git a/test/storage/index-db.test.ts b/test/storage/index-db.test.ts index 834503d7..d96de2c2 100644 --- a/test/storage/index-db.test.ts +++ b/test/storage/index-db.test.ts @@ -52,6 +52,7 @@ const sampleDoc: IndexedDocument = { supersededBy: null, validFrom: null, validUntil: null, + updatedBy: "", }; describe("index-db", () => { @@ -60,7 +61,7 @@ describe("index-db", () => { beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); @@ -183,13 +184,13 @@ describe("index-db", () => { // snapshot. `embeddings` is deliberately exempt from the drop — it is a // content-addressed cache, covered by schema-bump-embeddings.test.ts — // so the count below is 0 only because this test inserts none. - const reopened = openIndexDb(vault, LOCAL_MINILM_DIM); + const reopened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!reopened.ok) throw reopened.error; db = reopened.value; expect(documentCount(db)).toBe(0); expect(embeddingCount(db)).toBe(0); - expect(getMeta(db, "schema_version")).toBe("11"); + expect(getMeta(db, "schema_version")).toBe("14"); expect(getMeta(db, "vault_manifest")).toBeNull(); }); @@ -207,12 +208,12 @@ describe("index-db", () => { expect(documentCount(db)).toBe(1); db.close(); - const reopened = openIndexDb(vault, LOCAL_MINILM_DIM); + const reopened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!reopened.ok) throw reopened.error; db = reopened.value; expect(documentCount(db)).toBe(0); - expect(getMeta(db, "schema_version")).toBe("11"); + expect(getMeta(db, "schema_version")).toBe("14"); // All five expected tables now exist on a fresh index: three // regular tables (documents, chunks, embeddings, meta) plus two // virtual tables (documents_fts, embeddings_vec). @@ -460,7 +461,7 @@ describe("index-db", () => { // afterEach has a live handle to close. db.close(); const fresh = makeTempVault(); - const opened = openIndexDb(fresh, 4); + const opened = openIndexDb(fresh, 4, "float32"); if (!opened.ok) throw opened.error; db = opened.value; insertEmbeddingVec(db, "h1", MODEL, COLLECTION, v1); @@ -483,7 +484,7 @@ describe("index-db", () => { // suite-level afterEach has a valid db handle. db.close(); cleanupVault(fresh); - const reopened = openIndexDb(vault, LOCAL_MINILM_DIM); + const reopened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!reopened.ok) throw reopened.error; db = reopened.value; }); @@ -492,7 +493,7 @@ describe("index-db", () => { // First open creates the vec table at dim=4. db.close(); const fresh = makeTempVault(); - let opened = openIndexDb(fresh, 4); + let opened = openIndexDb(fresh, 4, "float32"); if (!opened.ok) throw opened.error; db = opened.value; expect(getMeta(db, "embeddings_vec_dim")).toBe("4"); @@ -510,7 +511,7 @@ describe("index-db", () => { // Reopen at a different dim — the vec table is dropped and recreated; // any rows in it are gone (the durable cache survives — `embeddings` // and `chunks` tables are not touched). - opened = openIndexDb(fresh, 8); + opened = openIndexDb(fresh, 8, "float32"); if (!opened.ok) throw opened.error; db = opened.value; expect(getMeta(db, "embeddings_vec_dim")).toBe("8"); @@ -532,7 +533,7 @@ describe("index-db", () => { db.close(); cleanupVault(fresh); - const reopened = openIndexDb(vault, LOCAL_MINILM_DIM); + const reopened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!reopened.ok) throw reopened.error; db = reopened.value; }); @@ -586,7 +587,7 @@ describe("getDocumentsInDateRange", () => { let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const o = openIndexDb(vault, LOCAL_MINILM_DIM); + const o = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!o.ok) throw o.error; db = o.value; }); @@ -624,7 +625,7 @@ describe("insertDocument date normalization (index is cleaned; source is not)", let db: IndexDb; beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); @@ -680,7 +681,7 @@ describe("chunks_fts", () => { writeFileSync(join(vault, "big.md"), `---\ntitle: Big\n---\n\n${para1}\n\n${para2}\n`); let r = await reindexVault(vault); if (!r.ok) throw r.error; - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const db = opened.value; @@ -739,7 +740,7 @@ describe("chunks_fts", () => { // We call deleteDocument + insertChunkRow directly to mirror that code path // without spinning up the embedding model or file system watcher. const vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; const testDb = opened.value; diff --git a/test/storage/schema-bump-embeddings.test.ts b/test/storage/schema-bump-embeddings.test.ts index 21c1cdfd..295735b5 100644 --- a/test/storage/schema-bump-embeddings.test.ts +++ b/test/storage/schema-bump-embeddings.test.ts @@ -28,7 +28,7 @@ function sampleVector(): Float32Array { } function open(vault: string): IndexDb { - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; return opened.value; } diff --git a/test/storage/schema-valid-from-migration.test.ts b/test/storage/schema-valid-from-migration.test.ts index ac66c459..0031c802 100644 --- a/test/storage/schema-valid-from-migration.test.ts +++ b/test/storage/schema-valid-from-migration.test.ts @@ -81,7 +81,7 @@ describe("upgrading an index built before the valid-time columns", () => { writePreValidityIndex(path, "10"); expect(columns(path, "documents")).not.toContain("valid_from"); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); // Before the bump to "11" this returned `no such column: valid_from` on the // first write, leaving the vault unable to serve. expect(opened.ok).toBe(true); @@ -92,7 +92,7 @@ describe("upgrading an index built before the valid-time columns", () => { const path = indexDbPath(vault); writePreValidityIndex(path, "10"); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (opened.ok) opened.value.close(); @@ -110,7 +110,7 @@ describe("upgrading an index built before the valid-time columns", () => { .run("vault_manifest", '{"stale":"entry"}'); seed.close(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(opened.ok).toBe(true); if (opened.ok) opened.value.close(); @@ -128,7 +128,7 @@ describe("upgrading an index built before the valid-time columns", () => { const path = indexDbPath(vault); writePreValidityIndex(path, "10"); - const first = openIndexDb(vault, LOCAL_MINILM_DIM); + const first = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(first.ok).toBe(true); if (first.ok) first.value.close(); diff --git a/test/storage/validity-index.test.ts b/test/storage/validity-index.test.ts index 397e2bed..f0c34556 100644 --- a/test/storage/validity-index.test.ts +++ b/test/storage/validity-index.test.ts @@ -49,7 +49,7 @@ describe("validity columns", () => { beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); @@ -106,7 +106,7 @@ describe("supersessionPredecessors", () => { beforeEach(() => { vault = makeTempVault(); - const opened = openIndexDb(vault, LOCAL_MINILM_DIM); + const opened = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); if (!opened.ok) throw opened.error; db = opened.value; }); diff --git a/test/tools/consumes.test.ts b/test/tools/consumes.test.ts index 8d911fc4..ae68f0cd 100644 --- a/test/tools/consumes.test.ts +++ b/test/tools/consumes.test.ts @@ -1,10 +1,13 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; -import { vaultConsumes } from "../../src/tools/consumes.js"; +import { consumesTools, vaultConsumes } from "../../src/tools/consumes.js"; import { vaultRead } from "../../src/tools/read.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:compiler"; +const consumesTool = consumesTools.find((t) => t.name === "vault_consumes"); +if (!consumesTool) throw new Error("vault_consumes not registered"); function frontmatter(overrides: Record = {}) { return { @@ -66,6 +69,7 @@ describe("vault_consumes (#233)", () => { ]); expect(forward.value.edges[0]?.edge_type).toBe("whole-doc-read"); expect(forward.value.edges[0]?.run_id).toBe("run-42"); + expectMatchesOutputSchema(consumesTool, forward.value); // Reverse: the unit's dependents. const reverse = await vaultConsumes(vault, { diff --git a/test/tools/context.test.ts b/test/tools/context.test.ts new file mode 100644 index 00000000..1a614b3b --- /dev/null +++ b/test/tools/context.test.ts @@ -0,0 +1,342 @@ +// vault_context handler-level tests (spec 2026-07-26-context-packs- +// progressive-disclosure-design.md, final plan Phase 2.7). +// +// Structural assertions only (C6) — inclusion, flag presence, RBAC omission, +// log contents. No golden-brief byte pin anywhere in this file. + +import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import type { AccessContext } from "../../src/access/rbac.js"; +import { readReadLog } from "../../src/curation/read-log.js"; +import { addTension } from "../../src/curation/tension.js"; +import { reindexVault } from "../../src/search/reindex.js"; +import { + contextTools, + DEFAULT_BUDGET, + MAX_BUDGET, + MIN_BUDGET, + parseBudget, + vaultContext, +} from "../../src/tools/context.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; + +const contextTool = contextTools.find((t) => t.name === "vault_context"); +if (!contextTool) throw new Error("vault_context not registered"); + +function frontmatter(fields: { + title: string; + collection: string; + status?: string; + supersededBy?: string; +}): string { + const lines = [ + "---", + `title: "${fields.title}"`, + `collection: ${fields.collection}`, + "domain: product", + `status: ${fields.status ?? "canonical"}`, + "confidence: high", + "created: 2026-01-01", + "updated: 2026-01-01", + "updated_by: human:alice", + "tags: []", + ]; + if (fields.supersededBy) lines.push(`superseded_by: ${fields.supersededBy}`); + lines.push("---", ""); + return lines.join("\n"); +} + +const WIDGET_QUERY = "widget launch plan announcement"; +const WIDGET_BODY = "widget launch plan announcement ".repeat(20); + +function publicRole(): AccessContext { + return { + user: "reader", + roleName: "public-reader", + role: { read: ["public"], write: [], promote: false, ratify: false }, + }; +} + +describe("vault_context", () => { + let vault: string; + + beforeAll(async () => { + vault = mkdtempSync(join(tmpdir(), "daftari-context-")); + mkdirSync(join(vault, "public"), { recursive: true }); + mkdirSync(join(vault, "secret"), { recursive: true }); + mkdirSync(join(vault, ".daftari"), { recursive: true }); + + const widgetBody = WIDGET_BODY; + + // Plain matching doc — no supersession, no tension. + writeFileSync( + join(vault, "public", "alpha.md"), + `${frontmatter({ title: "Alpha", collection: "public" })}${widgetBody}\n`, + ); + + // A supersession chain: old-widget (stale, matches the query, carries a + // tension) -> new-widget (the head, canonical, carries NO tension). + writeFileSync( + join(vault, "public", "old-widget.md"), + `${frontmatter({ title: "Old Widget", collection: "public", status: "superseded", supersededBy: "public/new-widget.md" })}${widgetBody}\n`, + ); + writeFileSync( + join(vault, "public", "new-widget.md"), + `${frontmatter({ title: "New Widget", collection: "public" })}The current widget plan supersedes the old one.\n`, + ); + writeFileSync( + join(vault, "public", "other.md"), + `${frontmatter({ title: "Other", collection: "public" })}Unrelated content.\n`, + ); + + // A restricted-hop chain: stale doc (public, matches query) superseded by + // a document in a collection the test role cannot read. + writeFileSync( + join(vault, "public", "restricted-stale.md"), + `${frontmatter({ title: "Restricted Stale", collection: "public", status: "superseded", supersededBy: "secret/restricted-head.md" })}${widgetBody}\n`, + ); + writeFileSync( + join(vault, "secret", "restricted-head.md"), + `${frontmatter({ title: "Restricted Head", collection: "secret" })}Secret current content.\n`, + ); + + // A hidden doc that ALSO matches the query lexically — an observable + // RBAC-dropped BM25-side candidate (C4's counted case). + writeFileSync( + join(vault, "secret", "hidden-match.md"), + `${frontmatter({ title: "Hidden Match", collection: "secret" })}${widgetBody}\n`, + ); + + // A hidden doc that is topically related but shares NO lexical terms + // with WIDGET_QUERY (paraphrased entirely differently) — used to prove + // hidden_remainder does NOT catch semantic-only hidden relevance (C4): + // the vector half is RBAC-pushdown-scrubbed before this document could + // ever become an observable drop, so it is structurally invisible to + // the count, not merely absent by chance. + writeFileSync( + join(vault, "secret", "quiet.md"), + `${frontmatter({ title: "Quiet", collection: "secret" })}Merchandise release schedule for retail partners.\n`, + ); + + // Many large filler docs so a small budget cannot include them all — + // forces a real budget cut for the C1 read-log test. + for (let i = 0; i < 8; i++) { + writeFileSync( + join(vault, "public", `filler-${i}.md`), + `${frontmatter({ title: `Filler ${i}`, collection: "public" })}${widgetBody.repeat(30)}\n`, + ); + } + + await addTension(vault, { + kind: "factual", + title: "widget scope disagreement", + sourceA: "public/old-widget.md", + claimA: "the widget ships in Q1", + sourceB: "public/other.md", + claimB: "the widget ships in Q2", + loggedBy: "human:alice", + }); + + const reindexed = await reindexVault(vault); + if (!reindexed.ok) throw reindexed.error; + }, 120_000); + + afterAll(() => { + // Best-effort; temp dirs are cleaned by the OS eventually either way, + // matching the sibling rerank fixture's posture. + }); + + it("assembles a brief for a matching task, output matches the schema", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 4000 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.manifest.included.length).toBeGreaterThan(0); + expectMatchesOutputSchema(contextTool, result.value); + }); + + it("zero-hit task returns the empty-pack shape (C9)", async () => { + // An empty vault (no documents at all) is the unambiguous zero-hit case: + // with any real corpus, vector similarity is never exactly zero for + // every candidate, so this vault is built with nothing to match at all + // rather than trying to word a query no document resembles. + const empty = mkdtempSync(join(tmpdir(), "daftari-context-empty-")); + mkdirSync(join(empty, ".daftari"), { recursive: true }); + const result = await vaultContext(empty, { task: "anything at all" }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.manifest.included).toEqual([]); + expect(result.value.manifest.omitted_over_budget).toBe(0); + expect(result.value.brief).toContain("No matching documents"); + }, 30_000); + + describe("budget parsing (C9)", () => { + it("absent/non-numeric budget defaults", () => { + expect(parseBudget(undefined)).toEqual({ ok: true, value: DEFAULT_BUDGET }); + expect(parseBudget("4000")).toEqual({ ok: true, value: DEFAULT_BUDGET }); + expect(parseBudget(Number.NaN)).toEqual({ ok: true, value: DEFAULT_BUDGET }); + }); + + it("a finite budget below the minimum is an error, never silently clamped up", () => { + const result = parseBudget(499); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain(String(MIN_BUDGET)); + }); + + it("a finite budget above the maximum clamps down silently", () => { + expect(parseBudget(MAX_BUDGET + 5000)).toEqual({ ok: true, value: MAX_BUDGET }); + }); + + it("the handler surfaces the same budget error", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 100 }); + expect(result.ok).toBe(false); + }); + }); + + describe("RBAC (omission over redaction, no existence leak)", () => { + it("a restricted role's pack never names an unreadable document", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 8000 }, publicRole()); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.brief).not.toContain("secret/"); + expect(result.value.brief).not.toContain("hidden-match"); + for (const e of result.value.manifest.included) { + expect(e.path.startsWith("secret/")).toBe(false); + } + }); + + it("hidden_remainder is coarsened (some/many), never an exact count, when a readable RBAC drop is observed", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 8000 }, publicRole()); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(["some", "many"]).toContain(result.value.manifest.hidden_remainder); + }); + + // C4: a lexically-quiet hidden document (no shared BM25 terms with the + // task, in an unreadable collection) yields hidden_remainder: "none" — + // asserted DELIBERATELY. "none" here means "no withholding observed", + // never "nothing withheld": the vector half of retrieval is RBAC- + // pushdown-scrubbed (2026-07-26 fusion spec, Decision 3), so this + // document was never a candidate to begin with, not merely filtered. + it("a lexically-quiet hidden document yields hidden_remainder: 'none' (C4)", async () => { + // Isolated vault: the ONLY hidden document is a paraphrase sharing no + // BM25 terms with the task, and no other secret doc exists to + // contaminate the count via a different channel. + const quietVault = mkdtempSync(join(tmpdir(), "daftari-context-quiet-")); + mkdirSync(join(quietVault, "public"), { recursive: true }); + mkdirSync(join(quietVault, "secret"), { recursive: true }); + writeFileSync( + join(quietVault, "public", "task-doc.md"), + `${frontmatter({ title: "Task Doc", collection: "public" })}${WIDGET_BODY}\n`, + ); + writeFileSync( + join(quietVault, "secret", "quiet.md"), + `${frontmatter({ title: "Quiet", collection: "secret" })}Merchandise release schedule for retail partners.\n`, + ); + const reindexed = await reindexVault(quietVault); + if (!reindexed.ok) throw reindexed.error; + + const result = await vaultContext( + quietVault, + { task: WIDGET_QUERY, budget: 8000 }, + publicRole(), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.manifest.hidden_remainder).toBe("none"); + }, 30_000); + }); + + describe("supersession collapse (C3 — head-keyed flags)", () => { + it("a chain head entry carries supersedes: N and none of the stale member's own flags", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 16000 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + // old-widget.md never appears — it collapsed into its head. + expect(result.value.manifest.included.map((e) => e.path)).not.toContain( + "public/old-widget.md", + ); + const head = result.value.manifest.included.find((e) => e.path === "public/new-widget.md"); + expect(head).toBeTruthy(); + expect(result.value.brief).toContain("supersedes 1 older document matching this task"); + // The stale member's tension (old-widget vs other.md) must NOT be + // borrowed onto the head's entry — the head carries no tension. + const headBlockStart = result.value.brief.indexOf("### New Widget"); + const nextHeading = result.value.brief.indexOf("\n### ", headBlockStart + 1); + const headBlock = result.value.brief.slice( + headBlockStart, + nextHeading === -1 ? undefined : nextHeading, + ); + expect(headBlock).not.toContain("contested"); + expect(headBlock).not.toContain("the widget ships in Q1"); + }); + }); + + describe("restricted supersession hop", () => { + it("a restricted hop yields the path-free 'current source: restricted' flag, kept as itself", async () => { + const result = await vaultContext(vault, { task: WIDGET_QUERY, budget: 16000 }, publicRole()); + expect(result.ok).toBe(true); + if (!result.ok) return; + const entry = result.value.manifest.included.find( + (e) => e.path === "public/restricted-stale.md", + ); + expect(entry).toBeTruthy(); + expect(result.value.brief).toContain("current source: restricted"); + expect(result.value.brief).not.toContain("secret/restricted-head.md"); + }); + }); + + describe("empty index — auto-reindex path", () => { + it("a fresh vault with no built index still returns a pack (ensureIndexReady auto-reindexes)", async () => { + const fresh = mkdtempSync(join(tmpdir(), "daftari-context-fresh-")); + mkdirSync(join(fresh, "notes"), { recursive: true }); + writeFileSync( + join(fresh, "notes", "a.md"), + `${frontmatter({ title: "A", collection: "notes" })}${widgetBodyFallback()}\n`, + ); + const result = await vaultContext(fresh, { task: "some task text" }, undefined); + expect(result.ok).toBe(true); + }, 60_000); + }); + + describe("read log (C1) — only survivors of the budget cut are logged", () => { + it("the read log contains exactly the included paths and none of the budget-cut entries", async () => { + // A dedicated, freshly-reindexed vault — a shared vault would have + // accumulated read-log entries from every earlier vault_context call + // in this file, which would make an exact-set comparison meaningless. + const logVault = mkdtempSync(join(tmpdir(), "daftari-context-log-")); + mkdirSync(join(logVault, "public"), { recursive: true }); + for (let i = 0; i < 8; i++) { + writeFileSync( + join(logVault, "public", `doc-${i}.md`), + `${frontmatter({ title: `Doc ${i}`, collection: "public" })}${WIDGET_BODY.repeat(30)}\n`, + ); + } + const reindexed = await reindexVault(logVault); + if (!reindexed.ok) throw reindexed.error; + + const budget = MIN_BUDGET; // small on purpose, forces a real cut + const result = await vaultContext(logVault, { task: WIDGET_QUERY, budget }); + expect(result.ok).toBe(true); + if (!result.ok) return; + // Sanity: this scenario must actually exercise a cut, or the assertion + // below would pass vacuously. + expect(result.value.manifest.omitted_over_budget).toBeGreaterThan(0); + + const logResult = await readReadLog(logVault); + expect(logResult.ok).toBe(true); + if (!logResult.ok) return; + const loggedPaths = new Set( + logResult.value.filter((e) => e.tool === "vault_context").map((e) => e.file), + ); + const includedPaths = new Set(result.value.manifest.included.map((e) => e.path)); + expect(loggedPaths).toEqual(includedPaths); + }, 60_000); + }); +}); + +function widgetBodyFallback(): string { + return "some task text ".repeat(5); +} diff --git a/test/tools/curation.test.ts b/test/tools/curation.test.ts index dbb3bb33..b26a64fe 100644 --- a/test/tools/curation.test.ts +++ b/test/tools/curation.test.ts @@ -16,9 +16,16 @@ import { vaultTensionLog, vaultTensionResolve, } from "../../src/tools/curation.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; const LINT_VAULT = resolve("test/fixtures/lint-vault"); +function curationTool(name: string) { + const t = curationTools.find((x) => x.name === name); + if (!t) throw new Error(`${name} not registered`); + return t; +} + describe("curation tools", () => { let vault: string; @@ -37,6 +44,7 @@ describe("curation tools", () => { if (!result.ok) return; expect(result.value.filter).toBeNull(); expect(result.value.totalFindings).toBe(9); + expectMatchesOutputSchema(curationTool("vault_lint"), result.value); // Derived from LINT_CHECKS rather than hardcoded: the point of this // assertion is that EVERY registered check appears in the report, not // that the vault happens to have N of them. @@ -101,6 +109,7 @@ describe("curation tools", () => { expect(result.value.status).toBe("unresolved"); expect(result.value.loggedBy).toBe("agent:claude-code"); expect(result.value.kind).toBe("factual"); + expectMatchesOutputSchema(curationTool("vault_tension_log"), result.value); const logged = await listTensions(vault); expect(logged.ok && logged.value).toHaveLength(1); @@ -171,6 +180,7 @@ describe("curation tools", () => { expect(cluster?.documents).toEqual(["a.md", "b.md", "c.md"]); expect(cluster?.id).toMatch(/^cluster:[0-9a-f]{8}$/); expect(cluster?.tension_count).toBe(2); + expectMatchesOutputSchema(curationTool("vault_tension_clusters"), result.value); }); it("drops accepted-resolution tensions from cluster scope", async () => { @@ -260,6 +270,7 @@ describe("curation tools", () => { if (!result.ok) return; expect(result.value.count).toBe(2); expect(result.value.history.map((e) => e.action)).toEqual(["create", "promote"]); + expectMatchesOutputSchema(curationTool("vault_provenance"), result.value); }); it("returns an empty history for a file with no recorded writes", async () => { diff --git a/test/tools/edge-staleness.test.ts b/test/tools/edge-staleness.test.ts index e0881cac..8ba11fb6 100644 --- a/test/tools/edge-staleness.test.ts +++ b/test/tools/edge-staleness.test.ts @@ -2,12 +2,15 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { observeEdge } from "../../src/curation/edges.js"; import { recordProvenance } from "../../src/curation/provenance.js"; import { readReadLog } from "../../src/curation/read-log.js"; -import { vaultStaleness } from "../../src/tools/edge-staleness.js"; +import { edgeStalenessTools, vaultStaleness } from "../../src/tools/edge-staleness.js"; import { vaultRead } from "../../src/tools/read.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:compiler"; +const stalenessTool = edgeStalenessTools.find((t) => t.name === "vault_staleness"); +if (!stalenessTool) throw new Error("vault_staleness not registered"); function frontmatter(overrides: Record = {}) { return { @@ -98,6 +101,7 @@ describe("vault_staleness (#234)", () => { expect(artifact.value.edges[0]?.edge_class).toBe("compiled"); expect(artifact.value.edges[0]?.staleness).toBe("pending-broken"); expect(artifact.value.summary.pending_broken).toBe(1); + expectMatchesOutputSchema(stalenessTool, artifact.value); const citer = await vaultStaleness(vault, { artifact: "pricing/citer.md" }); expect(citer.ok).toBe(true); @@ -171,6 +175,7 @@ describe("vault_staleness (#234)", () => { expect(report.value.broken_serves).toBeGreaterThanOrEqual(1); expect(report.value.broken_read_rate).toBeGreaterThan(0); expect(report.value.by_tool.vault_read?.broken_serves).toBeGreaterThanOrEqual(1); + expectMatchesOutputSchema(stalenessTool, report.value); }, 60_000); it("omits edges to unreadable units and coarsens them into hidden_pending", async () => { diff --git a/test/tools/edges.test.ts b/test/tools/edges.test.ts index 61c805cf..619b9448 100644 --- a/test/tools/edges.test.ts +++ b/test/tools/edges.test.ts @@ -1,12 +1,29 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import type { AccessContext } from "../../src/access/rbac.js"; -import { observeEdge } from "../../src/curation/edges.js"; +import { + computeInputsFingerprint, + edgeEvidenceClasses, + observeEdge, +} from "../../src/curation/edges.js"; import { listTensions } from "../../src/curation/tension.js"; -import { vaultEdgeContest, vaultEdgeObserve, vaultEdges } from "../../src/tools/edges.js"; +import { readFile, resolveVaultPath } from "../../src/storage/local.js"; +import { + edgeTools, + vaultEdgeContest, + vaultEdgeObserve, + vaultEdges, +} from "../../src/tools/edges.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:curation-loop"; + +function edgeTool(name: string) { + const t = edgeTools.find((x) => x.name === name); + if (!t) throw new Error(`${name} not registered`); + return t; +} const GUEST: AccessContext = { user: "guest", roleName: "guest", role: null }; function frontmatter(overrides: Record = {}) { @@ -59,6 +76,7 @@ describe("vault_edge_observe", () => { if (!result.ok) return; expect(result.value.status).toBe("candidate"); expect(result.value.kSurvived).toBe(0); // birth is not a survival + expectMatchesOutputSchema(edgeTool("vault_edge_observe"), result.value); }, 60_000); it("rejects an edge whose endpoint document does not exist", async () => { @@ -119,6 +137,7 @@ describe("vault_edge_observe", () => { const all = await vaultEdges(vault, {}); expect(all.ok && all.value.total).toBe(1); expect(all.ok && all.value.edges[0]?.fromPath).toBe("pricing/a.md"); + if (all.ok) expectMatchesOutputSchema(edgeTool("vault_edges"), all.value); }, 60_000); it("rejects an endpoint that escapes the vault", async () => { @@ -161,6 +180,159 @@ describe("vault_edge_observe", () => { }); }); +describe("vault_edge_observe — evidence fingerprint (2026-07-26 spec, Decision 1)", () => { + let vault: string; + beforeEach(() => { + vault = makeTempVault(); + }); + afterEach(() => { + cleanupVault(vault); + }); + + it("evidence_paths → the server-computed inputs hash matches a test-side recomputation", async () => { + await seed(vault, "pricing/a.md"); + await seed(vault, "pricing/b.md"); + // Seed the edge (blind:false — never registers a class), THEN the + // fingerprinted qualifying vote, so the vote's class is inspectable via + // edgeEvidenceClasses. + await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: false, + }); + const result = await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: true, + varied_axis: "prompt", + evidence_paths: ["pricing/a.md", "pricing/b.md"], + model: "claude-haiku-test", + prompt_id: "manual/spot-check", + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.kEff).toBeCloseTo(1, 6); // one fresh class + + // Recompute the expected hash the same way the tool does: over the + // CURRENT full bytes of the two evidence paths (the MCP path hashes full + // file content, unlike the loop's prompt-truncated form). + const aAbs = resolveVaultPath(vault, "pricing/a.md"); + const bAbs = resolveVaultPath(vault, "pricing/b.md"); + if (!aAbs.ok || !bAbs.ok) throw new Error("resolve failed"); + const aText = await readFile(aAbs.value.absPath); + const bText = await readFile(bAbs.value.absPath); + if (!aText.ok || !bText.ok) throw new Error("read failed"); + const expected = computeInputsFingerprint([ + { path: "pricing/a.md", text: aText.value }, + { path: "pricing/b.md", text: bText.value }, + ]); + + const classesRes = edgeEvidenceClasses(vault, "pricing/a.md", "pricing/b.md"); + expect(classesRes.ok).toBe(true); + if (!classesRes.ok) return; + const [classKey] = [...classesRes.value.keys()]; + expect(classKey).toBeDefined(); + const [inputsComponent] = (classKey as string).split("\n"); + expect(inputsComponent).toBe(expected); + }, 60_000); + + it("a nonexistent evidence path errors", async () => { + await seed(vault, "pricing/a.md"); + await seed(vault, "pricing/b.md"); + const result = await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: true, + evidence_paths: ["pricing/does-not-exist.md"], + }); + expect(result.ok).toBe(false); + if (!result.ok) expect(result.error.message).toMatch(/evidence path not found/); + }, 60_000); + + it("fp.principal is taken from the access context, never from args or observed_by", async () => { + await seed(vault, "pricing/a.md"); + await seed(vault, "pricing/b.md"); + // Seed the edge first (k=0, no class), then cast the fingerprinted vote + // under an access context — its class should reflect the AUTHENTICATED + // principal, not the caller-claimed observed_by. + await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: false, + }); + const access: AccessContext = { + user: "human:mihir", + roleName: "curator", + role: { read: ["*"], write: ["*"], promote: true, ratify: true }, + }; + const voted = await vaultEdgeObserve( + vault, + { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + // A caller-claimed observed_by that must NOT leak into fp.principal. + observed_by: "agent:someone-else-entirely", + blind: true, + varied_axis: "prompt", + evidence_paths: ["pricing/a.md", "pricing/b.md"], + }, + access, + ); + expect(voted.ok).toBe(true); + if (!voted.ok) return; + // One counted vote, one class (the seed contributes none) → kEff = 1. + expect(voted.value.kEff).toBeCloseTo(1, 6); + + // Independently: the SAME class, re-voted with a different claimed + // observed_by but the SAME access principal, must land in the SAME + // class (proving the class key used the access principal, not + // observed_by) — a second counted vote in that class gains 0.5. + const voted2 = await vaultEdgeObserve( + vault, + { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: "agent:yet-another-claimed-identity", + blind: true, + varied_axis: "model", + evidence_paths: ["pricing/a.md", "pricing/b.md"], + }, + access, + ); + expect(voted2.ok).toBe(true); + if (!voted2.ok) return; + expect(voted2.value.kEff).toBeCloseTo(1.5, 6); + }, 60_000); + + it("an observe with no fp-related args carries no fp (legacy/unfingerprinted)", async () => { + await seed(vault, "pricing/a.md"); + await seed(vault, "pricing/b.md"); + await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: false, + }); + const result = await vaultEdgeObserve(vault, { + from_path: "pricing/a.md", + to_path: "pricing/b.md", + observed_by: AGENT, + blind: true, + varied_axis: "prompt", + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + // No fp anywhere: this counted vote collapses into the single ∅ class, + // same as the pre-fingerprint behavior — kEff equals kSurvived for k=1. + expect(result.value.kEff).toBeCloseTo(1, 6); + expect(result.value.kSurvived).toBe(1); + }, 60_000); +}); + describe("vault_edge_contest", () => { let vault: string; beforeEach(() => { @@ -191,6 +363,7 @@ describe("vault_edge_contest", () => { if (!result.ok) return; expect(result.value.edge.status).toBe("revoked"); expect(result.value.tension_id).toMatch(/^tension-\d+$/); + expectMatchesOutputSchema(edgeTool("vault_edge_contest"), result.value); const tensions = await listTensions(vault); expect(tensions.ok).toBe(true); diff --git a/test/tools/read-anchors.test.ts b/test/tools/read-anchors.test.ts new file mode 100644 index 00000000..9ab03131 --- /dev/null +++ b/test/tools/read-anchors.test.ts @@ -0,0 +1,348 @@ +// Citation-anchors JIT verification on vault_read (2026-07-26 spec, +// Decisions 1-2, 4; role gate per the 2026-07-27 resolution). + +import { execFileSync } from "node:child_process"; +import { chmodSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import type { AccessContext } from "../../src/access/rbac.js"; +import { readReadLog } from "../../src/curation/read-log.js"; +import { readTools, vaultRead } from "../../src/tools/read.js"; +import { hashObjects } from "../../src/utils/git.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; +import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; + +const readTool = readTools.find((t) => t.name === "vault_read"); +if (!readTool) throw new Error("vault_read not registered"); + +const GIT_ENV = { + ...process.env, + GIT_AUTHOR_NAME: "t", + GIT_AUTHOR_EMAIL: "t@t", + GIT_COMMITTER_NAME: "t", + GIT_COMMITTER_EMAIL: "t@t", +}; + +function git(cwd: string, args: string[]): void { + execFileSync("git", args, { cwd, env: GIT_ENV, stdio: "ignore" }); +} + +function frontmatter(describes: string[], over: Record = {}): string { + const base: Record = { + title: "Retry logic notes", + domain: "accumulation", + collection: "engineering", + status: "canonical", + confidence: "high", + created: "2026-01-05", + updated: "2026-01-05", + updated_by: "agent:test", + provenance: "direct", + ...over, + }; + const lines = Object.entries(base).map(([k, v]) => `${k}: ${v}`); + const describesYaml = + describes.length > 0 + ? `describes:\n${describes.map((d) => ` - "${d}"`).join("\n")}\n` + : "describes: []\n"; + return `---\n${lines.join("\n")}\ntags: []\n${describesYaml}---\n\nThe retry loop lives in the code repo.\n`; +} + +describe("vault_read — citation anchors", () => { + let vault: string; + let codeRepo: string; + let sha: string; + + beforeEach(() => { + vault = makeTempVault(); + codeRepo = realpathSync(mkdtempSync(join(tmpdir(), "daftari-anchors-code-"))); + git(codeRepo, ["init", "-q"]); + writeFileSync(join(codeRepo, "retry.ts"), "export function retry() {}\n"); + git(codeRepo, ["add", "."]); + git(codeRepo, ["commit", "-q", "-m", "init"]); + const hashes = execFileSync("git", ["-C", codeRepo, "hash-object", "retry.ts"], { + env: GIT_ENV, + }) + .toString() + .trim(); + sha = hashes; + + mkdirSync(join(vault, ".daftari"), { recursive: true }); + mkdirSync(join(vault, "engineering"), { recursive: true }); + }); + + afterEach(() => { + cleanupVault(vault); + rmSync(codeRepo, { recursive: true, force: true }); + }); + + function writeConfig(extra = ""): void { + writeFileSync( + join(vault, ".daftari", "config.yaml"), + `code_repos:\n api: ${codeRepo}\n${extra}`, + ); + } + + it("returns null when there are no pinned bindings", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/no-pins.md"), frontmatter(["api:retry.ts"])); + const r = await vaultRead(vault, "engineering/no-pins.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + }); + + it("returns null when code_repos is empty (no config)", async () => { + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter([`api:retry.ts@${sha}`])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + }); + + it("returns null when jit_anchors: false", async () => { + writeConfig("jit_anchors: false\n"); + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter([`api:retry.ts@${sha}`])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + }); + + it("returns null when the configured repo dir does not exist", async () => { + writeFileSync(join(vault, ".daftari", "config.yaml"), "code_repos:\n api: /nowhere/at/all\n"); + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter([`api:retry.ts@${sha}`])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + }); + + it("classifies an intact whole-file pin, no access context (operator posture)", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter([`api:retry.ts@${sha}`])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).not.toBeNull(); + expect(r.value.anchors?.entries).toHaveLength(1); + expect(r.value.anchors?.entries[0]?.state).toBe("intact"); + expect(r.value.anchors?.checked).toBe(1); + expect(r.value.anchors?.skipped).toBe(0); + expect(r.value.anchors?.errored).toBe(0); + expect(r.value.anchors?.banner).toBeNull(); + expectMatchesOutputSchema(readTool, r.value); + }); + + it("reports a drift banner when the pinned blob has moved", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter(["api:retry.ts@0000000"])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors?.entries[0]?.state).toBe("moved"); + expect(r.value.anchors?.banner).toContain("CODE DRIFT"); + expectMatchesOutputSchema(readTool, r.value); + }); + + it("reports missing when the pinned file no longer exists", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/pinned.md"), frontmatter(["api:gone.ts@0000000"])); + const r = await vaultRead(vault, "engineering/pinned.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors?.entries[0]?.state).toBe("missing"); + expect(r.value.anchors?.banner).toContain("CODE DRIFT"); + }); + + it("caps at MAX_PINS_PER_READ (24): 25 pins -> checked 24, skipped 1", async () => { + writeConfig(); + // 25 distinct paths, all missing (cheap: no per-candidate git work needed + // to prove the cap, since missing short-circuits before hashObjects). + const describes = Array.from({ length: 25 }, (_, i) => `api:missing-${i}.ts@0000000`); + writeFileSync(join(vault, "engineering/many-pins.md"), frontmatter(describes)); + const r = await vaultRead(vault, "engineering/many-pins.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors?.checked).toBe(24); + expect(r.value.anchors?.skipped).toBe(1); + expect(r.value.anchors?.entries).toHaveLength(24); + }); + + it("bare (prefix-less) bindings are never JIT-checked even if pin-shaped", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/bare.md"), frontmatter([`retry.ts@${sha}`])); + const r = await vaultRead(vault, "engineering/bare.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + }); + + it("records anchors_moved/anchors_missing/anchors_errored in the read log, uncensored", async () => { + writeConfig(); + writeFileSync( + join(vault, "engineering/mixed.md"), + frontmatter([`api:retry.ts@${sha}`, "api:gone.ts@0000000"]), + ); + await vaultRead(vault, "engineering/mixed.md"); + const log = await readReadLog(vault); + expect(log.ok).toBe(true); + if (!log.ok) return; + const entry = log.value.find((e) => e.file === "engineering/mixed.md"); + expect(entry?.anchors_missing).toBe(1); + expect(entry?.anchors_moved).toBe(0); + expect(entry?.anchors_errored).toBe(0); + }); + + describe("Decision 4 — intact-pin softening of the decay banner", () => { + it("softens a past-TTL banner when every pin is intact", async () => { + writeConfig(); + const doc = frontmatter([`api:retry.ts@${sha}`], { + created: "2020-01-01", + updated: "2020-01-01", + }).replace("tags: []\n", "tags: []\nttl_days: 30\n"); + writeFileSync(join(vault, "engineering/stale-but-intact.md"), doc); + + const r = await vaultRead(vault, "engineering/stale-but-intact.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.decay?.banner).toContain("STALE"); + expect(r.value.decay?.banner).toContain("past TTL, but its 1 code pin"); + expect(r.value.decay?.banner).toContain("has not changed since the pins were written"); + }); + + it("does NOT soften when a pin is moved", async () => { + writeConfig(); + const doc = frontmatter(["api:retry.ts@0000000"], { + created: "2020-01-01", + updated: "2020-01-01", + }).replace("tags: []\n", "tags: []\nttl_days: 30\n"); + writeFileSync(join(vault, "engineering/stale-and-moved.md"), doc); + + const r = await vaultRead(vault, "engineering/stale-and-moved.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.decay?.banner).toContain("STALE"); + expect(r.value.decay?.banner).not.toContain("code pin"); + }); + + it("vault_status's staleness distribution stays byte-identical (computeDecay is untouched)", async () => { + // Spot check: computeDecay's pure output shape is unaffected by pins — + // the level and reasons are identical regardless of anchors, only the + // banner gains an appended line inside vaultRead. + writeConfig(); + const doc = frontmatter([`api:retry.ts@${sha}`], { + created: "2020-01-01", + updated: "2020-01-01", + }).replace("tags: []\n", "tags: []\nttl_days: 30\n"); + writeFileSync(join(vault, "engineering/stale-but-intact-2.md"), doc); + const r = await vaultRead(vault, "engineering/stale-but-intact-2.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.decay?.level).toBe("warn"); + }); + }); + + describe("role gate — code_repo_visibility (2026-07-27 resolution)", () => { + const baseRole = { read: ["engineering"], write: [], promote: false, ratify: false }; + + it("a role WITHOUT code_repo_visibility never sees the anchors field, even though the read succeeds", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/gated.md"), frontmatter([`api:retry.ts@${sha}`])); + const access: AccessContext = { user: "human:analyst", roleName: "analyst", role: baseRole }; + const r = await vaultRead(vault, "engineering/gated.md", access); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); + expectMatchesOutputSchema(readTool, r.value); + }); + + it("a role WITH code_repo_visibility sees the anchors field", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/granted.md"), frontmatter([`api:retry.ts@${sha}`])); + const access: AccessContext = { + user: "human:operator", + roleName: "operator", + role: { ...baseRole, codeRepoVisibility: true }, + }; + const r = await vaultRead(vault, "engineering/granted.md", access); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).not.toBeNull(); + expect(r.value.anchors?.entries[0]?.state).toBe("intact"); + }); + + it("the read log still records the true anchors_* counts for a gated-off role (telemetry is unfiltered)", async () => { + writeConfig(); + writeFileSync(join(vault, "engineering/gated2.md"), frontmatter(["api:gone.ts@0000000"])); + const access: AccessContext = { user: "human:analyst", roleName: "analyst", role: baseRole }; + const r = await vaultRead(vault, "engineering/gated2.md", access); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors).toBeNull(); // caller-facing surface: gated off + + const log = await readReadLog(vault); + if (!log.ok) return; + const entry = log.value.find((e) => e.file === "engineering/gated2.md"); + expect(entry?.anchors_missing).toBe(1); // telemetry: unfiltered + }); + + it("decay softening is also gated off for a role without code_repo_visibility", async () => { + writeConfig(); + const doc = frontmatter([`api:retry.ts@${sha}`], { + created: "2020-01-01", + updated: "2020-01-01", + }).replace("tags: []\n", "tags: []\nttl_days: 30\n"); + writeFileSync(join(vault, "engineering/gated-stale.md"), doc); + const access: AccessContext = { user: "human:analyst", roleName: "analyst", role: baseRole }; + const r = await vaultRead(vault, "engineering/gated-stale.md", access); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.decay?.banner).toContain("STALE"); + expect(r.value.decay?.banner).not.toContain("code pin"); + }); + }); + + describe("errored classification (C8)", () => { + it("a classifier failure is counted in errored and dropped from entries, never softens decay", async () => { + // Force the batch hash-object call to fail on an otherwise-confined, + // stat-visible file by revoking read permission on it: fs.statSync + // (used for confinement) does not require read permission, but `git + // hash-object` opening the file for reading does. + const restrictedRepo = realpathSync( + mkdtempSync(join(tmpdir(), "daftari-anchors-restricted-")), + ); + try { + git(restrictedRepo, ["init", "-q"]); + writeFileSync(join(restrictedRepo, "locked.ts"), "export const x = 1;\n"); + git(restrictedRepo, ["add", "."]); + git(restrictedRepo, ["commit", "-q", "-m", "init"]); + chmodSync(join(restrictedRepo, "locked.ts"), 0o000); + + // Skip this test outright if the sandbox runs as root (chmod 000 is + // then ineffective) — verify the precondition holds before asserting. + const check = await hashObjects(restrictedRepo, ["locked.ts"]); + if (check.ok) return; // running as root or on a fs that ignores perms; nothing to assert + + writeFileSync( + join(vault, ".daftari", "config.yaml"), + `code_repos:\n restricted: ${restrictedRepo}\n`, + ); + writeFileSync( + join(vault, "engineering/errored.md"), + frontmatter(["restricted:locked.ts@0000000"]), + ); + const r = await vaultRead(vault, "engineering/errored.md"); + expect(r.ok).toBe(true); + if (!r.ok) return; + expect(r.value.anchors?.errored).toBe(1); + expect(r.value.anchors?.entries).toHaveLength(0); + expectMatchesOutputSchema(readTool, r.value); + } finally { + chmodSync(join(restrictedRepo, "locked.ts"), 0o644); + rmSync(restrictedRepo, { recursive: true, force: true }); + } + }); + }); +}); diff --git a/test/tools/read.test.ts b/test/tools/read.test.ts index 4378e10b..d19bde80 100644 --- a/test/tools/read.test.ts +++ b/test/tools/read.test.ts @@ -16,10 +16,18 @@ import { vaultStatus, } from "../../src/tools/read.js"; import { sha256Hex } from "../../src/utils/hash.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const VAULT = resolve("test/fixtures/sample-vault"); +const readTool = readTools.find((t) => t.name === "vault_read"); +if (!readTool) throw new Error("vault_read not registered"); +const indexTool = readTools.find((t) => t.name === "vault_index"); +if (!indexTool) throw new Error("vault_index not registered"); +const statusTool = readTools.find((t) => t.name === "vault_status"); +if (!statusTool) throw new Error("vault_status not registered"); + const PRICING_ANALYST: AccessContext = { user: "human:test", roleName: "analyst", @@ -38,6 +46,7 @@ describe("vaultRead", () => { expect(result.value.frontmatter.status).toBe("canonical"); expect(result.value.content).toContain("## Questions Answered"); expect(result.value.validation.valid).toBe(true); + expectMatchesOutputSchema(readTool, result.value); }); it("succeeds on a document with invalid frontmatter, flagging issues", async () => { @@ -177,6 +186,7 @@ describe("vaultIndex", () => { if (!result.ok) return; expect(result.value.count).toBe(10); expect(result.value.entries).toHaveLength(10); + expectMatchesOutputSchema(indexTool, result.value); }); it("filters by collection", async () => { @@ -331,6 +341,7 @@ describe("vaultStatus", () => { expect(counts["competitive-intel"]).toBe(4); expect(counts.pricing).toBe(4); expect(counts.moonshot).toBe(1); + expectMatchesOutputSchema(statusTool, result.value); }); it("reports the count of documents with invalid frontmatter", async () => { @@ -491,7 +502,7 @@ describe("vaultStatus", () => { it("is 0 when all cached embeddings match the active provider dim", async () => { vault = makeTempVault(); - const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const db = dbResult.value; @@ -514,7 +525,7 @@ describe("vaultStatus", () => { it("is non-zero when cached embeddings have the wrong dim for the active model", async () => { vault = makeTempVault(); - const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const db = dbResult.value; diff --git a/test/tools/receipt.test.ts b/test/tools/receipt.test.ts index 8b28a412..3384caf0 100644 --- a/test/tools/receipt.test.ts +++ b/test/tools/receipt.test.ts @@ -6,9 +6,12 @@ import type { AccessContext } from "../../src/access/rbac.js"; import { addTension } from "../../src/curation/tension.js"; import { MAX_RECEIPT_PATHS, receiptTools, vaultReceipt } from "../../src/tools/receipt.js"; import { commit } from "../../src/utils/git.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const TODAY = new Date().toISOString().slice(0, 10); +const receiptTool = receiptTools.find((t) => t.name === "vault_receipt"); +if (!receiptTool) throw new Error("vault_receipt not registered"); function doc( relPath: string, @@ -73,6 +76,7 @@ describe("vaultReceipt", () => { expect(r.summary.flags).toEqual([]); expect(r.summary.byStatus).toEqual({ canonical: 1 }); expect(r.summary.openTensions).toBe(0); + expectMatchesOutputSchema(receiptTool, r); expect(r.summary.oldestUpdated).toBe(TODAY); expect(r.summary.newestUpdated).toBe(TODAY); // The temp vault copy strips .git, so there is no as-of anchor. diff --git a/test/tools/registry.test.ts b/test/tools/registry.test.ts new file mode 100644 index 00000000..9aa9c284 --- /dev/null +++ b/test/tools/registry.test.ts @@ -0,0 +1,141 @@ +// vault_tools + registry module tests (spec 2026-07-26-context-packs- +// progressive-disclosure-design.md, final plan Phase 1.5). + +import { mkdirSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { allRegisteredTools, registeredToolNames, vaultTools } from "../../src/tools/registry.js"; +import { clearConfigCache, configPath } from "../../src/utils/config.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; +import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; + +function writeConfig(vault: string, yaml: string): void { + mkdirSync(join(vault, ".daftari"), { recursive: true }); + writeFileSync(configPath(vault), yaml); + clearConfigCache(); +} + +const ONE_LINE_MAX = 120; +const INDEX_TOKEN_BUDGET = 1500; + +function estimateTokens(s: string): number { + return Math.ceil(s.length / 4); +} + +describe("ToolDefinition.oneLine — every registered tool", () => { + it("every tool has a non-empty oneLine no longer than 120 chars", () => { + for (const t of allRegisteredTools()) { + expect(typeof t.oneLine, `${t.name}.oneLine`).toBe("string"); + expect(t.oneLine.length, `${t.name}.oneLine is empty`).toBeGreaterThan(0); + expect( + t.oneLine.length, + `${t.name}.oneLine exceeds ${ONE_LINE_MAX} chars`, + ).toBeLessThanOrEqual(ONE_LINE_MAX); + } + }); + + it("the whole-registry index payload stays within the ~1,500 token budget", () => { + const vaultTool = allRegisteredTools().find((t) => t.name === "vault_tools"); + expect(vaultTool).toBeTruthy(); + const index = allRegisteredTools() + .map((t) => ({ name: t.name, oneLine: t.oneLine })) + .sort((a, b) => a.name.localeCompare(b.name)); + const payload = JSON.stringify({ mode: "index", count: index.length, tools: index }); + expect(estimateTokens(payload)).toBeLessThanOrEqual(INDEX_TOKEN_BUDGET); + }); +}); + +describe("vault_tools", () => { + let vault: string; + + beforeEach(() => { + vault = makeTempVault(); + clearConfigCache(); + }); + + afterEach(() => { + clearConfigCache(); + cleanupVault(vault); + }); + + const tool = allRegisteredTools().find((t) => t.name === "vault_tools"); + if (!tool) throw new Error("vault_tools not registered"); + + it("index mode (no expand arg) lists every registered tool, sorted by name", async () => { + const result = await vaultTools(vault, {}); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.mode).toBe("index"); + if (result.value.mode !== "index") return; + expect(result.value.count).toBe(registeredToolNames().length); + const names = result.value.tools.map((t) => t.name); + expect(names).toEqual([...names].sort((a, b) => a.localeCompare(b))); + expect(names).toContain("vault_tools"); + expect(names).toContain("vault_context"); + expectMatchesOutputSchema(tool, result.value); + }); + + it("index mode is byte-deterministic across two calls", async () => { + const a = await vaultTools(vault, {}); + const b = await vaultTools(vault, {}); + expect(a.ok && b.ok && JSON.stringify(a.value) === JSON.stringify(b.value)).toBe(true); + }); + + it("expand mode returns full definitions matching the ListTools serialization shape", async () => { + const result = await vaultTools(vault, { expand: ["vault_search"] }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.mode).toBe("expand"); + if (result.value.mode !== "expand") return; + expect(result.value.tools).toHaveLength(1); + expect(result.value.tools[0]?.name).toBe("vault_search"); + expect(result.value.tools[0]?.inputSchema).toBeTruthy(); + expect(result.value.tools[0]?.outputSchema).toBeTruthy(); + expect(result.value.unknown).toEqual([]); + expectMatchesOutputSchema(tool, result.value); + }); + + it("unknown expand names land per-name in `unknown`, never a batch error", async () => { + const result = await vaultTools(vault, { expand: ["vault_search", "vault_does_not_exist"] }); + expect(result.ok).toBe(true); + if (!result.ok) return; + if (result.value.mode !== "expand") throw new Error("expected expand mode"); + expect(result.value.tools.map((t) => t.name)).toEqual(["vault_search"]); + expect(result.value.unknown).toEqual(["vault_does_not_exist"]); + }); + + it("empty exclude yields the full registry in index mode", async () => { + const result = await vaultTools(vault, {}); + expect(result.ok).toBe(true); + if (!result.ok || result.value.mode !== "index") return; + expect(result.value.count).toBe(registeredToolNames().length); + }); + + it("excluded tool is absent from index mode (C2 — exclude always wins)", async () => { + writeConfig(vault, "roles: {}\ntools:\n tier: full\n exclude: [vault_lint]\n"); + const result = await vaultTools(vault, {}); + expect(result.ok).toBe(true); + if (!result.ok || result.value.mode !== "index") return; + expect(result.value.tools.map((t) => t.name)).not.toContain("vault_lint"); + expect(result.value.count).toBe(registeredToolNames().length - 1); + }); + + it("expand of an excluded name lands in `unknown`, identical to unregistered (C2)", async () => { + writeConfig(vault, "roles: {}\ntools:\n tier: full\n exclude: [vault_lint]\n"); + const result = await vaultTools(vault, { expand: ["vault_lint", "vault_search"] }); + expect(result.ok).toBe(true); + if (!result.ok || result.value.mode !== "expand") return; + expect(result.value.tools.map((t) => t.name)).toEqual(["vault_search"]); + expect(result.value.unknown).toEqual(["vault_lint"]); + }); + + it("tier does not affect vault_tools — a full registry with no exclude shows tiered-out tools", async () => { + writeConfig(vault, "roles: {}\ntools:\n tier: core\n"); + const result = await vaultTools(vault, {}); + expect(result.ok).toBe(true); + if (!result.ok || result.value.mode !== "index") return; + // vault_tension_log is full-tier only, but vault_tools shows the whole + // registry regardless of tier — tier and include never affect vault_tools. + expect(result.value.tools.map((t) => t.name)).toContain("vault_tension_log"); + }); +}); diff --git a/test/tools/search-rerank.test.ts b/test/tools/search-rerank.test.ts new file mode 100644 index 00000000..2cdd6b4d --- /dev/null +++ b/test/tools/search-rerank.test.ts @@ -0,0 +1,287 @@ +// Part B rerank pipeline integration (spec 2026-07-26-contextual-chunking- +// reranker-design.md, plan §4.3), exercised through vaultSearch with a FAKE +// RerankProvider (setRerankProviderForTests) — no real model, no network. +// +// Fixture: 60 documents in the `public` collection, all matching the shared +// probe term with varying repetition counts, so pure lexical BM25 (weights: +// {bm25: 1, vector: 0}, which also skips embedQuery entirely — no model +// load) produces a deterministic (if not hand-predictable — BM25 saturates +// term frequency non-linearly against document length) fused order. The +// baseline order is captured EMPIRICALLY in beforeAll (one real vaultSearch +// call with no reranker) rather than assumed from the repetition scheme, so +// the tests below never depend on BM25's internal scoring curve. One more +// document lives in a `secret` collection the test role cannot read, for the +// RBAC-ordering test. + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import type { AccessContext } from "../../src/access/rbac.js"; +import { ok } from "../../src/frontmatter/types.js"; +import { reindexVault } from "../../src/search/reindex.js"; +import type { RerankProvider } from "../../src/search/rerank-provider.js"; +import { + resetRerankProviderForTests, + setRerankProviderForTests, +} from "../../src/search/rerank-provider.js"; +import * as indexDb from "../../src/storage/index-db.js"; +import { vaultSearch } from "../../src/tools/search.js"; + +const QUERY = "zzzrerankprobe"; +const PROBE_COUNT = 60; +const RERANK_POOL = 50; // must match src/tools/search.ts's RERANK_POOL + +function probeBody(repetitions: number): string { + return Array.from({ length: repetitions }, () => QUERY).join(" "); +} + +function probeFrontmatter(title: string, collection: string): string { + return ( + `---\ntitle: "${title}"\ncollection: ${collection}\ndomain: product\n` + + "status: canonical\nconfidence: high\ncreated: 2026-01-01\nupdated: 2026-01-01\n" + + "tags: []\n---\n\n" + ); +} + +const FUSED_WEIGHTS = { bm25: 1, vector: 0 }; // pure lexical — never loads the embedding model + +function publicOnlyAccess(): AccessContext { + return { + user: "t", + roleName: "public-reader", + role: { read: ["public"], write: [], promote: false, ratify: false }, + }; +} + +function fakeProvider(overrides: Partial = {}): RerankProvider { + return { + id: "fake-rerank", + isReady: () => true, + warm: async () => ok(undefined), + rerank: async (_query, passages) => ok(passages.map(() => 0)), + ...overrides, + }; +} + +async function search(args: Record) { + return vaultSearch(vault, { query: QUERY, weights: FUSED_WEIGHTS, ...args }, publicOnlyAccess()); +} + +let vault: string; +// The natural fused order (no reranker), captured once in beforeAll — the +// ground truth every "fused order stands" assertion below compares against, +// instead of a hand-predicted BM25 ranking. +let fusedTop50: string[]; + +describe("vaultSearch — Part B rerank pipeline (fake provider)", () => { + beforeAll(async () => { + vault = mkdtempSync(join(tmpdir(), "daftari-rerank-")); + mkdirSync(join(vault, "public"), { recursive: true }); + mkdirSync(join(vault, "secret"), { recursive: true }); + + for (let i = 0; i < PROBE_COUNT; i++) { + const repetitions = PROBE_COUNT - i; + const name = `probe-${String(i).padStart(3, "0")}`; + writeFileSync( + join(vault, "public", `${name}.md`), + `${probeFrontmatter(name, "public")}${probeBody(repetitions)}\n`, + ); + } + // Matches everyone's repetition count heavily so it would rank at the + // very top if it were visible — the RBAC test's whole point. + writeFileSync( + join(vault, "secret", "hidden.md"), + `${probeFrontmatter("hidden", "secret")}${probeBody(1000)}\n`, + ); + + const reindexed = await reindexVault(vault); + if (!reindexed.ok) throw reindexed.error; + + resetRerankProviderForTests(); // ensure "none" for the baseline capture + const baseline = await search({ limit: RERANK_POOL }); + if (!baseline.ok) throw baseline.error; + fusedTop50 = baseline.value.hits.map((h) => h.path); + if (fusedTop50.length !== RERANK_POOL) { + throw new Error(`expected ${RERANK_POOL} fused hits, got ${fusedTop50.length}`); + } + }, 60_000); + + afterAll(() => { + rmSync(vault, { recursive: true, force: true }); + }); + + afterEach(() => { + resetRerankProviderForTests(); + }); + + it("baseline: the captured fused order is stable across repeated calls", async () => { + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits.map((h) => h.path)).toEqual(fusedTop50.slice(0, 10)); + expect(result.value.rerankUsed).toBe(false); // no reranker configured yet + }); + + // (a) reorder happens AFTER RBAC — a fake that would top-score a forbidden + // doc's passage never gets the chance, because RBAC drops it before the + // rerank stage ever sees it. + it("reorder happens after RBAC: a forbidden doc's passage never reaches the reranker or the hits", async () => { + let sawSecretPassage = false; + setRerankProviderForTests( + fakeProvider({ + rerank: async (_q, passages) => { + if (passages.some((p) => p.includes("hidden"))) sawSecretPassage = true; + return ok(passages.map(() => 0)); + }, + }), + ); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits.some((h) => h.path === "secret/hidden.md")).toBe(false); + expect(sawSecretPassage).toBe(false); + }); + + // (b) reorder happens BEFORE the slice — a hit ranked outside the default + // limit-10 page (fused rank 12, 1-based) can be promoted to #1. + it("reorder happens before the slice: a fused-#12 hit can land #1 in a limit-10 page", async () => { + const promoted = fusedTop50[11]; // 1-based rank 12 + if (!promoted) throw new Error("fixture too small"); + const promotedName = promoted.replace("public/", "").replace(".md", ""); + setRerankProviderForTests( + fakeProvider({ + rerank: async (_q, passages) => ok(passages.map((p) => (p.includes(promotedName) ? 1 : 0))), + }), + ); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits[0]?.path).toBe(promoted); + expect(result.value.hits).toHaveLength(10); + expect(result.value.rerankUsed).toBe(true); + }); + + // (c) a rerank Result.err degrades to the fused order, rerankUsed: false. + it("a Result.err from the provider degrades to the fused order", async () => { + setRerankProviderForTests( + fakeProvider({ + rerank: async () => ({ ok: false, error: new Error("boom") }) as const, + }), + ); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits.map((h) => h.path)).toEqual(fusedTop50.slice(0, 10)); + expect(result.value.rerankUsed).toBe(false); + }); + + // (d) a slow fake exceeding RERANK_TIMEOUT_MS (1500ms, src/tools/search.ts) + // degrades to the fused order exactly like a Result.err. + it("a slow provider exceeding the timeout degrades to the fused order", async () => { + setRerankProviderForTests( + fakeProvider({ + rerank: async (_q, passages) => { + await new Promise((resolve) => setTimeout(resolve, 1700)); + return ok(passages.map(() => 1)); // would reorder everything if it landed + }, + }), + ); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits.map((h) => h.path)).toEqual(fusedTop50.slice(0, 10)); + expect(result.value.rerankUsed).toBe(false); + }, 5_000); + + // (e) a not-ready provider skips reranking for THIS search and fires a + // background warm instead — never a synchronous model load inside the call. + it("a not-ready provider skips reranking and fires a background warm (C5)", async () => { + let warmCalls = 0; + setRerankProviderForTests( + fakeProvider({ + isReady: () => false, + warm: async () => { + warmCalls += 1; + return ok(undefined); + }, + }), + ); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankUsed).toBe(false); + expect(result.value.hits.map((h) => h.path)).toEqual(fusedTop50.slice(0, 10)); + // The background warm is fire-and-forget; give its microtask a tick. + await new Promise((resolve) => setTimeout(resolve, 10)); + expect(warmCalls).toBeGreaterThanOrEqual(1); + }); + + // (f) provider "none" (the default / no provider installed): no ref + // capture, rerankUsed: false, and passageRefs never leaks onto the result. + it("provider none: no ref capture, rerankUsed false, no passageRefs on the result", async () => { + resetRerankProviderForTests(); + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankUsed).toBe(false); + expect("passageRefs" in result.value).toBe(false); + }); + + // (g) the #3 agent-as-judge rerank_candidates pool reflects the RERANKED + // order, not the pre-rerank fused order (spec Decision 7). + it("rerank_candidates draws from the reranked order", async () => { + const promoted = fusedTop50[11]; + if (!promoted) throw new Error("fixture too small"); + const promotedName = promoted.replace("public/", "").replace(".md", ""); + setRerankProviderForTests( + fakeProvider({ + rerank: async (_q, passages) => ok(passages.map((p) => (p.includes(promotedName) ? 1 : 0))), + }), + ); + const result = await search({ limit: 10, rerank_candidates: 5 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerank?.candidates[0]?.path).toBe(promoted); + }); + + // (h) the pool is fixed at RERANK_POOL (50); an all-tied fake (stable sort + // preserves order) never disturbs the top-50 fused order — the closest + // observable proxy for "the tail past the pool keeps the fused order", + // since parseLimit's own max (50) coincides with RERANK_POOL and a caller + // can never request a page wide enough to see past index 49 directly. + it("an all-tied rerank score never disturbs the pool's fused order", async () => { + setRerankProviderForTests(fakeProvider()); // default: score 0 for everyone + const result = await search({ limit: 50 }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.hits).toHaveLength(RERANK_POOL); + expect(result.value.hits.map((h) => h.path)).toEqual(fusedTop50); + }); + + // (i) passage resolution touches only the top-RERANK_POOL permitted pool, + // never the full 60-candidate set — a spy on the batched chunk-text lookup + // sees a single call sized at most RERANK_POOL (C2). + it("passage resolution touches only the top-50 permitted pool, not the full candidate set", async () => { + setRerankProviderForTests(fakeProvider()); + const rowidsSpy = vi.spyOn(indexDb, "getChunkTextsByRowids"); + const hashSpy = vi.spyOn(indexDb, "getChunkByPathAndHash"); + const firstSpy = vi.spyOn(indexDb, "getFirstChunk"); + try { + const result = await search({ limit: 10 }); + expect(result.ok).toBe(true); + // Pure-lexical, chunk-mode, every doc matched by BM25 (no title/tag + // fallback and no vector signal at all with vector:0) => every ref is + // "lexical", resolved in exactly one batched call sized at RERANK_POOL. + expect(rowidsSpy).toHaveBeenCalledTimes(1); + const rowids = rowidsSpy.mock.calls[0]?.[1] ?? []; + expect(rowids.length).toBeLessThanOrEqual(RERANK_POOL); + expect(hashSpy).not.toHaveBeenCalled(); + expect(firstSpy).not.toHaveBeenCalled(); + } finally { + rowidsSpy.mockRestore(); + hashSpy.mockRestore(); + firstSpy.mockRestore(); + } + }); +}); diff --git a/test/tools/search.test.ts b/test/tools/search.test.ts index f23b4368..626c36f8 100644 --- a/test/tools/search.test.ts +++ b/test/tools/search.test.ts @@ -1,18 +1,29 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest"; +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; import type { AccessContext } from "../../src/access/rbac.js"; import { mintConsumesEdges } from "../../src/curation/consumes.js"; import { recordProvenance } from "../../src/curation/provenance.js"; import { readReadLog, recordRead } from "../../src/curation/read-log.js"; import { addTension, tensionsPath } from "../../src/curation/tension.js"; +import { err } from "../../src/frontmatter/types.js"; import { clearContestedCache } from "../../src/search/contested.js"; -import { vaultReindex, vaultSearch, vaultSearchRelated } from "../../src/tools/search.js"; +import * as vectorMod from "../../src/search/vector.js"; +import { + searchTools, + vaultReindex, + vaultSearch, + vaultSearchRelated, +} from "../../src/tools/search.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { clearConfigCache, configPath } from "../../src/utils/config.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const INSIGHT_DOC = "competitive-intel/vega-insight-positioning.md"; +const reindexTool = searchTools.find((t) => t.name === "vault_reindex"); +if (!reindexTool) throw new Error("vault_reindex not registered"); describe("search tools", () => { let vault: string; @@ -36,6 +47,7 @@ describe("search tools", () => { if (!result.ok) return; expect(result.value.documentCount).toBe(10); expect(result.value.vault).toBe(vault); + expectMatchesOutputSchema(reindexTool, result.value); }); }); @@ -48,6 +60,9 @@ describe("search tools", () => { if (!result.ok) return; expect(result.value.hits.length).toBeGreaterThan(0); expect(result.value.hits[0]?.path).toBe("pricing/helios-consumption-pricing.md"); + const searchTool = searchTools.find((t) => t.name === "vault_search"); + if (!searchTool) throw new Error("vault_search not registered"); + expectMatchesOutputSchema(searchTool, result.value); }); it("rejects a missing or empty query", async () => { @@ -746,3 +761,148 @@ describe("FTS5 lexical snippets (#108)", () => { expect(hit?.snippet).not.toContain("Filler paragraph 0"); }); }); + +// --------------------------------------------------------------------------- +// vault_search — query router wiring (spec 2026-07-26 fusion overhaul, +// Decision 2). router.test.ts covers classifyQuery/routeWeights/ +// makeDfLookup in isolation, including the rare-term signal's +// MIN_DOCS_FOR_RARE guard (which needs >= 100 documents — impractical to +// stand up here); these tests cover the WIRING in vaultSearch: config +// precedence, the `routed` field's presence/absence, and the +// distinguishability of a routed lexical-only result from an embedding +// degrade. A digit-heavy query exercises the "lexical" route cheaply +// (no MIN_DOCS_FOR_RARE dependency) to prove the same wiring path. +// --------------------------------------------------------------------------- +describe("vault_search — query router wiring (Decision 2)", () => { + let routedVault: string; + const DIGIT_HEAVY_QUERY = "PR 2026 release notes"; + const QUOTED_QUERY = `"exact phrase example" release notes`; + + beforeAll(async () => { + routedVault = makeTempVault(); + const result = await vaultReindex(routedVault); + if (!result.ok) throw result.error; + }, 60_000); + + afterAll(() => { + cleanupVault(routedVault); + }); + + function writeRoutingConfig(on: boolean): void { + mkdirSync(join(routedVault, ".daftari"), { recursive: true }); + writeFileSync(configPath(routedVault), `search:\n routing: ${on}\n`); + } + + afterEach(() => { + rmSync(configPath(routedVault), { force: true }); + clearConfigCache(); + vi.restoreAllMocks(); + }); + + it("explicit valid weights beat routing even when routing is on", async () => { + writeRoutingConfig(true); + const result = await vaultSearch(routedVault, { + query: DIGIT_HEAVY_QUERY, // would otherwise route to lexical {0.8, 0.2} + weights: { bm25: 0.9, vector: 0.1 }, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.vectorUsed).toBe(true); + expect(result.value.weights).toEqual({ bm25: 0.9, vector: 0.1 }); + expect(result.value.routed).toBeUndefined(); + }); + + it("an invalid weights arg yields static DEFAULT_WEIGHTS and no routed field, even with routing on", async () => { + writeRoutingConfig(true); + const result = await vaultSearch(routedVault, { + query: DIGIT_HEAVY_QUERY, + weights: { bm25: "nope" }, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.vectorUsed).toBe(true); + expect(result.value.weights).toEqual({ bm25: 0.5, vector: 0.5 }); + expect(result.value.routed).toBeUndefined(); + }); + + it("routing on + a lexical-signal query echoes {bm25: 0.8, vector: 0.2} with a routed field", async () => { + writeRoutingConfig(true); + const result = await vaultSearch(routedVault, { query: DIGIT_HEAVY_QUERY }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.routed).toBeDefined(); + expect(result.value.routed?.class).toBe("lexical"); + expect(result.value.routed?.signals).toContain("digit-heavy"); + expect(result.value.vectorUsed).toBe(true); + expect(result.value.weights).toEqual({ bm25: 0.8, vector: 0.2 }); + + const searchTool = searchTools.find((t) => t.name === "vault_search"); + if (!searchTool) throw new Error("vault_search not registered"); + expectMatchesOutputSchema(searchTool, result.value); + }); + + it("routing off restores the static 0.5/0.5 default with no routed field", async () => { + writeRoutingConfig(false); + const result = await vaultSearch(routedVault, { query: DIGIT_HEAVY_QUERY }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.routed).toBeUndefined(); + expect(result.value.vectorUsed).toBe(true); + expect(result.value.weights).toEqual({ bm25: 0.5, vector: 0.5 }); + }); + + it("no config at all behaves the same as routing off", async () => { + const result = await vaultSearch(routedVault, { query: DIGIT_HEAVY_QUERY }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.routed).toBeUndefined(); + expect(result.value.weights).toEqual({ bm25: 0.5, vector: 0.5 }); + }); + + it("a routed extreme-lexical result reports vectorUsed:false WITH routed present — distinguishable from an embedding degrade", async () => { + writeRoutingConfig(true); + const result = await vaultSearch(routedVault, { query: QUOTED_QUERY }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.vectorUsed).toBe(false); + expect(result.value.weights).toEqual({ bm25: 1, vector: 0 }); + expect(result.value.routed).toBeDefined(); + expect(result.value.routed?.class).toBe("extreme-lexical"); + expect(result.value.routed?.signals).toContain("quoted-phrase"); + }); + + it("an embedding-provider degrade also reports vectorUsed:false, but routed stays ABSENT (routing off)", async () => { + writeRoutingConfig(false); + vi.spyOn(vectorMod, "embedQuery").mockResolvedValue( + err(new Error("embedding provider unavailable")), + ); + const result = await vaultSearch(routedVault, { query: "pricing" }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.vectorUsed).toBe(false); + expect(result.value.weights).toEqual({ bm25: 1, vector: 0 }); + // The distinguishing assertion: unlike the routed extreme-lexical case + // above, `routed` is absent here — this is embedding degradation, not a + // router decision. + expect(result.value.routed).toBeUndefined(); + }); + + it("an embedding-provider degrade with routing ON still leaves routed absent when the router itself picked balanced/lexical weights that needed embedding", async () => { + // Routing on, a query with no extreme/lexical signal → router picks + // balanced (0.5/0.5, vector > 0) → hybridSearch attempts to embed → + // embedding fails → degrade rewrites weights to {1, 0}. `routed` is + // still attached (the router DID pick the weights hybridSearch started + // with), even though the actually-used weights differ after degrade. + writeRoutingConfig(true); + vi.spyOn(vectorMod, "embedQuery").mockResolvedValue( + err(new Error("embedding provider unavailable")), + ); + const result = await vaultSearch(routedVault, { query: "how do write locks expire" }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.vectorUsed).toBe(false); + expect(result.value.weights).toEqual({ bm25: 1, vector: 0 }); + expect(result.value.routed).toBeDefined(); + expect(result.value.routed?.class).toBe("balanced"); + }); +}); diff --git a/test/tools/staged-actions.test.ts b/test/tools/staged-actions.test.ts index 4c1c6278..2a704a2d 100644 --- a/test/tools/staged-actions.test.ts +++ b/test/tools/staged-actions.test.ts @@ -2,16 +2,27 @@ import { mkdirSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { readProvenanceLog } from "../../src/curation/provenance.js"; +import { BATCH_RATIFY_MAX } from "../../src/curation/risk.js"; import { getStagedActionById, stageAction } from "../../src/curation/staged-actions.js"; import { listTensions } from "../../src/curation/tension.js"; import { vaultRead } from "../../src/tools/read.js"; -import { vaultRatify, vaultStageAction } from "../../src/tools/staged-actions.js"; +import { + stagedActionTools, + vaultRatify, + vaultStageAction, +} from "../../src/tools/staged-actions.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:curation-loop"; const HUMAN = "human:mihir"; +const stageActionTool = stagedActionTools.find((t) => t.name === "vault_stage_action"); +if (!stageActionTool) throw new Error("vault_stage_action not registered"); +const ratifyTool = stagedActionTools.find((t) => t.name === "vault_ratify"); +if (!ratifyTool) throw new Error("vault_ratify not registered"); + function draftFrontmatter(overrides: Record = {}) { return { title: "Federation Spec", @@ -65,6 +76,7 @@ describe("vault_stage_action", () => { if (!result.ok) return; expect(result.value.id).toBe("stage-001"); expect(result.value.expires_at).toMatch(/^\d{4}-\d{2}-\d{2}T/); + expectMatchesOutputSchema(stageActionTool, result.value); }, 60_000); it("denies a role that lacks write access to the target collection", async () => { @@ -190,6 +202,49 @@ describe("vault_stage_action", () => { }); expect(result.ok).toBe(false); }); + + it("records the authenticated caller as staged_by_principal (C4)", async () => { + await seedDraft(vault, "pricing/foo.md"); + const access = { + user: "human:mihir", + roleName: "curator", + role: { read: ["*"], write: ["*"], promote: true, ratify: true }, + }; + const result = await vaultStageAction( + vault, + { + action_type: "promote", + target_path: "pricing/foo.md", + proposed_by: AGENT, + rationale: "Matured.", + proposed_diff: {}, + }, + access, + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + const staged = await getStagedActionById(vault, result.value.id); + expect(staged.ok).toBe(true); + if (!staged.ok || !staged.value) return; + expect(staged.value.stagedByPrincipal).toBe("human:mihir"); + // proposed_by remains the claimed-agent display string, untouched. + expect(staged.value.proposedBy).toBe(AGENT); + }); + + it("omits staged_by_principal when there is no access context (operator use)", async () => { + await seedDraft(vault, "pricing/foo.md"); + const result = await vaultStageAction(vault, { + action_type: "promote", + target_path: "pricing/foo.md", + proposed_by: AGENT, + rationale: "Matured.", + proposed_diff: {}, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + const staged = await getStagedActionById(vault, result.value.id); + expect(staged.ok && staged.value?.stagedByPrincipal).toBeNull(); + }); }); describe("vault_ratify", () => { @@ -222,6 +277,7 @@ describe("vault_ratify", () => { if (!ratified.ok) return; expect(ratified.value.applied).toBe(true); expect(ratified.value.commit).toMatch(/^[0-9a-f]+$/); + expectMatchesOutputSchema(ratifyTool, ratified.value); // The document is now canonical. const read = await vaultRead(vault, "pricing/federation.md"); @@ -247,13 +303,61 @@ describe("vault_ratify", () => { decision: "reject", principal: HUMAN, reason: "not ready", + reason_category: "stale-evidence", }); expect(result.ok).toBe(true); if (!result.ok) return; expect(result.value.applied).toBe(false); + expect((result.value as { decision_kind?: string }).decision_kind).toBe("reject"); const action = await getStagedActionById(vault, staged.value.id); expect(action.ok && action.value?.status).toBe("rejected"); + expect(action.ok && action.value?.reasonCategory).toBe("stale-evidence"); + }); + + it("errors on reject without a reason_category, enumerating the categories (C6)", async () => { + const staged = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/federation.md", + proposedBy: AGENT, + rationale: "Matured.", + proposedDiff: {}, + }); + if (!staged.ok) return; + + const result = await vaultRatify(vault, { + id: staged.value.id, + decision: "reject", + principal: HUMAN, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("reason_category"); + expect(result.error.message).toContain("wrong-conclusion"); + expect(result.error.message).toContain("other"); + + // The action is untouched — still pending. + const action = await getStagedActionById(vault, staged.value.id); + expect(action.ok && action.value?.status).toBe("pending"); + }); + + it("rejects an unknown reason_category", async () => { + const staged = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/federation.md", + proposedBy: AGENT, + rationale: "Matured.", + proposedDiff: {}, + }); + if (!staged.ok) return; + + const result = await vaultRatify(vault, { + id: staged.value.id, + decision: "reject", + principal: HUMAN, + reason_category: "not-a-real-category", + }); + expect(result.ok).toBe(false); }); it("approves a supersede: dispatches vault_supersede, marks ratified (§11.4)", async () => { @@ -463,7 +567,12 @@ describe("vault_ratify", () => { const result = await vaultRatify( vault, - { id: staged.value.id, decision: "reject", principal: "human:mihir" }, + { + id: staged.value.id, + decision: "reject", + principal: "human:mihir", + reason_category: "wrong-conclusion", + }, access, ); expect(result.ok).toBe(true); @@ -1013,7 +1122,12 @@ describe("vault_ratify", () => { proposedDiff: {}, }); if (!staged.ok) return; - await vaultRatify(vault, { id: staged.value.id, decision: "reject", principal: HUMAN }); + await vaultRatify(vault, { + id: staged.value.id, + decision: "reject", + principal: HUMAN, + reason_category: "wrong-conclusion", + }); const again = await vaultRatify(vault, { id: staged.value.id, decision: "approve", @@ -1021,4 +1135,363 @@ describe("vault_ratify", () => { }); expect(again.ok).toBe(false); }); + + // 2026-07-26 risk-triaged-ratification spec, Decision 3: edit-then-approve. + describe("edit-then-approve (amended_diff)", () => { + it("dispatches the amendment instead of the staged diff; the decision record keeps both", async () => { + const staged = await vaultStageAction(vault, { + action_type: "write", + target_path: "pricing/edited-analysis.md", + proposed_by: AGENT, + rationale: "Initial draft synthesis.", + proposed_diff: { + frontmatter: draftFrontmatter({ title: "Original" }), + body: "# Original\n\nOriginal content.\n", + }, + }); + expect(staged.ok).toBe(true); + if (!staged.ok) throw staged.error; + + const amendedDiff = { + frontmatter: draftFrontmatter({ title: "Amended" }), + body: "# Amended\n\nCorrected content.\n", + }; + const ratified = await vaultRatify(vault, { + id: staged.value.id, + decision: "approve", + principal: HUMAN, + reason_category: "wrong-conclusion", + amended_diff: amendedDiff, + }); + expect(ratified.ok).toBe(true); + if (!ratified.ok) throw ratified.error; + expect((ratified.value as { decision_kind?: string }).decision_kind).toBe( + "edit-then-approve", + ); + expectMatchesOutputSchema(ratifyTool, ratified.value); + + const read = await vaultRead(vault, "pricing/edited-analysis.md"); + expect(read.ok && read.value.content).toContain("Corrected content."); + expect(read.ok && read.value.content).not.toContain("Original content."); + + const action = await getStagedActionById(vault, staged.value.id); + expect(action.ok && action.value?.status).toBe("ratified"); + expect(action.ok && action.value?.decisionKind).toBe("edit-then-approve"); + expect(action.ok && action.value?.amendedDiff).toEqual(amendedDiff); + // proposedDiff still records the ORIGINAL proposal — history preserved. + const proposed = action.ok + ? (action.value?.proposedDiff as { frontmatter?: { title?: string } }) + : undefined; + expect(proposed?.frontmatter?.title).toBe("Original"); + }, 60_000); + + it("amended_diff requires reason_category (C6)", async () => { + const staged = await vaultStageAction(vault, { + action_type: "write", + target_path: "pricing/edited-2.md", + proposed_by: AGENT, + rationale: "Initial.", + proposed_diff: { frontmatter: draftFrontmatter(), body: "# X\n" }, + }); + if (!staged.ok) throw staged.error; + const result = await vaultRatify(vault, { + id: staged.value.id, + decision: "approve", + principal: HUMAN, + amended_diff: { frontmatter: draftFrontmatter(), body: "# Y\n" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("reason_category"); + }); + + it("an amended write payload still hits the tier-0 canonical-write gate", async () => { + await seedDraft(vault, "pricing/wip-source.md"); + const staged = await vaultStageAction(vault, { + action_type: "write", + target_path: "pricing/amend-target.md", + proposed_by: AGENT, + rationale: "Innocuous draft proposal.", + proposed_diff: { frontmatter: draftFrontmatter({ title: "Draft" }), body: "# Draft\n" }, + }); + if (!staged.ok) throw staged.error; + + const result = await vaultRatify(vault, { + id: staged.value.id, + decision: "approve", + principal: HUMAN, + reason_category: "overbroad", + amended_diff: { + frontmatter: draftFrontmatter({ + title: "Bold Claim", + status: "canonical", + sources: ["pricing/wip-source.md"], + }), + body: "# Bold Claim\n", + }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("tier-0 gate blocked canonical write"); + + const action = await getStagedActionById(vault, staged.value.id); + expect(action.ok && action.value?.status).toBe("pending"); + }, 60_000); + + it("errors under shadow_mode instead of discarding the amendment (C7)", async () => { + await seedDraft(vault, "pricing/shadow-target.md"); + const staged = await stageAction(vault, { + actionType: "confidence-up", + targetPath: "pricing/shadow-target.md", + proposedBy: AGENT, + rationale: "Survived re-derivation.", + proposedDiff: { confidence: "high" }, + }); + if (!staged.ok) throw staged.error; + + mkdirSync(join(vault, ".daftari"), { recursive: true }); + writeFileSync(join(vault, ".daftari", "config.yaml"), "version: 1\nshadow_mode: true\n"); + + const result = await vaultRatify(vault, { + id: staged.value.id, + decision: "approve", + principal: HUMAN, + reason_category: "wrong-conclusion", + amended_diff: { confidence: "medium" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("shadow mode"); + + const action = await getStagedActionById(vault, staged.value.id); + expect(action.ok && action.value?.status).toBe("pending"); + expect(action.ok && action.value?.decisionKind).toBeNull(); + expect(action.ok && action.value?.ratifiedAt).toBeNull(); + + // The plain-approve shadow path is unaffected: no amended_diff, no error. + const plain = await vaultRatify(vault, { + id: staged.value.id, + decision: "approve", + principal: HUMAN, + }); + expect(plain.ok).toBe(true); + if (!plain.ok) return; + expect((plain.value as { shadow?: boolean }).shadow).toBe(true); + }); + }); + + // Decision 2: batch ratify — an explicit id list, never a threshold. + describe("batch ratify (ids)", () => { + it("processes ids independently: a gate-blocked id stays pending, the rest land", async () => { + await seedDraft(vault, "pricing/batch-a.md"); + await seedDraft(vault, "pricing/batch-b.md"); + await seedDraft(vault, "pricing/wip.md"); + await seedDraft(vault, "pricing/batch-c.md", { sources: ["pricing/wip.md"] }); + + const a = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/batch-a.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + const b = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/batch-b.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + const c = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/batch-c.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + if (!a.ok || !b.ok || !c.ok) throw new Error("staging failed"); + + const result = await vaultRatify(vault, { + ids: [a.value.id, b.value.id, c.value.id], + decision: "approve", + principal: HUMAN, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expectMatchesOutputSchema(ratifyTool, result.value); + const batch = result.value as { + decision: string; + results: Array<{ action_id: string; ok: boolean; applied: boolean; error?: string }>; + succeeded: number; + failed: number; + }; + expect(batch.succeeded).toBe(2); + expect(batch.failed).toBe(1); + const cOutcome = batch.results.find((r) => r.action_id === c.value.id); + expect(cOutcome?.ok).toBe(false); + expect(cOutcome?.error).toContain("tier-0 gate blocked promote"); + + const aAction = await getStagedActionById(vault, a.value.id); + expect(aAction.ok && aAction.value?.status).toBe("ratified"); + const cAction = await getStagedActionById(vault, c.value.id); + expect(cAction.ok && cAction.value?.status).toBe("pending"); + }, 60_000); + + it("batch reject: one shared reason_category applies to every id", async () => { + await seedDraft(vault, "pricing/rej-a.md"); + await seedDraft(vault, "pricing/rej-b.md"); + const a = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/rej-a.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + const b = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/rej-b.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + if (!a.ok || !b.ok) throw new Error("staging failed"); + + const result = await vaultRatify(vault, { + ids: [a.value.id, b.value.id], + decision: "reject", + principal: HUMAN, + reason_category: "duplicate", + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + const batch = result.value as { succeeded: number; failed: number }; + expect(batch.succeeded).toBe(2); + expect(batch.failed).toBe(0); + + const aAction = await getStagedActionById(vault, a.value.id); + expect(aAction.ok && aAction.value?.reasonCategory).toBe("duplicate"); + const bAction = await getStagedActionById(vault, b.value.id); + expect(bAction.ok && bAction.value?.reasonCategory).toBe("duplicate"); + }); + + it("interrupted-batch recovery: re-issuing pins landed ids as not-pending and applies the rest", async () => { + await seedDraft(vault, "pricing/rec-a.md"); + await seedDraft(vault, "pricing/rec-b.md"); + await seedDraft(vault, "pricing/rec-c.md"); + const a = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/rec-a.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + const b = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/rec-b.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + const c = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/rec-c.md", + proposedBy: AGENT, + rationale: "r", + proposedDiff: {}, + }); + if (!a.ok || !b.ok || !c.ok) throw new Error("staging failed"); + + // Simulate "already landed before the interruption": decide `a` out of + // band, as if an earlier call to this same batch had already processed it. + const preDecided = await vaultRatify(vault, { + id: a.value.id, + decision: "approve", + principal: HUMAN, + }); + expect(preDecided.ok).toBe(true); + + // Re-issue the FULL original batch — the documented recovery path. + const result = await vaultRatify(vault, { + ids: [a.value.id, b.value.id, c.value.id], + decision: "approve", + principal: HUMAN, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + const batch = result.value as { + results: Array<{ action_id: string; ok: boolean; error?: string }>; + succeeded: number; + failed: number; + }; + expect(batch.failed).toBe(1); + expect(batch.succeeded).toBe(2); + const aOutcome = batch.results.find((r) => r.action_id === a.value.id); + expect(aOutcome?.ok).toBe(false); + expect(aOutcome?.error).toContain("not 'pending'"); + }, 60_000); + + it("caps the batch at BATCH_RATIFY_MAX ids", async () => { + const ids = Array.from({ length: BATCH_RATIFY_MAX + 1 }, (_, i) => `stage-${i}`); + const result = await vaultRatify(vault, { ids, decision: "approve", principal: HUMAN }); + expect(result.ok).toBe(false); + }); + + it("rejects an empty ids array", async () => { + const result = await vaultRatify(vault, { ids: [], decision: "approve", principal: HUMAN }); + expect(result.ok).toBe(false); + }); + + it("rejects a duplicate id within ids", async () => { + const result = await vaultRatify(vault, { + ids: ["stage-001", "stage-001"], + decision: "approve", + principal: HUMAN, + }); + expect(result.ok).toBe(false); + }); + + it("errors when both id and ids are supplied", async () => { + const result = await vaultRatify(vault, { + id: "stage-001", + ids: ["stage-002"], + decision: "approve", + principal: HUMAN, + }); + expect(result.ok).toBe(false); + }); + + it("errors when neither id nor ids is supplied", async () => { + const result = await vaultRatify(vault, { decision: "approve", principal: HUMAN }); + expect(result.ok).toBe(false); + }); + + it("errors when amended_diff is combined with ids — single-id only", async () => { + const result = await vaultRatify(vault, { + ids: ["stage-001", "stage-002"], + decision: "approve", + principal: HUMAN, + reason_category: "overbroad", + amended_diff: { confidence: "high" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("single-id only"); + }); + + it("denies a batch under a propose-only role", async () => { + const proposeOnly = { + user: "agent:proposer", + roleName: "proposer", + role: { read: ["*"], write: ["*"], promote: false, ratify: true, proposeOnly: true }, + }; + const result = await vaultRatify( + vault, + { ids: ["stage-001", "stage-002"], decision: "approve", principal: HUMAN }, + proposeOnly, + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("propose-only"); + }); + }); }); diff --git a/test/tools/summarizers.test.ts b/test/tools/summarizers.test.ts new file mode 100644 index 00000000..79af018a --- /dev/null +++ b/test/tools/summarizers.test.ts @@ -0,0 +1,509 @@ +// Degenerate-value coverage for every tool's `summarize`/`docLinks` (spec +// 2026-07-26, Decision 3, PR 1 gap closure, jugalbandi challenge C5). +// +// C5's finding: every summarizer is an unchecked cast, and the shipped test +// plan only exercised happy paths — a summarizer that indexes into an empty +// array, or reads a field only present in the non-degenerate branch, throws +// on a legal-but-sparse result and (pre-hardening) would have turned a +// successful tool call into an error response. These tests build the +// smallest legal value each tool's Result type allows — zero counts, empty +// arrays, null banners, coarsened "none"/"many" buckets — and assert the +// summarizer neither throws nor sharpens a coarsened disclosure, and that +// docLinks (where present) never re-derives a path absent from the value. +// +// Values are hand-built to the TypeScript result shape (not run through a +// live vault), so most also validate against the tool's own outputSchema — +// asserted here too, which additionally catches schema/shape drift. + +import { describe, expect, it } from "vitest"; +import { allRegisteredTools } from "../../src/server.js"; +import type { ToolDefinition } from "../../src/tools/read.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; + +function tool(name: string): ToolDefinition { + const t = allRegisteredTools().find((x) => x.name === name); + if (!t) throw new Error(`no such registered tool: ${name}`); + return t; +} + +// Runs `summarize`/`docLinks` against a value and asserts neither throws, +// summarize returns a non-empty string, and docLinks (if present) returns an +// array of strings. Optionally also pins the value against outputSchema. +function expectSummarizesCleanly( + name: string, + value: unknown, + opts: { checkSchema?: boolean } = {}, +): void { + const t = tool(name); + if (opts.checkSchema !== false) expectMatchesOutputSchema(t, value); + expect(t.summarize, `${name} has no summarize`).toBeTruthy(); + let summary = ""; + expect(() => { + summary = (t.summarize as (v: unknown) => string)(value); + }, `${name}.summarize threw on a degenerate value`).not.toThrow(); + expect(typeof summary).toBe("string"); + expect(summary.length).toBeGreaterThan(0); + if (t.docLinks) { + let links: string[] = []; + expect(() => { + links = (t.docLinks as (v: unknown) => string[])(value); + }, `${name}.docLinks threw on a degenerate value`).not.toThrow(); + expect(Array.isArray(links)).toBe(true); + for (const l of links) expect(typeof l).toBe("string"); + } +} + +const validation = { valid: true, issues: [] }; + +describe("read.ts summarizers — degenerate values", () => { + it("vault_read: no decay/validity/upstream/structural, no contested", () => { + expectSummarizesCleanly("vault_read", { + path: "a.md", + content: "", + frontmatter: { + title: "t", + domain: "accumulation", + collection: "c", + status: "draft", + confidence: "low", + created: "2026-01-01", + updated: "2026-01-01", + updated_by: "agent:x", + provenance: "direct", + tier: null, + sources: [], + superseded_by: null, + ttl_days: null, + tags: [], + describes: [], + questions_answered: [], + questions_raised: [], + }, + raw: {}, + validation, + hasFrontmatter: true, + decay: null, + validity: null, + upstream_staleness: null, + structural: null, + anchors: null, + version: "abc123", + }); + }); + + it("vault_index: zero entries", () => { + expectSummarizesCleanly("vault_index", { count: 0, entries: [] }); + }); + + it("vault_status: an empty vault", () => { + expectSummarizesCleanly("vault_status", { + vault: "/v", + fileCount: 0, + collections: [], + invalidCount: 0, + generatedAt: "2026-01-01T00:00:00Z", + stalenessDistribution: { fresh: 0, aging: 0, stale: 0, total: 0 }, + validityCoverage: { authored: 0, unknown: 0, total: 0 }, + unresolvedTensions: { count: 0, recent: [] }, + recentWrites: { count: 0, entries: [] }, + embeddingDimMismatches: 0, + }); + }); +}); + +describe("search.ts summarizers — degenerate values", () => { + it("vault_reindex: an empty vault, no warnings", () => { + expectSummarizesCleanly("vault_reindex", { + vault: "/v", + documentCount: 0, + chunkCount: 0, + vectorEnabled: false, + skipped: [], + invalidFrontmatter: [], + indexedAt: "2026-01-01T00:00:00Z", + embeddedCount: 0, + cacheHits: 0, + orphansRemoved: 0, + }); + }); +}); + +describe("write.ts summarizers — degenerate values", () => { + const base = { + path: "a.md", + commit: null, + committed: false, + status: "draft", + updated: "2026-01-01", + validation, + indexUpdated: false, + }; + + it("vault_write: a plain applied write, no advisory fields", () => { + expectSummarizesCleanly("vault_write", { ...base, action: "update" }); + }); + + it("vault_write: a staged proposal, uncontested", () => { + expectSummarizesCleanly("vault_write", { + ...base, + action: "staged", + status: "pending", + staged_id: "stage-1", + expires_at: "2026-01-08T00:00:00Z", + conflicts_with: [], + tension_id: null, + }); + }); + + it("vault_write: a staged proposal, contested", () => { + expectSummarizesCleanly("vault_write", { + ...base, + action: "staged", + status: "pending", + staged_id: "stage-1", + expires_at: "2026-01-08T00:00:00Z", + conflicts_with: ["stage-2", "stage-3"], + tension_id: "tension-1", + }); + }); + + it("vault_append: a shadow-mode write", () => { + expectSummarizesCleanly("vault_append", { ...base, action: "append", shadow: true }); + }); + + it("vault_merge: sources present — docLinks names both", () => { + const value = { ...base, action: "merge", sources: ["a.md", "b.md"] }; + expectSummarizesCleanly("vault_merge", value); + expect(tool("vault_merge").docLinks?.(value)).toEqual(["a.md", "a.md", "b.md"]); + }); +}); + +describe("staged-actions.ts summarizers — degenerate values", () => { + it("vault_stage_action: uncontested, no tension", () => { + expectSummarizesCleanly("vault_stage_action", { + id: "stage-1", + expires_at: "2026-01-08T00:00:00Z", + conflicts_with: [], + tension_id: null, + }); + }); + + it("vault_ratify: rejected", () => { + expectSummarizesCleanly("vault_ratify", { + action_id: "stage-1", + decision: "reject", + applied: false, + }); + }); + + it("vault_ratify: shadow-applied", () => { + expectSummarizesCleanly("vault_ratify", { + action_id: "stage-1", + decision: "approve", + applied: false, + shadow: true, + }); + }); +}); + +describe("curation.ts summarizers — degenerate values", () => { + it("vault_tension_log: a legacy entry with no id", () => { + const value = { + date: "2026-01-01", + title: "t", + kind: "factual", + sourceA: "a.md", + claimA: "x", + sourceB: "b.md", + claimB: "y", + status: "unresolved", + loggedBy: "agent:x", + resolved: false, + }; + expectSummarizesCleanly("vault_tension_log", value, { checkSchema: false }); + expect(tool("vault_tension_log").docLinks?.(value)).toEqual(["a.md", "b.md"]); + }); + + it("vault_tension_resolve: a resolved entry with an id", () => { + expectSummarizesCleanly( + "vault_tension_resolve", + { + id: "tension-1", + date: "2026-01-01", + title: "t", + kind: "factual", + sourceA: "a.md", + claimA: "x", + sourceB: "b.md", + claimB: "y", + status: "resolved", + loggedBy: "agent:x", + resolved: true, + }, + { checkSchema: false }, + ); + }); + + it("vault_tension_clusters: zero clusters", () => { + expectSummarizesCleanly("vault_tension_clusters", { cluster_count: 0, clusters: [] }); + }); + + it("vault_tension_blast: no downstream, hidden bucket 'none'", () => { + expectSummarizesCleanly("vault_tension_blast", { + contested_document: "a.md", + cluster_id: null, + cluster_documents: [], + downstream: [], + primary_blast: 0, + advisory_blast: 0, + max_depth: 0, + hidden_downstream: "none", + }); + }); + + it("vault_tension_blast: all-hidden — 'many' bucket, still zero visible downstream", () => { + const value = { + contested_document: "a.md", + cluster_id: null, + cluster_documents: [], + downstream: [], + primary_blast: 0, + advisory_blast: 0, + max_depth: 0, + hidden_downstream: "many", + }; + expectSummarizesCleanly("vault_tension_blast", value); + const summary = tool("vault_tension_blast").summarize?.(value) ?? ""; + // The coarsened bucket must appear VERBATIM, never sharpened to a number. + expect(summary).toContain("hidden: many"); + }); + + it("vault_provenance: no history", () => { + expectSummarizesCleanly("vault_provenance", { path: "a.md", count: 0, history: [] }); + }); +}); + +describe("edges.ts summarizers — degenerate values", () => { + const edge = { + fromPath: "a.md", + toPath: "b.md", + strength: 0, + kSurvived: 0, + kEff: 0, + strengthIndependent: 0, + firstObserved: "2026-01-01T00:00:00Z", + lastRederived: "2026-01-01T00:00:00Z", + status: "candidate", + directionVerdict: "directed", + observations: 0, + contestedAt: null, + contestReason: null, + }; + + it("vault_edge_observe: a freshly seeded zero-strength edge", () => { + expectSummarizesCleanly("vault_edge_observe", edge); + }); + + it("vault_edge_contest: no tension_id (reused legacy entry with none)", () => { + expectSummarizesCleanly( + "vault_edge_contest", + { edge: { ...edge, status: "revoked" }, tension_id: undefined }, + { checkSchema: false }, + ); + }); + + it("vault_edges: zero edges", () => { + expectSummarizesCleanly("vault_edges", { edges: [], total: 0 }); + }); +}); + +describe("consumes.ts summarizers — degenerate values", () => { + it("vault_consumes: zero edges", () => { + expectSummarizesCleanly("vault_consumes", { + direction: "forward", + anchor: "a.md", + edges: [], + total: 0, + include_history: false, + }); + }); +}); + +describe("themes.ts summarizers — degenerate values", () => { + it("vault_themes: zero themes", () => { + expectSummarizesCleanly("vault_themes", { + themes: [], + docMemberships: {}, + totalDocuments: 0, + totalChunks: 0, + skippedDocuments: 0, + selectedK: 10, + droppedClusters: 0, + clusteredAt: "2026-01-01T00:00:00Z", + }); + }); + + it("vault_themes: a theme with no retained primary member (all visitors) — no exemplar", () => { + const value = { + themes: [ + { + id: 0, + label: "x", + documentCount: 1, + primaryDocumentCount: 0, + coherence: null, + representativeDocs: [], + secondaryDocs: ["a.md"], + relatedTags: [], + }, + ], + docMemberships: {}, + totalDocuments: 1, + totalChunks: 1, + skippedDocuments: 0, + selectedK: 10, + droppedClusters: 0, + clusteredAt: "2026-01-01T00:00:00Z", + }; + expectSummarizesCleanly("vault_themes", value); + expect(tool("vault_themes").docLinks?.(value)).toEqual([]); + }); +}); + +describe("witness.ts summarizers — degenerate values", () => { + it("vault_witness: full report, nobody has written anything", () => { + expectSummarizesCleanly("vault_witness", { + principals: [], + unattributedDocs: 0, + concentration: { topPrincipal: null, topShare: 0 }, + flatCurveWarning: false, + }); + }); + + it("vault_witness: single-principal shape", () => { + expectSummarizesCleanly( + "vault_witness", + { + principal: { + principal: "agent:x", + writes: 0, + firstWriteAt: null, + lastWriteAt: null, + docsAuthored: 0, + liveClaims: 0, + openExposure: 0, + contestedOpen: 0, + stakeAtRisk: 0, + lost: 0, + burnedStake: 0, + survived: 0, + creditEarned: 0, + balance: 0, + proposals: { total: 0, ratified: 0, rejected: 0, expired: 0, pending: 0 }, + tensionsLogged: 0, + }, + concentration: { topPrincipal: null, topShare: 0 }, + flatCurveWarning: false, + }, + { checkSchema: false }, + ); + }); +}); + +describe("receipt.ts summarizers — degenerate values", () => { + it("vault_receipt: an empty summary, no flags", () => { + expectSummarizesCleanly("vault_receipt", { + claim: null, + sources: [], + summary: { + sourceCount: 0, + byStatus: {}, + openTensions: 0, + oldestUpdated: null, + newestUpdated: null, + flags: [], + }, + vaultHead: null, + generatedAt: "2026-01-01T00:00:00Z", + receiptHash: "h", + }); + }); +}); + +describe("tier1.ts summarizers — degenerate values", () => { + it("vault_tier1: no dependents, resolved at tier 1", () => { + expectSummarizesCleanly("vault_tier1", { + unit: "a.md", + changed_fields: [], + change_source: "explicit", + verdicts: [], + summary: { + unaffected: 0, + affected: 0, + possibly_affected: 0, + semantic_review: 0, + resolved_at_tier1: true, + }, + }); + }); +}); + +describe("tier2.ts summarizers — degenerate values", () => { + it("vault_tier2_queue: an empty queue", () => { + expectSummarizesCleanly("vault_tier2_queue", { items: [], total: 0 }); + }); + + it("vault_tier2_verdict: still-valid, no tension", () => { + expectSummarizesCleanly("vault_tier2_verdict", { + recorded: { + timestamp: "2026-01-01T00:00:00Z", + artifact: "a.md", + unit: "b.md", + edge_class: "declared", + judged_change_ts: "2026-01-01T00:00:00Z", + verdict: "still-valid", + reasoning: "r", + agent: "agent:x", + }, + tension_id: null, + }); + }); +}); + +describe("edge-staleness.ts summarizers — degenerate values", () => { + it("vault_staleness: artifact mode, nothing upstream", () => { + expectSummarizesCleanly("vault_staleness", { + mode: "artifact", + artifact: "a.md", + edges: [], + hidden_pending: "none", + summary: { current: 0, pending_unchecked: 0, pending_compatible: 0, pending_broken: 0 }, + }); + }); + + it("vault_staleness: artifact mode, all-hidden bucket 'many'", () => { + const value = { + mode: "artifact", + artifact: "a.md", + edges: [], + hidden_pending: "many", + summary: { current: 0, pending_unchecked: 0, pending_compatible: 0, pending_broken: 0 }, + }; + expectSummarizesCleanly("vault_staleness", value); + expect(tool("vault_staleness").summarize?.(value)).toContain("hidden_pending: many"); + }); + + it("vault_staleness: broken-read report mode, nothing instrumented", () => { + const value = { + mode: "report", + window_days: 30, + serves: 0, + broken_serves: 0, + broken_read_rate: null, + by_tool: {}, + uninstrumented: 0, + }; + expectSummarizesCleanly("vault_staleness", value); + expect(tool("vault_staleness").docLinks?.(value)).toEqual([]); + }); +}); diff --git a/test/tools/summary.test.ts b/test/tools/summary.test.ts new file mode 100644 index 00000000..74dcba7d --- /dev/null +++ b/test/tools/summary.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, it } from "vitest"; +import { clip, SUMMARY_DETAIL_CHARS, SUMMARY_MAX_ROWS } from "../../src/tools/summary.js"; + +describe("clip", () => { + it("returns short text unchanged", () => { + expect(clip("hello", 10)).toBe("hello"); + }); + + it("collapses internal whitespace, including newlines, to single spaces", () => { + expect(clip("hello\n\n world\tagain", 100)).toBe("hello world again"); + }); + + it("trims leading/trailing whitespace", () => { + expect(clip(" hello ", 100)).toBe("hello"); + }); + + it("truncates with an ellipsis when text exceeds max", () => { + const out = clip("abcdefghij", 5); + expect(out).toBe("abcd…"); + expect(out.length).toBe(5); + }); + + it("is idempotent on already-short, already-flat text", () => { + const once = clip("short text", 110); + expect(clip(once, 110)).toBe(once); + }); + + it("handles empty input", () => { + expect(clip("", 10)).toBe(""); + }); +}); + +describe("shared summary constants", () => { + it("are exported with the values every summarizer assumes", () => { + expect(SUMMARY_DETAIL_CHARS).toBe(110); + expect(SUMMARY_MAX_ROWS).toBe(20); + }); +}); diff --git a/test/tools/themes.test.ts b/test/tools/themes.test.ts index 514c3983..58948925 100644 --- a/test/tools/themes.test.ts +++ b/test/tools/themes.test.ts @@ -5,11 +5,14 @@ import { LOCAL_MINILM_DIM } from "../../src/search/providers/local-minilm.js"; import * as indexDb from "../../src/storage/index-db.js"; import { openIndexDb } from "../../src/storage/index-db.js"; import { vaultReindex } from "../../src/tools/search.js"; -import { __resetThemesCache, vaultThemes } from "../../src/tools/themes.js"; +import { __resetThemesCache, themesTools, vaultThemes } from "../../src/tools/themes.js"; import { loadConfig } from "../../src/utils/config.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const SAMPLE = resolve("test/fixtures/sample-vault"); +const themesTool = themesTools.find((t) => t.name === "vault_themes"); +if (!themesTool) throw new Error("vault_themes not registered"); const sampleConfig = loadConfig(SAMPLE); if (!sampleConfig.ok) throw sampleConfig.error; @@ -38,6 +41,7 @@ describe("vault_themes", () => { const result = await vaultThemes(vault, {}); expect(result.ok).toBe(true); if (!result.ok) return; + expectMatchesOutputSchema(themesTool, result.value); const v = result.value; expect(typeof v.totalDocuments).toBe("number"); expect(typeof v.skippedDocuments).toBe("number"); @@ -208,7 +212,7 @@ describe("vault_themes", () => { // Strip every embeddings-table row so each indexed document loses its // (chunk → embedding) join. Every doc should then be `skipped`. - const dbResult = openIndexDb(isolated, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(isolated, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const db = dbResult.value; @@ -304,7 +308,7 @@ describe("vault_themes", () => { // Mutate the index content: drop half the embeddings. The signature // must change and the next call must re-pool rather than serve stale // pooled vectors. - const dbResult = openIndexDb(isolated, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(isolated, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const db = dbResult.value; diff --git a/test/tools/tier1.test.ts b/test/tools/tier1.test.ts index 4926f2f9..c512a17b 100644 --- a/test/tools/tier1.test.ts +++ b/test/tools/tier1.test.ts @@ -2,11 +2,14 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { observeEdge } from "../../src/curation/edges.js"; import { recordProvenance } from "../../src/curation/provenance.js"; import { vaultRead } from "../../src/tools/read.js"; -import { vaultTier1 } from "../../src/tools/tier1.js"; +import { tier1Tools, vaultTier1 } from "../../src/tools/tier1.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:compiler"; +const tier1Tool = tier1Tools.find((t) => t.name === "vault_tier1"); +if (!tier1Tool) throw new Error("vault_tier1 not registered"); function frontmatter(overrides: Record = {}) { return { @@ -92,6 +95,7 @@ describe("vault_tier1 (#232)", () => { expect(result.value.change_source).toBe("provenance"); expect(result.value.changed_fields).toEqual(["tags"]); + expectMatchesOutputSchema(tier1Tool, result.value); const byArtifact = new Map(result.value.verdicts.map((v) => [v.artifact, v])); // Compiled whole-doc edge: certain hit (the run consumed everything). diff --git a/test/tools/tier2.test.ts b/test/tools/tier2.test.ts index 3190e121..e6990a77 100644 --- a/test/tools/tier2.test.ts +++ b/test/tools/tier2.test.ts @@ -2,13 +2,20 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { observeEdge } from "../../src/curation/edges.js"; import { DEFAULT_TENSION_STATUS, listTensions } from "../../src/curation/tension.js"; import { vaultStaleness } from "../../src/tools/edge-staleness.js"; -import { vaultTier2Queue, vaultTier2Verdict } from "../../src/tools/tier2.js"; +import { tier2Tools, vaultTier2Queue, vaultTier2Verdict } from "../../src/tools/tier2.js"; import { vaultWrite } from "../../src/tools/write.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:compiler"; const JUDGE = "agent:semantic-judge"; +function tier2Tool(name: string) { + const t = tier2Tools.find((x) => x.name === name); + if (!t) throw new Error(`${name} not registered`); + return t; +} + function frontmatter(overrides: Record = {}) { return { title: "Metric", @@ -90,6 +97,7 @@ describe("tier-2 queue and verdicts (#232)", () => { const queue = await vaultTier2Queue(vault, {}); expect(queue.ok).toBe(true); if (!queue.ok) throw queue.error; + expectMatchesOutputSchema(tier2Tool("vault_tier2_queue"), queue.value); const pairs = queue.value.items.map((i) => `${i.artifact}|${i.edge_class}`); expect(pairs).toContain("pricing/citer.md|declared"); expect(pairs).toContain("pricing/earned-dep.md|earned"); @@ -112,6 +120,7 @@ describe("tier-2 queue and verdicts (#232)", () => { expect(valid.ok).toBe(true); if (!valid.ok) throw valid.error; expect(valid.value.tension_id).toBeNull(); + expectMatchesOutputSchema(tier2Tool("vault_tier2_verdict"), valid.value); const after = await vaultTier2Queue(vault, {}); if (!after.ok) throw after.error; diff --git a/test/tools/witness.test.ts b/test/tools/witness.test.ts index 64082a1a..be7aa150 100644 --- a/test/tools/witness.test.ts +++ b/test/tools/witness.test.ts @@ -12,6 +12,10 @@ import { WAGER_GONE_STAKE, WAGER_SURVIVAL_CREDIT, } from "../../src/witness/track-record.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; + +const witnessTool = witnessTools.find((t) => t.name === "vault_witness"); +if (!witnessTool) throw new Error("vault_witness not registered"); const TODAY = new Date().toISOString().slice(0, 10); @@ -177,7 +181,15 @@ describe("buildWitness", () => { expect(r.ok).toBe(true); if (!r.ok) return; const loop = r.value.principals.find((p) => p.principal === "agent:loop"); - expect(loop?.proposals).toEqual({ total: 2, ratified: 0, rejected: 1, expired: 0, pending: 1 }); + expect(loop?.proposals).toEqual({ + total: 2, + ratified: 0, + rejected: 1, + expired: 0, + pending: 1, + edited: 0, + byCategory: {}, + }); }); it("raises the flat-curve warning when one principal holds ≥95% of writes", async () => { @@ -200,6 +212,67 @@ describe("buildWitness", () => { expect(r.value.principals).toEqual([]); }); + it("counts an edit-then-approve decision as edited, with its category (Decision 3)", async () => { + writeDoc("pricing/doc.md"); + const staged = await stageAction(vault, { + actionType: "confidence-up", + targetPath: "pricing/doc.md", + proposedBy: "agent:loop", + rationale: "r", + proposedDiff: { confidence: "high" }, + }); + expect(staged.ok).toBe(true); + if (!staged.ok) return; + const dec = await recordDecision(vault, staged.value.id, { + status: "ratified", + ratifiedAt: new Date().toISOString(), + ratifiedBy: "human:test", + decisionKind: "edit-then-approve", + reasonCategory: "overbroad", + }); + expect(dec.ok).toBe(true); + + const r = await buildWitness(vault); + expect(r.ok).toBe(true); + if (!r.ok) return; + const loop = r.value.principals.find((p) => p.principal === "agent:loop"); + // Status is authoritative: the edit still counts as ratified. edited is a + // SUBSET, not an addition. + expect(loop?.proposals.ratified).toBe(1); + expect(loop?.proposals.edited).toBe(1); + expect(loop?.proposals.byCategory).toEqual({ overbroad: 1 }); + }); + + it("does not fragment a principal's record when proposed_by rotates under one staged_by_principal (C4)", async () => { + writeDoc("pricing/x.md"); + writeDoc("pricing/y.md"); + const a = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/x.md", + proposedBy: "agent:rotating-1", + stagedByPrincipal: "human:mihir", + rationale: "r", + proposedDiff: {}, + }); + const b = await stageAction(vault, { + actionType: "promote", + targetPath: "pricing/y.md", + proposedBy: "agent:rotating-2", + stagedByPrincipal: "human:mihir", + rationale: "r", + proposedDiff: {}, + }); + expect(a.ok && b.ok).toBe(true); + + const r = await buildWitness(vault); + expect(r.ok).toBe(true); + if (!r.ok) return; + const mihir = r.value.principals.find((p) => p.principal === "human:mihir"); + expect(mihir?.proposals.total).toBe(2); + expect(r.value.principals.find((p) => p.principal === "agent:rotating-1")).toBeUndefined(); + expect(r.value.principals.find((p) => p.principal === "agent:rotating-2")).toBeUndefined(); + }); + it("scopes everything to readable collections under RBAC", async () => { writeDoc("pricing/open.md"); writeDoc("moonshot/secret.md"); @@ -225,8 +298,13 @@ describe("vaultWitness tool", () => { await logWrite("pricing/a.md", "agent:alpha"); const one = await vaultWitness(vault, { principal: "agent:alpha" }); expect(one.ok).toBe(true); + if (one.ok) expectMatchesOutputSchema(witnessTool, one.value); const miss = await vaultWitness(vault, { principal: "agent:nobody" }); expect(miss.ok).toBe(false); + + const full = await vaultWitness(vault, {}); + expect(full.ok).toBe(true); + if (full.ok) expectMatchesOutputSchema(witnessTool, full.value); }); it("denies a role with no read access", async () => { diff --git a/test/tools/write.test.ts b/test/tools/write.test.ts index aad49dd0..eba284ae 100644 --- a/test/tools/write.test.ts +++ b/test/tools/write.test.ts @@ -14,13 +14,18 @@ import { vaultDeprecate, vaultPromote, vaultWrite, + writeTools, } from "../../src/tools/write.js"; import { configPath } from "../../src/utils/config.js"; import { isGitRepo, log } from "../../src/utils/git.js"; +import { expectMatchesOutputSchema } from "../helpers/output-schema.js"; import { cleanupVault, makeTempVault } from "../helpers/temp-vault.js"; const AGENT = "agent:claude-code"; +const vaultWriteTool = writeTools.find((t) => t.name === "vault_write"); +if (!vaultWriteTool) throw new Error("vault_write not registered"); + // The write path stamps `new Date()` at write time (correct). Compare against a // date computed at ASSERTION time — never frozen at module load — and tolerate // the one-day window between the write and the read-back, so a run that crosses @@ -76,6 +81,7 @@ describe("write tools", () => { if (!result.ok) return; expect(result.value.action).toBe("create"); expect(result.value.commit).toMatch(/^[0-9a-f]+$/); + expectMatchesOutputSchema(vaultWriteTool, result.value); // The file is on disk with server-stamped updated / updated_by. const read = await vaultRead(vault, "pricing/new-note.md"); @@ -676,7 +682,7 @@ describe("write tools", () => { expect(update.value.commit).toMatch(/^[0-9a-f]+$/); // Indexed — the new content is searchable. expect(update.value.indexUpdated).toBe(true); - const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const doc = getDocument(dbResult.value, "pricing/oc-note.md"); @@ -729,7 +735,7 @@ describe("write tools", () => { if (!commitsAfter.ok) return; expect(commitsAfter.value.length).toBe(commitsBefore.value.length); // Index still holds the v2 content, not the rejected v3. - const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM); + const dbResult = openIndexDb(vault, LOCAL_MINILM_DIM, "float32"); expect(dbResult.ok).toBe(true); if (!dbResult.ok) return; const doc = getDocument(dbResult.value, path); diff --git a/test/utils/config-embeddings.test.ts b/test/utils/config-embeddings.test.ts index fd62c268..ba7215fb 100644 --- a/test/utils/config-embeddings.test.ts +++ b/test/utils/config-embeddings.test.ts @@ -39,6 +39,8 @@ describe("loadConfig — embeddings.provider", () => { expect(result.ok).toBe(true); if (!result.ok) return; expect(result.value.embeddingProvider).toBe("local-minilm"); + expect(result.value.embeddingDim).toBeNull(); + expect(result.value.embeddingQuantize).toBe("none"); }); it("defaults to local-minilm when the embeddings block is omitted", () => { @@ -118,3 +120,123 @@ describe("loadConfig — embeddings.provider", () => { expect(result.error.message).toMatch(/'embeddings' must be a mapping/); }); }); + +// New providers + dim/quantize (2026-07-26 embedding-refresh-quantization +// spec, Phase 2a). +describe("loadConfig — embeddings.provider (new local providers)", () => { + let dir: string; + + beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), "daftari-config-embeddings2-")); + }); + afterEach(() => { + rmSync(dir, { recursive: true, force: true }); + }); + + function writeConfig(yaml: string): void { + mkdirSync(join(dir, ".daftari"), { recursive: true }); + writeFileSync(configPath(dir), yaml); + } + + it("accepts provider: local-embeddinggemma, defaulting dim to 512 and quantize to int8", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingProvider).toBe("local-embeddinggemma"); + expect(result.value.embeddingDim).toBe(512); + expect(result.value.embeddingQuantize).toBe("int8"); + }); + + it("accepts provider: local-qwen3-0.6b, defaulting dim to 512 and quantize to int8", () => { + writeConfig("embeddings:\n provider: local-qwen3-0.6b\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingProvider).toBe("local-qwen3-0.6b"); + expect(result.value.embeddingDim).toBe(512); + expect(result.value.embeddingQuantize).toBe("int8"); + }); + + it("accepts an explicit dim: 768 for local-embeddinggemma", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n dim: 768\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingDim).toBe(768); + }); + + it("rejects dim: 384 for local-embeddinggemma (not a trained Matryoshka point)", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n dim: 384\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'embeddings\.dim' 384 is not valid/); + }); + + it("rejects any dim for local-minilm (fixed-dim provider)", () => { + writeConfig("embeddings:\n provider: local-minilm\n dim: 384\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'embeddings\.dim' is not accepted/); + }); + + it("rejects a non-integer or non-positive dim", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n dim: -5\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'embeddings\.dim' must be a positive integer/); + }); + + it("accepts quantize: none for local-embeddinggemma, overriding the int8 default", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n quantize: none\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingQuantize).toBe("none"); + }); + + it("accepts quantize: int8 for local-minilm (accepted for any provider)", () => { + writeConfig("embeddings:\n provider: local-minilm\n quantize: int8\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingProvider).toBe("local-minilm"); + expect(result.value.embeddingQuantize).toBe("int8"); + }); + + it("local-minilm still defaults to quantize: none", () => { + writeConfig("embeddings:\n provider: local-minilm\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.embeddingQuantize).toBe("none"); + }); + + it("rejects an unknown quantize value", () => { + writeConfig("embeddings:\n provider: local-embeddinggemma\n quantize: fp16\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'embeddings\.quantize' must be one of/); + }); + + it("rejects an unrecognised key under embeddings", () => { + writeConfig("embeddings:\n provider: local-minilm\n precision: high\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'embeddings\.precision' is not a recognised/); + }); + + it("the unknown-provider error message lists the new provider ids too", () => { + writeConfig("embeddings:\n provider: cohere-mighty-3\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/local-embeddinggemma/); + expect(result.error.message).toMatch(/local-qwen3-0\.6b/); + }); +}); diff --git a/test/utils/config-rerank.test.ts b/test/utils/config-rerank.test.ts new file mode 100644 index 00000000..cdc48d96 --- /dev/null +++ b/test/utils/config-rerank.test.ts @@ -0,0 +1,105 @@ +// Config parsing for the rerank.provider block (spec 2026-07-26-contextual- +// chunking-reranker-design.md Decision 5). +// +// The vault owner opts a reranker in via .daftari/config.yaml; the loader +// validates the choice. A missing or absent block defaults to "none" — the +// default install stays light, unlike embeddings (no OPENAI_API_KEY-style +// env check here: local-bge-m3 is a fully local model). Anything else is a +// hard config error — same trust model as embeddings.provider: a typo that +// meant to enable reranking must never silently no-op. + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { configPath, loadConfig } from "../../src/utils/config.js"; + +describe("loadConfig — rerank.provider", () => { + let dir: string; + + beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), "daftari-config-rerank-")); + }); + + afterEach(() => { + rmSync(dir, { recursive: true, force: true }); + }); + + function writeConfig(yaml: string): void { + mkdirSync(join(dir, ".daftari"), { recursive: true }); + writeFileSync(configPath(dir), yaml); + } + + it("defaults to none when no config file exists", () => { + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankProvider).toBe("none"); + }); + + it("defaults to none when the rerank block is omitted", () => { + writeConfig("auto_commit: true\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankProvider).toBe("none"); + }); + + it("defaults to none when the block is present but provider is omitted", () => { + writeConfig("rerank: {}\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankProvider).toBe("none"); + }); + + it("accepts provider: none explicitly", () => { + writeConfig("rerank:\n provider: none\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankProvider).toBe("none"); + }); + + it("accepts provider: local-bge-m3", () => { + writeConfig("rerank:\n provider: local-bge-m3\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.rerankProvider).toBe("local-bge-m3"); + }); + + it("rejects an unknown provider id with a helpful message", () => { + writeConfig("rerank:\n provider: cohere-rerank-3\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/unknown rerank\.provider/); + expect(result.error.message).toMatch(/none/); + expect(result.error.message).toMatch(/local-bge-m3/); + }); + + it("rejects non-string provider value", () => { + writeConfig("rerank:\n provider: 42\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/must be a string/); + }); + + it("rejects a rerank block that is not a mapping", () => { + writeConfig("rerank: not-a-mapping\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'rerank' must be a mapping/); + }); + + it("rejects a rerank block that is a list", () => { + writeConfig("rerank:\n - local-bge-m3\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toMatch(/'rerank' must be a mapping/); + }); +}); diff --git a/test/utils/config.test.ts b/test/utils/config.test.ts index c7e4a870..a98ffb0a 100644 --- a/test/utils/config.test.ts +++ b/test/utils/config.test.ts @@ -895,4 +895,171 @@ describe("malformed-comment hint on YAML parse errors (#26)", () => { expect(malformedCommentHint(text, 3)).toBeNull(); expect(malformedCommentHint(text, null)).toBeNull(); }); + + describe("search.routing (spec 2026-07-26 fusion overhaul, Decision 2)", () => { + it("defaults to routing: false when the block is absent", () => { + writeConfig("version: 1\nvault_name: v\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.search).toEqual({ routing: false }); + }); + + it("parses search.routing: true", () => { + writeConfig("version: 1\nvault_name: v\nsearch:\n routing: true\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.search.routing).toBe(true); + }); + + it("parses search.routing: false", () => { + writeConfig("version: 1\nvault_name: v\nsearch:\n routing: false\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.search.routing).toBe(false); + }); + + it('parses search.routing: "on" (quoted string)', () => { + writeConfig('version: 1\nvault_name: v\nsearch:\n routing: "on"\n'); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.search.routing).toBe(true); + }); + + it('parses search.routing: off (bare word — js-yaml 4\'s YAML-1.2 core schema loads it as the string "off", not a boolean)', () => { + writeConfig("version: 1\nvault_name: v\nsearch:\n routing: off\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.search.routing).toBe(false); + }); + + it("rejects an invalid routing value", () => { + writeConfig("version: 1\nvault_name: v\nsearch:\n routing: sometimes\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("search.routing"); + expect(result.error.message).toContain("on/off or true/false"); + }); + + it("rejects an unknown child key", () => { + writeConfig("version: 1\nvault_name: v\nsearch:\n routnig: true\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("'search.routnig' is not a recognised setting"); + }); + + it("rejects a non-mapping search block", () => { + writeConfig("version: 1\nvault_name: v\nsearch: true\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("'search' must be a mapping"); + }); + }); + + describe("code_repos / jit_anchors (2026-07-26 citation-anchors-jit spec)", () => { + it("defaults to an empty map and jit_anchors: true when absent", () => { + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.codeRepos).toEqual({}); + expect(result.value.jitAnchors).toBe(true); + }); + + it("resolves a relative path against the vault root", () => { + writeConfig("code_repos:\n api: ../code/api\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.codeRepos.api).toBe(resolve(dir, "../code/api")); + }); + + it("expands a leading ~ against the home directory", () => { + writeConfig("code_repos:\n api: ~/code/api\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.codeRepos.api).toBe(join(homedir(), "code", "api")); + }); + + it("does NOT check path existence at load — an absent repo loads fine", () => { + writeConfig("code_repos:\n ghost: /nowhere/does/not/exist\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.codeRepos.ghost).toBe("/nowhere/does/not/exist"); + }); + + it("rejects a repo name containing ':' (collides with the describes grammar)", () => { + writeConfig("code_repos:\n 'weird:name': ../code\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("must not contain ':'"); + }); + + it("rejects an empty path value", () => { + writeConfig("code_repos:\n api: ''\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + }); + + it("rejects a non-mapping code_repos block", () => { + writeConfig("code_repos: true\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("'code_repos' must be a mapping"); + }); + + it("parses jit_anchors: false", () => { + writeConfig("jit_anchors: false\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.jitAnchors).toBe(false); + }); + + it("rejects a non-boolean jit_anchors", () => { + writeConfig("jit_anchors: 'yes'\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("'jit_anchors' must be true or false"); + }); + }); + + describe("code_repo_visibility role flag (2026-07-27 resolution)", () => { + it("parses code_repo_visibility: true onto the role", () => { + writeConfig( + "roles:\n operator:\n read: ['*']\n write: ['*']\n code_repo_visibility: true\n", + ); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.roles.operator?.codeRepoVisibility).toBe(true); + }); + + it("defaults to absent/false when omitted — off by default for non-operator roles", () => { + writeConfig("roles:\n analyst:\n read: ['*']\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.roles.analyst?.codeRepoVisibility).toBeUndefined(); + }); + + it("rejects a non-boolean code_repo_visibility", () => { + writeConfig("roles:\n operator:\n read: ['*']\n code_repo_visibility: 'yes'\n"); + const result = loadConfig(dir); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain("code_repo_visibility"); + }); + }); }); diff --git a/test/utils/git.test.ts b/test/utils/git.test.ts index 08a08b12..89f1751b 100644 --- a/test/utils/git.test.ts +++ b/test/utils/git.test.ts @@ -4,10 +4,15 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { + blobAtHead, + blobSize, + catBlob, commit, ensureGitRepo, fileGitMeta, gitIdentity, + hashObject, + hashObjects, isGitRepo, log, } from "../../src/utils/git.js"; @@ -130,6 +135,82 @@ describe("git external git-dir", () => { }); }); +describe("citation-anchor plumbing (hashObjects/blobAtHead/blobSize/catBlob)", () => { + let vault: string; + + beforeEach(() => { + vault = makeTempVault(); + }); + + afterEach(() => { + cleanupVault(vault); + }); + + it("hashObject matches a single-file hashObjects call", async () => { + await writeFile(join(vault, "note.md"), "hello\n", "utf-8"); + const single = await hashObject(vault, "note.md"); + const batch = await hashObjects(vault, ["note.md"]); + expect(single.ok).toBe(true); + expect(batch.ok).toBe(true); + if (!single.ok || !batch.ok) return; + expect(single.value).toBe(batch.value[0]); + expect(single.value).toMatch(/^[0-9a-f]{40}$/); + }); + + it("blobAtHead returns the committed blob id, independent of a dirty working tree", async () => { + await writeFile(join(vault, "note.md"), "committed\n", "utf-8"); + await commit(vault, ["note.md"], "add note", "agent:tester"); + const head = await blobAtHead(vault, "note.md"); + expect(head.ok).toBe(true); + + await writeFile(join(vault, "note.md"), "dirty uncommitted content\n", "utf-8"); + const headAfterDirty = await blobAtHead(vault, "note.md"); + expect(headAfterDirty.ok).toBe(true); + if (!head.ok || !headAfterDirty.ok) return; + expect(headAfterDirty.value).toBe(head.value); // HEAD unaffected by the dirty write + + const workingHash = await hashObject(vault, "note.md"); + expect(workingHash.ok && workingHash.value).not.toBe(head.value); // working tree DID change + }); + + it("blobAtHead fails for a path that has never been committed", async () => { + await writeFile(join(vault, "untracked.md"), "x\n", "utf-8"); + const result = await blobAtHead(vault, "untracked.md"); + expect(result.ok).toBe(false); + }); + + it("blobSize and catBlob round-trip a committed blob's exact content", async () => { + const content = "line one\nline two\nline three\n"; + await writeFile(join(vault, "note.md"), content, "utf-8"); + await commit(vault, ["note.md"], "add note", "agent:tester"); + const sha = await hashObject(vault, "note.md"); + expect(sha.ok).toBe(true); + if (!sha.ok) return; + + const size = await blobSize(vault, sha.value); + expect(size.ok).toBe(true); + if (size.ok) expect(size.value).toBe(Buffer.byteLength(content, "utf-8")); + + const blob = await catBlob(vault, sha.value); + expect(blob.ok).toBe(true); + if (blob.ok) expect(blob.value).toBe(content); + }); + + it("blobSize/catBlob fail for a sha absent from the object database", async () => { + const missingSha = "0000000000000000000000000000000000dead"; + const size = await blobSize(vault, missingSha); + expect(size.ok).toBe(false); + const blob = await catBlob(vault, missingSha); + expect(blob.ok).toBe(false); + }); + + it("hashObjects errors the whole batch when one candidate is missing", async () => { + await writeFile(join(vault, "a.md"), "a\n", "utf-8"); + const result = await hashObjects(vault, ["a.md", "does-not-exist.md"]); + expect(result.ok).toBe(false); + }); +}); + describe("fileGitMeta", () => { it("reads add-date, last-date, and last author from history", async () => { const fmVault = buildFrontmatterLessVault(); diff --git a/test/utils/vault-gitignore.test.ts b/test/utils/vault-gitignore.test.ts index 7962e37b..d60b9527 100644 --- a/test/utils/vault-gitignore.test.ts +++ b/test/utils/vault-gitignore.test.ts @@ -49,4 +49,42 @@ describe("ensureVaultGitignore", () => { expect(after).toBe(before); expect(after.length).toBe(before.length); }); + + // C7 (2026-07-26 independence-aware-promotion spec): a vault whose + // .gitignore carries the marker but predates a later pattern-line addition + // to VAULT_GITIGNORE (e.g. the independence shadow journal, added after + // some vaults were already scaffolded) must pick up exactly what's missing. + it("per-line reconciliation: an old block gains exactly the new .daftari/ lines", async () => { + const path = join(dir, ".gitignore"); + const oldBlock = VAULT_GITIGNORE.split("\n") + .filter( + (l) => l !== ".daftari/independence-shadow.jsonl" && l !== ".daftari/revision-trace.jsonl", + ) + .join("\n"); + // Sanity: the marker survives the filter, so the block is still detected. + expect(oldBlock).toContain(".daftari/index.db"); + writeFileSync(path, oldBlock); + + const result = await ensureVaultGitignore(dir); + + expect(result).toBe("appended"); + const after = readFileSync(path, "utf-8"); + expect(after).toContain(".daftari/independence-shadow.jsonl"); + expect(after).toContain(".daftari/revision-trace.jsonl"); + // Idempotent: a second call against the now-current file is a no-op. + const second = await ensureVaultGitignore(dir); + expect(second).toBe("present"); + expect(readFileSync(path, "utf-8")).toBe(after); + }); + + it("a fully current .gitignore (all pattern lines present) is untouched", async () => { + const path = join(dir, ".gitignore"); + writeFileSync(path, VAULT_GITIGNORE); + const before = readFileSync(path, "utf-8"); + + const result = await ensureVaultGitignore(dir); + + expect(result).toBe("present"); + expect(readFileSync(path, "utf-8")).toBe(before); + }); });