Skip to content

docs: document search/neighbors/list wire shapes#954

Merged
ohdearquant merged 4 commits into
mainfrom
docs-900-wire-shapes
Jul 13, 2026
Merged

docs: document search/neighbors/list wire shapes#954
ohdearquant merged 4 commits into
mainfrom
docs-900-wire-shapes

Conversation

@ohdearquant

Copy link
Copy Markdown
Owner

Summary

Closes #900. search() and neighbors() returned rows whose wire shape was never documented and diverges from list()'s row shape, so integrators discovered it by probing a live server. This PR documents the current shapes precisely, field-by-field, verified against the handler and runtime source (crates/khive-pack-kg/src/handlers/search.rs, graph.rs, list.rs; crates/khive-storage/src/types/graph.rs, entity.rs, note.rs, event.rs). No behavior or response shape changes.

  • docs/guide/api-reference.md — full "Response shape" sections added under search, neighbors, and list, with a JSON example and explicit field list per verb/kind.
  • AGENTS.md — the tool-schemas table rows for search, neighbors, and list get a compact shape summary plus a pointer into the full reference, matching the existing style used for merge/propose/context.

Divergences documented as current behavior (not changed):

  • search() rows are a narrow ranking projection {id, entity_kind|note_kind, score, title, snippet} — never the full record. score is an RRF-fused rank score (1/(k+rank), k=10), not a normalized 0-1 similarity.
  • neighbors() rows are flat, one per edge: {origin_id, id, edge_id, relation, weight, name?, kind?, entity_type?}. name/kind/entity_type are omitted from the JSON entirely (not null) when the neighbor node can't be resolved — unlike search()'s kind field, which is always present and only ever null. kind is overloaded between an entity's base kind and a note's note_kind, with no separate substrate discriminant field.
  • list() rows are the full stored record, and the shape differs again by kind: notes have no top-level tags field (tags live in properties.tags, unlike entities which have a first-class tags column), and gain a lifecycle field (with status replaced by the pack-level properties.status value) when the note carries a properties.status.

Candidate follow-ups (not addressed here, flagging per the issue's ask)

  • The issue itself suggests an ADR for row-shape convergence across search/neighbors/list if that's wanted — this PR only documents current behavior, it doesn't propose or implement convergence.
  • neighbors()'s kind field relies on the entity-kind and note-kind vocabularies staying disjoint to be disambiguable by value alone (no substrate discriminant field). Works correctly today because the two closed vocabularies don't overlap, but it's a fragile implicit invariant worth a comment or an explicit substrate field if it's ever revisited.
  • list()'s per-kind row caps are inconsistent (entity 500, note 200, edge 1000, event 1000) — not a bug, just worth a note if row-shape/behavior convergence work happens later.

No .rs files were touched, so no clippy/test changes are needed. make fmt (cargo fmt + deno fmt docs/) and deno fmt --check AGENTS.md both pass with no reformatting required.

Test plan

  • make fmt — no changes (docs already fmt-clean)
  • make docs-check (deno fmt --check docs/) — passed
  • deno fmt --check AGENTS.md — passed
  • Every documented field cross-checked against source (handler code + storage struct definitions), not memory

🤖 Generated with Claude Code

search() and neighbors() rows were undocumented and diverge from
list()'s full-record rows, so agents were discovering the shapes by
probing a live server. Document all three, field-by-field, verified
against the handler and runtime source:

- search(): narrow ranking projection {id, entity_kind|note_kind,
  score, title, snippet}; score is an RRF rank score, not a
  normalized similarity.
- neighbors(): flat per-edge rows {origin_id, id, edge_id, relation,
  weight, name?, kind?, entity_type?}; name/kind/entity_type are
  omitted (not null) when the neighbor can't be resolved, and kind
  is overloaded between entity base-kind and note-kind.
- list(): the full stored record, shape depends on kind
  (entity/note/edge/event); notes have no top-level tags field and
  gain a lifecycle field when properties.status is set.

Full detail lives in docs/guide/api-reference.md (the verified
per-verb parameter/response contract); AGENTS.md's tool-schemas
table gets a compact pointer to each, matching the existing
merge/propose/context row style.

No behavior or response-shape change.

Refs #900
@ohdearquant

Copy link
Copy Markdown
Owner Author

REJECT

Findings: 0 Blocker, 2 High, 2 Medium, 1 Low

[High] Default list and search wire shapes are documented as their unprojected forms

Evidence: docs/guide/api-reference.md:227 says every list row is always the full stored record, and docs/guide/api-reference.md:352 says the search kind field is always present and can be null. The request server defaults an unspecified presentation mode to Agent at crates/khive-mcp/src/server.rs:1741. Agent presentation removes non-lifecycle null fields at crates/khive-runtime/src/presentation.rs:525, also dropping empty values and compacting IDs and timestamps at crates/khive-runtime/src/presentation.rs:535. Search is not one of the AlwaysVerbose verbs at crates/khive-types/src/pack.rs:145.

The listed fields are useful canonical/verbose shapes, but they are not the default MCP wire shape. In particular, absent entity_kind/note_kind, title, and snippet keys are omitted under Agent rather than represented by null. The same unqualified full-record claim is repeated in AGENTS.md:439 and AGENTS.md:443.

Fix: identify the listed object shapes as presentation="verbose" (or canonical handler) shapes, then document the default Agent projection and its conditional omissions. Alternatively, make every field statement explicitly conditional on presentation mode.

[High] The documented search score semantics conflate entity and note results

Evidence: docs/guide/api-reference.md:352 says note rows are identical except for note_kind, and docs/guide/api-reference.md:354 defines score as 1 / (10 + rank). Entity retrieval sums reciprocal-rank contributions and applies an exact-match boost at crates/khive-runtime/src/retrieval.rs:1000, crates/khive-runtime/src/retrieval.rs:1009, and crates/khive-runtime/src/retrieval.rs:1022. Notes use RRF_K = 60 at crates/khive-runtime/src/operations.rs:2971 and multiply the fused result by a salience-derived weight at crates/khive-runtime/src/operations.rs:3131.

This publishes a false numeric contract for note scores and makes the entity formula incomplete. It also makes the two shapes appear more equivalent than their construction is. The repeated summary in AGENTS.md:443 has the same issue.

Fix: describe score as an implementation-defined ranking value, or document entity and note scoring separately with their actual fusion, boost, and salience behavior. Do not state one K=10 formula for both kinds.

[Medium] The supported proposal list shape is omitted and contradicts the full-record rule

Evidence: docs/guide/api-reference.md:199 lists proposal as a supported list kind, while docs/guide/api-reference.md:227 limits the documented rows to entity, note, edge, and event and calls every row a full stored record. Proposal handling is selected by the list handler at crates/khive-pack-kg/src/handlers/list.rs:61, but its response is a purpose-built projection at crates/khive-pack-kg/src/handlers/proposal.rs:538, not a full stored record.

Fix: add the proposal projection to the row-shape documentation, or explicitly scope the full-record statement to entity, note, edge, and event while directing proposal consumers to its separate projection.

[Medium] The neighbors omission condition incorrectly names soft-deleted entities

Evidence: docs/guide/api-reference.md:415 says name/kind/entity_type may be omitted for a soft-deleted entity. The neighbors runtime removes hits for soft-deleted entity IDs before returning them at crates/khive-runtime/src/operations.rs:1919. Optional neighbor enrichment fields do omit when unresolved, as defined at crates/khive-storage/src/types/graph.rs:294, but a soft-deleted entity produces no row rather than a row with omitted enrichment.

Fix: replace the soft-deleted-entity example with an unresolved/dangling/bogus ID example, and retain the precise rule that the optional fields are omitted when enrichment has no value. Mirror the correction in AGENTS.md:445.

[Low] The added prose violates the no-em-dash requirement

Evidence: the PR adds 17 lines containing an em dash, including docs/guide/api-reference.md:228, docs/guide/api-reference.md:355, and AGENTS.md:439.

Fix: replace every added em dash with a colon, comma, parentheses, or a rewritten sentence.

Confirmed

  • git diff origin/main --stat reports only AGENTS.md and docs/guide/api-reference.md; the diff touches zero .rs files.
  • git diff --check origin/main...HEAD is clean.
  • The canonical entity/note/edge/event list fields and the core neighbors fields, including the include_entity_type gate, match their response-building code when read as verbose/canonical shapes.
  • The API reference is an appropriate consumer-facing location, and the documentation otherwise describes current behavior rather than a proposal.

Checks run

  • gh pr view 954 --repo ohdearquant/khive --json number,title,state,isDraft,baseRefName,headRefName,headRefOid,url
  • git diff origin/main --stat
  • git diff --name-only origin/main...HEAD -- '*.rs'
  • git diff --check origin/main...HEAD
  • Targeted source inspection of the KG handlers, retrieval, graph traversal, storage response types, and presentation layer.
  • CARGO_TARGET_DIR="$HOME/.cache/shared-cargo-target/khive" cargo test -p khive-pack-kg --lib handlers::search -- --nocapture (exit 0; the filter selected 0 tests).

Domain utility: SKIPPED - This field-level accuracy review used the implementation as the authoritative source.

- list/search: label full-record shapes as presentation=verbose, document
  the default Agent-mode projection (conditional null/empty drops, id and
  timestamp compaction, score truncation)
- search: describe entity vs note score construction separately instead of
  a single 1/(10+rank) formula (entities sum per-leg RRF + exact-match
  boost; notes use k=60 then a salience-derived weight)
- list: document the proposal row as its own projection, not a full record
- neighbors: fix the omission example (soft-deleted entities produce no
  row at all; omitted enrichment fields are for unresolved/dangling ids)
- remove all em dashes from the added prose

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ohdearquant

Copy link
Copy Markdown
Owner Author

APPROVE-WITH-FIXES

Findings: 0 Blocker, 0 High, 1 Medium, 0 Low

[Medium] Proposal list documentation incorrectly says its projection is presentation-invariant

Evidence: docs/guide/api-reference.md:262 through docs/guide/api-reference.md:266 identify the proposal projection correctly, but state that it is always returned in that shape regardless of presentation. The compact AGENTS summary at AGENTS.md:439 likewise limits Agent-mode reshaping to entity, note, edge, and event rows. list is not in the AlwaysVerbose set at crates/khive-types/src/pack.rs:145, so the MCP server applies the request's mode to it (crates/khive-mcp/src/server.rs:831). Successful results are passed through the presentation transform (crates/khive-mcp/src/server.rs:1416), and Agent mode recursively shortens IDs, compacts timestamps, and drops non-lifecycle null and empty fields (crates/khive-runtime/src/presentation.rs:420). The proposal handler produces an ID, timestamps, nullable expiry, and potentially empty last_decision fields (crates/khive-pack-kg/src/handlers/proposal.rs:532).

Why this matters: the new proposal projection is now documented, but its default MCP wire shape remains inaccurate. A default Agent-mode list(kind="proposal") can omit fields and compact IDs/timestamps; only the verbose projection has the stated unconditional field set.

Suggested fix: label the listed proposal object as the presentation="verbose" projection and state that the default Agent mode applies the same generic null/empty omission, ID shortening, and timestamp compaction rules. Make the corresponding AGENTS row include proposal in that qualification.

Looks Right

  • The entity, note, edge, and event list shapes are now explicitly verbose-only; their default Agent-mode omissions and compaction match server.rs:1741 and presentation.rs:525.
  • Search now distinguishes entity RRF plus exact-match boost from note k=60 RRF plus salience weighting, matching retrieval.rs:1000 and operations.rs:2971 / operations.rs:3131.
  • The proposal fields themselves match the purpose-built handler projection at proposal.rs:538; only the presentation qualification above remains.
  • The neighbors correction correctly distinguishes omitted enrichment for unresolved IDs from soft-deleted entity neighbors, which are filtered out before response construction (operations.rs:1919; graph.rs:294).
  • The PR delta contains only AGENTS.md and docs/guide/api-reference.md, with zero Rust files and zero added em-dash lines. The fix commit d65eebb6..d6762286 also adds zero em-dash lines.

Commands Run

  • git diff --check d65eebb6..d6762286: clean.
  • git diff --check origin/main...HEAD: clean.
  • git diff --name-only origin/main...HEAD -- '*.rs': zero files.
  • git diff origin/main...HEAD | grep '^+' | grep -c $'—': 0.
  • gh pr diff 954 --repo ohdearquant/khive --name-only: only AGENTS.md and docs/guide/api-reference.md.
  • Targeted inspection of the requested server, presentation, retrieval, operations, proposal, and graph source lines.

origin/main advanced after this branch's merge base (779872b3) with unrelated PR #953. Consequently, the literal git diff origin/main includes two unrelated Rust files and two unrelated added em-dash lines from that base drift; it is not the PR #954 delta. The merge-base/hosted-PR comparison above is the relevant scope check.

What I Did Not Check

  • No Rust tests were run: this is a documentation-only, source-verified re-review, and the requested PR delta contains no Rust changes.
  • I did not perform a broader review beyond the five original findings and the directly affected presentation claim.

Re-Review Guidance

  • A final narrow documentation check is sufficient after correcting the proposal presentation qualification.

Domain utility: SKIPPED - This narrow wire-contract review used the cited implementation paths as the authoritative source.

Agent mode applies the same generic null/empty omission, id shortening,
and timestamp compaction to proposal rows; only the verbose projection
has the unconditional field set. Addresses the round-2 review finding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ohdearquant

Copy link
Copy Markdown
Owner Author

APPROVE-WITH-FIXES

Findings: 0 Blocker, 0 High, 0 Medium, 1 Low

[Low] New API-reference sentence adds an em dash

Evidence: docs/guide/api-reference.md:267 adds an em dash between "rows" and "non-lifecycle". The targeted git diff d6762286..2368993e contains this one added em dash.

Why this matters: the requested round-3 acceptance criteria explicitly require no added em dashes. The semantic correction is sound, but this punctuation violates that narrow documentation constraint.

Suggested fix: replace the em dash at docs/guide/api-reference.md:267 with a colon, comma, or semicolon.

Looks Right

  • docs/guide/api-reference.md:262-268 identifies the proposal projection as presentation="verbose" and states that default Agent mode omits ordinary null/empty fields, shortens IDs, and compacts timestamps.
  • AGENTS.md:439 gives the proposal projection the same verbose qualification and includes proposal in the Agent-mode reshaping rule.
  • list is a Standard, not AlwaysVerbose, operation (crates/khive-types/src/pack.rs:145-149). The MCP server therefore retains the requested Agent mode (crates/khive-mcp/src/server.rs:831-839) and applies it to successful results (crates/khive-mcp/src/server.rs:915-924, crates/khive-mcp/src/server.rs:1416-1433).
  • Agent presentation recursively visits array rows (crates/khive-runtime/src/presentation.rs:461-498), drops ordinary null/empty values, shortens ID fields, and compacts metadata timestamps (crates/khive-runtime/src/presentation.rs:525-554). The proposal handler provides the documented fields, including nullable expiry and potentially empty last_decision (crates/khive-pack-kg/src/handlers/proposal.rs:538-550).
  • The fix range changes only AGENTS.md and docs/guide/api-reference.md; there is no scope creep and git diff --check is clean.

Commands Run

  • git diff --check d6762286..2368993e: clean.
  • git diff --name-only d6762286..2368993e: only AGENTS.md and docs/guide/api-reference.md.
  • git diff --word-diff=porcelain d6762286..2368993e -- AGENTS.md docs/guide/api-reference.md: confirmed the two substantive proposal wording changes and the one new em dash.
  • Targeted source inspection of crates/khive-mcp/src/server.rs, crates/khive-runtime/src/presentation.rs, crates/khive-types/src/pack.rs, and crates/khive-pack-kg/src/handlers/proposal.rs: documentation claims match implementation.

What I Did Not Check

  • No Rust tests were run. This was the requested documentation-only, source-verified check of the fix commit.

Re-Review Guidance

  • After replacing the single punctuation character, no further review is needed.

Domain utility: SKIPPED - This narrow wire-contract check used the cited implementation paths as the authoritative source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ohdearquant

Copy link
Copy Markdown
Owner Author

Round-3 finding applied: the single em dash at docs/guide/api-reference.md is now a colon. Per r3 re-review guidance no further review is needed. Marking ready (docs-only, fast lane).

@ohdearquant ohdearquant marked this pull request as ready for review July 13, 2026 16:13
@ohdearquant ohdearquant enabled auto-merge (squash) July 13, 2026 16:13

@oceanwaves630 oceanwaves630 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving: r3 clean at acaf292, docs wire-shape corrections verified through the codex rounds.

@ohdearquant ohdearquant merged commit c947e3a into main Jul 13, 2026
22 checks passed
@ohdearquant ohdearquant deleted the docs-900-wire-shapes branch July 13, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search() and neighbors() wire shapes are undocumented and diverge from list() row shape

2 participants