test: derive subc registration checks from the embedded tool schemas - #200
Conversation
Adding an agent tool to the subc surface takes six coordinated registrations, and every existing guard iterated a hand-maintained list, so a missed site was invisible until a live tool call failed. cortexkit#195 lists eight instances; the most recent shipped a tool that was green in standalone for a month and dead within hours under the daemon. Derive the checks from subc_tool_schemas.json instead. The generator has no emit list of its own -- it calls the real tool factories and throws on absence -- so the embedded artifact is the closest thing to a source of truth the Rust side has. command_lane grows an explicit counterpart. Its catch-all arm returned Mutating for unknown commands, so a missing lane could never be observed; command_lane_explicit returns None there, and command_lane keeps the old behaviour as a thin wrapper. Six tools that previously reached the fallthrough now carry explicit Mutating arms, which is the value they already had. CORE_TOOLS becomes a slice so it has no arity to drift, and both it and the manifest count assert against the schema keys rather than a literal. The test asserts its own denominator: a truncated or empty artifact would otherwise iterate zero keys and pass vacuously. TRANSLATION_EXEMPT is empty and documented -- a future exemption is a reviewed decision rather than a silent skip. Each assertion was observed failing before being trusted: removing a name from is_subc_agent_core_tool, from build_manifest, from supports_tool, or deleting its lane arm each produces a message naming the tool, the missing registration, and the file to edit.
| "artifact_version": "0.49.0", | ||
| "manifest_id": "MAN-V049-S5-AGENT-SURFACE-001", | ||
| "source_commit": "d842d702a6afa8395c810282591c062e17cff521", | ||
| "source_commit": "d34f8bd1b2825bb5e903d3a1de3601a33c677248", |
There was a problem hiding this comment.
When the v0.49 audit runs at this PR's head, the integration-test insertion leaves the regenerated allowlist's source locations and the surface manifest's artifact hashes stale, causing the legacy-vocabulary and exact-byte checks to fail. Regenerate all governed artifacts after the final source changes.
There was a problem hiding this comment.
1 issue found across 5 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="docs/v0.49-agent-surface-manifest.json">
<violation number="1" location="docs/v0.49-agent-surface-manifest.json:301">
P1: The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| "encoding": "UTF-8", | ||
| "byte_length": 214468, | ||
| "sha256": "46275e5fc1084539c92ca4a239027a11da273f8e4ea4a0f1ac5baff5b9c7d9ee" | ||
| "sha256": "32828e24553b1935c5f7a4ae5b6e01b341c07808bb120a5a2a5b7b2e6e3c116e" |
There was a problem hiding this comment.
P1: The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/v0.49-agent-surface-manifest.json, line 301:
<comment>The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).</comment>
<file context>
@@ -295,10 +295,10 @@
"encoding": "UTF-8",
"byte_length": 214468,
- "sha256": "46275e5fc1084539c92ca4a239027a11da273f8e4ea4a0f1ac5baff5b9c7d9ee"
+ "sha256": "32828e24553b1935c5f7a4ae5b6e01b341c07808bb120a5a2a5b7b2e6e3c116e"
},
{
</file context>
Closes #195, implementing the five points you pinned.
build_manifest, four assertions per keymanifest.rsembedded_subc_tools_are_registered_across_all_rust_surfacescommand_lane_explicit(&str) -> Option<Lane>,command_lanea thin wrappermanifest.rs:92TRANSLATION_EXEMPTCORE_TOOLSand the hardcoded counts derive from the keysmanifest.rs,subc_bridge_test.rs:6886SUBC_TOOL_SCHEMAS.len() == tools.len()Notes on two places where I made a judgement call
Point 4,
CORE_TOOLSshape. A fixed-size[&str; N]cannot derive its length from a runtime map, so I made it&[&str]— no arity to drift — and asserted bothlen()against the key count and exact set equality. The alternative (keep the array, assert its length) leaves an arity literal in the file, which is the thing #195 is about.Point 5, the denominator. You framed this as key count equals
BARE_TOOL_ORDER's length.BARE_TOOL_ORDERis TypeScript-only and not reachable from Rust, so the Rust-side denominator asserts key count equals the manifest tool count, with a comment saying why. The TS side already pins the other half —subc-tool-schemas-fresh.test.tsasserts the committed artifact matches in-memory generation byte-for-byte and thatSUBC_BARE_TOOL_NAMEScovers exactly the artifact's keys. Between the two, a truncated artifact fails on one side and a drifted selection list on the other. If you want the count itself pinned in Rust, say so and I will add it, but it would be a literal again.command_lanebehaviour is unchangedThe catch-all returned
Mutatingfor anything unmatched, which is exactly why a missing lane was unobservable.command_lane_explicitreturnsNonethere;command_laneiscommand_lane_explicit(c).unwrap_or(Lane::Mutating).Six tools reached that fallthrough and now carry explicit
Mutatingarms —apply_patch,delete,move,import,refactor,safety. Same lane they already had, now stated. Every other call site is untouched.Every assertion was observed failing before being trusted
A guard nobody has watched fail is the defect this PR is about, so each one got a red:
Messages name the tool, the missing registration, and the file to edit — I followed the register
subc_plumbing_drift_test.rsalready uses.Verification
cargo nextest run -p agent-file-tools --lib -E 'test(subc)'→ 122 passed. Full workspace run reached the complete selected count; the failures in my worktree are pre-existing there (read-only borrow worktree ⇒ callgraph artifacts unavailable, plus sandbox/permission cases), and I confirmed that by running the identical selection onupstream/mainin the same worktree — byte-identical failure sets, 25 both sides, none naming a file this PR touches. Lint, format, and the v0.49 audit pass; release artifacts untouched.One thing worth flagging separately: you mentioned your gate scripts moved to
--no-fail-fast..config/nextest.tomlstill setsfail-fast = trueat the profile level, so a barecargo nextest runinherits it. On a machine with any pre-existing environmental failure that silently truncates the run — mine cancelled at ~1320 of 1643 and hid the rest, which is how the count assertion in this file reached CI in the first place.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Derives subc tool registration checks from the embedded schema so missing registrations fail tests with clear errors, addressing #195. Also makes command lane mapping explicit and removes hardcoded tool counts.
subc_tool_schemas.json; for each tool we assert presence in core list, manifest, explicit lane, and translation support.command_lane_explicit(&str) -> Option<Lane>;command_lanewraps it with defaultMutating. Added explicitMutatingarms for apply_patch, delete, move, import, refactor, safety.Written for commit c55cf58. Summary will update on new commits.
Greptile Summary
This PR derives Rust-side subc registration checks and manifest counts from the embedded tool schemas while preserving existing command-lane behavior. It also refreshes governed v0.49 audit artifacts, but those artifacts were generated before the final test changes and are stale at the PR head.
Confidence Score: 4/5
The subc test changes appear sound, but the governed v0.49 artifacts must be regenerated from the final PR head before merging because the repository audit currently fails.
The final integration-test insertion shifted source occurrences after the governed allowlist and integrity manifest were generated, leaving stale locations and artifact hashes that the audit script rejects.
Files Needing Attention: docs/v0.49-agent-surface-manifest.json, docs/v0.49-legacy-vocabulary-allowlist.json, docs/v0.49-agent-prefix-capture.json
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Embedded subc schema keys] --> B[CORE_TOOLS exact-set check] A --> C[Manifest registration] A --> D[Core-tool gate] A --> E[Explicit command lane] A --> F[Translation support] C --> G[Integration manifest count]Reviews (1): Last reviewed commit: "test: derive subc registration checks fr..." | Re-trigger Greptile