Skip to content

test: derive subc registration checks from the embedded tool schemas - #200

Merged
ualtinok merged 1 commit into
cortexkit:mainfrom
iceteaSA:subc-registration-guard
Aug 9, 2026
Merged

test: derive subc registration checks from the embedded tool schemas#200
ualtinok merged 1 commit into
cortexkit:mainfrom
iceteaSA:subc-registration-guard

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Closes #195, implementing the five points you pinned.

# Point Where
1 Derived test beside build_manifest, four assertions per key manifest.rs embedded_subc_tools_are_registered_across_all_rust_surfaces
2 command_lane_explicit(&str) -> Option<Lane>, command_lane a thin wrapper manifest.rs:92
3 Documented exemption const, empty today TRANSLATION_EXEMPT
4 CORE_TOOLS and the hardcoded counts derive from the keys manifest.rs, subc_bridge_test.rs:6886
5 Test asserts its own denominator SUBC_TOOL_SCHEMAS.len() == tools.len()

Notes on two places where I made a judgement call

Point 4, CORE_TOOLS shape. A fixed-size [&str; N] cannot derive its length from a runtime map, so I made it &[&str] — no arity to drift — and asserted both len() against the key count and exact set equality. The alternative (keep the array, assert its length) leaves an arity literal in the file, which is the thing #195 is about.

Point 5, the denominator. You framed this as key count equals BARE_TOOL_ORDER's length. BARE_TOOL_ORDER is TypeScript-only and not reachable from Rust, so the Rust-side denominator asserts key count equals the manifest tool count, with a comment saying why. The TS side already pins the other half — subc-tool-schemas-fresh.test.ts asserts the committed artifact matches in-memory generation byte-for-byte and that SUBC_BARE_TOOL_NAMES covers exactly the artifact's keys. Between the two, a truncated artifact fails on one side and a drifted selection list on the other. If you want the count itself pinned in Rust, say so and I will add it, but it would be a literal again.

command_lane behaviour is unchanged

The catch-all returned Mutating for anything unmatched, which is exactly why a missing lane was unobservable. command_lane_explicit returns None there; command_lane is command_lane_explicit(c).unwrap_or(Lane::Mutating).

Six tools reached that fallthrough and now carry explicit Mutating arms — apply_patch, delete, move, import, refactor, safety. Same lane they already had, now stated. Every other call site is untouched.

Every assertion was observed failing before being trusted

A guard nobody has watched fail is the defect this PR is about, so each one got a red:

tool "zoom" is missing from is_subc_agent_core_tool in crates/aft/src/subc/manifest.rs
tool "zoom" is missing from build_manifest in crates/aft/src/subc/manifest.rs
tool "delete" is missing an explicit command_lane arm in crates/aft/src/subc/manifest.rs
tool "zoom" is missing from supports_tool in crates/aft/src/subc_translate.rs
denominator: left: 21, right: 22

Messages name the tool, the missing registration, and the file to edit — I followed the register subc_plumbing_drift_test.rs already uses.

Verification

cargo nextest run -p agent-file-tools --lib -E 'test(subc)' → 122 passed. Full workspace run reached the complete selected count; the failures in my worktree are pre-existing there (read-only borrow worktree ⇒ callgraph artifacts unavailable, plus sandbox/permission cases), and I confirmed that by running the identical selection on upstream/main in the same worktree — byte-identical failure sets, 25 both sides, none naming a file this PR touches. Lint, format, and the v0.49 audit pass; release artifacts untouched.

One thing worth flagging separately: you mentioned your gate scripts moved to --no-fail-fast. .config/nextest.toml still sets fail-fast = true at the profile level, so a bare cargo nextest run inherits it. On a machine with any pre-existing environmental failure that silently truncates the run — mine cancelled at ~1320 of 1643 and hid the rest, which is how the count assertion in this file reached CI in the first place.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Derives subc tool registration checks from the embedded schema so missing registrations fail tests with clear errors, addressing #195. Also makes command lane mapping explicit and removes hardcoded tool counts.

  • Refactors
    • Registration guard now derives from subc_tool_schemas.json; for each tool we assert presence in core list, manifest, explicit lane, and translation support.
    • Added command_lane_explicit(&str) -> Option<Lane>; command_lane wraps it with default Mutating. Added explicit Mutating arms for apply_patch, delete, move, import, refactor, safety.
    • Replaced fixed-size CORE_TOOLS with a slice and assert set equality and count vs schema keys; test also checks schema key count equals manifest tools.
    • Updated integration test to compare manifest tool count to schema key count; refreshed docs artifacts to the new source commit (no behavior change).

Written for commit c55cf58. Summary will update on new commits.

Review in cubic

Greptile Summary

This PR derives Rust-side subc registration checks and manifest counts from the embedded tool schemas while preserving existing command-lane behavior. It also refreshes governed v0.49 audit artifacts, but those artifacts were generated before the final test changes and are stale at the PR head.

  • Adds an explicit optional lane classifier and verifies every embedded subc tool across Rust registration surfaces.
  • Replaces a literal integration-test tool count with the embedded schema count.
  • Refreshes v0.49 prefix, surface-manifest, and vocabulary-allowlist artifacts.

Confidence Score: 4/5

The subc test changes appear sound, but the governed v0.49 artifacts must be regenerated from the final PR head before merging because the repository audit currently fails.

The final integration-test insertion shifted source occurrences after the governed allowlist and integrity manifest were generated, leaving stale locations and artifact hashes that the audit script rejects.

Files Needing Attention: docs/v0.49-agent-surface-manifest.json, docs/v0.49-legacy-vocabulary-allowlist.json, docs/v0.49-agent-prefix-capture.json

Important Files Changed

Filename Overview
crates/aft/src/subc/manifest.rs Adds behavior-preserving explicit lane classification and comprehensive schema-derived registration tests.
crates/aft/tests/integration/subc_bridge_test.rs Derives the expected manifest count from the embedded schema, but its inserted lines invalidate source locations in the separately changed governed allowlist.
docs/v0.49-agent-surface-manifest.json Records governed artifact provenance and hashes from an intermediate commit, so final-head audit verification fails.
docs/v0.49-legacy-vocabulary-allowlist.json Refreshes source locations before the final integration-test insertion, leaving tracked line positions stale.
docs/v0.49-agent-prefix-capture.json Refreshes capture provenance from the same intermediate artifact-generation commit and requires final-head regeneration with the manifest.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Embedded subc schema keys] --> B[CORE_TOOLS exact-set check]
  A --> C[Manifest registration]
  A --> D[Core-tool gate]
  A --> E[Explicit command lane]
  A --> F[Translation support]
  C --> G[Integration manifest count]
Loading

Reviews (1): Last reviewed commit: "test: derive subc registration checks fr..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Adding an agent tool to the subc surface takes six coordinated
registrations, and every existing guard iterated a hand-maintained list,
so a missed site was invisible until a live tool call failed. cortexkit#195 lists
eight instances; the most recent shipped a tool that was green in
standalone for a month and dead within hours under the daemon.

Derive the checks from subc_tool_schemas.json instead. The generator has
no emit list of its own -- it calls the real tool factories and throws on
absence -- so the embedded artifact is the closest thing to a source of
truth the Rust side has.

command_lane grows an explicit counterpart. Its catch-all arm returned
Mutating for unknown commands, so a missing lane could never be observed;
command_lane_explicit returns None there, and command_lane keeps the old
behaviour as a thin wrapper. Six tools that previously reached the
fallthrough now carry explicit Mutating arms, which is the value they
already had.

CORE_TOOLS becomes a slice so it has no arity to drift, and both it and
the manifest count assert against the schema keys rather than a literal.
The test asserts its own denominator: a truncated or empty artifact would
otherwise iterate zero keys and pass vacuously.

TRANSLATION_EXEMPT is empty and documented -- a future exemption is a
reviewed decision rather than a silent skip.

Each assertion was observed failing before being trusted: removing a name
from is_subc_agent_core_tool, from build_manifest, from supports_tool, or
deleting its lane arm each produces a message naming the tool, the missing
registration, and the file to edit.
"artifact_version": "0.49.0",
"manifest_id": "MAN-V049-S5-AGENT-SURFACE-001",
"source_commit": "d842d702a6afa8395c810282591c062e17cff521",
"source_commit": "d34f8bd1b2825bb5e903d3a1de3601a33c677248",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Governed artifacts are stale

When the v0.49 audit runs at this PR's head, the integration-test insertion leaves the regenerated allowlist's source locations and the surface manifest's artifact hashes stale, causing the legacy-vocabulary and exact-byte checks to fail. Regenerate all governed artifacts after the final source changes.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="docs/v0.49-agent-surface-manifest.json">

<violation number="1" location="docs/v0.49-agent-surface-manifest.json:301">
P1: The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

"encoding": "UTF-8",
"byte_length": 214468,
"sha256": "46275e5fc1084539c92ca4a239027a11da273f8e4ea4a0f1ac5baff5b9c7d9ee"
"sha256": "32828e24553b1935c5f7a4ae5b6e01b341c07808bb120a5a2a5b7b2e6e3c116e"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/v0.49-agent-surface-manifest.json, line 301:

<comment>The refreshed hashes are inconsistent with the declared source_commit. The new sha256/byte_length for LIST-V049-LEGACY-VOCABULARY-001 and ART-V049-S5-AGENT-PREFIX-CAPTURE-001 match only the content at the PR head commit (c55cf58), but source_commit is set to d34f8bd, which still holds the old bytes (46275e5f / 8e32fd). scripts/release-gate-v049.mjs checkGovernedBytes will reject the manifest because the source-commit bytes no longer match, and d34f8bd is not an ancestor of HEAD, failing its ancestor check too. Point source_commit at the commit that actually contains the newly-hashed bytes (or leave the old hashes).</comment>

<file context>
@@ -295,10 +295,10 @@
       "encoding": "UTF-8",
       "byte_length": 214468,
-      "sha256": "46275e5fc1084539c92ca4a239027a11da273f8e4ea4a0f1ac5baff5b9c7d9ee"
+      "sha256": "32828e24553b1935c5f7a4ae5b6e01b341c07808bb120a5a2a5b7b2e6e3c116e"
     },
     {
</file context>

@ualtinok
ualtinok merged commit 42b5ac8 into cortexkit:main Aug 9, 2026
16 checks passed
ualtinok added a commit that referenced this pull request Aug 9, 2026
@iceteaSA
iceteaSA deleted the subc-registration-guard branch August 9, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent tools need six registrations to work under subc, and nothing checks five of them

2 participants