Repository navigation
fix(ai-backend): declare the context nobody was declaring - #629
Merged
Merged
Conversation
A live packaged run recorded `undeclared_tokens: 8257` on every model call — 31% of measured input — through a field whose contract says the number is EXPECTED 0 and anything above it is a first-party defect. PR #627 had just stopped journey CB-8 failing on it, and the composer's context meter has been showing users an "N undeclared" notice since #625. It was three unrelated defects, and the largest row was not a tool at all. **Ten library-installed tools, ~3,459 tokens.** The declaration seam stamps tools at `factory._model_visible_tools`, and the PRD-02 AST gate sweeps that same function. But `create_deep_agent` installs the filesystem, todo and subagent middleware tools INSIDE the library, so they reach the provider having passed no append site: no stamp to read, and nothing for the gate to see. A gate scoped to one function cannot fail on contributors that never enter it. `ThirdPartyToolOrigins` now declares them on their author's behalf — §4.3's sanctioned exception, applied to the tool block the way it already was to the system block — resolving the owner from the tool's dispatch callable's `__module__` and injected as `ToolSchemaLedger.measure(fallback_origin=...)`. Resolution is by that callable and never by the tool's class or name. The class is `StructuredTool` whoever wrote the tool, so `type(tool).__module__` would relabel OUR tools as third-party `langchain_core` text. And the allowlist names the libraries we speak for rather than excluding the packages we own, because "unknown means theirs" fails by silent misattribution: `task` looks exactly like a deepagents built-in — same name, same middleware slot — but is `build_atlas_task_tool`, ours, and would have billed ~390 resident tokens of our text to a dependency that does not contain the string. It is declared at its authoring site instead, as is `web_search`. **`web_search`, ~198 tokens.** Declared correctly at the factory and destroyed downstream. `wrap_tools_with_display` copies a `StructuredTool` (the stamp rides along in the instance `__dict__`) but REBUILDS any other `BaseTool` via `from_function`, holding the inner in a closure — no stamp, and no attribute chain for `ContextOriginBinding.of` to walk. Every non-`StructuredTool` on the surface was silently un-declared on its way to the model. The declaration is now carried across the rebuild, so it is still made once, by the code that composed the text. **The system block, 4,798 tokens — 58% of the total, and not a tool.** Every row a packaged run wrote carries `assembly_record_id: None`: F2 prompt assembly does not run on the desktop, so there is no `PromptAssemblyPlan`, and the whole prompt fell through as one unattributed span. Without a plan the block has exactly one first-party contributor — the `system_prompt` the factory hands the builder — so naming it is a more accurate report than calling it undeclared. Keyed strictly on `plan is None`: a plan that exists and matches nothing means the plan and the prompt disagree, which IS a defect and still reports UNDECLARED. What is genuinely lost is the per-fragment breakdown, and that is carried in the segment's `detail` (`system[unassembled]`) rather than smuggled into a field whose contract is about broken declarations. The new tests are built on the real middleware and the real factory-plus-display chain, not on fixture tools — a hand-built tool list is declared by construction and can never notice that the live surface carries objects nobody stamped, which is why 9.5k green tests sat over this. Half the file is the counterweight: an undeclared first-party tool must STAY undeclared, or the fix would be an amnesty that zeroes the number by destroying the only signal the field carries. Verified end-to-end through the recorder against the pinned deepagents 0.7.4: a full desktop-shaped request reports `undeclared_tokens: 0`, with every row naming an owner someone can act on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A live packaged run recorded
undeclared_tokens: 8257on every model call — 31% of measured input — through a field whose contract says the number is EXPECTED 0 and anything above it is a first-party defect. #627 had just stopped journey CB-8 failing on it, and the composer's context meter has been showing users an "N undeclared" notice since #625.It was three unrelated defects, and the largest row was not a tool at all.
1. Ten library-installed tools — ~3,459 tokens
The declaration seam stamps tools at
factory._model_visible_tools, and the PRD-02 AST gate sweeps that same function. Butcreate_deep_agentinstalls the filesystem, todo and subagent middleware tools inside the library, so they reach the provider having passed no append site: no stamp to read, and nothing for the gate to see. A gate scoped to one function cannot fail on contributors that never enter it.ThirdPartyToolOriginsnow declares them on their author's behalf — §4.3's sanctioned exception, applied to the tool block the way it already was to the system block — resolving the owner from the tool's dispatch callable's__module__, injected asToolSchemaLedger.measure(fallback_origin=...).Two design points that are easy to get backwards:
type(tool).__module__islangchain_corewhoever wrote the tool, so falling back to it would relabel our tools as third-party text — misattribution rather than degradation.tasklooks exactly like a deepagents built-in (same name, same middleware slot) but isbuild_atlas_task_tool, ours, and would have billed ~390 resident tokens of our text to a dependency that does not contain the string. It is declared at its authoring site instead, as isweb_search.2.
web_search— ~198 tokensDeclared correctly at the factory and destroyed downstream.
wrap_tools_with_displaycopies aStructuredTool(the stamp rides along in the instance__dict__) but rebuilds any otherBaseToolviafrom_function, holding the inner in a closure — no stamp, and no attribute chain forContextOriginBinding.ofto walk. Every non-StructuredToolon the surface was silently un-declared on its way to the model. The declaration is now carried across the rebuild.3. The system block — 4,798 tokens, 58% of the total
Every row a packaged run writes carries
assembly_record_id: None: F2 prompt assembly does not run on the desktop, so there is noPromptAssemblyPlanand the whole prompt fell through as one unattributed span.Without a plan the block has exactly one first-party contributor — the
system_promptthe factory hands the builder — so naming it is a more accurate report than calling it undeclared. Keyed strictly onplan is None: a plan that exists and matches nothing means the plan and the prompt disagree, which is a defect and still reportsUNDECLARED. The lost per-fragment breakdown is carried in the segment'sdetail(system[unassembled]) rather than smuggled into a field whose contract is about broken declarations.Tests
test_context_tool_declaration_coverage.py(20 tests) is built on the real middleware and the real factory-plus-display chain, not on fixture tools — a hand-built tool list is declared by construction and can never notice that the live surface carries objects nobody stamped, which is why 9.5k green tests sat over this.Half the file is the counterweight: an undeclared first-party tool must stay undeclared, or the fix would be an amnesty that zeroes the number by destroying the only signal the field carries.
Verification
End-to-end through the recorder against the pinned deepagents 0.7.4 (the local venv lags at 0.7.1), on a full desktop-shaped request:
Full suite: 9,519 passed. The two remaining failures are pre-existing and unrelated — the local venv has deepagents 0.7.1 / langgraph 1.2.9 against pins of 0.7.4 / 1.2.10, and neither test touches anything here.
CB-8 already pins both directions ("no undeclared bytes must mean no notice"), so it will now exercise the healthy branch rather than the defect branch.
Follow-up, deliberately not in scope
Making the desktop bind the harness control plane so F2 gives real per-fragment system attribution. The coarse declaration is honest, but it is one row where the design intends a breakdown.
🤖 Generated with Claude Code