Repository navigation
fix(runtime): keep the execution budget reminder in history - #538
Open
yujiachen-y wants to merge 1 commit into
Open
yujiachen-y wants to merge 1 commit into
yujiachen-y wants to merge 1 commit into
Conversation
The execution budget reminder was a request-only marker appended after the last user message. The Anthropic adapter places the conversation cache breakpoint on that message, so each reminder request wrote a cache entry that ended at a marker the next request no longer contained. In the final two minutes every request carried such a marker, so cache reads stopped advancing and the uncached tail grew with every step until it fell out of the lookback window and the whole history was written again. runBeforeLLM now accumulates appendMessage decisions instead of ending the decision pipeline at the first append, so the background, session identity and todo reminders and the request filter no longer skip each other. All markers are committed once after the pipeline: at most one durable replacement carrying placed markers, then every tail marker in registration order with a single history flush. A durable replacement after a registered append fails closed. Compaction commit metadata and canonical tail validation accept more than one placed marker through a runner-owned placedMarkerCount (absent means 1, so existing metadata stays readable). The execution budget reminder becomes a hidden durable custom marker appended like the other reminders, so every provider request is an append-only extension of the previous one. Progress is derived from the latest reminder of the same deadline in canonical history: a repeated request does not duplicate it, a compaction that summarized reminders away announces the budget again, and in the final window a reminder is only appended when its 10-second text changes. The model-visible text, thresholds and position are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
mcode exec --timeoutadds an execution budget reminder to model requests. The reminder was request-only: it was appended after the last user message and never written to history. The Anthropic adapter puts the conversation cache breakpoint on the last user message, so every request that carried a reminder wrote its cache entry ending at a message the next request no longer contained. Outside the final two minutes this cost about one step of re-written context per reminder. In the final two minutes every request carried a reminder, so cache reads stopped advancing and the uncached tail grew with every step; once it falls out of the provider's lookback window, every request re-writes the whole conversation history, exactly when the run is shortest on time.The reminder could not simply be written to history because
runBeforeLLMended the decision pipeline at the firstappendMessage, which would have starved the background, session identity and todo reminders. This PR therefore changes both:Before-LLM pipeline (
agent-core):appendMessagedecisions accumulate instead of ending the pipeline, and later hooks see the registered markers. Everything is committed once after the pipeline: at most one durable replacement carrying thebefore-current-usermarkers, then every tail marker in registration order with a single history flush. A durablereplaceMessagesafter a registered append fails closed (production registers compaction before every appending reminder). This also stops the existing background reminder from skipping the request filter, session identity and todo reminders.Multiple placed markers: compaction commit metadata and the canonical tail validation accept more than one marker before the current user through a runner-owned
placedMarkerCount. Existing metadata without it is read as one marker.Execution budget reminder: it is now a hidden durable custom marker (
execution_budget_reminder,display: false) appended like the other reminders, so each provider request is an append-only extension of the previous one. The model-visible text, thresholds and position are unchanged. Progress is derived from the latest reminder of the same deadline in history instead of hook instance state: a repeated request does not duplicate it, a compaction that summarized reminders away announces the budget again, and in the final two minutes a reminder is only appended when its 10-second text changes.Tests: the BYOK cancellation tests run
mcode exec --timeout, so their persisted history now also contains the hidden budget reminder; they assert it explicitly.Labels:
bug,cli,perf:full(changes history construction in the tool loop).Validation
execution-budget-cache-prefix.integration.test.tsruns a realPiTurnRunnertool loop with the budget hook and a fake clock across the first announcement, quiet requests, the 50% and 25% thresholds and the final two minutes (including two requests in the same 10-second step), and asserts that every provider request starts with the previous one. It fails onmainat the first reminder request and passes with this change.mainreminder makes 22 tests fail (integration 1, reminder unit 12, executor 9); restoring themainrunBeforeLLMmakes 7 executor tests fail. Both pass again after restoring this change.pnpm verify(full profile, darwin/arm64, Node 22.22.2) on156f206: PASS for every applicable gate (source, tsconfig, export preview, release tools, TUI lint, typecheck, build, standalone, artifact, capabilities, status contract, smoke, BYOK, policy, sandbox);test:windowsandtest:release-packageare skipped on this platform. Run with the proxy environment variables unset, because this machine'sNODE_USE_ENV_PROXYmakes child Node processes print warnings that the unrelatedlocal-bash-outputtest counts as output lines.mcode exec --timeout 3magainst an Anthropic-protocol model, kept the previous request's breakpoint prefix on every request (12/12) and cache reads advanced on every request in the final two minutes (cache writes stayed at 105–242 tokens per request); the request-only reminder lost the prefix on every reminder request (6/10) and cache reads stayed flat while cache writes grew 105 → 500 tokens over the final-window requests. This was a single 3-minute sample with a ~19k-token context.perf:fullrequested.Publication and contribution checks
release/public-source.json; new tests are declared intest/vitest-suites.jsonwhere applicable.Maintainer handoff
Publication scope or license changes (if any): none.
Shared-source port: complete.
Need help on this PR? Tag
@codesmith-botwith what you need.