Skip to content

fix(runtime): keep the execution budget reminder in history - #538

Open
yujiachen-y wants to merge 1 commit into
mainfrom
fix/execution-budget-reminder-durable
Open

yujiachen-y wants to merge 1 commit into
mainfrom
fix/execution-budget-reminder-durable

Conversation

@yujiachen-y

@yujiachen-y yujiachen-y commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Change

mcode exec --timeout adds an execution budget reminder to model requests. The reminder was request-only: it was appended after the last user message and never written to history. The Anthropic adapter puts the conversation cache breakpoint on the last user message, so every request that carried a reminder wrote its cache entry ending at a message the next request no longer contained. Outside the final two minutes this cost about one step of re-written context per reminder. In the final two minutes every request carried a reminder, so cache reads stopped advancing and the uncached tail grew with every step; once it falls out of the provider's lookback window, every request re-writes the whole conversation history, exactly when the run is shortest on time.

The reminder could not simply be written to history because runBeforeLLM ended the decision pipeline at the first appendMessage, which would have starved the background, session identity and todo reminders. This PR therefore changes both:

  • Before-LLM pipeline (agent-core): appendMessage decisions accumulate instead of ending the pipeline, and later hooks see the registered markers. Everything is committed once after the pipeline: at most one durable replacement carrying the before-current-user markers, then every tail marker in registration order with a single history flush. A durable replaceMessages after a registered append fails closed (production registers compaction before every appending reminder). This also stops the existing background reminder from skipping the request filter, session identity and todo reminders.

  • Multiple placed markers: compaction commit metadata and the canonical tail validation accept more than one marker before the current user through a runner-owned placedMarkerCount. Existing metadata without it is read as one marker.

  • Execution budget reminder: it is now a hidden durable custom marker (execution_budget_reminder, display: false) appended like the other reminders, so each provider request is an append-only extension of the previous one. The model-visible text, thresholds and position are unchanged. Progress is derived from the latest reminder of the same deadline in history instead of hook instance state: a repeated request does not duplicate it, a compaction that summarized reminders away announces the budget again, and in the final two minutes a reminder is only appended when its 10-second text changes.

  • Tests: the BYOK cancellation tests run mcode exec --timeout, so their persisted history now also contains the hidden budget reminder; they assert it explicitly.

Labels: bug, cli, perf:full (changes history construction in the tool loop).

Validation

  • New execution-budget-cache-prefix.integration.test.ts runs a real PiTurnRunner tool loop with the budget hook and a fake clock across the first announcement, quiet requests, the 50% and 25% thresholds and the final two minutes (including two requests in the same 10-second step), and asserts that every provider request starts with the previous one. It fails on main at the first reminder request and passes with this change.
  • Reverse checks on this branch: restoring the main reminder makes 22 tests fail (integration 1, reminder unit 12, executor 9); restoring the main runBeforeLLM makes 7 executor tests fail. Both pass again after restoring this change.
  • Focused Vitest: execution budget reminder 13/13, cache-prefix integration 1/1, executor 159/159, context compaction (including new two-marker metadata cases), session identity reminder — 246 passed.
  • pnpm verify (full profile, darwin/arm64, Node 22.22.2) on 156f206: PASS for every applicable gate (source, tsconfig, export preview, release tools, TUI lint, typecheck, build, standalone, artifact, capabilities, status contract, smoke, BYOK, policy, sandbox); test:windows and test:release-package are skipped on this platform. Run with the proxy environment variables unset, because this machine's NODE_USE_ENV_PROXY makes child Node processes print warnings that the unrelated local-bash-output test counts as output lines.
  • Live check (offline evidence above is the gate): the same change, run through mcode exec --timeout 3m against an Anthropic-protocol model, kept the previous request's breakpoint prefix on every request (12/12) and cache reads advanced on every request in the final two minutes (cache writes stayed at 105–242 tokens per request); the request-only reminder lost the prefix on every reminder request (6/10) and cache reads stayed flat while cache writes grew 105 → 500 tokens over the final-window requests. This was a single 3-minute sample with a ~19k-token context.
  • Performance: perf:full requested.
  • NOT RUN: Windows and Linux locally (covered by CI), long runs whose uncached tail exceeds the provider lookback window, and live providers other than the Anthropic protocol.

Publication and contribution checks

  • I have permission to contribute these changes under the existing licenses applicable to the changed files/packages; imported material and its provenance are identified and existing notices are preserved.
  • No credentials, account data, real user content, internal source history or private review material is included.
  • Added/removed source files were reviewed before regenerating release/public-source.json; new tests are declared in test/vitest-suites.json where applicable.
  • Shared English/Chinese documentation and capability/verification records are updated where applicable. Mock/offline results are not described as live-service acceptance.

Maintainer handoff

Publication scope or license changes (if any): none.

Shared-source port: complete.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

The execution budget reminder was a request-only marker appended after the
last user message. The Anthropic adapter places the conversation cache
breakpoint on that message, so each reminder request wrote a cache entry that
ended at a marker the next request no longer contained. In the final two
minutes every request carried such a marker, so cache reads stopped advancing
and the uncached tail grew with every step until it fell out of the lookback
window and the whole history was written again.

runBeforeLLM now accumulates appendMessage decisions instead of ending the
decision pipeline at the first append, so the background, session identity
and todo reminders and the request filter no longer skip each other. All
markers are committed once after the pipeline: at most one durable
replacement carrying placed markers, then every tail marker in registration
order with a single history flush. A durable replacement after a registered
append fails closed. Compaction commit metadata and canonical tail validation
accept more than one placed marker through a runner-owned placedMarkerCount
(absent means 1, so existing metadata stays readable).

The execution budget reminder becomes a hidden durable custom marker appended
like the other reminders, so every provider request is an append-only
extension of the previous one. Progress is derived from the latest reminder
of the same deadline in canonical history: a repeated request does not
duplicate it, a compaction that summarized reminders away announces the
budget again, and in the final window a reminder is only appended when its
10-second text changes. The model-visible text, thresholds and position are
unchanged.
@yujiachen-y yujiachen-y added bug Something isn't working cli Standalone mcode: TUI, headless, ACP and source builds/tooling perf:full Run the full performance suite; required for performance-sensitive changes labels Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cli Standalone mcode: TUI, headless, ACP and source builds/tooling perf:full Run the full performance suite; required for performance-sensitive changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant