Skip to content

Summarize: all chunks share one budget, so smaller chunks make large sessions fail faster, and failing sessions retry forever #9

Open

Description

@stevepresley

Problem

All chunks of a summarize run share one time budget, so splitting a session into smaller chunks makes a large session fail faster, and sessions that cannot fit are retried and fail again.

src/functions/summarize.ts measures every chunk call against one run budget from getSummaryTimeoutMs() (the boundedProvider, lines 318 to 335). A session of N observations at chunk size c makes N / c chunk calls (SUMMARIZE_CHUNK_CONCURRENCY at a time), then one reduce call over all the partial summaries. All of it must finish inside the one budget.

Observed (2026-11-14, 0.9.31 with the hard-coded 120 s budget, chunk size 1)

  • A 179-chunk session finished about 10 chunks and lost 169: too_many_chunks_skipped: 169/179, every one summary_timeout.
  • Sessions queued at 1,268, 732 and 135 chunks.
  • Sessions too large to fit were retried repeatedly: a read of 0.9.31 shows a retry backoff of 30 s to 60 min (not re-checked in main). The status then showed 3,056 summarize calls against 49 sessions (119 successes).

Requested

  • Give each chunk its own budget, or scale the run budget with the chunk count.
  • Persist partial summaries so a retry resumes from the last finished chunk.
  • Stop re-queuing a session at the same size and budget that already failed. Report "cannot summarize: needs about N chunks or tokens" instead.

Acceptance

  • A session larger than one budget completes across retries without redoing finished chunks.
  • A session that cannot complete is not retried indefinitely, and the dashboard says why.
  • Tests with a slow stub provider.

Related

The budget is configurable in 0.9.33 (AGENTMEMORY_SUMMARY_TIMEOUT_MS, upstream rohitg00#1689). This issue is about how the budget is spent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions