Skip to content

fix(providers): honor the configured LLM timeout; triage upstream against this tree - #50

Merged
possiblyneal merged 6 commits into
mainfrom
fix/llm-timeout-hard-cap
Sep 22, 2026
Merged

possiblyneal merged 6 commits into
mainfrom
fix/llm-timeout-hard-cap

Conversation

@possiblyneal

@possiblyneal possiblyneal commented Sep 22, 2026 •

Copy link
Copy Markdown
Owner

Summary

Base branch of the four #31 cluster PRs (#45, #46, #47, #48). Merge this first so each cluster's diff against main shrinks to its own commits.

  • Honor the configured LLM timeout (d006520, 2c68018). fetchWithTimeout clamped every budget to a 170s HARD_BUDGET_CAP_MS while openai.ts still reported the unclamped value in its error, so AGENTMEMORY_LLM_TIMEOUT_MS=480000 aborted at 170s and claimed 480000ms. The cap guarded iii's 180s invocation limit; ADR 0001 removed that runtime, so the in-process Engine has no such limit. The caller's timeout is now the total budget for all attempts plus sleeps, honored exactly. test/fetch-timeout.test.ts asserts the full bound on the first attempt and that a retry attempt gets only the remaining budget, never a fresh ms.
  • Searchable sdk.ts and graph.ts (e6c0cbd, 2c68018). Both joined a map key with a raw NUL byte, which made git and grep treat the files as binary; Repair round: token-based summarize chunking, viewer/scrub/context hardening, hook env, API bounds #31 names graph.ts as the file needing grep -a. Each now uses a visible \u001f escape. Neither key is persisted: sdk.ts's is an in-memory stream-item Map key, graph.ts's is a function-local dedup Map key and nodes get generated ids.
  • Docs (04a9213, 304c40a, 01bd9e3, 2c68018). CLAUDE.md opens with the goal the memory system serves and records the fetchWithTimeout rule with a pointer to ADR 0001; ADR 0001 gains a consequence naming iii-derived ceilings as defects; docs/upstream-issue-triage.yaml and docs/upstream-pr-triage.yaml score upstream's open issues and orphan PRs against this tree with a disposition per row, the verdict fields hand-written; CONTEXT.md defines Session Summary.

Review notes

Standard-scan review at 2c68018. Left as follow-up (#51): two comments still budget against iii's 180s timeout (src/functions/export-import.ts, .env.example SUMMARIZE_CHUNK_CONCURRENCY); src/functions/summarize.ts's copy is rewritten by #45. Pre-existing and unchanged: fetchWithTimeout parses AGENTMEMORY_LLM_TIMEOUT_MS leniently ("30s" reads as 30ms) while openai.ts requires digits for OPENAI_TIMEOUT_MS.

Test plan

  • npx vitest run test/fetch-timeout.test.ts (29, including the retry-budget test, which fails when the clamp regresses to ms)
  • npm test — 1890 passed, 1 skipped
  • npm run build
  • grep -rl -P '\x00' src/ is empty

Refs #31, #43.

🤖 Generated with Claude Code

neal added 6 commits September 21, 2026 15:54
fetchWithTimeout clamped every budget to a 170s HARD_BUDGET_CAP_MS while
openai.ts still reported the unclamped value in its timeout error, so an
AGENTMEMORY_LLM_TIMEOUT_MS of 480000 aborted at 170s and claimed 480000ms.
Broker-side records confirm it: 86 client-disconnect 499s on the general
route, p50 duration 169,996ms, p90 170,004ms, none past 400s.

The cap existed to stay under iii's 180s invocation timeout. ADR 0001
deleted that runtime — the in-process Engine has no invocation timeout, so
the ceiling bounded against a limit that no longer exists and turned slow
local models into provider failures, which then tripped the circuit breaker
and failed unrelated calls.

The caller's timeout is now the total budget for all attempts plus sleeps,
honored exactly. test/fetch-timeout.test.ts asserted the clamp; it now
asserts the opposite.

Generated-By: Claude Opus 5 (1M context)
groupKey joined its two parts with a raw NUL byte. One NUL is enough for
git and grep to classify the whole file as binary, so every search over
src/engine/inproc/sdk.ts silently returned nothing and `file` reported it
as data rather than source.

U+001F separates just as well. The key is only ever a streamItems Map key
and is never persisted, so nothing migrates.

Generated-By: Claude Opus 5 (1M context)
CLAUDE.md opened straight into architecture, so every feature argument
started from mechanism. Lead with the goal instead: right Memory, right
moment, right scope, least record that restates it.

Generated-By: Claude Opus 5 (1M context)
Score every open upstream issue on severity x difficulty, cross-link the PRs
that claim to fix each one, and score the 168 PRs that reference no open issue
by how much of their diff lands in files this fork tracks.

Every row carries a disposition: wont-fix for what is out of scope here (no iii
engine, no npm publish, no Windows CI, no deploy tree, no adapter for the host),
already-fixed for what this tree already carries, candidate for the working set.

The 17 high-severity/low-difficulty issues and the 35 high-relevance candidate
PRs were then opened against this tree and read. Rows checked that way carry a
status whose leading token says what the read found; 15 claims turned out to be
already fixed here and moved out of the candidate set. A row with no status is
still upstream's claim, not a verified defect.

Generated-By: Claude Opus 5 (1M context)
Generated-By: Claude Fable 5.1
…drop the NUL in graph.ts

Code review of #50 (standard scan):

- CLAUDE.md: the fetchWithTimeout rule states the contract and cites ADR 0001
  instead of narrating the removed cap; the env precedence now says which
  provider resolves OPENAI_TIMEOUT_MS; the triage-file notes describe the
  hand-verified fields and list wont-fix reasons as examples
- CONTEXT.md: a Session Summary is produced each time the Session stops, not
  only at its end
- sdk.ts: GROUP_KEY_SEPARATOR = "\u001f" as a visible escape, not a raw byte
- graph.ts: the node-key separator was still a literal NUL, the file #31
  names; same visible escape, key is a function-local Map key
- _fetch.ts and its test: rationale cited once from ADR 0001; new test pins
  that a retry attempt is bounded by the remaining budget, not a fresh ms
- agent-sdk.ts: "cap" -> "deadline" now that no cap exists

Generated-By: Claude Fable 5.1
@possiblyneal

Copy link
Copy Markdown
Owner Author

Code review — standard scan (Standards, Spec, Correctness/Logic)

Fixed point: PR #50, base main, head fix/llm-timeout-hard-cap. Reviewed at 01bd9e3; corrections landed in 2c68018; every sub-agent signed off on the fixes to its own findings.

Standards

1 hard (fixed), 5 judgement. Hard: the CLAUDE.md fetchWithTimeout bullet narrated the removed 170s cap; it now states only the rule and cites ADR 0001, where the history lives. Judgement: GROUP_KEY_SEPARATOR = "\u001f" replaces an invisible byte in sdk.ts, and graph.ts's NUL takes the same visible escape; the no-clamp rationale is cited once from ADR 0001 in _fetch.ts and its test; the CLAUDE.md wont-fix reasons are worded as examples led by upstream-only housekeeping. Accepted as-is: branch packaging (this is the shared base of #45–#48); three surviving iii-derived comments go to #51, with summarize.ts's copy rewritten by #45. No "load-bearing" in the diff; triage YAML structure matches its CLAUDE.md description.

Spec

0 hard, 5 judgement, all resolved. graph.ts, the file #31 names for grep -a, is now NUL-free (Operator chose to fix it here); the PR body discloses the ADR 0001 edit; CLAUDE.md separates the row set refreshed from gh issue list from the hand-written verdict fields, consistent with #43; CONTEXT.md says a Session Summary is produced each time the Session stops; the spec's "every upstream claim carries status" is left as a loose spec sentence. Pre-existing, not introduced here: CLAUDE.md is CRLF.

Correctness/Logic

0 definite, 3 suspicions, all resolved. Removing budgetMs was semantically clean (the old Math.min(ms, …) was dead code) and both deadline checks measure from one start. New test pins that a retry attempt is bounded by the remaining budget: alive at 59999ms, aborted at 60001ms on a 60000ms budget, and it fails when the clamp regresses to ms. CLAUDE.md now attributes OPENAI_TIMEOUT_MS to the OpenAI LLM provider. agent-sdk.ts says "deadline" not "cap". Both separator keys are in-memory only. Lenient parseInt on AGENTMEMORY_LLM_TIMEOUT_MS is pre-existing and noted in the body. Full suite: 1890 passed, 1 skipped.

Totals: Standards 1 hard fixed/5 judgement, Spec 0/5 resolved, Correctness 0/3 resolved. Worst 3: none outstanding.

Reviewed-Head: 2c68018

@possiblyneal
possiblyneal merged commit 5904f1f into main Sep 22, 2026
6 checks passed
@possiblyneal
possiblyneal deleted the fix/llm-timeout-hard-cap branch September 25, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant