fix(platform): chat stall guard, atomic turn claim, sharing lifecycle - #3182
Draft
larryro wants to merge 5 commits into
Draft
fix(platform): chat stall guard, atomic turn claim, sharing lifecycle#3182larryro wants to merge 5 commits into
larryro wants to merge 5 commits into
Conversation
The direct model call ran under one whole-request AbortSignal.timeout of 180s that also covered body streaming, so every reply still streaming at minute three — a high reasoning effort with a large output ceiling — was cut mid-sentence and surfaced as a generic provider error. Replace the deadline with a silence clock (core/chat/stream_stall.ts): the guard restarts on every byte, so an actively producing stream is never aborted however long it runs, while a provider that stops sending for 180s still ends the round — with an error that names the stall and classifies as transient. The first byte gets the same allowance, and streamSse races every read against the clock so a hung read cannot outlive it.
beginTurn's contract is one transaction — user row, placeholder, and generation row commit together — but the Postgres store ran them as four pool statements: a crash between them left a question with no reply or a 'pending' bubble no watchdog would ever fail. The same insert also rebound an existing generation row (ON CONFLICT DO UPDATE), so two sends racing through the route's check-then-act busy gate both ran, streamed into one row, and the first to finish deleted it from under the other. Wrap the open in sql.begin and make the generation insert the claim: ON CONFLICT DO NOTHING, and no row means another turn holds the thread — the open throws ThreadBusyError and rolls back, so the loser leaves no trace and, raised before the pipeline's settle block, never runs the endGeneration that would close the winner. The send route answers it as the same 409 refusal its fast-path read gives, the arena side records no error row, the REST turn writes its busy row, and a deferred send parks itself again instead of deleting its tray row. endGeneration is one transaction too and fails a placeholder still pending after a failed finalize, so a settled turn never leaves an eternal pending bubble.
Two lifecycle leaks in thread sharing. A share link kept resolving for a thread its owner had moved to Trash (or that had aged to 'expired'): getSharedThread applied no status filter, while unshare went through the active-thread read and was a silent no-op on a trashed thread — the route still answered ok. And moving a project-shared thread to another project carried shared_with_project along, granting the new project's members the whole history with no consent and no audit row. Gate the share-link read on status = 'active' like every other read; match unshare on the owned row regardless of lifecycle and report whether it matched. A move that changes the project switches the project share off and audits the implicit unshare on the project the thread left; the owner re-shares in the new project deliberately. Docs (en/de/fr) name the rule.
ensureArenaPair copied A's agent and capabilities into column B but not its project filing, so in a project chat column A's turns carried the project instructions and knowledge while B's did not — a comparison between prompts, not models — and a winning B graduated with no project, dropping the conversation out of it and losing A's pin. Give B the project_id and reasoning_effort at birth (still hidden, so it never lists in the project's Chats tab), and on a B verdict carry A's filing, pin, and read watermark onto the survivor — project_id via coalesce so a pair opened before this fix still lands filed.
The chat shim's contact query — behind rag_search's contact leg and the assistant's list lane — selected from app.contacts with no lifecycle filter, while every read in the contacts domain excludes trashed rows, so a contact the user had deleted kept resurfacing in answers as current. Apply the same predicate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven verified medium-severity chat-domain defects from the backend deep review, all still live on
mainat 899fcc0 (post-#3157/#3158/#3164). Five atomic commits, one per seam.Per-finding outcome
core/chat/turn_action.ts:648hadAbortSignal.timeout(180_000)on the fetch, covering body streaming. Replaced by a silence clock (core/chat/stream_stall.ts).getSharedThreadhad notm.statusfilter;unshareThreadwent through the active-only read. Base harness probe: trashed link answered 200, unshare on trashed reported ok while doing nothing.moveThreadToProjectupdatedproject_idonly. Base probe:shared=truein the new project, no audit row.ensureArenaPaircopied agent + capabilities but notproject_id; settle graduated B unfiled. Base probe:B.project=false, winner unfiled + unpinned.["Zelda Trashed","Zelda Current"].sql.begin. Base probe (trigger-simulated crash at the generation insert): 2 stranded rows.ON CONFLICT (thread_id) DO UPDATE. Base probes: two concurrent opens both won (pending,pending), two concurrent HTTP sends bothcompletedwith 4 rows. Not addressed by #3164.The abort-timeout approach (1)
The model round's only clock is now a silence clock, never a whole-request deadline:
createStallGuard(180_000)aborts only after 180s with no bytes, andstreamSsecallstouch()on every chunk and races everyreader.read()against the guard, so an actively producing stream is never aborted however long it runs, while a genuinely hung one still ends — withThe model provider stopped sending data — the reply timed out after 180 seconds of silence., which the chat-error classifier buckets as transient (timed out). Headers count as first life; the first byte gets the same 180s allowance (a slow-thinking model is not a hung one). No env var, no migration.The claim (6, 7)
beginTurnruns in onesql.begin; the generation insert isON CONFLICT (thread_id) DO NOTHING RETURNING, and no row means another turn holds the thread →ThreadBusyError(new,lib/chat/turn.ts) rolls the open back. It is raised before the pipeline's settle block, so the loser never runs theendGenerationthat would close the winner. Callers: send route → 409{status:'refused'}(same shape as its fast-path read, which stays as a fast path); arena side → refused without an error row; REST turn → its existing busy error row; deferred send → re-parks the row and re-polls instead of deleting it.endGenerationis one transaction too and fails any placeholder stillpending(a failed finalize is the one remaining orphan path). No migration (0070 is taken by sibling branches; none needed here).Sharing lifecycle (2, 3) and arena (4)
getSharedThreadrequirestm.status = 'active';unshareThreadmatches the owned row regardless of lifecycle and returns whether it did — the route answers{ ok }honestly. Trash does not clearis_shared(the gate hides the link while trashed; restore returns the conversation as it was, and unshare is available throughout).shared_with_project = falseand auditsproject.thread.unsharedon the project the thread left (newState.movedToProjectId). Docsplatform/projects/concepts.md(en/de/fr) name the rule.project_id+reasoning_effort(still hidden — never lists in the project Chats tab); a B verdict carries A's filing (coalesce, so pairs opened before this fix land filed), pin, and read watermark onto the survivor.Tests
Colocated vitest (all red on base — run against a
git archive 899fcc08atree: 14 failed / 47 passed, the two stall files failing at import):core/chat/stream_stall.test.ts(5): silence clock never fires under activity, fires after a silent window, measures from the last byte, dispose/late-touch.core/chat/turn_action.test.ts(3): a healthy stream 3× longer than the window survives intact; a silent stream ends with the stall error; a user cancel passes through as itself.domains/chat/store.test.ts(+3): beginTurn inside ONEbeginwith NOTIFY after commit;DO NOTHINGclaim →ThreadBusyError+ rollback; endGeneration one tx failing a pending placeholder.lib/chat/turn.test.ts(+1): a busy open propagates and the loser never callsendGeneration.domains/chat/threads.test.ts(7): active-only share lookup; unshare without a lifecycle gate + honest boolean; move ends the share and audits (and not when the project is unchanged / never shared).domains/chat/arena.test.ts(2): B's birth carriesproject_id/reasoning_effort; the B-wins graduation copies filing/pin/read state.domains/chat/shim.test.ts(1): contacts query carries the lifecycle predicate.Integration harness (
backend/integration-check.ts), 7 new probes: share link 404 in trash + unshare works in trash + restored thread unshared; move ends share + audit row + new project's tab shows unshared; arena Bproject_id/effort + hidden from tab + B-wins filed and pinned; trashed contact absent from the shim query; trigger-simulated crash at the generation insert strands nothing; two concurrentbeginTurn→ one wins, oneThreadBusyError, 2 rows, endGeneration fails the orphan placeholder; two concurrent HTTP sends on a slow-drip provider →completed/refused,200/409, exactly one full exchange.Verification (observed)
bunx tsc --noEmit(platform): exit 0.bun run --filter @tale/platform lint: exit 0.bun run --filter @tale/platform test: 439/442 files, 72,742 tests pass; the 3 failing files areapp/routes/**suites failing to load withDenied ID …@fontsource…woff2— reproduced identically on the base tree (symlinkednode_modulesworktree artefact, unrelated).bun run --filter @tale/docs test: 30 files / 194 tests pass.backend:integrationon fresh tale-db + MinIO per run,SANDBOX_LLM_GATEWAY_ADMIN_PASSWORDset: branch 386 PASS / 0 FAIL (exit 0); base 379 PASS / 0 FAIL on its own probe set; base code + this branch's harness: 379 PASS / 7 FAIL — the seven failures are exactly the new probes.Cross-class discoveries (not in this batch)
app.messageshas noUNIQUE (thread_id, "order", step_order); with the claim in place the turn path can no longer race, but out-of-turn appenders (REST failure rows, arena side-error rows) still computemax(order)+1unguarded. Adding the index needs a dedupe backfill of existing ties first → own migration (0071+).pendingplaceholders orphaned before this fix (no generation row) are not swept; a periodic sweep would want a partial index onapp.messages (status) WHERE status = 'pending'.serverproject in a symlinked-node_modulesworktree cannot load 3app/routes/**suites (@fontsourcewoff2Denied ID) — environmental, present on base.