fix(platform): wire chat guardrails and policies and harden the turn - #3230
Merged
Conversation
Five modules exported helpers nothing in the repo reaches: - core/governance/file_utils.ts kept nine JSON/retention/secret-sidecar helpers "for historical migrations" that do not exist in this tree (0.5 migrations are SQL; guardrail secrets are app.governance_secrets rows). The header now describes the live yml layout. (governance-5) - domains/governance/service.ts readUsageBuckets / UsageBucketRow were called only by the integration harness and carried an unbounded org-wide scan; the harness reads its one bucket directly. (governance-8) - lib/chat/untrusted-content.ts containsSuspiciousInjection guarded tool inputs that no production tool has; the pointer comment in sanitize-untrusted-field.ts sent readers to it. (lib-chat-6) - lib/shared/chat-errors.ts PROVIDER_SCOPED_ERROR_CODES and buildHumanErrorSentence served a failover contract the 0.5 backend never invokes; the module doc named a classifyFailureScope that never existed. (chat-core-7) - domains/threads/store.ts getThread, ThreadRow and listThreadMessages had no reader — the chat domain keeps its own. (chat-core-8) Tests that only pinned the removed symbols go with them.
executeTurn built a Convex-era TurnStore/UsageLedger pair on every turn that its only host (runChatTurn) always overrode with the Postgres ports; the seven internal.chat.* / incrementUsageLedger names those ports dispatched had no shim handler anywhere, and shim.test.ts carried a replacedModules hole to hide it. The ports are now REQUIRED overrides, core/chat/turn_store.ts (with the never-called settleDeferredSendOnUserAppend decorator) is gone, the names leave handler_names.ts, and the reachability gate walks the host with no exclusion. (chat-core-5) ExecuteTurnArgs.sandbox only ever carried false: executionMode was a constant 'direct', the CHAT_EXECUTION_UNAVAILABLE throw in the direct model call could not fire, and loadHarnesses() ran per turn to feed a table the direct arm never consults. The host now states the direct lane plainly; TurnDeps.harnesses is optional (empty for a direct-only host) so the pipeline's sandbox seam stays what the task-agent hosts use, with its tests unchanged. (chat-core-6, lib-chat-7)
…at turns
The Guardrails and Custom-instructions settings were saved, audited and
displayed but never reached a turn: executeTurn built TurnDeps with no
inputFilters/outputFilters, TurnRequest.mandatoryInstructions was never
set, app.chat_filter_events had no producer, ModerationBackend had no
implementation, the stored moderation auth header had no reader, and
POST /moderation/test answered a permanent "offline while the platform
AI backend is rewritten" stub for a rewrite that shipped. (governance-1,
governance-2, governance-3, lib-chat-2, lib-chat-3)
- core/chat/guardrails.ts reads chat_filter / pii_config /
moderation_provider / system_prompt through the getPolicyConfigInternal
seam in the turn's parallel read slot, builds the chain (createChatFilter,
the PII scrubber — or the new createPiiTokenizeFilter round trip for
tokenize mode, which restores the tokens on the way out — and a
ModerationBackend over the new runModerationProvider seam), applies the
policy's fail behaviour, and writes one chat-filter event per non-pass
verdict through recordChatFilterEvent. The mandatory instructions become
the first system-prompt block.
- lib/chat/guardrails.ts gains an onOutcome observer on the chain (never
changes the verdict) and the shared moderation-run types; runTurn's
refusal now persists the user's message before the blocked reply so the
transcript shows what was refused (not on a regenerate).
- domains/governance/moderation.ts ports the 0.4 provider client onto
safeFetch + app.governance_secrets: JSON-safe template substitution, the
{{secret}} header, one retry on 5xx/429/network/timeout, the four response
shapes, category mappings, and a per-process circuit breaker. The same
path serves POST /moderation/test, which now round-trips the text and
reports not_configured honestly.
- domains/governance/shim.ts hosts the three seams; the chat shim spreads
it. The never-read inputGuardrailsActive flag (three policy reads per
feature-flags call, no app consumer) is dropped from the route, the app
contract and the harness.
- The integration proof sends a banned word (refused before the model,
user row + blocked row + event), then a message with an email under a
mask policy and a system_prompt policy (the wire carries [EMAIL] and the
mandatory text, a pii event lands), and round-trips the moderation probe
through a loopback provider checking the stored header.
…w lands
resolveDirectWire — and with it every resolveProviderCredential refusal
(CREDENTIAL_NONE_CONFIGURED / DISABLED / KEY_ROTATED / ENV_UNSET) plus
CHAT_CREDENTIAL_UNSUPPORTED and CHAT_PROVIDER_ENDPOINT_MISSING — ran
lazily on the model call's first chunk, inside runTurn's try, after
beginTurn had committed the user row and the placeholder. For every
catalog connector the fault therefore persisted a user message and a
failed bubble reading "An unexpected error occurred", with the remedy
buried in a JSON technical-details blob, on every send until an admin
noticed; six of the seven SERVING_REFUSAL_CODES the route promises to
show were unreachable. (chat-core-2)
The wire is now resolved in executeTurn right after the model, ahead of
the history read and of any write, so the fault throws its own code and
the send route answers {status:'refused', reason} with no rows.
createDirectModelCall takes the resolved wire.
classifyChatErrorCode reads an AppError's data.code / data.message (its
message is the serialized payload the regexes could never match) and
maps the credential codes to missing_api_key / auth_error; runTurn stores
the refusal sentence, not the JSON blob, as the failed turn's raw text
(describeChatError).
The integration proof disables the chat connector's default credential
and sends: refused, naming the disabled credential, zero message rows.
…rrow rag_search advertises kind="mail-attachment" as a search narrow, but no corpus leg named the kind: with the narrow every runLeg guard was false, the search ran nothing and answered "No matches — do not re-run" for an attachment the documents corpus held. Mail hits were also labelled kind "document" on the unnarrowed fan-out, so search and list spoke two vocabularies for the same row. (chat-core-3) The corpus leg now serves document, mail-attachment and web-page: both document kinds read the documents corpus, each hit is labelled by its provenance (a conversation id marks the mail attachment), a narrow keeps only its own kind, and `sources.mailAttachments` reports the leg like the others (searched / no matches / unavailable / denied).
…open A Stop that landed at the tool-round boundary (or a crash mid-round that the watchdog later failed) persisted the round's tool-call parts with no tool-result. The history read replays every non-pending row verbatim, and both wire dialects reject a tool call without its result, so one such row failed every later send and regenerate on the thread with a provider 400 the user had no way to repair. (lib-chat-1) - runTurn's boundary-cancel branch now records one stopped tool-result per settled call (TOOL_CALL_STOPPED_OUTPUT) before it breaks, and streamWithOutputGuardrails reports a cancel its final flush observed, so the loop never settles a round of calls for a Stop it already knows. - explodeAssistantMessage answers any replayed call whose result never landed with the same interrupted payload right after its call — this heals rows persisted before this change and the watchdog's failed rows.
The capability surface's knowledge port searched the whole org corpus
with no access scope ("an organization API key already speaks for the
whole org"). The REST/MCP door binds a key to its minting user and admits
any non-disabled member role, and the MCP developer gate covers only the
persisting tools, so a Developer-minted key could pull passages from team
libraries and projects its holder cannot open, and from other people's
thread uploads — unlike every other surface the same person has.
(lib-chat-4)
get_knowledge now resolves the holder's scope through the one resolver
the chat tools use (resolveAccessScope: teams + the org pseudo-team,
readable projects, the hub) and forwards it, stamped with the holder, to
searchKnowledgeForOrg; a key has no thread, so no thread uploads.
getOrgUsageMetricsPg read up to 20k+1 ledger buckets with no ORDER BY, so an org over the cap folded whichever heap pages Postgres handed back first — a subset that shifts after a vacuum or an update, and one that could drop the current period the summary cards are about. (governance-9) The scan now orders by period_key DESC (served by the (org_id, period_key) index), so the capped page is the newest window and the same on every call.
external_turn_shared.ts read the orphan-window override from the environment but no .env.example line or docs page named it, so an operator whose long agent turns were reaped had no discoverable knob. (chat-core-10) .env.example carries a commented default with the sliding-window semantics; the environment reference lists it in every locale.
generateThreadTitleImpl raced the model call against a 10s timer but never cancelled the loser: createBuilderModel had no abort plumbing and ran on the client's fixed 180s request timeout, so a slow provider kept working (and billing) for up to three minutes on every new thread while its late reply — and its usage — were discarded. (chat-core-9) safeFetch/safeFetchBinary take a caller `signal` (torn down at once as kind `aborted`, refused before any request when already aborted), BuilderModelArgs forwards one, and the title race's timer now fires an AbortController alongside the fallback title.
The generic policy save, the retention policy save, the retention shortening cancel and the DSAR tightening each wrote the policy file FIRST and then opened the transaction that records the hash-chained audit row (and, for retention, deletes or stages the pending row). A transaction failure therefore left a policy in force that the tamper-evident chain knew nothing about — and the retention cancel was worse: the revert was visible on disk while the pending shortening row survived to be applied anyway. `previousState` also came from the 15-second TTL cache, so two admins saving inside the window each audited the cached config, not the file they replaced. (governance-10) The file is now written LAST, inside the transaction, at all four sites: a write failure rolls the audit row back and a transaction failure never leaves an unaudited file. readGovernancePolicy(ForOrg) takes `fresh: true` so a writer reads the file as it is; writeGovernancePolicyFile is idempotent (a serializable retry that finds its own content on disk neither snapshots nor rewrites), so a re-run callback is safe.
The governance policy door accepted four policy types nothing read: `run_code` had a navigable admin editor (Settings > Governance > Run-code packages), a docs page, a screenshot, two video scenes and an e2e spec, but no run_code tool and no reader consulted the file; `agent_jobs` and `model_sync` were accepted by the generic save route and were inert; `agent_workforce` survived only in backend/core/governance/schema.ts as a "drop next release" literal. Three lists of policy types had drifted apart (POLICY_TYPES, GOVERNANCE_POLICY_TYPES, the app contract union). Remove the editor, its route, helper and test, the four schemas from POLICY_TYPES / POLICY_SCHEMAS, and converge the two copies on the shared list. Sweep the product surface that promised the feature: docs page in en/de/fr (redirected to the sandbox hardening page), nav, frontmatter and image manifests, the screenshot manifest and the SCREENSHOTS.md tile, the run-code scenes of the connectors and developers video episodes and their chapter tables, the e2e spec, the manual test plan, the locale catalogs and the marketing copy. Findings: governance-6, lib-shared-schemas-5, lib-shared-schemas-6, lib-shared-schemas-8.
…ions The capability registry declared five kinds (builtin, connector-action, skill, automation, mcp-tool) but the only host, domains/chat/capabilities, ever registered automations; the other four backends were "unavailable" stubs, createConnectorBackend and mcpToolsToCapabilities had no caller, the event-only refusal could not trigger (the store lists no triggers) and capabilityDocs was unreachable. The MCP tool description and the docs promised a search over "connector actions, skills and tools". Keep the one kind that exists, drop the dead adapters, the credential pass-through and the knowledge "not available yet" placeholder (the host always installs a backend), tell the truth in the MCP tool description and the docs, and lift lib/chat out of knip's ignore list so the next orphan is reported. Findings: lib-chat-5.
With lib/chat back under knip's eye (the parking entry left with the capability-surface narrowing) the audit reported 51 exports with no importer: forty barrel re-exports every consumer reaches by module path, five turn-pipeline steps and a poll interval only turn.ts calls, two token estimators only types.ts calls, the tool-result bounder only context.ts calls, a tool-name guard and a rag_search action type with no caller at all. Remove the barrel lines and the export keywords, delete the two callerless declarations. Behaviour unchanged; knip is green again. Findings: lib-chat-5 (follow-through).
Three lanes of the integration proof asserted the wrong thing once the
guardrail chain and the credential refusal became real:
- The disabled-default-credential lane wanted the refusal to name the
disabled credential. Serving reads the ACTIVE default only (the
documented contract in domains/provider_credentials), and the send
body carries no credential id, so the fault the composer can reach is
CREDENTIAL_NONE_CONFIGURED: the lane now expects that sentence, with
the refusal still landing before any row (rows=0).
- The moderation probe stored the whole "Bearer …" value as the secret
while its endpoint template — like the shipped presets and the docs'
"API key" — already prefixes `Bearer {{secret}}`, so the mock provider
saw "Bearer Bearer …". The lane stores the bare key, expects the masked
status to start with it and never leak its tail, and the provider to
see exactly one scheme.
- The chat-filter events listing counted every event in the org; the
guardrail lane earlier in the same run now legitimately records two
real ones (a chat_filter block, a pii detection), so the listing is
judged on the rows this section seeds, and each filtered listing on
keeping only its own kind or filter across the org.
Findings: governance-1, governance-3, chat-core-2 (proof lanes).
The guardrail wiring (78e6e6b) made the pipeline persist a pre-model refusal — the user row plus a blocked assistant row — but the send route answered `{status:'refused', reason}` exactly as it does for a zero-row refusal, and the client treats every refusal as "nothing landed": it put the text back into the composer, re-staged the attachments, unbound the video jobs and toasted, while the invalidation ALSO painted the two rows into the thread. Pressing Send again duplicated the message. `TurnOutcome`'s refused variant now carries `persisted`: true for every refusal the pipeline itself makes (input block, output block, failed stream — all on the record), false for the host's pre-pipeline refusals (model resolution, access policy, nothing to regenerate). The send and arena routes forward it on every refusal (busy, draining and serving refusals say false explicitly); the client contract reads it and the composer restores its text, attachments and video chips ONLY when the refusal wrote nothing. The PG store's appendMessage now NOTIFYs the thread stream — a refusal's two rows land through that write alone, so other viewers had no signal until a later invalidation. Review of PR #3230, blocking item 1.
lib-shared-schemas-8 named three hand-kept copies of the policy-type list; ecdb816 removed the retired literals from all three but left the copies, and they were still apart: `approval_policy` is in POLICY_TYPES and readable by the backend route, but missing from GOVERNANCE_POLICY_TYPES (backend/core/governance/schema.ts) and from the app contract's `getPolicy` union — so the app could not ask for it without a cast, and nothing pinned the lists together. backend/core/governance/schema.ts is gone; loader-preload.ts and the governance hooks type their policy type as `PolicyType` from lib/shared/schemas/governance, and the contract's `policyType` IS `PolicyType`, so a type added to the list is readable everywhere at once. A compile-time test pins the contract to the list, a runtime test pins POLICY_SCHEMAS to it, and the stale "MUST stay in sync" comment now says where the derivations are. Review of PR #3230, blocking item 2.
ecdb816 retired the `run_code` policy type but the docs-demo builtin config tree (the docs screenshot/video stack) still shipped a governance/run-code.json allowlist for it — the last run-code artefact in the repo, read by nothing (the seed reads policies by type name and the type no longer exists). Review of PR #3230, blocking item 3 (governance-6 sweep).
createOutputTransform judged each accumulated segment on its own and cut at whatever character the 120-char threshold fell on, so an address or phone number straddling the cut escaped the output mask and a `[EMAIL_1]` token split across two segments was never restored — the reader saw `[EMA` + `IL_1]`. Latent while no output filter was wired; live since 78e6e6b. The transform now keeps a 64-char tail in the buffer across pushes and ends each segment on a boundary no pattern spans — the last line break, else the last sentence end, else the last space — flushing the tail at the end. `holdbackChars: 0` keeps the old whole-buffer segment (tests). Review of PR #3230, non-blocking note on lib/chat/guardrails.ts.
Every chat_filter_events row was written with actorType 'user', including the ones an output filter produced on the model's own text, so the Security page could not tell whose text a detection judged. The producer now stamps the actor by direction: 'assistant' for output, 'user' for input. Review of PR #3230, non-blocking note on backend/core/chat/guardrails.ts.
runModerationProvider counted a `config` fault — no secret stored, a host safeFetch refuses — toward the circuit breaker, so a misconfigured org opened the breaker after ten turns and the settings probe then reported "repeated failures" instead of the honest config verdict. The breaker guards against a provider that is DOWN; a config fault never reached the provider, so it no longer records a failure. Review of PR #3230, non-blocking note on moderation.ts.
Resolves: the run-code policy docs pages (deleted here, re-described on main) stay deleted and the audit-logs description drops the run-code clause in main's shortened form; the docs frontmatter manifest is regenerated; chat/shim.ts keeps resolveAccessScope exported for the MCP key's get_knowledge; generate_title.test.ts carries both suites (main's model-choice tests and this branch's deadline-race tests).
runModerationProvider fetched the org-admin-supplied endpoint.url through safeFetch alone. safeFetch auto-allowlists the initial URL's own host, so its private-IP / metadata gate never fired for the URL it was handed: an org admin (not the deployment operator) could aim the moderation endpoint at http://169.254.169.254/... or a private host and, with responseShape custom_jsonpath, read the response back — the same class #3221 fixed for the credential broker. policeModerationEndpoint runs checkProviderHostPolicy once before the retry loop: cloud-metadata hosts are refused always, private hosts unless the operator set TALE_ALLOW_PRIVATE_PROVIDER_HOSTS=1 — a refusal is a ModerationHttpError('config') (step_error, the fail behaviour applies, off the breaker). A policy-admitted private host is named to safeFetch's allowedHosts so its own gate does not refuse what the policy just allowed. The endpoint schema comment now describes that gate. The moderation itest lane already opts in (mock on 127.0.0.1). governance-3 (SSRF), review of PR #3230.
The merge of origin/main (4866791) kept main's typed `fakeCtx` — its `runMutation` is `vi.fn(async () => null)`, an empty argument tuple — while this branch's two deadline-race cases still indexed `runMutation.mock.calls[0]?.[1]`, which tsc refuses (TS2493). The cases now assert the title write the way the file's other cases do: `toHaveBeenCalledWith(expect.anything(), expect.objectContaining(...))`. Test-only; no behaviour changes.
The providers settings' GET /vision-model built its own `getPolicyConfigInternal` handler — a vision_model-only variant that threw on any other type — next to the byte-identical copy the chat turn hosts in governanceShimHandlers. One concept, one seam: the route spreads governanceShimHandlers(sql) like the chat host does (the moderation and chat-filter-event seams it also carries are inert there). The third copy in domains/tasks/agent-turn-shim.ts is left in place on purpose: two open campaign branches (fix/tasks-integrity, fix/sandbox-harness-hardening) rewrite that file's import block and handler table, and a fourth edit would only hand the merger a conflict. Review of PR #3230, non-blocking note on governance/shim.ts.
… shim The task-agent turn host carried the last byte-identical copy of the `getPolicyConfigInternal` handler next to the one governanceShimHandlers hosts for the chat turn and (since 58aeab4) the providers vision-model read. One concept, one seam: the host spreads governanceShimHandlers(sql) like the other two; the moderation and chat-filter-event seams it also carries are inert for a task-agent turn. Two open campaign branches (tasks, sandbox harnesses) edit this file's import block and other handler regions — the merger resolves the import lines, nothing else overlaps. Review of PR #3230, non-blocking note on governance/shim.ts.
The lane's detail line printed `sync=error/ok` and nothing else, so a refused or failed task_upsert_by_external_ref dispatch could not be told apart from a serialization retry that ran out of attempts without a second run under a patched harness. A non-ok sync now carries the dispatch's raw response in the detail line, like the other lanes do.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
One PR for the chat-turn pipeline (governance + lib-chat + chat-core slices of the backend fix campaign). The org's guardrail chain (chat_filter → PII → moderation) and its mandatory instructions now reach every chat turn, a moderation provider has a real backend, a credential fault refuses the send before any row lands, a Stop at a tool-round boundary no longer poisons the thread,
rag_search kind=mail-attachmentruns its leg, MCPget_knowledgerespects the key holder's visibility, policy files are written inside their audit transaction, and the dead seams around the turn (Convex-era store ports, the always-directsandbox arm, the four-kind capability registry, four inert policy types incl. the whole Run-code policy editor) are gone. The three policy-type lists converge onPOLICY_TYPES.26 commits (15 original + 11 from repair round 1, incl. one merge of
origin/main3997f6a), each atomic; every behaviour change carries a vitest and the integration proof gained lanes for the guardrails, the credential refusal and the moderation probe.Findings fixed
executeTurnbuilds the guardrail chain fromchat_filter/pii_config/moderation_providerpolicies (both directions, fail behaviour honoured) and recordsapp.chat_filter_events(78e6e6b)system_prompt.mandatoryInstructionsbecomes the first system-prompt block of every turn (78e6e6b)ModerationBackendimplemented oversafeFetch+app.governance_secrets(template substitution,{{secret}}header, retry, four response shapes, circuit breaker);POST /moderation/testround-trips the text and reportsnot_configuredhonestly (78e6e6b); a config fault (no secret, refused host) no longer counts toward the circuit breaker (b2bb997); the deployment host policy now runs over the endpoint URL before any request (2b78287, see the SSRF finding below)core/governance/file_utils.ts"historical migration" helpers deleted (7bbbb5d)run_code,agent_jobs,model_sync(and the retiredagent_workforceliteral) leave the policy door; the Run-code policy editor, route, docs page (en/de/fr, redirected to Hardening), screenshot, video scenes, e2e spec, manual plan, locale keys and marketing copy removed (ecdb816); the docs-demo builtin config tree'sgovernance/run-code.jsonfixture — the last run-code artefact — dropped (1c8bc5a)readUsageBuckets/UsageBucketRowremoved; the harness reads its one bucket directly (7bbbb5d)getOrgUsageMetricsPgordersperiod_key DESCunder the 20k cap (14b1b4f)previousStateread fresh (78f19b8)get_knowledgeresolves the key holder's scope viaresolveAccessScope(3e6a496)automation); dead adapters, event-only refusal,capabilityDocs, credential pass-through removed; MCP tool description + docs tell the truth;lib/chatlifted from the knip ignore list and its 51 orphan exports swept (81b13d6, 0bc47bc)containsSuspiciousInjection/SUSPICIOUS_PATTERNSdeleted (7bbbb5d)ExecuteTurnArgs.sandbox/ the constantdirectexecution and the per-turn harness table removed from the chat host; the pipeline's sandbox seam stays for the task-agent hosts (6d48f32)executeTurnbefore any write; the send route answers{status:'refused', reason, persisted:false}with no rows;classifyChatErrorCodereadsAppError.data.code(3dc5dbd)mail-attachment(labelled by provenance,sources.mailAttachmentsreported) (6de9e4e)core/chat/turn_store.tsand its seven handler names deleted; the PG ports are required (6d48f32)PROVIDER_SCOPED_ERROR_CODES/buildHumanErrorSentencedeleted (7bbbb5d)threads/store.tsgetThread/ThreadRow/listThreadMessagesdeleted (7bbbb5d)safeFetchsignal plumbing) (86faac8)TALE_EXTERNAL_TURN_DEADLINE_MSin.env.exampleand the environment reference in every locale (1b38e37)backend/core/governance/schema.tsdeleted;loader-preload.ts, the governance hooks and the app contract'spolicyTypearePolicyType=(typeof POLICY_TYPES)[number], soapproval_policyis readable everywhere without a cast; a compile-time test pins the contract union and a runtime test pinsPOLICY_SCHEMAStoPOLICY_TYPES; the stale "MUST stay in sync" comment rewritten (ecdb816, 217f4f9)Skipped
fix/provider-resolution-hardening), which is MERGED and contained in this branch via the merge oforigin/main(68b26a4 is an ancestor of HEAD); nothing left to do here.Repair round 1 (review of PR #3230)
Blocking items — all three addressed:
TurnOutcome's refused variant carriespersisted: true for every refusal the pipeline itself makes (input block, output block, failed stream — the user row and the blocked/failed assistant row are on the record), false for the host's pre-pipeline refusals (model resolution, access policy, nothing to regenerate).domains/chat/routes.tsforwards it on every refusal (busy, draining and serving refusals saypersisted:falseexplicitly); the client contract (app/lib/backend/chat.ts,chat-backend.ts,arena-actions.ts) reads it andchat-surface.tsxrestores the composer text, re-stages the attachments and unbinds the video jobs ONLY whenpersisted !== true— the pendingSend overlay is dropped and the toast shown either way.createPgTurnStore.appendMessagenow callsnotifyThread, so the refusal's two rows ride the hint stream likebeginTurn's (store.test.ts pins it). Tests:chat-surface.test.tsx(persisted refusal keeps the composer empty and the video binding; a non-persisted one restores both),store.test.ts; the guardrail itest lane assertspersisted=trueon the blocked send and the credential lanepersisted=false.run-code.jsonsurvived the governance-6 sweep — 1c8bc5a.Non-blocking notes:
lib/chat/guardrails.ts) — fixed, c31af98:createOutputTransformkeeps a 64-char tail across pushes and cuts segments at a line/sentence/space boundary, flushing the tail at the end;guardrails.test.tscovers an email straddling the cut and a[EMAIL_1]token split across two pushes.chat_filter_events.actorTypealways'user'— fixed, 6d0e015: stamped by direction ('assistant'for output).messageIdis still not stamped: the input judgement runs BEFORE the pipeline appends the user row (a refusal writes both rows after the verdict), so no message id exists at judgement time; stamping it means pre-allocating ids in the turn store — a chat-domain-theme change, left as a follow-up.configfaults counted toward the breaker — fixed, b2bb997.mode: 'mask'folds intoflagged— not changed: dropping'mask'from the mapping schema would invalidate saved policy files, and an editor hint is a user-visible string in three locales; both are out of this repair's scope. Behaviour is unchanged from the original PR (amaskmapping records a detection and does not block).getPolicyConfigInternalshim copies (governance/shim.tsvstasks/agent-turn-shim.tsvsproviders/routes.ts) — fixed, 58aeab4 + 29cee2d: the providers vision-model route and the task-agent turn host spreadgovernanceShimHandlers(sql); the one handler lives ingovernance/shim.ts. Merger's note: the open tasks and sandbox-harness campaign branches also editagent-turn-shim.ts's import block (different lines, no handler overlap) — a trivial import-line resolution for whoever merges second.chat-filter.json/pii-config.jsonnow live in the docs screenshot/video stack — re-render/recheck docs media after merge (same follow-up as the ep7/ep10 note below).New finding (SSRF, governance-3) — 2b78287
fix(platform): run the host policy over the moderation endpoint.runModerationProviderfetched the org-admin-suppliedendpoint.urlthroughsafeFetchalone;safeFetchauto-allowlists the initial URL's own host, so its private-IP/metadata gate never fired for the URL it was handed — an org admin (not the deployment operator) could aim the moderation endpoint athttp://169.254.169.254/…or a private host and, withresponseShape: custom_jsonpath, read the response back (the class #3221 fixed for the credential broker). Fix:policeModerationEndpointrunscheckProviderHostPolicyonce before the retry loop — metadata hosts refused always, private hosts unlessTALE_ALLOW_PRIVATE_PROVIDER_HOSTS=1, in which case the host is named tosafeFetch'sallowedHosts; a refusal isModerationHttpError('config')→step_error, the policy's fail behaviour applies, off the breaker.moderation.test.ts+2 cases (metadata refused before any request; private refused unless opted in, then named inallowedHosts). The moderation itest lane already opts in (mock on 127.0.0.1); the endpoint schema comment now describes the gate.Merge: 4866791 merges
origin/main(3997f6a) into the branch — no rebase, no force-push; the branch contains #3221–#3229. One merge-only fix followed (17b4397): main's typedfakeCtxingenerate_title.test.tsmade this branch's two race cases'mock.calls[0]?.[1]a TS2493; they now assert withtoHaveBeenCalledWith.Tests & gates observed
Fresh, on the merged tree (2026-09-05;
bun run check, knip and the server suite at 29cee2d, the integration proof at 860d2ca — the one commit between them only changes a lane's detail line inintegration-check.ts):bunx tsc --noEmit(services/platform, 29cee2d and again on the harness edit):TSC_EXIT 0bunx oxlint --type-aware(services/platform): exit 0, no findings (a probe file with ananyis reported, so the zero output is a clean tree, not a silent run)bunx vitest --run --project server(services/platform, 29cee2d):Test Files 524 passed (524)/Tests 6140 passed (6140), exit 0bun run check(worktree root, 29cee2d):CHECK_EXIT 0;Tasks: 40 successful, 40 total; every workspace lintFound 0 warnings and 0 errors.;@tale/platform:test: Test Files 532 passed (532) / Tests 73379 passed (73379);@tale/platform:test:ui: Test Files 455 passed (455) / Tests 3510 passed (3510);@tale/docs:test: 31 files / 201 tests;@tale/web:test: 23 files / 201 tests;@tale/shared:test: 22 files / 205 tests;@tale/ui:test: 124 files / 1181 testsbun run knip:check:KNIP_EXIT 0(the one configuration hint,cron-parser … Remove from ignoreDependencies, pre-exists onmain)bunx vitest --run --project server backend/domains/governance/moderation.test.ts(2b78287's two SSRF cases ride in the suite above)run-itest.sh, real Postgres + MinIO, no cherry-pick — the branch contains fix(platform): pass the corpus scope rows as one jsonb parameter #3222 through the merge), four runs on the merged tree — the first three under a machine load average of 20–35 (four sibling itest stacks and a VM running alongside), the fourth after it dropped:[itest] 476/478 checks passed across 134/134 lanes, noRUN TRUNCATED; failures:webdav re-home(pre-existing onmain) andautomation-run tool lane … sync=error/ok— the firsttask_upsert_by_external_refexhausted its five serializable attempts on the audit chain head ([db] transient error on attempt 1..4/5 … could not serialize access due to concurrent update×4 immediately before, then ×3 for the second call which succeeded)RUN TRUNCATED at checkAutoRetryAndKickPlan(lane 119 of 134;416/418resp.414/416before the truncation,webdav re-homethe only other failure) —POST /api/app/tasks/ the lane's firstPOST500'd with an un-retriedPostgresError: could not serialize access due to concurrent update(40001,ExecCheckTupleVisible) thrown fromdomains/audit_logs/service.ts:57 lockChainHeadafter the route's four logged retriesorigin/main(3997f6a, a throwaway worktree with none of this branch's commits, run under the same load right after run C):RUN TRUNCATED at checkTasksCollabIntegrity(lane 121 of 134,426/428before it,webdav re-homethe only other failure) — the identical escapedlockChainHead40001 (PostgresError: could not serialize access due to concurrent update→ 500 → the harness's JSON parse throws). The same truncation is in the pii theme's review log (rv-pii-itest, lane 120). The contention is main's:domains/audit_logs/service.tslockChainHeadtakesSELECT … FOR UPDATEon the org'sapp.audit_chain_headsrow inside SERIALIZABLE callers, and the task-agent auto-retry burst the tasks lanes leave running audits every failed turn; none ofaudit_logs/,tasks/routes.ts,tasks/external-ref.ts,jobs/,core/tasks/is touched by this branch (git diff origin/main...HEAD --staton those paths: only theagent-turn-shim.tshandler-spread dedup). Named in the reviewer notes as a cross-theme follow-up, not fixed here.[itest] 477/478 checks passed across 134/134 lanes, noRUN TRUNCATED, no escaped 40001,sync=ok/ok— the ONLY failure iswebdav re-home(red onmainbefore the campaign).persisted=trueon the blocked send, two rows, masked wire, mandatory marker, events), the credential refusal lane (persisted=false, zero rows), the moderation probe lane, the governance policy/audit lanes, themessages: concurrent appendslane (green in all three runs this round)Original (pre-repair, base fe27bd3):
bun run checkTasks: 40 successful, 40 total;run-itest.sh[itest] 465/468 checks passed across 133/133 laneswith the three non-branch failures named in the first round.Notes for the reviewer
ep7-connectors,ep10-developers) lost their run-code scene in the episode specs and the chapter tables ofdocs/*/tutorials/videos/{connectors,tale-for-developers}.md; the rendered.mp4/.vtt/.webpunderservices/docs/public/videos/still carry the old segment until the episodes are re-rendered — flagged, not re-rendered here.services/platform/lib/chat/turn.ts(tool-call dedupe key) andservices/platform/backend/lib/org-config.ts(policy cache key) each carry one raw NUL byte inside a template literal, so git shows them as binary in a diff — both pre-exist onmainwith the same single byte; left untouched, worth their own escaped-literal fix.cron-parserinignoreDependencies) that pre-exists onmain; knip itself exits 0.backend/domains/chat/only where lib-chat-4 required it (capabilities.tsknowledge port +shim.tsseam spread); the chat-domain theme owns the rest of that folder.webdav re-homeintegration lane is red onmainbefore this campaign (known).domains/provider_credentials/service.ts,CREDENTIAL_DISABLED_DEFAULTcontract); a sentence naming the disabled credential would be a resolver change in the providers theme's files — not made here.messages: concurrent appends each take their own slotfailed once in the first round's runs (see gates) — an intermittent indomains/chat/store.ts's bounded slot retry, owned by the chat-domain theme; worth a look there.domains/audit_logs/service.tslockChainHeadtakesSELECT … FOR UPDATEonapp.audit_chain_headsinside the caller's transaction; when the caller runs SERIALIZABLE (everytransactSerializabledomain write) and a concurrent audited write commits between the snapshot and the lock, Postgres raises 40001 (ExecCheckTupleVisible). The task-agent auto-retry storm theauto-retry cascadelane leaves running audits every failed turn, so the next lanes' audited writes retry: in this branch's runstask_upsert_by_external_refexhausted its fivetransactSerializableattempts (sync=error/ok, 476/478 run) andPOST /api/app/tasks— which does not retry at all — 500'd and truncated a run at lane 119; the identical escape truncated the pii theme's review run (rv-pii-itest, lane 120,checkTasksCollabIntegrity) on a tree without any of this branch's commits. Neitheraudit_logs/nortasks/routes.tsis touched here; worth its own fix (a retry around the audit-bearing route writes, or a chain-head lock that is not a serializable read) in the audit/tasks theme.