Conversation
transactSerializable runs a postgres.js transaction at SERIALIZABLE and re-executes the callback on SQLSTATE 40001/40P01 with jittered backoff, layered on the existing connection-level withRetry (which only covers 08/57P/53-class transient errors). This is write-constitution rule 1 of the 0.5 Postgres backend: every mutation-shaped handler goes through it, so callbacks must stay pure apart from their database writes.
The Convex-to-Postgres port, increments 01-05 on the campaign ledger (services/platform/backend/MIGRATION.md): - Runtime: the platform image gains TALE_ROLE api/worker/all dispatch (Node runs backend/*.ts directly; compose profile 'backend'); a Node resolve hook + --experimental-transform-types let the backend import runtime-clean 0.4 modules unchanged instead of fork-copying them. - Constitution: serializable writes, pg-boss 12 jobs enqueued inside the caller's transaction (LISTEN/NOTIFY wake, singletonKey dedupe, per-job batch results), and the Tier-2 hint outbox fanned out over SSE with Last-Event-ID replay. postgres.js json/jsonb serialization is aligned with node-postgres semantics (strings pass through) so pg-boss's pre-stringified parameters are not double-encoded. - App DB: tale_app + an advisory-lock-guarded boot migrator; Better Auth and pg-boss own their schemas. - Auth at 0.4 parity: email+password with the login-throttle gate (per-IP flood guard, per-account exponential lockout, timing jitter), the organization plugin (teams, access-control role matrix, slug/name hooks, scaffold-on-create), apiKey (+suffix column), twoFactor and passkey plugins; membership reads are direct SQL - the 0.4 mirror apparatus is gone. - Ported domains: audit_logs (per-org chain head locked FOR UPDATE replaces the OCC genesis trick), login_attempts, notifications (the org-audience bell at the 0.4 shape), users, user_preferences, organizations (reads, org-switch record, deletion door, scaffold and cleanup jobs), members (list/context + guarded mutations), the PG rate limiter carrying the full 0.4 rule catalog, and file-direct org config reads (the configCache mirror dies). - Proof: backend/integration-check.ts runs 22 checks against a real throwaway Postgres, including lockout flow and audit-chain re-verification.
Core CRUD, settings (identity/sharing/instructions/knowledge/agents/ models/connectors), pin/archive/restore/delete (confirm-gated cascade mode), duplicate, project agents (create/update/delete with the 0.4 validation caps and admin-only secret grants), search, sidebar and overview reads. The pure access matrix, key derivation, and audit action constants are reused from 0.4 unchanged; per-org key/externalItemId uniqueness moves into partial unique indexes. Cross-domain touchpoints (document attach, delete-cascade walks, overdue rollups, automations guard, REST v1 surface) land with their domains - tracked in MIGRATION.md. Also fixes the token-bucket rate limiter's ratePerMs parameter type under bigint expression context (::float8).
Tier A of the tasks domain: create/update with label resolution and per-project numbering, status choreography with the subtask close-guard and completedAt semantics, polymorphic assignee validation (humans need project access, agents must be instances of the project), claim, board drag moves on the reused LexoRank module, the label catalog (create/rename/delete with task detach), advisory dependencies with write-time cycle rejection, saved board views, the per-task activity timeline, and the denormalized project rollup transitions. Project creation now seeds the default labels in the same transaction (0.4 parity). Tier B (agent runs, review arc, comments/mentions, notify and event fan-outs, REST, crons) lands with its infrastructure - tracked in MIGRATION.md.
S3-compatible object storage becomes THE blob backend (Convex _storage dies with the component): resolution prefers the org's BYO object-storage/connection.json and falls back to the deployment default under the 'default' config tree, failing closed when neither exists. The aws4fetch signing, presign lanes, and key scheme are reused from 0.4 unchanged, so existing BYO-org configs work verbatim. Adds the app.file_metadata ledger (RAG/transcription pipeline columns shipped nullable for the knowledge/tts ports) and the files surface: a two-step upload handshake with server-minted keys and HEAD-verified registration, presigned GET serving, and org-scoped delete. Integration exercises the full lane against a real MinIO (gated on ITEST_S3_ENDPOINT, visibly skipped otherwise).
pg-boss schedule() registry (worker-boot idempotent upsert, UTC) with the first maintenance sweeps - idle rate-limit rows (7d) and the loginAttempts 30-day TTL plus 90-day block counters. Ports the platform event seam at its exact 0.4 contract: dispatch stays a deliberate no-op until the automations domain rebuilds the subscription fan-out, and the producing domains (task created/status changed, project created) call it inside their transactions so the swap-in is one function body.
Documents Tier A: bind a registered upload as a document, hub vs project scoping with the 0.4 access-module semantics (project docs never surface in hub listings; team rules with org-wide default), rename/move/team edits, trash/restore soft lifecycle, project attach/detach with the mutual-exclusivity rule (closing the projects ledger gap), mention search, and presigned serving. Folders: tree CRUD with the depth cap, sibling-name uniqueness, scope inheritance and conflict rules, breadcrumbs - deleting refuses while any document lives in the subtree until the trash-cascade lands. Integration drives the whole vertical against real MinIO including the hub-visibility flips on attach/trash.
The 0.5 replacement for the @convex-dev/agent component's thread and message tables: app.threads + app.messages with the component's (order, step_order) turn model, AI-SDK-shaped jsonb parts plus derived plain text, and the thread_metadata sidecar (generation lifecycle columns shipped for the chat engine). Task discussion comments ride it end to end - lazily minted task_discussion threads, lockstep meta rows (author, mentions, editedAt, locale snapshots), comment counts, activity and audit trails, author-or-admin edit/delete. The mention directory and its fan-outs land with collab/agents/automations.
Contacts: the per-org correspondent directory with role-matrix gating, normalized emails, filterable keyset listing, find-or-create for the conversations lane, soft trash, and contact.* events. Message feedback: per-(message,user) upsert votes with toggle semantics and the org insights feed with rating stats. Support cases: the org-scoped ticket lifecycle with status timestamps, escalation levels, SLA fields, comments that stamp first response, the activity feed, and archive. Integration exercises all three verticals; the audit chain now verifies 36 rows across nine domains.
CRUD with the 0.4 field caps and role-matrix gating, per-org case-insensitive name uniqueness as a real expression index (the 0.4 full-table probe becomes a constraint, kept only for the friendly error), per-language translation upserts, and a filterable keyset listing. REST and connector ingest lanes land with the machine door.
The credential RESOLUTION path - api-key decryption via secret_box, the TALE_PROVIDER_KEY_ env gate re-checked at read time, and the subscription-broker pool fetch/pick - reuses the 0.4 module verbatim: a new ctx shim (lib/convex-shim.ts) re-points its two row lookups at app.provider_credentials by function name and fails loud on any un-shimmed ctx call. Admin surface: create (encrypt + masked preview, first row becomes the default), status/default/allowlist edits with the default swap enforced by a partial unique index, delete - all audit-trailed, secrets never leaving the server. The node loader also learns bare-specifier .js retries for CJS deep paths (validator/lib/*).
Search and fetch reuse the 0.4 modules verbatim - the ctx shim re-points their three seams (the org-row lookup, credential loads, and the retrievable-file access filter) at Postgres; the Tier-A filter covers document and chat-thread scopes and denies conversation refs until that domain lands. Ingest composes the same exported pieces (extractText, the org embedder behind provider credentials, indexDocument with the PII gate and scope stamps) into the rag.index_file job, enqueued transactionally when an upload binds as a document, with status writes on file_metadata. The deployment-default corpus bootstraps at worker boot. Integration drives the full loop against the real corpus database with a local OpenAI-compatible embedding endpoint (base64 Float32, the SDK default).
`jsonb_build_object` takes `"any"`, so the uncast parameter left Postgres nothing to infer from and the statement failed to PARSE — 42P18, before it ever ran. The endpoint answered 200 and wrote nothing, for as long as it has existed, filling the dev log with the error a user finally noticed. One `::numeric` fixes it; the sibling site in knowledge_entries already casts, so this was an omission, not a pattern. Nothing caught it because the lane had no integration coverage at all. It has one now: post a wait against a real assistant message, assert the value lands, then post a second one and assert the row is still stamped with the first — the once-only guard the query carries.
A turn that spent 27 seconds and seven searches showed nothing until it settled, then painted the whole trace at once. The engine was never at fault: `runTurn` persists the settled parts at each tool-round boundary, so the tool calls were in `app.messages.parts` as they happened. The transport dropped them. The progress lane carried text, reasoning and the cancel flag — `app.generations`, the row it polls, has no column for parts — and the client read only those three. The transcript itself is refetched at settle, which is why the trace arrived in one lump. Three edges close it. A parts write now bumps the generation clock: the lane watches that clock, and a parts-only write moved nothing, so a tool step was literally not progress. `progress` carries the row's current parts, sent only when they changed — a text tick fires at the store's 250ms throttle while a tool result can be a whole RAG page. And the view prefers the streamed parts when they outnumber the row's, which keeps the settle write authoritative the moment it lands. The renderer needed nothing; it already builds from `parts`. Guarded against the shape of the bug rather than its symptom: the integration check opens the stream on a fresh thread, runs a real tool round, and asserts a `tool-call` part appears in a frame BEFORE `event: settled` — finding it only in the settled payload fails. The fake provider pauses before its final round so the assertion turns on behaviour, not on whether the whole turn fit inside one poll.
The ENTITY_ID pattern is 16+ chars, so waitForURL resolved on the wizard path and every later dashboard navigation hit the wizard again.
Forced rotation 400ed as voluntary without currentPassword, so invite specs never left /forced-change-password. The automations list shows the title-cased name, not the slug.
Typecheck failed on the false branch reading init.body when init is undefined.
Every new thread's first message fires a second model call — the title generator — and it spent real tokens (one 48-output-token call per thread on the provider's bill) while writing nothing to usage_events: generate_title.ts contained no usage write at all. The parse layer had the numbers all along; nothing read them. The attempt now carries its spend out, and the impl books it through an injected recorder — the same ledger the turn writes through, handed in by the job door, because the ctx shim has no ledger of its own. Booked under its own `thread-title` agent slug so analytics can tell what the conversation cost from what naming it cost; booked whether or not the title was usable, because the tokens were spent either way; best-effort, so a ledger failure never costs the title. Deliberately unchanged: cost stays the catalog estimate everywhere. Provider-reported cost would make spend a number governance can only know after the fact — budget pre-checks and the ledger must price by the same rule. The alias-routing under-count is a property of estimating a routed request, not a config error. The guard books through a real call: the integration check pins the probe user's chat model to the fake provider (the fallback scan would take the shipped openai connector, whose fake env key an earlier block plants, and die on an unreachable host — masking the booking), then asserts both the usage row and the AI title itself landed. Found en route: the fake provider only spoke SSE, so the title lane's non-streaming JSON call always fell back in this harness — it now answers both dialects, like the providers it stands in for.
client.action skipped the write adapter's invalidate, so a successful package upload left the listing on its cached empty page. saveVersion now emits the automation hint the same-tab and SSE paths both need.
Clicking a task deliverable answered "Failed to load document": the preview asked GET /files/<ref>/url and got a 404. The app's file-identifier vocabulary is mixed by contract, not by accident. Listings hand out the row's own id, the POST upload lane's storageId IS the blob ref, task deliverables carry the ref, and the 0.4 getFileUrl query took the ref and resolved the row by storageId. The 0.5 port kept only the row-id half — /files/:fileId, its /url twin, the batch /urls and the delete all resolved WHERE id — so every ref-addressed read 404ed, while /files/statuses, ported separately, already resolved by ref: the tell that the vocabulary was mixed all along. One resolver closes it. The s3: prefix makes the two identifiers unambiguous, and a ref is org-scoped twice over — the WHERE, then the key's own org prefix at presign time. All four routes go through it. The guard round-trips the same upload through both identifiers, and the ref lane's body must match byte-for-byte.
`tale update` is line-pinned — a 0.4.x instance never moves to 0.5.x on its own — but the deploy guard is what stands behind the explicit move the update hint suggests, and its baseline still said 0.4.0. With that, `tale update --version 0.5.0 && tale deploy` on a 0.4 instance sailed through: both versions sit above the old baseline, so the guard had nothing to say while the deploy booted an empty Postgres and left every chat, user and document orphaned in the retired store. 0.5 is exactly the cutover the guard exists for — the application database moved from Convex to Postgres and no importer bridges them — and the constant's own contract says to bump it at the next one. The refusal text, the update-path downgrade warnings (which still told operators to run the deleted `tale migrate down`), and the upgrade docs in all three locales now describe the 0.4 → 0.5 cutover: what carries forward (the org config tree, on the shared volume), what does not (the database), and the fresh-deployment path. The org-config carry-forward is stated because it is the one asymmetry operators will probe.
The release workflow still built, gated, manifested and verified a `convex` image whose Dockerfile was deleted with the service: the build matrix would fail on the missing file, the container-test gate's pull loop on the missing image, and the manifest/verify loops behind them — the first tag push after the cutover would have died in four places. The gate tests were worse than stale: compose.test.yml overrode a `convex` service the base file no longer declares, which makes compose reject the WHOLE project — the smoke test would fail at `up` before probing anything. The overlay now wires the backend tiers and the object store into the test env instead, and the smoke test probes what actually ships: health-waits the 0.5 tier, `curl /ping` inside the backend container, platform → backend-api connectivity (every /api lane rides it), and backend → object-store health (an unreachable store means every upload 503s). The image test drops entries for services deleted long ago — convex, and the crawler/rag pair that consolidated into the platform — which it silently skipped as "image not found" while implying coverage; the vulnerability scan list loses the same two ghosts. Smoke has never actually completed on this branch (always skipped or cancelled), so these are proven locally against freshly built images rather than by CI history.
The opt-in docker-socket sidecar behind one-click "Apply & restart" was built to bounce rag+convex after a deployment-config change; both are gone, and the lane itself was broken against 0.5 — the app sent no services, the backend defaulted to ['convex'], and the controller's own allowlist stopped accepting convex at the teardown. Every operator who may edit deployment config is named in TALE_DEPLOYMENT_CONFIG_ADMINS, a host-side env var, so the manual path is always available. The whole lane goes: services/controller/, POST /deployment/restart and its wire-contract entry, adapter, hook and header button, the restart strings (en/de/fr), the CLI's compose emitter and deploy/restore hooks, the compose.yml block (the stack's last compose profile), the dev-loop sibling spawn, the CONTROLLER_* env docs, the build/cleanup CI matrices, and the trivy + commitlint entries. The post-save banner and the docs now name the real apply path: docker compose restart backend-api backend-worker, or tale deploy. cleanup-pr-images keeps a controller row so leftover PR image tags still get GC'd; release.yml's controller rows landed with the release-pipeline fix. Integration 260/260; docs suite 194/194; cli tests green; i18n parity green.
The backend entrypoint's egress firewall REJECTed all of RFC1918, but the compose bridge networks are RFC1918 too and the container-netns OUTPUT chain filters traffic to same-bridge peers — the claim that such traffic bypasses OUTPUT via the bridge driver was wrong. Both backend roles hold NET_ADMIN, so api and worker fenced themselves off from db and crash-looped on CONNECT_TIMEOUT (TCP treats the ICMP net-prohibited reject as a soft error and retries SYN until the client's timeout), leaving the smoke stack permanently 'starting'. The web tier survived only because it never gets NET_ADMIN. ACCEPT the container's directly-connected subnets (from ip -4 route) ahead of the RFC1918 rejects — IMDS and link-local stay rejected before everything — and ship iproute2 in the runner image. Without iproute2 the RFC1918 fence is skipped with a warning instead of self-severing. Verified in a bookworm-slim container with NET_ADMIN on a compose-like bridge: peer connects instantly, external RFC1918 and IMDS stay fenced, public egress unaffected. The CLI compose generator grants the same cap in production, so the deployed stack hit the identical loop — the fix rides in the image for both lanes.
Review follow-ups on the same-compose fence fix: derive the pass-list from scope-link routes only, so a via-learned route (a pushed 10.0.0.0/8 through some gateway) can never widen it past genuinely attached subnets; and when no subnet is derivable — iproute2 missing OR an empty scope-link table — skip the RFC1918 fence entirely instead of self-severing, keeping the IMDS + link-local rejects in every scenario. A guard test now locks the rule program: it runs the shipped install_ssrf_firewall against PATH-stubbed ip/iptables and asserts ACCEPTs cover exactly the scope-link subnets ordered between the IMDS/link-local rejects and the RFC1918 fence, and that both no-derivable-subnet paths emit no fence. The ip stub serves the via route to any non-scope-link query, so dropping the filter leaks a 10.0.0.0/8 ACCEPT and fails the guard (verified by mutation). Live-checked on bookworm-slim with NET_ADMIN and an injected via route: peer connects, via-routed and IMDS targets stay fenced.
This was referenced Aug 31, 2026
Closed
This was referenced Sep 3, 2026
larryro
added a commit
that referenced
this pull request
Sep 4, 2026
KNOWLEDGE_ENTRIES_FOLDER ("Knowledge entries") has had no importer since
the 0.5 rewrite (#3107): knowledge-entry backing documents are listed at
the Documents root, not filed into a reserved folder. knip never flagged
it because every backend/**/*.ts file is a knip entry for the platform
workspace, so backend exports are never reported unused. No message key,
helper, test, or docs sentence existed only for it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the Convex runtime with a Node + Hono + Postgres backend. The
running stack has no Convex process, service, control channel, or client
runtime; the app has its own wire contract; the reused 0.4 handler bodies
run against a SQL-backed ctx shim.
services/platform/backend/MIGRATION.mdis the campaign's source oftruth — one row per domain, one row per teardown step, each naming what
landed and what was found en route. 94 done / 5 dropped / 1 pending.
What this is
backend/, 57 SQL migrations applied at boot under an advisory lockconvex/tree.ts.ts— the reused handler bodies and what they reachThe port is domain-by-domain, not a rewrite: where a 0.4 handler body was
correct, it is REUSED verbatim behind a ctx shim that dispatches a
function NAME to a SQL handler. That is why the diff deletes twice what
it adds — the decision logic mostly survives; the runtime under it does
not.
Gates
Run on the tip, each with its own exit code checked:
tsc --noEmit— 0oxlint --type-aware— 0The integration run is the one that matters: it exercises every door
end-to-end against a throwaway database, including the machine journey
(presigned upload → bind → listing → content roundtrip), the automation
stepper, the task-agent arc, SSO/SCIM, governance holds and retention,
the WebDAV lane, and the in-sandbox bridges.
What is deliberately NOT done
The one
pendingrow.convexis still inpackage.json, used as aLIBRARY, not a runtime:
v/Infer/GenericIdfromconvex/valuestype the reused handlers' argument contracts across 32 modules. Nothing
imports it from
backend/orlib/. Those validators become plainTypeScript or zod as each domain ports, and then the reuse set moves out
of
convex/and the package leaves.Also priced rather than hidden, in the ledger:
WebDAVBackend's threemethods return
any— exactly what the retired client's genericresolved to for a name-addressed call, so nothing about strictness
changed here. Tightening it means declaring shapes at 66 call sites and
updating handler stubs that return partial ones; that is its own change.
Review notes
The ledger rows carry the reasoning, including the traps worth knowing
before touching this code:
through the shim (
internal.x.y.z) with no import to follow. Anyclosure analysis must walk static imports, dynamic
await import(),AND api-object references — a naive import closure was wrong by 350
files.
found and fixed this way: the connectors bridge's forensic audit lived
in the 0.4 wrapper, and the 0.5 door calls the impl directly.
_generated/is hand-maintained now. Adding a name there withoutgiving the shim a handler for it is a runtime refusal waiting to
happen.