Skip to content

fix(platform): make the 0.5 fresh deploy reach a working stack - #3108

Merged
larryro merged 1 commit into
mainfrom
fix/cli-deploy-image-pull
Aug 30, 2026
Merged

fix(platform): make the 0.5 fresh deploy reach a working stack#3108
larryro merged 1 commit into
mainfrom
fix/cli-deploy-image-pull

Conversation

@larryro

@larryro larryro commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

What broke

v0.5.0 cannot deploy itself on a clean host. Walking the released artifacts through the documented fresh-deploy journey (tale inittale deploy → onboarding → provider → chat → upload) hit five independent defects — the first one fatal at the very first step:

  1. Pull list invents imagesdeploy.ts derived image names mechanically as tale-${service}, so the new backend tier produced tale-backend-api:0.5.0 / tale-backend-worker:0.5.0, which the release pipeline never builds (the tier runs the platform image with TALE_ROLE dispatch). Every fresh deploy dies at "Failed to pull image".
  2. Backend crash-loop on env — the CLI-generated compose never set DATABASE_URL on the backend services (compose.yml does); both roles crash-looped on the env schema.
  3. The blob store never startsobject-store was in the stateful compose but in no deploy tier, and the deploy up -d names services explicitly: no store container, uploads dead, worker ENOTFOUND object-store at boot.
  4. Worker health-wait always times outwaitForHealthy only accepted healthy; backend-worker deliberately has no healthcheck (no HTTP surface) and can never report it. Every deploy burned the full 300 s and aborted.
  5. Proxy ships without the 0.5 lanes — the entrypoint kept auth/app routes, /events (SSE), both machine doors, control drain, SSO/SCIM and /<bucket>/* behind the cutover's BACKEND_UPSTREAM reversibility switch. tale deploy never sets it, and with the Convex runtime gone "unset" no longer means 0.4 lanes — it means uploads, live updates and every machine door 404. (Chat only worked because the base Caddyfile's /api/* carries its own default.)

Two more live in the platform image itself, inherited from the teardown — the retired convex image used to carry them:

  1. /app/system + /app/builtin (the shipped config catalogs) are baked nowhere: every provider read 500s (missing shipped config directory) and org scaffolding has no seed catalog. Neither tale deploy nor plain docker compose up works around it.
  2. /app/data (org-config volume mount point) is never owned: the volume initializes root-owned while every role runs as app — the backend's object-store bootstrap dies on EACCES writing the default connection.

The fix

  • Service → image goes through one shared map (imageRef / imageRepoForService / THIRD_PARTY_IMAGES in tools/cli/src/lib/compose/types.ts) used by both the compose creators and the deploy pull list (deduped), so they cannot drift again.
  • The backend services carry DATABASE_URL (fail-closed on a missing DB_PASSWORD, mirroring the object-store key) and depends_on: object-store.
  • object-store joins STATEFUL_SERVICES + STOP_GATED_SERVICES — started on first deploy, recreated only under --stop, and now visible to status/reset/-s.
  • waitForHealthy treats running with no configured healthcheck as success.
  • The proxy entrypoint injects the backend lanes unconditionally, defaulting BACKEND_UPSTREAM=backend-api:3005; the variable remains an override for split deployments.
  • The platform Dockerfile bakes configs/platform/system → /app/system and configs/platform/custom → /app/builtin (the dockerignore already strips **/*.secrets.json from the context) and owns /app/data; the entrypoint's root phase re-asserts ownership for volumes a v0.5.0 image already booted against.
  • Stale 0.4-era guard texts corrected (--accept-data-loss help, run-deploy comment, and the guard docblock that promised a container-side backstop which retired with the Convex runtime).

Guards added

  • compose-parity.test.ts: every generated tale image must equal imageRef for its service; every image repo must exist in release.yml's build matrix (the cross-artifact fact defect 1 violated); backend env keys must cover compose.yml's (defect 2); the minio pin is a single shared value; the proxy entrypoint must inject the lanes unconditionally (defect 5).
  • container-image-test.ts (runs in the release container gate): fails any platform image missing /app/system/providers, /app/builtin, or an app-owned /app/data (defects 6–7 — this is the check that would have stopped v0.5.0 from shipping).

Verified

On this machine, against the released ghcr 0.5.0 images with the fixed CLI (built from this branch, stamped 0.5.0):

  • clean-room first deploy (fresh volumes, no state) → Deployment complete!, all 10 containers healthy in one pass;
  • browser journey on https://localhost: owner onboarding → org created → providers page lists the full catalog → OpenRouter credential added → chat round-trip streams, → document upload lands and enters RAG indexing (defects 6/7/5 hot-patched into the running containers the same way the image/entrypoint now bake them);
  • proxy image built from this branch renders /events + /tale-blobs/* with no BACKEND_UPSTREAM set;
  • gates: CLI 319 tests green, tsc clean for tools/cli and services/platform, oxlint clean, bash -n on both entrypoints.

Not in this PR (follow-ups)

  • Cut v0.5.1 from this — v0.5.0's CLI binary and platform/proxy images are unfixable in place; the release notes already steer new installs to the install script.
  • tale update --version X (space form) is silently eaten by the root --version flag on every CLI line — real parser bug, needs its own change (the = form works; notes/doc snippets avoid the broken form).
  • Stale docs: upgrades.md still lists a convex tier and a TALE_ACCEPT_DATA_LOSS container backstop that no longer exists; overview / container-architecture / compose-reference pages still describe the 0.4 topology (needs the en/de/fr sweep).
  • Platform squash-stage env still carries CONVEX_URL / SANDBOX_STORAGE_INTERNAL_BASE_URL=http://convex:3210 residues (unused by the verified lanes; sandbox lanes not exercised in this pass).
  • Frontend still probes /api/auth/convex/token (404, cosmetic) — belongs to the http-api residues branch.

v0.5.0 cannot deploy itself on a clean host — five independent defects,
found by walking the released artifacts through the documented fresh-deploy
journey:

- tale deploy derived its pull list mechanically as tale-${service},
  inventing tale-backend-api / tale-backend-worker images that were never
  built (the backend tier runs the platform image). Service → image now
  goes through imageRef/imageRepoForService, shared by the compose
  creators and the pull list, with THIRD_PARTY_IMAGES carrying the minio
  pin.
- The generated backend services lacked DATABASE_URL, so both roles
  crash-looped on the env schema. Mirrors compose.yml, failing closed on
  a missing DB_PASSWORD.
- object-store was in the stateful compose but in NO deploy tier, so the
  explicit 'up -d <services…>' never started the blob store (uploads
  dead, worker ENOTFOUND at boot). It joins STATEFUL + STOP_GATED, which
  also makes status/reset see it; the backend tier now depends_on it.
- waitForHealthy only accepted 'healthy', so backend-worker — whose
  healthcheck is deliberately disabled (no HTTP surface) — burned the
  full 300s timeout on every deploy. Running + no configured healthcheck
  now passes.
- The proxy entrypoint kept the whole 0.5 lane block (auth/app routes,
  /events, machine doors, /<bucket>/*) behind the cutover's
  BACKEND_UPSTREAM reversibility switch; with Convex gone, unset no
  longer means 0.4 lanes — it means uploads, live updates and every
  machine door 404. The lanes inject unconditionally now, the variable
  stays as an override.

The platform image also lost the shipped config catalogs the retired
convex image used to bake (/app/system, /app/builtin — every provider
read 500'd, orgs seeded nothing) and never owned /app/data, so the
backend roles hit EACCES on the root-owned org-config volume. The
Dockerfile bakes both catalogs and owns the mount point; the entrypoint
re-asserts ownership for volumes a v0.5.0 image already booted against.

Guards: compose-parity locks service→image to release.yml's build
matrix, backend env keys to compose.yml, the minio pin to one value, and
the proxy entrypoint to unconditional lanes; container-image-test now
fails any platform image missing the catalogs or an app-owned /app/data.

Also corrects the 0.4-era guard texts (--accept-data-loss help, the
run-deploy comment, and the docblock promising a container-side backstop
that retired with the Convex runtime).
@larryro
larryro merged commit 8650fb8 into main Aug 30, 2026
58 checks passed
@larryro
larryro deleted the fix/cli-deploy-image-pull branch August 30, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant