Skip to content

fix(backup): clear AI job bindings with the provider list a restore clears - #4800

Open
Quick104 wants to merge 5 commits into
mainfrom
fix/restore-clear-ai-provider-bindings
Open

Quick104 wants to merge 5 commits into
mainfrom
fix/restore-clear-ai-provider-bindings

Conversation

@Quick104

@Quick104 Quick104 commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Related issue: N/A
Validation tasks: none

Other clients: N/A (no client-visible change)

A separate restore (silo backup restore --into) clears the backup's AI provider list, as it clears every credential for an outside service. It kept the text, speech-to-text and recommendation-embedding job bindings, though, and those name providers in that list. On the restored server every admin AI provider change then failed. Connecting a new provider returned 409 the text model provider "<id>" is not connected until someone cleared each binding by hand. A restore with another SECRET_KEY (--accept-lost-credentials) left the same bindings behind, because the restore step that clears settings the new key can't read had already deleted the list.

This change clears the bindings with the list. It also leaves an empty provider list in place, so the restored server's first start doesn't rebuild providers from what the backup still holds.

A full restore onto a live server is not affected: it takes the live server's list and bindings afterwards, with the other server-specific settings.

Approach

  • The AI providers' bundle in internal/backup/settings_outbound.go now clears the job bindings (config.AIProviderBindingKeys) along with the list. Bindings with no list left, as that step leaves them, also clear the bundle.
  • A cleared bundle now writes its off values, a field that replaces enabled. Mail still turns email.enabled off. The AI bundle writes an empty provider list at the current legacy-conversion version, which it takes from config.PlanAIProviderConversion(nil).
    • Deleting the list instead would reset that version to 0, and the next start would run the legacy conversion again. Once the bindings are gone, that conversion adopts the backup's leftover ai.base_url as a keyless provider. With recommendations on, it also adopts an Ollama default at http://ollama:11434 and binds embeddings to it.
  • The empty list is encrypted with the restore's SECRET_KEY, like any stored provider list. The startup backfill that would otherwise encrypt a plaintext row reads it and writes it back without a guard, so another API server could save a provider in between and lose it. CarryOutboundSettings and postLoad.carry now take the restore's cipher for this. A restore run without a key writes the list as plaintext, which Silo reads as it is and encrypts when it starts.
  • docs/architecture/backups.md records the rule.

The empty list comes from the conversion planner rather than a new exported helper because #4668 is changing internal/config/ai_providers.go.

Validation

  • Reproduced live on a Silo dev deployment at dec9d151 (main):
    • Setup: connected a custom provider and bound it to text, speech-to-text and embeddings, turned recommendations on, took a backup, restored it with --into … --database-only, and started the server on the copy.
    • Result: connecting a provider returned 409 the text model provider "custom-71e34b" is not connected.
  • With this change, the same backup:
    • restores with no bindings and an empty, encrypted provider list;
    • starts with no providers, and logs no conversion even though recommendations are on;
    • accepts a new provider (201), which can then be selected for text.
  • TestBackupRestoreRoundTripDB now covers the cleared AI bundle and the bindings-only case.
    • On main it fails with the reported error.
    • It also fails if only the bindings are cleared, because the next start would invent the two providers.
    • It passes with this change.
  • Go gate passes: build, gofmt, go mod tidy -diff, vet, make lint-changed (0 issues), make lint-router-recovery (0 issues) and make test-go. The internal/backup database tests also pass against a freshly migrated database.
  • Repository checks pass locally: size ratchet, bidi, local paths, case collisions, migrations and route inventory. The test helper moved to settings_outbound_postgres_test.go to keep the round-trip test file under the size limit. verify-settings-bindings-all wasn't run to the end because the web dependencies aren't installed here, and no settings contract changed.

Evidence

Nothing changes on screen. The evidence page has the steps, both build identities, the trimmed API responses and server log lines, and the test runs on main and on this branch.

Evidence: https://evidence.siloserver.org/r/silo-server/restore-ai-bindings/

Risks

  • The restore output is unchanged. It still lists "AI providers" among the cleared connections. On a replace of another deployment's backup, that line also appears when the live server's providers are then kept. Main already does this when the backup has a provider list, and this PR adds the case where only bindings survive a restore with another key. Follow-up: [bug] A replace of another deployment's backup says to reconnect AI providers it keeps #4813, which waits for this PR to merge.
  • Two states still let the first start convert the backup's leftover legacy endpoint rows, as they do on main. Neither leaves a binding naming a missing provider.

Checklist

  • I read and can explain the complete diff.
  • This pull request addresses one concern.
  • Each test this pull request adds meets "Decide what to test" in CONTRIBUTING.md.
  • The Evidence section shows every change a user can see, or how I checked a change nobody sees.

Council review

Plan round (before implementing):

  • Sol: found the restore with another key and the unguarded startup backfill. Both are handled above.
  • Opus: found the plan already minimal.
  • Gemini: confirmed the plan's premises against the source. Its notes on touched files and test setup were already covered.

Background review: round 1 reviewed c810ca1

Member Round 1
Sol (gpt-6.2-sol, xhigh) 8/10
Opus (claude-opus-5-5, high) 8/10
Gemini (gemini-4-neon, xhigh) 9/10

Sol's review couldn't be posted from its task, so its two low findings are handled here. Fixed: the test now also checks that the emptied list holds no providers (5102ede), which Sol and Gemini both raised. Answered: the misleading "AI providers" line after a replace of another deployment's backup is pre-existing and tracked in #4813. A test for a dropped restore cipher isn't added: a nil cipher only falls back to the plaintext path, which reads correctly, and covering it would mean adding AI settings to the shared round-trip seed. Opus found nothing to post.

Nothing on screen changes, so there was no visual review.

AI Disclosure

  • Harness: Claude Code in T3 Code
  • Tool(s): Claude Code with its review subagents; T3 Code delegated tasks for the plan round and the background review (Codex, Claude, Antigravity)
  • Model(s): claude-opus-5-5 (implementation, verification, code review); claude-sonnet-5-5 (code-review cleanup finder and validation check); gpt-6.2-sol, claude-opus-5-5, gemini-4-neon (plan round and background review)
  • Involvement: Fully AI-generated. The agent verified it live on a dev deployment; a person has not verified it yet.
  • Adversarial review: a three-member plan round before implementing; an xhigh code review (three finders and a sweep; its validation check couldn't read the v1.0 board), whose doc and test findings were fixed; and a simplify pass. See Council review.

🤖 Generated with Claude Code

Note

Fix restore to clear AI provider bindings with the provider registry

  • When a restore clears the AI outbound bundle, it now deletes provider bindings and writes an empty provider registry at the current conversion version, matching what a restore already does with the provider list. A binding without its registry also clears the AI bundle.
  • outboundBundle now has an off-values map instead of a single enabled switch, so a cleared bundle can write multiple off settings (settings_outbound.go).
  • The restore cipher is passed through postLoad.carry, carryLiveState, and PGLoader.CarryOutboundSettings, so sensitive off values (like the empty registry) are encrypted with the server's key when available. Restores with another key write them as plaintext.
  • Tests cover AI bundle clearing with and without a cipher, and the outbound-settings helper moved to settings_outbound_postgres_test.go.
  • Risk: DatabaseLoader.CarryOutboundSettings changed signature; any out-of-tree loader implementing it needs the new cipher parameter.

Macroscope summarized 1d9f3d1.

…lears

A separate restore (silo backup restore --into) cleared the backup's AI
provider list but kept the text, speech-to-text, and embedding bindings
that point into it. On the restored server every admin AI provider
change then failed with "provider is not connected".

The AI bundle now clears its bindings with the list, and a cleared bundle
writes its off values: the mail switch as before, and for AI an empty
provider list at the current conversion version, encrypted with the
restore's key when it has one, so the next start doesn't rebuild
providers from the backup's legacy endpoint settings or the Ollama
default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

  • Ask an admin to enable usage-based reviews

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5e192168-5426-4385-ac92-a52aa22e2a0a
Commits

Reviewing files that changed from the base of the PR and between 980e455 and 1d9f3d1.

Files selected for processing: 7

  • docs/architecture/backups.md
  • internal/backup/backup_postgres_test.go
  • internal/backup/restore.go
  • internal/backup/settings_outbound.go
  • internal/backup/settings_outbound_postgres_test.go
  • internal/backup/swap.go
  • internal/backup/swap_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@silo-triage silo-triage Bot added the size/M 50–199 added + deleted lines (all files); change volume, not priority. label Nov 21, 2026
@silo-kody

silo-kody Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌
⚠️ 4 Kody Rule(s) were not evaluated

These rules declare that they need repository context beyond this diff, and the review could not retrieve it, so they were not judged on this pull request:

  • Preserve optional store capabilities through production wrappers
  • Detach mandatory cleanup from request cancellation and bound its lifetime
  • Preserve durable media identity and share identity-evidence precedence
  • Bound decoder throughput trades and share direct-play eligibility

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T05:51:30.300956Z 1d9f3d1 New commits
🔒 Security Review ✅ Completed 2026-11-21T05:53:59.732494Z 1d9f3d1 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@silo-triage silo-triage Bot added bug Something isn't working go Pull requests that update go code impact: usability Core flow broken or severely blocked priority: P2 Limited scope, workaround exists, or polish labels Nov 21, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Quick104 Quick104 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review (Gemini 4 Neon, reviewed c810ca1)

One finding on checkAIReset in internal/backup/backup_postgres_test.go: the production changes and docs/architecture/backups.md match the restore paths and startup conversion code, but the new test helper does not verify that ai.providers is actually empty.

AI-assisted review: Antigravity (T3 Code) with Gemini 4 Neon.

Comment thread internal/backup/backup_postgres_test.go Resolved
The round-trip test file crossed the 2,000-line limit for test files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-triage silo-triage Bot added size/L 200–499 added + deleted lines (all files); change volume, not priority. and removed size/M 50–199 added + deleted lines (all files); change volume, not priority. labels Nov 21, 2026
@silo-kody

silo-kody Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌
⚠️ 4 Kody Rule(s) were not evaluated

These rules declare that they need repository context beyond this diff, and the review could not retrieve it, so they were not judged on this pull request:

  • Preserve optional store capabilities through production wrappers
  • Detach mandatory cleanup from request cancellation and bound its lifetime
  • Preserve durable media identity and share identity-evidence precedence
  • Bound decoder throughput trades and share direct-play eligibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working go Pull requests that update go code impact: usability Core flow broken or severely blocked priority: P2 Limited scope, workaround exists, or polish size/L 200–499 added + deleted lines (all files); change volume, not priority.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant