Skip to content

fix(catalog): keep durable provider IDs in step with an admin's ID edit - #4195

Merged
Quick104 merged 12 commits into
mainfrom
fix/admin-metadata-provider-id-normalize
Nov 15, 2026
Merged

Quick104 merged 12 commits into
mainfrom
fix/admin-metadata-provider-id-normalize

Conversation

@Quick104

@Quick104 Quick104 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Related issue: N/A
Validation tasks: changes #1210 C1, changes #1210 C2, changes #1211 C1, changes #1211 C2

Other clients: Apple N/A (no metadata edit surface); Android N/A (no metadata edit surface); Roku N/A (no metadata edit surface); Web: covered here (no web change needed; the dialog shows the new error)

When an admin changes a title's IMDb, TMDB or TVDB ID in Edit Metadata, Silo stores the
text exactly as typed and leaves the title's durable provider ID behind. A pasted
"550 " or 550-fight-club no longer matches exact TMDB lookups, so a title in the
library can read as absent to request presence checks. A corrected or cleared ID leaves
the old one in media_item_provider_ids, which lookups, AttachTMDBID's conflict check
and a refresh's ID seeding read. A refresh then puts a cleared ID back, and drops a value
that isn't a valid ID in favor of the old durable one, so the admin's edit quietly
reverts.

This change stores the ID clean and moves the durable provider ID with it in the same
transaction.

Approach

  • PATCH /api/v2/admin/items/{id}/metadata trims each ID, lowercases an IMDb ID, and
    drops a TMDB URL slug (550-fight-club saves as 550), using the existing
    NormalizeTMDBID and providerid checks. A value that isn't an ID of its provider is
    422 validation_failed at that member, reported with any bad date in the same
    response. An empty string still clears the ID. Seasons and episodes get the same
    checks.
  • ItemRepository.UpdateMetadataTx replaces or deletes the item's durable row for each
    provider whose ID the edit changes. An ID the edit resends unchanged is left alone,
    compared the way fix(catalog): hold curator-chosen provider IDs to the curator's libraries #3932's curator check compares it. If another item of the same type already holds the new ID, only the
    column changes, as before, and the next refresh's conflict recovery decides between
    the two items. A check before the insert keeps two edits that swap IDs from waiting on
    each other. A savepoint keeps a holder that commits mid-edit from failing the whole
    edit. Providers are written in the order ReplaceByContentIDTx uses. Three writers
    that changed an item's provider rows before locking the item now lock it first, as
    the edit does, so they queue behind an edit instead of deadlocking with it: the
    standalone ReplaceByContentID (book enrichment), the startup provider-ID drift
    repair, and an automatic rebind of a skeleton onto its matched item. Both drift-repair
    branches (clean insert and moving an ID off a provisional owner) also re-read the item's
    IDs under that lock and skip an item an edit changed while they waited, rather than
    writing back the IDs they read before. Moving an ID off a provisional owner also
    checks, under the owner's lock, that the owner still holds it.
  • The web dialog already sends only changed IDs, and its error toast shows the field's
    detail, so the web app needs no change. jellycompat has no metadata edit route.

Validation

  • Live, on a sandbox: on the base build, the web dialog stored 333371-10-cloverfield-lane
    and fight club as typed, and the API stored "7191 " with the durable row still on
    the old ID. On the fix build, the slug saves as 333371, fight club is refused with
    "expected a TMDB ID, such as 550", padded and uppercase IDs are cleaned, clearing an
    IMDb ID removes its durable row, an ID another title holds changes only the column, and
    a plain tagline edit with a lock behaves as before.
  • A new Postgres test (added to scripts/ci/db-contracts.txt) and a new apiv2 test both
    fail on the base for the reported reasons and pass with the fix. The Postgres test
    also covers another transaction claiming the ID while the edit waits on it; that case
    fails with a unique-key 500 without the savepoint. Two more Postgres tests (also in
    db-contracts.txt) hold the item as an edit does while a drift repair or a rebind
    runs; both fail on the previous head with deadlock detected and pass now.
  • go test for internal/catalog, internal/apiv2, internal/api/handlers and
    internal/metadata against a migrated Postgres; make lint-changed; the OpenAPI, web
    type, contract, fixture, route-inventory, size and local-path checks all pass.
  • Validation cases: Manual metadata edit / identify / artwork — Server #1210 and Manual metadata edit / identify / artwork — Web admin #1211 C1 and C2 edit through this endpoint. Only an edit
    that sends an ID behaves differently: the ID is cleaned or refused, and a saved ID now
    survives a refresh. Their other field edits, locks and Match Item are unchanged.

Evidence: https://evidence.siloserver.org/r/silo-server/admin-provider-id-edit/

Risks

  • A client that sends a value that isn't an ID now gets a 422 instead of a save that a
    refresh would later undo. The web dialog sends only the IDs the admin changed, so a
    title already holding a bad stored ID can still be edited.
  • When two titles claim the same ID at once, the second keeps the ID on its column only
    until its next refresh, the same as before this change.

Checklist

  • I read and can explain the complete diff.
  • This pull request addresses one concern.
  • Each test this pull request adds meets "Decide what to test" in CONTRIBUTING.md.
  • The Evidence section shows every change a user can see, or how I checked a change nobody sees.

AI Disclosure

  • Harness: Claude Code (via T3 Code); Codex for the plan review
  • Tool(s): Claude Code; Codex; GitHub CLI
  • Model(s): claude-opus-5-5; gpt-6.1-sol (plan review); claude-sonnet-5-5 (code-review subagents)
  • Involvement: Fully AI-generated, human verified
  • Adversarial review: before implementation, gpt-6.1-sol attacked the plan and claude-opus-5-5 checked it for cuts. That round added TMDB slug normalization, the targeted conflict handling and a corrected account of what a refresh does. A high-effort multi-agent code review (claude-opus-5-5 and claude-sonnet-5-5 finders, plus a validation check) then found the 500 when a holder commits mid-edit and a provider-order deadlock with refresh writes, both fixed. It also found doc wording that limited the sync to movies and series, now corrected. Not changed: leading zeros, which refresh also keeps; sharing refresh's ID sanitizer, which would change refresh behavior. A simplify pass ran on the production code. The gpt-6.1-sol background review of c722c15 found two more, both fixed: a curator's ID edit could deadlock with a book enrichment's provider-ID replace (reproduced; ReplaceByContentID now locks the item first), and the test's lock-wait check could match a waiter in another test database. Kody's index finding is fixed in 3b4addc, and Codex's finding that a resent, unchanged ID still synced its durable row is fixed in 0b739d8. A maintainer /silo review (claude-opus-5-5, claude-sonnet-5-5 subagents, and a gpt-6.1-sol round) found the drift-repair and rebind deadlocks, fixed in ec7f0ae and 43e028c with tests that fail on 0b739d8; Kody then found the repair could write back IDs it read before a waited-for edit, fixed in 7497947, and CodeRabbit found the same gap in the provisional-owner branch, fixed in 53718d2, and Kody's follow-up about the owner changing meanwhile is fixed in 1d68964; plus a stale comment, a doc sentence and duplicate provider constants, fixed in 10b14d5.

🤖 Generated with Claude Code

Note

Fix durable provider IDs to track admin metadata ID edits

  • Admin metadata PATCH requests now normalize provider IDs (trim, lowercase IMDb, strip TMDB URL slugs) and reject invalid nonempty IDs before reaching the service, in admin_catalog_item_metadata.go
  • ItemRepository.UpdateMetadataTx in item_repo.go locks the item row, then syncs changed provider-ID columns with the durable provider-ID table via the new syncEditedProviderIDTx. Clearing an ID deletes the durable row; resent IDs equal after trimming and case folding are skipped; a uniqueness conflict leaves the durable row absent without failing the update
  • Drift repair in provider_id_integrity.go now locks and re-checks the item (and provisional owner) before writing, so a repair does not restore provider IDs read before an admin edit; skipped writes are reported as unresolved
  • Behavioral Change: ProviderIDRepository.ReplaceByContentID takes a FOR NO KEY UPDATE lock on the item row, and MetadataService.rebindItemOnto locks the source item with FOR UPDATE before provider-ID deduplication

Macroscope summarized 1d68964.

Quick104 and others added 2 commits November 14, 2026 16:19
The admin metadata edit stored an IMDb, TMDB or TVDB ID exactly as typed and
left the item's durable provider ID behind. A padded ID ("550 ") missed exact
lookups, and a corrected or cleared one kept the old durable ID, which
lookups, AttachTMDBID's conflict check and a refresh's ID seeding read; a
refresh restored a cleared ID from it.

The v2 PATCH now stores each ID the way a refresh stores it: trimmed, IMDb in
lowercase, a TMDB URL slug dropped. It answers 422 at the member for a value
that is no ID of its provider. On a movie or series, the same transaction
replaces or removes the durable row for each edited provider. An ID another
item already holds stays on the column only, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The six updateDownloadSubscription entries merged with #3535. An approval
lives only on the pull request that needs it, so the next pull request
removes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-triage silo-triage Bot added the size/L 200–499 added + deleted lines (all files); change volume, not priority. label Nov 14, 2026
@coderabbitai

coderabbitai Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Walkthrough

Admin metadata updates now normalize and validate IMDb, TMDB, and TVDB IDs. Catalog writes synchronize durable provider-ID rows, while row locks coordinate edits with provider-ID replacement, drift repair, and item rebind. API descriptions and database tests document and verify these behaviors.

Changes: Provider ID edits

Layer / File(s) Summary
Normalize and validate metadata IDs
contracts/api/v2/openapi.json, web/src/api/v2/schema.ts, internal/apiv2/admin_catalog_item_metadata.go, internal/apiv2/admin_catalog_item_metadata_test.go
The metadata endpoint trims provider IDs, lowercases IMDb IDs, normalizes TMDB IDs, and rejects invalid nonempty values. Empty values remain clearing edits. API descriptions document these rules.
Synchronize durable provider IDs
internal/catalog/item_repo.go, internal/catalog/provider_id_repo.go, internal/catalog/metadata_provider_ids_db_test.go, docs/admin-catalog-api.md, scripts/ci/db-contracts.txt
Metadata edits synchronize durable rows for changed IDs. Clearing an ID removes its row. A duplicate claim leaves the metadata column updated without creating a conflicting row. Tests cover these edits and provider-ID replacement lock waits.
Coordinate provider-ID repair and rebind
internal/metadata/provider_id_integrity.go, internal/metadata/service.go, internal/metadata/provider_id_integrity_db_test.go, scripts/ci/db-contracts.txt
Drift repair checks the locked item against its batch snapshot before writing provider rows. Rebind locks the source item before provider-ID deduplication and movement. PostgreSQL tests cover both operations waiting on edits.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Suggested reviewers: rhainland, blurbery

Merge Risk: Moderate · up to 74979

Provider-ID drift repair can overwrite a provider ID an admin has just edited when it moves a provisional claim. The item then keeps the edited ID in its metadata but the old ID in its durable provider rows. This needs an admin edit to race with repair, so it is unlikely, but it defeats the purpose of this change for that path. Add the same item lock and snapshot re-check to that branch before merging.

Security architecture risk: Moderate · up to 74979

Authentication and catalog-edit permissions remain in place. However, a conflicting edit can remove an item's last durable provider ID and route a later refresh through a less-protected merge path, potentially combining populated catalog entries and their associated state.

Retained concerns

  • Medium · reliability · inferred: A conflicting edit now deletes the item's previous durable row even when the replacement ID belongs to another item. If that was its last durable ID, a subsequent refresh can enter the empty-durable-ID rebind branch with allowMatchedSource enabled. That branch moves library memberships, file links, and user state and can delete the matched source, without the refusal to merge two populated entries present in canonicalization. The base retained the old durable row and therefore skipped this early branch. Same-title checks and curation authorization limit the scenario, but they do not distinguish separate populated editions. Whether alternative base reanchoring paths already produce an equivalent outcome was not fully resolved.
Security review details

Security Blast Radius

  • inferred — The inspected request path requires authenticated curation authority rather than anonymous access. Direct synchronization is limited to the edited item's provider rows, but downstream rebind can affect both involved catalog entries, their library memberships and file associations, and whole-item profile state. The ownership concern therefore extends beyond the edited text field.

Security Findings and Attack Paths

  • inferred — The retained concern is a state-transition path, not an established authorization exploit: an authorized edit selects an already-held ID, synchronization removes the source's last durable ID, and a successful later refresh can select the holder and invoke matched-source rebind. Identity-agreement checks reject contradictory titles, but agreement on external identity does not distinguish two populated editions.

Trust Boundaries and Controls

  • observed — The normalization concern raised by the supplied handler excerpt is resolved for the public route: normalization occurs before the handler copies ID values. Missing authentication or permission wiring fails closed. Durable synchronization neither deletes another item's row nor targets another item's key in its upsert, and the type-scoped unique constraint remains an independent ownership guard.

Resilience and Maintainability Implications

  • observed — The existing canonicalization path refuses to merge two confirmed entries when both have files or library memberships. The early empty-durable-ID path instead invokes rebind, whose inspected transaction moves associations and user state and commits independently before subsequent refresh persistence. Transactional atomicity within each operation does not provide rollback of an already committed rebind if later refresh work fails.

Hardening Proposals

  • proposed — Apply the populated-entry ownership safeguard consistently to the empty-durable-ID fallback, with eligibility checked under the merge locks before associations or user state move. Absence of durable rows should not by itself grant permission to collapse a confirmed, populated source into another populated entry.
Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: admin metadata edits of provider IDs now keep the durable provider-ID rows in step. The summary, file changes, and objectives all match this. The title is specific and…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description is detailed and directly related to the changeset. It explains the problem with IMDb, TMDB, and TVDB ID edits, and describes the fix: normalizing and validating IDs in the PATCH handle…
Full details: Title check

Explanation

The title names the main change: admin metadata edits of provider IDs now keep the durable provider-ID rows in step. The summary, file changes, and objectives all match this. The title is specific and concise, and it uses the fix(catalog) convention.

Full details: Description check

Explanation

The description is detailed and directly related to the changeset. It explains the problem with IMDb, TMDB, and TVDB ID edits, and describes the fix: normalizing and validating IDs in the PATCH handler, and syncing durable provider-ID rows in UpdateMetadataTx. It also covers lock-ordering changes in ReplaceByContentID, drift repair, and rebind, plus the tests and validation performed. This matches the summary of changes, so the check passes.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T23:45:56.841085Z 1d68964 New commits
🔒 Security Review ✅ Completed 2026-11-14T23:49:12.644315Z 1d68964 New commits

Security findings

Advisory findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@silo-triage silo-triage Bot added bug Something isn't working evidence-required Needs before-and-after evidence before the Evidence check passes needs-review Needs maintainer review to resolve uncertain classification, scope, or changed evidence. priority: P2 Limited scope, workaround exists, or polish labels Nov 14, 2026
@silo-triage

silo-triage Bot commented Nov 14, 2026

Copy link
Copy Markdown

Please add phone-width before-and-after captures of Edit Metadata showing the changed ID-save and validation-error behavior, or explain why this behavior is desktop-only. The repository’s Evidence policy treats desktop and mobile web as separate surfaces; desktop captures alone do not cover this change.

Comment on lines +2992 to +2998
if _, err := tx.Exec(ctx, `
DELETE FROM media_item_provider_ids
WHERE content_id = $1 AND lower(provider) = $2`, contentID, provider); err != nil {
return fmt.Errorf("deleting edited %s provider id: %w", provider, err)
}
if value == "" {
return nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug critical

Concurrent provider replacement can undo a cleared ID

A concurrent ReplaceByContentIDTx can restore the old provider ID after an admin clears it. Serialize replacement and edits on the same item before deleting or inserting provider IDs.

Prompt for LLM
File internal/catalog/item_repo.go, lines 2992-2998:

Concurrent provider replacement can undo a cleared ID

A concurrent `ReplaceByContentIDTx` can insert the old provider ID after this delete has found no row, restoring a durable ID that the admin just cleared. Serialize replacement and edits on the same item before deleting or inserting provider IDs.

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this here. The race is real, but it isn't new. A refresh that overlaps an admin edit already overwrites the edited column too: persistItemAndProviderIDsOnceTx waits on the edit's row lock, then upserts the values it read before the edit. The book and manga enrichers that call ReplaceByContentID rewrite the item's durable IDs from their providers on every run, whatever an admin edited. Before this PR the durable row never moved, so the edit now loses to a concurrent refresh in the same way its column already did. Serializing refreshes against edits would be its own change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That distinction makes sense. If overlapping refreshes already overwrite the edited column with values read before the edit, the durable-ID race is part of a broader existing concurrency issue rather than a regression introduced by this PR.

I’m okay leaving the suggestion out of scope here. Serializing refreshes and edits would need to address both the item column and its durable provider IDs together.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: a37e05e now has ReplaceByContentID lock the item before it deletes the item's provider rows, as the edit does. An enrichment replace and an admin edit on the same item now take turns instead of interleaving. A replace that starts after the edit commits still writes its own provider IDs, as it did before this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. The item lock prevents an enrichment replace and an admin edit from interleaving on the same item. A replace that runs after the edit commits can still write its own provider IDs, consistent with the existing behavior.

That addresses the interleaving concern; the broader behavior of subsequent refreshes or enrichment writes remains separate.

Comment thread internal/catalog/item_repo.go Resolved
Wrapping held.provider in lower() kept the check from using the
(provider, provider_id, item_type) unique key, so every edited ID scanned
media_item_provider_ids. That key compares the provider exactly, so an exact
match is also what decides whether the insert would conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: c722c15e7d

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread internal/catalog/item_repo.go Resolved
WHERE mi.content_id = $1
AND NOT EXISTS (
SELECT 1 FROM media_item_provider_ids held
WHERE held.provider = $2 AND held.provider_id = $3

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug high

Case-sensitive holder lookup permits duplicate logical provider IDs

Revises an earlier Kody suggestion on internal/catalog/item_repo.go:3010 (2026-11-14 21:46 UTC).

A holder stored as TMDB can be missed, allowing two items to retain the same logical provider ID. Preserve case-insensitive holder detection with a matching expression index, or canonicalize existing provider rows before using exact comparison.

Prompt for LLM
File internal/catalog/item_repo.go, line 3012:

**Revises an earlier Kody suggestion** on `internal/catalog/item_repo.go:3010` (2026-11-14 21:46 UTC).

Case-sensitive holder lookup permits duplicate logical provider IDs

A holder stored as `TMDB` is missed when an admin assigns the same ID to another item, so the case-sensitive unique constraint accepts two durable IDs for the same logical provider and type. Preserve case-insensitive holder detection, using a matching expression index or canonicalizing existing provider rows before relying on exact comparison; this revises the earlier performance suggestion because the indexed comparison changes conflict semantics.

Suggested code:

WHERE lower(held.provider) = $2 AND held.provider_id = $3

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the exact match. Every reader that decides who owns a durable ID matches the provider name exactly, in lowercase: AttachTMDBID (provider = 'tmdb'), the external-ID lookups (item_repo.go mip.provider = 'tmdb' and mip.provider = r.provider), and #3932's curation owner query ((provider, provider_id) IN (('tmdb', $3), ...)). A row stored as TMDB is already invisible to all of them, so treating it as the holder here would make this check disagree with those decisions. It would block an edit on account of a row no lookup resolves to. If mixed-case provider rows turn up, normalizing them in the writers that store them (normalizeDurableProviderIDs) would fix every reader at once, and that belongs in its own change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Keeping the exact match is consistent with the existing ownership lookups, so I withdraw the suggestion for this PR.

If mixed-case provider rows are possible, addressing them separately should include existing rows as well as the writers; changing writers alone would not make existing TMDB rows visible to lowercase lookups.

A curator's ID edit locks the item FOR UPDATE, then deletes its provider
row. A standalone ReplaceByContentID (book enrichment, a legacy refresh)
deleted the item's rows first and then waited on that lock for its
inserts, so the two deadlocked. It now locks the item first, as the edit
does, and the two queue instead.

The test's lock-wait check now looks only at this database and the
statement it expects to block, since CI runs packages against separate
databases on one server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

An edit form resends IDs the item already has. lockCuratedProviderIDs
treats those as unchanged and skips its checks, so the durable-ID sync now
skips them too and writes only IDs the edit actually changes, compared the
same way (trimmed, any case).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

Comment thread internal/catalog/item_repo.go Outdated
Comment on lines +2979 to +2980
if edited.value == nil || strings.EqualFold(strings.TrimSpace(*edited.value), strings.TrimSpace(storedIDs[edited.provider])) {
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug high

Resending the correct ID preserves a conflicting durable ID

A refresh can restore a stale durable ID even after an admin resaves the correct value. Reconcile divergent durable IDs when the caller is authorized to choose that identity.

Prompt for LLM
File internal/catalog/item_repo.go, lines 2979-2980:

Resending the correct ID preserves a conflicting durable ID

When the scalar ID already equals the submitted value but the durable row is stale, this branch leaves the stale row untouched, as the new test explicitly demonstrates. A refresh can therefore restore the wrong ID even after an admin resaves the correct one; reconcile divergent durable IDs when the caller is authorized to make that identity choice.

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A refresh doesn't restore the stale ID here. prepareProcessRequest (internal/metadata/service.go:1473) loads the durable IDs into the request, and processInternal copies them in (:1547). The scheduled and manual refresh branch then overwrites them with the item's columns (:1897, accumulatedIDs["tmdb"] = existing.TmdbID, and the same for TVDB and IMDb). The save then rewrites the durable rows from that result through ReplaceByContentIDTx. With the correct ID in the column, the next refresh replaces the stale durable row. The durable row can only win when the column is empty or invalid, and an edit that sets a valid ID covers neither case.

Until that refresh, durable-row lookups still see the old row. That's the trade-off of skipping unchanged IDs, which keeps the sync to IDs the curator scope check has judged a change. The web dialog sends only changed IDs, so this needs an API client that resends an ID over a row that has already drifted. I'm leaving it as is rather than adding an admin-only branch.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That distinction makes sense. I withdraw the claim that a refresh necessarily restores the stale durable ID: for a scheduled or manual refresh with a valid ID in the item column, the flow you described gives the column precedence and rewrites the durable row.

The remaining issue is the temporary inconsistency before refresh, when durable-ID lookups can still see the stale row. Given the curator scope check and the web dialog’s changed-fields behavior, leaving unchanged IDs alone is a reasonable trade-off.

Quick104 and others added 4 commits November 14, 2026 17:46
…der IDs

An admin's ID edit now locks the item and then deletes the edited
provider's row. The startup provider-ID drift repair deleted the item's
rows first and needed the item for its inserts' foreign-key check, so an
edit and a repair of the same item could deadlock and fail the edit.
The repair now locks the item first, as ReplaceByContentID does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drops the package's second copy of the tmdb, imdb and tvdb names, keeps
each stored ID beside its provider instead of in a map, and corrects the
sync's comment now that ReplaceByContentID locks the item. The admin API
doc now says a resent, unchanged ID leaves the durable ID alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An automatic rebind moved the skeleton's provider rows and only locked
the skeleton at its final delete. An admin's ID edit locks the item and
then deletes one of those rows, so the two could deadlock and fail the
edit or the refresh. The rebind now locks the skeleton after its files
and before its provider rows, the order the edit and refresh use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Quick104 Quick104 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four findings, one medium and three minor. I'm fixing all four on this branch; a summary comment follows once the fixes are pushed.

🤖 Claude Code (claude-opus-5-5)

Comment thread internal/catalog/item_repo.go Resolved
Comment thread internal/catalog/item_repo.go Resolved
Comment thread docs/admin-catalog-api.md Resolved
Comment thread internal/catalog/provider_id_repo.go Resolved
@silo-triage silo-triage Bot added size/XL 500–999 added + deleted lines (all files); change volume, not priority. and removed size/L 200–499 added + deleted lines (all files); change volume, not priority. labels Nov 14, 2026
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

@Quick104

Quick104 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor Author

Review summary

Reviewed head: 1d689649

Fit: 9/10 · Readiness: 9/10 — ready (assuming CI is green)

The PR fixes a catalog-correctness bug at the endpoint that causes it: an admin's ID edit now stores a clean ID and moves the durable provider ID with it, so lookups and refreshes stop using the old one. It adds no new setting or field; the contract change is descriptions plus a 422 for values that were never IDs.

Fixed on this branch

  • Deadlocks with two background writers (ec7f0ae9, 43e028c5). The edit holds the item and then deletes one of its provider rows. The startup provider-ID drift repair and an automatic rebind of a skeleton onto its matched item did the reverse: they changed the item's provider rows first and needed the item afterwards. Either could deadlock with an edit of the same item, failing the save with a 500 or failing the refresh. Both now lock the item first, as ReplaceByContentID does in this PR. TestProviderIDDriftRepairWaitsForItemEditPostgres and TestRebindWaitsForItemEditPostgres fail on 0b739d8a with deadlock detected and pass now. A gpt-6.1-sol review round found the rebind case.
  • Drift repair writing stale IDs (74979470, from Kody). A repair that waited behind an ID edit then wrote the IDs it had read before the edit. It now re-reads the item's IDs under its lock and skips an item that changed. CodeRabbit then found the same gap, plus the deadlock, in the repair's provisional-owner branch (53718d2f); both branches now share one locked check. The drift test runs the race against both branches: the clean-insert case fails on 43e028c5, the provisional-owner case on 74979470. Kody then noted the provisional owner itself could change while the repair waited; the repair now also checks, under the owner's lock, that the owner still holds one of the IDs (1d689649, with a test that fails without the check).
  • Comment, doc and constants (10b14d5c). The sync's comment no longer names ReplaceByContentID as a writer that skips the item lock. The API doc now says a resent, unchanged ID leaves the durable ID alone. The sync uses contentid's provider names instead of a second copy.
  • Branch update. Main merged in cleanly. fix(plugins): reuse saved secrets in connection tests only with the saved settings #3923 had already emptied breaking-approvals.json, so this PR no longer changes it, and the description no longer lists that commit.

Left for the maintainers

  • An ID with leading zeros (0550) still passes and is stored as typed, which the PR discloses and refresh also does. Stripping them would change the shared ID rules, so it belongs in its own change.
  • Three Kody threads are answered, and Kody withdrew or accepted each one; they can be resolved.

Tests

  • The PR's TestMetadataUpdateKeepsDurableProviderIDsDB and TestAdminItemMetadataNormalizesProviderIDs fail on the base for the reported reasons: the durable row stays on the old ID, and the API stores the value as typed.
  • go test for internal/metadata, internal/catalog and internal/apiv2 passes against a freshly migrated Postgres 18. make lint-changed and the OpenAPI and local-path checks pass. The generated web types are left to CI.

Validated features

#1210 and #1211 C1 and C2 go through this endpoint, and each gets a minor change: an edited ID is cleaned or refused, and a saved ID now survives a refresh. No regression. The Validation tasks: line matches.

Other checks

  • User manual: unaffected. It doesn't describe the External IDs fields, and the refusal message names the expected format.
  • Clients: Apple and Android have no metadata edit screen; the web dialog sends only changed IDs and shows the refusal. jellycompat has no route that writes provider IDs.

Evidence: https://evidence.siloserver.org/r/silo-server/admin-provider-id-edit/

🤖 Claude Code (claude-opus-5-5)

The repair reads an item's ID columns in its batch query and only locks
the item later. When it waited behind an admin's ID edit, it then wrote
the IDs it had read before the edit over the edited durable ID. It now
re-reads the columns under the lock and leaves the item for the next run
when they changed, or when the item is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Major · Re-check the edited item before moving a provisional claim. · provider_id_integrity.go:375

internal/metadata/provider_id_integrity.go:375
Major · Data Integrity & Integration · Heavy lift

Re-check the edited item before moving a provisional claim.

If repair reads item A with TMDB X and provisional owner B, an admin can then change A to Y and commit its durable Y row. This branch locks only B. Line 375 deletes A's new row, and the following insert restores stale X. A then has column Y but durable X.

Lock A and re-check its type and IDs before deleting either item's provider rows. Use a consistent lock order for A and B. The new check in insertProviderIDsForDriftRow does not protect this branch.

Based on learnings, prevent an intervening state change from being silently overwritten; use “row locks held from SELECT through commit” or an equivalent checked predicate.

Source: Learnings

Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4f5b8062-5880-489d-9a76-de09142bb66d
Commits

Reviewing files that changed from the base of the PR and between 0b739d8 and 7497947.

Files selected for processing: 7

  • docs/admin-catalog-api.md
  • internal/catalog/item_repo.go
  • internal/metadata/provider_id_integrity.go
  • internal/metadata/provider_id_integrity_db_test.go
  • internal/metadata/service.go
  • scripts/ci/db-contracts.txt
  • web/src/api/v2/schema.ts
Files skipped from review as they are similar to previous changes: 1

  • web/src/api/v2/schema.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

… claim

The drift repair's provisional-owner branch locked only the owner, then
deleted the drifted item's provider rows and wrote the IDs it had read in
its batch. Like the clean-insert branch before 7497947, it could
deadlock with an admin's ID edit of that item, or write back the IDs the
edit replaced. Both branches now share one check that locks the item
first and skips it when its IDs changed; the owner is locked after it,
the order the edit takes the item and the holders of its IDs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Quick104

Copy link
Copy Markdown
Contributor Author

Re: CodeRabbit's outside-diff finding on moveProvisionalProviderClaim (provider_id_integrity.go:375): fixed in 53718d2. The provisional branch locked only the owner, so it could write back IDs an admin's edit had just replaced, and, like the clean-insert branch before ec7f0ae, it could deadlock with that edit: its inserts needed the edited item while the edit waited on the rows it had deleted. Both branches now share lockUnchangedDriftItem, which locks the item first and skips it when its type or IDs changed since the batch read; the owner is locked after it, the order the edit takes the item and then the holders of its IDs. TestProviderIDDriftRepairWaitsForItemEditPostgres now runs the race against both branches; its provisional-owner case fails on 7497947 with deadlock detected (SQLSTATE 40P01) and passes now.

🤖 Claude Code (claude-opus-5-5)

@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌

The repair picks a provisional owner before it locks anything, and after
53718d2 it can wait on the drifted item first. It then deleted all of the
owner's provider rows even when an edit had moved the owner off the ID in
the meantime. It now checks, under the owner's lock, that the owner still
holds one of the IDs, and leaves the owner for the next run otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@silo-kody

silo-kody Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo
  • Include the related issue, expected behavior, and validation steps in the PR description.
  • For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
  • For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
  • Follow this repository's AGENTS.md and CONTRIBUTING.md.
  • Request another review with @kody start-review in a PR comment.
  • React with 👍 or 👎 to give feedback on individual suggestions.
Review settings
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ❌
⚠️ 4 Kody Rule(s) were not evaluated

These rules declare that they need repository context beyond this diff, and the review could not retrieve it, so they were not judged on this pull request:

  • Preserve optional store capabilities through production wrappers
  • Detach mandatory cleanup from request cancellation and bound its lifetime
  • Preserve durable media identity and share identity-evidence precedence
  • Bound decoder throughput trades and share direct-play eligibility

@Quick104
Quick104 merged commit 3f73ecb into main Nov 15, 2026
22 checks passed
@Quick104
Quick104 deleted the fix/admin-metadata-provider-id-normalize branch November 15, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working evidence-required Needs before-and-after evidence before the Evidence check passes needs-review Needs maintainer review to resolve uncertain classification, scope, or changed evidence. priority: P2 Limited scope, workaround exists, or polish size/XL 500–999 added + deleted lines (all files); change volume, not priority.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant