Repository navigation
fix(boxsets): fetch box sets in the library's metadata language - #4513
Conversation
The box set sync asked TMDB for each collection without a language, so box set names, overviews and posters were always English. It now asks in the metadata language of the first box set library holding one of the collection's movies, records that language with the stored collection, and fetches a collection again when the language it is due in changes. Closes Silo-Server#4485
TMDB leaves a collection's name or overview empty where its translation lacks them, so the sync now takes those from TMDB's English answer, as item metadata falls back. Also share the box set membership SQL between the fetch and cleanup queries, read the stored language back, simplify the TMDB language mapping, and record the language rule in the box set architecture note.
A failed English request no longer holds back a collection TMDB already returned in the library's language; it is stored as translated unless it has no name.
|
Warning Review limit reachedThis organization requires a seat for paid usage-based reviews, including automatic and on-demand reviews. Ask an admin to assign a seat or enable paid usage-based reviews for users without a seat in Billing. Next included review available in 52 minutes. View limit details
|
Silo Kody — review completeReview finished. Check the inline comments for findings and verify each suggestion against the code and tests. Reviewing changes in Silo
Review settingsReview OptionsThe following review options are enabled or disabled:
|
| return nil, err | ||
| } | ||
| localized := *c | ||
| localized.Name = cmp.Or(c.Name, english.Name) |
There was a problem hiding this comment.
English fallback names lose their language provenance
An English fallback name becomes indistinguishable from one supplied in the requested language because the collection records only fetch.Language. Track and persist the provenance of localized.Name independently.
Also found in:
internal/boxsets/sync.go:234-234
Kody rule violation: Track localization provenance independently for each metadata field
Prompt for LLM
File internal/boxsets/sync.go, line 233:
English fallback names lose their language provenance
Preserve the name's language provenance when filling it from the English response. The stored collection records only `fetch.Language`, so an English fallback name becomes indistinguishable from a name supplied in the requested language. Track the name's provenance independently and persist it with the collection.
**Also found in:**
- `internal/boxsets/sync.go:234-234`
Kody rule violation: [Track localization provenance independently for each metadata field](https://kodus.siloserver.org/settings/code-review/1247142351/kody-rules/705436fb-f63f-4782-bc30-9184ebbd56df?teamId=ba9922fc-c0ea-4cf8-87bb-9d4f42ffc73e)
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.
There was a problem hiding this comment.
Not changing this. The row is refetched every 7 days, and any time the language it is due in changes, so a translation TMDB adds later replaces the filled-in name or overview on the next refresh without per-field tracking. Nothing reads a separate language for a box set's name or overview. In practice TMDB fills a missing collection name with its default name itself (/collection/10?language=ta returns "Star Wars Collection"), so the fill mostly applies to the overview. The fill now uses the server's metadata.fallback_language rather than English (4ab7be8).
🤖 Claude Code (claude-opus-5-5)
There was a problem hiding this comment.
That makes sense. The stored language tracks the requested collection language, not the provenance of every field. Since the collection is refreshed every seven days and when its requested language changes, a newly available translation can replace fallback text without per-field tracking.
I withdraw the suggestion for this PR. Using the configured metadata.fallback_language also makes “English fallback” an outdated description.
| if err != nil { | ||
| if c.Name != "" { | ||
| slog.WarnContext(ctx, "box sets: English collection fallback failed; storing the translation as it is", "component", "boxsets", "tmdb_collection_id", fetch.ID, "error", err) | ||
| return c, nil |
There was a problem hiding this comment.
A failed English fallback leaves the overview empty for seven days
Preserve the translated collection in c while tracking the failed fallback separately so it can be retried on the next run.
Prompt for LLM
File internal/boxsets/sync.go, lines 225-228:
A failed English fallback leaves the overview empty for seven days
A transient English fallback failure stores an empty overview as a successful, fresh collection, so the missing English text is not retried for seven days even if TMDB recovers immediately. Preserve the translated collection while tracking the failed fallback separately so it can be retried on the next run.
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.
There was a problem hiding this comment.
This holds up: a failed fallback request stored the translation with a blank overview, which overwrote any overview already stored and marked the row fresh for 7 days. Fixed in 45c12e0. A failed fallback answer now fails the fetch, so the stored row keeps its text and the collection is tried again on the next run. The error isn't wrapped as ErrNotFound, so a 404 on the fallback request can't mark the row fetched. TestRunRetriesWhenTheFallbackFails fails on the previous head (stored = true, failures = 0) and passes now.
🤖 Claude Code (claude-opus-5-5)
There was a problem hiding this comment.
Thanks for the update. The reported fix addresses the retry issue: a failed English fallback leaves the existing collection unchanged and allows the next sync to retry. The regression test also covers the previously observed behavior.
| if err != nil { | ||
| if c.Name != "" { | ||
| slog.WarnContext(ctx, "box sets: English collection fallback failed; storing the translation as it is", "component", "boxsets", "tmdb_collection_id", fetch.ID, "error", err) | ||
| return c, nil |
There was a problem hiding this comment.
Fallback failures bypass the TMDB outage circuit breaker
An English-endpoint outage triggers retries for every collection with a translated name instead of stopping the run. Count fallback failures toward res.Failures while preserving the translated data.
Prompt for LLM
File internal/boxsets/sync.go, lines 225-228:
Fallback failures bypass the TMDB outage circuit breaker
English fallback failures for collections with translated names bypass `res.Failures` and the 25-failure circuit breaker, so an English-endpoint outage triggers retries for every such collection instead of stopping the run. Count fallback failures toward the existing failure budget while still preserving the translated data.
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.
There was a problem hiding this comment.
Fixed by the same change in 45c12e0. A failed fallback answer now counts as a failed fetch, so it counts toward the run's 25-failure limit (res.Failures); TestRunRetriesWhenTheFallbackFails checks for one failure.
🤖 Claude Code (claude-opus-5-5)
There was a problem hiding this comment.
Thanks for the update. Based on your description, commit 45c12e082 addresses the circuit-breaker concern by counting fallback failures toward the 25-failure limit. I haven’t independently verified the change or test.
Item metadata fills text a translation lacks from metadata.fallback_language and leaves it empty when that setting is the original language. The box set sync always asked TMDB for English instead. It now reads the same setting, hot-reloaded through the metadata service. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A library holding fewer of a collection's movies than its box set minimum does not show the box set, so it no longer picks the language the shared row is fetched in while another library shows it. A failed fallback-language answer now fails the fetch. Storing the translation without it wrote a blank overview over the stored one, marked the row fresh for a week, and kept the failure out of the run's budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Quick104
left a comment
There was a problem hiding this comment.
🤖 Claude Code (claude-opus-5-5)
Review of 1e8af96. Two new findings below, and two of the existing Kody threads hold up (answered in those threads). I'm fixing all of them in commits on this branch, so there's no need to change anything yourself.
Silo Kody — review completeReview finished. Check the inline comments for findings and verify each suggestion against the code and tests. Reviewing changes in Silo
Review settingsReview OptionsThe following review options are enabled or disabled:
|
| ORDER BY m.tmdb_collection_id, count(*) >= GREATEST(lib.box_set_min_movies, $4) DESC, lib.sort_order, lib.id | ||
| ) due | ||
| LEFT JOIN tmdb_movie_collections c ON c.tmdb_collection_id = due.tmdb_collection_id | ||
| WHERE c.tmdb_collection_id IS NULL OR c.fetched_at < $1 OR c.language <> due.language |
There was a problem hiding this comment.
Fallback language changes do not invalidate fresh collections
Changes to metadata.fallback_language can leave freshly fetched collections unchanged for up to seven days because the due predicate checks only the primary language and fetch time. Track the fallback language used for each collection or invalidate affected collections when the setting changes.
Prompt for LLM
File internal/catalog/box_sets.go, line 237:
Fallback language changes do not invalidate fresh collections
Changing `metadata.fallback_language` leaves freshly fetched collections unchanged for up to seven days because the due predicate only compares the primary language and fetch time. Track the fallback language used for each collection or invalidate affected collections when the setting changes.
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.
There was a problem hiding this comment.
Not changing this in this PR. metadata.fallback_language only fills a name or overview the collection's translation lacks. When an admin changes it, the affected collections pick up the new fallback on their next refresh, at most 7 days later. Item metadata works the same way: changing the fallback language doesn't refetch items either. Tracking the fallback language per collection would need another column and refetch rule for a rare settings change. If a refetch on change is wanted, it should apply to items and box sets together.
|
🤖 Claude Code (claude-opus-5-5) Review summaryReviewed 1e8af96. I pushed fixes 4ab7be8 and 45c12e0 on top, so the head is now 45c12e0. There was no rebase. Fixed on this branch
Verification
Left for the maintainers
Other checks
Evidence: https://evidence.siloserver.org/r/silo-server/pr-4513-review/ Fit: 9/10 · Readiness: 9/10 — ready (assuming CI is green) Fit: box sets were the one part of a movie library that ignored its metadata language, and this fixes that inside the existing box set sync. The added column is internal and the API is unchanged. Readiness: the fix is shown end to end against real TMDB on main and on the head, and each review fix has a test that fails without it. The rolling-update window above is the only known gap, and it is disclosed. |
Problem
Closes #4485
Related issue: N/A
Validation tasks: none
Other clients: Apple N/A (no client change; clients show the name and poster the server sends); Android N/A (same); Roku N/A (same)
Box sets built from TMDB collections always came in English, whatever the library's metadata language. A French or German movie library showed "Star Wars Collection" with the English poster and overview, while its movies, series and seasons were localized. The box set sync asked TMDB for each collection without a
languageparameter, so TMDB answered in English. Collections already stored would also stay English, because a stored collection was only fetched again when it went stale.With this change the sync asks for each collection in its library's metadata language and stores which language it used. Collections stored in another language are fetched again on the next sync, so existing installs pick up the translation, and so does a library whose language changes later.
Approach
tmdb_movie_collectionsgains alanguagecolumn. Rows stored before this change default toen, which is what TMDB returned for them, so English libraries do not refetch anything.enasen-US, other codes as given. TMDB picks the matching poster itself.metadata.fallback_language), the way item metadata falls back, and stay empty when that setting is the original language. If that second request fails, the fetch fails: the stored row keeps its text and the next run tries again.Validation
go build ./...,gofmt,go mod tidy -diff,go vet ./...,make lint-changed,make lint-router-recovery: pass.make verify-migrations,verify-local-paths,verify-bidi,verify-size-ratchet,verify-route-inventory: pass.language, with a German movie library holding two movies of one collection: reproduced onmain, fixed on this branch (see Evidence).TestBoxSetCollectionLanguageDB(a library below its box set minimum) andTestRunRetriesWhenTheFallbackFailsfail on the commit before 45c12e0 and pass now;TestRunFillsNothingWhenTheFallbackIsOffcovers the fallback setting.make test-gosuite and the web half of the gate; this change touches no web code.Evidence
A local build with a movies library set to
deand box sets on, two synthetic movies in TMDB collection 10, and a fake TMDB that answers/collection/10in German only when asked forlanguage=de.Before, on
main, the sync requests/3/collection/10with no language and the API returns:After, on this branch, against the same database, the sync requests
/3/collection/10?language=deand the API returns:Switching the library to
enand syncing again requestslanguage=en-USand returns "Star Wars Box Set" with the English poster. When the German answer has no overview, the sync makes a second request without a language and stores the German name with the English overview. Box set search (GET /api/v2/catalog/box-sets?q=star) returns the same localized fields.The review fixes change no UI; their tests and a query timing are on the evidence page.
Evidence: https://evidence.siloserver.org/r/silo-server/pr-4513-review/
Risks
ADD COLUMN ... NOT NULL DEFAULT 'en'ontmdb_movie_collections, a metadata-only change in PostgreSQL. An older node that runs the sync during a rolling update still fetches stale collections in English without touching the column, so such a row can hold English text while recorded as another language until it goes stale again (7 days).Checklist
AI Disclosure
Note
Fetch movie box sets in the library's metadata language
catalog.BoxSetRepository.CollectionsNeedingFetch(box_sets.go). Collections stored in a different language are scheduled for a refetch.tmdb.Client.GetCollectionaccepts a language parameter, and the syncer fills missing name or overview text from the metadata service's configured fallback language via the newSyncer.WithFallbackLanguagesetter (sync.go).languagecolumn, added by 20261118183932_box_set_collection_language.sql with an English default for existing rows.CollectionsNeedingFetchandTouchCollectionsignatures changed, andMetadataService.FallbackLanguageis now exported.Macroscope summarized 45c12e0.