Repository navigation
fix(metadata): preserve concurrent edits during hydration (#2758) - #2794
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Early look since it's still a draft, no action needed yet. Reading `CAST(updated_at AS TEXT)` and comparing against that same raw text is the right fix for the problem that sank the earlier version: it's correct whatever shape the row holds, so none of the nine historical layouts can make it silently refuse. The guard is only as good as "every writer of a guarded column bumps `updated_at`", so I went through main's `UPDATE books` statements that don't:
The main import path is fine: `AddBookFile` goes through `refreshBookStatus` into `Update`, which bumps it. |
c4677db to
4588a40
Compare
|
Thanks for walking through the The Verifying that turned up one more pre-existing problem in the same fallback, now open as #2819. The fallback takes the legacy write on any #2759 has merged, so I've rebased onto main. The diff is now just this PR's three commits. I'll mark it ready for review once CI is green. |
|
Thanks! Heads up that #2824 landed after your fix and removed the |
) Guard metadata writes with the original update timestamp and edition writes with the parent identity observed before fetching. Advance the book timestamp when imported EPUB language changes. Reload current book state before callers search, including duration-only, empty, unchanged, and failed fetch outcomes. Preserve ingestion provider context only while the work identity still matches. Keep the repository interfaces, hydration callers, concurrency regression tests, and documented behavior together. Signed-off-by: magrhino <12738722+magrhino@users.noreply.github.com>
…vallee#2758) books.updated_at holds several historical shapes: CURRENT_TIMESTAMP from migrations 082 and 084, and Go's time.String form from vavallee#914. Comparing the column against a reformatted time.Time never matched those rows, so every refresh silently discarded the hydrated metadata, and an unparseable value failed with an invalid-snapshot error. Carry the stored text on the book as UpdatedAtRaw, read through CAST(updated_at AS TEXT) because the driver normalises DATETIME columns, and compare that text in UpdateHydratedMetadata. No schema change. Cover all four stored shapes, a concurrent edit through every writer that bumps updated_at, and the identity the duration reload returns. Signed-off-by: magrhino <12738722+magrhino@users.noreply.github.com>
4588a40 to
3763daa
Compare
|
Rebased onto main and dropped the fallback hunk and subtest now that #2824 removed it. |
#2926) (#3008) * fix(metadata): guard audnex enrichment writes against concurrent edits (#2926) Three paths made an Audnex call and then wrote the whole book row from the snapshot taken before it, so an edit saved during the call was lost: the ABS importer, the Hardcover list syncer and the manual enrich endpoint. They now write through the same updated_at guard hydration uses since #2794. The background paths use UpdateHydratedMetadata, which writes only the enrichment columns and, on a lost guard, reloads the row so the rest of the import or sync works from the user's edit. The manual endpoint retries once from a fresh read (the Audnex answer is cached, so this normally costs no second request) and returns 409 if the row changes again. When the endpoint also maps the book onto the canonical work for its ASIN it needs the whole row, so BookRepo gains UpdateIfUnchanged, the guarded sibling of Update. Rebind had the same shape around its provider fetch and now uses UpdateIfUnchanged too, answering 409 on a lost guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(web): tell a changed book apart from a duplicate in rebind conflicts (#2926) The rebind modal turned every 409 without force_required into "This metadata record conflicts with an existing book", so a user whose book was edited during the rebind was told the target id was taken. Both new 409s (rebind and enrich) now carry reason "changed", and the modal shows its own string for it. The enrich button already showed the server's message; a test now pins that. List sync logs a dropped enrichment at INFO, since it is not retried on later syncs, and the comment no longer claims the reloaded row reaches the widen path's queued search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> --------- Signed-off-by: vavallee <vavallee@protonmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
#2423) (#3107) * fix(db): refresh a book's status without rewriting the whole row (#2375) refreshBookStatus read the book, derived its status and file paths, then wrote every column back through Update from that snapshot, so a user edit that landed between the read and the write was reverted. It now writes only the five columns it derives, guarded on the media_type and status it derived them from, and derives again when either changed. The second call site #2758 found (HydrateHardcoverEditions) was already moved onto the guarded UpdateHydratedMetadata in #2794. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): queue a library scan requested while one is running (#3014) POST /library/scan answered 409 while a scan was walking and dropped the request, so a file placed in a folder the walk had already passed waited for the next scheduled scan. A request made during a scan now sets a pending flag and gets 202 with queued=true; the running scan runs exactly one follow-up when it finishes, however many requests arrived. The gate and the flag change together under a mutex so a request racing the end of a scan is never lost. The cron path still skips rather than queues, but runs a manual request queued during it. Reported by foobarbigtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): reconcile an untracked audiobook beside a tracked one in a flat layout (#1985) scanLibrary marked the parent folder of every tracked audio file as tracked so the sibling tracks of a multi-track audiobook count as already tracked (#1436). In a flat Author/Title.mp3 layout that parent is the author folder, so every other audiobook by the author was absorbed and never reached the matching tiers or Unmatched. An audio file with no book folder of its own (directly in a root or in an author folder) no longer marks its parent. The folder instead remembers the track stems of its tracked files (the name with part, track and chapter numbers removed), and only a sibling with the same stem, or a bare track number, is absorbed. Book folder layouts are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(web): refresh the book page while its download is importing (#2423) The book page's live refresh (#1161) armed on book statuses nothing ever wrote, so it never ran, and #2374 removed it. GET /book/{id} now carries importInFlight, set while a download row for the book is grabbed, downloading, completed, importPending or importing. The parked hand-off states are left out so a page does not poll for hours. The page polls the book every 5s while the flag is set and refreshes the history on the reload that clears it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): let Fix match correct the link without moving the file (#2055) Fix match ran the full import, so correcting which book a file belongs to also moved it into the target's folder and renamed it from the naming template. #2224 added the warning and destination preview; this adds the other half the issue asked for. POST /queue/manual-import/reassign takes relocate. false moves the book_files row to the target book through MoveBookFile, keeps the stored path spelling, records bookFileMoved in history and answers 200; no import runs and nothing on disk changes. true or absent is the old import, so API callers are unchanged. The modal now asks what should happen to the file and defaults to Correct the match only; the move option keeps the destination preview and the no undo warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): compare only the text before a track number in a flat folder (#1985) Review of the first pass: audioTrackStem stripped numbers anywhere, so track names carrying chapter titles (01 - Chapter One, Title - 02 - The Middle), spelled parts (Part Two), n of m and a glued CD1 got different stems and became unmatched rows where main had absorbed them. The stem is now the text before the first track number (digits, or a spelled number after a track label), with track labels removed; a name that starts with its number is a bare track and is absorbed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(api): scope the book page's import in flight flag to the queue's owner (#2423) HasImportInFlight counted every download for the book, so under tenancy a user viewing a book nobody owns could learn that another user had a grab in flight. It now takes the same QueryScope owner filter as the queue, with ListScopeUserID from the request (admins and API key unscoped). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): name the scan a library scan request will be answered by (#3014) The Settings page matched its scan's result by comparing the browser clock with ran_at, which can pick up the scan it queued behind, and gave up after two minutes even while the server was still scanning. Each scan now gets an id (boot id plus sequence), the 202 returns the id the requested scan's result will carry (scanId), and the stored result carries scan_id. The status endpoint adds the live running and queued state. The page waits for its own scan_id and only gives up after two minutes when the server reports nothing walking or queued. Also tests the scheduled path: a scheduled scan walking, a manual request queued behind it, a second scheduled tick skipping, and exactly one follow-up carrying the manual request's id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(db): guard the status refresh on the stored file paths too (#2375) The guarded write compared only media_type and status, so a concurrent refresh that moved a path without changing either (a reorganize running UpdateBookFilePath) could be overwritten by an older derivation, putting the old path back. The guard now also compares the raw ebook_file_path and audiobook_file_path columns, read directly rather than through the bookColumns book_files view, with IS so a NULL read still matches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(importer): record history when Fix match links a file nothing tracked (#2055) RelinkFile wrote a bookFileMoved row only when it took the file from another book, so linking an untracked file changed the book with nothing on its History. It now writes the row in that case too, without the from fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * fix(web): translate the library scan give up message (#3014) The message the scan poll shows when it stops waiting was a hard coded English string with a dash; it is now settings.general.scanCheckBack, read through a ref so the poll does not restart when t changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> * test(api): expect the live scan state in the admin scan status body (#3014) TestLibraryScanStatusRouteAllowsAdmin compared the admin body byte for byte with the stored blob, which no longer holds now that the status adds the live running and queued flags. It now checks every stored field comes back unchanged and that the only additions are those two booleans. The non admin case is unchanged and still refused before the handler with no paths in the body. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com> --------- Signed-off-by: vavallee <vavallee@protonmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Split out of #2759 at the maintainer's request. Hardcover edition hydration no longer overwrites book edits made while editions are being fetched, and it no longer silently drops its own results on rows whose
updated_atwas stored in a legacy shape.Closes #2758.
What changed
BookRepo.UpdateHydratedMetadatawrites only the hydrated columns, and only if the row hasn't changed since the snapshot. When a concurrent edit wins, the book is reloaded, so the caller can't act on a discarded ASIN or discarded metadata.EditionRepo.UpsertMetadatachecks that the parent still has the foreign ID and provider seen before the fetch. The check runs in the same statement as the insert or conflict update.SetLanguagenow bumpsupdated_at, so a pending hydration sees an imported EPUB language.How the precondition compares
updated_atThe first version of the guard compared
updated_atagainst a reformattedtime.Time.books.updated_atholds several historical shapes, and for the two legacy ones the guard never matched: it returnedupdated=false, err=nil, so every refresh discarded its results.The guard now keeps the column text exactly as stored, in
Book.UpdatedAtRaw, and compares that. It reads the column throughCAST(updated_at AS TEXT), because the SQLite driver turnsDATETIMEcolumns into timestamps even when the scan target is a string. There is no schema change.updated_atshape2026-09-01 10:00:00CURRENT_TIMESTAMP(migrations 082, 084)2026-09-01 10:00:00.123456789 +0000 UTCtime.String()(#914, still written by author merge, #2792)invalid book snapshoterrorI chose this over a row-version column because it needs no migration. Like any timestamp guard, two writes in the same clock tick would look identical. A version column is the only full fix for that, and that would be a separate change.
Tests
TestBookRepoUpdateHydratedMetadataLegacyUpdatedAtShapescovers all four stored shapes:updated_atstill wins:SetLanguage,SetImageURL,MarkWantedMonitored,SetExcluded, and author merge. It fails for a writer that stops bumping the timestamp.TestHydrateHardcoverEditionsPersistsOverLegacyUpdatedAtruns hydration end to end on both legacy shapes and checks that the result flags and persisted metadata survive. It fails against the old reformatted comparison.TestBookRepoFillMissingAudiobookDurationRequiresMatchingIdentity(from fix(metadata): retain hardcover audiobook runtime before search (#2755) #2759) now also checks that the reloaded duration comes with its own identity, not a stale one.Follow-ups (not in this PR)
books.updated_atin the Books page blank — unsupported Scan error on legacy date formats from Calibre import #914 shape.SetFilePathfallback changesstatuswithout bumpingupdated_at.Checklist
git commit -sdocs/DEPLOYMENT.mdupdated if env vars, config, or upgrade path changed (not applicable)changelog.d/(2758-hydration-concurrent-edits.md)docs/User-Guide-Wiki.md)Test plan
go test -count=1 ./cmd/... ./internal/...go vet ./internal/...cd web && npm run build(not run; no web changes)