Skip to content

fix: data integrity and library scan batch (#2055, #2375, #1985, #3014, #2423) - #3107

Merged
vavallee merged 14 commits into
mainfrom
fix/data-integrity-scan
Oct 8, 2026
Merged

vavallee merged 14 commits into
mainfrom
fix/data-integrity-scan

Conversation

@vavallee

@vavallee vavallee commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Closes #2055
Closes #2375
Closes #1985
Closes #3014
Closes #2423

Not included: #2793 is already fixed on main by #2824 (64095b0): SetFilePath no longer has the fallback write at all, it returns the load error instead. It can be closed against that.

What

One commit per issue.

#2375 refreshBookStatus rewrote the whole row. It read the book, derived status and paths, then wrote every column back through Update from that snapshot, so a user edit landing in between (unmonitor, rename, media type) was reverted by an import recording its file. It now writes only the five columns it derives plus updated_at, guarded on the media_type and status it derived them from, and derives again (up to five attempts) when either changed. The second call site from #2758 (HydrateHardcoverEditions) was already moved onto UpdateHydratedMetadata in #2794.

#3014 scan requested during a scan. StartScan now sets a pending flag instead of refusing and returns ErrScanQueued; the endpoint answers 202 {"message":"library scan queued","queued":true}. The running scan runs exactly one follow-up when it finishes, however many requests arrived. Claim, release and the flag change together under a mutex so a request racing the end of a scan is never lost; a cancelled context drops the queued request. The cron path still skips rather than queues, but runs a manual request queued during it. Settings says the scan was queued instead of "Scan failed". Reported by foobarbigtime.

#1985 flat layout audiobooks. An audio file with no book folder of its own (directly in a root, or in Author/) no longer marks its parent as tracked. That folder instead remembers the track stems of its tracked files (name with part, track, chapter, disc numbers stripped), and only a sibling with the same stem or a bare number (02.mp3) is absorbed. Book folder layouts are unchanged. Known limit, documented: titles differing only by a number (Saga 1.mp3, Saga 2.mp3) still read as tracks of one book, which is the old behaviour for them.

#2423 book page live refresh. GET /book/{id} carries importInFlight, true while a download row for the book is grabbed, downloading, completed, importPending or importing. The parked states (importExternal, importHeld) are left out so a page never polls for hours. The page polls the book every 5s while it is set (via usePolling, so it pauses on hidden tabs) and refreshes history on the reload that clears it.

#2055 Fix match, correct match only. #2224 shipped the warning and destination preview; this is the deferred half. POST /queue/manual-import/reassign takes relocate. false moves the book_files row to the target through MoveBookFile, keeps the stored path spelling, writes a bookFileMoved history row and answers 200: no import, nothing on disk changes. true or absent is the old import, so API callers are unchanged. The modal asks what should happen to the file and defaults to Correct the match only; Also move and rename the file keeps the destination preview and the no undo warning.

Fail before evidence

Each test below was run against the main code and failed, then passed with the fix:

Checks

gofmt -l: clean
go vet ./...: clean
golangci-lint (v2.11.4) on internal/api, db, importer, models: 0 issues
go test internal/api internal/db internal/importer internal/models: ok
web: npm ci, lint (0 errors, existing warnings only), typecheck, vitest 150 files / 1845 tests passed, build ok

No migration.

Docs

docs/API.md (scan queued answer, importInFlight, relocate), docs/User-Guide-Wiki.md (Fix match modes, flat layout scan), docs/Migrating-From-Readarr-Wiki.md (Fix match now matches Readarr by default). One changelog fragment per issue.

Review follow ups

Merged main (no conflicts). Rechecked: vet, golangci-lint 0 issues, go test api/db/importer/models ok, web lint/typecheck, vitest 150 files / 1852 tests, build.

🤖 Generated with Claude Code

https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9

vavallee and others added 5 commits October 7, 2026 23:21
refreshBookStatus read the book, derived its status and file paths, then
wrote every column back through Update from that snapshot, so a user edit
that landed between the read and the write was reverted. It now writes
only the five columns it derives, guarded on the media_type and status it
derived them from, and derives again when either changed.

The second call site #2758 found (HydrateHardcoverEditions) was already
moved onto the guarded UpdateHydratedMetadata in #2794.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…3014)

POST /library/scan answered 409 while a scan was walking and dropped the
request, so a file placed in a folder the walk had already passed waited
for the next scheduled scan. A request made during a scan now sets a
pending flag and gets 202 with queued=true; the running scan runs exactly
one follow-up when it finishes, however many requests arrived. The gate
and the flag change together under a mutex so a request racing the end of
a scan is never lost. The cron path still skips rather than queues, but
runs a manual request queued during it.

Reported by foobarbigtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…in a flat layout (#1985)

scanLibrary marked the parent folder of every tracked audio file as tracked
so the sibling tracks of a multi-track audiobook count as already tracked
(#1436). In a flat Author/Title.mp3 layout that parent is the author
folder, so every other audiobook by the author was absorbed and never
reached the matching tiers or Unmatched.

An audio file with no book folder of its own (directly in a root or in an
author folder) no longer marks its parent. The folder instead remembers the
track stems of its tracked files (the name with part, track and chapter
numbers removed), and only a sibling with the same stem, or a bare track
number, is absorbed. Book folder layouts are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
The book page's live refresh (#1161) armed on book statuses nothing ever
wrote, so it never ran, and #2374 removed it. GET /book/{id} now carries
importInFlight, set while a download row for the book is grabbed,
downloading, completed, importPending or importing. The parked hand-off
states are left out so a page does not poll for hours. The page polls the
book every 5s while the flag is set and refreshes the history on the
reload that clears it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…#2055)

Fix match ran the full import, so correcting which book a file belongs to
also moved it into the target's folder and renamed it from the naming
template. #2224 added the warning and destination preview; this adds the
other half the issue asked for.

POST /queue/manual-import/reassign takes relocate. false moves the
book_files row to the target book through MoveBookFile, keeps the stored
path spelling, records bookFileMoved in history and answers 200; no import
runs and nothing on disk changes. true or absent is the old import, so API
callers are unchanged. The modal now asks what should happen to the file
and defaults to Correct the match only; the move option keeps the
destination preview and the no undo warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
vavallee and others added 7 commits October 8, 2026 00:36
…folder (#1985)

Review of the first pass: audioTrackStem stripped numbers anywhere, so
track names carrying chapter titles (01 - Chapter One, Title - 02 - The
Middle), spelled parts (Part Two), n of m and a glued CD1 got different
stems and became unmatched rows where main had absorbed them. The stem is
now the text before the first track number (digits, or a spelled number
after a track label), with track labels removed; a name that starts with
its number is a bare track and is absorbed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…owner (#2423)

HasImportInFlight counted every download for the book, so under tenancy a
user viewing a book nobody owns could learn that another user had a grab in
flight. It now takes the same QueryScope owner filter as the queue, with
ListScopeUserID from the request (admins and API key unscoped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…by (#3014)

The Settings page matched its scan's result by comparing the browser clock
with ran_at, which can pick up the scan it queued behind, and gave up after
two minutes even while the server was still scanning. Each scan now gets an
id (boot id plus sequence), the 202 returns the id the requested scan's
result will carry (scanId), and the stored result carries scan_id. The
status endpoint adds the live running and queued state. The page waits for
its own scan_id and only gives up after two minutes when the server reports
nothing walking or queued.

Also tests the scheduled path: a scheduled scan walking, a manual request
queued behind it, a second scheduled tick skipping, and exactly one
follow-up carrying the manual request's id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
The guarded write compared only media_type and status, so a concurrent
refresh that moved a path without changing either (a reorganize running
UpdateBookFilePath) could be overwritten by an older derivation, putting
the old path back. The guard now also compares the raw ebook_file_path and
audiobook_file_path columns, read directly rather than through the
bookColumns book_files view, with IS so a NULL read still matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…cked (#2055)

RelinkFile wrote a bookFileMoved row only when it took the file from
another book, so linking an untracked file changed the book with nothing
on its History. It now writes the row in that case too, without the from
fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
Signed-off-by: vavallee <vavallee@protonmail.com>
The message the scan poll shows when it stops waiting was a hard coded
English string with a dash; it is now settings.general.scanCheckBack,
read through a ref so the poll does not restart when t changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee marked this pull request as ready for review October 8, 2026 03:58
@vavallee
vavallee enabled auto-merge (squash) October 8, 2026 03:58
vavallee and others added 2 commits October 8, 2026 01:18
…3014)

TestLibraryScanStatusRouteAllowsAdmin compared the admin body byte for
byte with the stored blob, which no longer holds now that the status adds
the live running and queued flags. It now checks every stored field comes
back unchanged and that the only additions are those two booleans. The
non admin case is unchanged and still refused before the handler with no
paths in the body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
Signed-off-by: vavallee <vavallee@protonmail.com>

# Conflicts:
#	internal/db/books.go
@vavallee
vavallee merged commit 5e317d9 into main Oct 8, 2026
38 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment