feat: adopt unmatched library books from the Import page (#2547) - #2671
Conversation
…eusable cores Library adoption and request approval both need to add a book or an author without an HTTP request. Move the bodies of AddBook and Create into addBookCore (add_book_core.go) and createAuthorCore (create_author_core.go). The handlers now decode the body, call the core, and map its errors back to the exact statuses and bodies they returned before. Ownership still comes from the context. This is a pure move: no behaviour change, and the existing tests pass unmodified. addBookResult reports whether the book and the author were created by the call, which the later callers need for compensation. authors.go shrinks by about 480 lines (plan item C1). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…dd cores Library adoption creates books unmonitored and must not pull in a catalogue behind them; request approval needs the same control over the author sync. Both overrides default to what the handlers already do, and the handlers never set them. addBookParams.Monitored nil keeps the forced monitored flag. addBookParams.SkipCatalogueSync skips the single work fallback sync and looks for the row once instead of polling, since nothing the core started could still create it. createAuthorParams.SkipCatalogueSync skips the catalogue sync on both the create and the relink path. New tests call the cores directly: default params match the handler, Monitored=false stores an unmonitored book, SkipCatalogueSync makes no author works call (with a control case that does), and the handler's error table matches the literal bodies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Migration 088 adds unmatched_units, one row per book a library scan could not match, keyed by path so an ignore or an adoption survives the next scan. UnmatchedUnitRepo.ReconcileScan upserts in chunks of 500 and never moves an ignored row; it removes unseen pending rows and purges decisions unseen for 30 days, and does neither when the scan found no files. Adopt, undo and ignore claim a row with a single conditional UPDATE so two requests cannot both act on it. The list maps sort keys through a switch, binds every filter, and hydrates the books a page points at in one query. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…ts (#2547) The library scan used to put its first 1000 unmatched files into the lastScan blob, one entry per file, so a 193 track audiobook filled 193 rows and a large library lost everything past the cap. The scan now groups unmatched files into book units (audio by folder, disc folders by the book folder above, same stem ebooks together), ranks up to three suggestions per unit from the catalogue already in memory with the reconcile's own title measure, and stores them through WithUnmatchedUnits. Bounds are 50,000 files and 20,000 units with a truncated flag. Only regular files whose folder resolves inside a scanned root become rows (S10); size and mode come from the walk. The blob keeps its counters and the #1436 invariant, gains unmatched_units, ignored_units and units_truncated, and writes unmatched_files as an empty list for one release. The disc folder regex, AllDiscFolders and SameStem move from internal/api/scan_walk.go into importer so both scans share one definition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…thout sync Review found that changing the 422 and 404 add book statuses, or the 502 create author status, left the whole internal/api suite green. The error mapping tests now cover every sentinel and typed error of both cores with the exact status and the exact JSON body, including the 409 bodies that carry the existing book or canonical author. Documents what SkipCatalogueSync skips on each core. On the add book core it removes only the single work fallback; SearchOnAdd still runs. On the create author core it skips the whole sync, and the search on add only ever runs inside that sync, so SearchOnAdd with SkipCatalogueSync would accept a search and never perform it. createAuthorCore now refuses that pair with errCreateAuthorSearchNeedsSync. No HTTP request can set SkipCatalogueSync, so the handler's behaviour is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Adds the admin only /library/unmatched routes. The list filters, sorts through a fixed switch, returns facets only when asked, and hydrates every book a page points at in one query. Adopt registers a unit's files in place against a book already in the library, or against a metadata result added through addBookCore unmonitored, in the adopted format, with no search and SkipCatalogueSync set. No route accepts a path. A request claims its row with a compare and swap, re-checks every file with Lstat and LibraryRoots.ResolveContained, refuses with 409 when a file already belongs to a book, and compensates on failure. Undo untracks exactly what the adoption registered and removes a book or author only when the row recorded creating it and nothing else holds it. The scanner is wired to store unmatched units, and ScanRunning exposes the in flight flag for the page's scan status. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
A multi disc audiobook's tracks parse their title from the disc folder, so the unit read "CD 1". The unit is the book folder above the discs, and it is named after that folder now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
/import becomes one page with two views: "In your library", the new default, and "From a folder" (?view=folder), the existing folder import. The library view has a summary strip with Scan now, a rail of the author folders holding the most undecided books with the one action each calls for, and a list whose rows say in one sentence why the book is here and end in the thing to do. A suggestion is one click to confirm; any row opens an editor in place with the suggestions and their scores, a library search, and a collapsed metadata search that asks a provider only on submit. Decisions show at once with Undo and revert inline on error. Rows are keyboard operable (arrows, Enter, Esc, i, u, /) and stack as cards on narrow screens. Admins see the pending count on the Import nav entry. ManualImportPage is split into ImportPage, FolderImportView and FolderImportRow; BookPicker and CatalogueAdder move to components/import, and SortHeader is extracted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…rt page Settings > General no longer lists up to 1000 unmatched files with reason hints nobody could act on; it shows how many books need a decision and a link to review them. The bulk folder scan in Settings > Import becomes a link card to /import?view=folder, and its test goes with it. The empty Queue and Wanted hints point at the two Import views. The per file reason and hint strings are removed from en.json and the one stale translation of them from ko.json. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
A new user guide section, "Adopting files already in your library", the /library/unmatched routes and their answers in API.md, a troubleshooting entry for files a scan leaves unmatched, a README feature line, and the changelog fragment. References to Manual Import, the bulk folder import and the Settings unmatched table point at the Import page views now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
There was a problem hiding this comment.
ESLint found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
… actions Books whose author is not in the library and share a folder are one group row with a single Add author and a disclosure for the books, and the rail is plain folder navigation. Only a strong suggestion (title similarity of at least 0.92 with a matching author, one constant in adoptionMatch.ts) gets a one click Confirm; anything weaker is a quiet "Possible match" link that opens the editor preselected. Every row has the same fixed action cell, one primary button and a More menu, and a one line fact instead of a wrapping sentence, with the full sentence in the tooltip and the editor. The summary is one number with quiet stats, sort arrows are SVG, counts use tabular numerals, and the library search no longer repeats the metadata search prompt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…es not own From the backend review of library adoption: - Undo removes a book_files row only while it still belongs to the book the adoption registered it to (each registered path now records its book), so a path since deleted with its book and registered to another keeps its new owner. - A created author counts as unused only with no books at all, excluded ones included, since deleting it cascades to them. - A created book is fingerprinted when the adoption finishes (monitored flag, updated_at, history, downloads, pending releases, series links); if that changed, Undo untracks the files but keeps the book and author and says so in the response message. - An adopt writes its created ids and each registered file into the row before the next side effect. Abandoned adopting or undoing claims, dated by a new claimed_at column, are reversed the same way Undo reverses and then returned to pending: at startup for any claim, and on list for claims older than 15 minutes. A scan no longer releases claims itself. - Adopted rows are kept while their book exists and purged once it is gone; ignored rows are purged after 30 days only under a root that produced files in the scan. - A truncated scan removes no pending row (SkipDeletion, now wired). - A lost undo completion answers 409; a format that contradicts the files answers 400. - Disc sets follow the folder import walker's rule (every subfolder is a disc folder with audio, via AllDiscFolders) with CD, Disc or bare number names only, and never at a root or author folder, so Mistborn/Book 1..3 and Author/1, Author/2 stay separate. A disc set is registered disc folder by disc folder, and the scanner's disc tracked rule is reverted. Migration 088 is unreleased and gains claimed_at and created_book_fingerprint in place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Adopt no longer accepts a format that contradicts the files, so the editor's Ebook or Audiobook toggle could only ever produce an error. The user guide and API docs describe what Undo keeps, disc set grouping, recovery after a crash, and how long adopted and ignored rows last. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
When Undo answers with a message because the book the adoption added has been used since, the row that returned to pending shows a quiet line: Files removed. The book stayed because it is now in use. It clears with the next fetch. Also replaces two NUL bytes that had been written into a template string in useAdoptionList.ts, which made git treat the file as binary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
…tes to the claim From the second backend review of library adoption: - Disc sets: subfolders with no audio beneath them, and hidden or system folders (a leading dot, at sign or hash), no longer stop a folder from grouping; a book folder directly under a library root can group its CD or Disc folders (the root itself still cannot); bare numbers are no longer disc names, so Series/1 and Series/2 stay separate books. - reverse returns its first error instead of logging it. Undo then answers 503 and keeps the claim and the record; a failed adopt whose reversal fails answers 503 the same way; recovery leaves the row for its next pass. The record is only cleared once every step succeeded. - Claims carry a token (claim time plus a random nonce) stored in claimed_at. Adopt progress, completion, reset and undo completion all match it, so a request whose claim was recovered and retaken cannot write into the new claim. - The created book fingerprint also covers blocklist entries, the Calibre id, editions and provider identifiers. Its known false positives are documented as the safe direction. - Old ignored rows under a root that is no longer configured are purged; a configured root that produced no files still keeps its rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
Adopt and bulk ignore decoded their JSON leniently, so a body carrying a field the route does not take, such as "path": "/etc/passwd", was silently ignored. No adoption route takes a path; both now answer 400 naming the unknown field, and nothing is changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9 Signed-off-by: vavallee <vavallee@protonmail.com>
There was a problem hiding this comment.
Reviewed the full diff. Here is what I checked and what I found.
Auth / tenancy — registerAdoptionRoutes wraps every route in r.Use(auth.RequireAdmin) (sensitive_routes.go). TestAdoptionRoutesRequireAdmin tests role user and no-role for all seven routes; TestAdoptionRoutesEnumerated catches any route added without a row in the table. Clean.
SQL injection — sort/dir go through the unmatchedOrderBy switch (fixed strings only); state, reason, authorFolder, format are all bound as ? parameters; search goes through FoldForSearch → escapeLike → LIKE bound parameter. TestUnmatchedList_FiltersSortsAndEscapes sends "score); DROP TABLE unmatched_units; --" as the sort and "Other' OR '1'='1" as the folder and confirms neither matches extra rows. The nolint:gosec G202 lines are correctly annotated — where contains only fixed predicates and bound values, and ORDER BY comes from the switch. Clean.
Path traversal / symlinks (S10) — registrationPaths calls os.Lstat on each member (catches a symlink swapped in after the scan) and then ResolveContained on every file and derived folder. TestAdopt_RejectsSymlinkEscape covers both cases: a file replaced by a symlink, and a path reached through a symlinked directory that leaves the library. Clean.
No path in request body — decodeAdoptionBody uses json.Decoder.DisallowUnknownFields. The comment at adoption.go:963 names the intent. adoption_undo_safety_test.go line 2475 sends {"bookId": ..., "path": "/etc/passwd"} and expects a 400 that names the field. Clean.
TOCTOU / concurrent state changes — Claim returns a random-nonce token stored in claimed_at; every subsequent write (progress, complete, release, undo complete) requires claimed_at IS token. TestAdopt_DoubleAdoptAndDoubleUndo races six concurrent requests and expects exactly one 200 and five 409s. TestClaimToken_ScopesEveryWrite (listed in the PR body) shows that a late write from a dead request cannot land on a new claimant's row. Clean.
Crash / partial-adopt recovery — Every side effect is persisted to the row before the next step. RecoverStaleClaims(ctx, 0) at startup reverses any claim from a previous process. TestRecoverStaleClaims_ReversesADeadAdopt simulates a panic after the first of two files and confirms recovery untracks the file, deletes the created book and author, and resets the row to pending. TestAdopt_KeepsTheClaimWhenItsReversalFails confirms a failed reversal leaves the claim intact for the next recovery pass. Clean.
Undo safety — UntrackFilePathForBook deletes only the row matching both path and book_id (no gap between check and delete). bookIsOnlyOurs runs after untracking, so the file count it reads is post-untrack. ListByAuthorIncludingExcluded prevents cascading into excluded books. BookFingerprint is read before untracking (untracking itself updates updated_at). All five reviewer items from the first pass and all five from the second have named tests.
No new deps — Checklist confirms no new Go module or npm package. The diff contains no go.mod / package.json changes.
Backward compat — unmatched_files is kept as an empty list in the scan blob for one release so an older cached web bundle still parses.
rootPath in API response — The absolute server path is intentional, admin-only, and consistent with the existing /library/scan/status gate that the PR description explicitly references. Not a concern.
One note on the IgnoreMany guard — (len(req.IDs) == 0) == (req.AuthorFolder == "") is correct but reads as a double-negation; something like noIDs && noFolder || hasIDs && hasFolder would be easier to reason about at a glance. Not a blocker.
Overall: the change is correct, the security surface is covered by tests, and no blocking defect was found.
— 🤖 Bindery triage bot (automated). Reply to correct me; a human will see it.
…kip-tracked Resolves the manual-import wizard's move from web/src/pages/ManualImportPage.tsx to web/src/pages/import/FolderImportView.tsx (upstream vavallee#2671's Import-page refactor split it into FolderImportView/FolderImportRow/folderImport.ts and added the "In your library" adoption view alongside it): the skip-tracked scan and the "Show already imported" toggle now live in the new files, and the equivalent Settings > Import bulk-scan UI is gone there too, replaced by upstream's link out to the Import page. internal/api/manual_import.go's Scan handler keeps our resolveImportFolder extraction, now using upstream's outsideRootsMessage for the 403 body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L3vXCo9PwmffPWUik6Gwvh Signed-off-by: Trevor Swanson <83826109+trevorswanson@users.noreply.github.com>
Summary
Library adoption: the books a library scan could not match stop being a read only, 1000 file table buried in Settings and become decisions on the Import page.
/importopens on In your library, one row per book, with suggested matches (a one click Confirm only for a strong match), an in place editor, one group row per missing author with a single Add author, ignore, and Undo. Adoption is the scanner's own reconcile with a person supplying the match: files are registered where they are, nothing moves, nothing is queued or searched.Stacks on #2662 and must merge after it. It reuses
addBookCorefrom that PR; the base of this draft isrefactor/add-book-core.Closes #2547 (193 tracks are one row).
Design
Layout of
/import(default view), desktop:Principles:
STRONG_MATCH_SCORE = 0.92inadoptionMatch.ts, and the author the scan read must match the book's author. A strong suggestion shows a green "Strong match" chip and a primary Confirm. Anything weaker shows a neutral "Possible match" chip and the title as a quiet link that opens the editor with it preselected; the row's primary action is Choose book.Row states:
Designed empty and status states: never scanned (explains a scan, Scan library), scan running (pulsing dot, polling every 3 s only while running), everything matched, truncated, scan error, a scan that found no files ("your decisions were kept"), nothing ignored, nothing adopted, filters match nothing (Clear filters). Mobile: rows and groups stack as cards with the same primary action and More menu, and the rail scrolls as chips. Keyboard: roving tabindex over rows and group rows, arrows / Home / End, Enter opens a book's editor or a group's books, Esc closes and returns focus to the row,
iignores,uundoes,/focuses search. The table has a caption naming the list and the keys.Screenshots were taken from a local instance seeded with a demo library in headless Chromium with Inter installed (the wide word spacing is the headless renderer): list light and dark, weak suggestion opened in the editor, a strong suggestion confirmed, a group expanded, a row's More menu, 390 px mobile, and the folder view. They are local files, listed in the builder's report, not attached here.
Implementation notes
088_unmatched_units.sqlunmatched_units,unit_path UNIQUE, statespending/adopting/adopted/undoing/ignored,created_book_id/created_author_id/registered_paths_jsonfor exact undo,member_paths_json, candidates, indexes on(state, top_score DESC, rel_path),(state, author_folder, rel_path),(state, reason),(book_id); book and author FKsON DELETE SET NULLinternal/db/unmatched_units*.goReconcileScan(chunks of 500, never moves an ignored row, adopted row back to pending only if adopted before the scan started, deletes unseen pending, purges decisions unseen 30 days, deletes nothing on zero files or error, releases claims stale for 15 min),ClaimStatecompare and swap, list with switch mapped sort, facets, one query book hydrationinternal/importer/unmatched_units.gogroupUnmatched(audio by folder, disc folders by the book folder, root level audio alone, ebooks by folder + stem), S10 eligibility, top 3 candidates frombooksByAuthorwith the reconcile's Jaro-Winkler, threshold 0.60,WithUnmatchedUnits,ScanRunning,audioTrackedByFolderinternal/importer/unit_boundaries.godiscFolderRe(+IsDiscFolderName),AllDiscFolders,SameStemmoved fromscan_walk.go;dirSubtreeHasAudiomoves withAllDiscFoldersbecause it is its only callerinternal/importer/scanner.gounmatched_units,ignored_units,units_truncated, keepsunmatched_files: []with a dated comment; the disc tracked check callsaudioTrackedByFolderinternal/api/adoption.go,adoption_adopt.goregisterAdoptionRoutesunderRequireAdminpages/import/ImportPage,FolderImportView+FolderImportRow(split fromManualImportPage),AdoptionView,AdoptionSummary,AdoptionRail,AdoptionFacets,AdoptionTable,AdoptionRow,AdoptionEditor,useAdoptionList+adoptionReducer(named actions),adoptionHint;components/import/BookPicker,CatalogueAdder;components/SortHeader,useUnmatchedCount;api/adoption.ts. Largest new component 170 linesRegistration.
AddBookFileIfMissing, which isAddBookFile(the scanner's write,scanner.goreconcile tiers) plus whether the row was inserted, so undo removes only what adoption added. An audiobook folder unit registers its folder, the shape an imported audiobook download already has (SetFormatFilePath(destDir)); a disc set registers the book folder, and the scan's tracked check now treats a disc folder's tracks as tracked when the folder above is. Ebook units register each member. The walked path is registered, not the resolved one, so the next scan recognises it; containment is still proven byResolveContained. Undo usesBookRepo.UntrackFilePath, skipping any path that has since moved to another book.SkipCatalogueSync: true. Adding a book never pulls the author's bibliography anyway (#1816), so the flag only affects the single work fallback. With it set, a provider failure answers at once instead of polling for 15 seconds, there is no second provider call, and no background sync can create the book after the adopt request has given up and compensated, which would strand a row undo does not know about. A person adopting a file by a new author gets that one book and that author, which is what they asked for.
BookCreated race. When the add created the book, every file on it at the end of the adopt is recorded as registered (the add's own library lookup can attach one), and undo deletes the created book only if it has no files left and no other row references it; same for a created author with no books.
Overlap with #2480 (contributor PR, changes requested)
internal/api/scan_walk.go@@ -51,7 +51,15(walk limits),@@ -86,6 +94,9(insideenumerateImportUnits)regexp/stringsimports anddiscFolderReabove the limits, rewrites the two helper calls toimporter.AllDiscFolders/importer.SameStemin the unit heuristic, deletes the three helpers at the end of the fileweb/src/pages/ManualImportPage.tsxpages/import/FolderImportView.tsx+FolderImportRow.tsxverbatim apart from theCatalogueAdderprops, so #2480's hunks port by pathweb/src/pages/ManualImportPage.test.tsxpages/import/FolderImportView.test.tsx, import paths onlyweb/src/pages/settings/ImportTab.tsx,FolderScanSection.test.tsxFolderScanSectionchangesFolderScanSectionretired to a link card and its test deleted, so those hunks dropinternal/db/books.go@@ -749,6 +749,21useFolderScandoes not exist on this base (it arrives with #2480), so C5's "delete it if unused" has nothing to delete yet; if #2480 lands first, its only caller isFolderScanSection, which this PR removes.Second backend review fixes (
3cb3bdde, web note63750f6e).,@,#) are ignored; a book folder directly under a root can group CD/Disc folders, the root itself never; bare numbers are no longer disc names;Part Nstays apartTestGroupUnmatched_DiscSetsFollowTheWalkerRule: every layout from the review table plus@eaDir,Book N, a non disc audio subfolder andRoot/CD1,CD2reversereturns its first error; Undo answers 503 and keeps the claim and record; a failed adopt whose reversal fails answers 503 and keeps its claim; recovery skips and retries next pass; the record is cleared only after every step succeededTestUndo_KeepsTheRecordWhenReversalFails,TestAdopt_KeepsTheClaimWhenItsReversalFails(both then finished by recovery)Claimreturns a token (claim time plus random nonce) stored inclaimed_at; progress, completion, reset, undo completion and release all requireclaimed_at IS token; recovery uses the stale row's tokenTestClaimToken_ScopesEveryWrite(A claims, recovery releases, B claims, every late A write refused)calibre_id, editions and provider identifiers; the known false positives (status refresh, cover writes, author refresh movingupdated_at) are documented onBookFingerprintas the safe direction and left aloneTestUndo_KeepsACreatedBookSomeoneStartedUsinggains blocklist and calibre id casesConfiguredRootsoption; old ignored rows purge under a root that produced files or a root no longer configured; a configured root that produced nothing keeps themTestReconcileScan_PurgesIgnoresUnderARemovedRootFail before, on
63750f6e:Checked by reverting each fix in place (these tests need the new seams):
Backend review fixes (
e8d7e1a1,b3e8287e)BookRepo.UntrackFilePathForBookdeletes onlypath AND book_id; a gone book means nothing matches, the unit just returns to pendingTestUndo_LeavesAFileThatNowBelongsToAnotherBookListByAuthorIncludingExcluded; the only other "unused" check (bookIsOnlyOurs) readsbook_filesand unmatched rows, no excluded filterTestUndo_KeepsACreatedAuthorWithAnExcludedBookAllDiscFoldersover every subfolder) plus CD / Disc / bare number names only, never a root or author folder; disc sets register each disc folder; scanner disc tracked rule revertedTestGroupUnmatched_DiscSetsFollowTheWalkerRule,TestScanLibrary_TrackedFolderDoesNotHideALaterSibling,TestAdopt_RegistersEachDiscFoldermessagesays soTestUndo_KeepsACreatedBookSomeoneStartedUsing(monitored, series link)RecoverStaleClaimsreverses like Undo, then resets: at startup for any claim, on list for claims over 15 min; the scan no longer releases claimsTestRecoverStaleClaims_ReversesADeadAdoptbook_idis NULLTestReconcileScan_KeepsAdoptionsWhileTheirBookExistsSkipDeletionnow set for a truncated scan andSkipPurgeremovedTestScanLibrary_ZeroFileScanKeepsUnitsclaimed_atcolumn in 088TestStaleClaims_DatedByClaimedAtSkipDeletion: truncatedTestScanLibrary_TruncatedScanRemovesNothingCompleteUndofalse answers 409TestUndo_ReportsALostCompletionRootsWithFilesTestReconcileScan_PurgesOldDecisionsTestAdopt_RejectsAFormatThatContradictsTheFilesDecided differently from the review, with reasons:
AllDiscFoldersacceptsBook N,Part N,Vol NandChapter N, so a series folder holdingBook 1,Book 2,Book 3with audio is one unit to the folder import walker too. Adoption uses the walker's structure check through the moved helper unchanged, and adds two narrowing conditions of its own: the subfolder names must be CD, Disc or a bare number, and the folder must not be a root or directly under one.IsDiscFolderNameandAllDiscFoldersare untouched, so the folder import behaviour and the helper move are unchanged.BookRepo.UntrackFilePathForBook, rather thanUntrackFilePathplus a separate ownership read, so there is no gap between the check and the delete.Fail before, on
270739a6(the tree before these fixes) with the new tests copied in:Tests that need the new seams (the recovery entry point, the unit cap variable, the new options) were checked by reverting each fix in place:
Security
sortanddirgo throughunmatchedOrderBy's switch;searchis folded then LIKE escaped and bound;authorFolder,reason,formatare bound.TestUnmatchedList_FiltersSortsAndEscapesfeeds SQL insortandauthorFolderTestClaimState_ExactlyOneWinnerandTestAdopt_DoubleAdoptAndDoubleUndo(6 concurrent requests, one 200, five 409, under-race)created_book_id/created_author_idthe row recorded, only with no files / books left and no other row referencing themGET /library/unmatched/summary, admin only; the nav asks only whenisAdmin(App test asserts no call for a non admin)mode.IsRegular()from the walk's Lstat info, and a folder whose resolved path is under a resolved scanned root;TestEligibleUnmatched_SymlinkOutsideRoot,TestScanLibrary_SymlinkedBookIsNotListedLibraryRoots.ResolveContained(library roots only, not download dirs) on every member and the folder;TestAdopt_RejectsSymlinkEscape(file swapped for a symlink, and a symlinked folder leaving the library) answer 422 and register nothingregisterAdoptionRoutesinRequireAdmin;TestAdoptionRoutesEnumeratedwalks the router so an unlisted route fails,TestAdoptionRoutesRequireAdmin(role user and no role: 403, handler not run),TestAdoptionRoutesAllowAdminaddBookCore's context; choosing an existing book leaves owner and monitored as they were (asserted inTestAdopt_ExistingBookRegistersInPlaceAndUndoIsExact)Performance
os.FileInfo; grouping does no stat. S10 resolves each folder once, not each fileTestUnmatchedList_QueryPlansUseIndexes: every sort searches a state index, default and folder sorts need no temp b-tree. Facets only withfacets=1, which the hook sends when state or search changesscan.running; nav badge reads once, then from an event after adopt, ignore or scanTestAdoptionList_NoProviderCallsAndOneHydrationQuery: 5 rows with 2 candidates each, with and without facets, oneBookRefscall and zero provider calls.TestScanLibrary_MakesNoOutboundCalls:http.DefaultTransportspy sees nothing during a scan. Web:asks a metadata provider only when the search is submittedspiesfetchthrough render, expand, open and typing, then sees exactly one/search/bookon submitBenchmarkGroupUnmatched(50,000 files)BenchmarkReconcileScan(20,000 units)Fail before evidence
Blob level tests copied onto the base (
58292dea) and run there:Web tests copied onto the base:
The adopt, undo, ignore, reconcile, grouping and view tests exercise code that does not exist on the base, so they do not compile there. The provider on submit test was checked by mutation: making
CatalogueAddersearch on every keystroke fails it.Deviations and follow ups
userEvent;@testing-library/user-eventis not installed and no new npm package was added, so the keyboard path usesfireEvent.keyDownon real focus.adopting/undoingstates, so every list stays a single state equality in index order; a crashed request's claim is released by the next scan.SortHeaderis the third copy of the sortable header; movingBooksPageandAuthorDetailPageonto it is left for a follow up to keep this diff reviewable (C6)./importwith a plain anchor because the Settings page tree has no router in its tests.Suggested review order
9825f2e6db and migration85868532,8f330151scanner grouping and candidatesdceb018bAPI and routes, thene8d7e1a1and3cb3bddereview fixesae8293afweb, then21126ad6design pass, then92cb5d12removals4a531701docsChecklist
docs/User-Guide-Wiki.mdnew section,docs/API.md,docs/Troubleshooting-Wiki.md, README Features,changelog.d/library-adoption.md)Test plan
go build ./...,go vet ./...,GOOS=windows go build ./...,GOOS=darwin go build ./...golangci-lint run ./...(0 issues)go test ./cmd/... ./internal/...(exit 0, after the review fixes)-race:./cmd/binderywhole package ok;./internal/api -run 'TestAdopt|TestUndo|TestIgnore_|TestAdoptionList|TestRecoverStaleClaims|TestEnumerateImportUnits|TestAddBookCore'ok;./internal/db -run 'Unmatched|ReconcileScan|ClaimState|ClaimToken|BookRefs|Migrate088|StaleClaims|KeepsAdoptions|RemovedRoot'ok;./internal/importer -run 'GroupUnmatched|EligibleUnmatched|SymlinkedBook|RanksCandidates|NoOutboundCalls|AudioTrackedByFolder|TrackFolderIsOne|NoThousandCap|UnmatchedReason|ClaimRank'ok. No race reported. The wholeinternal/dbandinternal/importerpackages exceed the 10 minute test timeout under-raceon this machine (asinternal/apialready does, make test: internal/api race package hits the 30-minute timeout #2293), so they were run by filter.cd web && npx tsc --noEmit,npx vitest run(1025 passed),npm run build,npx eslint .(0 errors; warnings are pre-existing files)🤖 Generated with Claude Code
https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9