Skip to content

feat(author): drop thin edition clusters from catalogue sync (#2235) - #2673

Merged
vavallee merged 3 commits into
vavallee:mainfrom
gchahcg:feat/2235-min-edition-count
Sep 27, 2026
Merged

vavallee merged 3 commits into
vavallee:mainfrom
gchahcg:feat/2235-min-edition-count

Conversation

@gchahcg

@gchahcg gchahcg commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Metadata profiles gain an opt-in minimum edition count filter for author catalogue syncs: works whose title cluster reports fewer known editions than the profile's floor are skipped before they become book rows. Closes #2235.

This is the smaller, safer shape of the request from the issue thread — a per-profile number, not a global catalogue-wide policy — and it leaves the closed PR #2628's cluster-signalling approach behind.

Implementation notes

Aspect Behaviour
Cluster key Normalised (lowercased, trimmed) work title
Cluster count Max of the known EditionCount across the works in the cluster — one well-editioned work saves the title's thinner siblings
Unknown count Passes the filter ("unknown is not zero", same rule as MinPages). Critical for Hardcover-primary authors, whose records never carry an edition count; a sum-based rule would have emptied their catalogues
Already-tracked books Maintained in place, never dropped, never counted as skipped
Single-work adds Exempt, matching the #1612 rule that explicit user picks are not vetoed by catalogue heuristics
Default 0 = filter off; the cluster map is only built when the profile sets a floor, so the default is zero-cost
  • Migration 094_metadata_profile_min_edition_count.sql adds min_edition_count INTEGER NOT NULL DEFAULT 0 to metadata_profiles.
  • The skip is reported in the sync summary (SkippedThinCluster + sample, capped at 5) and surfaced as its own line in the author page's sync notice.

Where it's mounted

  • fetchAuthorBooks (internal/api/authors.go): cluster map built in the candidate loop, filter applied in the create loop after existing-book resolution.
  • Profile CRUD: internal/db/metadata_profiles.go, internal/models/metadata_profile.go.
  • Web: profile editor form + profile-list chip (MetadataTab.tsx), sync notice breakdown (AuthorSyncNotice.tsx), en i18n keys (other locales fall back to en).

Follow-ups (not in this PR)

Checklist

  • Tests added or updated — 4 new fetchAuthorBooks tests (drop, opt-in default, owned-book exemption, single-work exemption) plus the twin-title fixture fix in EditAuthorModal.test.tsx
  • Doc-update gate cleared — docs/User-Guide-Wiki.md profile-filter list
  • Wiki pages updated if user-facing behaviour changed

Test plan

  • make test (full Go suite, CI gate)
  • gofmt -l ., go vet ./..., golangci-lint v2.11.4, govulncheck
  • make smoke (real binary, exercises the new migration)
  • cd web && npm run typecheck && npm run build && npm test (1002 tests) and npm run lint (0 errors; remaining warnings pre-existing)

@github-actions github-actions Bot added the bindery-notified Discord notification already sent for this PR label Sep 17, 2026
@gchahcg
gchahcg force-pushed the feat/2235-min-edition-count branch from b2072bc to f454ab4 Compare September 17, 2026 21:28
@codecov

codecov Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@vavallee

Copy link
Copy Markdown
Owner

Heads up on a migration number collision, not a review.

This PR adds a migration numbered 086, and so does #2673 or #2633, whichever this is not. Both are open and both claim it. assertUniqueMigrationVersions (internal/db/db.go:685) refuses to boot when two files share a numeric prefix, so whichever merges second breaks startup for anyone with both.

086 is a genuine gap on main, which goes 085 then 087, and it has never been filled by a merged commit. Three separate branches have each created their own 086 at some point, which is how the gap became attractive to all of them at once.

Main tops out at 089. Current claim map across the open PRs:

Suggest renaming to 093 or above rather than reusing the gap, and picking the number by checking open PRs rather than main. The reason this keeps happening is that each branch computes "next free" against a main that excludes everyone else's in flight work.

@vavallee vavallee added the needs-triage New issue, not yet reviewed label Sep 24, 2026
@gchahcg
gchahcg force-pushed the feat/2235-min-edition-count branch 2 times, most recently from 1854e09 to 198360b Compare September 25, 2026 01:30
@gchahcg

gchahcg commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

Renamed to 093_metadata_profile_min_edition_count.sql — checked all 33 open PRs, 093 is genuinely free (086 was also claimed by #2633, and 090/091/092 by #2761/#2750/#2713/#2714/#2737). No other file referenced the old number. Force-pushed.

@vavallee

Copy link
Copy Markdown
Owner

My fault, and a correction to my own advice on this PR.

I told four PRs that 093 was the first free number. Two of you took it, so #2673 and #2761 now both claim 093, which is the same collision I was flagging. Telling several branches one free number just moves the crash.

Assigning distinct numbers instead, so nobody has to guess:

Migration PR
086 #2633
090 #2713
091 #2714
092 #2737
093 #2761
094 #2673, this one
095 #2750

So please rename to 094_metadata_profile_min_edition_count.sql. I picked 093 for #2761 only because it renamed first; there is nothing better about its claim.

One other thing on this PR: the body says 086 while the file says 093. Those need to agree whichever number it ends on.

Note for anyone reading this later: 086 is a real gap on main, which runs 085 then 087, and it was never filled by a merged commit. The apply loop keys on the filename number rather than on "greater than last applied", so a back filled 086 does apply cleanly to existing installs. That is why #2633 can keep it.

…e#2235)

A metadata profile can now set a minimum edition count per title
cluster. During author catalogue sync, works whose normalised title has
fewer known editions than the floor are skipped before creation; the
cluster's best-known count decides, so a title with one well-editioned
work keeps its thinner siblings.

Unknown is not zero: works reporting no edition count at all pass the
filter, which keeps Hardcover-primary authors (whose records never
carry a count) from losing their whole catalogue. Already-tracked books
are maintained, not dropped, and explicit single-work adds are exempt,
matching the vavallee#1612 catalogue-heuristic exemption.

The skip is opt-in (profile default 0 = off) and shows up in the sync
summary and the author page notice as its own line.

Signed-off-by: gch ahcg <gchahcg@proton.me>
@gchahcg
gchahcg force-pushed the feat/2235-min-edition-count branch from 198360b to 53931a4 Compare September 25, 2026 19:24
@gchahcg

gchahcg commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

Renamed to 094_metadata_profile_min_edition_count.sql and fixed the PR body to match (it still said 086). Force-pushed.

@vavallee vavallee left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the shape we agreed on #2235, and the unknown passes rule is the part that matters most: Hardcover primary authors never carry an edition count, so a sum or zero based rule would have emptied their catalogues. I checked it's pinned by turning a missing count into a zero, and `TestFetchAuthorBooks_SkipsThinCluster` fails. Merged onto current main it builds, vets, and `db`, `models` and the catalogue sync tests pass. 094 is unique across main and every open PR.

Two small things before merge:

  • Changelog fragment. Every PR needs one under `changelog.d/` (the release notes are assembled from them). Something like `changelog.d/2235-min-edition-count.md` with an `### Added` bullet. House style is no dashes in the text.
  • One line in the docs about series fill. Every metadata profile filter, this one included, is skipped by the series fill path today. That's #2208 and not yours to fix, but someone who sets a floor and then fills a series will see thin works arrive and think the filter is broken. A sentence in the User Guide section you added saves that support question.

Tiny nit, take or leave: the cluster key is normalised inline in two places (building the map and reading it). They match today; a small helper would keep them from drifting.

…s fill gap (vavallee#2235)

Adds the changelog fragment for the opt in minimum edition count, and a
sentence in the user guide that series fill skips every metadata profile
filter today, this one included (vavallee#2208).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee

Copy link
Copy Markdown
Owner

Pushed two small follow ups so this can make the next release: a changelog fragment (changelog.d/2235-min-edition-count.md), and a sentence in the user guide that series fill skips every metadata profile filter today, this one included (#2208). No code changes. Thanks gchahcg!

Resolves the one conflict in internal/api/authors.go: this branch adds
resolveMinEditionCount and main (vavallee#2754) adds the author work language
evidence fetch at the same point in runCatalogueSync. Independent, so
both are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee enabled auto-merge (squash) September 27, 2026 01:57
@vavallee
vavallee merged commit d81b068 into vavallee:main Sep 27, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bindery-notified Discord notification already sent for this PR needs-triage New issue, not yet reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Additive scored signals instead of first-drop-wins boolean filtering

2 participants