Repository navigation
Conversation
Callers that only want a thumbnail (e.g. codex's CoverThread) don't need the full ComicInfo/CoverImage hint resolution. Parsing the metadata for every cover dominates the cost of cover extraction and emits a flood of debug-bucket Union ValidationErrors that look like real failures in DEBUG logs. When skip_metadata=True, bypass generate_cover_paths entirely and read archive index 0 directly. This drops per-call schema instantiation, Union resolution, and path normalization. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps
…122) ClearingErrorStoreSchema previously split each schema's errors into two buckets: ignored ones logged at DEBUG, real ones at WARNING. The DEBUG bucket only ever held errors from ``_ignore_errors`` — ``Field may not be null.`` (sparse-field tolerance) and ``Invalid input type.`` (Union variant misses) — both of which are internal mechanics, not operator-actionable signal. Each Union miss emitted one ``ValidationError - {'_schema': ['Invalid input type.']}`` line per field per archive, drowning the genuinely useful per-source DEBUG messages emitted by ``_except_on_load``. Filter ignored errors at split time, log only WARNINGs. Real schema failures still surface with full context (path, schema class, normalized message). Collapses the dual-bucket _split_*_errors methods into _filter_* + _log_warnings. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps * metron: drop broken URL slugs for genre, location, reprint, role, story, tag Metron has no public web pages for these types — only API endpoints — so URLs like https://metron.cloud/genre/3 always 404. Stop emitting them. The numeric Metron ID is still preserved on the identifier. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shortens the import path for the helper from comicbox.enums.maps.age_rating to comicbox.enums.maps so downstream callers can reach it without drilling into the submodule. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Remove unused module/class constants: _COMMENT_ARCHIVE_TYPES, SUFFIXES, _LOG_FORMAT, comet.py IDENTIFIER_TAG/IS_VERSION_OF_TAG, comictagger.py IDENTIFIER_TAG/PAGES_TAG, XmlCountryField (and now-orphaned imports RarFile, ZipFile, CountryField). - Fix latent bug in TrapExceptionsMeta: `attr_name in "deserialize"` was a substring check that wrapped any callable whose name was a substring of "deserialize" (e.g. "er", "size", "ali"). Use the existing _WRAP_METHODS tuple instead so only the exact `deserialize` method is wrapped. - Simplify _get_pdf_enabled() to a plain `import pdffile` probe; the except-arm stub import had no effect. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidate the optional comicbox-pdffile integration into one module (comicbox/_pdf.py) and delete the hand-maintained pdffile_stub.py. Previously six call sites each duplicated a `try: from pdffile import X / except: from pdffile_stub import X` block, and the stub class mirrored the real PDFFile API method-for-method — silent drift risk every time upstream pdffile shipped. Now: - comicbox/_pdf.py is the single source of truth for PDF_ENABLED, PDFFile, and PAGE_FORMAT_VALUES. When pdffile is absent, PDFFile is None at runtime; type checkers see the real class via TYPE_CHECKING. - Every call site that touches PDFFile is gated by `if PDF_ENABLED`. - The `case PDFFile():` arm in box/archive/archive.py is lifted to an `if PDF_ENABLED and isinstance(archive, PDFFile):` guard above the match (the match form would fail when PDFFile is None). - config/__init__.py reads PAGE_FORMAT_VALUES instead of iterating an empty stub Enum. Verified with `pdffile` installed (307/307 tests pass) and in a fresh venv without it (PDF_ENABLED=False, CBZ archives still work, PDF files raise UnsupportedArchiveTypeError, CLI shows the "not installed" hint). Net: -70 lines across 9 files. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps * update news and version to alpha 4 * update deps * rename function path in NEWS * bump alpha version to 3.0.0a5
* require comicbox-pdffile 0.6.x for image-dominant page detection Widens the optional ``[pdf]`` extra to require comicbox-pdffile 0.6.x. The new minor release adds image-dominant page detection ( ``PDFFile.classify_page``, ``PDFFile.read_image_if_dominant``, ``PDFFile.read_full_pixmap_jpeg``) used by browser readers to serve scanned-comic PDF pages as plain ``<img>`` instead of routing through pdf.js on the client. comicbox itself doesn't use the new API — the bump is purely a pin update so downstream callers (Codex, OPDS readers) can adopt it. The ``[tool.uv.sources]`` block is transient: it points at the pdffile PR branch so this CI can resolve dependencies before 0.6.x lands on PyPI. Drop it once 0.6.x publishes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * just use the released pdffile --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add bin/regenerate-pdf-test-pages.py — drives Comicbox.get_page_by_index
against tests/files/test_pdf.pdf to refresh tests/files/pdf/{N}.pdf when
pymupdf or pdffile change page-extraction output. Run on the next drift.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Telemetry Brian can act on, plus the floating-burst-limit behaviour he described, pinned by a test. W2 — responses without rate-limit headers. Not "429s without headers", which is structurally zero from Metron's checked-in stack: DRF's throttles run in `initial()`, after auth but before any view code, before the conditional-GET 304 and before `X-Cache`, and Metron's middleware copies `X-RateLimit-*` onto every response whose request reached them. So a genuine `/api/` answer of ANY status carries the headers, and their absence proves the answer came from nginx or Anubis instead. Anubis ALLOWs `^/api/.*$` ahead of its challenge rules and serves both its challenge and deny pages as HTTP 200 HTML by default, so bucketing by status catches a bot-check page as readily as a 429 — with no body inspection. - `_ApiCounts.unthrottled: dict[int, int]`, fed by `record_unthrottled_response`, reported as e.g. "3 responses without rate-limit headers (200: 2, 429: 1)". - One `warn_once` line carrying status, Content-Type and Content-Length only. No body (mokkari's `ApiError` already embeds `response.text`), no `Server` header (nginx overwrites it). - `connection_failures`, counted on `_execute_http_request`'s exception path. Only transport failures reach that frame: mokkari re-raises `requests` ConnectionError and ReadTimeout as `ApiError` there, while HTTP status errors surface later out of `_handle_http_response`. This is the shape a fail2ban ban takes, where the client only sees timeouts. - A header-less 429 no longer calls `gate.observe` or `record_rate_limit_windows`. Behaviour is unchanged — `observe` with no headers only flips UNKNOWN to OPEN, logging a false "pacing disabled" that `cooldown` contradicts a moment later, and the windows call is a no-op with all-None arguments. The cooldown itself is untouched: the full-window rebuild stays, or the paced retry path would fire its whole budget back-to-back. W3 — the floating burst limit. 20/minute is the documented floor; production commonly reports 45-60 and is dialled back toward 20 on peak days. The gate already adopts whatever `X-RateLimit-Burst-Limit` says, in both directions, and the estimator prefers the live gate's number over the constant. What was missing was a test and honest wording, so `test_rate_gate` now pins a limit that drops mid-run shrinking the window and rising re-widening it, and `rate_limits` says the constant is a starting pace and an estimator fallback rather than a floor. Gates: ruff, basedpyright, ty, vulture, complexipy, codespell all clean; 2178 passed, 1 skipped. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A Metron search that missed ran a year cycle (Y, then Y-1 and Y+1) twice,
once with `series_volume` and once without, because both filters are
guesses a filename made: cover-date drift is real, and a scanner's
`Vol. N` is inconsistent. Six requests to answer one question, and on
Metron a request is a request whether it finds anything or not.
bpepple shipped `cover_date_range_after` / `cover_date_range_before`
(Metron #628), which collapses the year cycle into one filter. Dropping
`series_volume` collapses the two cycles into one call. So the miss path
is now the exact call plus one wide fallback:
series_name + number
+ cover_date_range_after=(Y-1)-01-01
+ cover_date_range_before=(Y+1)-12-31
The hit path is byte-identical — call 1 is unchanged, including
series-cache population.
Precedence is reproduced client-side rather than abandoned. The six-call
cascade never ranked anything: it stopped at the first call that returned
rows, so the ORDER of the calls WAS the ranking. `_select_precedence_tier`
returns the first non-empty tier of [volume+exact year], [volume, any
year in range], [any volume, exact year], [any volume, any year in
range], which for every profile with an issue number is the same
candidate set the six calls produced. Letting the matcher rank the whole
three-year window instead was considered and rejected: the matcher has no
volume signal (it scores the ISSUE cover year, not the series start
year), and `_candidate_sort_key` breaks the resulting tie on the LOWEST
`volume_id` — so an adjacent-year reboot would flip from a solo
auto-write to a prompt, or under `eager` to the wrong volume.
`CandidateSummary` grew the `volume` field that tiering needs, fed from
`BasicSeries.volume` (a required int on every list row).
Guard: DRF ignores filter params it does not recognize, so an
un-upgraded or rolled-back Metron would answer the wide call with every
issue of the series ever published — silently, with a 200. Rows whose
cover year falls outside {Y-1, Y, Y+1} are dropped locally and
`warn_once` says why. `cover_date` is non-null on both sides, so the
check is exact.
Degradations, explicit:
- Year None: no range params, which IS the old drop-volume call
(name + number). Guard and tiering are skipped — every row comes back,
which is what that call returned.
- Year None and volume None: no fallback. It would repeat call 1
verbatim.
- Number None: no fallback. Deliberately NOT parity — the old cascade ran
in full here, and a wide call with only a series name paginates every
issue of every matching series, for rows the matcher has no issue
number to choose between anyway.
Failure semantics are unchanged in kind: call 1 keeps its raise-on-
failure contract, the fallback logs and returns [] the way the per-year
retries swallowed, and `OnlineLookupAbortedError` is re-raised from both.
Housekeeping: calibration harness cost model 6 -> 2, `_resolve_volume`
and `_issues_list_with_retry` docstrings, the B2 entry and status line in
`tasks/metron-rate-limit-plan.md`, and the "Not measured" section of the
2026-09-16 batch-cost note.
Not done: the live parity run against the 47 Metron-labelled fixtures.
It needs `/Volumes/Media`, which is not mounted here. It is confirmation
rather than a gate — the tiering is asserted against a mixed-volume,
mixed-year fake — and the exact command is recorded in the calibration
note.
Gates: ruff, basedpyright, ty, vulture, complexipy, radon, codespell all
clean; 2177 passed, 1 skipped.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* bump version * docs(calibration): record the Metron miss-cascade parity run W1.8, and the last open half of B2 in the rate-limit plan. Before `53a17f5` (the six-call cascade), after `3e69fff` (#213), on the 46 runnable Metron-labelled fixtures of `fixtures-bigmedia.json`, both passes cold and independent. Parity is exact: `top_issue_id` identical on all 46, `n_candidates` identical, outcomes identical (36 correct, 0 wrong, 10 no-candidates, 100% accuracy on labelled fixtures). `issues_list` calls 66 → 56, and wrapper counts equal real HTTP sends in both passes, so nothing paginated. Two things the run does NOT show, recorded rather than rounded off: - It never exercised the six-call path. All ten misses had a year and no parsed volume, so the old code ran the year cycle alone and never reached the drop-volume cycle — zero `retrying without the volume filter` lines against twenty year retries. This set measures 3 → 2; 6 → 2 stays derived until a `Vol. N` fixture that misses is added. - Recall is unchanged, not improved. All ten misses stayed misses; the set has no cover-date-drift case. `cover_date_range_*` is confirmed deployed on metron.cloud by a direct two-call probe rather than inferred from the guard's silence — the guard was silent because every fallback returned zero rows. `Wolverine #1` returns 132 rows spanning 1982-2027 unbounded and 5 rows in a three-year window. That also prices the rejected year-less variant. Incidentally confirms W3 against production: Metron reported `X-RateLimit-Burst-Limit: 60`, not the documented 20, and the gate adopted it for zero rejections across 122 sends — with zero header-less responses and zero connection failures, which is the clean baseline the W2 counters need. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
`pyproject.toml` went to 5.1.2 but NEWS had no section for it, so the entries from #212 and #213 were sitting under v5.1.1 — a release that shipped at 09cb320, before either landed. They go under their own heading; v5.1.1 is now byte-identical to what it shipped as. Those two are the whole user-facing surface since 5.1.1. The rest of the range is the parity-run calibration note (#214) and a lint-tooling dependency bump, neither of which a user sees. Also "instead of 6" is now "instead of up to 6": 6 is the worst case, and the parity run measured 3 for the common shape, where the profile carries no volume. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A write's destination collision was only ever found after the work: a
CBR was fully repacked before `_create_zipfile` noticed its CBZ twin
already existed, and two archives in one batch converging on the same
name were decided by whichever thread claimed the destination first.
- `DestinationOccupiedError` replaces the untyped `ArchiveWriteError` at
the three collision sites, carrying `source`, `destination`, `kind`
("convert" / "rename" / "inflight") and the rival `occupant`. Its
message texts are unchanged, and `__reduce__` keeps it picklable --
`WriteResult.error` is a public field and Python rebuilds an unpickled
exception as `cls(*self.args)`.
- `Comicbox.get_write_destination()` is now the one definition of where a
write lands. It was inlined twice, once from the sniffed archive type
and once from the suffix.
- `dump()` checks the destination before loading any metadata, so the CLI
benefits as well as the API.
- New `comicbox.predict.predict_write_destination()` reports the
destination without opening the archive.
- `bulk_write(preflight=True)` sniffs every destination on the first
`next()`, serially and in submission order, so an in-batch collision
loses deterministically. `preflight=False` restores the old path.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
… url (#218) mokkari 4.8.0 shipped all three upstream asks from tasks/metron-rate-limit-plan.md. U1 (#167) — `Session(rate_limiter=...)`. comicbox's `PacedSession` subclass, which overrode the private `_execute_http_request` and disabled `_check_rate_limit`, collapses to `GateRateLimiter`: a registration through the public hook. `RateGate` stays rather than mokkari's `HeaderPacedRateLimiter`; the paced_session module docstring has the point-by-point comparison so nobody "simplifies" it later. One private reach remains, `Session._http`, for the per-endpoint and header-less telemetry the hook cannot see -- a `requests` response hook, which also keeps the observe/cooldown/release order the override had. U2 (#168) — bounded pagination 429 retries. With a limiter set a page is retried through it without sleeping, so a paginated call now waits on the gate's rebuilt window, and a spent daily quota aborts mid-pagination instead of being slept out. U3 (#169) — a pooled `requests.Session`: one TLS handshake per run instead of per request. `close_shared_sessions()` releases the pool at the end of a run and from the new `OnlineSession.close()` / context manager; an `os.register_at_fork` handler drops inherited sessions in a child rather than letting two processes share sockets. Also: - `CandidateSummary.cover_url_full` — display only; the matcher keeps hashing `cover_url`, because hashing a larger image moves every score. ComicVine picks largest-first and never falls back to a thumbnail tier; Metron's image is already full size. Zero API budget either way. - `RateLimiterError` classifies as INVALID, like `CacheError`. - `outcome_stats.record_gate_wait` splits the gate wait off the send, so `requests` counts responses received -- what the server's logs show. - Tests move off `mokkari.session.requests.request`, which intercepts nothing under 4.8.0, onto a `requests` adapter mounted on the pooled session. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…re (#220) `make lint` has been exiting 2 since #218: vulture flags `stream`, `verify`, `cert` and `proxies` on both fake `BaseAdapter.send` overrides. The names cannot change. `requests.Session.send` calls `adapter.send(request, **kwargs)` with those exact keywords, so renaming them is a TypeError at runtime, and collapsing them into `**kwargs` fails basedpyright's override check (positional parameter count mismatch). So the signature stays and the body consumes them. `del` rather than a `vulture_ignorelist.py` entry: the whitelist matches by bare name across the whole tree, and `stream` / `verify` / `cert` / `proxies` are ordinary enough words that listing them would blind vulture to real dead code elsewhere. The existing `option_string` entry is safe there precisely because nothing else would ever be called that. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…222) * feat(comicvine): release its connections and sqlite handles on close `OnlineSession.close()` and the CLI's end-of-run release covered Metron only. simyan's Comic Vine client is `requests`-backed too, and holds more besides: the response cache's sqlite connection, one rate-limit bucket connection per endpoint pool, and pyrate-limiter's leaker thread. Left to finalization they surface as the `ResourceWarning` that asked for this. Closing the session is not enough on its own, and on its own is worse than nothing. requests-ratelimiter's `close` does ask pyrate-limiter to close the buckets, but `BucketFactory.close()` drops its leaker before iterating `get_buckets()`, which reads that leaker, so the loop closes nothing (pyrate-limiter 4.5.0). What it does do is drop the references that had been keeping those buckets alive, so a session-only close turns a silent leak into one `ResourceWarning` per pool: 0 to 5 measured on a run that touches every endpoint comicbox calls. So the close empties the factory's bucket registry and closes each bucket itself. Emptying first is what keeps a closed bucket from being handed back out to a racing lookup, which would die on its `None` connection; each bucket's own lock covers an acquire already in flight. A closed client stays usable, as Metron's does. The response cache reopens its connection lazily, and the hourly budget lives in the bucket file rather than the bucket object, so a rebuilt bucket resumes where the closed one left off -- `shared_client_rate_limit_status` reads the same numbers across a close. The one sharp edge, documented on `OnlineSession.close()`: a Comic Vine lookup that already holds a bucket can fail outright rather than merely reconnect, so close between files, not under one. tests/util's `close_comicvine_client` had grown the same bucket dance for test hygiene; it now delegates to the production close instead of being a second copy of it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * bump version Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* update devenv & deps - devenv: eslint base config (presets as separate flat-config entries via defineConfig extends), pyproject template (T201 per-file ignores for the copied bin/ scripts, prettier-plugin-toml 3 formatting), bin/roman.py drops its now-redundant T201 noqas. - Retire CircleCI leftovers: .circleci lines in .dockerignore, .prettierignore and .remarkignore; the prettier "**/*Dockerfile" sh-parser override in package.json. - Node dev deps: eslint plugins, prettier 3.9.9, prettier-plugin-toml 3.0.2; bun.lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: move to devenv's modular CI ci.yml is now devenv's standard caller: devenv-check (gate with Release Preflight and main-push dist reuse, one CI image pulled by a fail-fast Lint / Test / Build Dist matrix, and the "CI / Lint, Test & Build Dist" required check), then PyPI through the devenv-pypi action, then devenv-release (tag, GitHub Release from NEWS.md, merge main into develop). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The schema markdown generator treated every $ref as a local "#/..." pointer and chopped two characters off, so links like identifiers.schema.json became the dead fragment #entifiers.schema.json. Point each link at its sibling doc page instead. Fixes the markdown/no-missing-link-fragments lint errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The site/schema pages described a stale v2.0 snapshot. Regenerate them from comicbox/schemas/v3.0: adds comicbox-v3.0, credit, role and alternative-name pages, drops comicbox-v2.0 and credits, and picks up v3.0 field changes (identifiers id_type, page key as string, reprint name/identifiers, series alternative_names, volume minimums). Add generate-schema/gen-schema-docs.sh so the pages can be rebuilt. It pins jsonschema2md 1.7.0 and rewrites the bare relative $ref links that jsonschema2md can't resolve to point at the sibling pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The devenv update dropped the `.*` skip that made codespell skip the
whole tree, so these are its first real findings.
- Spell "unparsable" and "preempts" the way codespell expects, including
a hyphenated "unparseable-body" it can't see.
- Reword four false positives ("retuned", "GET's", "UIs", "msdos")
rather than widen the project-wide ignore list.
- Skip the per-developer calibration fixtures and outcomes, mirroring
tests/calibration/.gitignore; the tracked fixtures.example.json is
still checked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ty's truthiness-test-of-iterable check caught three guards that could never fire: - get_page_by_index() truth-tested the get_pages() generator, so an index past the last page raised StopIteration from next() instead of returning None. Look the page name up and read it instead. - extract_covers() fed the raw generate_cover_paths() generator through, so "No pages to extract." never logged, and a cover found by more than one heuristic was written and counted once per heuristic. Use the deduplicated get_cover_paths() tuple; the extract path now only takes sequences. - A dead `if not matches` guard on a finditer() result in notes. Mint v5.2.2 for the fixes since v5.2.1 is tagged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_page_by_index()returnsNonefor an index past the last pageinstead of raising
StopIteration.heuristic was written, and counted, once per heuristic.