Repository navigation
Conversation
* upgrade confuse to 2.2.0; replace AttrDict with typed Settings dataclass confuse 2.2.0 makes AttrDict properly generic, so per-key types resolve to `object` and consumers across the box mixins fail typecheck. Convert the validated AttrDict into a frozen `Settings` dataclass once in get_config() and propagate that typed object everywhere; confuse stays confined to comicbox/config. - New comicbox/config/settings.py defines `Settings` and `ComputedSettings` (frozen, slots). - get_config() returns Settings; new _build_settings() does the conversion. post_process_set_for_path() rebuilt around dataclasses.replace. - FrozenAttrDict deleted — frozen dataclass enforces immutability. - process.py passes Settings through pickle directly so workers skip re-running confuse. - Drops dead `dest_path is None` checks now that the field is required. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * rename Settings to ComicboxSettings So that client programs that already define their own `Settings` type don't collide on import. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * flatten ComputedSettings into ComicboxSettings The hierarchical split was a confuse-template setup convenience, not a logical grouping — there's no API benefit to keeping client code chained through `cfg.computed.X`. Promote the six computed fields onto ComicboxSettings under a clearly labeled comment block. The confuse template's nested `computed` MappingTemplate is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`_get_source_config_metadata` early-returned an empty list whenever the caller set `metadata_format`, because `fmt not in self._config.read` compared a string against a frozenset of `MetadataFormats` enums — always True. The conversion + correct membership check happens in the try block on the next lines, so the early return was both wrong and redundant. Adds tests/unit/test_sources.py covering the four behavioral cases: fmt-in-read, no-fmt, fmt-not-in-read, invalid-fmt. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
) read_config_sources used config.add() for the Mapping branch, which appends to the BOTTOM of confuse's source priority stack — below the config_default.yaml loaded by config.read() at the top of the function. So any caller passing a dict / Mapping override (e.g. `get_config({"comicbox": {"compute_pages": True}})`) silently got the default instead. Switch to config.set() so Mapping args land on top, matching set_args() for the Namespace branch. Surfaced by a downstream Codex migration that hit dead Mapping overrides; covered now by tests/unit/test_config_layering.py. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The template arms for `read`, `write`, `export`, `delete_keys`, `read_ignore`, and `print` previously combined `frozenset` (a pass-through marker) and `Sequence(str)` (list-of-strings coercion). That works for the common YAML/CLI list path but rejects callers passing a `set` / `tuple` / `frozenset` literal — which is logically fine for fields whose post-compute value is always a frozenset. Replaces the per-field unions with `OneOf((set, frozenset, tuple, list))` (`print` also accepts `str` for the historical phase-char form). The `_build_settings` boundary already calls `frozenset(...)` on these values, so any of the four containers normalize correctly. Also adapts `compute_config`'s helpers — Subview iteration only supports dict/list source values, so user-supplied set/frozenset/tuple inputs would error before reaching the template. New `_raw_or_empty` pulls the Python value via `.get()` and explicitly rejects mappings with a clear error (dict iteration would silently accept dict input otherwise). `_parse_print` now accepts a phase-char string OR any iterable of phase chars. Path-list fields (`paths`, `import_paths`, `metadata_cli`) keep their existing `Sequence(...)` form with element-type validation — that trade-off felt worth keeping. 14 new tests in tests/unit/test_config_container_inputs.py cover the four container types per field and assert mapping rejection. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Callers that only want a thumbnail (e.g. codex's CoverThread) don't need the full ComicInfo/CoverImage hint resolution. Parsing the metadata for every cover dominates the cost of cover extraction and emits a flood of debug-bucket Union ValidationErrors that look like real failures in DEBUG logs. When skip_metadata=True, bypass generate_cover_paths entirely and read archive index 0 directly. This drops per-call schema instantiation, Union resolution, and path normalization. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps
…122) ClearingErrorStoreSchema previously split each schema's errors into two buckets: ignored ones logged at DEBUG, real ones at WARNING. The DEBUG bucket only ever held errors from ``_ignore_errors`` — ``Field may not be null.`` (sparse-field tolerance) and ``Invalid input type.`` (Union variant misses) — both of which are internal mechanics, not operator-actionable signal. Each Union miss emitted one ``ValidationError - {'_schema': ['Invalid input type.']}`` line per field per archive, drowning the genuinely useful per-source DEBUG messages emitted by ``_except_on_load``. Filter ignored errors at split time, log only WARNINGs. Real schema failures still surface with full context (path, schema class, normalized message). Collapses the dual-bucket _split_*_errors methods into _filter_* + _log_warnings. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps * metron: drop broken URL slugs for genre, location, reprint, role, story, tag Metron has no public web pages for these types — only API endpoints — so URLs like https://metron.cloud/genre/3 always 404. Stop emitting them. The numeric Metron ID is still preserved on the identifier. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shortens the import path for the helper from comicbox.enums.maps.age_rating to comicbox.enums.maps so downstream callers can reach it without drilling into the submodule. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Remove unused module/class constants: _COMMENT_ARCHIVE_TYPES, SUFFIXES, _LOG_FORMAT, comet.py IDENTIFIER_TAG/IS_VERSION_OF_TAG, comictagger.py IDENTIFIER_TAG/PAGES_TAG, XmlCountryField (and now-orphaned imports RarFile, ZipFile, CountryField). - Fix latent bug in TrapExceptionsMeta: `attr_name in "deserialize"` was a substring check that wrapped any callable whose name was a substring of "deserialize" (e.g. "er", "size", "ali"). Use the existing _WRAP_METHODS tuple instead so only the exact `deserialize` method is wrapped. - Simplify _get_pdf_enabled() to a plain `import pdffile` probe; the except-arm stub import had no effect. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidate the optional comicbox-pdffile integration into one module (comicbox/_pdf.py) and delete the hand-maintained pdffile_stub.py. Previously six call sites each duplicated a `try: from pdffile import X / except: from pdffile_stub import X` block, and the stub class mirrored the real PDFFile API method-for-method — silent drift risk every time upstream pdffile shipped. Now: - comicbox/_pdf.py is the single source of truth for PDF_ENABLED, PDFFile, and PAGE_FORMAT_VALUES. When pdffile is absent, PDFFile is None at runtime; type checkers see the real class via TYPE_CHECKING. - Every call site that touches PDFFile is gated by `if PDF_ENABLED`. - The `case PDFFile():` arm in box/archive/archive.py is lifted to an `if PDF_ENABLED and isinstance(archive, PDFFile):` guard above the match (the match form would fail when PDFFile is None). - config/__init__.py reads PAGE_FORMAT_VALUES instead of iterating an empty stub Enum. Verified with `pdffile` installed (307/307 tests pass) and in a fresh venv without it (PDF_ENABLED=False, CBZ archives still work, PDF files raise UnsupportedArchiveTypeError, CLI shows the "not installed" hint). Net: -70 lines across 9 files. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* compact news * update deps * update news and version to alpha 4 * update deps * rename function path in NEWS * bump alpha version to 3.0.0a5
* require comicbox-pdffile 0.6.x for image-dominant page detection Widens the optional ``[pdf]`` extra to require comicbox-pdffile 0.6.x. The new minor release adds image-dominant page detection ( ``PDFFile.classify_page``, ``PDFFile.read_image_if_dominant``, ``PDFFile.read_full_pixmap_jpeg``) used by browser readers to serve scanned-comic PDF pages as plain ``<img>`` instead of routing through pdf.js on the client. comicbox itself doesn't use the new API — the bump is purely a pin update so downstream callers (Codex, OPDS readers) can adopt it. The ``[tool.uv.sources]`` block is transient: it points at the pdffile PR branch so this CI can resolve dependencies before 0.6.x lands on PyPI. Drop it once 0.6.x publishes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * just use the released pdffile --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
simyan 4.1.0 ships Simyan#309/PR #310 — the pagination fix this project filed during the 4.0 adoption. `_offset`/`_paginate` now stop on a short page instead of looping until an empty one, so every fan-out list call spends one request against Comic Vine's 200/hour pool instead of two. No comicbox code change was needed. Diffing the 4.0.0 and 4.1.0 wheels, the release is the two loop exits, a defensive `params` copy, a version bump and one stale docstring line — no new endpoints, resources, schema fields or parameters. The `params`-mutation fix can't reach comicbox either: all three `params=` call sites build a fresh dict literal inline. `COMICVINE_ISSUE_LIST_REQUESTS_BY_EFFORT` needs no re-derivation. Its constants were anchored to `api_call_counts`, which counts comicbox-level calls, so they always described logical calls; one logical call is now one request, which turns the table from a floor at half the true cost into an accurate request count. The 2026-09-08 decision not to double them to describe an upstream bug is what makes this a no-op. The floor and lock landed in af8caab "update deps"; this records it: - online_estimate: drop the now-false 2x caveat from the module docstring. - NEWS: v5.0.1, Performance + the simyan >= 4.1.0 floor. - simyan-4-plan: close Phase 3, with a both-wheels probe table (identical result sets, 2 requests -> 1 in every non-boundary case). - calibration note: close the simyan under-count; the Metron one stands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The series-first batching warm path called `source.lookup_issue(volume_id,
profile.issue)` whenever a cached volume_id existed for the comic's series
fingerprint, guarding only on `profile.series`. A comic with a series,
publisher and year but no issue number reached that call with
`profile.issue` None or "".
Both sources then dropped the issue filter instead of refusing the call.
Metron sent `issues_list({"series_id": N})` with no `number`, which mokkari
paginates across every issue in the series, and accepted `issue_list[0]`.
ComicVine's bare `volume:N` filter had the same hole. The result was a
mis-tag against whatever issue happened to sort first, on top of an
unbounded number of HTTP requests.
Guard in two places:
- `_try_series_cache_lookup` declines when the profile has no usable issue
number, so the comic falls through to the search path, which already
handles issue-less profiles.
- Both sources' `_lookup_issue_in_volume` return None without a request
rather than listing a whole series or volume. ComicVine now normalizes
the number before filtering, matching Metron and the search path.
Regression tests cover the box guard and both sources; each fails without
its corresponding fix.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat: pace Metron requests through a rate gate (#207) Metron's DRF backend throttles with a per-token sliding LOG of send timestamps, and it evaluates each throttle class independently — so a burst 429 debits the 5,000/day sustained quota exactly like a successful request. A user in issue #207 took 1044 of them in one day. Every one cost them a day-slot and told them nothing. Nothing in comicbox paced requests. `--jobs` bounded WORKERS, which is not the unit Metron throttles: 20 workers still send far more than 20 requests a minute, because one comic costs several. mokkari's own `_check_rate_limit` is a fail-fast that reads the last response's headers; under a thread pool it is advisory at best, since the check is not serialized with the send and `_update_rate_limit_status` is last-write-wins, so a slow response can put a stale `remaining` back on record after a fresher `0` was seen. Model the server's window instead of discovering it by rejection: - `RateGate` (comicbox/formats/base/online/rate_gate.py) is a sliding log mirroring DRF's, one per credential set. It starts serialized, switches to the limit `X-RateLimit-Burst-Limit` reports, and stops gating entirely against a Metron that does not throttle. Header feedback only ever TIGHTENS, so an out-of-order response cannot widen the estimate. Every deadline is monotonic plus a relative server hint; the epoch `-Reset` headers are never trusted as local time, because comicbox runs on NAS boxes and in containers whose clocks drift. - On a rejection the gate rebuilds the window the server must be holding, with its oldest slot freeing at `Retry-After`. That admits exactly one worker at the deadline and paces the rest behind it. Parking everyone on the hint and releasing them together is what turns one 429 into a burst — `Retry-After` is when ONE slot frees, not when the window refills. - `PacedSession` puts the gate at mokkari's one real choke point, `_execute_http_request`, so list calls, detail fetches, every page of a paginated result and conditional GETs are all paced. It is a private seam; tests/unit/test_paced_session.py drives real mokkari methods through a fake transport and asserts one acquisition per HTTP send, so an upstream rename fails loudly instead of silently unpacing us. The upstream ask that retires it is U1 in the plan. Also here: - Sustained-quota watermarks: warn once at 10% remaining, stop starting cold searches near the floor so the rest of the day finishes comics that already matched, abort at zero. - Per-endpoint request counts, rejections, paced seconds and remaining budgets in the end-of-run summary, so a user can paste numbers that line up with the server's logs. - Remove the `--jobs` cap; the gate is the rate control now. - Honor `rate_limit.per_minute` again as a ceiling, for embedders splitting one token across processes. `per_day` still warns. - Wire `online.tuning.retry_budget`, which was parsed and dropped. - Stop retrying `OnlineLookupAbortedError` — no classifier claimed it, so the gate's quota-exhausted abort would have been replayed eight times. - User-Agent carries the entry point: `comicbox/5.0.1 (cli; jobs=8)`. Plan: tasks/metron-rate-limit-plan.md, PR A. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf: resolve each series once per batch and prefetch long runs (#207) Two ways a batch overpays Metron, both of which only show up at the scale users actually run at. Series batching resolved a series once — but only once the first file of a cluster had FINISHED. A `-j N` pool takes the first N paths at the same instant, and the dispatcher had just sorted them so those N were the same series. All N missed the cache and all N paid for the same cold search: the optimization saved nothing for exactly the files it was added for. - `SeriesCache` now arbitrates single-flight resolution. One caller leads a key and the rest wait on its event, then re-read and take the warm volume-scoped path. A leader that resolves nothing — no match, declined prompt, source failure — hands leadership to the next waiter instead of stranding the cluster, and the wait is bounded so a wedged leader costs latency rather than the whole cluster. - The dispatcher hands the pool one file per series FIRST, then the rest still clustered. The first N tasks are then N different series, so waiting is rare rather than merely safe. Serial runs keep plain clustering, which reads better in a log and races nothing. Second: with `PAGE_SIZE=100` a whole series is one or a few pages, so a long run was buying one `issues_list` per comic to learn things one list call knows. `MetronOnlineSource.prefetch_volume` now asks `series(id)` for the issue count and, only when `1 + pages` beats the per-comic lookups it would replace, lists the series once and serves the rest of the cluster from memory. The leader pays for it while still holding the lead, so followers wait for a warm list rather than racing it. A 100-issue single-series batch goes from ~200 requests to ~102. The floor is the per-comic `issue(id)`: `BaseIssue` carries no credits or characters, so a match always costs one detail fetch. The prefetch is an optimization with a working fallback one line away, so it must never be able to fail or delay a comic: every failure degrades to the per-comic path, and the `series(id)` probe pins `max_retries=1` rather than spending the user's whole retry budget and its 31s of backoff. That made `max_retries` mean what it says again — an explicit value at a call site now wins over `online.tuning.retry_budget`, which applies where the call site expressed no opinion. NOT done, from the plan's §B2: dropping `series_volume` from queries, and shortening the year cascade. The plan justified the first on the matcher already scoring year distance, and that premise does not hold — `CandidateSummary` has no volume field, the year signal scores the ISSUE cover year, and ties break on the LOWEST volume_id, which is the oldest series record and the wrong answer for a reboot. Both changes need a calibration run against live Metron for `top_issue_id` parity, and there is no offline cassette harness to do it with. Written up in tasks/online-tagging/calibration-notes/2026-09-16-metron-batch-cost.md. Plan: tasks/metron-rate-limit-plan.md, PR B and PR C. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: prove the zero-429 goal offline, and count 429s in the stress run The acceptance criterion for #207 is "a `-j N` batch earns no 429s", and until now the only thing that could answer it was a live stress run — credentials, minutes, and spending the very quota the change protects. So nothing would have caught a regression between runs of it. `tests/unit/test_rate_gate_concurrency.py` answers it offline: a real thread pool, a real `PacedSession`, and a transport that enforces the throttle the way Metron's DRF does — a sliding log where a slot frees one window after the request that took it, headers on every response including the 429s. Four assertions, of which the second is the one that makes the first mean anything: - a paced pool of 8 sending 50 requests at a 20-per-window limit is refused nothing; - the identical load with no gate IS refused, which is what comicbox did before; - the compliance comes from pacing rather than from crawling — it still finishes in the ~2.5 windows the rate allows; - a window already spent by another client on the same token costs exactly one rejection, not one per worker, with `with_retry` replaying the refused call and the gate holding the rest behind it. Windows are scaled to 1s so it runs in about a second; the arithmetic on both sides is the real thing. `tests/stress/jobs_accuracy.py` now reports requests sent and 429s received per `-j` value, and says FAIL in the summary when any appear. It was comparing match decisions across `-j` values and silently ignoring what the run cost, which is half of what that harness is for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
5.0.1 was never released (v5.0.0 is the latest tag and the latest on PyPI), so this renames the in-progress version rather than minting a new one on top of it. Minor because the release no longer only fixes things. It adds public API — `OnlineSession(client_name=...)`, `OnlineSource.prefetch_volume`, `metron_requests_for_batch`, `source_rate_per_minute`, `user_agent` / `set_user_agent_context`, `SeriesCache.lead`/`wait`/`release` — and, more to the point for anyone upgrading, two config knobs that used to do nothing now do something. `rate_limit.per_minute` was warned about as ignored and is now enforced as a ceiling; `retry_budget` was parsed and dropped and is now applied. An existing config file changes behavior on upgrade without being edited, which is not a patch-release promise. `--jobs` also stops being clamped to 20 for Metron runs. Both knob changes now say so explicitly in NEWS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two unrelated warnings, both now silent. basedpyright flagged four `# pyright: ignore[reportUnusedFunction]` comments on autouse pytest fixtures as unnecessary. They are: the autouse fixtures added in the rate-gate work needed no such comment. Removed rather than left to rot, since a stale ignore is worse than no ignore — it suppresses a rule nobody has checked is still firing. `radon mi` had `online_lookup.py` at B. That is mine: the file was at A (19.80) before the single-flight work, which is 0.2 off the boundary, and ~110 lines of series-cache machinery tipped it. Two changes, both of which stand on their own: - The `lead`/`wait`/`release` adaptation moved next to `claim_series` in `series_cache.py`, which already owns the "works on any MutableMapping" contract. The box was doing `getattr(cache, "lead", None)` in three places to ask a question the cache module should answer. - The cover-hash cache and download pool split into a `ComicboxOnlineCovers` mixin under `ComicboxOnlineLookup`. They are box-lifetime resources with their own `close()`, the dependency runs one way (the lookup flow reaches down for a hash; nothing here calls back), and they took the file's most complex function with them. This is the "one focused concern per mixin layer" the architecture in CLAUDE.md describes. `online_lookup.py` is now A (23.01) — better than before the feature work, not just back over the line. No behavior change: 2161 tests pass, and the moved pHash path is exercised end to end against a real archive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both changes exist so an embedding application can tell "nothing was found" from "nothing was looked at". The two look identical from outside and call for opposite handling. - `OnlineSession.abort_reason` keeps the reason a lookup aborted the batch. A spent daily quota and a caller's own pause both surface as a stream of cancelled results; only one of them is the caller's doing, and only one should leave the batch resumable. - A search the daily-quota reserve stopped now emits `Skipped` with `SKIP_QUOTA_RESERVED` instead of passing for a source that searched and missed. The reason lives on the source, which owns its whole lifetime (set when declining, cleared at the top of its own `search()`), and the lookup reads it structurally — a source is whatever implements the search protocol, and most never decline. The matcher's own reason string is now the exported constant `SKIP_MATCHER_DECLINED` alongside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Telemetry Brian can act on, plus the floating-burst-limit behaviour he described, pinned by a test. W2 — responses without rate-limit headers. Not "429s without headers", which is structurally zero from Metron's checked-in stack: DRF's throttles run in `initial()`, after auth but before any view code, before the conditional-GET 304 and before `X-Cache`, and Metron's middleware copies `X-RateLimit-*` onto every response whose request reached them. So a genuine `/api/` answer of ANY status carries the headers, and their absence proves the answer came from nginx or Anubis instead. Anubis ALLOWs `^/api/.*$` ahead of its challenge rules and serves both its challenge and deny pages as HTTP 200 HTML by default, so bucketing by status catches a bot-check page as readily as a 429 — with no body inspection. - `_ApiCounts.unthrottled: dict[int, int]`, fed by `record_unthrottled_response`, reported as e.g. "3 responses without rate-limit headers (200: 2, 429: 1)". - One `warn_once` line carrying status, Content-Type and Content-Length only. No body (mokkari's `ApiError` already embeds `response.text`), no `Server` header (nginx overwrites it). - `connection_failures`, counted on `_execute_http_request`'s exception path. Only transport failures reach that frame: mokkari re-raises `requests` ConnectionError and ReadTimeout as `ApiError` there, while HTTP status errors surface later out of `_handle_http_response`. This is the shape a fail2ban ban takes, where the client only sees timeouts. - A header-less 429 no longer calls `gate.observe` or `record_rate_limit_windows`. Behaviour is unchanged — `observe` with no headers only flips UNKNOWN to OPEN, logging a false "pacing disabled" that `cooldown` contradicts a moment later, and the windows call is a no-op with all-None arguments. The cooldown itself is untouched: the full-window rebuild stays, or the paced retry path would fire its whole budget back-to-back. W3 — the floating burst limit. 20/minute is the documented floor; production commonly reports 45-60 and is dialled back toward 20 on peak days. The gate already adopts whatever `X-RateLimit-Burst-Limit` says, in both directions, and the estimator prefers the live gate's number over the constant. What was missing was a test and honest wording, so `test_rate_gate` now pins a limit that drops mid-run shrinking the window and rising re-widening it, and `rate_limits` says the constant is a starting pace and an estimator fallback rather than a floor. Gates: ruff, basedpyright, ty, vulture, complexipy, codespell all clean; 2178 passed, 1 skipped. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A Metron search that missed ran a year cycle (Y, then Y-1 and Y+1) twice,
once with `series_volume` and once without, because both filters are
guesses a filename made: cover-date drift is real, and a scanner's
`Vol. N` is inconsistent. Six requests to answer one question, and on
Metron a request is a request whether it finds anything or not.
bpepple shipped `cover_date_range_after` / `cover_date_range_before`
(Metron #628), which collapses the year cycle into one filter. Dropping
`series_volume` collapses the two cycles into one call. So the miss path
is now the exact call plus one wide fallback:
series_name + number
+ cover_date_range_after=(Y-1)-01-01
+ cover_date_range_before=(Y+1)-12-31
The hit path is byte-identical — call 1 is unchanged, including
series-cache population.
Precedence is reproduced client-side rather than abandoned. The six-call
cascade never ranked anything: it stopped at the first call that returned
rows, so the ORDER of the calls WAS the ranking. `_select_precedence_tier`
returns the first non-empty tier of [volume+exact year], [volume, any
year in range], [any volume, exact year], [any volume, any year in
range], which for every profile with an issue number is the same
candidate set the six calls produced. Letting the matcher rank the whole
three-year window instead was considered and rejected: the matcher has no
volume signal (it scores the ISSUE cover year, not the series start
year), and `_candidate_sort_key` breaks the resulting tie on the LOWEST
`volume_id` — so an adjacent-year reboot would flip from a solo
auto-write to a prompt, or under `eager` to the wrong volume.
`CandidateSummary` grew the `volume` field that tiering needs, fed from
`BasicSeries.volume` (a required int on every list row).
Guard: DRF ignores filter params it does not recognize, so an
un-upgraded or rolled-back Metron would answer the wide call with every
issue of the series ever published — silently, with a 200. Rows whose
cover year falls outside {Y-1, Y, Y+1} are dropped locally and
`warn_once` says why. `cover_date` is non-null on both sides, so the
check is exact.
Degradations, explicit:
- Year None: no range params, which IS the old drop-volume call
(name + number). Guard and tiering are skipped — every row comes back,
which is what that call returned.
- Year None and volume None: no fallback. It would repeat call 1
verbatim.
- Number None: no fallback. Deliberately NOT parity — the old cascade ran
in full here, and a wide call with only a series name paginates every
issue of every matching series, for rows the matcher has no issue
number to choose between anyway.
Failure semantics are unchanged in kind: call 1 keeps its raise-on-
failure contract, the fallback logs and returns [] the way the per-year
retries swallowed, and `OnlineLookupAbortedError` is re-raised from both.
Housekeeping: calibration harness cost model 6 -> 2, `_resolve_volume`
and `_issues_list_with_retry` docstrings, the B2 entry and status line in
`tasks/metron-rate-limit-plan.md`, and the "Not measured" section of the
2026-09-16 batch-cost note.
Not done: the live parity run against the 47 Metron-labelled fixtures.
It needs `/Volumes/Media`, which is not mounted here. It is confirmation
rather than a gate — the tiering is asserted against a mixed-volume,
mixed-year fake — and the exact command is recorded in the calibration
note.
Gates: ruff, basedpyright, ty, vulture, complexipy, radon, codespell all
clean; 2177 passed, 1 skipped.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* bump version * docs(calibration): record the Metron miss-cascade parity run W1.8, and the last open half of B2 in the rate-limit plan. Before `53a17f5` (the six-call cascade), after `3e69fff` (#213), on the 46 runnable Metron-labelled fixtures of `fixtures-bigmedia.json`, both passes cold and independent. Parity is exact: `top_issue_id` identical on all 46, `n_candidates` identical, outcomes identical (36 correct, 0 wrong, 10 no-candidates, 100% accuracy on labelled fixtures). `issues_list` calls 66 → 56, and wrapper counts equal real HTTP sends in both passes, so nothing paginated. Two things the run does NOT show, recorded rather than rounded off: - It never exercised the six-call path. All ten misses had a year and no parsed volume, so the old code ran the year cycle alone and never reached the drop-volume cycle — zero `retrying without the volume filter` lines against twenty year retries. This set measures 3 → 2; 6 → 2 stays derived until a `Vol. N` fixture that misses is added. - Recall is unchanged, not improved. All ten misses stayed misses; the set has no cover-date-drift case. `cover_date_range_*` is confirmed deployed on metron.cloud by a direct two-call probe rather than inferred from the guard's silence — the guard was silent because every fallback returned zero rows. `Wolverine #1` returns 132 rows spanning 1982-2027 unbounded and 5 rows in a three-year window. That also prices the rejected year-less variant. Incidentally confirms W3 against production: Metron reported `X-RateLimit-Burst-Limit: 60`, not the documented 20, and the gate adopted it for zero rejections across 122 sends — with zero header-less responses and zero connection failures, which is the clean baseline the W2 counters need. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
`pyproject.toml` went to 5.1.2 but NEWS had no section for it, so the entries from #212 and #213 were sitting under v5.1.1 — a release that shipped at 09cb320, before either landed. They go under their own heading; v5.1.1 is now byte-identical to what it shipped as. Those two are the whole user-facing surface since 5.1.1. The rest of the range is the parity-run calibration note (#214) and a lint-tooling dependency bump, neither of which a user sees. Also "instead of 6" is now "instead of up to 6": 6 is the worst case, and the parity run measured 3 for the common shape, where the profile carries no volume. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A write's destination collision was only ever found after the work: a
CBR was fully repacked before `_create_zipfile` noticed its CBZ twin
already existed, and two archives in one batch converging on the same
name were decided by whichever thread claimed the destination first.
- `DestinationOccupiedError` replaces the untyped `ArchiveWriteError` at
the three collision sites, carrying `source`, `destination`, `kind`
("convert" / "rename" / "inflight") and the rival `occupant`. Its
message texts are unchanged, and `__reduce__` keeps it picklable --
`WriteResult.error` is a public field and Python rebuilds an unpickled
exception as `cls(*self.args)`.
- `Comicbox.get_write_destination()` is now the one definition of where a
write lands. It was inlined twice, once from the sniffed archive type
and once from the suffix.
- `dump()` checks the destination before loading any metadata, so the CLI
benefits as well as the API.
- New `comicbox.predict.predict_write_destination()` reports the
destination without opening the archive.
- `bulk_write(preflight=True)` sniffs every destination on the first
`next()`, serially and in submission order, so an in-batch collision
loses deterministically. `preflight=False` restores the old path.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
… url (#218) mokkari 4.8.0 shipped all three upstream asks from tasks/metron-rate-limit-plan.md. U1 (#167) — `Session(rate_limiter=...)`. comicbox's `PacedSession` subclass, which overrode the private `_execute_http_request` and disabled `_check_rate_limit`, collapses to `GateRateLimiter`: a registration through the public hook. `RateGate` stays rather than mokkari's `HeaderPacedRateLimiter`; the paced_session module docstring has the point-by-point comparison so nobody "simplifies" it later. One private reach remains, `Session._http`, for the per-endpoint and header-less telemetry the hook cannot see -- a `requests` response hook, which also keeps the observe/cooldown/release order the override had. U2 (#168) — bounded pagination 429 retries. With a limiter set a page is retried through it without sleeping, so a paginated call now waits on the gate's rebuilt window, and a spent daily quota aborts mid-pagination instead of being slept out. U3 (#169) — a pooled `requests.Session`: one TLS handshake per run instead of per request. `close_shared_sessions()` releases the pool at the end of a run and from the new `OnlineSession.close()` / context manager; an `os.register_at_fork` handler drops inherited sessions in a child rather than letting two processes share sockets. Also: - `CandidateSummary.cover_url_full` — display only; the matcher keeps hashing `cover_url`, because hashing a larger image moves every score. ComicVine picks largest-first and never falls back to a thumbnail tier; Metron's image is already full size. Zero API budget either way. - `RateLimiterError` classifies as INVALID, like `CacheError`. - `outcome_stats.record_gate_wait` splits the gate wait off the send, so `requests` counts responses received -- what the server's logs show. - Tests move off `mokkari.session.requests.request`, which intercepts nothing under 4.8.0, onto a `requests` adapter mounted on the pooled session. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…re (#220) `make lint` has been exiting 2 since #218: vulture flags `stream`, `verify`, `cert` and `proxies` on both fake `BaseAdapter.send` overrides. The names cannot change. `requests.Session.send` calls `adapter.send(request, **kwargs)` with those exact keywords, so renaming them is a TypeError at runtime, and collapsing them into `**kwargs` fails basedpyright's override check (positional parameter count mismatch). So the signature stays and the body consumes them. `del` rather than a `vulture_ignorelist.py` entry: the whitelist matches by bare name across the whole tree, and `stream` / `verify` / `cert` / `proxies` are ordinary enough words that listing them would blind vulture to real dead code elsewhere. The existing `option_string` entry is safe there precisely because nothing else would ever be called that. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…222) * feat(comicvine): release its connections and sqlite handles on close `OnlineSession.close()` and the CLI's end-of-run release covered Metron only. simyan's Comic Vine client is `requests`-backed too, and holds more besides: the response cache's sqlite connection, one rate-limit bucket connection per endpoint pool, and pyrate-limiter's leaker thread. Left to finalization they surface as the `ResourceWarning` that asked for this. Closing the session is not enough on its own, and on its own is worse than nothing. requests-ratelimiter's `close` does ask pyrate-limiter to close the buckets, but `BucketFactory.close()` drops its leaker before iterating `get_buckets()`, which reads that leaker, so the loop closes nothing (pyrate-limiter 4.5.0). What it does do is drop the references that had been keeping those buckets alive, so a session-only close turns a silent leak into one `ResourceWarning` per pool: 0 to 5 measured on a run that touches every endpoint comicbox calls. So the close empties the factory's bucket registry and closes each bucket itself. Emptying first is what keeps a closed bucket from being handed back out to a racing lookup, which would die on its `None` connection; each bucket's own lock covers an acquire already in flight. A closed client stays usable, as Metron's does. The response cache reopens its connection lazily, and the hourly budget lives in the bucket file rather than the bucket object, so a rebuilt bucket resumes where the closed one left off -- `shared_client_rate_limit_status` reads the same numbers across a close. The one sharp edge, documented on `OnlineSession.close()`: a Comic Vine lookup that already holds a bucket can fail outright rather than merely reconnect, so close between files, not under one. tests/util's `close_comicvine_client` had grown the same bucket dance for test hygiene; it now delegates to the production close instead of being a second copy of it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * bump version Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OnlineSession.close()releases Comic Vine's connections and sqlitehandles too, not just Metron's.