Skip to content

refactor(eq2db): base read helpers + dead code removal#151

Merged
VortexUK merged 1 commit into
mainfrom
refactor/catalogue-dedup
Jul 11, 2026
Merged

refactor(eq2db): base read helpers + dead code removal#151
VortexUK merged 1 commit into
mainfrom
refactor/catalogue-dedup

Conversation

@VortexUK

Copy link
Copy Markdown
Owner

Summary

Dedup + dead-code pass over the rearchitected eq2db catalogues (#144#150).

Deduplication — _fetchall / _fetchone on BaseCatalogue

The single biggest repeated shape left was the read preamble — 24 copies of:

if not self.path.exists():
    return []
with sqlite3.connect(self.path) as conn:
    conn.row_factory = sqlite3.Row
    rows = conn.execute(...).fetchall()

Two base helpers now own the missing-file guard, Row factory, per-call open/close, and OperationalError graceful degradation. Converted: spells (all 5 reads), recipes (all 5), raids (both revision reads + the 4 ACT trigger/timer reads), zones catalogue-level reads, and aas (_query and total_max_points become thin wrappers). The zones active-record models keep their own connections — hydration needs the same conn for child fetches.

Two behaviour improvements fall out uniformly:

  • unbuilt-DB resilience everywhere — previously only aas._query and zones.list_expansions caught OperationalError; now every read logs + degrades instead of 500ing on a provisioned-but-empty file.
  • connections actually close — most reads used with sqlite3.connect(...) which commits but never closes (handle left to GC); the helpers close in a finally.

Dead code (zero references anywhere — code, tests, or docs)

  • ZoneCatalogue.zone_count, Zone.find_by_id
  • RaidCatalogue.find_zone_by_name / list_encounters_for_zone / list_zones_by_expansion / stats — the "for the future web routes" reads from the raids module's PoC era; the strategy routes ended up with their own targeted SQL and nothing ever called these.
  • Their orphaned SQL blocks (raids.sql: the zone/encounter column fragments, 3 list queries, 5 stats queries; zones.sql: count_zones).

conftest

The seven duplicated DB_PATH + catalogue.path re-point pairs collapse into one loop — the next eq2db module can't forget the catalogue.path half.

Verification

1521 tests green, ruff + pyright clean. Live spot-checks on the dev backend across the converted read paths: character spells tab (find_by_ids/upgradeable_crcs), /api/spell-scroll (recipes find_by_spell), /api/aa/spell/{crc} (find_by_crc + aas caches).

🤖 Generated with Claude Code

…moval

Dedup pass over the rearchitected catalogues:

- BaseCatalogue gains _fetchall/_fetchone: one place for the
  missing-file guard, Row factory, per-call connection open/close, and
  OperationalError graceful degradation (previously only aas._query and
  zones.list_expansions handled unbuilt DBs; now every read does, and
  connections are explicitly closed instead of leaking to GC).
- 24 copies of the connect/row_factory/exists() boilerplate collapse to
  helper calls across spells (5 reads), recipes (5), raids (6 incl. the
  ACT trigger/timer reads), zones (list_expansions, expansion_counts)
  and aas (_query + total_max_points become thin wrappers).
- Dead code removed (zero references anywhere, code/tests/docs):
  ZoneCatalogue.zone_count, Zone.find_by_id,
  RaidCatalogue.{find_zone_by_name, list_encounters_for_zone,
  list_zones_by_expansion, stats}, plus their now-orphaned SQL blocks
  (raids.sql select_zone_cols/select_encounter_cols/find_zone_by_name_ci/
  list_encounters_for_zone/list_zones_by_expansion/stats_*;
  zones.sql count_zones).
- conftest: the seven duplicated DB_PATH + catalogue.path re-point
  pairs become one loop, so the next module can't forget the
  catalogue.path half.

Behaviour unchanged on live paths: 1521 tests green, pyright clean,
character spells / spell-scroll / AA-spell endpoints spot-checked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@VortexUK
VortexUK merged commit d7f2ec7 into main Jul 11, 2026
6 checks passed
@VortexUK
VortexUK deleted the refactor/catalogue-dedup branch July 11, 2026 09:17
VortexUK added a commit that referenced this pull request Jul 11, 2026
…152)

Adversarial review of #144-#151 surfaced three shipped breakages plus
regressions and drift; this fixes the lot:

Broken pipelines (CONFIRMED crashes, no pytest coverage):
- scripts/download_spells.py + scripts/backfill_spell_effects.py still
  called catalogue.init_db(DB_PATH) against the new no-arg signature
  (TypeError). Both verified working now (backfill run end-to-end
  against a fresh temp DB).
- scripts/dev/_smoke_test_zones_db.py had five stale module-level
  zones_db.find_by_name calls (AttributeError). Now runs to completion;
  it surfaces one pre-existing DATA staleness (bosses_total meta=56 vs
  62 actual encounters — curator edits postdate the build stamp).

Regressions:
- SpellCatalogue.find_by_crc cache went lru_cache(maxsize=4096) →
  unbounded dict caching misses, fed by client ints via /aa/spell.
  Restored a 4096 bound with FIFO eviction (+ regression test).
- BaseCatalogue._fetchall/_fetchone swallowed ALL OperationalError
  (incl. 'database is locked') as empty results. Now only the
  unbuilt-schema class ('no such table/column') degrades — logged at
  WARNING per CLAUDE.md levels — and real faults propagate again
  (+ tests for both behaviours).

Hazards & drift:
- zones model-layer path params: 31 `path: Path = DB_PATH` defaults
  (bound at import, unre-pointable by conftest, never exercised)
  become required keywords; ZoneCatalogue.__init__ keeps its default.
- :memory: half-support removed from BaseCatalogue.init_db — reads
  guard on path.exists() so a memory catalogue silently returned
  empties; docstring now states why it's unsupported.
- items.py _str_field/_int_field_zero re-implemented
  backend/census/_coerce helpers — now delegate to them like
  spells/recipes already do (_int_field keeps its 0→NULL variant).
- New BaseCatalogue._find_exact_then_like: the duplicated
  exact-then-LIKE name search in spells/recipes now shares one
  implementation AND runs both queries on one connection (undoing the
  double-connect the #151 helper conversion introduced).
- Log prefixes normalised to CLAUDE.md's [lowercase-with-hyphens]:
  [spells_db]→[spells-db], [recipes_db]→[recipes-db], [db]→[items-db].

1524 tests green (3 new), ruff + pyright clean, live /aa/spell and
recipe search verified on the dev backend.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant