chore: technical debt remediation — audit, PRD, and work items WI-01…WI-15 (Kafka backbone retired) by PetrefiedThunder · Pull Request #3699 · regengine/RegEngine

PetrefiedThunder · 2026-10-28T10:56:46Z

Summary

Tech-debt remediation from docs/audits/TECH_DEBT_AUDIT_2026-10-28.md and docs/plans/TECH_DEBT_REMEDIATION_PRD_2026-10-28.md, executed one commit per work item on this branch. Both documents carry a dated Status block per item.

Delivered (phase order):

WI	Commit	What landed
01	ADR, precheck SQL, runbook, `scripts/check_schema_channels.py`	Alembic is the only schema authority; CI guard in `alembic-revision-check`; Appendix B decisions still open
02 (first half)	`fix(imports)`	production code imports `services.<svc>.app.*`; Dockerfile `PYTHONPATH`; bare-`app` guard in `check_legacy_dead_code.py`. Tests (210 files) + conftest shim + legacy entrypoints are the second half
03	`services/shared/ruff.toml`, noqa ratchet, mypy report job	wider rule set on `services/shared`; `scripts/check_noqa_ratchet.py` (baseline now 136)
04	`docs/engineering/CI_TIMING_2026-11.md`, `.github/actions/setup-python-locks`	timing report, one Python+locks setup for the 10 pytest jobs; rerun workflow kept
05	`refactor(events)`	Kafka backbone retired: 16 modules and 33 Kafka-only tests moved with `git mv` to `docs/archive/retired-topology/kafka/` (README has the call graph and the env vars no longer read); `confluent-kafka`, `avro`, `fastavro` out of `requirements.lock`; `server.main` imports with those packages blocked; dead-code guard fails on new `confluent_kafka` / `shared.kafka_*` imports
06	`requirements-dev.in/.lock`, `requirements-experimental.in/.lock`	production lock 200 → 187 distributions; `image-import-check` job
07	`scripts/check_docs_counts.py`, directory merges	ARCHITECTURE/CODEBASE_MAP counts checked against the tree; LEGACY runbooks under `docs/runbooks/legacy/`
08	`scripts/README.md`, `scripts/check_scripts_registered.py`	103 scripts registered; 14 archive candidates listed for Christopher's approval by name
09	`docs/archive/retired-topology/`	k8s manifests, gateway config, swarm runtime, infra-k8s workflow archived
10	`test(db)`	`pytest --require-db` (exports the existing `REGENGINE_REQUIRE_DB=1` contract and converts any remaining database-reason skip into a failure), `skip_or_fail_without_db()` with the local setup recipe, flag on the six pytest calls of `live-db-suite` / `seeded-migration-rehearsal`, CONTRIBUTING "Running database tests locally". A10: 95 files / 146 sites → 81 / 123
11	`frontend/src/lib/api-client.ts` `rawFetch` / `requestPublicJson`, eslint rule	no raw `fetch(` in app/components/hooks
12 (PR B)	`feat(persistence)`	`CANONICAL_LEGACY_DUAL_WRITE` (default `on`); `off` skips the `cte_events` dual-write and counts `legacy_dual_write_skipped_total{path}`; DEVOPS_NOTES rollout plan. PR A (re-point the 12 readers with parity tests) needs a session with a scratch Postgres; PR C needs the two-week window
13, 15		delivered per their status blocks
14	`refactor(server)`	`server/main.py` 1,271 → 450 lines; `startup_checks.py`, `workers_boot.py`, `routers_mount.py`, `readiness.py`; `tests/test_server_route_snapshot_wi14.py` pins the 259 production `METHOD path` routes from the pre-split file
16	`refactor(frontend)`	StatusBoard lost its `set-state-in-effect` suppression (13 remain, each on its file's next PR)

Open: WI-02 second half, WI-12 PR A and PR C, WI-04 follow-up measurement after merge, WI-10 requirement 4 (coverage floors, same measurement).

fix(ci) repairs five gates the WI-04/06/07/11 commits broke on 788ab86: factory-local lock path, E2E job missing the dev lock, Semgrep run-shell-injection on the composite action, the trust-safety scanner reaching the moved LEGACY runbooks, and a StatusBoard test fixture.

Type of Change

Refactor
CI / build
Documentation

Test Plan

Backend suites run locally on the final tree (admin 1729, compliance 282, graph 823 + 2 pre-existing ADMIN_DATABASE_URL failures, nlp 473, scheduler 407, ingestion 6109, services/shared + tests/shared serial 2915; counts are lower than before because the archived Kafka tests left). pytest --collect-only across tests and services reports no errors. The server wiring, readiness, idempotency, route-parity, OpenAPI-surface and docs-count tests pass on the split server/ package; tests/security shows the same database-gated and Redis-backed failures on the pre-split tree without services and is green only in CI.
Guards: check_legacy_dead_code, check_docs_links, check_docs_counts, check_scripts_registered, check_noqa_ratchet, check_trust_safety_copy, the ruff bug-class gate and ruff check services/shared all pass.
python -c "import server.main" with confluent_kafka, fastavro and avro blocked in sys.modules succeeds.
Frontend: npm run lint and npm run test:run pass on the final tree; tsc --noEmit is clean.

Checklist

My code follows the existing patterns in the codebase
I have not introduced hardcoded secrets or credentials
I have updated relevant documentation (if applicable)
My changes generate no new warnings or lint errors

Still needs Christopher

Commit Message Check stays red: commit c282cdb has the subject scripts: ...; rewording it needs a force-push of this branch (say "force-push the reworded subject" or merge with that check red).
Engine Review Pass needs a REVIEW: PASS <engine> <sha> marker from a second engine on the final head.
WI-01 Appendix B decisions, WI-08 archive candidates, WI-12 flag flip and PR C.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

Sixteen items scored on impact, risk and effort, with evidence measured against the tree, a Jev second opinion on the top three risks, and a four-phase remediation plan sized for one engineer beside feature work. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

vercel · 2026-10-28T10:56:52Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
regengine	Ready	Preview	Nov 5, 2026 6:46am UTC

coderabbitai · 2026-10-28T10:56:55Z

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

github-actions · 2026-10-28T11:00:59Z

PR Blocker Checklist

Head SHA: 8fe939169fc2ff613b0ea3592ea29fec8592331f

Status: Ready

All review threads resolved
No reviews requesting changes
No blocked review engines

Two (not three) of the preceding five commits are security fixes; 25 (not 24) modules exceed 1,000 lines; the codebase map lists 14 (not 7) experimental routers; every frontend `any` is in a test file; the admin bare import is a compat shim that raises rather than loading the wrong module. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

PetrefiedThunder · 2026-10-28T11:28:27Z

visual-report (non-required) is red on this head for a reason unrelated to this PR.

This PR changes one Markdown file under docs/audits/. The job's log on 9f6efb3 shows 164 of 364 visual tests failing with Screenshot name "…" must have a '.png' or '.webp' extension (Playwright 1.64 rejects extensionless toHaveScreenshot() names) plus two 60 s click timeouts on the preview-forms spec. The same job fails the same way on other open PRs, and the workflow is continue-on-error and report-only by design.

The fix already exists in #3666 (adds .png to every screenshot name and skips Vercel-protected previews). I am not porting it here: it touches six frontend test files that a docs-only PR has no reason to carry, and the check does not gate merge. Once #3666 lands, this job goes green on its own.

Generated by Claude Code

…plementer Turns the sixteen audit findings into sixteen work items with evidence, requirements, acceptance criteria, verification commands, out-of-scope lines and rollback. Adds the owner's standing rules (never delete, one item per PR, no production changes), the repo mechanics an agent hits (engine branch prefix, cross-engine review marker, gitignored audit names, docs link and workflow inventory gates), phase sequencing, a definition of done, the decisions only Christopher can make, and baseline measurements to compare against. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

PetrefiedThunder · 2026-10-28T12:16:35Z

Passwordless Auth E2E (Chromium) is red on f87ed2a for a reason outside this PR.

One test failed, on the attempt and on Playwright's retry: tests/e2e/authenticated-core-journeys.spec.ts:400 ("invite collaborator → email sent → accept invite → appears in member list"). The assertions passed; the finally cleanup threw:

Error: database is locked
  at cleanupInviteFixture (frontend/tests/e2e/authenticated-core-journeys.spec.ts:119:93)

cleanupInviteFixture opens the admin SQLite file with new DatabaseSync(adminSqlitePath()) (no busy timeout) and issues DELETEs while the admin backend still holds a write lock from the accept-invite request. The 22 other tests passed. The same code passed on the previous head 9f6efb3; the only change since is a Markdown file under docs/plans/.

No fix is open for this. Proposed patch (not pushed here, to keep a docs PR docs-only):

// frontend/tests/e2e/authenticated-core-journeys.spec.ts, both DatabaseSync call sites
const database = new DatabaseSync(adminSqlitePath());
database.exec('PRAGMA busy_timeout = 5000');

Re-running the failed job once to confirm.

Generated by Claude Code

PetrefiedThunder · 2026-10-28T23:47:31Z

Re-run result: Passwordless Auth E2E (Chromium) passed on the same head f87ed2a (job 122874475838), which confirms the database is locked failure above was a race in the test's SQLite cleanup, not a product or doc change. Frontend CI Status Gate, PR Gate: Frontend and CI Status Gate are now green. The only red check is visual-report (non-required), covered by #3666. Remaining gate: a non-author engine's REVIEW: PASS <engine> f87ed2adffcd698a09dbd7f78a92e966fbc5ec63.

Generated by Claude Code

…precheck and runbook Production (the Supabase project) was last observed at Alembic v079 with 71 revisions pending behind a role cutover that has been prepared but not executed since 2026-09-05, while 45 Supabase-CLI migrations and 12 prod-apply scripts kept reaching it by hand. This records the decision that the Alembic chain is the only authority, inventories every file in the two other channels against its Alembic counterpart (10 of 45 have one; 35 do not, with eight decisions listed for Christopher), and adds: - sql/prod-apply/PRECHECK_alembic_version_reconcile.sql: read-only; one catalog signature per revision after v079 (71 rows) and a summary of the chain position the catalog supports. Verified on a scratch PostgreSQL 16. - docs/runbooks/ALEMBIC_VERSION_RECONCILE.md: how to read the precheck and the one case in which the version row may be edited; otherwise the existing DATABASE_ROLE_CUTOVER and v079-to-head procedure applies. - docs/MIGRATION_CONSOLIDATION.md reduced to a pointer; it described March. - PRD WI-01 status and audit item 1 status updated; CI guard still open. Jev second opinion recorded in the ADR (run the chain 0.89; stamping to head unsafe p=0.95). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…d files Completes WI-01 requirement 5 from the remediation PRD and the header rule in docs/decisions/2026-10-30-schema-change-channel.md. - scripts/check_schema_channels.py: every .sql under supabase/migrations/ and sql/prod-apply/ must start with `-- alembic-revision: <id>` and `-- rendered-from: alembic/versions/<file>.py (upgrade|downgrade|precheck)` naming an existing revision whose file matches. The 38 pre-decision files without a counterpart are grandfathered by name; the list only shrinks, and a stale entry fails the run. The reconcile precheck is exempt because it checks every revision rather than rendering one. - tests/scripts/test_check_schema_channels.py: 17 tests, including one that scans the real tree and one that keeps the grandfather list honest. - backend-ci.yml: run the guard in the "Alembic revision ID guard" job and add the two channel directories to the push and pull_request paths so a channel-only change still triggers it. - Add the two header lines to the 19 files that render a known revision (v092, v128 x2, v131, v134, v135, v136, v137). Comment lines only; no statement changes. - ADR, runbook, PRD WI-01 and audit item 1 updated to "delivered"; the open items are the Appendix B decisions. Verified: guard clean on the tree (58 files scanned); 17 tests pass; scripts/check_workflow_inventory.py, check_alembic_revisions.py and check_orphan_migrations.py still pass; docs link check passes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…ard tests Two test lines exceeded 120 characters; ruff format applied to both new files. No behaviour change; 17 tests still pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

graph_update was declared in PURGEABLE_TASK_TYPES and given a 120 s visibility timeout, but nothing enqueued it and TASK_HANDLERS never had a handler. Remove the type and the timeout, say so in the worker docstring and ASYNC_PROCESSES.md, and drop it from the two tests that listed it as a mapped type. No database object names it. 47 queue tests still pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…st-dom majors (WI-13) copy-gate pinned Node 20 while .nvmrc says 22 and package.json engines says 22 or 24. Point it at .nvmrc. The one remaining literal pin is the Node 24 job in frontend-ci.yml, kept because authenticated-core-journeys needs the built-in node:sqlite module and its comment says so. With no workflow on Node 20, the jsdom 30 and jest-dom 7 Dependabot ignores lose their reason; removed so CI decides those upgrades. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…d (WI-08) scripts/README.md gives every top-level file (99) and subdirectory (11) a row: status, purpose, and what invokes it. Status values are used, data, manual, one-time, retire-with-WI-12 and archive candidate. The 14 archive candidates stay in place; moving them to scripts/_archive/ is a decision for the owner by name and becomes a one-commit follow-up. scripts/check_scripts_registered.py fails when a top-level entry has no row or a row names a missing entry, so the directory cannot silently collect orphans again. Ten tests under tests/scripts/. Wired into backend-ci lint-repo-roots, which already triggers on scripts/**. Audit §1 and the PRD now record items 8, 13 and 15 as delivered. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…-0aw2am

…ive (WI-09) Moves, with git mv so history follows: infra/k8s -> k8s/, infra/gateway -> gateway/ (ARCHITECTURE.md's reference-only claim holds: no Dockerfile, railway.toml or workflow loads it), regengine/swarm -> swarm/regengine_swarm (renamed so it is no longer importable), tests/swarm and scripts/compliance_sweep.py (its only importer) -> swarm/, the three tests/infra files that pinned the k8s manifests -> k8s/tests/, and infra-k8s-ci.yml -> workflows/. docs/archive/retired-topology/README.md says what the topology was and that nothing under it is loaded. Inventories follow: infra/gateway/tests leaves pyproject testpaths and test-suite-check's root list; tests/swarm leaves the offline slice and the AGENTS.md verification list; infra-k8s-ci.yml leaves scripts/workflow_inventory.txt; the seven k8s image-digest entries leave scripts/ci/supply_chain_baseline.json (the checker only scans infra/); the ingress.yaml entry in .secrets.baseline is re-pathed. CODEBASE_MAP, ARCHITECTURE.md, AGENTS.md and AGENT_OPERATING_MODEL.md describe the new location. regengine/setup.py already packages only the regengine package. Not moved: the five LEGACY-bannered runbooks (still linked and link-checked) and tests/infra/test_otel_collector_tls.py, because infra/otel-collector-config.yaml stays in the tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…ck; allowlist revision ids Merging main brought three Supabase files and three Alembic revisions (v139 e7a1c9d2f4b0, v140 b8c1e4f2a901, v141 08c7cf36ddbe) that the new guards did not know. The v139 and v140 files render those revisions and now carry the two header lines; partner_uploads_bucket.sql is a Supabase storage-schema object with no Alembic counterpart and is grandfathered with a dated note (decision B1). The precheck gains signature rows 72 to 74 and a privilege_column_select kind for the v139 column grant; verified on a scratch PostgreSQL 16: 74 rows, no errors, writes nothing. The ADR inventory and the runbook counts follow. check_edi_event_time_mismatch.py and jev_review_eval.py get registry rows. detect-secrets flagged the 12-hex revision ids in the precheck and its tests as high-entropy strings; each carries the repository's inline allowlist pragma, the same convention alembic/versions uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…app in production code (WI-02, first half) Every bare `from app.X` in non-test code now reads `from services.<svc>.app.X` (28 files: the four kept entrypoints, five ingestion app modules, four graph scripts, three admin scripts, eight scripts under scripts/). Inside an app package the lazily imported siblings use relative imports instead, so the module binds to whichever package name it was loaded under; the one dynamic import in edi_ingestion/routes.py resolves through __package__ for the same reason. Scripts that put a service directory on sys.path now put the repo root there. Dockerfile PYTHONPATH drops the five service directories (/app:/app/services): no production module depends on a bare `app` any more. pyproject testpaths loses the non-existent services/test_data/tests. scripts/check_legacy_dead_code.py gains a bare-app-import rule for production Python under services/ (tests in tests/scripts/), so the form cannot return. Appendix A command A2 of the remediation PRD returns zero files. Service suites before and after on this tree, no database: admin 1758/1 skipped/3 xfailed, graph 823/8/3 plus the same two ADMIN_DATABASE_URL failures, nlp 604, compliance and scheduler untouched; ingestion 6166/58/6 before, four module-identity failures after the absolute form, zero after the relative form on the affected files (full re-run recorded in the PR). The one test that asserts the entrypoint's import text now expects the services form. The 366 test files that still use the bare form, the conftest sys.path shim, and the seven legacy entrypoints (every one is imported by a service-local test, and inflow-contract-ci runs services.ingestion.main) are the second half, as the PRD's split rule directs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

… pytest jobs (WI-04) docs/engineering/CI_TIMING_2026-11.md records the last 30 pull_request runs of pr-gate, pr-guards, backend-ci, frontend-ci and security: run wall-clock, and per job the median, p90, failure count and first failing step, produced by scripts/ci/ci_timing_report.py (gh api, one page per run). The critical path is backend-ci's Test ingestion job (median 20.2 of a 24.1-minute run); pr-gate's p90 is waiting on it; the four PR Gate: Frontend failures all end in the 12-minute wait, and no single spec fails more than once, so there is no flaky spec to fix and pr-gate-frontend-rerun.yml stays. .github/actions/setup-python-locks is the one definition of the backend test environment (Python, pip cache keyed on both locks, hash-verified install of requirements.lock and requirements-dev.lock); the ten pytest jobs in backend-ci, test-suite-check, security and qa-health use it. Every pull-request workflow already carried a concurrency group with cancel-in-progress except the two that must not cancel. The supply-chain baseline entry for security.yml follows the simplified install line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

github-actions · 2026-11-04T06:39:39Z

✅ Bundle Size Report

Metric	Size
Total JS	5693.7KB (5.56MB)
Total CSS	143.8KB
Limit	10.0MB

Bundle is within the 10.0MB limit.

github-actions · 2026-11-04T06:41:13Z

Vercel preview: https://regengine-hc3xc4p4r-petrefiedthunders-projects.vercel.app

Access is via Vercel login (Deployment Protection).

The PostgreSQL task queue has been the only production event backbone since EVENT_BACKBONE defaulted to pg (#1159). Every Kafka producer and consumer was gated off, yet the modules, their 33 test files and confluent-kafka[schema-registry,avro] + fastavro still shipped in the image. This moves the whole path to docs/archive/retired-topology/kafka/ with `git mv` (nothing deleted; the archive README carries the call graph and the env vars no longer read) and removes the live-path hooks. Live-path changes - ingestion: `_process_and_emit` returns the NormalizedEvent instead of publishing it; the >1MB claim-check stays. `/health` reports `"kafka": "retired"` with no broker probe. KAFKA_COUNTER and the kafka_* settings are gone. - scheduler: `_process_new_items` acks on webhook delivery + compliance alerting; the emit_batch / emit_fsma_batch branch, `kafka_producer` and its shutdown close are gone. - admin: the review-consumer thread and its EVENT_BACKBONE gate are gone; `/health/consumer` returns a static 200. - nlp: `services/nlp/app/consumer/` keeps only what the task processor imports (extractor flags and lazy getters, conversion, thresholds, schema_validation, tenancy, `_build_review_reasons`); the pure `_is_fsma_event` / `_collect_ftl_flags` moved to a new `events.py`. The consumer thread left `services/nlp/main.py`. - graph: OverlayWriter logs `tenant_control_committed` / `control_mapping_committed` instead of publishing to graph.audit. - secrets: `validate_production_secrets` requires DATABASE_URL only; `get_kafka_credentials` and the Kafka fields in the admin, graph, nlp, ingestion and scheduler configs, `.env.example` and `scripts/export_env_bundle.py` are gone. Dependencies and guards - `requirements.in` drops confluent-kafka and fastavro; the lock was recompiled in place (avro, confluent-kafka, fastavro removed, no other pin moved; dev and experimental locks unchanged). Lock headers now match the CI CUSTOM_COMPILE_COMMAND form. - `scripts/check_legacy_dead_code.py` fails on `confluent_kafka` and `shared.kafka_*` imports in Python files (patterns can now be scoped by suffix). `scripts/noqa_baseline.txt` ratchets 138 -> 136. - `server.main` imports with confluent_kafka, fastavro and avro blocked in sys.modules. Tests - Kafka-only tests moved to the archive; mixed files (test_extractor_wiring, test_scheduler_hardening, test_observability_correlation, test_qa5_shared_misc, the secrets tests, test_config_1342, test_placeholder, the ingest/NLP contract test) were copied to the archive where they lost sections and trimmed in place. The production-detection heuristic test now injects a synthetic required credential instead of relying on the Kafka branch. Docs: ARCHITECTURE.md (backbone bullet, large-module table), ASYNC_PROCESSES.md, README.md, the retired-topology READMEs, PRD and audit WI-05 status blocks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

Five checks went red on 788ab86; none was a product defect: - factory-local offline tests: packages/factory-local carries its own requirements.lock (pytest included) and runs from that directory, so the repo-root requirements-dev.lock added in WI-06 could not be found. Install only its lock again. - Passwordless Auth E2E: scripts/setup_e2e_postgres.py imports tests/migrations/test_seeded_mid_chain_upgrade.py, which imports pytest; the lock split moved pytest to requirements-dev.lock. Install both locks in the E2E job. - SAST (Semgrep run-shell-injection): the WI-04 composite action interpolated `${{ inputs.dev }}` inside `run:`. Pass it through `env` and read the variable in the script. - Lint & Format (check_trust_safety_copy): WI-07 moved the LEGACY runbooks from the unscanned repo-root runbooks/ to docs/runbooks/legacy/, where the scanner found "Redpanda". Skip that directory like docs/archive. - Frontend unit tests: the WI-11 StatusBoard test fed a payload without `services`, which React rendered into an uncaught TypeError that vitest reports as an unhandled error. The fixture now matches HealthPayload. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

… (WI-10) The fail-not-skip contract already existed as REGENGINE_REQUIRE_DB=1 (tests/migrations/rls_db.py, tests/integration_requirements.py, the ingestion _DB_READY modules) and the live-db-suite and seeded-migration-rehearsal jobs already set it. What was missing was a CLI entry point, one helper, and a backstop for skips raised with other wording: - root conftest.py: `pytest --require-db` exports REGENGINE_REQUIRE_DB=1 for the module-level gates and, through a pytest_runtest_makereport hook, turns any remaining skip whose reason names a database condition into a failure. Package, service-not-running and platform skips are untouched. - tests/migrations/rls_db.py: `skip_or_fail_without_db(reason, gate=)` whose skip and fail messages carry the local recipe (dev compose postgres, CREATE DATABASE regengine_test, REGENGINE_MIGRATION_TEST_DB, alembic upgrade head, pytest --require-db). skip_or_fail_db, tests/integration_requirements._skip_or_fail and the bootstrap-staging helper delegate to it; 24 inline pytest.skip database sites under tests/ and services/ call it directly. - backend-ci.yml: --require-db on the six pytest invocations of the two jobs. - CONTRIBUTING.md: "Running database tests locally". - tests/scripts/test_require_db_option.py runs a probe module in a subprocess with a minimal environment and proves both modes. - tests/migrations/test_v140_partner_file_storage_retention.py no longer assigns its local-PG fallback to os.environ for the rest of the session (MonkeyPatch.context); the leak made every later guarded_live_database_url() caller fail as "set but not reachable". Appendix A command A10: 95 files / 146 sites before, 81 / 123 after. Requirement 4 (coverage floors) waits for the post-merge CI measurement. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…s dual-write (WI-12 PR B) `CANONICAL_LEGACY_DUAL_WRITE` (default `on`) is declared on BaseServiceSettings and read at write time by legacy_dual_write.legacy_dual_write_enabled(). With `off`, the single and batch paths of CanonicalEventStore skip dual_write_legacy and increment `legacy_dual_write_skipped_total{path="single"|"batch"}`; with `on` the #1277 behaviour is unchanged. The per-caller `dual_write=False` opt-out is untouched and not counted. tests/shared/test_canonical_legacy_dual_write_flag.py proves, against the mocked session, that no `INSERT INTO fsma.cte_events` runs while the flag is off, that the canonical insert still succeeds, and that the counter moves by one per single write and by N per batch. DEVOPS_NOTES.md carries the rollout plan (staging off for two weeks with scripts/verify_canonical_parity_live.py daily, then production; Christopher flips the flag). PR A (re-point the 12 cte_events readers with parity tests) is deferred to a session with a scratch Postgres; PR C waits on the window. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…ot, router mounting and readiness (WI-14) server/main.py drops from 1,271 to 450 lines and keeps app construction, the middleware order, the router policy and the lifespan. No behaviour change: - server/startup_checks.py: run_import_time_checks (auth config + the two production secrets gates, evaluated at import as before), init_admin_database, register_integration_connectors, log_integration_config (the jev_configured / document_extract_configured deploy-log lines), configure_audit_logger. Lazy imports kept so tests that monkeypatch the source modules still work. - server/workers_boot.py: start_background_workers (ingestion task worker + embedded scheduler, same app.state attributes for /readiness) and stop_background_workers (same shutdown order). - server/routers_mount.py: the include_owned_router calls in their original first-match order. - server/readiness.py: /readiness unchanged; main re-exports `readiness` and registers the route with the RouterPolicy predicate. The lifespan keeps every log event in order and the source anchors the wiring tests assert on (ORM alignment calls, RLS posture gate before auth and workers, the auth-runtime shutdown step). tests/test_server_route_snapshot_wi14.py imports server.main in a child under production posture with the release-approved router set and compares every `METHOD path` with tests/snapshots/server_main_production_routes.json (259 entries, generated from the pre-split file). scripts/check_docs_counts.py counts the router calls in routers_mount.py (81; main mounts the scheduler status router itself); tests/test_server_readiness_probe.py patches server.readiness; ARCHITECTURE.md and docs/CODEBASE_MAP.md updated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…ion (WI-16) WI-11 touched frontend/src/app/status/StatusBoard.tsx, so per WI-16 its react-hooks/set-state-in-effect suppression goes: the fetch is now a module-level fetchHealthState() returning the next FetchState, and the effect writes state only from the promise callback (with a cancelled flag for unmount). The refresh button and the 30 s interval behave as before; eslint reports nothing on the file and the StatusBoard tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…checks.py (WI-14) Two services/shared tests read server/main.py as text to prove the auth config validator and the two production secrets gates run at import. WI-14 moved those calls into server/startup_checks.py (run_import_time_checks); the tests now assert the imports and calls there and that server/main.py invokes run_import_time_checks() at module level. These tests run only in the Test shared CI job, which is why the split did not trip them locally. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

…napshot test detect-secrets' Secret Keyword plugin flagged the four import-only placeholder values the snapshot test feeds its child process; they are synthetic (same shape as the OpenAPI contract test's) and now carry the pragma the PR Gate: Secrets job expects. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

Brings in #3780 (relative imports in three ingestion modules plus the test_no_absolute_app_imports guard), #3793, #3794, #3807 and #3809. Both conflicts (stripe_billing/rate_limiting.py, webhook_compat.py) were the same relative-import change made on both sides; main's wording kept. The new guard test, the WI-02 bare-import guard, the route snapshot test and the repository gates pass on the merged tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

PetrefiedThunder · 2026-11-05T06:17:08Z

Engine Review Pass is red for a reason this PR cannot fix from its own diff.

The check reports "No passing review from a non-author engine for current head SHA". This session authored every commit on the branch, so a marker from it would not count, and posting one under another engine's name is not allowed.

What it needs: a second engine (Codex, Copilot, or another non-author reviewer) to review the current head and post its own marker:

REVIEW: PASS <engine> 8fe9391 (full SHA from the PR head; it moves on every push)

History: 9ccc935 → 1e06d69 was a force-push that reworded one commit subject (c282cdb → 8fb5760, now chore(scripts): …) with no tree changes. 1e06d69 → 2ed17b3 merged origin/main and added one ARCHITECTURE.md row so PR Gate: Docs passes on the merge ref. 2ed17b3 → 8fe9391 merged origin/main again after #3782 landed: it conflicted with this PR's WI-02 fix to the lazy rejection_log import in edi_ingestion/routes.py; both sides made the same fix, main's relative-import form was kept. 37 EDI route tests pass on the merged tree.

Nothing else is outstanding on this check. The PR Blocker Checklist reports Ready.

Generated by Claude Code

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

… table main's #3720 and #3726 grew services/ingestion/app/stripe_billing/routes.py to 1029 lines, past the 1000-line threshold scripts/check_docs_counts.py (WI-07) checks ARCHITECTURE.md against, so PR Gate: Docs went red on the merge ref. Add the row; no code change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

vercel Bot deployed to Preview October 28, 2026 10:58

View deployment

github-actions Bot added the docs label Oct 28, 2026

vercel Bot deployed to Preview October 28, 2026 11:06

View deployment

vercel Bot deployed to Preview October 28, 2026 11:37

View deployment

PetrefiedThunder changed the title ~~docs(audits): technical debt audit 2026-10-28~~ docs: technical debt audit and remediation PRD (2026-10-28) Oct 28, 2026

PetrefiedThunder changed the title ~~docs: technical debt audit and remediation PRD (2026-10-28)~~ docs: technical debt audit, remediation PRD, and WI-01 schema-channel decision (2026-10-28) Oct 30, 2026

PetrefiedThunder and others added 5 commits October 30, 2026 06:50

github-actions Bot added backend ci agent:infra Issues found by Bot-Infra agent database labels Oct 30, 2026

PetrefiedThunder and others added 3 commits November 4, 2026 05:34

Merge remote-tracking branch 'origin/main' into claude/fervent-clarke…

7795b1c

…-0aw2am

vercel Bot deployed to Preview November 4, 2026 05:38

View deployment

github-actions Bot added frontend agent:ui Issues found by Bot-UI agent labels Nov 4, 2026

vercel Bot deployed to Preview November 4, 2026 06:40

View deployment

claude added 2 commits November 4, 2026 07:11

PetrefiedThunder changed the title ~~docs: technical debt audit, remediation PRD, and WI-01 schema-channel decision (2026-10-28)~~ chore: technical debt remediation — audit, PRD, and work items WI-01…WI-15 (Kafka backbone retired) Nov 4, 2026

vercel Bot deployed to Preview November 4, 2026 07:15

View deployment

claude added 2 commits November 4, 2026 07:26

vercel Bot deployed to Preview November 4, 2026 07:30

View deployment

vercel Bot deployed to Preview November 4, 2026 07:39

View deployment

vercel Bot deployed to Preview November 4, 2026 07:42

View deployment

claude added 2 commits November 4, 2026 07:48

vercel Bot deployed to Preview November 4, 2026 07:50

View deployment

vercel Bot deployed to Preview November 4, 2026 08:12

View deployment

PetrefiedThunder force-pushed the claude/fervent-clarke-0aw2am branch from 9ccc935 to 1e06d69 Compare November 5, 2026 06:15

vercel Bot deployed to Preview November 5, 2026 06:17

View deployment

claude added 2 commits November 5, 2026 06:18

chore: merge origin/main into claude/fervent-clarke-0aw2am

af4b437

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

vercel Bot deployed to Preview November 5, 2026 06:22

View deployment

chore: merge origin/main into claude/fervent-clarke-0aw2am

8fe9391

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyhDYGPjbfDzVjRUqkvz4B

vercel Bot deployed to Preview November 5, 2026 06:46

View deployment

Navigation Menu

Repository navigation

chore: technical debt remediation — audit, PRD, and work items WI-01…WI-15 (Kafka backbone retired) - #3699

chore: technical debt remediation — audit, PRD, and work items WI-01…WI-15 (Kafka backbone retired)#3699
PetrefiedThunder wants to merge 31 commits into
mainfrom
claude/fervent-clarke-0aw2am

PetrefiedThunder commented Oct 28, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 28, 2026 •

edited

Loading

Draft PR not reviewed

Uh oh!

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

github-actions Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

PetrefiedThunder commented Nov 5, 2026 •

edited

Loading

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

PetrefiedThunder commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Test Plan

Checklist

Still needs Christopher

Uh oh!

vercel Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Draft PR not reviewed

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Blocker Checklist

Status: Ready

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

PetrefiedThunder commented Oct 28, 2026

Uh oh!

github-actions Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Bundle Size Report

Uh oh!

github-actions Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PetrefiedThunder commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

PetrefiedThunder commented Oct 28, 2026 •

edited

Loading

vercel Bot commented Oct 28, 2026 •

edited

Loading

coderabbitai Bot commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Nov 4, 2026 •

edited

Loading

github-actions Bot commented Nov 4, 2026 •

edited

Loading

PetrefiedThunder commented Nov 5, 2026 •

edited

Loading