Skip to content

Faster request counts, cancel abandoned page reads, and a Background work setting - #85

Open
TheMightyTRex wants to merge 2 commits into
logabell:mainfrom
TheMightyTRex:contrib/speed-and-background-work
Open

TheMightyTRex wants to merge 2 commits into
logabell:mainfrom
TheMightyTRex:contrib/speed-and-background-work

Conversation

@TheMightyTRex

Copy link
Copy Markdown

The problem

On my server (about 1,500 requests, 52,000 operations, a modest home server), Dewarr used a lot of CPU and database time while idle, and some pages were very slow:

  • Request counts took about 4 minutes. The counts behind Activity look up each request's latest operation by the request id inside payload['command'], and by kind and status. Neither had an index, so every lookup read the whole operations table: 2.1 million full scans since the database started. Repeated polling used up every database connection, and Dewarr stopped responding.
  • Postgres JIT cost more than it saved: 17 seconds of compiling on one count query.
  • Leaving a slow page kept its query running to the end, so moving back and forth piled the queries up.
  • Scheduled jobs run every 10 seconds to 15 minutes, which is right for a busy install but too much for a small server that also does other work. I had been patching the cron schedules at container start to slow them down (related to High idle CPU from periodic jobs — make schedules configurable #83).

The changes

  1. Two indexes on operations (migration 0081_operation_lookups):

    • (kind, status, created_at)
    • ((payload['command'] ->> 'intent_id'), kind, created_at), written with subscripts exactly as the app's queries compile, or Postgres ignores it.

    On a copy of my database, the request counts went from 4 min 2 s to 0.26 s, with identical results.

  2. JIT off for the app's database connections (-cjit=off in the connect options).

  3. Abandoned page reads are cancelled. A small ASGI middleware cancels GET and HEAD requests on the requests, following and library pages when the browser disconnects before the response starts. Cancelling the task makes psycopg cancel the running query. It never cancels writes, reads that call outside services (MAM, Prowlarr, Hardcover), or clean-up after a response has started.

  4. Settings → Background work (admins): Normal, Relaxed (×5), Light (×20) or Minimal (×50).

    • Normal is the default and keeps every schedule exactly as it is.
    • Each scheduled job keeps its cron schedule and skips the runs that are not due at the chosen pace, decided from the scheduled timestamp alone. Every worker agrees without extra state, and a new pace applies within a minute, without a restart.
    • Stored in a one-row background_work_settings table (migration 0082_background_work), like onboarding_settings.
    • The settings page says "New in this version" and explains what each pace delays (at Minimal, a finished download can take up to an hour to import).

Database changes and how to undo them

Migration Change Downgrade
0081_operation_lookups Two indexes on operations Drops them
0082_background_work background_work_settings table Drops it

Both run on start like every Dewarr migration, and nobody needs to do anything after updating. To go back: alembic downgrade 0080_member_setup. Both directions are tested, including on a real Dewarr v0.3.9 install.

Tests

  • New: tests/unit/test_background_pace.py, tests/integration/test_background_work.py (API, admin only, both migrations down and up, the planner uses the new index), tests/unit/test_disconnect.py, apps/web/tests/ui/background-work.spec.ts.
  • Full suites: server 4,668 passed and 6 skipped; mocked browser journeys 148 passed; live browser journeys 41 passed (downloader-simple timed out once while the server suite ran alongside, then passed twice alone).
  • Upgrade and downgrade on a real install: Dewarr v0.3.9 → this branch (both migrations, pace set to Light, no errors over a minute and a half) → alembic downgrade 0080_member_setup → Dewarr v0.3.9 starts with no errors.

🤖 Generated with Claude Code

TheMightyTRex and others added 2 commits October 31, 2026 10:18
Leaving a page while a slow request is still running kept its database query busy until it finished,
and moving back and forth piled the queries up. Reads (GET and HEAD) on the requests, following and
library pages are now cancelled when the browser disconnects before the response starts; the
database driver then cancels the running query. Writes, reads that call outside services, and
clean-up after a response always run to the end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Request counts and background jobs looked operations up by kind and status, and by the request id
inside the command payload, with no index for either, so each lookup read the whole operations table.
On a library with 52,000 operations the request counts took about 4 minutes; with the two indexes in
migration 0081 they take under a second. JIT is turned off for the app's connections, because
compiling the large status queries cost more than it saved (17 seconds on one count).

Settings -> Background work chooses how often scheduled jobs do their work: Normal (unchanged),
Relaxed (x5), Light (x20) or Minimal (x50). Each job keeps its schedule and skips the runs that are
not due at the chosen pace, so the setting applies within a minute without a restart. Migration 0082
stores it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant