Skip to content

Job options, worker limits, queue commands, fake fixes - #18

Merged
necessarylion merged 1 commit into
mainfrom
low-fixes
Nov 6, 2026
Merged

necessarylion merged 1 commit into
mainfrom
low-fixes

Conversation

@mydoxie

@mydoxie mydoxie commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Low items from todo.md for dox-queue. All changes are opt-in. Jobs and rows from the old version work as before.

  • [56] dox-queue#17 nits: done.
    • The fake said "a UnusedJob". Now it says "an UnusedJob", and still "a UniJob" and "a UserJob".
    • When Mail.fake() was on before Queue.fake(), Queue.restore() lost a Mail.useQueue() queue. Now Queue.fake() leaves the mail queue alone while Mail.fake() is on.
    • The _a doc comment now sits on _a.
    • The tests reset Mail.useQueue(null) with addTearDown, so a failed expect does not leak it.
  • [180] job options: mostly done.
    • New opt-in job settings: retryUntil, maxExceptions and failOnTimeout. retryUntil is read once at dispatch and saved in the payload. The exception count is saved in the payload too. Both keys are left out when not set, so old rows decode as before.
    • New dispatch(delayUntil: DateTime). delay: Duration still works. Giving both throws an ArgumentError.
    • New dispatchIf and dispatchUnless.
    • Skipped dispatchAfterResponse. dox_core has no after-response hook that a job can reach. It only has route-level TerminableMiddleware, and the job has no handle on the current request.
  • [181] worker: done.
    • New flags: --backoff, --max-jobs, --max-time and --memory.
    • New commands in bin/queue.dart: clear [queue], restart and prune-failed --hours=N.
    • restart writes a time to the app cache. A running worker reads that cache key every 3 seconds and stops after its current jobs when the value changes.
    • New QueueDriver.clear() and QueueDriver.pruneFailed(). The default versions throw UnsupportedError, so a custom driver that extends QueueDriver still compiles.

Note: QueueDriver.release() now also saves job.payload. This keeps the exception count. A custom driver that ignores it still works, but maxExceptions then never reaches its limit.

Follow-up in other repos:

  • dox-cli needs queue:clear, queue:restart and queue:prune-failed entries in command_registry.dart.
  • The docs page dox/docs/digging-deeper/queue.md needs the same lines as the README.

- Jobs: retryUntil, maxExceptions, failOnTimeout. dispatch(delayUntil:),
  dispatchIf, dispatchUnless.
- Worker: --backoff, --max-jobs, --max-time, --memory.
- Commands: clear, restart, prune-failed.
- QueueFake: 'an UnusedJob'; keep Mail.useQueue() when Mail.fake() was on.
@mydoxie

mydoxie commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Bot review

Blocking: none.

Checked: old rows decode the same (new keys are left out when not set). A job without the new options takes the same path as before. A custom driver that extends QueueDriver still compiles, and its old release() override still works. --max-jobs, --max-time and --memory only call stop(), so the running jobs end and no job is lost. restart reads the cache value when the worker starts, so a worker started after restart keeps running. The SQL in clear and pruneFailed uses parameters. failed_at is in milliseconds, so prune-failed compares the right unit. retryUntil is saved as epoch milliseconds, so the time zone does not matter.

Non-blocking:

  1. retry keeps the old exceptions count and the old retryUntil (withAttempts(0) copies them). A retried job with maxExceptions: 3 fails at its first error. A retried job past its retryUntil gets only one try. Consider resetting exceptions to 0 on retry (database_queue_driver.dart:164, redis_queue_driver.dart:145).
  2. clear also deletes reserved (running) jobs. This is the same as Laravel, and the doc comment says so. The README only says "delete the jobs of a queue". If a running job then fails, Redis release() puts it back in the delayed set, and the database driver adds a failed row. A short README note would help.
  3. No test checks that a worker started after restart keeps running. The code is right. A test would guard it.
  4. A class that implements QueueDriver (not extends) must now add clear and pruneFailed. This is rare. A CHANGELOG note is enough.
  5. Old flaky test, not from this PR: queue_mail_test "redis encrypt: true: a send error shows no mail data" failed in 1 of 3 local full runs. The 3-letter secret Ann can show up by chance in the random base64 text.

Checks: dart format clean. dart analyze --fatal-infos clean. dart test with Postgres, MariaDB and Redis: 245 passed (one earlier run hit the old flaky test in point 5; it passed 6 more times on its own). CI test: pass.

@necessarylion
necessarylion merged commit 3d4de23 into main Nov 6, 2026
1 check passed
@necessarylion
necessarylion deleted the low-fixes branch November 6, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants