Skip to content

Fix mail integrity, send recovery, concurrency, and maintenance - #116

Merged
bermanto merged 4 commits into
mainfrom
fix/audit-reliability
Sep 5, 2026
Merged

Fix mail integrity, send recovery, concurrency, and maintenance#116
bermanto merged 4 commits into
mainfrom
fix/audit-reliability

Conversation

@bermanto

@bermanto bermanto commented Sep 5, 2026

Copy link
Copy Markdown
Member

Summary

Prevent incomplete inbound mail, duplicate sends after storage failures, lost draft edits, and removal of the last owner during concurrent requests. This applies the codebase audit fixes to main and adds the related performance and maintenance improvements.

  • Stage complete inbound messages and attachments before the D1 commit. Keep large plain-text bodies in R2 and honor validated Reply-To addresses.
  • Record send identity, provider acceptance, and recovery state. Retry identified sends without sending another copy. Protect drafts while delivery is pending, and wait for the database guards during updates.
  • Make draft version checks and owner protection atomic. Require the exact Origin for session writes and limit request bytes before parsing.
  • Give maintenance jobs leases, saved progress, queue continuation, page-wide reference checks, and default Trash cleanup. Remove expired auth and operation records in small batches.
  • Refresh all loaded inbox pages, use the draft journal after bootstrap, page draft rows, and reconcile a connected event socket every two minutes.
  • Load settings, contacts, agents, and the editor on demand. Omit message bodies from normal conversation queries. Separate send storage from message preparation and job progress from maintenance work.
  • Search every relevant zone page, patch vulnerable dependencies, add a recurring dependency audit, and verify recovery checkpoints against resource IDs, release state, Worker state, and mail object references.

Adds migration 0029_mail_reliability.sql, after-deploy migration 0004_mail_reliability_guards.sql, and release schema version 4. Fresh-install and populated-upgrade checks cover the migration sequence. Product version and release publication remain part of the normal release process.

Verification

  • pnpm check on Linux and Windows: 882 unit tests passed on Linux (2 Windows-only tests skipped), 884 on Windows; 214 Worker integration tests passed on both. Coverage, architecture, API checks, and production builds passed.
  • pnpm deploy:dry-run on Linux and Windows
  • Production dependency audit: zero reported advisories
  • Canonical documentation pnpm check
  • HQBase staging E2E for commit 7766795c35b15063b9fdd42281d4179a4fc0dcd1: provisioning recovery, populated migration upgrade and retry, eight browser checks, hostname move and return, populated backup/restore, and disposable resource cleanup all passed.

The app chunk fell from 902.00 kB to 355.00 kB (271.63 kB to 106.23 kB gzip). A local synthetic 10,000-message query took 19.4 ms with the narrower projection versus 74.1 ms with the wide projection. This is a local measurement, not a production latency claim.

The regression suite covers partial R2 and D1 failures, accepted-send recovery, uncertain-send protection, concurrent draft saves and owner changes, job leases and failed continuation publication, scans beyond 10,000 objects, large text bodies, request stream limits, stale inbox rows, draft paging, and missed socket wake-ups. Worker files previously excluded from unit coverage are now included; integration results remain separate evidence.

Notes

API callers need a saved draft or an idempotencyKey for a safe send retry. An uncertain provider result requires review; it is never sent again automatically. Full-text search covers the first 256 KiB of large plain-text bodies. Recovery checkpoints do not copy R2 objects; independent object backups are still required. New verified checkpoints use format v2, while older checkpoint files require manual recovery.

Canonical specifications and the operations guide are updated in HQBase/hqbase-site#49. Merge the paired changes together. No production deployment or release is included.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 109 files, which is 9 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: e5ce0bac-9ed8-4fa4-96fc-d2564283c59b

📥 Commits

Reviewing files that changed from the base of the PR and between 895757e and 7766795.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (109)
  • .github/workflows/dependency-audit.yml
  • .github/workflows/staging-e2e.yml
  • api/hqbase-mail-api-v1.openapi.json
  • api/hqbase-mail-api-v2.openapi.json
  • app/app.tsx
  • app/features/compose/compose-state.ts
  • app/features/drafts/api.ts
  • app/features/drafts/drafts-page.tsx
  • app/features/drafts/sync-client.ts
  • app/features/drafts/use-drafts.ts
  • app/features/events/use-mail-events.ts
  • app/features/messages/conversation-window.ts
  • app/features/messages/types.ts
  • app/features/messages/use-mail-sync.ts
  • app/workspace-pages.tsx
  • migrations-after-deploy/0004_mail_reliability_guards.sql
  • migrations/0029_mail_reliability.sql
  • package.json
  • playwright.config.ts
  • pnpm-workspace.yaml
  • scripts/hqbase/backup.mjs
  • scripts/hqbase/cli.mjs
  • scripts/hqbase/empty-r2.mjs
  • scripts/hqbase/recovery-verification.mjs
  • scripts/hqbase/reset-d1.sql
  • scripts/hqbase/restore.mjs
  • scripts/release/after-deploy-state.mjs
  • scripts/release/deploy.mjs
  • scripts/release/package.mjs
  • test/e2e/staging/event-socket.spec.ts
  • test/e2e/staging/lifecycle.spec.ts
  • test/e2e/staging/setup.ts
  • test/integration/worker/agent-mailboxes.test.ts
  • test/integration/worker/auth.test.ts
  • test/integration/worker/contacts.test.ts
  • test/integration/worker/conversations.test.ts
  • test/integration/worker/labels.test.ts
  • test/integration/worker/local-reset.test.ts
  • test/integration/worker/mail-api.test.ts
  • test/integration/worker/mail-reliability.test.ts
  • test/integration/worker/mcp.test.ts
  • test/integration/worker/oauth-connections.test.ts
  • test/integration/worker/signature-routes.test.ts
  • test/integration/worker/users.test.ts
  • test/unit/app/drafts/draft-list-paging.test.tsx
  • test/unit/app/drafts/sync-client.test.ts
  • test/unit/app/events/use-mail-events.test.tsx
  • test/unit/app/messages/use-mail-sync.test.tsx
  • test/unit/scripts/after-deploy-state.test.mjs
  • test/unit/scripts/d1-migrations.test.mjs
  • test/unit/scripts/recovery-objects.test.mjs
  • test/unit/scripts/recovery.test.mjs
  • test/unit/scripts/sql-migrations.test.mjs
  • test/unit/scripts/staging-workflow.test.mjs
  • test/unit/worker/auth/mailbox-access.test.ts
  • test/unit/worker/features/messages/conversation-scale.test.ts
  • test/unit/worker/features/messages/message-thread.test.ts
  • test/unit/worker/features/send/forward-service.test.ts
  • test/unit/worker/features/send/routes.test.ts
  • test/unit/worker/features/send/send-service.test.ts
  • test/unit/worker/features/updates/migration-state.test.ts
  • test/unit/worker/features/updates/service.test.ts
  • test/unit/worker/features/updates/zone-account.test.ts
  • test/unit/worker/jobs/consumer.test.ts
  • test/unit/worker/lib/request-body.test.ts
  • vitest.config.ts
  • worker/auth/mailbox-access.ts
  • worker/auth/session.ts
  • worker/db/drizzle.ts
  • worker/db/schema-core.ts
  • worker/db/schema-messages.ts
  • worker/db/schema-send.ts
  • worker/db/schema.ts
  • worker/email/parse-email.ts
  • worker/email/store-email.ts
  • worker/features/drafts/queries.ts
  • worker/features/drafts/routes.ts
  • worker/features/labels/queries.ts
  • worker/features/mcp/mail-tools.ts
  • worker/features/mcp/send-tools.ts
  • worker/features/messages/conversation-queries.ts
  • worker/features/messages/object-references.ts
  • worker/features/messages/queries.ts
  • worker/features/messages/routes.ts
  • worker/features/messages/storage.ts
  • worker/features/messages/text-storage.ts
  • worker/features/messages/types.ts
  • worker/features/operations/routes.ts
  • worker/features/send/attachment-storage.ts
  • worker/features/send/delivery.ts
  • worker/features/send/forward.ts
  • worker/features/send/operations.ts
  • worker/features/send/persistence.ts
  • worker/features/send/routes.ts
  • worker/features/send/service.ts
  • worker/features/send/validation.ts
  • worker/features/updates/migration-names.ts
  • worker/features/updates/migration-state.ts
  • worker/features/updates/service.ts
  • worker/features/updates/zone-account.ts
  • worker/features/users/queries.ts
  • worker/features/users/routes.ts
  • worker/jobs/consumer.ts
  • worker/jobs/maintenance.ts
  • worker/jobs/object-scan.ts
  • worker/lib/errors.ts
  • worker/lib/json.ts
  • worker/lib/request-body.ts
  • worker/routes/index.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@socket-security

socket-security Bot commented Sep 5, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updated@​tiptap/​extension-placeholder@​3.27.3 ⏵ 3.30.410010065 -4100 +1100
Updated@​tiptap/​starter-kit@​3.27.3 ⏵ 3.30.49910075100 +1100
Updated@​tiptap/​extension-image@​3.27.3 ⏵ 3.30.4100 +110075100 +1100
Updated@​tiptap/​react@​3.27.3 ⏵ 3.30.499 +110079100 +1100
Updatedsanitize-html@​2.17.6 ⏵ 2.17.7100 +1100 +210092100

View full report

@bermanto
bermanto deployed to hqbase-staging September 5, 2026 01:48 — with GitHub Actions Active
@bermanto
bermanto deployed to hqbase-staging September 5, 2026 01:57 — with GitHub Actions Active
@bermanto
bermanto marked this pull request as ready for review September 5, 2026 02:08
@bermanto
bermanto merged commit ddf1ac4 into main Sep 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant