Skip to content

feat(crypto+store): single AAD-bound enc:v1 format + ULID identifiers everywhere (spec 20)#513

Merged
PaulDotterer merged 9 commits into
mainfrom
feat/spec-20-crypto-one-format
Jul 4, 2026
Merged

feat(crypto+store): single AAD-bound enc:v1 format + ULID identifiers everywhere (spec 20)#513
PaulDotterer merged 9 commits into
mainfrom
feat/spec-20-crypto-one-format

Conversation

@PaulDotterer

@PaulDotterer PaulDotterer commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Closes #504. Implements spec 20 — the last spec-19 prerequisite (after #511/#512). Spec 19's per-user DEK envelope wraps under this single format.

Stage 1 — one at-rest format (F-06, the WS10 deferral)

  • internal/crypto ships ONE format: AAD-bound AES-256-GCM under enc:v1. The naked nil-AAD Encrypt/Decrypt pair is deleted; EncryptWithContext refuses an empty AAD at runtime; DecryptWithContext opens only enc:v1 and errors loudly on any other enc:* tag (reporting only the format tag, never ciphertext) — a retired enc:v2 blob must never silently pass through as plaintext. Plain non-prefixed values still pass through (pre-encryption plaintext + empty-string identity).
  • New RowAAD(rowID, purpose) + shared Purpose* constants: IdP client secrets bind idp_id|idp-client-secret (create/update + both SSO decrypts), TOTP secrets bind user_id|totp-secret (setup + both verify paths). LPS/LUKS keep SecretAAD. testutil factories seed with the same AADs as the read paths.
  • Guard TestAEADStaysInCryptoPackage: AEAD Seal/Open (at AEAD arity) forbidden outside internal/crypto; literal-nil AAD forbidden at *WithContext call sites.

Stage 2 — ULID identifiers (F-15)

  • events.id, lps_passwords_projection.id, luks_keys_projection.id, luks_tokens.id, security_alerts_projection.event_id: uuid → text ULID. gen_random_uuid() defaults removed — the DB never mints a random identifier.
  • AppendEvent/AppendEventWithVersion mint the event ULID in Go; secret-history projection rows take the rotating event's ULID as their id and created_at from the event's occurred_at — so a rebuild reproduces them byte-identically. The extended full-fidelity fixture (lps password, luks key, security alert + ack) proves it; before this change those ids/timestamps were silently non-deterministic under replay (invisible only because the fixture left those tables empty).
  • SecurityAlertAckParamsFromEvent validates alert_id as a ULID — preserving the fail-loud contract of the retired ::uuid cast, pinned by the existing malformed-alert-id test.
  • google/uuid is gone from all first-party code (only an indirect module dep remains). Guard TestULIDOnlyIdentifiers: no google/uuid import anywhere (generated code included, so a uuid column can't sneak back via sqlc), no uuid column type / gen_random_uuid() in migrations. Empty allowlist — red-checked at 13 findings before the migration.
  • ADR 0030 records both decisions (supersedes ADR 0009's dual-version framing).

Migration stance

Beta Path A (per the spec): reprovision; no dual-read shim, no backfill. Pre-change ciphertext fails decryption with an error that names the retired tag; pre-change uuid rows are discarded with the reprovision.

Verification

  • go vet, staticcheck, gofmt, docref check clean; full go test ./... -count=1: 33 packages ok, 0 failures.
  • Red-checks: prefix pin flipped → format test fails; ULID guard was red (13 findings) before the migration; wrong-AAD / cross-row / cross-purpose / retired-format / tampered paths unit-tested.
  • Local coderabbit review on the rebased branch: no findings.

Summary by CodeRabbit

  • New Features

    • Introduced stricter at-rest encryption handling with context-bound secrets, improving protection for identity provider, SSO, and TOTP data.
    • Standardized record identifiers to ULIDs across stored events and related projections.
  • Bug Fixes

    • Fixed audit and search reindexing to use the stored event ID consistently.
    • Improved rebuild consistency so projection data and timestamps can be reproduced byte-for-byte during recovery.
  • Tests

    • Expanded coverage for encryption format validation, identifier rules, and deterministic replay behavior.

…ed (#504)

Spec 20 stage 1 / F-06 (WS10 deferral): collapse the two at-rest
formats (nil-AAD enc:v1 + AAD-bound enc:v2) into ONE AAD-bound
AES-256-GCM format under enc:v1.

- crypto: Encrypt/Decrypt (nil-AAD) deleted; EncryptWithContext refuses
  an empty AAD; DecryptWithContext accepts only enc:v1 (+ plaintext
  passthrough) and errors LOUDLY on any other enc:* tag — a retired
  enc:v2 blob must never silently pass through as plaintext. New
  RowAAD(rowID, purpose) + shared Purpose* constants so write and read
  paths cannot drift.
- IdP client secrets bind to RowAAD(provider_id, idp-client-secret)
  (create/update handlers, both SSO decrypt sites); TOTP secrets bind
  to RowAAD(user_id, totp-secret) (setup + both verify decrypts);
  testutil factories seed with the same AADs.
- New archtest TestAEADStaysInCryptoPackage: AEAD Seal/Open (at AEAD
  arity) forbidden outside internal/crypto, literal-nil AAD forbidden
  at *WithContext call sites — the compile-time F-06 anti-regression.
- lps_sealed_transport test updated to the enc:v1 prefix.

Red-checked: prefix pin fails when flipped back to v2; retired-format
and wrong-AAD paths covered by unit tests. Beta Path A: pre-change
ciphertext requires reprovision (documented in the error itself).
…c regenerate + all caller fixes (spec 20, F-15)
…ID, luks tokens Go-minted; ack validates alert_id (spec 20, F-15)

- lps_passwords/luks_keys projection rows take the rotating event's
  ULID as id (deterministic under replay — a rebuild reproduces the
  same row ids); luks_tokens mint a ULID at creation.
- SecurityAlertAckParamsFromEvent validates alert_id as a ULID again —
  the fail-loud contract the retired ::uuid cast enforced and the
  malformed-alert-id test pins (fixture corrected to a 26-char ULID).
- google/uuid is gone from all first-party code; only an indirect
  module dependency remains.
…ty fixture covers them; ADR 0030 (#504)

The extended full-fidelity fixture (lps password, luks key, security
alert + ack) caught the next replay-infidelity layer after the ULID
ids: created_at DEFAULT now() re-minted on rebuild. The appliers now
supply created_at from the event's occurred_at, so the rows reproduce
byte-identically. ADR 0030 records the single-format + ULID decision
(supersedes ADR 0009's dual-version framing).
@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@PaulDotterer, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b68ce7e7-1f30-4403-8ad2-6e2efbf2fbfe

📥 Commits

Reviewing files that changed from the base of the PR and between 40b9019 and 5af7e31.

📒 Files selected for processing (1)
  • internal/archtest/ulid_only_test.go
📝 Walkthrough

Walkthrough

This PR unifies at-rest encryption into a single AAD-bound enc:v1 format with row/purpose-scoped AAD (removing the nil-AAD API), and migrates event/projection identifiers from UUID to Go-minted ULID strings across schema migrations, SQL queries, store/projector code, API handlers, and tests, adding architecture guard tests for both changes.

Changes

AAD-bound Encryption Format

Layer / File(s) Summary
Single enc:v1 format core
internal/crypto/crypto.go, internal/crypto/crypto_test.go, docs/adr/0030-...md
Replaces dual-prefix encryption with a single enc:v1: format, adds RowAAD/purpose constants, fail-closed EncryptWithContext, strict DecryptWithContext rejecting retired tags, and an ADR documenting the design.
Row-scoped AAD wiring
internal/api/idp_handler.go, internal/api/sso_handler.go, internal/api/totp_handler.go, internal/testutil/factories_idp.go, internal/api/lps_sealed_transport_test.go
IdP, SSO, and TOTP handlers and test factories now encrypt/decrypt via EncryptWithContext/DecryptWithContext with row/purpose-bound AAD; a sealed-transport test now expects enc:v1:.
Architecture guard tests
internal/archtest/aead_containment_test.go, internal/archtest/ulid_only_test.go
New AST-based tests enforce AEAD calls stay in the crypto package with non-nil AAD, and enforce no google/uuid imports or UUID-typed migration columns.

ULID-only Identifiers

Layer / File(s) Summary
Schema and query updates
internal/store/migrations/002_event_store.sql, internal/store/migrations/004_devices.sql, internal/store/queries/*.sql, go.mod
Changes events.id and projection id/event_id columns from uuid to text, updates insert queries to bind explicit id/created_at, and moves google/uuid to an indirect dependency.
ULID minting in store/repositories
internal/store/store.go, internal/store/postgres/luks.go, internal/store/postgres/lps.go, internal/store/luks.go
Store.AppendEvent* and LUKS token creation mint ULIDs via ulid.Make().String(); LuksKey/LuksToken ID fields change from uuid.UUID to string.
Projector persistence and parsing
internal/projectors/lps_password_listener.go, internal/projectors/luks_key_listener.go, internal/projectors/security_alert.go, internal/projectors/security_alert_test.go
Projectors now insert event-derived ID/CreatedAt, and security alert acknowledgement parses alert_id as ULID instead of UUID.
API/search read-path updates
cmd/control/valkey.go, internal/api/audit_handler.go, internal/search/index.go
Audit reindexing, valkey listener, and search warm-rebuild use raw event ID strings instead of ULID-formatting a stored UUID.
Projector test fixtures
internal/projectors/*_test.go
Numerous test files switch PersistedEvent.ID and fixture identifiers from uuid.New()/uuid.NewString() to ulid.Make().String().
Rebuild fidelity coverage
internal/store/rebuild_fidelity_test.go
Extends the seed fixture with LPS password, LUKS key, and security alert events to verify byte-identical rebuild of new ULID-derived rows.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the two main changes: the new AAD-bound enc:v1 crypto format and the ULID identifier migration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/spec-20-crypto-one-format

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/archtest/ulid_only_test.go`:
- Around line 56-84: The migration scan in ulid_only_test.go ignores the result
of f.Close(), which triggers errcheck. Update the file-handling logic around
os.Open, bufio.NewScanner, and sc.Err() so the close error is checked and
reported with t.Fatalf or otherwise handled consistently with the existing
open/scan error paths.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 12b2fc80-4d87-412b-96ec-4097f3cf904a

📥 Commits

Reviewing files that changed from the base of the PR and between 3846eda and 40b9019.

⛔ Files ignored due to path filters (6)
  • go.sum is excluded by !**/*.sum
  • internal/store/generated/events.sql.go is excluded by !**/generated/**
  • internal/store/generated/lps.sql.go is excluded by !**/generated/**
  • internal/store/generated/luks.sql.go is excluded by !**/generated/**
  • internal/store/generated/models.go is excluded by !**/generated/**
  • internal/store/generated/security_alerts.sql.go is excluded by !**/generated/**
📒 Files selected for processing (43)
  • cmd/control/valkey.go
  • docs/adr/0030-single-aad-at-rest-format-ulid-ids.md
  • go.mod
  • internal/api/audit_handler.go
  • internal/api/idp_handler.go
  • internal/api/lps_sealed_transport_test.go
  • internal/api/sso_handler.go
  • internal/api/totp_handler.go
  • internal/archtest/aead_containment_test.go
  • internal/archtest/ulid_only_test.go
  • internal/crypto/crypto.go
  • internal/crypto/crypto_test.go
  • internal/projectors/action_set_test.go
  • internal/projectors/action_test.go
  • internal/projectors/assignment_test.go
  • internal/projectors/compliance_policy_test.go
  • internal/projectors/compliance_test.go
  • internal/projectors/definition_test.go
  • internal/projectors/device_group_test.go
  • internal/projectors/device_test.go
  • internal/projectors/execution_test.go
  • internal/projectors/identity_provider_test.go
  • internal/projectors/lps_password_listener.go
  • internal/projectors/luks_key_listener.go
  • internal/projectors/role_test.go
  • internal/projectors/security_alert.go
  • internal/projectors/security_alert_test.go
  • internal/projectors/totp_test.go
  • internal/projectors/user_group_test.go
  • internal/projectors/user_test.go
  • internal/search/index.go
  • internal/store/luks.go
  • internal/store/migrations/002_event_store.sql
  • internal/store/migrations/004_devices.sql
  • internal/store/postgres/lps.go
  • internal/store/postgres/luks.go
  • internal/store/queries/events.sql
  • internal/store/queries/lps.sql
  • internal/store/queries/luks.sql
  • internal/store/queries/security_alerts.sql
  • internal/store/rebuild_fidelity_test.go
  • internal/store/store.go
  • internal/testutil/factories_idp.go
💤 Files with no reviewable changes (1)
  • internal/projectors/totp_test.go

Comment thread internal/archtest/ulid_only_test.go
@PaulDotterer PaulDotterer merged commit 16a1150 into main Jul 4, 2026
11 checks passed
@PaulDotterer PaulDotterer deleted the feat/spec-20-crypto-one-format branch July 4, 2026 19:56
PaulDotterer added a commit that referenced this pull request Jul 13, 2026
Spec 20 (#513) switched identifiers from uuid to text ULIDs by editing
the already-shipped 002/004 migration files in place rather than adding
a forward migration. goose tracks applied migrations by version number,
not content, so a database created on v2026.07 (which recorded 002/004
as applied with uuid columns) never re-runs them — it keeps the uuid
columns while the code now mints ULID strings, and the first AppendEvent
after upgrade fails with 22P02 (invalid input syntax for type uuid).

Migration 016 introspects and converts every remaining uuid-typed column
in the public schema to text, working around the single
security_alerts_projection.event_id -> events.id FK. Idempotent: a no-op
on fresh v2026.08 databases where the columns are already text, verified
via goose up (version 16) and a second-run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

crypto: collapse at-rest encryption to a single enc:v1 AAD-bound format (finishes WS10 / F-06)

1 participant