Skip to content

fix(reports): correct the Security & Compliance Posture report#2518

Merged
ToddHebebrand merged 46 commits into
mainfrom
ToddHebebrand/Security-Compliance-Posture-report
Jul 16, 2026
Merged

fix(reports): correct the Security & Compliance Posture report#2518
ToddHebebrand merged 46 commits into
mainfrom
ToddHebebrand/Security-Compliance-Posture-report

Conversation

@ToddHebebrand

Copy link
Copy Markdown
Collaborator

The headline: every posture report has been reporting 0 CVEs

Every Security & Compliance Posture report Breeze has generated reported 0 critical / 0 high vulnerabilities, regardless of actual exposure — on a document whose stated purpose is filling out cyber-insurance applications. Two independent causes, either sufficient alone:

  1. RLS context. vulnerabilities is FORCE ROW LEVEL SECURITY with a system-only policy (2026-06-22-vulnerability-management.sql:104-115). The old code .innerJoined it from the org-scoped request context, so the join matched nothing. A silent zero-row read.
  2. Severity casing. NVD writes 'CRITICAL' (nvdClient.ts:136), MSRC writes 'Critical' (msrcClient.ts:51). The old comparison was v.severity === 'critical' — case-sensitive.

The fix keeps tenant-scoped findings in the request context and reads the global CVE catalog via runOutsideDbContext(() => withSystemDbAccessContext(...)), joining in memory — the pattern CLAUDE.md prescribes. Locked by a real-Postgres integration test seeding mixed-case severities and asserting cross-org isolation, in CI's blocking integration-test job.

Also in here

Posture report corrections

  • All detected security products now reach the inventory (not just Huntress/S1/DNS/backup/identity), deduped by device across sources.
  • Long inventories continue onto extra PDF pages instead of clipping.
  • New backupRequired option — see the caveat below.
  • antivirus product category added; category ranking is now keyed by the union so a new category is a build error, not a silent front-sort.

API test stability — route-graph reuse, metric/state resets without module reloads, the validator import guard moved from a test to an eslint rule, migration-ordering contract isolated, worker parallelism bounded.

Fixes found in review

  • ReportBuilder replaced config wholesale on save, so renaming a posture report silently reset backupRequired, includeCis, sites and every threshold to defaults. executive_summary had the same hole. Fixed via baseConfig, which also restores full editing for posture reports (they'd been narrowed to a lone checkbox to dodge the wipe).
  • Scheduled report failures were silent and unretried: BullMQ default single attempt, lastGeneratedAt stamped before generation, no notification. Now retried with backoff, with a failure email that deliberately omits the raw error (it reaches customer inboxes and can carry Zod/PG internals). The up-front stamp staysisDue() reads it, so stamping after success would re-find a failing report due every 5 minutes forever.
  • Redis-less inline path no longer lets one throwing report abort the whole due-batch.

Please look at this before merging

backupRequired defaults to false on every UI-created posture report, so a customer with no backup renders Backup: Not required with the "Configure backups" recommendation suppressed.

This is deliberate and load-bearing, not an oversight: Breeze can only see its own backup. backup_configs.provider is a storage target (local|s3|azure_blob|google_cloud); c2c_connections.provider is the source Breeze backs up (microsoft_365|google_workspace). Nothing represents "a third-party product protects this customer" — no Cove, Dropsuite, Veeam, Datto or Acronis anywhere in the repo. For any MSP not using Breeze's own backup the control would be permanently red, so defaulting it off was the only sane move.

Flipping it to true would make things worse, converting a misleading "Not required" into a flatly wrong "No".

Design for fixing this properly is parked at docs/superpowers/specs/2026-07-14-posture-backup-evidence-design.md, deliberately unimplemented — it would make Breeze a reporting plane for software it doesn't manage, and the same argument generalises to EDR, DNS filtering and MFA. That's a product-direction call, not a technical one.

Known wart: reports.mdx still justifies the off-default as suiting "workstation-oriented assessments". The report can't express that — the device query filters on org, sites and permissions, with no OS or device-type filter. The honest sentence is "Breeze currently detects only its own backup." Left as-is pending the decision above.

Verification

Gate Result
API suite 13,287 passed, 0 failed
Web suite 3,690 passed, 0 failed
Shared suite 1,211 passed, 0 failed
tsc --noEmit (api + shared) clean
eslint src clean
astro check 0 errors, 0 warnings
Docs build 146 pages, clean
Cold run (.vite cleared) green

Test-name diff across the whole API suite: 13,153 → 13,168 cases, three names gone, all accounted for — the eslint-replaced import guard (proven by probe to catch import, export … from and export *) and two renamed *.disabled tests that are now stronger. No coverage lost.

New guards are mutation-proven rather than assumed: removing the provider === 'other' guard, adding a schema key missing from the field map, reintroducing the 1 devices plural bug, and downgrading the em-dash each kill exactly one intended test.

Follow-up filed: #2517 (product active ||= OR-merge paints a product green off one RTP-on device — low severity, the metric grid contradicts it on the same page).

🤖 Generated with Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 15, 2026

Copy link
Copy Markdown

Deploying breeze with  Cloudflare Pages  Cloudflare Pages

Latest commit: c9c74e7
Status: ✅  Deploy successful!
Preview URL: https://e6c26d46.breeze-9te.pages.dev
Branch Preview URL: https://toddhebebrand-security-compl.breeze-9te.pages.dev

View logs

@ToddHebebrand
ToddHebebrand force-pushed the ToddHebebrand/Security-Compliance-Posture-report branch from 95fdebc to 2b0b354 Compare July 16, 2026 04:37
Todd Hebebrand added 28 commits July 15, 2026 22:53
Todd Hebebrand and others added 18 commits July 15, 2026 22:54
PUT /reports/:id replaces `config` wholesale, and ReportBuilder rebuilds
that object from its own state alone. Editing a report whose type carries
extra config therefore dropped it: renaming a posture report silently reset
backupRequired, includeCis, sites and every threshold to schema defaults on
the next generation. executive_summary had the same hole.

Pass the stored config in as `baseConfig` and spread it under the
reconstructed one, so keys the builder doesn't own survive while live
builder state still wins.

This lets the posture edit page go back to the full builder — it had been
narrowed to a lone backup checkbox to dodge the wipe, which cost name,
schedule, format and recipient editing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A failed scheduled report was silent end to end: jobs were enqueued with
BullMQ's default single attempt, lastGeneratedAt is stamped before
generation, and nothing told recipients. One transient blip cost the whole
occurrence — next run next month — and the only trace was a failed
report_runs row nobody watches.

Add attempts + exponential backoff so a blip no longer burns the
occurrence. The up-front lastGeneratedAt stamp stays: isDue() reads it, so
stamping after success would re-find a failing report due every check
interval forever. Retries live inside the job instead, where that stamp
doesn't reach.

Once the attempts are spent, email the recipients. The notice omits the
underlying error — it reaches customer inboxes and the raw message can
carry Zod issue arrays or PG internals; operators keep it on the run row
and in Sentry.

Also guard the Redis-less inline path, where an unhandled throw aborted the
due-batch loop and silently starved every remaining org's reports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CATEGORY_ORDER was an array read with indexOf(), which returns -1 for an
unranked member and silently sorts it to the front. Key it by
PostureProductCategory so a new category is a build error instead — the
'antivirus' member added on this branch had to be remembered by hand.

The generator skips provider 'other' because there is no pretty name for it,
so without the skip a client-facing insurance PDF ships a green-dotted
product literally called "other". The fixture already carries such a row but
nothing asserted on it; deleting the guard left the suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The posture config is declared twice: once with defaults for generation, once
without for persistence. Drift is silent and one-directional — a key added to
the generation schema but missed in the field map is stripped by zod on save,
then reappears at generation as its default, so the user's setting is quietly
ignored rather than rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extracting drawPostureProductRow retyped its separators as hyphens, leaving
the product inventory and the backup recommendation reading "-" while the
39 other rendered strings — including their neighbours in the same
Recommended actions list — still use an em-dash.

jsPDF encodes an em-dash correctly (WinAnsi 0x97), so the hyphens bought
nothing. What they did buy was a passing test: the helper reads the raw page
command strings as latin1, where the em-dash arrives as \x97 and never
matches '—'. Decode that range in the helper instead of downgrading the
document's punctuation to suit it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The backup-option rewrite dropped the sentence listing every other config
default, though all of them still apply — thresholds that decide pass/fail on
an insurance attestation, left undocumented for API callers.

Restore them, fold in backupRequired, and state the asymmetry plainly: the
template card opts out of required backup while an API call that omits the
field opts in. Only the UI default was documented, so a caller following the
docs got the opposite of what they read.

Also drop the note about vulnerability totals being "scoped through tenant
findings and the global CVE catalog" — that describes an RLS-context split
inside the generator and answers no question an MSP tech would ask.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The posture report can only see Breeze's own backup: backup_configs.provider is
a storage target, c2c_connections.provider is the source Breeze backs up. Nothing
represents "a third-party product protects this customer", so a customer covered
by Cove renders "Backup: No" in red on an insurance document.

That gap is also why the posture template defaults backupRequired to false — for
any MSP not using Breeze's own backup the control is permanently red. The default
is a workaround for missing evidence, not a preference, which is why flipping it
to true would make things worse rather than better.

Records the design reached before parking (org-scoped attestation, derived
relevance, evidence precedence) and the objection it parked on: this would make
Breeze a reporting plane for software it does not manage, and the same argument
generalises to EDR, DNS filtering and MFA. That is a product-direction call to
settle before any code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…15 baseline perms

buildAuthFromApiKey's org branch re-validates the key's stored scopes
(ai:read + ai:execute) against the creator's live permissions before
building an AuthContext. The mocked getUserPermissions returned an
empty permissions array, so the coarse re-clamp denied the request
with 403 before ever reaching the tools/list + tools/call dispatch
this test exercises.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ToddHebebrand
ToddHebebrand force-pushed the ToddHebebrand/Security-Compliance-Posture-report branch from 2b0b354 to c9c74e7 Compare July 16, 2026 04:55
@ToddHebebrand
ToddHebebrand merged commit 528e0f1 into main Jul 16, 2026
43 checks passed
@ToddHebebrand
ToddHebebrand deleted the ToddHebebrand/Security-Compliance-Posture-report branch July 16, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant