feat(risk): add listRiskResultsForAgent for AI Insights with redacted match#2922
Conversation
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
🦋 Changeset detectedLatest commit: a0194d2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
64b8965 to
d0cb9bc
Compare
|
@claude review |
There was a problem hiding this comment.
Code review is billed via overage credits. To resume reviews, an organization admin can raise the monthly limit at claude.ai/admin-settings/claude-code.
Once credits are available, comment @claude review on this pull request to trigger a review.
🚀 Preview Environment (PR #2922)Preview URL: https://pr-2922.dev.getgram.ai
Gram Preview Bot |
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
d0cb9bc to
e725aff
Compare
e143315 to
adc37db
Compare
… match Adds a redacted variant of risk.results.list designed for MCP / AI assistant consumption. The raw `match` field is replaced with `match_redacted`, an opaque token of the form `<redacted len=N sha=XXXXXXXX>`, where the sha prefix is deterministic so agents can dedupe identical secrets across chats without ever seeing secret content. `shadow_mcp` findings pass `match` through verbatim because the value is a non-sensitive server identifier already shown unmasked in the dashboard. Also wires `<InsightsConfig>` overrides into the Security Overview and Policy Center pages with risk-aware suggestions, and extends the Insights system prompt with a hard rule against echoing `match_redacted` values. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's nolintlint flags //nolint suppressions that the underlying linter wouldn't have raised — gosec didn't actually warn on int32(len(match)) in the test fixture, so the directive is dead weight. Removing it satisfies nolintlint without changing test behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
adc37db to
971b943
Compare
Mixes the active organization ID into sha256(match) with a NUL byte separator so the redacted fingerprint can never correlate the same secret across organizations. Within an org the fingerprint stays deterministic so agents can still dedupe findings of the same secret across chats. Defense in depth: the endpoint is already gated by org:admin and the auth context is project-scoped, so cross-org reads aren't possible today. The salt removes the dependency on access-control correctness for the cross-org non-correlation property. Adds white-box tests for the helper (cross-org differs, same-org deterministic, shadow_mcp ignores salt, empty collapses, NUL separator prevents boundary ambiguity). The integration test environment hard- codes mockidp.MockOrgID across instances so the cross-org assertion has to live at the unit-test layer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
main's risk-overview-analytics PR added category, rule_id, unique_match, from, and to filter fields to listRiskResults. The agent endpoint constructs a ListRiskResultsPayload literal directly, so exhaustruct rightly flagged the new fields as missing. Add them to the agent endpoint's payload too (the LLM benefits from the same filters — and from/to in particular lets the agent honor the dashboard's selected date range) and thread them through. Also reverts mise.lock to main's version; the only diff was the autogenerated doc-URL header (mise.jdx.dev vs mise.en.dev) from a locally-stale mise install, which CI rebuilds clean and was failing git:porcelain. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Adds tools for the dashboard's AI Insights sidebar to reason about risk overview, policies, and findings — without ever loading raw secret content into the model context.
risk.results.listForAgentmirrorsrisk.results.listbut replacesmatchwithmatch_redacted, an opaque token<redacted len=N sha=XXXXXXXX>whereshaissha256(orgID || 0x00 || match)truncated to the first 8 hex chars. Per-org salting means identical secrets dedupe within an org but never correlate across orgs.shadow_mcpfindings passmatchthrough verbatim (server URL / stdio command — already shown unmasked in the dashboard).start_pos/end_posare coarsened to a singleposition_knownboolean to remove reconstruction signals.<InsightsConfig>is now mounted onSecurityOverviewandPolicyCenterwith risk-aware suggestion prompts and page-specificcontextInfo.match_redactedvalues.The existing
risk.results.listendpoint is unchanged — the dashboard UI still uses it for the click-to-reveal flow on the findings table.Why this shape
The safest of the options discussed: dedicated redacted endpoint over a
?redact=trueflag, so the LLM cannot opt out of redaction. Theshaprefix gives agents enough signal to dedupe within an org (e.g. "12 findings of the same OPENAI_KEY across 5 chats") without ever holding a secret in context. MixingorgIDinto the hash with a NUL separator is defense-in-depth: even if a future code path widens the endpoint surface beyond the current org-scoped access control, the fingerprints can never correlate the same secret across orgs.TODOs
speakeasy-team-gramMCP toolset (production Gram instance) to expose the new + safe-to-reuse risk tools. Include:listRiskResultsForAgent(new, redacted)listRiskResultsByChat(already safe — nomatchfield)listRiskPoliciesgetRiskPolicygetRiskCapabilitiesgetRiskPolicyStatuslistShadowMCPApprovalsapproveShadowMCP,revokeShadowMCPApproval,triggerRiskAnalysis,create/update/deleteRiskPolicy)./securityand/security/policies, try each suggestion prompt, confirm nomatch_redactedvalues appear in assistant output.Non-TODOs (deliberately deferred)
middleware.TraceMethods, and the Insights sidebar tags requests withX-Gram-Source: dashboard-ai-insights. Those two signals together answer "how often does the AI Insights feature use risk tools" better than a per-endpoint counter would. Revisit only if we want a hard SLI panel.orgIDdirectly, which prevents cross-org fingerprint correlation. A secret-keyed HMAC would additionally defend against an attacker with raw DB read access guessing fingerprints offline, but that threat model isn't in scope at this layer today. Revisit if redaction outputs ever flow to a less-trusted store.Test plan
mise gen:goa-server+mise gen:sdkregenerate cleanlymise build:server— greengo test ./server/internal/risk/...— all tests pass including 9 new ones:results_test.go):TestListRiskResultsForAgent_RedactsGitleaksMatch,_ShadowMCPPassthrough,_DeterministicFingerprintWithinOrg,_Unauthorizedredact_internal_test.go):TestRedactMatch_FingerprintDiffersAcrossOrgs,_FingerprintDeterministicWithinOrg,_ShadowMCPIgnoresSalt,_EmptyMatchCollapses,_NULSeparatorPreventsBoundaryAmbiguitycd client/sdk && pnpm build— greencd client/dashboard && pnpm tsc -p tsconfig.app.json --noEmit— 3 unrelated pre-existing errors (Table.Body / AppPortal JSX-component issues), zero new errors from this change/security-review): no findings (authz delegation verified, sha256 prefix non-reversible at the threat-model boundary, shadow_mcp passthrough matches existing dashboard exposure, redacted shape exposes no new side channels)🤖 Generated with Claude Code