feat(mcp): reduce PII in $mcp_intent (redaction + stronger prompt) - #4783
feat(mcp): reduce PII in $mcp_intent (redaction + stronger prompt)#4783gesh wants to merge 6 commits into
Conversation
$mcp_intent is agent-narrated free text the calling LLM writes into the injected `context` argument, so it could carry personal data a model read aloud despite the "no personal data" instruction. Add an always-on redactor that strips well-defined structured identifiers — emails, phone numbers, IPv4/IPv6 addresses, Luhn-valid card numbers, and US SSNs — from the intent before capture. Scoped to the intent only: structured tool arguments and responses keep the same shapes since they are often legitimate data there. Best-effort for those patterns rather than free-form names or addresses, which regex cannot catch without over-redacting prose. `context: false` and `beforeSend` remain the ways to drop the field entirely. Generated-By: PostHog Desktop Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
Rewrite DEFAULT_CONTEXT_PARAMETER_DESCRIPTION so agents are less likely to
write personal data into the injected `context` argument that becomes
$mcp_intent. The privacy rule is now explicit and mandatory, names the
identifiers to avoid (real names, emails, phone numbers, IPs, IDs,
credentials, secrets, copied user/tool text), and tells the agent to refer
to people and accounts by role ('a user', 'the customer') rather than by
identity. The example now models that role-based phrasing.
Pairs with the always-on intent PII redaction as a second, prompt-level
layer. Tests reference the constant symbolically, so no assertions change.
Generated-By: PostHog Desktop
Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
|
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
Review of the redactor surfaced real gaps, now fixed: - ReDoS: the email pattern used open-ended `+` quantifiers, quadratic on a long local-part run with no valid TLD. `$mcp_intent` is attacker-influenceable free text seen before truncation, so a pathological intent could stall the event loop (~13s at 80k chars). Bound the quantifiers to RFC-ish limits; 80k chars now runs in ~26ms. - Under-redaction: normalize horizontal Unicode spaces (NBSP etc.) so copy-pasted identifiers still match; accept dot/slash-grouped card numbers (Luhn-gated, so no new false positives) which previously leaked or produced a garbled partial redaction; accept space/dot-separated SSNs. A bare 9-digit number is still not treated as an SSN, to avoid eating numeric IDs. Adds tests for each. Full @posthog/mcp suite (699) and oxlint pass. Generated-By: PostHog Desktop Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
|
Note 🤖 Automated comment by QA Swarm — not written by a human Multi-perspective review across two rounds. Findings verified with deterministic reproductions, then fixed. Verdict: 💬 APPROVE WITH NITS (round 2 @ a764145)Redaction is correctly scoped to Fixed
Remaining best-effort limitations (by design)
Reviewers
Automated by QA Swarm — not a human review |
Fold the near-identical card/phone `replace` callbacks into one `redactConfirmedCandidate` helper (strip digits, range-check, confirm). The phone grouping check reuses the digit-count instead of a second regex scan, and the digit-strip regex is hoisted to a module constant. Pure refactor; all 699 tests and oxlint pass unchanged. Generated-By: PostHog Desktop Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
From PR review (marandaneto, greptile): - IPv6: `2001:db8::` (trailing `::`) leaked, and hex-looking C++ scope like `std::bad` was over-redacted. Rework the pattern into four bounded forms with a `(?<![\w:])` boundary on the compressed branches: trailing/middle/leading `::` now redact, while `std::bad` (left side is not a valid hex group) is left intact. - Phone: replace the "any 10-15 digits with a separator" heuristic — which over-redacted dates (`2024-01-15 12:30`) and dotted versions, missed slash-grouped numbers, and left a trailing digit on long international numbers — with two structural patterns: a NANP 3-3-4 grouping requiring a real separator, and an international number that must start with `+` and a country code. Dates, versions, bare digit runs, and order IDs are now left intact. Convert the redactPii cases to table-driven `it.each` per the suite's style. Full @posthog/mcp suite (711) and oxlint pass. Generated-By: PostHog Desktop Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
Review follow-up (veria-ai): the structural NANP pattern required a separator after the area code, so `(415)555-0142` (parens, no following separator) leaked. Allow the parenthesized `(415)` form to omit the separator while the bare `415` form still requires one — so bare digit runs are still never taken for phones. Test added; full @posthog/mcp suite (712) and oxlint pass. Generated-By: PostHog Desktop Task-Id: fc71ed52-e03a-4f5b-a863-acf3b23aa472
marandaneto
left a comment
There was a problem hiding this comment.
left one last comment otherwise lgtm
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
| // the digits decides — a non-card digit run of the same length is left intact. | ||
| result = result.replace(CREDIT_CARD_CANDIDATE_PATTERN, (match) => { | ||
| const digits = match.replace(/[ ./-]/g, '') | ||
| return digits.length >= 13 && digits.length <= 19 && passesLuhn(digits) ? REDACTED_VALUE : match |
There was a problem hiding this comment.
blocking: Adjacent expiry digits prevent card redaction — For Charging card 4111 1111 1111 1111 12/30 for renewal., the greedy candidate includes the expiry month. The combined number fails Luhn, so the entire match is returned unchanged, exposing the complete valid card in $mcp_intent. Recognize the valid card without absorbing adjacent numeric fields. Reproduction: reproduced — the focused pi-review-card.test.ts Vitest regression failed on this head with the complete card retained; the same card without expiry passed, and a temporary 16-digit candidate control made both cases pass.
Problem
$mcp_intentis free text the calling agent writes into the injectedcontextargument. Agents sometimes narrate personal data (emails, phone numbers, names) into it, and the SDK previously stored it almost verbatim — the only cleanup removed PostHog tokens and large binary blobs, and the "no personal data" instruction was prompt-only, not enforced.Changes
Two layers, both scoped to
@posthog/mcp:redactPiiinmcp-payloads.ts, applied touserIntentinsanitization.ts): strips structured identifiers — emails, phone numbers, IPv4/IPv6, Luhn-valid card numbers, US SSNs — from$mcp_intentbefore capture. Covers both theinstrument()andPostHogMCPpaths via the sharedsanitizeEvent. Scoped to the intent only; structured toolarguments/results are untouched, since the same shapes are often legitimate there.contextprompt (DEFAULT_CONTEXT_PARAMETER_DESCRIPTION): now forbids repeating/paraphrasing/inferring personal data and tells the agent to generalize entities to roles ("a user", "the customer") — covering names, which regex cannot. Tests reference the constant symbolically, so no assertions changed.Unit tests added for both. All 694
@posthog/mcpunit tests pass;oxlintclean.Libraries affected
Checklist
$mcp_intentcontent)pnpm changesetto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Built with PostHog Desktop (Claude Opus). The human driver chose the scope (redaction limited to the intent, always-on, no config flag), the structured-identifier pattern set, and the final wording of the default prompt. Names/addresses were deliberately left to the prompt layer rather than regex, to avoid over-redacting ordinary prose.
tsc/rslib buildOOM locally in this environment (pre-existing, unrelated to the diff); verified via the vitest suite and oxlint instead.Created with PostHog Desktop