You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Capture redaction corrupts structured raw data and misses JSON-shaped secrets #435
Capture redaction loses data and misses secrets, three ways:
mem::observe redacts JSON.stringify(payload.data) with regexes, then JSON.parses the result. A secret assignment at the end of a string value swallows the closing quote, the parse fails, and the fallback stores stripPrivateData(String(payload.data)) — the literal "[object Object]". Reproduced: {"command":"export API_KEY=abc123"} → stored raw "[object Object]".
The key-name patterns need key[=:], so JSON-shaped secrets pass untouched. Reproduced: {"tool_input":{"password":"hunter2hunter2"}} and {"accessToken":"…30 chars…"} are stored verbatim.
Hooks cut text to length before the daemon redacts (post-tool-use.ts 8000-char output cut), so a <private> block or secret split by the cut survives redaction. The same .slice calls can split a surrogate pair.
Redaction walks structured data value by value, so the stored raw keeps its shape; {"command":"export API_KEY=abc123"} is stored as an object with the value redacted
JSON-shaped secret keys (password, accessToken, apiKey, client_secret, …) have their values redacted
A hook redacts (or the daemon receives untruncated text) before any length cut, so a secret spanning the cut is not stored
Verdict: apply criteria 1, 2 and the <private> half of 3; reject surrogate truncation (verified 2026-11-18)
Reproduced on fdc3d459.{"command":"export API_KEY=abc123"} was stored as the string "[object Object]", and {"tool_input":{"password":"hunter2hunter2"}} / {"accessToken":"…"} were stored verbatim. In the live store, 10 Observations already have the "[object Object]" fallback shape.
Fix on branch verify/435-capture-redaction (not committed yet):
stripPrivateDataDeep() (src/functions/privacy.ts) walks the raw value by value, so the stored raw keeps its shape. A string under a secret-named key (password, accessToken, apiKey, client_secret, authorization, private_key, …) is replaced whole; other strings go through stripPrivateData. mem::observe calls it in place of the stringify/parse round trip.
The key-name patterns now also match quoted JSON keys ("password": "…", escaped quotes included) and skip true/false/null values.
PRIVATE_TAG_RE also strips an unclosed <private> block to the end of the string, so a block cut off by a hook's length limit is still removed.
Not done:
Redacting before the hook's length cut. With the unclosed-tag rule and value-level redaction, a cut no longer lets a <private> block or a secret-named value through. What remains is a secret assignment whose value the cut shortens below the 20-char token pattern. No case of this was found.
Lone surrogates from .slice. Rejected: 3 of 105,699 stored records have one, and none came from a hook cut.
Narrowed patterns: rerunning the new redaction over the live store changes 86 of 103,152 records, against 91 for the first, broader patterns. Regression tests pin the over-redactions found: a token count ("max_tokens": 4096), booleans and null, and code after an escaped close tag.
Evidence:test/capture-redaction.test.ts (3 tests) and test/privacy.test.ts (describes "JSON-shaped secrets (#435)" and "stripPrivateDataDeep"). They fail without the fix and pass with it. npm run typecheck, npm test (3127 passed, 1 skipped), eval:gate and eval:gate:rules all pass. eval:suite --tier fast: all 15 metrics equal to base fdc3d459 (Δ 0.000).
Activity
possiblyneal commentedon Nov 18, 2026
Not verified on this tree yet. Held at
needs-triageuntil Verify: capture redaction corrupts data and misses JSON secrets gives a tested verdict: a failing reproduction and a fix that works here, or a reject.possiblyneal commentedon Nov 18, 2026
Verdict: apply criteria 1, 2 and the
<private>half of 3; reject surrogate truncation (verified 2026-11-18)Reproduced on
fdc3d459.{"command":"export API_KEY=abc123"}was stored as the string"[object Object]", and{"tool_input":{"password":"hunter2hunter2"}}/{"accessToken":"…"}were stored verbatim. In the live store, 10 Observations already have the"[object Object]"fallback shape.Fix on branch
verify/435-capture-redaction(not committed yet):stripPrivateDataDeep()(src/functions/privacy.ts) walks the raw value by value, so the stored raw keeps its shape. A string under a secret-named key (password,accessToken,apiKey,client_secret,authorization,private_key, …) is replaced whole; other strings go throughstripPrivateData.mem::observecalls it in place of the stringify/parse round trip."password": "…", escaped quotes included) and skiptrue/false/nullvalues.PRIVATE_TAG_REalso strips an unclosed<private>block to the end of the string, so a block cut off by a hook's length limit is still removed.Not done:
<private>block or a secret-named value through. What remains is a secret assignment whose value the cut shortens below the 20-char token pattern. No case of this was found..slice. Rejected: 3 of 105,699 stored records have one, and none came from a hook cut.Narrowed patterns: rerunning the new redaction over the live store changes 86 of 103,152 records, against 91 for the first, broader patterns. Regression tests pin the over-redactions found: a token count (
"max_tokens": 4096), booleans andnull, and code after an escaped close tag.Evidence:
test/capture-redaction.test.ts(3 tests) andtest/privacy.test.ts(describes "JSON-shaped secrets (#435)" and "stripPrivateDataDeep"). They fail without the fix and pass with it.npm run typecheck,npm test(3127 passed, 1 skipped),eval:gateandeval:gate:rulesall pass.eval:suite --tier fast: all 15 metrics equal to basefdc3d459(Δ 0.000).Upstream: rohitg00#1569, rohitg00#1600, rohitg00#1695, rohitg00#1701, rohitg00#1597, rohitg00#1598 (read for the diagnosis, not the patch).