security(logging): stop treating exc_info tracebacks as exception-detail redaction

[seonghobae]

Finding

PR #1696 generated a repository-wide exception rewrite that replaced many f"...{exc}" log messages with static text plus exc_info=True. That is not a complete CWE-532 mitigation: Python traceback rendering still includes the exception type/value, so provider responses, connection strings, paths, or other sensitive values can remain in the log sink.

The broad sweep has therefore been removed from #1696. The public tool-response leak has a bounded repair there; logging confidentiality needs its own sink-aware owner rather than a generated blanket rule.

A CodeRabbit outside-diff finding on backend/scripts/import_fixtures.py also confirms a real raw-exception logging site. Other former #1696 paths included mail import/workers, runner connectivity, email endpoints, and LLM service logging. Those sites must be evaluated by actual data flow and log audience, not by string-pattern replacement alone.

Required contract

Define a shared structured exception telemetry contract that preserves operational correlation without logging raw exception values by default. Candidate evidence fields include exception class, stable traceback/location fingerprint, operation code, and bounded non-secret identifiers. Full traceback/value capture, when genuinely required, must be explicitly classified, access-controlled, retained appropriately, and tested against representative secret-bearing exceptions.

Do not assume exc_info=True is redacted. Do not suppress all diagnostics either.

Acceptance

• inventory production exception-log sinks and their trust/retention boundary;
• RED tests inject representative API keys/tokens, connection strings, provider response text, and internal paths and assert they do not appear in normal production logs;
• structured correlation evidence remains sufficient for RCA;
• no public/API response behavior is changed accidentally;
• security/operations doctoring records the chosen policy and exceptions;
• hosted security tests run on the final exact head.

References: CWE-532; OWASP Logging Cheat Sheet. Refs #1696.

[seonghobae]

Owner-path update: Draft #1700 (fix/structured-exception-telemetry@6b515f5b2e08241eeecd4b587a50a68a79f60c76) is now the first structured-telemetry successor, stacked on canonical bounded exception owner #1612 3da3ae8e... rather than competing with it.

Verified current gap:

• fix(security): stabilize exception-redaction successor #1612 removes raw exception values from its bounded surface, but its shared helper still attaches the original traceback; fix(security): bound exception telemetry without raw traceback #1700 replaces only that helper behavior with bounded exception class + 16-hex fingerprint and no traceback while preserving all fix(security): stabilize exception-redaction successor #1612 caller source.
• protected develop@042b0c7... still has additional sink classes outside fix(security): stabilize exception-redaction successor #1612: code search currently finds exc_info=True in agent registry, Reply-SLA scheduler, Context Search, runner WebSocket persistence, Newsdom worker, provider writeback retry, email import projection, batch embedding cleanup, project-graph extractor fallback and others. Raw exception interpolation also remains on protected source in several paths.
• these remaining sinks are not declared fixed by fix(security): bound exception telemetry without raw traceback #1700; each still needs trust/retention/data-flow review and representative secret-bearing RED coverage before security(logging): stop treating exc_info tracebacks as exception-detail redaction #1698 can close.

Do not merge #1700 into #1612 while #1612 exact-head evidence is active. Parent must integrate normally first, then #1700 should be reconciled/retargeted and revalidated without force push or predecessor-evidence transfer.

[seonghobae]

#1700 exact 2d6800aea110ac04eeedcd5faa91479f62877408 now tightens the structured telemetry foundation after a second RCA pass. The first fingerprint key (exception type + module + function) merged distinct raise statements inside one function into one incident signature. Current source adds traceback line + bytecode offset to the hash input only while still emitting no raw traceback/path/function/line/offset. Regression coverage now requires same-site/different-message stability and different fingerprints for two raise statements in the same function. Doctoring records build-scoped stability and rejects module/function-only correlation. #1698 remains open: the remaining protected-develop exc_info=True / raw interpolation sinks still require sink-by-sink trust/retention review and migration. #1700 is stacked on #1612 and has zero PR-triggered workflow runs on the new exact head, so hosted evidence remains blocked by the stacked-PR trigger foundation #1691; no synthetic evidence or temporary retargeting was used.

[seonghobae]

Fresh protected-develop@042b0c70531b229af3acbd0421a2f23098d848b3 sink inventory, deduplicated against the live #1612 security owner:

#1612 already owns backend/api/emails.py, backend/services/imap_worker.py, backend/services/pop3_worker.py, backend/core/safe_logging.py, fixture import, prompts, and llm_service.py; do not open parallel writers for those files while that owner remains unintegrated.

Still outside #1612 and currently using exc_info=True on protected develop:

• backend/api/search.py — Search failed and Grounded answer failed; API/RAG boundary, highest next priority because provider/DB/internal exception values and traceback locations can enter a normal production log sink.
• backend/services/agent_registry.py — registration-file read/JSON failures; also logs the path as a %s argument, so migration must decide whether that path is an allowed bounded identifier rather than merely replacing traceback handling.
• backend/services/reply_sla_scheduler.py — scheduler-loop/config failure paths.
• backend/api/runner_ws.py — runner signal/command persistence SQLAlchemy failures.
• backend/services/newsdom_worker.py — recognition worker loop.
• backend/services/provider_writeback_retry_service.py — provider-writeback retry loop.
• backend/services/email_import_service.py — project-graph projection failure after rollback.
• backend/services/batch_embedding_service.py — pg-llm-batch config-store cleanup failure.
• backend/services/project_graph/extractor_registry.py — extractor fallback loop.

This is not a blanket-replace authorization. #1700 exact 2d6800aea110ac04eeedcd5faa91479f62877408 is the shared structured-correlation foundation; each remaining sink still needs its own trust/retention and required-debug-context check. A safe next bounded slice is backend/api/search.py after #1700's unchanged-head review, with RED tests injecting secret-bearing/provider/DB exceptions into the real logging formatter and asserting the normal log preserves operation/fingerprint evidence but not exception value or raw traceback/source location. Worker/scheduler sinks should follow separately because their operator audience and bounded identifiers differ.

[seonghobae]

#1700 advanced without moving source: exact 2d6800aea110ac04eeedcd5faa91479f62877408 now has a formal CodeRabbit APPROVED review (PRR_kwDOSNjZ2s8AAAABNs7L5g, submitted 2026-09-15T18:52:19Z) covering exactly the three effective files and no actionable current-head finding. The review gate for the shared structured-correlation foundation is therefore satisfied on this source head; repository-local hosted evidence is still absent because the PR is stacked on #1612 and #1691 still owns stacked-PR workflow admission.

The next bounded sink remains backend/api/search.py, not a repository-wide replacement. Protected develop has two exc_info=True API/RAG failure sinks (Search failed, Grounded answer failed). Any successor should ordinary-stack on #1700, inject representative provider/DB/secret-bearing exceptions through the real formatter, preserve the public search response contract, and migrate only those two sinks to the reviewed redacted_exception_info() contract. Do not duplicate #1612-owned mail/fixture/LLM paths or mark #1698 complete.

[seonghobae]

Fresh sink/owner audit on 2026-09-16: the remaining exc_info=True inventory is real, but the next repair must not open a second writer in paths that already have active product owners.

Verified protected sinks still include backend/services/agent_registry.py, backend/api/search.py, Reply-SLA scheduler, IMAP/POP3 workers, email send, runner WebSocket persistence, Newsdom, provider-writeback retry, email-import projection, batch-embedding cleanup, and project-graph extractor fallback. The first two have live overlapping owner lanes:

• backend/services/agent_registry.py: refactor(agent-registry): use semantic agent identifiers #1516 exact 71ea032006665ebb63f0b5616b764e63d63a897f is the active semantic registry owner; its current file still contains logger.debug(..., exc_info=True) on registration-file read/JSON failures. refactor(agent-registry): publish semantic registry fields #1537 is adjacent registry contract work.
• backend/api/search.py: refactor(search): name email result identifiers semantically #1511 exact f78222dda6ac961fa3c1c4fb2acd9d7625e7b672 is the active Context Search naming owner; feat: forward ZDR policy to contextual orchestrator #1480 also carries broader search/orchestrator changes.

Therefore do not create a new #1698 source branch over either path while those owners are active. #1700 exact 2d6800aea110ac04eeedcd5faa91479f62877408 remains the approved shared structured-telemetry foundation; it is still stacked and has no repository-local hosted receipts because of #1691. The next source slice should be chosen only after a fresh overlap check finds an unowned sink, or be handed into the existing path owner as an explicit dependency on #1700. No blanket exc_info=True replacement and no duplicated security writer.

[seonghobae]

Fresh owner-map refinement — 2026-09-16:

Protected develop@042b0c70531b229af3acbd0421a2f23098d848b3 still contains multiple exc_info=True sinks, but several are not available for a new #1698 writer:

• backend/services/reply_sla_scheduler.py sits inside the active broad Reply-SLA/Noema owner feat(noema-agent): add calendar conflict-check tool #1486;
• backend/services/provider_writeback_retry_service.py is actively owned by fix(retry): claim due provider writeback retries with SKIP LOCKED #1654 (current retry/concurrency/policy lane);
• backend/services/project_graph/extractor_registry.py is inside the active project-graph extraction owner path led by fix(project-graph): stop Naruon from choosing its own orchestrator pool/model #1525;
• backend/services/agent_registry.py and backend/api/search.py remain on the previously identified active owner paths and should likewise consume the shared telemetry contract through those owners rather than a duplicate sweep.

Therefore do not open a blanket exc_info=True replacement PR from this issue. #1700 remains the shared structured-exception telemetry foundation. The next implementation slice must either (a) be an actually unowned sink after exact PR/file overlap verification, stacked on the telemetry foundation, or (b) be handed to the existing file owner for adoption. exc_info=True alone is still not a redaction mechanism.

[seonghobae]

Sink-by-sink migration advanced with #1709 (fc56d609045e47093fdf455234121eb7cfe31715) stacked directly on canonical telemetry #1700. It removes raw exc_info=True and exception-bearing path disclosure from backend/services/agent_registry.py read/JSON-parse failures by consuming redacted_exception_info(...), and adds formatter-level regressions with API-key/connection-string/token/path injections plus doctoring. Effective delta is exactly 3 files; compare against #1700 is ahead 3 / behind 0. Exact-head repository workflow lookup is currently 0, so this is source/test progression only and remains Draft under #1691; no hosted GREEN or parent-review transfer is claimed.

[seonghobae]

#1709 review gate update: CodeRabbit formally APPROVED exact fc56d609045e47093fdf455234121eb7cfe31715 at 2026-09-16T15:51:58Z (PRR_kwDOSNjZ2s8AAAABN2-vow) with zero review threads. Exact-head repository workflow lookup is still zero, so the sink migration now has source/test + independent-review evidence but still lacks hosted stacked evidence under #1691. Keep #1698 open; this closes only one sink/review slice, not the remaining production sink inventory.

[seonghobae]

Ownership correction after a fresh open-PR sweep: #1486 exact 22c173bb93ef15c3244477fe32540aeb2bcf1199 is already the canonical broad Reply-SLA/Noema writer and currently modifies backend/services/reply_sla_scheduler.py plus its scheduler tests. The earlier version of this handoff, which described #1710 as the sink owner directly on #1700, is superseded.

Current state:

• fix(logging): stage Reply SLA telemetry successor on canonical owner #1486 #1710 exact eee00bda624c58795a8adb17a5b2b734fd20d0a1 has been retargeted to feat(noema-agent): add calendar conflict-check tool #1486 and remains Draft/non-mergeable while ordinary ancestry reconciliation is pending;
• no force rebase or synthetic merge tree was used, because either would risk dropping valid feat(noema-agent): add calendar conflict-check tool #1486 or fix(security): bound exception telemetry without raw traceback #1700 deltas;
• protected develop still has the raw Reply-SLA loop exc_info=True finding;
• feat(noema-agent): add calendar conflict-check tool #1486 has already independently removed raw owner interpolation from its per-owner failure path and emits a bounded exception type there, so the eventual telemetry successor must preserve feat(noema-agent): add calendar conflict-check tool #1486's live physical-lease/workspace/rollback semantics rather than overwrite them with the older protected-develop implementation;
• fix(security): bound exception telemetry without raw traceback #1700 remains the structured telemetry prerequisite. Once fix(security): bound exception telemetry without raw traceback #1700 is protected-integrated and feat(noema-agent): add calendar conflict-check tool #1486 has repaired its own prerequisite ancestry, fix(logging): stage Reply SLA telemetry successor on canonical owner #1486 #1710 can ordinary-adopt both lineages and rerun its formatter-level RED tests against the real owner head.

Keep #1698 open. The valid Reply-SLA confidentiality finding is preserved, but no current #1710 source delta is merge authority until that owner-path repair is complete.

[seonghobae]

Context Search ownership finding: do not open a telemetry PR for backend/api/search.py yet. Fresh changed-file inspection found three active Draft writers on the same file: #1480 0bc675932e2db7481368215c27751cd7b6de9e38 (ZDR/orchestrator forwarding), #1511 f78222dda6ac961fa3c1c4fb2acd9d7625e7b672 (semantic email_id with legacy wire id), and #1486 22c173bb93ef15c3244477fe32540aeb2bcf1199 (workspace-aware owner scope). Their current hunks are mostly orthogonal, but the ancestry is parallel.

Opened #1711 to reconcile those active writers before #1698 migrates the two protected-develop Search failed / Grounded answer failed raw exc_info=True sinks. The eventual logging slice must stack on the reconciled search owner plus #1700, preserve all three valid product deltas, and test both formatter confidentiality and traceback-chaining suppression on the public 500 path. This prevents repeating the #1710 Reply-SLA ownership mistake.

[seonghobae]

Runner telemetry sink migration is now isolated in #1712 on top of #1700.

Exact topology: #1700 2d6800aea110ac04eeedcd5faa91479f62877408 → #1712 a085240288baac27d4bb1373a62ed8ab73aa2abf, ahead 4 / behind 0, exactly three files (backend/api/runner_ws.py, focused regression, doctoring). The slice covers both remaining raw-exception paths in that module: ConnectionManager.dispatch_command() (logger.exception) and connector-signal persistence (exc_info=True). Both now consume #1700 redacted_exception_info(); API/retry/DB behavior is unchanged.

RED history is preserved at 566f1f737d5a... and 4e8e800e31d1...; production repair starts 3026bac5d7c2.... Exact current head has zero repository-local PR workflow receipts, so #1712 remains Draft and no predecessor evidence is transferred. Independent current-head review was requested. Keep #1698 open: this closes only the runner_ws.py sink group, not the remaining repository inventory.

[seonghobae]

#1712 authority correction after immediate test-isolation review: current exact head is now c57b2d110c079fa862c97251aa835215f014365d (same three effective files, #1700 merge base exact, ahead 5 / behind 0). The only post-doctoring change adds an autouse runner_ws.manager.reset() fixture so the singleton connection state cannot leak across tests. Prior a085240... is predecessor evidence only. Current-head review has been re-requested; hosted workflow receipt must be checked on c57b2d1... only.

[seonghobae]

#1712 review follow-up on exact c57b2d110c079fa862c97251aa835215f014365d: CodeRabbit re-inspected exact #1700→#1712 and reported no blocking issue. It explicitly confirmed the autouse manager.reset() fixture fixes the predecessor test-isolation finding and that both logging sinks use the bounded helper with the intended secret/traceback regressions. This is a source-inspection comment, not a formal GitHub APPROVED review; formal review inventory is still empty. Hosted exact-head workflow inventory is also still 0. #1712 therefore remains Draft; #1698 remains open for other owned sinks.

[seonghobae]

#1698 sink migration update — OpenTelemetry setup

Fresh protected-source inventory found an unowned raw logger.exception() sink in backend/core/telemetry.py; no open PR was writing that file. I created Draft #1713 on the canonical structured-telemetry parent #1700 rather than on develop.

• base: fix(security): bound exception telemetry without raw traceback #1700 2d6800aea110ac04eeedcd5faa91479f62877408
• exact head: 76d54e20a020fb96e31554fd43fa3de2bf5053af
• exact effective delta: 3 files (backend/core/telemetry.py, focused regression, doctoring)
• RED: d7289799e2661ad3ebbf5c309579ec8bdc45738b injects token/connection-string/path data at the real OTEL import boundary and formats the resulting log record
• minimal fix: 96446d87fdbf59de1868a55f01be999990d31df8 consumes fix(security): bound exception telemetry without raw traceback #1700 redacted_exception_info(); operation message/severity and continue-without-tracing behavior are preserved
• current exact-head repository workflow lookup: 0 runs on the valid stacked base; no hosted GREEN claimed
• independent exact-head review requested; no approval claimed yet

The repair intentionally does not change OTEL enablement/config/bootstrap ownership. Search/Reply-SLA/other actively owned sinks remain separate owner-path work.

[seonghobae]

Current owner/sink handoff — 2026-09-17

Protected develop@042b0c70531b229af3acbd0421a2f23098d848b3 still has raw exc_info=True/exception-logging sinks, but the remaining initial inventory is no longer an unowned queue that can safely receive parallel micro-PRs.

• migrated on the canonical fix(security): bound exception telemetry without raw traceback #1700 telemetry stack: fix(logging): redact agent registry exception telemetry #1709 agent_registry.py@fc56d609045e47093fdf455234121eb7cfe31715, fix(logging): redact runner exception telemetry #1712 runner_ws.py@c57b2d110c079fa862c97251aa835215f014365d, fix(logging): redact OpenTelemetry setup exception telemetry #1713 core/telemetry.py@76d54e20a020fb96e31554fd43fa3de2bf5053af;
• backend/api/search.py: overlapping active owners feat: forward ZDR policy to contextual orchestrator #1480/refactor(search): name email result identifiers semantically #1511/feat(noema-agent): add calendar conflict-check tool #1486; convergence is tracked by architecture(search): reconcile active search.py writers before telemetry migration #1711, so do not add a fourth writer;
• Reply-SLA scheduler: feat(noema-agent): add calendar conflict-check tool #1486 owner path;
• provider writeback retry: fix(retry): claim due provider writeback retries with SKIP LOCKED #1654 owner path;
• project-graph extractor registry: fix(project-graph): stop Naruon from choosing its own orchestrator pool/model #1525 owner path;
• Newsdom worker: fix(attachments): align deferred parse budget with import transport #1469 owner path;
• email-import/batch-embedding surfaces: active owner overlap including feat: forward ZDR policy to contextual orchestrator #1480/feat(email): deterministic dedupe provenance — gate strong fingerprints on genuine Date (naruon#1086) #1195/fix(import): preserve attachment source on security owner stack #1701.

Accordingly no new source writer was created merely to reduce the grep count. Each remaining sink must migrate through its existing canonical owner or a verified successor, preserving that owner's valid domain delta.

#1713 remains a material three-file child of exact #1700 with zero repository-local workflow receipts. Its first exact-range CodeRabbit request was capacity-limited; a fresh review request has now been issued against unchanged 2d6800aea110ac04eeedcd5faa91479f62877408 → 76d54e20a020fb96e31554fd43fa3de2bf5053af rather than pushing a no-op wake commit. Formal review is not assumed until it actually appears.