[Multi-Agent Privacy] Integrate privacy into the RAG pipeline

[JCHAVEROT]

Summary

Related issue: #295
Subsequent to: #338

The privacy pipeline stopped at the gate, never producing the answer, checking it, or recording it. This PR adds the post-cloud half: after the gate approves, the answer model answers from the sanitized context alone, a verifier agent checks for residual leakage and unfaithful claims, and every request ends with a short PII-free record.

The chain is now:

... -> HITL -> answer -> verifier -> report

Changes

• AnswerModel (answer.py): answers from the sanitized context only, never the raw chunks; records the backend and model used
• AdvisoryVerifierAgent (agents/verifier.py): reads the full context plus the answer, runs the configured leakage and faithfulness checks, warns when confidence clears the threshold; never re-runs detection or cleaning
• Added a report record (report.py) and builder (report_builder.py) to give context to the user
• Wired the graph (pipeline.py): added answer, verifier, and report nodes, routing the proceed, reject, and abort paths through the report
• Integrated into RAG (rag/pipeline.py, run_rag.py): the privacy graph replaces the answer step and surfaces the report and warning summary alongside the answer
• Also integrated into RAG CLI
• Config (config.py): added VerifierCheck enum, VerifierConfig, and CloudLLMConfig
• JSON adapter (dspy_llm.py): tolerant parsing for small local models that have trouble with formats
• Tests: full E2E tests now that we have full pipeline (TBD soon in another PR)

Demo with RAG CLI

Note that adversarial agent is turned off here, but it works also fine with (just a bit longer)

CleanShot.2026-07-17.at.16.11.15-compressed.mp4

[Copilot]

Pull request overview

This PR completes the “post-cloud” half of the multi-agent privacy pipeline and integrates it into the RAG flow, so that (when enabled) retrieval outputs are sanitized/gated before an answer is generated, then the answer is advisorially verified and a PII-free report record is returned alongside the final response.

Changes:

• Extends the privacy graph beyond the HITL gate with new answer, verifier, and report stages (and wires the new routing).
• Integrates the privacy graph into the RAG pipeline as an optional replacement for the normal answer step, surfacing privacy_report and privacy_warnings in outputs.
• Adds supporting config/schema, reporting utilities, and DSPy JSON parsing tolerance improvements.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
src/mmore/run_rag.py	Adds --privacy flag support, builds/loads privacy graph, and exposes privacy fields in JSON/API output.
src/mmore/rag/pipeline.py	Adds optional privacy_graph path that replaces the answer step and surfaces privacy report/warnings.
src/mmore/privacy/verification.py	Introduces verifier verdict/warning dataclasses + summary helpers.
src/mmore/privacy/runner.py	Adds config validation and a single-query runner that drives the compiled privacy graph.
src/mmore/privacy/report.py	Defines the PII-free ReportRecord schema and related enums.
src/mmore/privacy/report_builder.py	Builds/serializes the final report record from final graph state.
src/mmore/privacy/pipeline.py	Wires answer/verifier/report nodes into the LangGraph pipeline and updates routing.
src/mmore/privacy/escalation.py	Ensures detection-engine switching also resets per-engine default params.
src/mmore/privacy/dspy_llm.py	Adds a tolerant JSON adapter for small/local model formatting issues.
src/mmore/privacy/config.py	Adds verifier/answer config types and new enums; adjusts leakage adversary defaults.
src/mmore/privacy/answer.py	Adds post-gate answer model that uses sanitized context only and records model identity.
src/mmore/privacy/agents/verifier.py	Adds post-cloud advisory verifier agent (residual leakage + faithfulness checks).
src/mmore/privacy/agents/state.py	Extends shared privacy graph state to include answer/verifier/report metadata.
src/mmore/privacy/agents/analyzer.py	Switches DSPy adapter usage to the new tolerant JSON adapter.
src/mmore/privacy/agents/adversary.py	Tweaks adversary probe instructions wording.
src/mmore/privacy/agents/init.py	Exports the new verifier agent.
src/mmore/cli.py	Adds --privacy option to the mmore rag CLI command.
examples/rag/privacy.yaml	Adds an example privacy config including answer + verifier sections.
docs/source/index.md	Adds privacy mode doc page to the docs index.
docs/source/core_features/privacy_mode.md	Documents privacy mode pipeline, configuration, and report schema.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 42 changed files in this pull request and generated 1 comment.