Skip to content

[Multi-Agent Privacy] Add Adversarial agent with initial pipeline#338

Open
JCHAVEROT wants to merge 12 commits into
feat/privacy-agents-cleanfrom
feat/adversarial-agent
Open

[Multi-Agent Privacy] Add Adversarial agent with initial pipeline#338
JCHAVEROT wants to merge 12 commits into
feat/privacy-agents-cleanfrom
feat/adversarial-agent

Conversation

@JCHAVEROT

@JCHAVEROT JCHAVEROT commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Related issue: #294
Depends on: #337

Note that these commits were originally on a fork repo, a rebase was needed to target the branch on the origin repo feat/privacy-agents-clean

The privacy pipeline cleaned the retrieved context but never checked that the cleaned text was actually safe before sending it to the cloud model. After cleaning, an adversary agent looks for leftover identifiers. If it finds one, it writes a short PII-free remediation report (which detection engine, cleaning strategy, threshold, or entity labels to change) and the pipeline applies it and runs detection and cleaning again, up to a set number of tries. Once the text is clean, a human approves it (HITL)

Changes

  • Added AdversarialAgent (agents/adversary.py): probes the cleaned text for leftover identifiers and flags a leak when confidence is high enough. On a leak it also emits a remediation report, with knowledge of the available engines/strategies/params and of its own past reports, so it can see whether they were applied
  • Added report-driven escalation: the analyzer applies the adversary's report through the same path as human gate feedback (extra entity labels, engine/strategy/threshold overrides, guidance pinned into the prompt). No fixed escalation ladder; without a usable report the policy is only hardened with the escalation note. Stops after max_iterations and marks the request unsafe
  • Added the HITL gate (agents/gate.py): shows a short PII-free summary and waits for a human choice (approve / revise / reject)
  • Wired the graph (pipeline.py) with the loop and the routing for the adversary and gate
  • Added config (config.py): AttackVector enum, LeakageAdversaryConfig, and an interactive flag

Notes

  • Only leaks count against max_iterations, a human revision does not so he/she can ask changes as many times as wanted
  • The adversary's report cannot override fields the user pinned in the config (engine, strategy, threshold), but human feedback at the gate can
  • The gate summary has no PII, so the review itself does not leak

@JCHAVEROT JCHAVEROT self-assigned this Jun 29, 2026
@JCHAVEROT JCHAVEROT added the enhancement New feature or request label Jun 29, 2026
@JCHAVEROT JCHAVEROT linked an issue Jun 29, 2026 that may be closed by this pull request
@fabnemEPFL
fabnemEPFL requested a review from Copilot June 30, 2026 15:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a pre-cloud privacy “leakage loop” to the existing analyzer→detector→sanitizer flow by introducing an adversarial probe stage, a bounded escalation mechanism, and an optional HITL approval gate before any sanitized context is sent to a cloud model.

Changes:

  • Introduces AdversarialAgent + LeakageVerdict to probe sanitized context for residual identifiers and quasi-identifiers.
  • Adds an escalation ladder to progressively tighten policy (entities/threshold/engine/strategy) on detected leakage, with an iteration budget.
  • Adds an optional HITL gate agent to summarize actions taken and pause for human approve/revise/reject, and wires the full LangGraph pipeline.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/mmore/privacy/sanitization/constants.py Adds per-strategy guidance strings so the analyzer can map descriptive feedback to sanitization choices.
src/mmore/privacy/pipeline.py New LangGraph wiring for analyzer→detector→sanitizer→adversary→gate, including bounded escalation routing.
src/mmore/privacy/leakage.py Defines structured leakage verdict and escalation record dataclasses used across the loop.
src/mmore/privacy/escalation.py Implements policy escalation steps (threshold/engine/strategy) and entity guidance merging.
src/mmore/privacy/config.py Adds AttackVector, LeakageAdversaryConfig, and an interactive flag to PrivacyConfig.
src/mmore/privacy/agents/state.py Extends shared PrivacyState with leakage + escalation + HITL fields and introduces PreCloudOutcome.
src/mmore/privacy/agents/gate.py New HITL approval agent using interrupt() with a PII-free operational summary.
src/mmore/privacy/agents/analyzer.py Enhances analyzer to handle loop re-entry, leak-targeted label expansion, and human-feedback-driven policy tweaks.
src/mmore/privacy/agents/adversary.py New adversary agent that runs multiple attack vectors and returns the strongest leakage signal.
src/mmore/privacy/agents/init.py Exports new adversary and gate agent symbols for package consumers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/mmore/privacy/agents/adversary.py
Comment thread src/mmore/privacy/agents/adversary.py Outdated
Comment thread src/mmore/privacy/agents/analyzer.py Outdated
Comment thread src/mmore/privacy/agents/gate.py Outdated
Comment thread src/mmore/privacy/escalation.py Outdated
Comment thread src/mmore/privacy/pipeline.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Comment thread src/mmore/privacy/escalation.py Outdated
Comment thread src/mmore/privacy/agents/adversary.py Outdated
Comment thread src/mmore/privacy/config.py
Comment thread src/mmore/privacy/config.py Outdated
@JCHAVEROT
JCHAVEROT force-pushed the feat/privacy-agents-clean branch 5 times, most recently from 1b801cf to 0f0d2a3 Compare July 11, 2026 19:12
@JCHAVEROT
JCHAVEROT force-pushed the feat/adversarial-agent branch from 19fccde to 201382c Compare July 11, 2026 19:29
@JCHAVEROT
JCHAVEROT force-pushed the feat/privacy-agents-clean branch 3 times, most recently from 0f0d2a3 to 92a2b79 Compare July 13, 2026 09:22
@JCHAVEROT
JCHAVEROT force-pushed the feat/adversarial-agent branch from 201382c to 4d7562a Compare July 13, 2026 09:52
@fabnemEPFL
fabnemEPFL requested a review from Copilot July 13, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comment thread src/mmore/privacy/agents/analyzer.py
Comment thread src/mmore/privacy/agents/adversary.py Outdated
Comment thread src/mmore/privacy/agents/adversary.py Outdated
return "\n".join(f"- {name}: {desc}" for name, desc in guidance.items())


def _clamp_confidence(value: object) -> float:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it have to have object as type hint?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, replaced for float | int | str | None

Comment thread src/mmore/privacy/agents/adversary.py Outdated
Comment thread src/mmore/privacy/agents/adversary.py
Comment thread src/mmore/privacy/agents/gate.py Outdated
Comment thread src/mmore/privacy/agents/gate.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

src/mmore/privacy/agents/detector.py:124

  • from_config() still accepts dict-like configs at runtime via load_config(...) (which supports Union[str, Dict, T]), but the type annotation was narrowed to PrivacyConfig | str. This is an API typing regression for callers that pass a dict. Consider reflecting the actual accepted types in the signature.
    @classmethod
    def from_config(
        cls,
        config: PrivacyConfig | str,
        checkpointer: Optional[BaseCheckpointSaver] = None,
    ) -> Self:
        if not isinstance(config, PrivacyConfig):
            config = load_config(config, PrivacyConfig)
        return cls(config, checkpointer=checkpointer)

src/mmore/privacy/agents/sanitizer.py:82

  • from_config() still accepts dict-like configs at runtime via load_config(...), but the signature was narrowed to PrivacyConfig | str. If dict configs are supported (as load_config indicates), include dict in the annotation to avoid misleading API typing for callers.
    @classmethod
    def from_config(
        cls,
        config: PrivacyConfig | str,
        checkpointer: Optional[BaseCheckpointSaver] = None,
    ) -> Self:
        if not isinstance(config, PrivacyConfig):
            config = load_config(config, PrivacyConfig)
        llm_config = config.sanitization.llm if config.sanitization else None
        return cls(config, llm_config, checkpointer=checkpointer)

Comment thread src/mmore/privacy/pipeline.py Outdated
Comment thread src/mmore/privacy/agents/gate.py Outdated
@JCHAVEROT
JCHAVEROT force-pushed the feat/privacy-agents-clean branch from 6971012 to fae2eb3 Compare July 21, 2026 16:12
@JCHAVEROT
JCHAVEROT force-pushed the feat/adversarial-agent branch from fe55399 to c5fa763 Compare July 21, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adversarial agent implementation

3 participants