Skip to content

fix(test): scrub personal PII from provider fixtures + add recurrence guard#456

Merged
plauzy merged 1 commit into
awslabs:mainfrom
plauzy:security/scrub-pii-test-fixtures
Jul 17, 2026
Merged

fix(test): scrub personal PII from provider fixtures + add recurrence guard#456
plauzy merged 1 commit into
awslabs:mainfrom
plauzy:security/scrub-pii-test-fixtures

Conversation

@plauzy

@plauzy plauzy commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Removes personal email addresses (PII) that were committed into provider test
fixtures, and adds a guard so it cannot happen again.

  • Deleted 9 orphaned test/providers/fixtures/antigravity_cli_*.txt fixtures
    that captured a maintainer's Antigravity/Google login banner
    (…@gmail.com (Google AI Ultra)). These files are referenced by no test
    the antigravity suite loads the separate agy_*.txt fixtures — so removing
    them has zero test impact.
  • Redacted a second contributor's account email in
    test/providers/fixtures/claude_code_new_tui_completed_raw.txt (which is
    used by test_claude_code_unit.py) to user@example.com. The address sat in
    the header banner; the test only asserts COMPLETED status and the bottom-box
    pattern, so status detection is unaffected.
  • Added test/test_fixtures_no_personal_pii.py — a fail-closed guard that
    scans every test/**/fixtures/** file for personal-provider email addresses
    and fails CI if any are found.

No personal email remains in any tracked file after this change.

How this happened

Provider status-detection fixtures are recorded from live CLI TUIs. Several
CLIs (Antigravity, Claude Code) print the authenticated account's email in
their startup/login banner. When those raw terminal captures were saved verbatim
as fixtures and committed, the account emails rode along inside the ANSI escape
stream — not obvious in a rendered terminal, but plainly visible in the file and
in the PR diff. One set shipped in #436 (which merged before the removal landed);
another had been present in an older claude_code capture.

Because the addresses are embedded among ANSI escape codes, they slipped past
manual diff review.

Prevention

  • Automated guard (this PR): test/test_fixtures_no_personal_pii.py fails CI
    the moment a fixture contains an address at a personal mail provider (gmail,
    icloud, outlook, proton, …). Synthetic/tooling addresses (@example.com,
    git@github.com) are intentionally allowed so legitimate fixtures still pass.
  • Fixture-recording hygiene: when capturing a new live-TUI fixture, scrub the
    login/banner line before committing — replace the account email with
    user@example.com. Prefer capturing on a throwaway/synthetic account where
    possible.
  • History note: because feat(agui): AG-UI protocol adapter + generative UI (L1, default-off) — supersedes #387 #436 already merged, the removed addresses remain in
    git history at the merge commit. If full history scrubbing is desired, that
    requires a maintainer-run git filter-repo/BFG pass on the default branch
    (out of scope for a normal PR); the affected account owners may also wish to
    treat the addresses as publicly exposed.

Testing

  • test/test_fixtures_no_personal_pii.py — passes (no PII remains).
  • test_claude_code_unit.py, test_antigravity_cli_unit.py,
    test_tool_mapping.py — pass (redaction/deletion broke nothing).
  • black/isort clean.

… guard

Provider status-detection fixtures are captured from live CLI TUIs whose login
banners print the authenticated account email. Two such addresses were committed:

- Delete 9 orphaned `test/providers/fixtures/antigravity_cli_*.txt` fixtures
  (shipped via awslabs#436) that captured a maintainer's Google login banner. They are
  referenced by NO test — the antigravity suite uses the separate `agy_*.txt`
  fixtures — so deleting them removes the PII with zero test impact.
- Redact a contributor's account email in
  `test/providers/fixtures/claude_code_new_tui_completed_raw.txt` (used by
  `test_claude_code_unit.py`) to `user@example.com`. The email sits in the
  header banner; the test asserts only COMPLETED status and the bottom-box
  pattern, so detection is unaffected.
- Add `test/test_fixtures_no_personal_pii.py`: a fail-closed guard that scans
  every `test/**/fixtures/**` file for personal-provider email addresses
  (gmail, icloud, …) and fails CI, so a live-capture PII leak cannot recur.

Co-authored-by: Kiro Agent <244629292+kiro-agent@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@a9af159). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #456   +/-   ##
=======================================
  Coverage        ?   89.35%           
=======================================
  Files           ?      156           
  Lines           ?    18510           
  Branches        ?        0           
=======================================
  Hits            ?    16539           
  Misses          ?     1971           
  Partials        ?        0           
Flag Coverage Δ
unittests 89.35% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@plauzy plauzy added the security issue Anything related to leaked secrets, PII, or other security-related topics label Jul 17, 2026

@fanhongy fanhongy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@plauzy
plauzy merged commit 52dd96b into awslabs:main Jul 17, 2026
11 checks passed
@plauzy

plauzy commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author

@fanhongy TY for fast approval - i created #457 in light of the issue which we can all discuss at a later point.

This issue was low-severity but probably a good reason to think about how we would deal with a more severe issue if/when that happens. cc @haofeif @call-me-ram

@plauzy
plauzy deleted the security/scrub-pii-test-fixtures branch July 17, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

security issue Anything related to leaked secrets, PII, or other security-related topics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants