The Prompt Stealing Fallacy: Rethinking Metrics, Attacks, and Defenses

Overview

This repository contains the implementation of The Prompt Stealing Fallacy: Rethinking Metrics, Attacks, and Defenses. We present one novel attack method and two defense strategies to address the vulnerability of prompt stealing in T2I Models.

Project Structure

pthieft_v01/
├── attack/                     # Attack code and docs
│   ├── README.md               # Attack run guide and usage notes
│   ├── data/                   # Data utilities (process_json.py, resample.py, eval.py, etc.)
│   ├── eval/                   # Evaluation helpers│ 
│   ├── script/                 # Training/attack scripts
│   └── environment.yml         # Attack env dependencies
├── defense/                    # Defense code and docs
│   ├── README.md               # Defense run guide and usage notes
│   ├── scripts/                # Runner scripts
│   ├── src/                    # Core defense implementation
│   ├── data/                   # Datasets / assets for defense
│   ├── requirements*.txt       # Defense env dependencies
│   └── utils*.py               # Utility modules
└── README.md                   # Repository overview (this file)

Where to find run instructions

• Attack run steps: see ./pthieft_v01/attack/README.md.
• Defense run steps: see ./pthieft_v01/defense/README.md.

Attack Method

Attack

Located in ./pthieft_v01/attack/

Our attack leverages the MS-SWIFT framework to perform prompt stealing attacks on T2I models. The attack method is designed to extract high-contribution prompts.

Given that our attack methodology may potentially impact commercial platforms and the interests of real-world prompt engineers (see the Ethical Considerations section), we do not publicly release the full attack implementation (i.e., our modified ms-swift framework). To balance scientific transparency with responsible disclosure, the attack code is available upon reasonable request for research and verification purposes only.

For the artifact evaluation process, we provide the complete codebase to the reviewers. The access link has been submitted via the AE HotCRP platform, ensuring full availability during the evaluation phase.

Defense Methods

PSA Defense Framework

Located in ./pthieft_v01/defense See ./pthieft_v01/defense/README.md for defense setup and run instructions.