Fingerprinting Pre-trained Encoders under Arbitrary Downstream Fine-Tuning via Adversarial Shifting

About the Project

Pre-trained encoders have emerged as the foundational backbone for diverse downstream applications, representing significant intellectual property (IP) value. However, protecting the ownership of pre-trained encoders is critical yet challenging, as downstream fine-tuning fundamentally alters output semantics, rendering traditional end-to-end fingerprinting ineffective. Moreover, existing encoder-specific methods typically require impractical white-box access to embeddings. To address these limitations, we propose a downstream-agnostic and label-only framework for encoder ownership verification. Our approach utilizes Adversarial Shifting to craft fingerprint samples that form cohesive ``feature islands'' within the encoder's latent space. Exploiting the intrinsic stability of these clusters, we design a Group Voting mechanism that verifies ownership through output consistency across unknown downstream tasks. Extensive experiments demonstrate that our method achieves superior robustness and stealthiness against model extraction and adaptation, providing a practical safeguard for high-value encoders.

Getting Started

File Structure

Adversarial_Shifting-master
├── utils
│   ├── __init__.py
│   ├── data.py
│   ├── downstream.py
│   ├── extract_verify.py
│   └── models.py
├── main.py
├── README.md
├── requirements.txt
└── LICENSE

The repository is organized as follows:

• utils (fold): This directory houses the core source code for the main experiments.
• data.py:Handles raw dataset preprocessing and implements the data loading pipelines.
• downstream.py: Implements the training routines for the two specific downstream fine-tuning strategies.
• extract_verify.py: Encapsulates the core functions for fingerprint extraction and ownership verification.
• models.py: Defines the model architectures and handles model loading operations.
• main.py: The main function of MFUE.

Environment Setup

To ensure reproducibility, we recommend creating a new Conda environment with the specified dependencies.

Prerequisites:

• Python 3.11.5
• CUDA 11.0+ (Recommended for GPU)

Main Dependencies:

• torch==2.1.1 & torchvision==0.16.1
• numpy==1.26.2
• scikit-learn==1.2.2
• tqdm==4.66.4

Quick Start:

You can set up the environment using the following commands:

Create a new conda environment

conda create -n fingerprint python=3.11.5 conda activate fingerprint

Install dependencies

pip install -r requirements.txt

OR install manually

pip install torch==2.1.1 torchvision==0.16.1 numpy==1.26.2 scikit-learn==1.2.2 tqdm==4.66.4

Hyper-parameters

The settings are determined by the args parser in main.py. Here, we introduce the crucial hyper-parameters aligned with our methodology:

• --device: Calculation device specification (e.g., 'cuda' for GPU, 'cpu', or 'mps').
• --dataset: The dataset used for both fingerprint generation and downstream evaluation (e.g., 'stl10', 'cifar10').
• --arch: The architecture of the Victim Encoder ($E_{vct}$), such as 'res18'.
• --pretrain_style: The training paradigm of the pre-trained encoder (e.g., 'supervised', 'simclr', 'moco').
• --FT_mode: The fine-tuning strategy employed by the attacker for the downstream task (e.g., 'FTLL' for training the classifier head only, 'FTAL' for fine-tuning the entire model).
• --num_group: The total number of disjoint fingerprint groups ($L$) used for the final voting mechanism.
• --num_member: The number of base samples ($N$) to be shifted within each group to form a "feature island".
• --num_anchor: The number of anchor samples ($M$) selected from the target cluster to guide the adversarial shifting direction.
• --eps_l2: The $\ell_2$-norm constraint ($\epsilon$) for the adversarial perturbations to ensure visual imperceptibility.
• --num_iters: The number of PGD iterations utilized during the fingerprint optimization process.
• --member_lr: The step size ($\alpha$) for the Projected Gradient Descent (PGD) optimizer.
• --fp_path: The directory path where the generated fingerprint samples will be saved.

Run

You can run main.py directly to reproduce our method. The example below demonstrates the standard workflow for generating fingerprints and verifying the target encoder:

def main(args):
    # Train downstream task
    encoder = load_pretrain(args.pretrain_style, args.device, args.arch)
    MLP_layers = MLP(args.num_classes)
    head = classify(args.dataset, encoder, MLP_layers, args, mode=args.FT_mode)

    # Extract Fingerprints
    generate_FP(args)

    # Verify
    matching_rate = verify_FP(args.sus_model_dir, args.fp_dir, args)

if __name__ == '__main__':
    utils.random_seed(args.random)
    main(args)