FreqBrand detects the Silent Branding Attack (CVPR 2025) — a trigger-free data poisoning attack that embeds a logo into diffusion model training images so that any model finetuned on the poisoned data reproduces the logo in all generated outputs, with no inference-time trigger required. No existing defense handles this. We build the first detector.
Team: Yevin Goonatilake (lead), Sina Mansouri (theory), Philip Stavrev (baselines) — GMU CS, advisor: Prof. Ateniese.
Text-to-image diffusion models are routinely fine-tuned on community-shared datasets. The Silent Branding Attack exploits this by poisoning those datasets with a subtly embedded logo — one that survives fine-tuning and appears in every generated image, with no trigger required. Image quality metrics (FID, CLIP score) remain unchanged, so the user has no easy signal.
FreqBrand is the first detector designed for this threat. Rather than looking for a trigger to invert, it operates directly in weight space: given a suspect LoRA checkpoint and the public base model it was fine-tuned from, FreqBrand computes the Spectral Concentration Score (SCS) — the fraction of top singular energy in the fine-tuning weight delta — and compares it against a bootstrap-calibrated threshold derived from clean reference fine-tunes.
The key insight: trigger-free poisoning forces the optimizer to repeatedly reinforce a single recurring visual pattern, concentrating weight updates along a small number of singular directions — a measurable low-rank spike above the Marchenko-Pastur bulk expected under benign fine-tuning.
BreakingBranding/
├── configs/ # Experiment configs and YAML files
│ ├── phase1_pilot.yaml # Phase 1 config (7 models, 128x128, bootstrap)
│ ├── phase2_plan.md # Phase 2 attack variant plan
│ └── n_sweep_hypothesis.md # Pre-registered N-sweep expectations
│
├── results/ # All phase results
│ ├── phase0_residuals/ # Phase 0 gate: BM3D/DnCNN/wavelet preservation
│ ├── phase1_diagnostics/ # N-sweep, patch size, overlap experiments
│ ├── phase1_svd/ # SVD metrics per model (64/128/256 patches)
│ ├── phase1_svd_128/ # PRIMARY: bootstrap detection at 128x128
│ ├── phase2_attack_success/ # OWLv2 attack success per variant
│ ├── phase2_svd/ # Per-variant SVD + bootstrap results
│ └── phase2_5/ # AC/PS/AC-SVD results (all failed)
│
├── scripts/
│ ├── core/ # Main SVD detection pipeline
│ │ ├── svd_patch_analysis.py # Patch SVD, MP fit, bootstrap detection
│ │ ├── n_sweep_analysis.py # Detection vs sample size (N=25..500)
│ │ ├── extract_residuals.py # BM3D sigma=0.25 residual extraction (CPU)
│ │ ├── generate_population.py # Generate N images from any SDXL/custom model
│ │ ├── generate_coco_prompts.py# Sample COCO val2014 captions
│ │ ├── owlv2_scan.py # OWLv2 attack success measurement
│ │ ├── freqbrand_ac.py # Phase 2.5: autocorrelation split-half (failed)
│ │ ├── freqbrand_ps.py # Phase 2.5: power spectrum SVD (failed)
│ │ └── freqbrand_ac_svd.py # Phase 2.5: AC features + SVD (failed)
│ ├── train/ # SLURM finetune scripts
│ │ ├── finetune_poisoned.sh # LoRA finetune on poisoned dataset
│ │ ├── finetune_clean.sh # LoRA finetune on clean subset
│ │ └── finetune_clean_seeds.sh # K=5 clean-FT seed replicates
│ ├── phase0/ # Phase 0 gate scripts (complete)
│ ├── diagnostics/ # Phase 1 diagnostic scripts (complete)
│ ├── dct_pipeline/ # Prior work: DCT + CNN (Tier-3 ablation)
│ ├── failed_methods/ # 7+ methods that didn't work (paper content)
│ ├── analysis/ # One-off analysis scripts
│ ├── setup/ # One-time setup, downloads, logo creation
│ ├── tarot/ # Tarot domain transfer test
│ └── _deprecated/ # Old shell scripts (preserved)
│
├── obsidian-vault/ # Research notes (concepts, papers, daily logs)
├── _archive/ # Pre-pivot versions and old setup files
├── term-cmds.sh # Master SLURM orchestrator (all phases)
├── timeline.md # 15-week timeline to submission
├── CLAUDE.md # Project instructions for Claude Code
├── requirements.txt
└── requirements-frozen.txt # Exact reproducibility pins
Not in repo (too large): checkpoints/, data/, generated images, .cache/, logs/.
Given a suspect LoRA checkpoint and its public base model, FreqBrand extracts the per-layer weight delta for each attention layer:
ΔW_ℓ = W_suspect,ℓ − W_base,ℓ
It then computes the top-50 singular values via randomized SVD and defines the per-layer SCS as the fraction of spectral energy captured by the top 3:
SCS_ℓ = (σ₁² + σ₂² + σ₃²) / (σ₁² + ··· + σ₅₀²)
Scores are aggregated across layers weighted by spectral norm, then compared against a bootstrap-calibrated threshold from K=5 clean reference fine-tunes. No image generation, no GPU, no prompts — the full pipeline runs in seconds on a laptop CPU.
Under benign fine-tuning, the optimizer distributes updates across many directions simultaneously; SCS stays near 3/50 = 0.06. Under poisoning, the optimizer must repeatedly reinforce a single recurring visual pattern (the logo), concentrating updates along a few weight directions and producing a measurable spike. Flynn & Granziol (2025) formalize this as a low-rank perturbation above the Marchenko-Pastur bulk; the BBP phase transition gives the minimum poisoning rate below which the spike is statistically invisible.
Population-level DCT spectra + ResNet-18 classifier. Achieved AUROC = 1.0 with cross-logo and cross-domain generalization. Preserved as a Tier-3 ablation — the CNN detects something structural about logo injection. The SCS method formalizes what the CNN learned with a principled, calibrated threshold.
| Test | Result |
|---|---|
| ResNet-18 AUROC | 1.0000 |
| Permutation test | p = 0.000 |
| 5-fold CV | 1.0 ± 0.0 |
| Cross-logo (Avengers → HF logo) | P(poisoned) = 1.000 |
| Wild model (Juggernaut-XL) | FPR 0% after diverse retrain |
| Population size ablation | AUROC ≥ 0.999 for N ≥ 25 |
| Phase | Name | Status | Key Result |
|---|---|---|---|
| Phase 0 | Residual preservation gate | COMPLETE | BM3D 19/20, DnCNN 14/20, wavelet 8/20. Gate: PROCEED. |
| Phase 0.5 | Eigenvalue baseline | COMPLETE | No spurious spike in base or clean-FT. MP bulk OK. |
| Phase 0.7 | Attack success on COCO prompts | COMPLETE | OWLv2 tau=0.20: poisoned 39%, base 5.5%. |
| Phase 1 | Pilot spectral analysis | COMPLETE | TPR@FPR=5%=100% (N=500), TPR@FPR=1%=100% (N=1000). |
| Phase 1+ | N=1000 extension | COMPLETE | 1% FPR gap closed. Margin=0.115. |
| Phase 2 | Attack variant sweep | COMPLETE | 2/8 variants detected. Detection boundary characterized. |
| Phase 2.5 | Alternative detection methods | COMPLETE | AC/PS/AC-SVD all failed. BM3D-SVD remains only working method. |
| Phase 3 | Baseline comparison | not started | Philip's track. Elijah, T2IShield, Spectral Signatures. |
| Phase 4 | Generalization (multi-dataset) | not started | LAION + Midjourney. Non-negotiable for paper. |
| Phase 5 | Adaptive attacks | not started | Denoiser-aware, sparse poisoning. Min 2 attacks. |
| Phase 6 | Ablations | not started | N-sensitivity, residual extractor, covariance window. |
| Phase 7 | Writing & submission | not started | Target: early August 2026. |
Setup: 7 models (1 poisoned Avengers + 5 clean-FT seeds 42–46 + 1 base SDXL), COCO-prompted images per model, BM3D sigma=0.25 residuals, 128×128 non-overlapping patches, deterministic CPU randomized SVD (seed=42).
| Metric | N=500 | N=1000 |
|---|---|---|
| Detection statistic | σ₁ / σ₂ | σ₁ / σ₂ |
| Suspect ratio (poisoned) | 1.366 | 1.333 |
| Bootstrap 95th pct (5% FPR) | 1.252 | 1.105 |
| Bootstrap 99th pct (1% FPR) | 1.386 | 1.218 |
| TPR at FPR=5% | 100% | 100% |
| TPR at FPR=1% | 0% (gap=0.020) | 100% (margin=0.115) |
| N images | N_eff patches | Poisoned ratio | Max clean ratio | Gap | z-score | Detected? |
|---|---|---|---|---|---|---|
| 25 | 1,600 | 1.038 | 1.158 | −0.120 | −1.2 | NO |
| 50 | 3,200 | 1.067 | 1.070 | −0.002 | 1.4 | NO |
| 100 | 6,400 | 1.019 | 1.067 | −0.048 | −0.7 | NO |
| 250 | 16,000 | 1.200 | 1.067 | +0.133 | 7.9 | YES |
| 500 | 32,000 | 1.367 | 1.053 | +0.314 | 19.8 | YES |
Sharp phase transition at N~250. Below N=100, poisoned is indistinguishable from clean.
| Patch | D | γ | Poisoned ratio | Max clean ratio | Gap | Role |
|---|---|---|---|---|---|---|
| 64×64 | 12,288 | 0.096 | 1.311 | 1.125 | 0.186 | Ablation |
| 128×128 | 49,152 | 1.536 | 1.366 | 1.053 | 0.314 | Primary |
| 256×256 | 196,608 | 24.576 | 1.398 | 1.178 | 0.220 | Interpretability only |
128×128 gives the widest detection margin. γ=1.5 places us in the principled RMT regime.
Setup: Same pipeline as Phase 1 (500 images per model, BM3D sigma=0.25, 128×128 patches, bootstrap from K=5 clean-FT seeds 42–46, 1000 iterations).
| Variant | Axis | Key Change | σ₁/σ₂ | FPR=5% | FPR=1% | Verdict |
|---|---|---|---|---|---|---|
| avengers_default | baseline | 15% area, 100% opacity, semantic placement, ~50% rate | ~1.37 | YES | YES | DETECTED |
| placement_fixed | placement | fixed corner instead of semantic | 1.236 | YES | NO | DETECTED (marginal) |
| size5 | size | 5% logo area | 1.097 | NO | NO | not detected |
| complexity_simple | complexity | solid cyan circle | 1.065 | NO | NO | not detected |
| opacity_low | opacity | 40% opacity | 1.018 | NO | NO | not detected |
| rate10 | rate | 10% poisoning rate | 1.004 | NO | NO | not detected |
| logo_hf | identity | HuggingFace smiley (smooth) | 1.008 | NO | NO | not detected |
| text_logo | modality | "BRANDX" text, random placement | ~1.0 | NO | NO | not detected |
| ext_juggernaut (clean) | external | Juggernaut-XL-v9, no poisoning | 1.008 | NO | NO | clean ✓ |
| ext_juggernaut_poisoned | external | Juggernaut + Avengers LoRA (99.2% ASR) | 1.019 | NO | NO | not detected |
Detection boundary: structured logo + high opacity (≥100%) + poisoning rate (≥50%) + consistent placement. Weakening any single axis drops below threshold.
Phase 2.5 — Alternative Methods (all failed): FreqBrand-AC, FreqBrand-PS, and FreqBrand-AC-SVD all failed because all images from the same diffusion model share ~99.99% consistent structure (VAE decoder patterns, attention artifacts, sampling schedule). The logo is a ~0.01% perturbation invisible at this scale. BM3D-SVD works by subtracting this dominant structure first; split-half methods cannot.
- GPU with ≥ 80GB VRAM for training (A100.80gb on GMU Hopper)
- Python 3.10+, CUDA 12.x
- See
requirements-frozen.txtfor exact package versions
# On Hopper:
cd /scratch/ygoonati/freqbrand
source /scratch/ygoonati/ai/temp/ai-watermark/unmarker-original/img-data/venv-detector-cu121/bin/activate
export HF_HOME=/scratch/ygoonati/freqbrand/.cache/huggingface
# Generate images (7 models x 500 COCO-prompted images)
bash term-cmds.sh phase1gen
# Extract BM3D residuals (CPU, parallelized)
bash term-cmds.sh phase1bm3d
# Run SVD at 128x128 + bootstrap detection
bash term-cmds.sh phase1svd128
# Check results
cat results/phase1_svd_128/bootstrap_test/bootstrap_results.jsonSee term-cmds.sh for the complete SLURM orchestrator. Phases: coco, phase07, phase05, seeds, phase1gen, phase1bm3d, phase1svd, phase1svd128, nsweep128, n1000gen, n1000bm3d, n1000svd.
The Phase 1 results validate the RMT story:
- γ=1.536 at 128×128 (N=500), γ=0.768 at N=1000 — both in the Marchenko-Pastur regime. Bulk eigenvalues follow MP as expected.
- σ₁/σ₂ ratio is the correct detection statistic — raw σ₁ fails because clean models can have higher absolute σ₁ due to higher bulk noise.
- N=1000 closes the 1% FPR gap: suspect ratio 1.333 vs 99th pct threshold 1.218, margin=0.115.
- Tracy-Widom comparison: σ₁ does NOT exceed the MP λ₊ edge at either N. Bootstrap is strictly necessary. Paper framing: "TW assumes i.i.d. entries; BM3D residuals violate this; bootstrap handles the real distribution."
- Key files:
configs/phase1_pilot.yaml,results/phase1_svd_128/phase1_report.md
Phase 3 baseline work can start in parallel. Priority order:
- Elijah — most cited trigger-based defense. Expected: fails completely (no trigger to invert).
- T2IShield — text-trigger defense. Expected: fails (no text trigger).
- Spectral Signatures (Tran et al.) — SVD on representations. Closest to our method but operates on classifier features, not residual covariance. Expected: may partially work.
All baselines run on our existing 7 Phase 1 models. See .claude/context/baselines.md for full setup notes.
- Bootstrap threshold is primary. Tracy-Widom is secondary comparison only.
- Every poisoned model has a matched clean-finetuned control. No exceptions.
- Tier A threat model (auditor has base checkpoint).
- 128×128 patches primary, 64×64 ablation.
- TPR@FPR=5% is headline metric, AUROC is supporting.
- COCO prompts for generation, not logo-biased prompts.
torch
diffusers
transformers
bm3d
numpy
scipy
scikit-learn
matplotlib
Pillow
@inproceedings{silentbranding2025,
title = {Silent Branding Attack},
booktitle = {CVPR},
year = {2025}
}MIT License — see LICENSE for details.