SN-VSD: Generalizable AI-Generated Image Detection

This repository studies detection of AI-generated images from generators that were not seen during detector training.

The current direction is decoder-and-detector-guided diffusion augmentation. A frozen Stable Diffusion 2.1 model constructs fake-origin counterexamples that a frozen RGB detector snapshot predicts as real. The accepted images are then intended to be used as hard fake examples for detector training.

known fake image
    -> frozen SD2.1 VAE encoder
    -> partial forward noise
    -> frozen SD2.1 U-Net
    -> predicted clean latent
    -> frozen SD2.1 VAE decoder
    -> frozen RGB detector snapshot
    -> guidance toward the real label
    -> decoded hard fake image

No SD2.1 component is fine-tuned. During guidance, gradients pass through the U-Net, VAE decoder, and detector only to the current noisy latent.

Read this file before running experiments. Model weights, datasets, checkpoints, generated images, logs, caches, and API keys must not be committed.

1. Research problem

A binary detector trained on one generator can learn that generator's easiest fingerprint rather than evidence shared by generated images. It may therefore perform extremely well in-domain while predicting unseen fake images as real.

The existing one-epoch ResNet-50 baseline demonstrates this gap:

Evaluation	AP	AUROC	Accuracy	Fake recall
AIGIBench validation, micro	0.9998	0.9997	0.9937	--
AIGIBench test, micro	0.7462	0.7148	0.6299	0.3107
AIGIBench test, macro over 25 generators	0.7008	0.7025	0.6266	0.3068

The goal is to improve unknown-generator fake recall without creating an unacceptable real-image false-positive rate.

2. Current proposal

The guidance classifier is not a small latent detector. It is:

clean latent z0_hat
    -> frozen SD2.1 VAE decoder
    -> RGB image in [-1, 1]
    -> differentiable ResNet preprocessing
    -> frozen baseline ResNet-50
    -> real/fake logit

The ResNet checkpoint uses:

real = 0
fake = 1

Therefore, the guidance loss targets the real class:

L_guide = BCEWithLogits(D_g(VAE.decode(z0_hat)), 0)

This replaces the previous objective:

old:
    guide a small latent detector toward p_fake = 0.5

current:
    guide the decoded RGB image toward y = real

The intended training curriculum is:

Warm-up epoch:
    train RGB detector D on clean real and SD1.4 fake images

Guided epochs:
    freeze a snapshot D_g
    retain ordinary real/fake training
    select 25 percent of fake samples
    generate short SD2.1 img2img counterexamples against D_g
    detach accepted decoded images
    train D to classify them as fake

Fake-origin guidance is the primary configuration. Guiding a real image toward the real label usually leaves it unchanged and does not create a hard negative.

Full motivation, losses, controls, risks, and experiment requirements are in proposal2.md.

3. What has been implemented

3.1 ResNet-50 baseline

The baseline is an ImageNet-pretrained ResNet-50 trained for one epoch on the AIGIBench training mixture.

checkpoint:
    checkpoints/baseline_resnet50_b64_1ep/best.pt

trainer:
    scripts/train_baseline.py

evaluator:
    scripts/evaluate_baseline.py

Checkpoints are local and intentionally excluded from GitHub.

3.2 Previous latent ADBE experiment

The earlier implementation used:

image
    -> SD2.1 VAE latent
    -> small latent detector
    -> guidance toward p_fake = 0.5

Relevant historical code:

models/latent_boundary_detector.py
scripts/train_genimage_adbe.py
scripts/evaluate_genimage_adbe.py
scripts/generate_adbe_guided_samples.py

The five-epoch experiment completed, but later guided epochs did not improve aggregate seven-generator performance over the epoch-1 warm-up checkpoint. Several decoded boundary-guided samples contained severe mixed-texture artifacts. Treat this as a negative or inconclusive mechanism result, not the current proposed method.

3.3 Decoded ResNet guidance diagnostic

The current diagnostic is implemented in:

scripts/generate_resnet_real_guided_samples.py

It loads the frozen SD2.1 VAE and U-Net plus the frozen baseline ResNet-50. For each predicted clean latent, it decodes an RGB image and minimizes the detector's real-label loss.

The first 5-real/5-fake diagnostic used:

inference steps:        25
img2img strength:       0.4
maximum guided steps:   10
guidance scale:         0.15
stop threshold:         p_fake <= 0.10

Four attempted fake images reached the real target after one guidance step:

p_fake 1.000 -> 0.001
p_fake 0.128 -> 0.003
p_fake 0.918 -> 0.000
p_fake 0.108 -> 0.000

One additional fake image was already predicted real. All sampled real images were confidently predicted real and were unchanged.

The generated images were less chaotic than the previous latent-boundary samples, but some became smooth, painterly, or structurally distorted. This is evidence that gradient routing works; it is not evidence that training on the images improves unknown-generator detection. The rapid one-step confidence collapse may be classifier-specific reward hacking.

Generated diagnostic files remain local under:

artifacts/resnet_real_guided_samples/

Do not commit that directory.

4. Important interpretation

The guided image is not sampled from the unmodified SD2.1 distribution.

ordinary sampler:
    SD2.1 denoising direction

guided sampler:
    SD2.1 denoising direction + detector-dependent correction

Using a frozen U-Net does not guarantee that the altered trajectory remains in a high-density region of the SD2.1 image distribution. A VAE decoder can render an off-manifold latent.

The mandatory native-distribution control is:

generate many images with ordinary SD2.1
    -> score completed images with the detector
    -> retain naturally difficult samples
    -> train them as fake

If native hard-sample mining matches gradient guidance, the external gradient is unnecessary.

5. Repository status

Completed

ResNet-50 baseline training and AIGIBench evaluation
SD2.1 latent ADBE five-epoch training
seven-generator GenImage evaluation for preserved ADBE checkpoints
ten-sample latent-guidance visual diagnostic
ten-sample decoded-ResNet real-target diagnostic

Implemented but not integrated into training

decoded VAE + ResNet real-target sample generator
best-candidate tracking
real-target early stopping
local image and trajectory metadata output

Not implemented

offline guided-fake cache builder at training scale
RGB detector trainer consuming guided fake images
latent/pixel/perceptual trust-region acceptance
independent-detector transfer evaluation
native SD2.1 hard-sample-mining baseline
full robustness and unknown-generator comparison

6. Repository map

README.md
    current project handoff and operating instructions

proposal2.md
    current decoder-and-detector-guided augmentation proposal

proposal.md
    older source-erasure/VSD proposal; historical context

implementation_experiment_plan.md
    older implementation plan; not the current execution specification

experiment_record.md
    experiment ledger and comparison tables

literature_review.md
literature table.xlsx
    AIGC detection literature review

scripts/generate_resnet_real_guided_samples.py
    current decoded-ResNet guidance diagnostic

scripts/train_baseline.py
scripts/evaluate_baseline.py
    RGB ResNet-50 baseline

scripts/train_genimage_adbe.py
scripts/evaluate_genimage_adbe.py
scripts/generate_adbe_guided_samples.py
models/latent_boundary_detector.py
    previous latent ADBE implementation

models/stable-diffusion-2-1-base/
    local SD2.1 components; weights excluded from Git

paper/
    local literature PDFs and notes; not intended for Git

7. Environment setup

This repository uses uv. Do not create a Conda environment or install project dependencies using direct pip commands.

uv venv --python 3.10
uv sync

Run Python through the managed environment:

uv run python -c "import torch; print(torch.__version__); print(torch.cuda.is_available())"

The existing experiments used an RTX 5080 with 16 GB VRAM and BF16.

GPU access may be unavailable inside a restricted Codex sandbox even when nvidia-smi can query the host. Run CUDA experiments with the required device permission rather than changing the PyTorch installation immediately.

8. Dataset layout

Datasets live outside the repository.

Windows:
    F:\datasets

WSL:
    /mnt/f/datasets

Expected GenImage SD1.4 structure:

/mnt/f/datasets/GenImage/stable_diffusion_v_1_4/
    imagenet_ai_0419_sdv4/
        train/
            ai/
            nature/
        val/
            ai/
            nature/

The most recent audit found SD1.4, SD1.5, Midjourney, Wukong, ADM, VQDM, and BigGAN. GLIDE was missing. Re-audit the dataset before relying on that list.

Do not train on final unseen-generator test splits.

9. SD2.1 components

The local model directory must contain:

models/stable-diffusion-2-1-base/
    scheduler/
    text_encoder/
    tokenizer/
    unet/
    vae/
    model_index.json

Model weights are excluded from Git. The local components can be reconstructed through hf-mirror:

HF_ENDPOINT=https://hf-mirror.com uv run hf download Manojb/stable-diffusion-2-1-base \
  --revision 0094d483a120f3f33dafbd187ea4aa60d10de75c \
  --local-dir models/stable-diffusion-2-1-base \
  --include model_index.json \
  --include scheduler/scheduler_config.json \
  --include 'tokenizer/*' \
  --include text_encoder/config.json \
  --include text_encoder/model.fp16.safetensors \
  --include unet/config.json \
  --include unet/diffusion_pytorch_model.fp16.safetensors \
  --include vae/config.json \
  --include vae/diffusion_pytorch_model.fp16.safetensors

10. Run the current diagnostic

The default command generates five real-origin and five fake-origin examples:

UV_CACHE_DIR=.uv-cache uv run python -m scripts.generate_resnet_real_guided_samples

Explicit form:

UV_CACHE_DIR=.uv-cache uv run python -m scripts.generate_resnet_real_guided_samples \
  --data-root /mnt/f/datasets/GenImage/stable_diffusion_v_1_4/imagenet_ai_0419_sdv4 \
  --model-root models/stable-diffusion-2-1-base \
  --resnet-checkpoint checkpoints/baseline_resnet50_b64_1ep/best.pt \
  --real-count 5 \
  --fake-count 5 \
  --inference-steps 25 \
  --strength 0.4 \
  --max-guidance-steps 10 \
  --guidance-scale 0.15 \
  --target-fake-probability 0.10 \
  --output-dir artifacts/resnet_real_guided_samples

The script writes original crops, VAE reconstructions, guided images, guided latents, a contact sheet, and metadata. These are diagnostics and must remain outside Git history.

11. Reproduce the baseline

Training:

UV_CACHE_DIR=.uv-cache uv run python scripts/train_baseline.py \
  --data-root /mnt/f/datasets/AIGIBench \
  --epochs 1 \
  --batch-size 64 \
  --num-workers 8 \
  --output-dir checkpoints/baseline_resnet50_reproduction

Evaluation:

UV_CACHE_DIR=.uv-cache uv run python scripts/evaluate_baseline.py \
  --data-root /mnt/f/datasets/AIGIBench \
  --checkpoint checkpoints/baseline_resnet50_reproduction/best.pt \
  --output results/baseline_resnet50_reproduction_aigibench_test.json \
  --batch-size 128 \
  --num-workers 8 \
  --threshold 0.5

12. Required next experiments

Run these in order:

1. Generate at least 500 fake-origin trajectories while sweeping weaker
   guidance scales and one to ten steps.

2. Quantify content preservation, latent displacement, target success, visual
   rejection, runtime, and peak VRAM.

3. Test whether guided images also fool an independent non-ResNet detector.

4. Implement matched augmentation controls: VAE reconstruction, unguided
   img2img, Gaussian/pixel distortion, and native SD2.1 hard-sample mining.

5. Train matched RGB detectors with clean versus guided fake examples.

6. Evaluate every model on all available unseen GenImage generators with one
   fixed threshold.

7. Add JPEG, resizing, blur, screenshot, crop, and color robustness tests.

The immediate scientific question is:

Do detector-guided samples expose a transferable synthetic-image blind spot,
or only an adversarial weakness of the ResNet used for guidance?

13. Evaluation policy

Use one fixed binary threshold, normally 0.5, for every final test generator. Do not select a separate threshold for each generator.

Always report:

per-generator AP and AUROC
accuracy and balanced accuracy
real accuracy and false-positive rate
fake recall
macro-generator and micro-image means
worst-generator performance

For guidance also report:

attempted and accepted trajectories
initial and final p_fake
guidance steps
latent, pixel, and perceptual displacement
independent-detector transfer
visual rejection rate
runtime and peak VRAM

Record completed experiments in experiment_record.md and the corresponding machine-readable result table. Do not present an incomplete run as a completed comparison.

14. Common pitfalls

• requires_grad=False freezes model weights; it does not permit wrapping the guidance forward pass in torch.no_grad. Autograd through the U-Net, decoder, and detector is required to obtain a latent gradient.
• The detector must score the predicted clean decoded image, not a noisy latent.
• The detector preprocessing must exactly match baseline training.
• A lower ResNet fake probability is not proof of greater realism.
• One-step confidence collapse can indicate adversarial reward hacking.
• VAE decodability is not evidence of high SD2.1 probability.
• Keep clean real and fake losses when training on guided examples.
• Use fake-origin guidance as the primary method.
• Validate hardness using an independent detector.
• Compare against native SD2.1 hard-sample selection.
• High SD1.4 validation accuracy is not evidence of unknown-generator generalization.
• Never commit datasets, model weights, checkpoints, artifacts, caches, logs, SwanLab data, or secrets.

15. Reading order for a new agent

1. README.md
   Current state, commands, and operating constraints.

2. proposal2.md
   Current decoded-detector guidance hypothesis and required controls.

3. scripts/generate_resnet_real_guided_samples.py
   Implemented mechanism diagnostic.

4. experiment_record.md
   Completed experimental evidence and metric conventions.

5. scripts/train_baseline.py and scripts/evaluate_baseline.py
   RGB detector checkpoint format and preprocessing.

6. scripts/train_genimage_adbe.py
   Historical latent-guidance implementation and diffusion utilities.

7. literature_review.md and literature table.xlsx
   Related AIGC detection work.

8. proposal.md and idea.md
   Historical source-erasure and VSD directions only.

Keep the README, proposal, implementation state, and experiment ledger synchronized. A successful detector attack is not yet a successful detector training method.