GGGT

GGGT is a research codebase for AIGC image detection. The current v2 line focuses on prototype-based real/fake separation with DINOv3 and CLIP backbones, source-aware fake prototypes, LoRA/EFFORT-SVD/Stiefel projector variants, and evaluation/visualization on GenImage, Chameleon, UniversalFakeDetect, and Community Forensics.

This repository intentionally tracks source code, configs, and reports only. Local checkpoints, pretrained weights, extracted embeddings, SwanLab logs, and dataset caches are ignored because they are large and machine-specific.

Repository Layout

• configs/: training configs for the DINOv3, CLIP, multilayer, Stiefel, source-prototype, and VGGT-Omega-style variants.
• datasets/: GenImage and cached dataset loaders plus source-balanced samplers.
• losses/: real-prototype contrastive loss, center loss, frequency alignment, and distribution-alignment utilities.
• models/: DINOv3/CLIP backbones, LoRA, EFFORT-SVD, projector heads, sample pools, and local/global adapter blocks.
• scripts/: training, prototype precomputation, evaluation, embedding extraction, and visualization entry points.
• utils/: metrics, prototype loading, pretrained path resolution, and logging helpers.
• report.md: running experiment table and analysis notes.
• modify.md: change log for implementation edits.

Local Files Not Tracked

The following paths are expected to exist locally but are not committed:

• pretrained_weight/: downloaded DINOv3 and CLIP weights, for example dinov3-l16, dinov3-b16, and clip-vit-large-patch14.
• data/: prototypes, extracted embeddings, evaluation JSON files, and generated visualizations.
• checkpoints*/: experiment checkpoints.
• swanlog/: local SwanLab run backups.

Move or regenerate these artifacts on a new machine before running experiments.

Environment

The project currently uses Python 3.12. pyproject.toml is minimal, so install the research stack explicitly in the environment you use for training:

uv venv --python 3.12
source .venv/bin/activate
uv pip install torch torchvision transformers safetensors pyyaml tqdm numpy scipy scikit-learn matplotlib pillow pandas swanlab

For multi-GPU training, make sure the installed PyTorch build matches the CUDA version on the machine.

Data Expectations

Configs use absolute dataset paths from the development machine. Update them before running elsewhere. Common paths used by the current experiments include:

• /home/data/JinchengLiu/GenImage/
• /home/data/JinchengLiu/Chameleon/
• /home/data/JinchengLiu/UniversalFakeDetect/
• /home/data/JinchengLiu/CommunityForensics-Eval/
• /home/data/JinchengLiu/imagenet-1k/

Pretrained weights are resolved through utils/pretrained_paths.py; place them under pretrained_weight/ or edit the relevant config path.

Prototype Precomputation

Real-prototype based variants require prototypes produced with the same backbone and feature space used during training:

torchrun --standalone --nproc_per_node=8 scripts/precompute_prototype.py --config configs/train.yaml
torchrun --standalone --nproc_per_node=8 scripts/precompute_multilayer_prototype.py --config configs/train_multilayer_rpc_svd_dinov3_8gpu.yaml

Regenerate prototypes after changing backbone architecture, token layout, projector space, or checkpoint feature output.

Training

Most training entry points use scripts/train.py with a YAML config:

CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --standalone --nproc_per_node=4 scripts/train.py \
  --config configs/train_layer18_lora24_stiefel_rpc256_frozenout_semantic0.1_ema0.2_dinov3_4gpu_b128.yaml

The VGGT-Omega-style source adapter has a separate entry point:

CUDA_VISIBLE_DEVICES=0,1 torchrun --standalone --nproc_per_node=2 scripts/train_vggt_omega.py \
  --config configs/train_vggt_omega_dinov3_b_rpc_sourceproto_2gpu_b64.yaml

Check the config before launching:

• experiment_name controls SwanLab naming and checkpoint directory naming.
• backbone selects DINOv3 or CLIP.
• real_prototype_path must match the feature space used by the run.
• use_blending, lambda_rpc, lambda_semantic, lambda_bce, and source-prototype settings define the active objective.

Evaluation

Typical Chameleon evaluation is done by extracting checkpoint embeddings and then computing AUC/ACC from the saved shards:

torchrun --standalone --nproc_per_node=4 scripts/extract_checkpoint_embeddings_multigpu.py \
  --config configs/train_layer18_lora24_stiefel_rpc256_frozenout_semantic0.1_ema0.2_dinov3_4gpu_b128.yaml \
  --checkpoint checkpoints_layer18_lora24_stiefel_rpc256_frozenout_sem0.1_ema0.2_nobce_dinov3_4gpu_b128/checkpoint_epoch_0008.pt \
  --dataset-root /home/data/JinchengLiu/Chameleon/ \
  --output-dir data/extracted_embeddings/chameleon_example

python scripts/eval_chameleon_from_embedding_shards.py \
  --embedding-dir data/extracted_embeddings/chameleon_example

Community Forensics has a dedicated extractor:

torchrun --standalone --nproc_per_node=4 scripts/extract_community_forensics_embeddings_multigpu.py \
  --config <config.yaml> \
  --checkpoint <checkpoint.pt> \
  --dataset-root /home/data/JinchengLiu/CommunityForensics-Eval/ \
  --output-dir data/extracted_embeddings/community_forensics_example

Visualization

Visualization scripts save plots under data/visualizations/ by default. Examples:

python scripts/visualize_genimage_tsne.py --help
python scripts/visualize_universalfakedetect_object_tsne.py --help
python scripts/visualize_community_forensics_generators_projected_tsne.py --help

Use cached embeddings when possible; full image forward passes are much slower than plotting from saved feature shards.

Current Research Directions

Implemented variants include:

• RPC-only and RPC plus center loss baselines.
• Real-pool blending with detached pool entries.
• Layer 18 to 24 LoRA adaptation with frozen-output semantic consistency.
• Fixed and trainable Stiefel subspace projectors with RPC in the null-space feature.
• Source-prototype aggregation with EMA updates.
• A VGGT-Omega-style local/global adapter over frozen DINOv3-B final-layer tokens.

See report.md for experiment outcomes and modify.md for implementation-level change history.