Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ on:
required: false
default: "12"
type: string
platform:
description: "GitHub-hosted runner label for the bench job"
required: false
default: "ubuntu-latest"
type: choice
options:
- ubuntu-latest
- ubuntu-24.04-arm

# Bench is on-demand: each workflow_dispatch run is independent and
# parallel-safe (different `runs` get different run IDs and thus
Expand All @@ -41,7 +49,7 @@ permissions:

jobs:
bench:
runs-on: ubuntu-latest
runs-on: ${{ inputs.platform }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -126,6 +134,7 @@ jobs:
env:
BENCH_MODELS: ${{ steps.variants.outputs.BENCH_MODELS }}
BENCH_VARIANT_INPUT: ${{ inputs.variant }}
BENCH_PLATFORM: ${{ inputs.platform }}
run: |
set -euo pipefail
provenance_path=benchmark_provenance.txt
Expand All @@ -138,7 +147,7 @@ jobs:
printf 'models=%s\n' "$BENCH_MODELS"
printf 'image_path=assets/tench.jpg\n'
printf 'build_configuration=Release\n'
printf 'platform=ubuntu-latest\n'
printf 'platform=%s\n' "$BENCH_PLATFORM"
printf 'quantization=f16 only\n'
} > "$provenance_path"
variant_list="${BENCH_MODELS//,/ }"
Expand All @@ -164,17 +173,20 @@ jobs:
BENCH_MODELS: ${{ steps.variants.outputs.BENCH_MODELS }}
BENCH_REPEATS: ${{ inputs.repeats }}
BENCH_THREADS: ${{ inputs.threads }}
BENCH_PLATFORM: ${{ inputs.platform }}
BENCH_BUILD_CONFIG: Release
run: |
set -euo pipefail
echo "models selected: $BENCH_MODELS"
echo "repeats: $BENCH_REPEATS"
echo "threads: $BENCH_THREADS"
echo "platform: $BENCH_PLATFORM"
chmod +x scripts/bench.sh
bash scripts/bench.sh \
--models "$BENCH_MODELS" \
--repeats "$BENCH_REPEATS" \
--threads "$BENCH_THREADS" \
--platform "$BENCH_PLATFORM" \
--image assets/tench.jpg \
--bin build/bin/dinov2-cli \
--out benchmark_results.txt
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ each input gets one JSON line on stdout, identical to running it alone
./bin/dinov2-cli -m models/model.gguf -i a.jpg -i b.jpg --batch 2 --print-embeddings
```

That's it. Up to **3x faster than PyTorch on CPU** with up to **4x less memory** (see [docs/benchmarks.md](docs/benchmarks.md)).
That's it. Up to **3x faster than PyTorch on CPU** with up to **4x less memory** (i9-14900HX, see [benchmarks methodology](docs/benchmarks.md)).

## C API

Expand Down Expand Up @@ -138,6 +138,11 @@ Opt-in surfaces, off by default and not covered by the stability contract
| **PyTorch-parity outputs** | CLS + patch embeddings as JSON, matching the reference implementation. |
| **Cross-platform prebuilts** | macOS arm64, Linux x64/arm64, Windows x64. |

**Limitations:** image decode is vendored stb_image only (JPEG, PNG, BMP,
TGA; no HEIC, WebP, RAW, or AVIF, convert those first with `sips`,
`heif-convert`, or `ffmpeg`). Embeddings are image-to-image: this is not
CLIP and there are no text queries. Classification is ImageNet-1k only.

## Pre-converted GGUF weights

Ready-to-download f16 GGUF weights, published by CI to the [`dinov2-cpp-core`](https://huggingface.co/dinov2-cpp-core) Hugging Face profile:
Expand Down
19 changes: 19 additions & 0 deletions benchmark_results.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# ===== platform: ubuntu-latest =====
# Generated by scripts/bench.sh on 2026-09-21T17:14:58Z. Actual Ubuntu workflow evidence.
# Platform: ubuntu-latest (GitHub-hosted runner), reported runner: linux-x86_64
# Git SHA: 9d0a6265ae53088e423e4c2417ec8589de181e99
Expand All @@ -14,3 +15,21 @@

# Measurement: forward-pass wall time from dinov2-cli; image load, preprocessing, classification, and PCA are excluded.
# Unmeasured: register-token variants, base/large/giant sizes, non-Ubuntu platforms, GPU backends, and quantization comparisons.

# ===== platform: ubuntu-24.04-arm =====
# Generated by scripts/bench.sh on 2026-09-22T22:59:52Z. Actual Ubuntu ARM64 workflow evidence.
# Platform: ubuntu-24.04-arm (GitHub-hosted runner), reported runner: linux-aarch64
# Git SHA: 1f5c16148f75fc3b75fe5ff964943d406f3bfd10
# Build configuration: Release
# Precision: f16 GGUF; backend: CPU; batch: 1
# Image path: assets/tench.jpg
# Methodology: 5 timed runs after 1 warmup; threads: 4.
# Command: scripts/bench.sh --models small --repeats 5 --threads 4 --platform ubuntu-24.04-arm --image assets/tench.jpg --bin build/bin/dinov2-cli --out benchmark_results.txt
# Workflow run: https://github.com/espetro/dinov2.cpp/actions/runs/35794974726

| Model | GGUF path | GGUF SHA256 | mean (ms) | stddev (ms) | min (ms) | max (ms) | peak RSS (MB) |
|:------|:----------|:------------|----------:|------------:|---------:|---------:|---------------:|
| small | models/dinov2-vit-small-patch14/model.gguf | b7ca009aa416f6be85ea95363f4ef13d3c139999f96ca3fd47226849afb712e6 | 133.8 | 8.2 | 128 | 148 | 96 |

# Measurement: forward-pass wall time from dinov2-cli; image load, preprocessing, classification, and PCA are excluded.
# Unmeasured: register-token variants, base/large/giant sizes, non-Ubuntu platforms, GPU backends, and quantization comparisons.
50 changes: 40 additions & 10 deletions docs/benchmarks.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@
# Benchmarks

The current committed benchmark evidence is Ubuntu-only, CPU-only, and generated by
`.github/workflows/bench.yml` on a GitHub-hosted `ubuntu-latest` runner. It is
not a multi-platform performance claim. No quantization comparison is included.
The current committed benchmark evidence is Ubuntu-only (x64 and arm64),
CPU-only, and generated by `.github/workflows/bench.yml` on GitHub-hosted
`ubuntu-latest` and `ubuntu-24.04-arm` runners. It is not a multi-platform
performance claim. No quantization comparison is included.

## Current evidence

The following row is an actual measurement from workflow run
[35630688827](https://github.com/espetro/dinov2.cpp/actions/runs/35630688827),
not a seed or estimate.
The following rows are actual measurements from workflow runs
[35630688827](https://github.com/espetro/dinov2.cpp/actions/runs/35630688827)
and
[35794974726](https://github.com/espetro/dinov2.cpp/actions/runs/35794974726),
not seeds or estimates.

| Model | GGUF | Precision | Backend | Mean (ms) | Stddev (ms) | Min (ms) | Max (ms) | Peak RSS (MB) |
|:------|:-----|:----------|:--------|----------:|------------:|---------:|---------:|---------------:|
| small | `dinov2-cpp-core/dinov2-small-gguf` | f16 | CPU | 255.0 | 9.1 | 249 | 271 | 104 |
| Model | Platform | GGUF | Precision | Backend | Mean (ms) | Stddev (ms) | Min (ms) | Max (ms) | Peak RSS (MB) |
|:------|:---------|:-----|:----------|:--------|----------:|------------:|---------:|---------:|---------------:|
| small | `ubuntu-latest` (x86_64) | `dinov2-cpp-core/dinov2-small-gguf` | f16 | CPU | 255.0 | 9.1 | 249 | 271 | 104 |
| small | `ubuntu-24.04-arm` (aarch64) | `dinov2-cpp-core/dinov2-small-gguf` | f16 | CPU | 133.8 | 8.2 | 128 | 148 | 96 |

### Provenance
The two rows use different thread counts (12 on x64, 4 on arm64, matching the
runner CPU counts) and different microarchitectures; they are not directly
comparable as an x64-versus-arm64 claim.

### Provenance: `ubuntu-latest` row

- Date: 2026-09-21T17:14:58Z
- Platform: `ubuntu-latest`, reported by the runner as `linux-x86_64`
Expand All @@ -30,6 +38,25 @@ not a seed or estimate.
- Timed runs: 5
- Measurement: forward-pass wall time from `dinov2-cli`; image loading, preprocessing, classification, and PCA are excluded
- Command: `scripts/bench.sh --models small --repeats 5 --threads 12 --image assets/tench.jpg --bin build/bin/dinov2-cli --out benchmark_results.txt`
- Workflow run: [35630688827](https://github.com/espetro/dinov2.cpp/actions/runs/35630688827)

### Provenance: `ubuntu-24.04-arm` row

- Date: 2026-09-22T22:59:52Z
- Platform: `ubuntu-24.04-arm`, reported by the runner as `linux-aarch64`
- Repository SHA: `1f5c16148f75fc3b75fe5ff964943d406f3bfd10`
- GGUF path: `models/dinov2-vit-small-patch14/model.gguf`
- GGUF SHA-256: `b7ca009aa416f6be85ea95363f4ef13d3c139999f96ca3fd47226849afb712e6`
- Image: `assets/tench.jpg`
- Precision: f16 GGUF
- Backend: CPU
- Threads: 4
- Batch: 1
- Warmup: 1
- Timed runs: 5
- Measurement: forward-pass wall time from `dinov2-cli`; image loading, preprocessing, classification, and PCA are excluded
- Command: `scripts/bench.sh --models small --repeats 5 --threads 4 --platform ubuntu-24.04-arm --image assets/tench.jpg --bin build/bin/dinov2-cli --out benchmark_results.txt`
- Workflow run: [35794974726](https://github.com/espetro/dinov2.cpp/actions/runs/35794974726)

The runner's exact result and checksum provenance are retained in the workflow
artifact alongside the committed result. Runner scheduling and CPU frequency
Expand All @@ -47,6 +74,9 @@ gh workflow run bench.yml --ref feat/batched-inference \
-f variant=small -f repeats=5 -f threads=12
```

Add `-f platform=ubuntu-24.04-arm` to run the same job on the ARM64 runner
(the committed arm64 row above used `-f threads=4` to match its 4 CPUs).

The workflow supports the regular `small`, `base`, `large`, and `giant` models
and their explicitly named `small-registers`, `base-registers`,
`large-registers`, and `giant-registers` variants. `all` currently means the
Expand Down
9 changes: 9 additions & 0 deletions examples/dedup/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ open dupes.html
grouping merely similar images. Tune on a small folder first.
- `-o/--out`: where the HTML review page goes (default `dedup-report.html`).

## Supported formats

The CLI decodes images with vendored stb_image, so only `.jpg`, `.jpeg`,
`.png`, `.bmp`, and `.tga` are scanned. WebP, HEIC, RAW, and AVIF files are
skipped by `dedup.py` and would fail decode in `dinov2-cli` anyway; convert
them first. On macOS, `sips -s format jpeg in.heic --out out.jpg` covers
HEIC/WebP/RAW; elsewhere `heif-convert in.heic out.jpg` (libheif) or
`ffmpeg -i in.webp out.jpg` do the same.

The script uses `pooled` (`[cls || mean(patches)]`) when present and falls
back to `cls`. For large folders the pairwise pass is O(n^2) in Python; a
few thousand images is fine, hundreds of thousands will want a real index
Expand Down
4 changes: 3 additions & 1 deletion examples/dedup/dedup.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
import sys
from pathlib import Path

EXTS = {".jpg", ".jpeg", ".png", ".bmp", ".tga", ".webp"}
# Formats decodable by vendored stb_image (v2.30). WebP, HEIC, RAW, and AVIF
# are not supported: convert them first (sips, heif-convert, ffmpeg).
EXTS = {".jpg", ".jpeg", ".png", ".bmp", ".tga"}


def embed(cli: str, model: str, images: list[Path]) -> list[dict]:
Expand Down
Loading