From 9e6e1e32b99878cf84665828163fef46e0bbc201 Mon Sep 17 00:00:00 2001 From: Joaquin Terrasa Date: Wed, 23 Sep 2026 00:59:46 +0200 Subject: [PATCH 1/2] feat(wasm): hostable demo on GitHub Pages Default the demo to the dinov2-with-registers-small GGUF served from Hugging Face (resolve URLs send CORS headers, so no model needs to ship with the page), and add a pages.yml workflow that rebuilds the wasm bundle on pushes to main touching wasm/ and deploys it via actions/deploy-pages. Requires enabling Pages with GitHub Actions as the source in repo settings; documented in docs/wasm.md. --- .github/workflows/pages.yml | 71 +++++++++++++++++++++++++++++++++++++ docs/wasm.md | 23 +++++++++--- wasm/index.html | 12 ++++--- 3 files changed, 97 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/pages.yml diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml new file mode 100644 index 0000000..0d7d280 --- /dev/null +++ b/.github/workflows/pages.yml @@ -0,0 +1,71 @@ +name: pages + +# Builds the wasm demo and deploys it to GitHub Pages so index.html is +# hostable without committing a model (the page defaults to a Hugging Face +# /resolve/ URL, which sends CORS headers). +# +# One-time prereq: in repo Settings -> Pages, set Source to "GitHub Actions". +# The github-pages environment is created automatically on first deploy. +on: + push: + branches: [main] + paths: + - "wasm/**" + - ".github/workflows/pages.yml" + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + with: + submodules: recursive + + - name: Setup emsdk + uses: mymindstorm/setup-emsdk@v14 + with: + version: latest + actions-cache-folder: emsdk-cache + + - name: Configure (wasm preset) + run: cmake --preset wasm + + - name: Build dinov2-wasm + run: cmake --build --preset wasm -j"$(nproc)" + + - name: Stage site + run: | + mkdir -p site + cp wasm/index.html wasm/main.js site/ + cp build-wasm/bin/dinov2-wasm.js build-wasm/bin/dinov2-wasm.wasm site/ + ls -lh site/ + + - name: Configure Pages + uses: actions/configure-pages@v5 + + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@v3 + with: + path: site + + deploy: + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/docs/wasm.md b/docs/wasm.md index b9c34c6..ed4598f 100644 --- a/docs/wasm.md +++ b/docs/wasm.md @@ -52,6 +52,19 @@ embind. For a node smoke test reconfigure with ## Running the demo +### Hosted demo (GitHub Pages) + +`.github/workflows/pages.yml` rebuilds the wasm bundle on every push to +`main` that touches `wasm/` and deploys `index.html`, `main.js` and +`dinov2-wasm.{js,wasm}` to GitHub Pages (it can also be triggered manually +via `workflow_dispatch`). One-time prerequisite: in the repo settings under +**Pages**, set **Source** to "GitHub Actions"; the `github-pages` +environment is created automatically on first deploy. The page defaults to +fetching the `dinov2-with-registers-small-gguf` model straight from Hugging +Face, so no weight needs to be committed or served. + +### Local + The demo (`wasm/index.html` + `wasm/main.js`) expects the build artifacts next to it: @@ -63,12 +76,14 @@ cd wasm && python3 -m http.server 8000 Model source, in the URL field: -- `model.gguf` (default): fetched relative to the page, so drop a GGUF next to +- `https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-small-gguf/resolve/main/model.gguf` + (default): the ViT-S with registers variant, fetched straight from Hugging + Face. Register tokens give cleaner patch heatmaps. huggingface.co + `/resolve/` URLs send CORS headers (verified), so remote URLs work when the + host allows them. +- `model.gguf`: fetched relative to the page, so drop a GGUF next to `index.html`. Get one with `hf download dinov2-cpp-core/dinov2-small-gguf model.gguf --local-dir wasm` (~50 MB f16 ViT-S). -- A remote URL also works when the host sends CORS headers. huggingface.co - `/resolve/` URLs do (verified), e.g. - `https://huggingface.co/dinov2-cpp-core/dinov2-small-gguf/resolve/main/model.gguf`. Then pick two images: the page decodes them via `createImageBitmap` + canvas, encodes each (RGB8/RGBA8 in, preprocessing happens inside the library), and diff --git a/wasm/index.html b/wasm/index.html index a86c4f7..f7de5b9 100644 --- a/wasm/index.html +++ b/wasm/index.html @@ -30,14 +30,16 @@

dinov2.cpp wasm demo

1. Model

- Default model.gguf is fetched relative to this page: drop a GGUF next to - index.html. huggingface.co /resolve/ URLs send CORS headers, so - e.g. https://huggingface.co/dinov2-cpp-core/dinov2-small-gguf/resolve/main/model.gguf - (~50 MB f16 ViT-S) also works directly. + Default is the ViT-S with registers GGUF fetched straight from Hugging Face: + /resolve/ URLs send CORS headers, so the hosted demo works with + no model committed. Register tokens give cleaner patch heatmaps. A relative + model.gguf also works if you drop a GGUF next to + index.html.

From 57cce1dde25c1d6f43e46fa902a346280349c243 Mon Sep 17 00:00:00 2001 From: Joaquin Terrasa Date: Wed, 23 Sep 2026 01:00:58 +0200 Subject: [PATCH 2/2] ci(extras): visual-regression end-to-end proof job Add tiny synthetic fixture screenshots under examples/ci-visual-regression/fixtures/ (generated by make_fixtures.py, Pillow only) and a continue-on-error visual-regression job that materializes baseline/ + screenshots/ from them, then runs the same release-tarball, checksum, embed and cosine-gate steps as the example workflow. Two nearly identical pairs pass and a screenshots-only settings.png reports NEW, so the public run log shows the gate working end to end without going red. --- .github/workflows/extras.yml | 100 ++++++++++++++++ examples/ci-visual-regression/README.md | 25 +++- .../fixtures/baseline/detail.png | Bin 0 -> 1810 bytes .../fixtures/baseline/home.png | Bin 0 -> 1366 bytes .../fixtures/screenshots/detail.png | Bin 0 -> 1815 bytes .../fixtures/screenshots/home.png | Bin 0 -> 1368 bytes .../fixtures/screenshots/settings.png | Bin 0 -> 1782 bytes .../ci-visual-regression/make_fixtures.py | 111 ++++++++++++++++++ 8 files changed, 233 insertions(+), 3 deletions(-) create mode 100644 examples/ci-visual-regression/fixtures/baseline/detail.png create mode 100644 examples/ci-visual-regression/fixtures/baseline/home.png create mode 100644 examples/ci-visual-regression/fixtures/screenshots/detail.png create mode 100644 examples/ci-visual-regression/fixtures/screenshots/home.png create mode 100644 examples/ci-visual-regression/fixtures/screenshots/settings.png create mode 100644 examples/ci-visual-regression/make_fixtures.py diff --git a/.github/workflows/extras.yml b/.github/workflows/extras.yml index 51f22d9..ec15aa1 100644 --- a/.github/workflows/extras.yml +++ b/.github/workflows/extras.yml @@ -133,3 +133,103 @@ jobs: run: | bash <(curl -s https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash) ./actionlint -oneline examples/ci-visual-regression/visual-regression.yml + + # End-to-end proof of examples/ci-visual-regression/visual-regression.yml: + # materializes baseline/ + screenshots/ from the committed fixtures, then + # runs the same release-tarball + cosine-gate steps so the public run log + # shows the gate working on real images. Fixtures are synthetic UI mockups + # (regenerate with examples/ci-visual-regression/make_fixtures.py): two + # nearly identical pairs pass, and a screenshots-only settings.png is + # reported NEW instead of failing, so a healthy run stays green. + visual-regression: + continue-on-error: true + runs-on: ubuntu-latest + timeout-minutes: 15 + env: + THRESHOLD: "0.98" + MODEL_REPO: "dinov2-cpp-core/dinov2-small-gguf" + steps: + - uses: actions/checkout@v4 + + - name: Materialize baseline/ and screenshots/ from fixtures + run: | + cp -r examples/ci-visual-regression/fixtures/baseline baseline + cp -r examples/ci-visual-regression/fixtures/screenshots screenshots + find baseline screenshots -type f | sort + + - name: Download dinov2-cli release binary + run: | + TAG=$(basename "$(curl -sIL -o /dev/null -w '%{url_effective}' \ + https://github.com/espetro/dinov2.cpp/releases/latest)") + curl -LO "https://github.com/espetro/dinov2.cpp/releases/download/$TAG/dinov2-$TAG-bin-ubuntu-x64.tar.gz" + curl -LO "https://github.com/espetro/dinov2.cpp/releases/download/$TAG/SHA256SUMS.txt" + + - name: Verify checksums and unpack + run: | + TARBALL=$(ls dinov2-*-bin-ubuntu-x64.tar.gz) + grep " $TARBALL\$" SHA256SUMS.txt > check.sums + sha256sum -c check.sums + tar xzf "$TARBALL" + find . -name dinov2-cli -type f + + - name: Download GGUF weight + run: | + mkdir -p models + curl -L -o models/model.gguf \ + "https://huggingface.co/$MODEL_REPO/resolve/main/model.gguf" + + - name: Embed baseline and screenshot fixtures + run: | + CLI=$(find . -name dinov2-cli -type f | head -1) + mkdir -p emb + embed() { + args=() + while IFS= read -r f; do args+=(-i "$f"); done \ + < <(find "$1" -maxdepth 1 \( -name '*.png' -o -name '*.jpg' \) | sort) + if [ "${#args[@]}" -eq 0 ]; then + echo "no images in $1/"; return 0 + fi + "$CLI" -m models/model.gguf --preprocess crop518 --batch 8 \ + --print-embeddings "${args[@]}" > "$2" + } + embed baseline emb/baseline.jsonl + embed screenshots emb/current.jsonl + + - name: Compare embeddings (cosine >= THRESHOLD) + run: | + python3 - <<'PY' + import json, math, os, sys + + def load(path): + out = {} + for line in open(path): + r = json.loads(line) + out[os.path.basename(r["image"])] = r.get("pooled") or r["cls"] + return out + + def cosine(a, b): + dot = sum(x * y for x, y in zip(a, b)) + na = math.sqrt(sum(x * x for x in a)) + nb = math.sqrt(sum(x * x for x in b)) + return dot / (na * nb) if na and nb else 0.0 + + threshold = float(os.environ["THRESHOLD"]) + base = load("emb/baseline.jsonl") + curr = load("emb/current.jsonl") + failed = [] + for name, bvec in sorted(base.items()): + if name not in curr: + print(f"MISSING {name}: no current screenshot") + failed.append(name) + continue + s = cosine(bvec, curr[name]) + print(f"{'ok' if s >= threshold else 'FAIL'} {s:.4f} {name}") + if s < threshold: + failed.append(name) + for name in curr: + if name not in base: + print(f"NEW {name}: add to baseline/ if intentional") + if failed: + print(f"\n{len(failed)} screenshot(s) below {threshold}: {failed}") + sys.exit(1) + PY diff --git a/examples/ci-visual-regression/README.md b/examples/ci-visual-regression/README.md index faf7d1e..2d42848 100644 --- a/examples/ci-visual-regression/README.md +++ b/examples/ci-visual-regression/README.md @@ -4,9 +4,28 @@ A copy-paste GitHub Action that uses release `dinov2-cli` builds as a visual regression gate: screenshots from your PR are compared against committed baselines by embedding cosine similarity instead of pixel diffs. -**This is an example, not a gate on this repo.** It is not referenced by any -workflow in `.github/workflows/`; copy `visual-regression.yml` into your own -project and adapt it. +**This is an example, not a gate on this repo.** `visual-regression.yml` is +not referenced directly by `.github/workflows/`; copy it into your own +project and adapt it. It does get exercised end to end though: the +`visual-regression` job in `.github/workflows/extras.yml` (advisory, +`continue-on-error`) materializes `baseline/` and `screenshots/` from the +committed `fixtures/` and runs the same download, embed and cosine-gate +steps, so every extras run log is a live proof that the gate works. + +## Fixtures + +`fixtures/` holds tiny synthetic UI mockups generated by +`make_fixtures.py` (Pillow only, no fonts): + +- `baseline/home.png` + `screenshots/home.png` and + `baseline/detail.png` + `screenshots/detail.png`: nearly identical pairs + (a 2 px element nudge stands in for font/AA jitter), so both pass the + 0.98 gate. +- `screenshots/settings.png`: clearly different, with no baseline pair, so + the comparison prints it as `NEW` instead of failing. A healthy proof run + stays green. + +Regenerate with `python3 examples/ci-visual-regression/make_fixtures.py`. ## What it does diff --git a/examples/ci-visual-regression/fixtures/baseline/detail.png b/examples/ci-visual-regression/fixtures/baseline/detail.png new file mode 100644 index 0000000000000000000000000000000000000000..6ff338c8465e4c09e0c134d53ffadfb277ced4af GIT binary patch literal 1810 zcmeAS@N?(olHy`uVBq!ia0y~yV4MKNIvh+uk)*3dQyCc8ZhE>nhE&XXdsi_}CS9W8 zp=anbZIKWyEsbpd1c@6SY>bSS4me=7U#|Nnk|x|p!o1jPu4wgg7WCe{=VoE)3t)l~})wct~NqS)N&miP3+%86u*%$k|)9(Mz>rwu!-hV#)EpL~9^Sx}o{q)7`*|V0-ultjD zG`*Q||NoDb@23CQd-wVIr}jTRIPmxg#kuI5jnBkRewZ>h{;utxO19B>l*@>k3KyR2)@ zRsH;RR;2LnySHZgDTkVFPCMiEY>g_|TQ{m)j8b@oLGkv0>hX&JiH`R@oKf9 zvkxR*?1T>SY@eRKdX0z0ldePG&E=QNR+N6Mxy1{PI)n3{a@XecnEHT%xyjdEhhe4ZNQib4O5XRa((!FcK>C*J-aIG-nMaqa>I=DpWDLB zj!CJ3g7Dxn~Y><-gfjxTpO(6+?@u`*hCX zrpMQ^epJXAB_8b9_-ju5U0J*Iy)}Q+x^~okFPkk66wdKZQ!Ty*j7ne<&aqXJ+|CXzTzrXkE zz10NUyXFc0tan)Wq8S?3(-9X`9zD9=mJ);$kab^2%FRtD2U}t(m*V@L|QgbN6qDZvy2n zPVQ?9eWeTd@DIQ#NCN-T>%d^+*QiTNAc~ujGoLtbHfW**D-jy`njxgN@xNALSp1B literal 0 HcmV?d00001 diff --git a/examples/ci-visual-regression/fixtures/baseline/home.png b/examples/ci-visual-regression/fixtures/baseline/home.png new file mode 100644 index 0000000000000000000000000000000000000000..0c37c76858e9a9052657a4c15e86a9eca1cf6dbb GIT binary patch literal 1366 zcmeAS@N?(olHy`uVBq!ia0y~yV4MKNIvh+uk)*3dQyCaoWj$RSLn`LHy{nrS5-8E| z(0U19@9GSrRg0#vGOXivP|%8Ckn^|Yb_h|3aFD&rQ08(~rNc_+a)?sFihp+=o~wRd znKIe3c=o(KKh<8&++6YX<#&dF4>_;c8FuI&WHMsmJ|Uo^;Jl%M2PdbexqaV*SC2m{ zKKlCe*~{)9wa@>4UwX8KHHAZLf?|Y2TLPnG6HX3y_*G_6H3Bvu>`vaebMNe5k__*N zH6ORl5Sc=QYfi7{nNBL8Tw$M51{~@EPHn6k21@v(TC^E z*=sBNt&NjFj3sYcY*}48k3pi9kVW%#%8#+`eAe;YTiYyve(HO;tqw_-VoLu{$VX6hE&XXdsi_}CS9W8 zp=anbZIKWyEsbpd1c@6SY>bSS4m6G(m8_wV-0>-ZVi4HFq9 zn^;pg#3m?4IJ6~@&8gXaO}O^o=k;6XaZUU2@b`N@`y1!q+U?hO=f7s+YybDh;jZ&+ z%ys`Cf4-ytv3U3S_owSWO%xz1V6X&P@>$(U4>hg#@2dXMzI}T7@(TU`(M_!Go>!;y zpS%{u!reD%ReSr%YenZ04;E&d*4S9j+EBN%vT)V(Wlt|3ZO^Pf_Nut9wEV}Mub;lo z)_nBq?(N*T6D_Qpb!SdJ6X^{OiVa^oQcg&zfr8{9XPdI~H3@j+9bRhr;YxUN+YCsI z)~K{d=INggi=0?+LbUaLzMY@-hgXk&ZjlB@UBbMlx1(-KWX}MH?Cd^|W2;qJxVOFT zi_%qzFv!TOy^>{lS}NgAZL?hL3xjE^V%MtQ;5^u*oOU{7tKPw;(u~V&a}MK6HdsOj zk<>Q08m&1z@13-OPQ)wi^0PbZPn5mS7SxgW{(^VkJ{zBH(~9}H`=VDrd|V0izA|N@ zawxFk)oMd$A4n>)6FS5LjG5JIJS?7c9r|uAzg)JW^kdB}USUu|GdTY#cWq9OsShX! zn|$3>9$yt|V!d7JzE(^~N5W*gUDnn)sk{d7_Br#RuO-*<1#^nt=Rr=9V7wgwoLz$AR5%Ec&!R~VF2AFv#r zsF>XY&qj$4HEh0KouLTL#`2trZMRQv4h>V0DRO=Idv^b2zCF7t?B2G4bMB1vpWDLB zjzLr1GUZ7hvw%tA?XJaP+?--Ov!dm&(f^$*PLZe8yEVe9rfwS=dM4{;7li5>Ob1z3 zAX0W}`Gj^_r6QZJ32#$+IgfuX`M1ATHG0{q(wXQ1ho!9UnY^~7-s*ZTLpHTaOK=%; zVDg@cuin?!#C^P(AYGT|Jn!z`zvoxpXFlLqqr!rxrofx!k)1z7>mTc)XrmYB7iq)- PD?A2IS3j3^P6-Ln`LHy{qUalPJ;f z@FrXPy44w*Rx#aXp0Z1hA+YfUi(l zQsv}7>jO{j<_p#RJGy&)J$plwcyK(!fm%+n35pR8Z3&E$O{^&#I5}@}e>HXM|8_p| z@Z_(D-9PG{|NXu6XbV0a2(!4|uQH3O=_ojFXy7@>WW>UKLI58pdE>sFbAL%Oyd&0p zybcFCW25;sr`50TC+o7FIsfVF>U5Dci4uJGblhWjXY`(4eO2Rig$fHnSK)Phk0JlU z8z+wY&tG1Z{p{`W@A<*;O#Q?kRW$`&$Sn7)10V+pzn) z{oU~E`;SL;bv=C@etYkQyPKM)-M**WU&?Xx;OwUzt3RHx{cuLG{IuUrnU^h6-|Z=HB}@>hY9kE!*qyl=SH-`g+S6!-6s=~vY=y|@D&C8?_2{*xna_&r_R@c&Fg zDOv6IEava!Rb}svfBn+eZ^v`K2WZKRoI{^x?mRtz@&;L9oEh!LEa!&(wNLGVDhzM7 zJT(@^moOmiJG|1e*x+}q-?@k?mE&8jj}yw453k5D6!s1Tng@uq%)nfiuw;aXru z>sW1=|4wY%hKT!j@;+NNs6U@5fWIUl&ent#KiD(4gC2`!S-JyDOa@O^KbLh*2~7a= CUg3BE literal 0 HcmV?d00001 diff --git a/examples/ci-visual-regression/fixtures/screenshots/settings.png b/examples/ci-visual-regression/fixtures/screenshots/settings.png new file mode 100644 index 0000000000000000000000000000000000000000..6c0eb63bb4e5e9cf7bb4a22d9e07fdf9a123af98 GIT binary patch literal 1782 zcmeAS@N?(olHy`uVBq!ia0y~yV4MKNIvh+uk)*3dQyCc8_IkQFhE&XXdv{}2$ZM&F zhbNCk>FjB6P!JH}SR%rG`mT&cxX_V9-5UfxdhxwUn=Y0lxnY9Cx`R$m3Yz@V+^so_ zjI|Tjmj8RMscxKIx2)o4+SB=4%gVm@h244i?AIxE7Vi(o?{G8xh;3p`;Sigk7~#;C zz$i&R=VQiQ@i+2w);!WXcV6z@ga5x@zi+p;DEa%{zTVHigOmG&fR2Lmh6bL4OhzoE zbGCg~6XCC7XK%2cL4+AMZr-(?Rn6;NUR-PXU8td+7(FoMT^HV;)&DUQ&f1WMo zTzF}3)N0N3O&SNIS~Txo)Hw!@zO(tiI#Ny~2>>JU&6{_!XZN-(-Mw>P)sEXWcOMr# zv2W9vdAo)&`Quap)VL_v+BQ>mp4|P@=h{lYo7hzUOZqiCZ@$00{h7Ti?~*|N&91v_ zUUSRnlmaN`*0|0vQj%OR(Vn1tH0^GncAHWJ_olqsD_s|FT-%^?U;4}GUcR8CGg!2z z>8?ylS=Vm*`Rj%Rf$8`60h2*7u2i;Rul3#U{C;7D;?L8I^%-xppd^@>xU^1tPxgOP zQA7zPqr*>A-`#wy?*IPxgbloU?;oG?&SpO?E7P#^hZhI_B>!-GOLNuFS6}aVKWHz{ zzyHIct{rHzL89*U>EGw)+gDUn*&L5b5M{WZCnWO6R?Yu46} zpBMUVn3lBpSB99gATYQt$(~J;0p+Y~f0p%owCS(`W9!(tclR#d$&T1H`}OMf-Dj`c z^0}8;Zuq+Wr%@tiF5D4y_za(a|NGSR!@H`b=GoR8{`K8{etEzAvpA-5!$ij0)`@SS zSugCiY06DtO!ZEdMP?=6U**F0Sd zn8r5dc#0|uuX$Q^!R)4@=R};bgi%T$`(o#GhCiUhOnhPTG27I=7c4Vi7P8 zEtfw+i~;wB_+C9-eZ5$}QU2cE+Q*N7$g1fm9B$bf6MwIM-`@uhUw)LyZgdq`~j5&v2UCny@+<7i25r4g?15Cu-21~7H0mBNE z#DDtAPvnW@aoW((GxzS^#arVH9&BqviK&#l&#XO1!<|seG+>fTE32rg(YKbqcK7m! zH$S^y=I^SttNml}U1&!)DD?u3zW*Wfv_>*(F_uH`_%|PYW^Jv~_*E=>qn>a$-D>KHa|llD&|R=(4n+ ta;EA(71