Skip to content

Batched export indexing, Add Missing Search Previews, optional AuraFace face recognizer - #460

Merged
bmachek merged 10 commits into
mainfrom
claude/zen-keller-ezc6eq
Oct 30, 2026
Merged

bmachek merged 10 commits into
mainfrom
claude/zen-keller-ezc6eq

Conversation

@bmachek

@bmachek bmachek commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

1. Indexing exports in batches, with fixed settings

  • macOS pipeline: it used one LrExportSession for the whole selection. Lightroom keeps every rendition until a session ends, so its memory climbed through long runs. It now opens one session per batch (default 25, Photos per export batch in Plug-in Manager). The producer waits while a batch's worth of JPEGs is still queued, so temp files cannot pile up.
  • Windows / sequential path: before, every photo got its own session, so nothing rendered in parallel. Now the next batch is rendered in one session and consumed photo by photo. It stays a single worker.
  • Pixel-relevant settings are pinned (7574d29): colour space sRGB and no enlarging of photos smaller than the export size. Two exports of the same photo then always match, instead of depending on Lightroom's defaults. Settings are built fresh per session (exportSettingsFor).
  • JPEG quality on Lightroom's 0–1 scale (a383823): LR_jpeg_quality got the preference in percent unchanged, and Lightroom takes anything above 1 as full quality, so every export was written at 100% (1.44 MB vs 160 KB for a 2048 px raw at 50%). This predates the PR; Util.jpegQualityFromPercent converts it now, and the preview task passes 0.8.
  • Lightroom previews are not used (dropped in caf262c). requestJpegThumbnail delivers a few and then stops answering, and it returns whatever size is cached, so embeddings drift. Reading Previews.lrdata / .lrprev directly has the same size and staleness problems, plus an undocumented format.

2. Add Missing Search Previews

  • Backend:
    • POST /v1/photos/thumbnails/missing lists the indexed photos without a search-page preview.
    • POST /v1/photos/thumbnail (multipart) stores one for an indexed photo, scaled to the indexing size. Photos without an index row get 404 not_indexed.
  • Plugin: a new task, Add Missing Search Previews…, makes small batched exports (512 px) and uploads them. Failures are listed in the final dialog. It sits in the Help menu since 1ae4a82, which also moved Save Edits as AI Training Examples and Deduplicate Keyword Synonyms; docs, the search page banner and messages follow that.

3. Optional AuraFace recognizer

  • lrg-ml:
    • Recognizer {FaceNet, AuraFace}, each with its own face_model id.
    • AuraFace uses the existing 112×112 ArcFace crop, RGB, (v-127.5)/127.5; the input name is read from the graph.
    • detect_faces_with_model returns the id it actually used.
  • API:
    • GET/POST /v1/faces/recognizer: saved next to the models and restored at start; 409 until the model is downloaded.
    • /v1/faces/recognizer/downloads: downloads fal/AuraFace-v1/glintr100.onnx (Apache-2.0) from Hugging Face, pinned to commit af6d057c9b0ec4071d4c49c80e3539258798b609 (08000d7).
  • Names survive a switch: re-detection carries person assignments from the other recognizer's rows by box overlap (same YuNet detector, IoU ≥ 0.6).
  • /v1/faces/search now ranks only rows of the query's model, and says in warnings how many it left out (bedb2d6): right after a switch it used to answer "ok" with no results. Find Similar Faces shows those warnings.
  • Plug-in Manager: a Face recognition box to switch between FaceNet and AuraFace.
  • Numerics verified (08000d7): scripts/auraface_reference.py recorded InsightFace's own embedding (InsightFace picks mean/std 127.5, as auraface.rs assumes); auraface_matches_insightface_reference passes at cosine 0.9988 (needs > 0.99).

Tested live in Lightroom 15.6 (test catalog, 197 raws, macOS)

  • In-Lightroom plugin tests 24/24, catalog-write tests 4/4.
  • Batched indexing: 157 photos in 7 batches, 396 s; at most 11 JPEGs in temp, none afterwards. Same run with one session (batch size 200): 405 s and the same memory (median 15 GB, p90 18 GB). At this size batching makes no measurable difference; whether it keeps memory down over thousands of photos is still open.
  • Two exports of the same photo differ by at most 2–3 levels on under 0.01% of pixels.
  • Search previews: missing-list and upload routes incl. every error case; 10 previews regenerated through the plugin functions, matching the originals.
  • AuraFace: download 260 MB (sha256 matches the pinned commit); FaceNet → AuraFace → FaceNet with "New or unprocessed photos" re-detects all 197 photos each way, and every person keeps exactly its photos (86/86, 10/10, 6/6).

Known issue, not fixed here

  • Cancel does not stop rendering. rendition:skipRender() always raises "must not be called after exportSession has started rendering" (swallowed by pcall), so on cancel Lightroom keeps rendering the rest of the batch for ~30 s and those JPEGs stay in the temp folder (15 files / 47 MB in the test). Same in exportPhotosForIndexing. The old rendition:cancel() did not work either. Fix: waitForRender() and delete the file instead.

Still to note

  • Small photos: with no enlarging, photos smaller than the export size now index at their own size instead of an upscaled one. Their embeddings change slightly on the next re-index, as they do from the JPEG quality fix.
  • The recognizer choice is per computer (~/.cache/lrgenius/models/face_recognizer.txt), while faces are stored per database: switching in one catalog sets aside the faces of every other catalog until it is re-indexed.
  • Cluster thresholds are FaceNet's for both recognizers. In a small sample, AuraFace at the default 0.5 left 70 of 135 faces unassigned (FaceNet: 33). Named people are not affected (pinned).

Checks

  • cargo fmt --check and cargo clippy --workspace --all-targets are clean.
  • cargo test --workspace passes; face goldens pass with LRG_REQUIRE_GOLDENS=face,auraface.
  • luacheck and stylua pass; busted 613/614 locally. The one failure, aesthetic_refine_spec, comes from the local Lua 5.5 (1.0 vs 1) and is untouched by this PR.
  • scripts/check-docs.py reports no errors.

https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C

🤖 Generated with Claude Code

https://claude.ai/code/session_01HaMMvtFGGZAyc9nbCZiG6T

…exing

Photos indexed before the backend stored search-page previews had no way to
get one short of sending them through Analyze & Index again.

Backend: POST /v1/photos/thumbnails/missing lists the indexed photos (of a
selection or a catalog) without a preview; POST /v1/photos/thumbnail stores
one for an indexed photo from any rendered JPEG, scaled to the indexing size,
and refuses photos that have no index row.

Plugin: Library > Plug-in Extras > Add Missing Search Previews asks which
photos lack one, takes Lightroom's cached preview (requestJpegThumbnail) when
it is at least 256 px, and exports small JPEGs in chunked export sessions for
the rest. LightroomPreviews.lua wraps requestJpegThumbnail (deadline, size
check via Util.jpegDimensions); export settings are now built per session.

Docs-Reviewed: Dev-Backend-API.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C
One LrExportSession over the whole selection (the macOS pipeline) kept every
rendition until the run ended, so Lightroom's memory climbed through long
runs; on Windows each photo got its own session, so nothing rendered in
parallel. Both paths now export in batches (25 by default, 'Photos per export
batch' in Plug-in Manager): the macOS producer opens one session per batch
and waits while a batch's worth of JPEGs is still queued, the sequential path
renders the next batch in one session and consumes it photo by photo.

New opt-in 'Use Lightroom's previews where they are large enough': indexing
takes the cached preview (requestJpegThumbnail) when its long edge is at
least 3/4 of the export size and exports the rest; the completion dialog says
how many were exported and how to build previews first.

Docs-Reviewed: Dev-Backend-API.md
Docs-Reviewed: Getting-Started.md
Docs-Reviewed: Help-Local-AI-Models.md
Docs-Reviewed: Help-Other-AI-Server.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C
FaceNet stays the default. AuraFace (fal/AuraFace-v1, glintr100.onnx,
Apache-2.0, ~260 MB) can be downloaded from Hugging Face and selected in
Plug-in Manager > Face recognition.

lrg-ml: Recognizer {FaceNet, AuraFace} with its own model id
(yunet-2023mar/auraface-v1); AuraFace runs on the canonical 112x112 ArcFace
crop, RGB, (v-127.5)/127.5, the recognizer input name read from the graph.
detect_faces_with_model loads and runs under one lock and returns the id
the embeddings were made with, so a concurrent switch cannot mislabel rows.

lrg-api: GET/POST /v1/faces/recognizer (saved in face_recognizer.txt next
to the models, restored at start; 409 until downloaded) and
/v1/faces/recognizer/downloads (download_url_assets, generalised from the
release downloader). Re-detection carries person assignments from the
other recognizer's rows by box overlap (same detector, IoU >= 0.6), so
names survive a switch. /v1/faces/search only ranks rows of the query's
model. Health reports face_recognizer.

scripts/auraface_reference.py records InsightFace's own embedding of the
golden face; the AuraFace golden test compares against it and skips until
that reference is committed.

Docs-Reviewed: Help-Choosing-AI-Model.md
Docs-Reviewed: Help-Other-AI-Server.md
Docs-Reviewed: Troubleshooting.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C
requestJpegThumbnail delivers a few previews and then stops answering, and
what it delivers comes in whatever size is cached: embeddings and aesthetic
scores made from those drift from the ones made from exports. Indexing and
Add Missing Search Previews now always export, in batches. Removes
LightroomPreviews.lua, the Plug-in Manager option, Util.jpegDimensions and
Util.previewFallbackWarning; the docs explain why previews are not used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C
@bmachek bmachek changed the title Faster export indexing, Add Missing Search Previews, optional AuraFace face recognizer Batched export indexing, Add Missing Search Previews, optional AuraFace face recognizer Oct 30, 2026
bmachek and others added 6 commits October 30, 2026 05:34
Indexing exports now state the colour space (sRGB) and never enlarge a photo
smaller than the export size, so two exports of the same photo always match
instead of depending on Lightroom's defaults.

The menu rearrangement moved Add Missing Search Previews, Save Edits as AI
Training Examples and Deduplicate Keyword Synonyms to the Help menu; the
wiki, README, search page banner, the no_thumbnail error and the AI Edit
hint now say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvqfgsKcoNWQaKy3c9C
…ision

scripts/auraface_reference.py, run against fal/AuraFace-v1 at
af6d057c9b0ec4071d4c49c80e3539258798b609 (glintr100.onnx, sha256
a7933ea5330113b01c9b60351d8f4c33003f145d8470ac5f0e52ee2effe25c60, the same
file the backend's download fetches), wrote auraface_embedding into
face_goldens.json. InsightFace picked input_mean/std 127.5, as auraface.rs
assumes, and the Rust chain matches the reference at cosine 0.9988
(auraface_matches_insightface_reference, needs > 0.99), so the golden test
no longer skips once the model is downloaded.

The download is pinned to that commit instead of `main`, so a later push to
the Hugging Face repository cannot change the model under a verified chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvtFGGZAyc9nbCZiG6T
LR_jpeg_quality runs from 0 to 1, and Lightroom takes anything above 1 as
full quality. The "Export JPEG quality in percent" preference went through
unchanged, so every indexing and AI export was written at 100%: measured in
Lightroom 15.6 on a 2048 px raw export, 1.44 MB at "50" against 160 KB at
0.5. The Add Missing Search Previews task passed 80 the same way.

Util.jpegQualityFromPercent converts the preference (clamped to 1-100, 60
when unset, as before), exportSettingsFor applies it per session, and the
preview task passes 0.8. Exports get several times smaller; embeddings
shift slightly the next time a photo is indexed, which Help-Performance-Tips
now says, together with the real defaults (3072 px, 50%).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvtFGGZAyc9nbCZiG6T
After a switch between FaceNet and AuraFace, /v1/faces/search skips the
rows the other recognizer embedded, as it should, but said nothing: right
after a switch it answered "ok" with no results, which reads as "nobody
matches". It now counts the rows it left out and says so in `warnings`,
with what to run to bring them back (none searched at all, or N of M).

Find Similar Faces read only the old single `warning` field. It now goes
through Util.responseWarnings, shows the warnings in place of the generic
"no similar faces" text when nothing was found, and as a warning dialog
next to a result otherwise.

Checked in Lightroom 15.6 against the running backend: with AuraFace active
and only FaceNet rows stored, 0 results and the "None of the 135 faces"
warning; with FaceNet, 20 results and no warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvtFGGZAyc9nbCZiG6T
Once the `id` index covers a table, LanceDB runs merge_insert as a hash
join that loads its whole input into a fixed 150 MB pool it cannot spill
out of. Face clustering writes every face back in one upsert, and at about
2.5 KB a face a catalog with ~60,000 faces failed with "Resources
exhausted … HashJoinInput".

Store::upsert now builds its input into batches of at most 32 MB of row
data (UPSERT_CHUNK_BYTES) and runs one merge_insert per batch. Each is a
commit of its own, so a failure partway leaves the earlier chunks written.
Raising the pool through LANCE_MEM_POOL_SIZE would only move the limit.

Tests cover the split (updated and new rows, one commit per chunk) and an
oversized record; a 70,000-face reproduction is #[ignore]d. The server-rs
README's memory section notes the write ceiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvsPvtNfysffwnVi9vb
@bmachek
bmachek merged commit ebe4894 into main Oct 30, 2026
4 of 5 checks passed
@bmachek
bmachek deleted the claude/zen-keller-ezc6eq branch October 30, 2026 19:09
bmachek added a commit that referenced this pull request Oct 30, 2026
#460 added the paragraph on the upsert write ceiling (UPSERT_CHUNK_BYTES)
to server-rs/README.md without rerunning scripts/build-wiki-pages.sh, so
check-docs.py has failed on main since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvuCsggAAAZqFC3FdDX
bmachek added a commit that referenced this pull request Oct 30, 2026
* deps: bump rustls to 0.23.46 for four security advisories

Fixes Dependabot alerts #79-#82, all in rustls < 0.23.46, which the
backend pulls in only through reqwest/hyper-rustls/tokio-rustls:

- GHSA-c8m5-c952-6cm4 (medium): Debug formatting of a session exposes
  TLS master secrets and resumption PSKs
- GHSA-h2v8-394m-p7vv (low): Brotli certificate decompression can
  allocate up to 1 GiB before authentication
- GHSA-w44r-93j7-qv28 (low): an unauthenticated ServerHello poisons the
  key-exchange record, so later ClientHellos send only X25519
- GHSA-vp88-798v-h67f (low): ECH rejection can expose buffered data to
  the outer server

Lockfile change is rustls alone; cargo update also re-resolved
tempfile's getrandom (0.4.3 -> 0.3.4), which was dropped as unrelated.
Builds with --locked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvuCsggAAAZqFC3FdDX

* docs: regenerate Dev-Server-README.md from server-rs/README.md

#460 added the paragraph on the upsert write ceiling (UPSERT_CHUNK_BYTES)
to server-rs/README.md without rerunning scripts/build-wiki-pages.sh, so
check-docs.py has failed on main since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvuCsggAAAZqFC3FdDX

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bmachek added a commit that referenced this pull request Oct 31, 2026
* ci: fetch AuraFace for the nightly golden tests

#460 added the `auraface` golden family together with a committed reference
embedding, so `LRG_REQUIRE_GOLDENS=all` now demands auraface_glintr100.onnx —
but golden-tests-full.yml never downloaded it, and the nightly went red
(auraface_matches_insightface_reference: assets missing).

AuraFace is not a release asset; the server pulls it from Hugging Face at a
pinned commit. The workflow now does the same, reading repo, revision, file
and sha256 from `auraface_source` in face_goldens.json (the source the
reference was made from) and failing on a hash mismatch.

Docs list `auraface` among the LRG_REQUIRE_GOLDENS family names
(CLAUDE.md, AGENTS.md, CONTRIBUTING.md, server-rs/README.md,
Dev-Server-README.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvs5Zqtm9RhCkDHer9F

* docs: the nightly golden job fetches every family, not three

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HaMMvs5Zqtm9RhCkDHer9F

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants