Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/golden-tests-full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ jobs:
--pattern 'face_mesh_v2.onnx' \
--pattern 'face_blendshapes_v2.onnx'

# RF-DETR-Seg Nano (the subject for culling), published by
# model-assets-subject.yml; its goldens are committed
# (testdata/subject_goldens.json).
gh release download subject-assets-v1 --repo "$GITHUB_REPOSITORY" --dir "$dest" \
--pattern 'subject_rfdetr_seg_nano.onnx'

gh release download bioclip-assets-v1 --repo "$GITHUB_REPOSITORY" --dir "$dest" \
--pattern 'bioclip2_image_fp16.onnx' \
--pattern 'bioclip2_taxa.bin' \
Expand Down
89 changes: 89 additions & 0 deletions .github/workflows/model-assets-subject.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
name: Export Subject Model Assets

# RF-DETR-Seg Nano (~123 MB) for the backend's subject detection
# (server-rs/crates/lrg-ml/src/subject.rs), which culling uses to measure focus
# on the subject and on the background. Same reasoning as the other model
# jobs: the upstream model does not change per app release, so this publishes
# to a stable tag that only moves when the export changes.
#
# The checkpoint comes from rfdetr (pinned in the script's PEP 723 header),
# pinned again by SHA-256 in the export script. The rfdetr README states that
# RF-DETR-Seg, every size, is Apache 2.0. The script compares the export with
# the PyTorch model on the reference portrait before anything is written.
#
# Its **own** tag, like the face-state family: without the subject model
# culling judges sharpness frame-wide, and no other family may report "not
# ready" because of it. If the export changes in a way older binaries cannot
# read, bump both this workflow's `tag_name` and `SUBJECT_ASSETS_RELEASE_TAG`
# in routes/assets.rs together. That is a different question from
# `lrg_ml::subject::MODEL_ID` and `SUBJECT_VERSION`, which mark stored readings
# as no longer comparable.
#
# macos-latest for the same reason as the other model jobs: ubuntu-latest
# intermittently segfaulted during ONNX Runtime verification.
on:
workflow_dispatch: {}
push:
branches: [main]
paths:
- 'server-rs/scripts/export_subject_rfdetr.py'
- '.github/workflows/model-assets-subject.yml'

# Publishes to a FIXED release tag with overwrite_files: true, so two runs are
# delete-then-upload racing each other. Never cancelled: cancelling
# mid-upload is the one thing that could truncate a published asset.
concurrency:
group: model-assets-subject
cancel-in-progress: false

permissions:
contents: read

jobs:
export-subject-assets:
runs-on: macos-latest
timeout-minutes: 45
name: Export RF-DETR-Seg Nano
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Setup uv
uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
with:
enable-cache: true

# --no-project: the script carries its own PEP 723 dependencies (rfdetr
# with its torch and transformers pins), which do not belong in the
# shared server-rs/scripts export project.
- name: Export the subject model
run: |
set -euo pipefail
uv run --no-project --python 3.12 server-rs/scripts/export_subject_rfdetr.py \
--output-dir subject-assets
echo "=== Exported subject assets ==="
ls -lh subject-assets/

- name: Publish to the stable subject-assets release
uses: softprops/action-gh-release@8a0cd651e5b2365bd640854a3fa5dfc70e4875f3 # v3.0.4
with:
tag_name: subject-assets-v1
name: Subject model assets (RF-DETR-Seg Nano)
body: |
ONNX asset for the backend's subject detection, fetched by
`POST /v1/models/assets/downloads`.

- `subject_rfdetr_seg_nano.onnx` — RF-DETR-Seg Nano (Roboflow),
COCO objects with masks from a 312 px square, Apache-2.0

See `server-rs/scripts/export_subject_rfdetr.py` and
`.github/workflows/model-assets-subject.yml`.
files: subject-assets/*
draft: false
prerelease: false
make_latest: false
overwrite_files: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ cargo run -p lrg-server -- --db-path /path/to/lrgenius.db --debug

**The ML golden tests skip when their model files are absent, and a skip is
reported as a pass.** Set `LRG_REQUIRE_GOLDENS` to the families whose assets
are present (`face`, `auraface`, `facestate`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a
are present (`face`, `auraface`, `facestate`, `subject`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a
failure — this is what stops CI being green on numerical checks that never ran.
PR CI provisions the face pair (89 MB) and requires it; the nightly
`golden-tests-full.yml` fetches every family. If you add a golden test, gate it
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ cargo run -p lrg-server -- --db-path /path/to/lrgenius.db --debug

**The ML golden tests skip when their model files are absent, and a skip is
reported as a pass.** Set `LRG_REQUIRE_GOLDENS` to the families whose assets
are present (`face`, `auraface`, `facestate`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a
are present (`face`, `auraface`, `facestate`, `subject`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a
failure — this is what stops CI being green on numerical checks that never ran.
PR CI provisions the face pair (89 MB) and requires it; the nightly
`golden-tests-full.yml` fetches every family. If you add a golden test, gate it
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ must survive every edit.
- **Lightroom Tests over MCP**: With Lightroom open (the agent bridge starts with it) and `LRGENIUS_MCP_TOKEN` set to the agent token from the Plug-in Manager, `cd server-rs && cargo test -p lrg-mcp --test live_lightroom -- --ignored --test-threads=1` runs the plugin's in-Lightroom tests through the MCP endpoint (see `docs/wiki/Dev-Agent-Bridge.md`). They work on whichever catalog is open, so use a test catalog that holds copies of your photos, above all with `LRG_LIVE_WRITES=1`, which changes photos. AI coding agents run the full live test from `.claude/skills/lrgenius-mcp-full-test/SKILL.md` and must not test in your production catalog without your approval.
- **Plugin Unit Tests**: Pure Lua logic (helpers in `Util.lua`, keyword and photo-id handling) is tested headlessly with [busted](https://lunarmodules.github.io/busted/): run `busted` from the repo root (`luarocks install busted`). Specs live in `plugin/spec/`, `.busted` configures them, and `plugin/spec/spec_helper.lua` stubs the Lightroom SDK. CI runs them via `lua-tests.yml`. Add specs when you touch pure helpers.
- **Backend Tests**: `cd server-rs && cargo test --workspace`.
- The ML golden tests **skip when their model files are absent, and a skip is reported as a pass** — so a green run on a machine without the model assets has not checked the model numerics. Set `LRG_REQUIRE_GOLDENS` to the families whose assets you have (`face`, `auraface`, `facestate`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a failure. PR CI requires `face`; the nightly `golden-tests-full.yml` runs `all`. See [`server-rs/README.md`](server-rs/README.md#golden-tests-and-lrg_require_goldens) for fetching the assets.
- The ML golden tests **skip when their model files are absent, and a skip is reported as a pass** — so a green run on a machine without the model assets has not checked the model numerics. Set `LRG_REQUIRE_GOLDENS` to the families whose assets you have (`face`, `auraface`, `facestate`, `subject`, `siglip`, `aesthetic`, `bioclip`, or `all`) to turn that skip into a failure. PR CI requires `face`; the nightly `golden-tests-full.yml` runs `all`. See [`server-rs/README.md`](server-rs/README.md#golden-tests-and-lrg_require_goldens) for fetching the assets.

---

Expand Down
24 changes: 24 additions & 0 deletions docs/doc-sources.toml
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,18 @@ sources = [
"server-rs/crates/lrg-api/src/species_links.rs",
]

# The benchmark page reports numbers measured with these; a change to how
# they score or what the script makes is a change to the page.
[page."Dev-Image-Culling-Benchmark.md"]
sources = [
"server-rs/scripts/cull_bench.py",
"server-rs/crates/lrg-analysis/src/eval.rs",
"server-rs/crates/lrg-analysis/examples/cull_eval.rs",
"server-rs/crates/lrg-analysis/src/selection.rs",
"server-rs/crates/lrg-llama/examples/tiebreak_probe.rs",
"server-rs/testdata/cull_bench/fixtures/*.json",
]

[page."Help-Cull-Photos.md"]
sources = [
"plugin/LrGeniusAI.lrdevplugin/TaskCullPhotos.lua",
Expand All @@ -167,6 +179,18 @@ sources = [
"server-rs/crates/lrg-imaging/src/focus.rs",
"server-rs/crates/lrg-ml/src/face_state.rs",
"server-rs/crates/lrg-analysis/src/face_aggregate.rs",
# "Focus on the subject": how the subject is found and its focus measured.
"server-rs/crates/lrg-ml/src/subject.rs",
"server-rs/crates/lrg-imaging/src/subject_focus.rs",
# "Key people", and the small faces of the tile pass.
"server-rs/crates/lrg-analysis/src/key_people.rs",
"server-rs/crates/lrg-api/src/key_people.rs",
"server-rs/crates/lrg-ml/src/yunet.rs",
# "Keep" and "Highlights": choosing to a target across a shoot.
"server-rs/crates/lrg-analysis/src/selection.rs",
# "Your culling style".
"server-rs/crates/lrg-analysis/src/taste.rs",
"plugin/LrGeniusAI.lrdevplugin/TaskLearnCullingStyle.lua",
]

[page."Help-AI-Agents.md"]
Expand Down
3 changes: 3 additions & 0 deletions docs/wiki/Credits.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ LrGeniusAI is made possible by these amazing open-source projects and AI framewo
- **YuNet**: Fast, lightweight face detector from the OpenCV Model Zoo; provides the face boxes and five landmarks. [GitHub](https://github.com/opencv/opencv_zoo)
- **FaceNet (facenet-pytorch)**: Inception-ResNet-v1 with VGGFace2 weights; produces the face embeddings that group photos by person. [GitHub](https://github.com/timesler/facenet-pytorch)
- **MediaPipe Face Mesh V2 and Blendshape V2** (Apache-2.0): Google's face landmark and facial-expression models from the Face Landmarker; culling reads eye state (open, half closed, closed), smile and gaze per face from them. Converted to ONNX with tf2onnx. [Guide](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker) · [GitHub](https://github.com/google-ai-edge/mediapipe)
- **RF-DETR-Seg Nano** (Apache-2.0): Roboflow's real-time detector with instance masks, trained on COCO, built on a DINOv2 backbone; culling finds the subject of a photo with it to tell focus on the subject from focus on the background. Exported to ONNX with Roboflow's `rfdetr` package. [GitHub](https://github.com/roboflow/rf-detr)
- **AuraFace (fal.ai)**: ArcFace-style ResNet-100 trained on commercially usable data, Apache-2.0; the optional, more accurate face recognizer. [Hugging Face](https://huggingface.co/fal/AuraFace-v1)
- **SigLIP2 (NaFlex)** (Apache-2.0): Google's `siglip2-so400m-patch16-naflex` ([SigLIP 2, Tschannen et al., 2025](https://arxiv.org/abs/2502.14786)), the vision-language model behind semantic search and the photo genre; it keeps each photo's aspect ratio. Exported with Hugging Face Transformers. [Model](https://huggingface.co/google/siglip2-so400m-patch16-naflex)
- **AestheticSigLIP**: Gowthami Somepalli's `AestheticSigLIP`, a SigLIP 2 So400m NaFlex encoder fine-tuned to score photos from 1 to 9, which provides the *Aesthetic score*. Apache-2.0. [Model](https://huggingface.co/somepago/AestheticSigLIP)
Expand Down Expand Up @@ -47,6 +48,8 @@ LrGeniusAI is made possible by these amazing open-source projects and AI framewo
## Tooling & Project Infrastructure
- **Automaat/lightroom-mcp** (MIT): A second open-source Lightroom Classic plugin, by Marcin Skalski, pairing an `.lrplugin` with an MCP server. Its CI/CD setup is where this project's workflow trigger scoping, per-workflow concurrency groups, commit-SHA pinning of *every* action (not just third-party ones), the pinned `busted` revision, and the "stamp the version, then read the stamp back" rule came from — the last of which turned up a bug that had shipped in every release. Its plugin also documents a couple of Lightroom SDK behaviours worth knowing — among them how LrSocket behaves across plug-in reloads and on Windows (its issues #110, #121, #128, #134, #137), which decided the shape of LrGeniusAI's agent bridge: a receive-only socket as a doorbell, everything else over HTTP. No code was taken from it. [GitHub](https://github.com/Automaat/lightroom-mcp)
- **StyLua, luacheck, busted**: Formatting, linting and headless unit tests for the plugin's Lua.
- **Wikimedia Commons photographers**: The public culling benchmark is made from 120 quality images on Wikimedia Commons, each under the free licence named with it, with its author, in [`testdata/cull_bench/sources.toml`](https://github.com/LrGenius/LrGeniusAI/blob/main/server-rs/testdata/cull_bench/sources.toml). Only measurements made from them are in the repository, not the photographs. See [Dev-Image-Culling-Benchmark.md](Dev-Image-Culling-Benchmark.md).
- **torchvision** (BSD-3-Clause): Its Mask R-CNN and Keypoint R-CNN place the defects of that benchmark — where the subject and the eyes are. A build-time tool in `server-rs/scripts`, never shipped.
- **pre-commit**: Runs the formatters, linters and the docs checker before every commit.

Full licence texts for the third-party source vendored into this repository are in [THIRD-PARTY-NOTICES.md](https://github.com/LrGenius/LrGeniusAI/blob/main/THIRD-PARTY-NOTICES.md).
Expand Down
Loading
Loading