[bmachek]

Summary

Download AI models (POST /v1/models/assets/downloads) put the files of every missing model family into one list and handed it to download_url_assets. That function sent a HEAD request per file and stopped the whole job at the first failure. So one unreachable release meant nothing downloaded, not even the search model for a new user. A release can be unreachable when its tag is not published yet, when a model-assets-*.yml run failed, or when a host is blocked.

This is the case on main right now. face-state-assets-v1 does not exist: the only run of model-assets-face-state.yml, after the #471 merge (run 41688897316), was cancelled after 15 minutes (its limit is 45). Until that release is published, every user who is still missing any model gets nothing from "Download AI models" on main. With this PR the other families download, and "Eye and expression analysis" is named as missing. Re-running that workflow (it has workflow_dispatch) publishes the release.

Now each family downloads on its own:

• A family whose files cannot be resolved (HEAD) or fetched (GET or write) is skipped, and the others still download.
• The job still ends in error when any family failed:
  • error_messages holds one "<family name>: <reason>" per failed family.
  • error names each failed family and says what to do. The status stays downloading until every family has been tried.
• Progress stays one bar. Sizes are resolved for every family before the first byte, so total covers only what will be fetched. A family that fails part-way takes the bytes it will not fetch out of total, so the bar still ends full.
• The single-family routes do not change. CLIP, BioCLIP and AuraFace share the split-out HEAD pass (resolve_sizes) and transfer (fetch_assets). They stay all-or-nothing, with the same messages. A test pins that.

The progress total was 0

The HEAD pass used resp.content_length(). reqwest computes it from the received body, and a HEAD response has none, so it is always 0 (seanmonstar/reqwest#1814). Every download on this machinery therefore had total = 0 until set_done() filled it in at the end. Sizes now come from the Content-Length header. Switching back to content_length() makes 3 of the new tests fail (left: 0, right: 57344).

Changes

• routes/clip.rs
  • Splits download_url_assets into resolve_sizes, fetch_assets and release_urls.
  • ModelDownloadStatus gets error_messages, omitted from the JSON when empty so the other download routes answer exactly as before, plus set_errors.
• routes/assets.rs
  • A Family enum is the one place for each family's id, human name, size estimate and readiness. GET /v1/models/assets uses it and returns the same JSON, now pinned by a contract test.
  • download_families is the per-family loop, and failure_summary builds the user-facing text.
  • With main merged in, feat(cull): eye focus, eye state, smile and gaze per face #471's face_state is Family::FaceState: "Eye and expression analysis", face-state-assets-v1.
• src/fake_release.rs (#[cfg(test)]): a local axum stand-in for GitHub release downloads with three behaviours: files that answer, a 404, and HEAD-OK/GET-502. It sits outside routes/ because check-docs.py reads every .route( there as an endpoint.
• Plugin (APISearchIndex.lua startAssetDownload): all three callers end in this loop: the Plug-in Manager button, the setup wizard and the run preflight. The loop already showed the backend's error. When error_messages is non-empty, the headline now reads Not all AI models could be downloaded. The Plug-in Manager's 5-second /v1/models/assets poll picks up what landed.
• Docs
  • Dev-Backend-API.md: the assets download section.
  • Troubleshooting.md: §3 resolution steps, and the intro now lists all five models.
  • server-rs/README.md, and the Dev-Server-README.md generated from it: the BioCLIP 404 note.
  • Getting-Started.md: main's text, about 4.2 GB.
  • Credits.md is still accurate (Docs-Reviewed trailer).

Testing

• CI is green on 3292f52, the merged head: cargo-test (cargo test --workspace with LRG_REQUIRE_GOLDENS=face), format-lint, format-lint-rust, docs-check and busted.
• New unit tests, against a real HTTP server on a local port:
  • a family whose release 404s doesn't stop the others; the status names only that family
  • a family whose transfer breaks part-way is skipped, and total shrinks
  • every family landing ends completed
  • the summary wording
  • the single-model path stops at its first missing file with the unchanged message
  • sizes come from the announced length
• Contract tests for the GET /v1/models/assets shape (five families, now including face_state), and for no error_messages without failures.
• Locally: cargo fmt --check and cargo clippy --workspace --all-targets -- -D warnings are clean. stylua --check, luacheck (0/0) and busted (614) pass. scripts/check-docs.py exits 0.
• Live, the debug binary against the real releases. Search and quality-scoring models were dummy files, so they counted as ready.
  • /v1/models/assets listed five families with missing_approx_bytes: 977000000.
  • The download chose exactly bioclip, face, face_state and skipped each one with its own reason.
  • The final error, as the plugin shows it:

    These AI models could not be downloaded:
    - Species identification: bioclip2_image_fp16.onnx not found at …/bioclip-assets-v1/bioclip2_image_fp16.onnx (HTTP 401 Unauthorized) — the release may not exist yet
    - Face detection: yunet_face_detection.onnx not found at …/face-assets-v1/yunet_face_detection.onnx (HTTP 401 Unauthorized) — the release may not exist yet
    - Eye and expression analysis: face_mesh_v2.onnx not found at …/face-state-assets-v1/face_mesh_v2.onnx (HTTP 404 Not Found) — the release may not exist yet
    
    The other AI models are downloaded and ready. Press "Download AI models" in the Plug-in Manager again later; it fetches only what is still missing. If it keeps failing, check that the computer the backend runs on can reach github.com, and that the backend can write to its model folder.
    

  • The 401s come from the test sandbox. Its proxy mediates github.com, and the release redirect lands on a pre-signed S3 URL that refuses HEAD (curl: HEAD → 401, ranged GET → 206). So no family could land there. The unit tests cover the successful path against a real server.

Notes

• Follow-up, not in this PR: routes/llm.rs (the GGUF download) has the same content_length()-on-HEAD bug in its own loop, so its progress total is 0 as well.
• Possible hardening, not done here: if real users ever see HTTP 401/403 … the release may not exist yet, resolve_sizes could fall back to a one-byte ranged GET, which pre-signed GET URLs accept. The download counts on the asset releases suggest HEAD works outside the sandbox, so I left it out.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HaMMvtwbAwMWB9jcpdCc3T