Fix search, export and API gaps; update dependencies and CI actions

.github/workflows/ci.yml

@@ -30,9 +30,9 @@ jobs:
       HF_HUB_OFFLINE: "1"
       TRANSFORMERS_OFFLINE: "1"
     steps:
-      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
+      - uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
         with:
           version: "0.12.9"
           enable-cache: true
@@ -73,7 +73,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
-      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
       - uses: denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed # v2.0.5
         with:
@@ -90,13 +90,13 @@ jobs:
     runs-on: macos-14
     timeout-minutes: 30
     steps:
-      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
+      - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable
         with:
           toolchain: stable
 
-      - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
+      - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
         with:
           workspaces: tauri/src-tauri
 

ARCHITECTURE.md

@@ -114,6 +114,13 @@ channel still returned something, so the result list never looked broken
 enough to report. Stripping the hyphen turns it into an AND, which is what
 people mean anyway.
 
+The words `AND`, `OR`, `NOT` and `NEAR` were another. FTS5 treats them as
+operators only in capitals, and the sanitiser joins the words with explicit
+ANDs, so `rock OR roll` became `rock AND OR AND (roll* OR roll)` and failed
+the same silent way. Lowercasing those four words makes them ordinary
+words, so the query finds the text that contains them. `ANDROID` and `NOTE`
+are untouched, because the rule is whole-word.
+
 Unbalanced quotes are the same class of problem. Search-as-you-type means that
 the moment someone presses `"` the query is syntactically invalid, so an odd
 quote count gets a closing quote appended.
@@ -131,7 +138,9 @@ first, because it is the most expensive part of a query. The code's own
 estimate is 50–150 ms for the warm embed against 10–30 ms per FTS5 channel,
 so running the two side by side hides the FTS5 time under the embed instead
 of adding them up. `scripts/qa_smoke.py` holds the whole
-request to a p50 under 100 ms and a p99 under 200 ms on Apple Silicon.
+request to a p50 under 100 ms and a p99 under 200 ms on Apple Silicon (500 ms
+and 800 ms when `CI=true`; CI does not run the script, so this is a local
+check).
 
 Hits are then bucketed by `(file_id, ts_ms // 5000)`. A bucket holding more
 than one source is the strongest signal Tern has: speech, on-screen text and
@@ -188,21 +197,27 @@ so the writers are defensive.
   or file name with `&` or `"` cannot break the document or inject
   structure. Clip names and marker values are truncated *before* escaping,
   so a length cap can never cut an entity like `&` in half, which Final
-  Cut and DaVinci reject as a corrupt import.
+  Cut and DaVinci reject as a corrupt import. The `src` of each asset is a
+  percent-encoded `file://` URL (`_file_url` in `clip.py`), not a raw path,
+  because a raw path with a space, `#` or `%` in it is not a URL and Final
+  Cut reports that media as missing.
 - **CSV formula injection.** A cell that starts with `=`, `+`, `-`, `@`, a
   tab or a carriage return, including after leading spaces, gets a leading
   single quote, so a spreadsheet treats it as text rather than a formula.
 - **Bounded input.** CSV and FCPXML exports take at most 200 hits, project
   names are capped at 200 characters, and a clip may not be longer than 30
   minutes.
-- **Source allowlist.** `/api/export/clip` accepts a source only if it is
-  inside the workspace or is an indexed file, the same check `/api/file`
-  uses. Without it, exporting an arbitrary file and then downloading the
-  result from the workspace would be a way to read any file on the machine.
+- **Source allowlist.** `/api/export/clip` and `/api/export/fcpxml` accept a
+  source only if it is inside the workspace or is an indexed file, the same
+  check `/api/file` uses, and make it before anything touches the disk.
+  Without it, exporting an arbitrary file and then downloading the result
+  from the workspace would be a way to read any file on the machine.
 - **Timeouts.** ffmpeg gets 120 s for an MP3 extract and 300 s for a video
   re-encode, and each ffprobe call during FCPXML generation gets 30 s. A
-  timeout comes back as a 504 whose message names the likely cause (a
-  corrupt source or a stalled mount) and is logged to `tern-crash.log`.
+  timeout while reading a file's duration comes back as a 504 whose message
+  names the likely cause (a corrupt source or a stalled mount) and is logged
+  to `tern-crash.log`; a timeout while reading its frame size and rate falls
+  back to 1080p30, so the export still ships.
 
 Video clips are encoded with `h264_videotoolbox` rather than libx264, which
 is GPL. `clip.py` records the measurement behind the `-q:v 75` setting: it
@@ -261,7 +276,7 @@ this stops casual copying, not a determined user.
 
 ## Testing
 
-539 tests across three suites: 168 for the pipeline, 358 for the API, and
+556 tests across three suites: 171 for the pipeline, 372 for the API, and
 13 for the licence server's seat rules.
 
 ```bash

README.md

@@ -22,7 +22,7 @@
   <tr>
     <td align="center" width="200"><b>3 search channels</b><br><sub>speech · on-screen text · picture</sub></td>
     <td align="center" width="200"><b>On-device</b><br><sub>no media leaves the Mac</sub></td>
-    <td align="center" width="200"><b>539 tests</b><br><sub>pipeline · API · licence rules</sub></td>
+    <td align="center" width="200"><b>556 tests</b><br><sub>pipeline · API · licence rules</sub></td>
     <td align="center" width="200"><b>5 export formats</b><br><sub>MP4 · MP3 · SRT · CSV · FCPXML</sub></td>
   </tr>
 </table>
@@ -66,8 +66,9 @@ FCPXML timeline for Final Cut Pro export from the result list.
 Media and the index never leave the Mac: transcription, OCR and embedding
 all run locally. The network is used to fetch the models on first use (the
 Hugging Face library also revalidates its cached copy when it loads one),
-to activate a licence, for the update check, and to open Apple Maps when
-you click a photo's location.
+to activate a licence, and to open Apple Maps when you click a photo's
+location. The update check will use it too once an update endpoint is
+configured, which it is not yet (see [Status](#status)).
 
 ![Searching "hurricane" across NASA footage: the burned-in caption and the narration match the same moments, so the top hits are badged On-screen + Speech](docs/media/multi-channel.png)
 
@@ -96,7 +97,7 @@ demo podcast (the episodes are not included in this repository).*
 
 <picture>
   <source media="(prefers-color-scheme: dark)" srcset="docs/media/fusion-dark.svg">
-  <img src="docs/media/fusion-light.svg" alt="Four channels (speech, on-screen text, picture and filename) are searched in parallel and weighted; hits are grouped into five-second windows per file, where agreement between channels earns a capped bonus; a filename boost, a per-file cap and a visual noise floor then produce one ranked list">
+  <img src="docs/media/fusion-light.svg" alt="Four channels (speech, on-screen text, picture and filename) are searched independently and weighted; hits are grouped into five-second windows per file, where agreement between channels earns a capped bonus; a filename boost, a per-file cap and a visual noise floor then produce one ranked list">
 </picture>
 
 Each channel is searched on its own, so one failing channel degrades the list
@@ -164,18 +165,24 @@ The parts that took real work, each with the code that does it:
   for footage with few scene cuts. ([`vision.py`](service_pipeline/tern/vision.py))
 - **Latency budget.** The SigLIP text embedding runs on a background thread
   while the FTS5 channels run, so their costs overlap instead of adding up;
-  the smoke test holds a search to a p50 under 100 ms.
+  `scripts/qa_smoke.py` holds a search to a p50 under 100 ms on Apple Silicon.
+  It is a local check; CI does not run it.
 - **The loopback API is treated as hostile.** CORS allows loopback and the
   Tauri origin only, a Host-header guard refuses any name that is not a
   loopback name on the port the server listens on (so a DNS-rebinding page
   is turned away), the licence endpoint is fixed by configuration, every
   endpoint that serves, opens or exports a file
-  checks a workspace-and-index allowlist before it checks existence, and
-  `/api/diagnostics` strips the home directory out of every path.
+  checks a workspace-and-index allowlist before it checks existence,
+  responses carry a Content-Security-Policy header (including
+  `frame-ancestors 'none'`, which a `<meta>` policy cannot express) so no
+  other page can frame the UI, and `/api/diagnostics` strips the home
+  directory out of every path.
   ([`api/main.py`](api/main.py))
 - **Exports that cannot corrupt a timeline.** Text exports are written
   through a temp file, fsync and rename; FCPXML strings are truncated before
-  they are escaped so an entity is never cut in half; CSV cells that would
+  they are escaped so an entity is never cut in half, and asset paths are
+  written as percent-encoded `file://` URLs so a folder with a space in it
+  still resolves in Final Cut; CSV cells that would
   run as spreadsheet formulas are neutralised.
   ([`clip.py`](service_pipeline/tern/clip.py), [`api/main.py`](api/main.py))
 - **Every blocking subprocess has a timeout,** sized to the job, and
@@ -216,10 +223,11 @@ scripts/init_demo.sh            # indexes demo/, then runs a smoke search
 ./run.sh                        # serves the app on the first free port from 18765
 ```
 
-The first index downloads the Whisper model (about 547 MB) into
+The first index downloads the Whisper model (about 574 MB) into
 `service_pipeline/models/` and SigLIP-2 (about 1.5 GB) into the Hugging Face
 cache. With those in place, `init_demo.sh` indexes the roughly forty minutes
-of demo media in about two minutes on an M-series MacBook Pro. Indexing
+of demo media in about two minutes on Apple Silicon (101 and 117 seconds in
+two runs on a Mac Studio M4 Max). Indexing
 through the app or the API counts against the 120-minute trial (the `tern`
 CLI does not); `TERN_STATE_DIR` moves the trial file.
 
@@ -252,7 +260,7 @@ permissions and timeouts.
 
 ## Tests
 
-539 tests: 168 for the pipeline, 358 for the API, and 13 for the licence
+556 tests: 171 for the pipeline, 372 for the API, and 13 for the licence
 server's seat rules.
 
 ```bash
@@ -269,9 +277,9 @@ clone runs green. [CI](.github/workflows/ci.yml) runs the same suites on a
 macOS runner with an empty workspace and the Hugging Face hub offline, the
 Deno suite on Linux, and a `cargo check` of the desktop shell.
 With the full runtime and the downloadable demo workspace, 15 API tests
-that need indexed podcast audio still skip; on the author's machine, whose
-demo also holds eight synthetic podcast episodes that are not in this
-repository, all 500 pass.
+that need indexed podcast audio still skip. On the author's machine, whose
+demo also holds seven synthetic podcast episodes and a screencast video that
+are not in this repository, all 543 Python tests pass.
 
 ## Layout
 
@@ -312,12 +320,19 @@ What is not finished:
 - **The updater is unconfigured.** `tauri.conf.json` carries a placeholder
   public key and endpoint, and `release.sh` refuses to build a signed release until it is
   replaced.
+- **No licence endpoint is compiled in.** The desktop shell does not set
+  `TERN_LICENSE_SERVER`, so a built app cannot activate a licence until a
+  build supplies the endpoint (see [license_server/](license_server/)).
 - **One dependency blocks distribution.** pillow-heif's macOS wheel bundles
   the GPL x265 encoder, so HEIC decoding needs another library before any
   build ships ([third_party/NOTICE.md](third_party/NOTICE.md)).
 - **The trial gate is advisory.** The quota is enforced in the API, so the
   bundled CLI indexes without it, and licence verdicts are cached unsigned.
   It stops casual copying, not a determined user.
+- **Symlinked media files are indexed but not served.** The index stores a
+  file's path as walked, and the file endpoints compare the resolved target
+  against it, so playing or exporting a symlinked file is refused with a 403.
+  Real files are unaffected.
 - **Moving a workspace leaves stale paths.** Only the seeded demo's paths
   are re-rooted in SQLite on start, and the keyframe metadata in ChromaDB
   never is, so visual hits in a moved workspace show blank thumbnails.

SECURITY.md

@@ -23,3 +23,28 @@ README's Status section lists the known limitations, and those are not new
 findings. Reporting a vulnerability does not grant any right to use, copy or
 redistribute the code beyond what the LICENSE and GitHub's Terms of Service
 allow.
+
+## Dependency advisories
+
+The lockfiles were audited on 2026-10-23 (`pip-audit` for `api/uv.lock` and
+`service_pipeline/uv.lock`, the OSV database for `tauri/src-tauri/Cargo.lock`)
+and every advisory with a compatible fix was applied. Three Python packages
+are still flagged, and none is used in the affected way:
+
+- `chromadb`: no fixed release. The advisories concern Chroma's server mode
+  (authorization and `trust_remote_code` on the server). Tern uses the
+  embedded `PersistentClient` and never starts a Chroma server.
+- `accelerate`: no fixed release. The advisory is a path traversal in its
+  checkpoint-loading helpers (`load_checkpoint_in_model` and
+  `load_checkpoint_and_dispatch`), which no Tern code calls.
+- `oauthlib`: fixed only in 4.0.0. The advisory is a timing side-channel in
+  its PKCE provider code, which Tern does not use. It is a transitive
+  dependency.
+
+In the Rust lockfile, the network-facing crates (`rustls`, `rustls-webpki`,
+`tokio`, `webpki-root-certs`) were updated. Of the advisories that remain,
+`zip`, `glib` and `proc-macro-error` are not in the dependency graph for
+macOS (`cargo tree --target aarch64-apple-darwin -i <crate>` finds nothing).
+The five `unic-*` crates are unmaintained-crate notices pulled in through
+Tauri's `urlpattern`; a full `cargo update` drops them, but it also moves
+Tauri itself, so it is a separate change.