Skip to content

Fix served-model CI: record lineage for the July v5 incumbent - #100

Merged
jcdavis131 merged 2 commits into
masterfrom
cursor/served-model-lineage-b86c
Oct 19, 2026
Merged

jcdavis131 merged 2 commits into
masterfrom
cursor/served-model-lineage-b86c

Conversation

@jcdavis131

@jcdavis131 jcdavis131 commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Root cause

served-model has been red since it landed, not since PR #99.

The served tree is still the July 2026 v5 export (assets/mtnn_embeddings.f32 git blob a4918f09, content sha256 2574ef58…). pipeline/data/ is gitignored, the original promote.py run directory is not in this repo, and PR #97 showed that retraining v5 does not reproduce the incumbent. The check was correctly reporting “no lineage file.” Making that job a required check before anyone could write the file is why master and PRs #96/#97/#98 stay red.

What changed

Did not skip, weaken, or delete the check. scripts/check_served_model.py still hashes the f32, the vectors.json key order, meta keys/metrics, sidecar lineage.run_id, and assets/ vs public/ parity.

  • Added scripts/record_served_lineage.py (stdlib). It writes mtnn_lineage.json from the served bytes (f32 sha256, keys sha256, metrics already in mtnn_meta.json) for run july-v5-a4918f09, with forced: true and a written reason. A replacement model still has to go through promote.py + export_mtnn_embeddings.py.
  • Stamp + sync_public so ?v= tokens and the Vercel public/ tree match.

Review follow-up

  1. Check-side pin for forced adoption. served_model.forced_adoption_pin_problem (run by check_served_model.py) rejects a lineage with forced: true and no checkpoint_sha256, or adoption.kind == served-incumbent, unless embedding_f32_sha256 is the pinned July v5 sha (2574ef581ac3…). The recorder is no longer the only gate: a hand-written forced lineage for a different model fails the check. promote.py --force of a real bundle still writes checkpoint_sha256 and is not this path (needed for tests/test_e2e_smoke.py). Test: test_a_forced_lineage_with_a_different_sha_fails.

  2. Mislabeled 48-d sidecars. mtnn_jacobian.json (built 2026-07-14, dEmb 48) and mtnn_attr_pop.json (built 2026-07-11, different checkpoint bytes) are no longer stamped run_id: july-v5-a4918f09. They carry stale / not_from_served_net and their own origin run ids (pre-v5-48d-2026-07-14 / pre-v5-48d-2026-07-11). The check fails if a sidecar marked not-from-served-net claims the served run. network-viz.js throws not from served net (stale 48-d export) and leaves Jacobian/attribution unused rather than treating them as the shipped 64-d net.

Evidence that CI passes on this PR

Follow-up commit e11c2a5b. All required checks green:

Check Result Run
served-model pass (9s) 39178511097
site pass (14s) same
offline-build pass (1m30s) same
ruff pass (10s) 39178511030
budgets pass (11s) 39178511040
Vercel pass preview deployment

PR #96 (opening-night 2026-27)

This PR does not touch index.html / public/index.html. Once this lands, #96 needs only a rebase. Any index.html conflicts would be from #99’s nav work, not from this lineage fix. No model/asset changes required on #96.

Open in Web Open in Cursor 

The served-model job has been red since it landed (before PR #99): master
serves f32 blob a4918f0 with no mtnn_lineage.json, and the original
promote.py run directory is not in this repo. Record that incumbent's
bytes, keys hash, and existing meta metrics, stamp sidecar run ids, and
keep the check strict — a different f32 is still refused.

Co-authored-by: Cam Davis <jcdavis131@gmail.com>
@vercel

vercel Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vector-hoops Ready Ready Preview Oct 18, 2026 11:55pm UTC

Request Review

The recorder already refused a non-incumbent f32; the check did not, so a
hand-written forced lineage could bless a future model. Reject forced
adoption unless embedding_f32_sha256 is the pinned July v5 sha.

Stop stamping mtnn_jacobian.json and mtnn_attr_pop.json with the v5 run
id: they are 48-d Jul 11–14 exports. Mark them not_from_served_net and
have network-viz refuse to treat them as the shipped net.

Co-authored-by: Cam Davis <jcdavis131@gmail.com>
@jcdavis131
jcdavis131 marked this pull request as ready for review October 19, 2026 03:02
@jcdavis131
jcdavis131 merged commit 61c2c86 into master Oct 19, 2026
9 checks passed

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Cursor Bugbot was not present after the first poll, remaining CI checks completed successfully, and no approval policy required human review.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

This branch was successfully deployed

1 active deployment
Preview — e11c2a5b Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants