Skip to content

refactor(zoo): name the transformer models' modules like their checkpoints, and warn on unread tensors - #101

Open
chmjkb wants to merge 10 commits into
mainfrom
@chmjkb/checkpoint-names-tier1
Open

chmjkb wants to merge 10 commits into
mainfrom
@chmjkb/checkpoint-names-tier1

Conversation

@chmjkb

@chmjkb chmjkb commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Description

Tier 1 of "modules own their checkpoint layout" (#70): the transformer models in the zoo name their modules exactly like their Hugging Face checkpoints, so their state-dict hooks no longer rename, drop or tie tensors.

  • loadStateDict({ strict }), PyTorch-style. A tensor in the file that the model doesn't read now gets a console.warn listing it and stays unbound; strict: true throws CHECKPOINT_MISMATCH as before. A missing tensor always throws. The progress total counts only bound tensors. No runner passes strict.
  • RMSNorm.scale → RMSNorm.weight, PyTorch's name. privacy-filter's hosted int8g640 file stores the old .scale names; its detector maps them (renameExportedNormScales).
  • Moonshine and Moonshine Streaming: fields follow model.encoder.* / model.decoder.* (self_attn, encoder_attn, input_layernorm, ...). The conv stem lives in the encoder as in the checkpoint; Streaming's decoder gets its own decoder.ts for pos_emb / proj.
  • MiniLM: embeddings.{word_embeddings,position_embeddings,LayerNorm}, encoder.layer.N.{attention.{self,output},intermediate,output}.
  • EmbeddingGemma 2: language_model.*, vision_tower.* + embed_vision.*, audio_tower.* + embed_audio.*.
  • Code style: a rule that a model's module tree matches its checkpoint, and that a hook only builds derived tensors, with a bad/good example.

Hook files after this, with zero move() calls left in all four:

model hook lines
MiniLM 102 → 79
Moonshine 141 → 84
Moonshine Streaming 93 → 71
EmbeddingGemma 2 334 → 256

What's left in them is fusion (q/k/v concatRows), scale folds, RoPE and conv permutes, and tensors computed on the CPU. Those move into core in tier 2.

Introduces a breaking change?

  • Yes
  • No

The public API (create<Task>(), models) is unchanged. Hosted files that carry tensors the model never reads now print one warning per load:

  • all-MiniLM-L6-v2 f16/f32: embeddings.position_ids, pooler.dense.weight, pooler.dense.bias
  • Moonshine Streaming tiny f16/f32: proj_out.weight

Re-exporting those files without the unused tensors removes the warning; that's left for the Hub upload step.

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Performance (kernel tuning, graph building, anything related to performance)
  • Other (chores, tests, code style improvements etc.)

Testing instructions

  1. pnpm typecheck && pnpm lint && pnpm format:check && pnpm --filter runntime test:unit (1104 pass)
  2. pnpm --filter runntime exec vitest run --project gpu test/zoo test/core/nn test/core/kernels/matmul (427 pass, 3 expected fail that exist on the base too)

Checked outside this branch:

  • A model-level A/B (kept on a local branch, not part of this PR) loaded the real hosted f32, f16 and int8 g32 files on both sides and compared every output: max difference 0 on MiniLM and all three EmbeddingGemma 2 towers. Breaking one hook on purpose made it report 4.1.
  • Through the public runners, base vs this branch: identical MiniLM and EmbeddingGemma 2 embeddings, identical Moonshine tiny and Streaming tiny transcripts on 3 clips, identical privacy-filter spans on the hosted int8g640 file.

Related issues

Part of #70. Stacked on #99.

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

  • Field names copy the checkpoint, including snake_case and capitals (self_attn, LayerNorm); the code-style skill now says so.
  • Derived tensors (qkv, kv, qkv_norm, rel, q_scale, pos_x/pos_y) sit under the checkpoint-named parent.

@chmjkb
chmjkb added this pull request to stack #119 November 13, 2026 14:19
@chmjkb
chmjkb force-pushed the @chmjkb/checkpoint-names-tier1 branch 3 times, most recently from d6737d9 to 8d5d3a7 Compare November 16, 2026 10:11
Base automatically changed from @chmjkb/quant-embeddinggemma2-models to main November 16, 2026 10:14
@chmjkb
chmjkb force-pushed the @chmjkb/checkpoint-names-tier1 branch from 8d5d3a7 to 89c85f5 Compare November 16, 2026 10:15
@chmjkb
chmjkb marked this pull request as ready for review November 16, 2026 12:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant