Skip to content

tenstorrent: the rms_norm fp32-acc fix + the teacher-forced B2b-ii instrument - #3442

Closed
lu-zero wants to merge 0 commit into
localai-org:mainfrom
lu-zero:row/tt-kolibri-residue
Closed

lu-zero wants to merge 0 commit into
localai-org:mainfrom
lu-zero:row/tt-kolibri-residue

Conversation

@lu-zero

@lu-zero lu-zero commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Two keepers from the kolibri P150 residue hunt.

The instrument (cherry-picked from the near-tie row): the B2b-ii gate gains the W3-methodology teacher-forced adjudication — golden prefix, per-divergence gap and top-5 membership, hard-flip counter. "0/8" style verdicts are now "which prompt, which step, how many nats".

The substrate fix: ttnn::rms_norm's rmsnorm_default_compute_config hardwired fp32_acc = false and approx_mode = true — the sum-of-squares accumulated in bf16 even for f32 tiles (which is what vllm.cpp's shadow arm feeds it). With f32 accumulation and accurate rsqrt, the RMSNorm micro moves 0.9899 -> 0.9997 (1 bf16 ULP — the output-store floor). The trial tt-metal tree (/tmp/tt-metal-umdtrial, installed at /tmp/umdtrial-install) carries this on top of the port fixes; the stack-pin record must note it.

Verdicts after: the RMSNorm kernel bias is GONE, but the B2b-ii argmax chain stays 26/33 on both arms — the remaining flips are attributed to the two known op-level drifts (kGdnDecode, kMatmulBTQuantGrouped), which are exactly the TT suite's 2 standing failures. The teacher-forced counts improved (worst gap 5.17 -> 3.19 nats ON) but the hard flips persist: the next localization pass runs on the fixed substrate. The streaming slot-pool throw also still fires on 4 of 8 walks (its message is now just "1" — source unidentified, separate defect).

TT unit suite: only the 2 known failures; W4 EnsureDevice2D and matmul region class split fail only in full-suite order state (pass isolated — order-state flake, recorded).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]

This branch was successfully deployed

1 active deployment
github-pages — 31b48a3a Deployed Oct 11, 2026 by lu-zero via deploy #2742
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant