Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,34 @@ B2b-i landed the dense-resident device forward with the routed-expert tier delib
driver is the upstream drift class. CLASSIFICATION: selection flips
DRIVEN by continuous bf16 device-arm precision drift through the
attention/norm chain; per the row's adjudication rule this is an
upstream adjudication decision (f32-enabling the device attention/norm
upstream adjudication decision ( f32-enabling the device attention/norm
chain or accepting the band), not a local patch. ISSUE STAYS OPEN on the
device gates and the OOM debt.

- 2026-10-12 (branch row/tt-kolibri-readj, off row/tt-b2ii-index
@ 2c4efce88): the teacher-forced adjudication RE-RUN on the CORRECTED
instrument — the prior classification was computed on a SHIFTED chain
(generated_ids[step-1] fed instead of generated_ids[d-1]), so every
prior per-step teacher-forced number is invalid as step evidence. The
corrected walk reproduces the #3453-era corrected counts exactly (ON 69
tf flips: 29 near-tie, 40 HARD, worst 10.617 nats; OFF 44: 27 near-tie,
17 HARD, worst 8.647) — the residual is REAL, not an instrument
artifact. Drift origin UNCHANGED: L0 dh2 (the L0 MoE input) is the first
divergent stage at EVERY decode step from step 0 on all three diverging
prompts (SCRATCH dbg TF prompt, CPU-vs-TT bit hashes), 0.03-2.5% rel per
step — the 1-bf16-ULP-class L0 input-norm origin stands. Router-flip
mechanism CONFIRMED: per-layer CPU-vs-TT top-6-of-384 selected-expert
overlap under the corrected walk averages 3.0-3.7 of 6 (full agreement
only 6-7% of layer-steps) at EVERY layer including L0, median router
logit_sum rel diff 6.7-9.7%; the route_scores probe shows top1-top2
margins are robust (median 0.13-0.48) while top6-top7 margins sit below
0.05 nats at 26-41 of 50 layers at every hard-flip step — the incoming
noise exceeds the rank-6/7 boundary margin at most layers. Distribution:
hard flips SPREAD, not concentrated — 7 of 8 prompts, opening bursts
(steps 0-4) on some prompts and isolated late flips (steps 14-26) on
others, both arms. Gate-standard input: the P150 band-2.5 residual is
STRUCTURAL (continuous step-0-resident drift x router gain onto a
sub-0.05-nat selection boundary), not a single-site repair; levers in
order: host-free shadow reconciliation (17 vs 40 hard shows it halves
the count), then f32 device attention/norm chain. Evidence:
docs/bench-evidence/kolibri1-tt-readjudication-20261012.md.
257 changes: 257 additions & 0 deletions docs/bench-evidence/kolibri1-readj/OFF.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
{
"arm": "OFF",
"summary": [
158,
44,
27,
17,
8.64696
],
"chain": [
12,
20,
8,
6,
2
],
"hard_flips": [
{
"prompt": "der Mond scheint hell \u00fcber den Bergen",
"step": 0,
"golden": 948,
"tfarg": 46,
"gap": 3.181908,
"intop5": "no",
"layer_top12_min": 0.0016739999999999533,
"layer_top12_med": 0.24162299999999998,
"layer_top67_min": 0.0002620000000002065,
"layer_top67_med": 0.028259000000000034,
"n_layers_gap67_below_0p05": 34,
"n_layers_gap12_below_0p05": 6
},
{
"prompt": "der Mond scheint hell \u00fcber den Bergen",
"step": 1,
"golden": 46252,
"tfarg": 3389,
"gap": 2.273598,
"intop5": "no",
"layer_top12_min": 0.0012659999999999894,
"layer_top12_med": 0.167883,
"layer_top67_min": 0.0007520000000000009,
"layer_top67_med": 0.020367000000000024,
"n_layers_gap67_below_0p05": 40,
"n_layers_gap12_below_0p05": 12
},
{
"prompt": "The capital of Australia is",
"step": 18,
"golden": 75361,
"tfarg": 543,
"gap": 3.553511,
"intop5": "yes",
"layer_top12_min": 0.016432000000000002,
"layer_top12_med": 0.484023,
"layer_top67_min": 0.0007150000000000212,
"layer_top67_med": 0.04172799999999999,
"n_layers_gap67_below_0p05": 28,
"n_layers_gap12_below_0p05": 5
},
{
"prompt": "Wissen ist Macht, aber Zeit ist Geld",
"step": 1,
"golden": 936,
"tfarg": 420,
"gap": 1.92732,
"intop5": "no",
"layer_top12_min": 0.003607000000000027,
"layer_top12_med": 0.21200000000000002,
"layer_top67_min": 0.00011499999999997623,
"layer_top67_med": 0.04392200000000002,
"n_layers_gap67_below_0p05": 34,
"n_layers_gap12_below_0p05": 9
},
{
"prompt": "Wissen ist Macht, aber Zeit ist Geld",
"step": 2,
"golden": 2973,
"tfarg": 936,
"gap": 2.048908,
"intop5": "no",
"layer_top12_min": 0.0017440000000000788,
"layer_top12_med": 0.316585,
"layer_top67_min": 0.0001020000000000465,
"layer_top67_med": 0.044141999999999904,
"n_layers_gap67_below_0p05": 30,
"n_layers_gap12_below_0p05": 10
},
{
"prompt": "Wissen ist Macht, aber Zeit ist Geld",
"step": 3,
"golden": 476,
"tfarg": 735,
"gap": 3.018167,
"intop5": "yes",
"layer_top12_min": 0.008242,
"layer_top12_med": 0.19427700000000003,
"layer_top67_min": 0.0009399999999999964,
"layer_top67_med": 0.02517999999999998,
"n_layers_gap67_below_0p05": 41,
"n_layers_gap12_below_0p05": 8
},
{
"prompt": "Wissen ist Macht, aber Zeit ist Geld",
"step": 14,
"golden": 18331,
"tfarg": 502,
"gap": 7.306021,
"intop5": "no",
"layer_top12_min": 0.0006349999999999412,
"layer_top12_med": 0.34341299999999997,
"layer_top67_min": 0.0001620000000001065,
"layer_top67_med": 0.034442000000000084,
"n_layers_gap67_below_0p05": 34,
"n_layers_gap12_below_0p05": 13
},
{
"prompt": "Wissen ist Macht, aber Zeit ist Geld",
"step": 17,
"golden": 936,
"tfarg": 286,
"gap": 3.616066,
"intop5": "yes",
"layer_top12_min": 0.002801999999999999,
"layer_top12_med": 0.19718500000000005,
"layer_top67_min": 0.0009519999999998419,
"layer_top67_med": 0.036826000000000025,
"n_layers_gap67_below_0p05": 33,
"n_layers_gap12_below_0p05": 7
},
{
"prompt": "Translation to German: the weather is beautiful today",
"step": 5,
"golden": 4484,
"tfarg": 286,
"gap": 3.819496,
"intop5": "yes",
"layer_top12_min": 0.0007070000000000132,
"layer_top12_med": 0.287526,
"layer_top67_min": 0.000280000000000058,
"layer_top67_med": 0.03786900000000004,
"n_layers_gap67_below_0p05": 33,
"n_layers_gap12_below_0p05": 7
},
{
"prompt": "Translation to German: the weather is beautiful today",
"step": 10,
"golden": 5445,
"tfarg": 2302,
"gap": 3.030571,
"intop5": "no",
"layer_top12_min": 0.003015999999999991,
"layer_top12_med": 0.19559200000000004,
"layer_top67_min": 5.100000000002325e-05,
"layer_top67_med": 0.027216000000000018,
"n_layers_gap67_below_0p05": 35,
"n_layers_gap12_below_0p05": 9
},
{
"prompt": "Translation to German: the weather is beautiful today",
"step": 22,
"golden": 5445,
"tfarg": 2302,
"gap": 1.899872,
"intop5": "no",
"layer_top12_min": 0.0015570000000000306,
"layer_top12_med": 0.29370300000000005,
"layer_top67_min": 0.001670999999999978,
"layer_top67_med": 0.042990000000000084,
"n_layers_gap67_below_0p05": 31,
"n_layers_gap12_below_0p05": 5
},
{
"prompt": "Ein gutes Buch liest man am besten",
"step": 7,
"golden": 51092,
"tfarg": 2459,
"gap": 3.285328,
"intop5": "yes",
"layer_top12_min": 0.004497999999999891,
"layer_top12_med": 0.278218,
"layer_top67_min": 0.0005789999999998852,
"layer_top67_med": 0.04965699999999995,
"n_layers_gap67_below_0p05": 26,
"n_layers_gap12_below_0p05": 6
},
{
"prompt": "counting: one two three four five six seven",
"step": 0,
"golden": 12843,
"tfarg": 14031,
"gap": 6.156881,
"intop5": "no",
"layer_top12_min": 0.008246999999999893,
"layer_top12_med": 0.170104,
"layer_top67_min": 0.0010459999999999914,
"layer_top67_med": 0.02381000000000011,
"n_layers_gap67_below_0p05": 39,
"n_layers_gap12_below_0p05": 13
},
{
"prompt": "counting: one two three four five six seven",
"step": 1,
"golden": 12843,
"tfarg": 14031,
"gap": 8.646961,
"intop5": "no",
"layer_top12_min": 0.002374999999999905,
"layer_top12_med": 0.24289000000000005,
"layer_top67_min": 0.0004789999999998962,
"layer_top67_med": 0.028886000000000023,
"n_layers_gap67_below_0p05": 38,
"n_layers_gap12_below_0p05": 12
},
{
"prompt": "counting: one two three four five six seven",
"step": 2,
"golden": 12843,
"tfarg": 14031,
"gap": 5.815979,
"intop5": "no",
"layer_top12_min": 0.004530999999999952,
"layer_top12_med": 0.128412,
"layer_top67_min": 0.00014899999999995472,
"layer_top67_med": 0.018912999999999958,
"n_layers_gap67_below_0p05": 41,
"n_layers_gap12_below_0p05": 10
},
{
"prompt": "counting: one two three four five six seven",
"step": 3,
"golden": 12843,
"tfarg": 14031,
"gap": 4.99181,
"intop5": "yes",
"layer_top12_min": 0.01208100000000012,
"layer_top12_med": 0.24573,
"layer_top67_min": 0.00014399999999997748,
"layer_top67_med": 0.030302999999999997,
"n_layers_gap67_below_0p05": 34,
"n_layers_gap12_below_0p05": 10
},
{
"prompt": "counting: one two three four five six seven",
"step": 4,
"golden": 12843,
"tfarg": 14031,
"gap": 3.630234,
"intop5": "yes",
"layer_top12_min": 0.000136000000000025,
"layer_top12_med": 0.20996,
"layer_top67_min": 0.0004939999999999944,
"layer_top67_med": 0.033832,
"n_layers_gap67_below_0p05": 32,
"n_layers_gap12_below_0p05": 10
}
]
}
Loading
Loading