Skip to content

kolibri1 TT: the P150 teacher-forced adjudication re-run on the corrected walk — the residual is structural - #3454

Open
lu-zero wants to merge 2 commits into
localai-org:mainfrom
lu-zero:row/tt-kolibri-readj
Open

lu-zero wants to merge 2 commits into
localai-org:mainfrom
lu-zero:row/tt-kolibri-readj

Conversation

@lu-zero

@lu-zero lu-zero commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

The teacher-forced instrument's earlier counts were computed against a shifted chain (fixed in #3453); this re-runs the adjudication on the corrected walk and delivers the decision input for the P150 band-gate standard. Adjudication table (reproduced exactly, both arms, cold-cache recipe): host-free ON 170 free divergences, 69 tf flips (29 near-tie, 40 HARD, worst 10.6 nats); host-free OFF 158 free divergences, 44 tf flips (27 near-tie, 17 HARD, worst 8.6). The zero-artifact stop condition did not fire — the residual is real.

Findings (corrected walk):

  1. Drift origin unchanged: the L0 MoE input (dh2) at 1-bf16-ULP class, every decode step from step 0, all three diverging prompts.
  2. The router-flip mechanism HOLDS decisively: top-6-of-384 expert-set overlap vs CPU averages 3.05-3.71 of 6 (full agreement only 6-7% of layer-steps); top6-top7 margins below 0.05 nats at 26-44 of 50 layers at every hard-flip step. A new route_scores probe (RouteDump, STAGE_DUMP-gated) supplies the margins.
  3. Distribution: SPREAD (structural) — hard flips on 7 of 8 prompts, BOTH arms, opening bursts AND isolated late flips, first-divergent-layer L0 everywhere. No localized defect site.

Recommended gate-standard input: keep the 2.5-nat/top-5 adjudication; record the P150 band as STRUCTURAL (spread), not fixable-by-a-patch. The levers, in order: (1) host-free shadow reconciliation (halves the hard count: 40 -> ~20 ON), (2) f32-enabling the device attention/norm chain (removes the origin — a perf/substrate question), (3) router-margin widening is a model change, out of scope for this row. The flip step is decided by the LM-head margin at that step, not by where the drift begins.

Raw evidence in the doc; probes committed behind VT_KOLIBRI1_TT_STAGE_DUMP. Issue: ISSUE-LOCAL-01M4ER0E9HHM95YZYJB7T5FECN (Resolution updated with the corrected analysis).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mudler @richiejp Focused source review at 1a905450ce4e7995afa9a205c26e35953ef180ac, compared with current main 50d425bec:

The new evidence needs an explicit mapping between the scratch diagnostic's step labels and the gate's decode positions. In tests/vllm/models/test_kolibri1_tt_b2ii.cpp:988-1016, the final prompt token produces decode-position-0 logits but prints TFstep -1. The next iteration feeds generated_ids[0], produces decode-position-1 logits, and prints TFstep 0. Thus the new document's correlation of SCRATCH dbg TF prompt stages with hard flips at the same numbered decode step is not established by those labels. Please include the mapping used by the parser, or correct the labels and regenerate the affected tables. This finding concerns the new stage/step interpretation; it does not establish that the separate 69/44 gate totals are wrong.

The cited raw evidence is only under /tmp/readj/. Please attach durable logs and the parsing commands so the mapping and derived overlap tables can be checked. The conclusion that no single-op repair can clear the residual also needs evidence beyond its spread across prompts; the report itself identifies a shared L0 origin and further implementation hypotheses.

This PR currently conflicts with main, which already contains the memo/residency fixes and corrected gate indexing. Reconcile that base before requesting acceptance. I have not reproduced the P150 runs or completed the mutation/hardware gates: Python, the native build toolchain, and device assets are unavailable in this session. No merge recommendation yet.

…cted walk

The prior hard-flip classification was computed on a shifted chain
(generated_ids[step-1] fed instead of generated_ids[d-1]), so its per-step
teacher-forced numbers answered the wrong question. This re-runs the
adjudication on 2c4efce's corrected walk and re-derives the residual.

Both arms reproduce the corrected counts exactly (ON 69 tf flips: 29
near-tie, 40 HARD, worst 10.617 nats; OFF 44: 27/17, worst 8.647) - the
residual is real. The drift origin is unchanged (L0 MoE input, first
divergent stage at every decode step from step 0, 1-bf16-ULP class), and
the router-gain mechanism HOLDS: CPU-vs-TT top-6-of-384 expert-set overlap
averages 3.0-3.7 of 6 at every layer under the golden prefix, and the new
route_scores probe shows the top6-top7 selection margin below 0.05 nats at
26-44 of 50 layers at every hard flip, while top1-top2 stays robust. The
flips SPREAD across 7 of 8 prompts and the whole step range in both arms,
so the band-2.5 residual is classified STRUCTURAL (drift x router gain),
not a single-site repair; the levers are host-free shadow reconciliation
(halves the hard count) and f32 device attention/norm.

A scratch route_scores dump (sorted top-8 biased selection scores behind
VT_KOLIBRI1_TT_STAGE_DUMP) accompanies the existing stage probes; the
adjudication table lives in
docs/bench-evidence/kolibri1-tt-readjudication-20261012.md and the issue
resolution records the supersession.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]
@lu-zero
lu-zero force-pushed the row/tt-kolibri-readj branch from 1a90545 to 531deaa Compare October 15, 2026 03:39
…on evidence and parsers

The review on the P150 readjudication found the SCRATCH dbg TF log's raw
TFstep label is not the gate's decode position: the walk prints
d = i - input_len before the +1 that the logits' context position implies,
so raw `TFstep N` is decode position N+1 (the last prompt token's
TFstep -1 line is decode position 0). The test now prints both labels and
one short prompt-4 device leg verified the new wording. The doc gains an
explicit mapping section; its SCRATCH-derived step numbers are restated as
decode positions (the tf-p legs' decode-position-0 forward diverges at L0
dh2 like every later one, so the origin claim survives the relabel), and
the Q2 example is restated as the walk's first prompt forward, not a step.

The structural verdict is restated to what the evidence supports: the flip
spread across 7 of 8 prompts and the whole decode range is consistent with
the shared L0-origin drift amplification, and it rules out single-site
patches — it does not rule out that a chain-wide f32 fix (lever 2) clears
most of the residual, since that targets the origin itself.

The evidence the tables derive from is now durable: docs/bench-evidence/
kolibri1-readj/ carries the parsed OFF/ON.json and the corrected-label
tf_origin.json, the tf-p{1,5,4}.log raw legs gzipped, and the three
parsers that regenerate every doc table from them (the 22-98 MB full-gate
arm logs stay off-tree with their sizes and regeneration recipe recorded).
parse_overlap.py and parse_tf_origin.py were rerun against the committed
logs and reproduce the doc's Q1/Q2 numbers exactly.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants