Repository navigation
Conversation
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
@mudler @richiejp Focused source review at 1a905450ce4e7995afa9a205c26e35953ef180ac, compared with current main 50d425bec:
The new evidence needs an explicit mapping between the scratch diagnostic's step labels and the gate's decode positions. In tests/vllm/models/test_kolibri1_tt_b2ii.cpp:988-1016, the final prompt token produces decode-position-0 logits but prints TFstep -1. The next iteration feeds generated_ids[0], produces decode-position-1 logits, and prints TFstep 0. Thus the new document's correlation of SCRATCH dbg TF prompt stages with hard flips at the same numbered decode step is not established by those labels. Please include the mapping used by the parser, or correct the labels and regenerate the affected tables. This finding concerns the new stage/step interpretation; it does not establish that the separate 69/44 gate totals are wrong.
The cited raw evidence is only under /tmp/readj/. Please attach durable logs and the parsing commands so the mapping and derived overlap tables can be checked. The conclusion that no single-op repair can clear the residual also needs evidence beyond its spread across prompts; the report itself identifies a shared L0 origin and further implementation hypotheses.
This PR currently conflicts with main, which already contains the memo/residency fixes and corrected gate indexing. Reconcile that base before requesting acceptance. I have not reproduced the P150 runs or completed the mutation/hardware gates: Python, the native build toolchain, and device assets are unavailable in this session. No merge recommendation yet.
…cted walk The prior hard-flip classification was computed on a shifted chain (generated_ids[step-1] fed instead of generated_ids[d-1]), so its per-step teacher-forced numbers answered the wrong question. This re-runs the adjudication on 2c4efce's corrected walk and re-derives the residual. Both arms reproduce the corrected counts exactly (ON 69 tf flips: 29 near-tie, 40 HARD, worst 10.617 nats; OFF 44: 27/17, worst 8.647) - the residual is real. The drift origin is unchanged (L0 MoE input, first divergent stage at every decode step from step 0, 1-bf16-ULP class), and the router-gain mechanism HOLDS: CPU-vs-TT top-6-of-384 expert-set overlap averages 3.0-3.7 of 6 at every layer under the golden prefix, and the new route_scores probe shows the top6-top7 selection margin below 0.05 nats at 26-44 of 50 layers at every hard flip, while top1-top2 stays robust. The flips SPREAD across 7 of 8 prompts and the whole step range in both arms, so the band-2.5 residual is classified STRUCTURAL (drift x router gain), not a single-site repair; the levers are host-free shadow reconciliation (halves the hard count) and f32 device attention/norm. A scratch route_scores dump (sorted top-8 biased selection scores behind VT_KOLIBRI1_TT_STAGE_DUMP) accompanies the existing stage probes; the adjudication table lives in docs/bench-evidence/kolibri1-tt-readjudication-20261012.md and the issue resolution records the supersession. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5.3-flash [maki]
1a90545 to
531deaa
Compare
…on evidence and parsers
The review on the P150 readjudication found the SCRATCH dbg TF log's raw
TFstep label is not the gate's decode position: the walk prints
d = i - input_len before the +1 that the logits' context position implies,
so raw `TFstep N` is decode position N+1 (the last prompt token's
TFstep -1 line is decode position 0). The test now prints both labels and
one short prompt-4 device leg verified the new wording. The doc gains an
explicit mapping section; its SCRATCH-derived step numbers are restated as
decode positions (the tf-p legs' decode-position-0 forward diverges at L0
dh2 like every later one, so the origin claim survives the relabel), and
the Q2 example is restated as the walk's first prompt forward, not a step.
The structural verdict is restated to what the evidence supports: the flip
spread across 7 of 8 prompts and the whole decode range is consistent with
the shared L0-origin drift amplification, and it rules out single-site
patches — it does not rule out that a chain-wide f32 fix (lever 2) clears
most of the residual, since that targets the origin itself.
The evidence the tables derive from is now durable: docs/bench-evidence/
kolibri1-readj/ carries the parsed OFF/ON.json and the corrected-label
tf_origin.json, the tf-p{1,5,4}.log raw legs gzipped, and the three
parsers that regenerate every doc table from them (the 22-98 MB full-gate
arm logs stay off-tree with their sizes and regeneration recipe recorded).
parse_overlap.py and parse_tf_origin.py were rerun against the committed
logs and reproduce the doc's Q1/Q2 numbers exactly.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]
The teacher-forced instrument's earlier counts were computed against a shifted chain (fixed in #3453); this re-runs the adjudication on the corrected walk and delivers the decision input for the P150 band-gate standard. Adjudication table (reproduced exactly, both arms, cold-cache recipe): host-free ON 170 free divergences, 69 tf flips (29 near-tie, 40 HARD, worst 10.6 nats); host-free OFF 158 free divergences, 44 tf flips (27 near-tie, 17 HARD, worst 8.6). The zero-artifact stop condition did not fire — the residual is real.
Findings (corrected walk):
Recommended gate-standard input: keep the 2.5-nat/top-5 adjudication; record the P150 band as STRUCTURAL (spread), not fixable-by-a-patch. The levers, in order: (1) host-free shadow reconciliation (halves the hard count: 40 -> ~20 ON), (2) f32-enabling the device attention/norm chain (removes the origin — a perf/substrate question), (3) router-margin widening is a model change, out of scope for this row. The flip step is decided by the LM-head margin at that step, not by where the drift begins.
Raw evidence in the doc; probes committed behind VT_KOLIBRI1_TT_STAGE_DUMP. Issue: ISSUE-LOCAL-01M4ER0E9HHM95YZYJB7T5FECN (Resolution updated with the corrected analysis).
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]