[novkien]

Overview

The generic MMVF tuner chooses 160 threads for the BF16 K320/M10240/W4 hyperconnection projection on Volta. This adds cross-warp reductions and barriers for each output column. Use one 32-thread warp per output row for this exact non-fused ordinary matmul. Other types, shapes, architectures, ID and fusion paths retain the existing selection.

Addresses #9.

Validation

The production-form native/RPC pair passed six CPU-reference cases on V100 and RTX 5060 Ti: W1/W3/W4/W5, padded K stride, and broadcast weight channels. Independent source review found no blocker. Model-naming validation, EditorConfig 3.0.3 checks, and git diff --check pass. A separate instrumented build confirmed the actual hc_gate route; its timings are excluded.

Five alternating baseline/candidate pairs used the same three frozen 8192-token prompts, 512 generated tokens, one fixed warmup after each fresh load, and the owner's C266240/ub768/two-slot/pipeline-on/full-GPU configuration. Each measured request used one active slot; the final two-slot capacity checks are functional validation, not a concurrent-throughput gain claim. Candidate source is c7de202 plus the exact production diff committed as 8af9376; final merged-revision packages remain a deployment step. The unchanged paired analyzer validates all 30 samples, identities, fixture hashes and VRAM records.

Fixture	Baseline median TG	Candidate median TG	Paired geometric gain	Paired 95% interval
code	57.87 tok/s	60.05 tok/s	3.34%	1.79% to 4.50%
prose	60.21 tok/s	61.92 tok/s	2.38%	1.49% to 3.27%
structured	56.21 tok/s	70.01 tok/s	22.76%	19.22% to 24.92%

Structured passes the predeclared 5% TG gain/above-parity interval gate. Code and prose pass TG nonregression but do not reach the 5% gain threshold. The three-family geometric result is 9.11% (7.28% to 10.41%), descriptive only: each pair combines its three families before the same exact five-pair bootstrap. Actual MTP verification-step speed improves 4.32%; changes in accepted draft tokens also contribute to TG. PP ratios are 0.9982-1.0019, and observed peak VRAM is unchanged.

Client wall time and TTFT are mixed; their 0.97 point checks fail for prose and structured. In the final candidate group, those two requests have about 13.2 and 16.1 seconds outside the reported native PP+TG intervals, followed by the configuration-read timeout below. The cause of that overhead is unconfirmed. All samples and client timings are retained; this PR claims native TG improvement, not general client-latency improvement.

The last post-measurement configuration GET timed out after all measurements and counter checks completed. Its retained normal-restoration configuration read, taken before any configuration write, exactly matches the pre-measurement route and revision. A separate derived recovery record uses that observation and preserves the failed original receipt and every raw sample; no timings were repeated, removed or changed. Baseline restoration and identity verification completed normally.

The 512-token timing samples stop during reasoning and do not establish answer quality. Three separate bounded final-answer fixtures show no new failing case: both arms pass prose/structured and fail the same code-comprehension fixture. This pre-existing answer failure is retained, not counted as a numerical or semantic-equivalence pass.

Final exact-merged build, deployment and full-pool one-/two-slot runtime acceptance remain under issue #9. Merge does not close acceptance.

Requirements

• Contributing guidelines reviewed for this owner-authorized private-fork change.
• AI usage disclosure: YES. Codex prepared the implementation, tests, review, and validation under the owner's approved scope.

[novkien]

Draft PR #22: #22

Reviewed head: 8af9376 (base c7de202).

The production pair passed six CPU-reference cases on the native V100 and RPC RTX 5060 Ti. Model-naming validation (152 mappings), EditorConfig 3.0.3 checks on both changed files, and git diff --check passed; independent source review found no blocker.

Five alternating TG pairs and final runtime/context acceptance remain pending. Keep the PR in draft until those results are available.

[novkien]

Work report - TG qualification and pre-merge handoff

Owner and scope: Codex task /root, issue #9, PR #22 at reviewed head 8af9376af703ec4710b4cbed626457c52d5b9069. The owner-approved endpoint includes native/RPC build, deployment and live acceptance; merge alone is not completion.

Done and evidence: The exact production diff passed six unchanged CPU-reference cases on both V100 and RTX 5060 Ti, focused style/model checks, and independent source review. Five alternating pairs with three identical frozen cold8192+512 fixtures completed on the fixed C266240/ub768 configuration. Paired native TG gains are code 3.34% (95% interval 1.79-4.50%), prose 2.38% (1.49-3.27%), and structured 22.76% (19.22-24.92%). Structured meets the 5% gain/above-parity interval gate; the other TG cells pass nonregression. PP and peak VRAM are unchanged within the recorded measurements. Per-verification-step speed gain is 4.32%; TG also reflects different MTP acceptance.

The original outer run ended on a 10-second configuration-read timeout after all 30 measurements and final counters completed. Independent recovery review confirmed that the retained restoration pre-write configuration snapshot is byte-identical to the pre-measurement snapshot, postdates all measurements, and precedes the first route write. A separate derived record preserves the failed original receipt and every raw measurement. Existing analysis reports complete, no issues, five valid pairs per cell; no timings were rerun, removed or changed. Native/RPC baseline restoration and private hold release were verified.

Client wall/TTFT results remain mixed and fail their 0.97 point flags for prose/structured. The final candidate group has about 13.2/16.1 seconds outside native PP+TG timings on those requests; its cause is unconfirmed. No general client-latency gain is claimed. Timing samples end during reasoning. Separate finished-answer probes pass prose/structured on both arms and fail the same code fixture on both; no new failing case was observed, without claiming broad semantic equivalence. Candidate quality probing also completed normal baseline restoration and hold release.

Remaining: Exact merged-revision paired build and source convergence, then full-pool one-/two-slot generation, MTP cache/cancellation and sequence-isolation checks on that pair. The unchanged C266240/ub768/two-slot/pipeline/full-GPU constraints remain mandatory. Issue #9 stays open.

Next: The delivery subagent may mark this exact reviewed PR ready and merge within the existing owner authorization after rechecking the open issue, head and enforced gates. Root retains runtime build/deployment and acceptance responsibility. No proxy source change or proxy restart is part of this delivery.

[novkien]

Work report - exact merged pair built, runtime acceptance next

PR #22 is merged as 5423bddf352adc41a8cc59ed02d38a511b333d30; its entire source tree matches reviewed head 8af9376. Both physical fork checkouts fast-forwarded cleanly from c7de202 to this exact merge on master, with no other source changes.

Immutable package final-bf16-5423bddf35 was built for native V100 and RPC RTX 5060 Ti. Both manifests identify the merged revision with an empty source diff, with 33/24 hashed runtime/test files. Both focused allocator runs pass. The maintained CPU-only recurrent-state checkpoint/rollback regression and its small-model-generation fixture pass (2/2 CTest cases); this proves those native API cases on dummy models, not Flash GPU checkpoint equivalence.

The build/source-convergence hold was released and verified inactive. Live binaries were not replaced by the build. Codex /root now proceeds with normal model activation and final C266240/ub768 two-/one-slot near-full generation, MTP/cache/cancellation/isolation, bounded answer comparison, and identity/configuration checks. The previous exact c7 pair remains the rollback artifact. Issue #9 stays open until runtime acceptance completes; no proxy restart is involved.

[novkien]

Final deployment and acceptance

Exact merge 5423bddf352adc41a8cc59ed02d38a511b333d30 is installed and active as the matched native/RPC pair. Both physical checkouts converged cleanly; final running identities and persisted route match. C266240/ub768, b1024, two slots, pipeline, Q8 KV, MTP max3/min1, full 49 target/50 draft GPU layers and safe P2P remain intact.

The final checks passed:

• Two cold 116224+16384-token lanes, each with its own nonce and no other-lane nonce; actual both-generating population reached 265029/266240.
• One cold 233216+27904-token sequence, 261120/262144 per-slot capacity, with clean completion and 1024 tokens of slack.
• MTP accept/reject, cached continuation, cancellation and a successful follow-up.
• A 98304-token long-prefix request reused 72941 tokens, processed 25363 new tokens and generated 512 tokens. This is functional cache coverage, not a speed comparison.
• Three fresh finished-answer fixtures retained the baseline's two passing fixtures and one failing code fixture. The wrong code answer differed between arms; no broad answer-equivalence or quality-pass claim is made.

The first outer acceptance receipt failed on an RPC SSH sampler timeout before the one-slot request began, after which normal rollback and lease release completed. The model request itself passed. Hash-bound, independently reviewed request-window telemetry recovery preserved that failed receipt and reused only the valid near-full runs; activation, MTP, quality and final identity/configuration checks then ran fresh. The final long-cache check also finished with unchanged identities/route, idle slots and its lease released.

The five-pair TG qualification and its client-wall/TTFT limitations remain as documented above. The exact-merge CPU dummy checkpoint/rollback CTest passed; no Flash GPU checkpoint-equivalence claim is added. The current full outcome, family dispositions, evidence hashes and rollback record are in issue #9. Independent final artifact/report review found no blocker. All task-owned idle leases were released and verified inactive.