Fall back to a preset of the same weights format when no (model, weights) row matches: prefill-cost prediction goes from 3.1x to 1.15x on a registered artifact by MichaelDementii · Pull Request #195 · Neroued/ninfer

MichaelDementii · 2026-09-06T19:26:57Z

Base: origin/master @ 487f8977. Rebased onto master 487f8977; the diff applies to it cleanly, with the same diff on its own base as the control. The measurements below were taken on ad0f3d38, which was master when the work was done, 83 commits back. I have not re-taken them here and I say so rather than implying they are fresh.

Originally checked on ad0f3d38 2026-09-05 with
git ls-remote https://github.com/Neroued/ninfer.git refs/heads/master rather than
through a clone's origin. This branch was written against a140e7ae and has been rebased
straight onto ad0f3d38 across every intervening head: no conflict, and
patch-id --stable of the change is unchanged. Head fb925985 -> 05f4f4a9, 1 commit(s)
above the base.

The two commits added since b8786751 are 863aa8a5 (fix(dflash): allow vision prompts) and ad0f3d38 (chore: add project funding information). Nothing here is
re-measured because of them, and that is settled by a line of source rather than by the
commit subject: everything the DFlash fix adds is allocated under
if (layout.spec.enable_dflash) (src/targets/qwen3_6/impl/state/round_state.cpp:176),
behind a speculative backend whose default is SpeculativeBackend::None in every entry
point (include/ninfer/types.h:78, src/serve/serve_options.h:46), reachable only
through an explicit --spec dflash that no measurement here passes.

Note that master and dev no longer agree: dev is one commit behind, at 863aa8a5.

Scope

One mechanism, four lines: when the compiled context-cost table has no row for an artifact's
(model_id, weights_id) pair, fall back to a row of the same weights_id before falling through to
the generic profile.

This corrects the prediction, not the throughput. Time to first token does not move; the
evidence for that is below, and the report does not claim performance.

It improves one of the three unmatched combinations. The other two are unchanged by construction

the paragraph on that is first, because it is the thing worth knowing before reading anything else.

Environment

One RTX 5090, sm_120a, driver 580.105.08, CUDA 13.1.115, gcc 13.3, Ubuntu 24.04, Release,
-DCMAKE_CUDA_ARCHITECTURES=120a. Dedicated host, no other tenant.
Base a140e7ae (origin/master at the time of writing). context_cost.cpp and
context_cost_defaults.cpp are untouched between that commit and the branch point.
Artifact Qwen3.6-27B NVFP4 (model_id qwen3.6-27b, weights_id nvfp4).
Stock ninfer-serve with --request-log-jsonl. Both arms are built in the same run from a
clean tree - origin/master and this one-file commit, nothing else applied - and each arm is
measured twice with the arms alternated.

Observation

find_prefill (src/runtime/engine/context_cost.cpp:210) matches on both fields:

return preset.model_id == model_id && preset.weights_id == weights_id;

The compiled table (context_cost_defaults.cpp) holds two rows:

`model_id`	`weights_id`	`chunk_ns`
`qwen3.6-27b`	`groupwise-int`	40 813 570
`qwen3.8-27b`	`nvfp4`	14 672 989

The targets register five combinations (src/targets/*/impl/package.cpp):

combination	row
`qwen3.6-27b` + `groupwise-int`	present
`qwen3.8-27b` + `nvfp4`	present
`qwen3.6-27b` + `nvfp4`	absent
`qwen3.8-27b` + `groupwise-int`	absent
`qwen3.6-35b-a3b` + `groupwise-int`	absent

An unmatched pair keeps generic_context_prefill_cost(), whose five coefficients are byte-identical
to the qwen3.6-27b + groupwise-int row. So an artifact on NVFP4 weights is costed with the
integer-weight profile, which the table itself puts at 2.78x the measured NVFP4 one.

The comment calls the generic profile a conservative estimate for an unknown model, which is
reasonable. The model here is not unknown: the combination is registered and supported
(src/targets/qwen3_6_27b/impl/package.cpp:92); it simply has no row.

Measured consequence on the stock binary, means over 27 requests, two runs:

quantity	predicted	actual	error
`predicted_now_ns` (this prefill step)	240.6 / 240.4 ms	78.2 / 76.3 ms	3.08x / 3.15x
`predicted_total_ns`	1259 / 1251 ms	-	-

That 3.1x agrees with the 2.78x ratio between the two profiles' coefficients, which is what makes
the substituted profile the explanation rather than a coincidence.

The same shape appears on a second artifact: qwen3.6-35b-a3b has no row either, and its prediction
reads 139.7 ms against 24.7 ms actual (5.7x). Different model, different machine, same mechanism.

Root cause

The lookup key is a pair; the table is populated on a diagonal. Two of the five registered
combinations are covered, and the fallback for the rest is not neutral - it is one of the two
measured profiles, the slower one.

Change

find_prefill_by_weights returns the first row whose weights_id matches, and
resolve_context_machine_cost consults it only after the exact pair misses:

if (const ContextPrefillPreset* prefill = find_prefill(*machine, model_id, weights_id)) {
    ...
} else if (const ContextPrefillPreset* by_weights = find_prefill_by_weights(*machine, weights_id)) {
    model.prefill  = by_weights->cost;
    prefill_source = ContextCostPresetSource::CompiledDefault;
}

Exact matches still win. An artifact whose weights format has no row at all still receives the
generic profile.

Correctness evidence

What changes, and what provably does not.

gap	costed today with	costed after with	change
`qwen3.6-27b` + `nvfp4`	generic (= integer profile)	`qwen3.8-27b` + `nvfp4` row	3.14x -> 1.15x
`qwen3.8-27b` + `groupwise-int`	generic	`qwen3.6-27b` + `groupwise-int` row - identical coefficients	none
`qwen3.6-35b-a3b` + `groupwise-int`	generic	same row - identical coefficients	none

The two unchanged rows are not a measurement but a reading of the table: all five coefficients of
generic_context_prefill_cost() equal the qwen3.6-27b + groupwise-int row exactly, so the new
path substitutes the same numbers those combinations already receive. The change cannot make any
registered combination worse.

The one combination it corrects, same artifact, same probe, two runs per arm, arms alternated,
means over 27 requests each:

arm	`predicted_now_ns`	actual prefill	error
`origin/master`, run 1	240.6 ms	78.2 ms	3.08x
`origin/master`, run 2	240.4 ms	76.3 ms	3.15x
this change, run 1	87.8 ms	76.3 ms	1.15x
this change, run 2	87.8 ms	76.3 ms	1.15x

predicted_total_ns falls from 1251-1259 ms to 456 ms. The corrected arm reproduces to the first
decimal across both runs, because the prediction is a function of the coefficients and the prompt,
not of the machine's state.

Time to first token does not move. Same runs, shared-prefix traffic, three prefix sizes:
69.4 vs 69.4 ms at 150 tokens, 83.3 vs 83.2 at 400, 122.2 vs 122.1 at 800. The materialization
planner's selection is empty on this traffic either way (stop_reason: time_budget,
last_selection.frontier_tokens: 0), so a corrected cost does not change what it picks. That is
reported because it bounds the claim, and because the opposite would have been the natural thing to
assume.

ctest: 104 tests, 104 passed, 0 failed on 36a3cb04. The submitted commit fb925985 differs
from it in the commit message only - both have tree 5a6f6c4b97ce7e6d4f385dd365d8e071f4c7bd3f and
the same patch-id 5bc6da3418d9, so the tested tree is the submitted tree. The suite is run one
test at a time -
build, run, delete - because the test executables are ~184 MB each and 104 of them do not fit
beside the model artifact on this host's 32 GB disk. Four of the 104 registered names are
variants of two parent executables (ninfer_kv_cache_append_{k8v4,nvfp4}_test,
ninfer_softmax_attention_{k8v4,nvfp4}_test) and are built through their parent target.

Tradeoffs

The heuristic's limit, stated plainly. Matching on weights format alone assumes two models of
the same format have comparable per-token cost. That holds for today's table, where both
groupwise-int entries are identical. It would not hold between a dense model and a sparse one of
the same format: qwen3.6-35b-a3b is MoE with about three billion active parameters,
qwen3.6-27b is dense. If a measured MoE row is ever added, the nearest profile should be chosen
by weights format and architecture class together. This change does not create that situation,
but it does not prevent it either.

Three consumers. Under load, none of them changes its decisions. ResolvedContextMachineCost
reaches the materialization planner, the shared-capture planner, and resource_manager.h, where
it supplies rebuild_ns - the cost of rebuilding a dropped checkpoint, which feeds retention and
eviction. That last one is the one to worry about: an over-predicted prefill makes checkpoint
rebuild look three times more expensive than it is. It was measured, and it does not move.

Four long shared prefixes with varying tails, 96 requests, at --max-concurrency 1, 4 and 8 -
8 being the ceiling the server itself enforces (--max-concurrency must be in [1,8]), so this is
the whole range the engine allows. Two passes per arm, arms alternated, both binaries built in the
same run. Counters are summed across the server's stats windows, because the throughput event
reports per-interval deltas rather than running totals.

concurrency	`predicted_now` base -> fix	`checkpoints_dropped`	`private_owners_evicted`	`searches`	`captures`	req/s
1	346.1 -> 126.7 ms	194 / 194	97 / 97	98 / 98	103 / 103	3.347 / 3.339
4	346.3 -> 126.9 ms	102 / 102	91 / 91	96 / 96	19 / 19	5.653 / 5.663
8	346.2 -> 126.8 ms	94 / 94	83 / 83	93 / 93	19 / 19	6.381 / 6.383

The prediction changes by 2.73x and not one pressure counter moves by one event. The only
disagreement anywhere is private_owners_degraded 3 against 4 in one of the four runs at
concurrency 4 - within that arm's own run-to-run spread, since the other three runs at that point,
including both base runs, all read 4. Base against base agrees exactly on every counter, so the
noise floor on these quantities is zero and the agreement is a statement rather than an inability
to tell them apart.

The counters do respond to something, and it is not the cost. Between concurrency 1 and 8
dropped checkpoints halve, captures fall fivefold, three counters that are zero at 1 become
non-zero at 8, and the degradation-unit distribution shifts from {0:1, 1:1, 3:97} to
{0:6, 1:7, 2:81, 3:5}. So the instrument is not stuck.

Which part of concurrency does that, I tested and could not confirm. Setting
--device-state-slots to 8 and then 16 at concurrency 1 - the same checkpoint capacity the
engine gives itself at concurrency 8 - raises reserved memory by a gigabyte per step, and leaves
checkpoints_dropped and private_owners_evicted at exactly 194 and 97, unchanged from the
default. I had predicted, in writing before the run, that this knob would move them; it does
not. What it does move is the planner's regime: budget_exhausted goes from 95 requests to 0,
stop_reason from time_budget to queue_exhausted, and the enumerated candidate set from 558
targets to 256. So checkpoint capacity changes how the planner searches without changing what
gets evicted, and whatever concurrency carries that does move eviction is something else.

That failed prediction strengthens the arms comparison rather than weakening it. Base and fix
agree exactly at every one of those slot settings too - including the regime where the search
exhausts its queue instead of its clock. The obvious objection to the original table, that the
price never had time to act because every search ran out of budget, is answered by measurement:
with budget_exhausted = 0 the answer is still identical.

The planner is working, and that makes the agreement stronger. It evaluates 400-600 targets
per request (targets_evaluated) and selects one, at 3 degradation units on 97 requests of 99.
The cost model's own output does change with this patch - predicted_future_loss_ns drops from a
maximum of 1.59 s to 0.58 s, the same ~2.7x - while the selected degradation distribution and
every pressure counter stay identical. So this is not a case of a planner that never runs; it is
a planner whose inputs move and whose choices do not.

The constant 3 is not a ceiling. degradation_units
(src/targets/qwen3_6/impl/runtime/pressure_planner.h:129) is a tally of the consequences of a
decision already taken - evicted continuation, plus state, KV and checkpoint changes - and no term
in it comes from the cost model. It decomposes exactly: at concurrency 1 the units sum to 292 over
99 requests, and the independently reported counters give 97 evictions plus 194 checkpoint drops
plus one state change, which is 292. Three units is "evict one continuation and drop its two
checkpoints", repeated because the traffic is uniform.

Why it does not move, read in the code rather than inferred. rebuild_ns is genuinely read
from this cost model (resource_manager.h:1908, :1966, feeding baseline_saving in
context_portfolio_value.h:69), so it is not a dead computation. But it prices an outcome
rather than choosing it: the victim's disposition arrives already decided in
assessment.owner_outcomes, and the planner only folds its consequences into a candidate cost
(materialization_planner.h:857). And the quantity that looks like the outcome of that
comparison is not one: degradation_units is a tally, as shown above. The planner's
own search budget cannot move either: min(5 ms, incumbent.cost.total_ns / 20) binds at 5 ms
whenever total_ns >= 100 ms, and both arms are far above it (1875 ms and 686 ms).

A second consequence, and it is about the clamp rather than about this change. The budget is
written as a proportion of the incumbent's cost with a ceiling: total_ns / 20, capped at 5 ms.
The proportional half only has an effect below total_ns = 100 ms. With the prediction inflated
threefold, the cap is saturated on every request measured here - 1875 ms - so the proportional
term never takes effect at all. Corrected, it is still saturated on this artifact (686 ms), but
the distance to the point where the proportion begins to act is three times shorter. Where a
correct preset puts the prediction under 100 ms, that term would take effect for the first
time. I have not measured that regime and this artifact cannot reach it; it is named because
whoever measures the three missing rows will reach it before anyone else, and should look.

A note on why nothing is reused here, since the logs invite the wrong conclusion. Across all
twelve runs context_cache.selections.root is 99 of 99 and reused_prompt_tokens is 0, on
traffic built from four repeated prefixes. That is expected, not a defect: this probe drives the
OpenAI chat endpoint with no cache markers, and docs/serving.md:696 states that block-level
cache_control is what "creates explicit shared-prefix candidates". Reuse that needs no marker
is turn closure, and it works - a separate multi-turn probe on the same stock binary reports
reused_prompt_tokens = 38 and private_turn_closure = 4. So the zero above measures a probe
that did not ask for caching, and it carries no claim about the engine.

What this does not establish. That cost never affects eviction. Only that on this traffic, at
this cache geometry, across the whole concurrency range the engine permits, a threefold change in
the predicted cost changes no decision.

No new coefficients. The change adds no measured numbers, so it carries none of this host's
hardware into the product. Filling the three missing rows properly is a separate job and belongs
on the maintainer's hardware; the tooling for it already exists in the tree
(bench/context_cost/, target ninfer_context_cost_bench, emitting
ninfer_context_cost_calibration).
Workspace, memory, kernels: untouched. This is host-side cost resolution at startup.

Reproduction

No patch is needed to see the defect - it reproduces on a stock build:

ninfer-serve models/qwen3_6_27b_nvfp4.ninfer --port 18080 --max-context 40960 \
  --prefill-chunk 8192 --max-concurrency 1 --greedy --no-thinking \
  --request-log-jsonl /tmp/req.jsonl
# send any request, then:
python3 -c "
import json
for line in open('/tmp/req.jsonl'):
    o = json.loads(line)
    m, t = o.get('materialization'), o.get('timings_seconds')
    if m and t:
        print(m['predicted_now_ns']/1e6, 'ms predicted vs', t['prefill']*1000, 'ms actual')
"

--context-cost-presets FILE does not mask this in normal operation: docs/serving.md:766
describes it as an optional runtime registry whose default is "generic + compiled defaults", and
the repository ships no such file.

🤖 Generated with Claude Code

…t row matches The compiled context-cost table is keyed by the pair (model_id, weights_id) and carries two rows: qwen3.6-27b/groupwise-int and qwen3.8-27b/nvfp4. The targets register five combinations, so three of them - qwen3.6-27b/nvfp4, qwen3.8-27b/groupwise-int and qwen3.6-35b-a3b/groupwise-int - find no row and fall through to generic_context_prefill_cost(). That generic profile is, coefficient for coefficient, the qwen3.6-27b/groupwise-int row. For an artifact whose weights are nvfp4 that is the wrong shape of cost: the same table already holds a measured nvfp4 profile whose chunk_ns is 14.7 ms against the integer profile's 40.8 ms. find_prefill_by_weights picks a row of the same weights_id when the exact pair misses, before falling through to the generic profile. Nothing else changes: the exact-pair lookup still wins, and an artifact whose weights format has no row at all still receives the conservative generic estimate. For the two groupwise-int gaps the new path selects the qwen3.6-27b/groupwise-int row, whose coefficients are byte-identical to the generic profile, so their behaviour is unchanged by construction. The one combination this actually corrects is qwen3.6-27b/nvfp4. Measured on that artifact with a stock server and --request-log-jsonl, two runs per arm with the arms alternated and both binaries built in the same run from a clean tree, means over 27 requests: predicted_now_ns falls from 240.6/240.4 ms to 87.8/87.8 ms against an actual prefill of 78.2/76.3 ms, i.e. from 3.08-3.15x over to 1.15x, and predicted_total_ns from 1251-1259 ms to 456 ms. Time to first token does not move - 69.4 vs 69.4, 83.3 vs 83.2, 122.2 vs 122.1 ms at shared prefixes of 150, 400 and 800 tokens. This corrects the prediction, not the throughput, and does not claim otherwise. Under load the decisions do not change either. Four repeated prefixes with varying tails, 96 requests, at --max-concurrency 1, 4 and 8 - the whole range the server accepts - two passes per arm with the arms alternated: the prediction changes by 2.73x and every pressure counter is identical, including checkpoints dropped, owners evicted, searches and completed captures. Base against base agrees exactly, so the noise floor on those counters is zero. The same holds with --device-state-slots raised to 8 and 16, where the planner stops exhausting its time budget and exhausts its candidate queue instead, so the agreement is not an artifact of searches being cut short. Matching on the weights format alone is a heuristic and its limit is worth stating: it assumes two models sharing a weights format have comparable per-token cost. That holds for the rows in the table today, since both groupwise-int entries are identical, but a dense and a sparse model of the same format would not be interchangeable. A future table with a measured MoE row would want the nearest profile chosen by weights format and architecture class together.

chatgpt-codex-connector · 2026-09-06T19:27:06Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-09T22:34:02.257763Z	`f0391ca`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

MichaelDementii · 2026-09-09T19:26:45Z

Checked on b88c0f6. Nothing here to re-measure — the claim is a prediction ratio, not a time — so
what mattered was whether the change and its target are still real:

merges into today's master clean, ctest 114/114;
the gap is intact: find_prefill still matches on the (model, weights) pair alone, so a
registered model without a row of its own still falls through to the generic profile rather than
to another model in the same weight format.

src/runtime/engine/context_cost.cpp has had no commits since ad0f3d38.

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f84adafea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-09T19:30:10Z

+        } else if (const ContextPrefillPreset* by_weights =
+                       find_prefill_by_weights(*machine, identity.weights_id)) {


Add behavioral coverage for the weights fallback

The passing suite does not exercise this new branch: test_context_cost.cpp covers an exact compiled match and a completely unknown identity, but not a known hardware/weights pair with an unmatched model. Consequently, the suite would still pass if this fallback were removed or returned the wrong preset, even though selecting the NVFP4 coefficients is the entire behavioral change and affects planner inputs. Add a resolution test for qwen3.6-27b/nvfp4 that verifies the selected coefficients and source while retaining exact-match precedence.

AGENTS.md reference: AGENTS.md:L78-L81

Useful? React with 👍 / 👎.

The suite passed an exact match and a wholly unknown identity, so it would still pass with the fallback removed. Resolve a registered model that has no row of its own and assert it takes the coefficients of the same weights format, that an exact (model, weights) row still wins, and that an unregistered format still reaches the generic profile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

MichaelDementii · 2026-09-09T21:36:21Z

Correct — the suite would have passed with the fallback removed. Covered in 5ae1335b:
test_weights_fallback resolves qwen3.6-27b / nvfp4, which has no row of its own, and asserts
it takes the nvfp4 coefficients, that the exact (model, weights) row still wins over them, and
that an unregistered weight format still reaches the generic profile.

Strength control, since a passing test proves nothing by itself: with
src/runtime/engine/context_cost.cpp reverted to the base and nothing else changed, the suite
fails on exactly the two new assertions —

FAIL: a model with no row of its own did not take the coefficients of the same weights format
FAIL: the weights fallback picked a row of a different weight format

and passes again when it is restored. The comparison is ContextPrefillCost as a whole, not one
coefficient, so no number is hardcoded into the test. clang-format clean.

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ae1335b26

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-09-09T21:40:07Z

+    expect(int_row.model.prefill.chunk_ns != nvfp4_row.model.prefill.chunk_ns &&
+               resolve("qwen3.6-27b", "groupwise-int").model.prefill == int_row.model.prefill,
+           "an exact (model, weights) row no longer wins over the same-weights fallback");


Test exact precedence with distinguishable same-weight rows

This assertion does not actually protect exact-match precedence: int_row is resolved from the only compiled groupwise-int row, so removing the exact (model, weights) lookup would still return that same row through find_prefill_by_weights and the assertion would pass. The fresh evidence in this revision is that both sides resolve qwen3.6-27b/groupwise-int, rather than two distinct models sharing a weights format with different costs. Use distinguishable same-weights rows so the test fails if the fallback is consulted before the exact match.

AGENTS.md reference: AGENTS.md:L78-L81

Useful? React with 👍 / 👎.

Reporting a borrowed profile as CompiledDefault hid two things: a reader of the resolution summary could not tell coefficients measured for this model from ones taken from a sibling, and the order of the two lookups was not observable from outside, so a test could not hold it. Both follow from naming the source. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

MichaelDementii · 2026-09-09T22:29:41Z

Right, and thank you — that assertion was worthless. groupwise-int has exactly one compiled row,
so both lookups return it and the check passed with the exact match removed.

There is no second row of any format to build the test on, so f0391cad makes the order observable
instead of guessing at it: the fallback now reports its own source, CompiledWeightsFallback,
rather than borrowing CompiledDefault. The test then asserts that the two exact identities still
resolve as CompiledDefault — which is false the moment the fallback is consulted first.

Strength control, with the two lookups swapped in resolve_context_machine_cost and nothing else
changed:

FAIL: the two compiled rows this test compares are not distinguishable
FAIL: an exact (model, weights) row no longer wins over the same-weights fallback
FAIL: compiled transfer and prefill defaults did not resolve independently

and with the fallback removed entirely it still fails on the two fallback assertions. Both pass
when restored.

The new source is worth having on its own: a reader of the resolution summary can now tell
coefficients measured for this model from ones taken from a sibling — the request log prints the
name, compiled-weights-fallback. ninfer_request_log_test passes unchanged. clang-format
clean on all three files.

@codex review

chatgpt-codex-connector · 2026-09-09T22:34:00Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: f0391cadc5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… pick list corrected - Neroued#222 parent + Neroued#195 series were missing from the audit)

MichaelDementii mentioned this pull request Sep 7, 2026

perf(ops): warm L2 for the next projection from the MoE down tail, issuing the hint before the block barrier #206

Merged

chatgpt-codex-connector Bot reviewed Sep 9, 2026

View reviewed changes

Gevil added a commit to Gevil/ninfer that referenced this pull request Sep 11, 2026

docs(adoption-v2): V2-T11 launched (12-commit branch v2/t11-md-clean;…

b945cba

… pick list corrected - Neroued#222 parent + Neroued#195 series were missing from the audit)

Navigation Menu

Uh oh!

Fall back to a preset of the same weights format when no (model, weights) row matches: prefill-cost prediction goes from 3.1x to 1.15x on a registered artifact - #195

Fall back to a preset of the same weights format when no (model, weights) row matches: prefill-cost prediction goes from 3.1x to 1.15x on a registered artifact#195
MichaelDementii wants to merge 3 commits into
Neroued:masterfrom
MichaelDementii:fix/context-cost-weights-fallback

MichaelDementii commented Sep 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 6, 2026 •

edited

Loading

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Sep 9, 2026

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Sep 9, 2026

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		} else if (const ContextPrefillPreset* by_weights =
		find_prefill_by_weights(*machine, identity.weights_id)) {

Uh oh!

Conversation

MichaelDementii commented Sep 6, 2026

Scope

Environment

Observation

Root cause

Change

Correctness evidence

Tradeoffs

Reproduction

Uh oh!

chatgpt-codex-connector Bot commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

MichaelDementii commented Sep 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chatgpt-codex-connector Bot commented Sep 6, 2026 •

edited

Loading