Skip to content

llama: retry pipeline reservation with fewer copies - #20

Merged
novkien merged 1 commit into
masterfrom
codex/flash9-copy-fallback-109
Oct 3, 2026
Merged

novkien merged 1 commit into
masterfrom
codex/flash9-copy-fallback-109

Conversation

@novkien

@novkien novkien commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Addresses #9

Overview

When pipeline copy-buffer reservation fails, retry with two copies before the existing one-copy generic fallback. Successful four-copy reservations stay unchanged; this does not set a global two-copy default. The Flash acceptance checks reject the one-copy result; generic one-copy fallback remains available.

Validation

  • Focused test-alloc passed on CPU and on the matched native/RPC build.
  • Matched source 109ab654 plus this patch passed a proxy :8082 capacity screen with RPC-first placement, ub=1024, two slots, pipeline enabled and all 49 target plus 50 draft layers on GPU. At shared contexts 233472, 249856 and 266240, the initial four-copy reservation failed and the two-copy retry succeeded. One cold 8192+512 request completed at each context.
  • PP/TG were 492.56/69.69, 503.59/71.43, and 485.60/70.01 tokens/s at those contexts, respectively. These are single cold requests per context, not a performance qualification.

Remaining qualification

This PR does not claim maximum-context or near-full-pool acceptance. Long-context behavior, cancellation, simultaneous two-slot qualification, combined-route qualification and final deployment remain outstanding in issue #9. Root will continue this qualification after merge; this PR is not final task acceptance.

Requirements

  • I have read and agree with the contributing guidelines.
  • AI usage disclosure: YES - Codex authored and independently reviewed the implementation and tests under the owner-approved plan; this submission is AI-assisted.

@novkien

novkien commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Pre-merge handoff to Codex /root: this issue-linked draft is at reviewed head 68136f094fcec0d898eb037552dc1f7af621fbca on base 109ab6544090ba9922496c8ca4b34fe6751120ca. The issue #9 publication report records the scope and evidence.

Focused test-alloc passed on CPU and the matched native/RPC builds. The proxy :8082 capacity screen retained all target/draft layers on GPU with two slots and pipeline enabled. It passed one cold 8192+512 request at each of c=233472, 249856 and 266240 after four-copy reservation failed and two-copy retry succeeded. This is capacity evidence only.

Maximum-context and near-full-pool acceptance, long-context behavior, cancellation, simultaneous two-slot qualification, combined-route qualification and final deployment remain outstanding. PR remains draft; merge is on hold for root's continuing qualification.

@novkien
novkien marked this pull request as ready for review October 3, 2026 12:44
@novkien
novkien merged commit c09fa60 into master Oct 3, 2026
11 of 32 checks passed
@novkien

novkien commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Post-merge handoff: PR #20 is squash-merged at c09fa602cc64e30eef5db0cb3784667a91172452, from reviewed head 68136f094fcec0d898eb037552dc1f7af621fbca. Issue #9 remains OPEN; the post-merge issue report records the remaining acceptance work.

Root will continue exact merged-source deployment and proxy verification, maximum-context/near-full-pool, long-context, cancellation, simultaneous two-slot and combined-route qualification after merge. The combined trace reported by root (70,000 new tokens plus 512 generated, GPU-resident layers, two-copy pipeline) is state/route evidence only. No final acceptance or deployment is claimed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants