Repository navigation
External pull requests of 2026-10-22 (33 PRs, one CI) - #2284
Merged
Merged
Conversation
Ported from the qlora-train branch (pre-v1.0 base): safetensors adapter format with base-fingerprint gating (LORA_UNSAFE=1 to override), atomic writer, CPU residual application via ADAPTER=<dir>. Without an adapter, inference output is bit-identical (tiny oracle TF 32/32 on this base). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full training path from the qlora-train branch, ported to colibri.c: manual backward through the GLM block with activation checkpointing (routing replay; recompute bitwise-equal to retention), expert-grouped streaming with row-batched fwd/bwd, budget manager (shed expert cache before the OS swaps), masked-CE SFT dataset reader, atomic checkpoints with bitwise resume, coli_train CLI. Gates: test_train_linear (PyTorch f64 parity), test_train_tiny (trajectory vs torch, rel 2.4e-6), test_train_resume (bitwise) — all wired into make test. Real-model probe on this base reproduces the M7 baseline exactly (loss 3.5764, 239s/step, identical expert I/O). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged around the new r_top8_par work; metal-test green for both suites. Dispatch is opt-in (COLI_METAL=1, COLI_TRAIN_METAL_MIN rows) and off by default: no wall-clock win yet (experts are fmt=4 grouped on CPU) and the f32 GEMM forward numerics differ from the CPU idot path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
m7_smoke.sh (build -> doctor -> 1+10 steps -> adapter-in-inference) and m8_overfit.sh (overfit + on/off eval), adapted to the colibri binary name; persona + dolly tokenized datasets; the M7 runbook and the implementation brief the milestones were built against. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tiny-oracle end-to-end repro for reviewers: base greedy matches the PyTorch reference 20/20 -> coli_train overfits a rank-16 adapter onto a chosen alternative continuation (~90s CPU) -> ADAPTER set reproduces the trained continuation 20/20 -> ADAPTER unset matches the reference 20/20 again. Needs torch once (tiny model generation); training and inference are pure C. tools/make_tiny_sft.py builds the coli-sft-v1 set from the oracle's own prompt ids, no tokenizer involved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One forward per question: force-closed think block, 'Answer:' suffix, 2 greedy tokens (DRAFT=0), first A-D letter wins. Same protocol both sides so the gap is the adapter's. Deterministic seeded subset spread across subjects; question pool fetched once from the HF datasets server and cached locally (nothing committed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The engine build broke on the Windows CI runners (lora.h:183) because mkdir(path, 0755) is POSIX; MSVC/mingw expose _mkdir(path) only. Also split on backslash so nested Windows-style paths create correctly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…3 missed Same UCRT64 failure class as lora.h: mkdir(dir,0755) is POSIX-only. The test already includes lora.h via colibri.c, so reuse LORA_MKDIR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- c/data/m8_tokenized is no longer committed: m7_smoke/m8_overfit generate it from data/m8_persona.jsonl via tools/prepare_sft.py on first use (tokenizer from the snapshot). Verified byte-identical to the removed fixtures when regenerated from the zai-org/GLM-5.2 tokenizer (seed 0, all six files). - lora.h added to the colibri rule prerequisites — caught by the new test_makefile_deps meta-test on current dev. - test_train_tiny stays in TEST_BINS (unchanged; noted per review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…the fork Not part of the training path (per review); the milestone brief stays at github.com/pavolbauer/colibri-qlora. The runbook under docs/plans/ remains, to be moved under docs/ as a follow-up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
dev now derives TEST_BINS from the tests/NAME$(EXE) rules, so the explicit TEST_BINS line made test_lora and the three test_train_* binaries run twice. The rules alone keep them in the default suite. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Conflicts: - .gitignore: union of the MiMo fixture lines and the training lines. - c/Makefile: header prerequisites are compiler-generated since #1741, so the colibri rule is taken from dev as is (lora.h no longer needs listing) and the four training test rules are reduced to their sources. coli_train has no coli_train.c for the engine discovery to find, so its generated dependencies are hooked up beside its own rule, the same way AUTODEP_BINS does it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The root .gitignore is taken from dev unchanged. The training lines sat at its tail, which is where every new engine appends its own, so the file conflicted after each merge to dev (#1813, then again within hours). The entries now live only in c/.gitignore, in its leading block: the trainer and its four test binaries, adapters, and the generated datasets. glm_tiny/, *.exe and *.d were already covered there. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Reuse the decoder's convolution, residual, normalization, and attention primitives for RGB/RGBA image encoding. Implement residual downsampling, single-frame temporal padding, posterior-mode extraction, and latent normalization; reject unsupported configurations and invalid inputs. Add dependency-free unit tests and a pinned Diffusers tiny oracle with encoder-only checkpoint, invalid-input, decoder-regression, and ASan/UBSan coverage. Vision encoding and DiT reference conditioning remain outside this change.
Run the normal and ASan/UBSan encoder oracles in the existing Qwen-Image CI job. Keep the encoder's default CPU unit tests dependency-free. Format the changed C code with the repository's clang-format settings. Document the sanitizer coverage and the limits of tiny-weight validation.
Clarify that the default CPU build and encoder unit tests do not require Torch or Diffusers. Other make check tests may use optional dependencies when installed. This changes documentation only.
Clarify encoder usage, input requirements and reference-test coverage. Simplify comments and test-tool descriptions without changing behavior.
The cross-slot KV prefix adoption loop in mux_submit only copied the f32 row arrays Lc/Rc (and Ic). Under KV8=1 or KV_TQ=1, kv_alloc leaves those pointers NULL and stores the cache in the byte rows Lc8/Rc8 plus the per-row scales Lsc/Rsc, so the adoption guards never fired: no row was copied, yet sc->len was advanced past the adopted range and prefill for those positions was skipped. Attention then read uninitialized Lc8/Rc8 rows, silently corrupting generated tokens. Extend the loop to copy the Lc8/Rc8 row bytes and the Lsc/Rsc scale rows for the adopted range when g_kv8 || g_tq is set, using the same byte widths (coli_kvq_row_bytes for TQ, kv_lora/qk_rope for KV8) and scale strides (coli_kv8_nscale groups for KV8) as kv_alloc. The f32 paths and the always-f32 Ic copy are unchanged; Ic has no quantized counterpart. Fixes #2096 Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
APIServer never set request_queue_size, so the kernel's accept queue -- the completed handshakes the accept loop has not drained yet -- held five. Once it is full Linux drops the next SYN (ListenOverflows in /proc/net/netstat) and that client sits out the SYN retransmit timer, 1 s then 2 s then 4 s, before the server has seen it; MAX_CONNECTIONS (64) and the per-address cap run after accept(), so they never saw it either. A dashboard's parallel fetches or a benchmark client's burst turned into multi-second stalls on a server with nothing to do. Measured on a 16-core box at load ~40: 100 barrier-synchronised connects, each GET /v1/models, the server in its own process, three runs per setting. Backlog 5: 85, 47 and 59 of 100 waited over 900 ms (max 3.3, 2.2 and 7.9 s) and ListenOverflows rose by 307, 91 and 245; with the caps raised to 1000 so only the queue is in play, 87, 57 and 65 of 100 (max 4.4, 3.1 and 2.9 s), +309, +175 and +187. Backlog 128: 0 of 100 in all six runs (max 0.40, 0.21 and 0.84 s with the default caps, 0.22, 0.16 and 0.30 s with them raised) and ListenOverflows unchanged. What is left with the default caps -- 1 to 38 connections closed right after accept -- is the per-address cap of 8 doing its job, the count the server logs as refused. 128 is werkzeug's and uvicorn's figure. The kernel caps it at net.core.somaxconn (4096 on current Linux, 128 on older ones), so 128 is honoured everywhere and holds the 64 connections MAX_CONNECTIONS admits at once with room for the next burst. tests/test_serve_listen_backlog.py: the attribute (128, at least MAX_CONNECTIONS, not socketserver's default) everywhere; on Linux, the kernel's backlog for the listening socket read back through ss (the LISTEN socket's Send-Q, min(128, somaxconn)) and the 100-connection burst with the post-accept caps raised: every connection gets 200, and the kernel's overflow counter and a client waiting a second or more must not both happen (the counter alone is another listener's, the wait alone a slow scheduler's; both together are our dropped SYN). Against openai_server.py unchanged it fails 3 of 3 runs -- 388 and 173 overflows with 91 and 58 of 100 waiting 1.0..7.6 s, the third run 7 of 100 timed out at 15 s -- and with the change 3 of 3 ok.
CANCEL was checked only between prefill chunks, so a cancel that arrived early in a chunk held the engine until the chunk finished. Smaller chunks are not the answer: routed experts are re-read once per chunk per layer, so every prompt would pay for a shorter wait (measurements in #1748). run_layers now asks a halt hook before each layer. forward_prefill arms it only for the duration of a chunk, after copying the KDA state (~149 MiB on GLM-5.3-Flash, once per chunk). If the hook fires, run_layers returns NULL, forward_span leaves `filled` where it was, and forward_prefill puts the KDA state back to the start of the chunk and stops. The layers already run have written their DSA rows for the chunk's positions; those are positional and the next prefill rewrites them. The cache is left at the previous chunk boundary, exactly as a between-chunk cancel leaves it, so a retry reuses it the same way. Decode and the other callers of run_layers never see the hook. If the copy can't be allocated, that chunk falls back to the between-chunk check. GLM53_VERBOSE prints `HALT layer <i> of <n>, back to <filled>`. The serve harness adds a case: prime 10 tokens, then a 1000-token prompt in one chunk with CANCEL sent after a delay. The delays are fractions of the chunk's time, measured first on the same machine without a CANCEL, since fixed delays miss the chunk on a machine ten times faster. It needs a HALT line, and retries other delays rather than passing without one. It expects `CANCEL 51 1000 0 10`, a retry that reuses the 10 tokens, and the same answer as a fresh engine. The parent engine never halts mid-chunk, and with the restore removed the answer differs; the harness fails on both. The pin, multimodal, vision-serve harnesses and the dashboard and context-exceeded unit tests pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With COLI_VK_CHAIN, run_layers hands every layer to g53c_forward before its own loop, so the halt between layers never ran there and a CANCEL waited for the whole chunk. g53c_forward_seg now checks g_layer_halt between layers (and between its row chunks), waits for the device, and returns -1; g53c_forward marks what the devices hold from `start` stale (g53c_halted) and run_layers returns NULL, so forward_prefill throws the chunk away and restores the KDA state exactly as on the CPU. The HALT line says "on the device" when the chain stopped it. A partial chain set run_layers' `begin` to its first CPU layer, so the `i > begin` check skipped that boundary; it now compares against the `begin` the caller passed. The serve harness's CANCEL-inside-a-chunk case timed its delays against the whole request; they now come from the forward in the PROF line. --vk-chain requires the cancelled chunk's HALT line to say "on the device" (the chain's forward count would not do: the priming request and decoding run on it either way) and, unless COLI_VK_CHAIN_ROWS is set, runs the case at 128 rows: the fixture's dense layers are recorded in a few ms and the device is then awaited for the whole chunk, so no delay lands between layers, while at 128 rows it is awaited, and the halt checked, per group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extract the COLI_KV_SHARE cross-slot adoption block from mux_submit into coli_kv_share_adopt_rows (behavior-identical move; the donor-selection invariant makes copying the donor hist equivalent to copying tmp) and add tests/test_kv_share_kv8.c, which exercises the routine directly for both KV8 and KV_TQ, per review feedback on #2103. The test builds a donor KVState by natively quantizing deterministic f32 reference rows (the 'full prefill') and a receiver KVState holding its own [0,from) prefix plus a poisoned [from,blen) window, then adopts [from,blen) and checks: the adopted Lc8/Rc8 byte rows and Lsc/Rsc scales are bit-identical to the donor's (including the asymmetric scale strides — Lsc holds coli_kv8_nscale groups per row under KV8 with group scales, Rsc a single float per row; one radius each under KV_TQ); the adopted rows dequantize to exactly the donor rows and within quantization tolerance of the pre-quantization reference; the receiver's own prefix and the token history are intact. Covers KV8 with gs=0 and gs=128, and KV_TQ with both codecs at 4 bits. Red/green verified: against the pre-fix adoption semantics the test fails (312 failures — adopted rows stay poisoned, nothing is copied); on this head it passes. Full 'make -C c test-c' is green with no regressions. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
…tocol v2 The expert-worker protocol moves every routed row as raw f32 both ways, so on a LAN the transfer, not the expert FFN, bounds a delegated layer. distributed-llama syncs its activations as blocks of 32 int8 behind one scale for a ~4x cut. Same idea here, behind an env switch and off by default: the cluster parity gate is token-exact and int8 rows are not. - COLI_CLUSTER_ACT=q8: the coordinator speaks a v2 header (magic, version 2, act, layer, D, I, n) whose rows are ceil(D/32) f32 block scales followed by D int8 -- qrow_i8 per block, the engine's own activation rounding, so no new quantizer -- and the worker answers in the same format. f32, the default, sends the v1 bytes unchanged, so a cluster of mixed binaries keeps working. - The worker accepts v1 and v2, names the versions it speaks when it refuses one, and refuses an unknown act word by name. A coordinator whose v2 request meets a closed socket names the worker and the COLI_CLUSTER_ACT=f32 escape hatch instead of just "failed". The worker's response-header writes free the batch when the peer is gone mid-header, as a bad request already did; the act word had widened that leak. - The coordinator reports bytes sent and received at exit, so the cut is measurable: on the tiny GLM oracle, 8203 -> 2316 B/position (3.54x) with the same argmax at every position. - test_cluster_protocol: the q8 codec byte for byte against qrow_i8 (partial last block, zero row, decode within half a block scale), a v2 q8 round trip over a socketpair, and a v2 header with act=f32 still carrying raw rows.
test_cluster_sharding's f32 gate is unchanged: the same assertions, now on shared _expert_worker/_coordinator helpers. ClusterQ8DriftTest runs the delegated oracle with COLI_CLUSTER_ACT=q8 on both fixtures and asserts that the f32 delegation stays exact, that the q8 run scores the same positions and mismatches at most Q8_MAX_MISMATCHES of them (measured 0 on fmt6 and fmt4; the bound carries no slack, raise it with a measurement), and that q8 moves less than a third of f32's bytes; it prints bytes per position for the record.
…ert on the worker
On top of the q8 protocol v2 (COLI_CLUSTER_ACT=q8, header magic version act
layer D I n): the default still speaks v1 bytes, so a cluster of mixed engine
builds keeps working. The coordinator speaks v2 when it needs q8 rows or the
shared expert, and v2 gains two words v1 has none for:
- hello: an empty v2 request (n==0, act as the coordinator will speak it) at
connect time, sent only when v2 will be spoken at all. A worker built from
an older engine closes on version 2, and cluster_init refuses it by name
("expert worker HOST:PORT closed on the COLIEX01 v2 hello: it runs an older
engine (v1); rebuild the worker, or run this coordinator without
COLI_CLUSTER_ACT=q8 and CLUSTER_SHARED=1") before any token is generated. A
worker answers the hello before its model checks, with an empty response in
the same act; its refusals of an unknown version or act are unchanged.
- item eid 0xFFFFFFFF: the layer's shared expert. With CLUSTER_SHARED=1 the
coordinator adds one item of every row to the layer's first routed request,
on the worker layer%n picks, in the request's act; the worker loads the
three shared matrices once (qt_load + qt_planarize, the coordinator's own
resident branch) and runs FASE E's CPU sequence (matmul_qt, siluf,
matmul_qt); the coordinator adds the rows where FASE E adds its own, so
with f32 rows the f32 order of the sum does not move. Off by default.
GLM-5.2: 2.83 B params a token off the coordinator. A v2 request may carry
65 items (64 routed + the shared one); v1 stays at 64.
DUMP=<path> (raw f32 logits rows) was compiled only under COLI_VULKAN; it is
now on every build so the cluster gate can compare runs byte for byte.
Tests:
- tests/test_cluster_protocol.c: the hello against a v1 worker (refused, by
name and version, read back from stderr) and a v2 worker (accepted); the
real cluster_moe_batch over a socketpair with the shared item (a v2 request)
and without it (v1 bytes, the default).
- tests/test_cluster_sharding.py: the f32 parity gate gains a third run with
CLUSTER_SHARED=1 that must match the local run's teacher-forcing signature,
show zero oracle mismatches, and match the local run's logits bytes; the
worker's stderr must name every sparse layer it loaded the shared expert
for. A 0.1 % scale on the worker's shared output flips no token on the tiny
fixtures but fails the byte comparison; zeroing it fails the oracle on
27/32 and 30/32 positions. The q8 drift gate is unchanged; its transfer
summary now includes the hello's 56 bytes per worker per run.
…nnect The coordinator's exchange with a worker -- the request header and items, then the reply's header and items -- is one function now, cluster_exchange, and a transport or protocol failure in it (a short read, EOF mid-reply, a magic, version, act, status, item id or row count that is not the one asked for) marks that worker's link failed. Every later exchange on a failed link returns -1 before touching the socket: nothing is sent, nothing is read, so the bytes a worker still sends after a failed exchange -- the rest of a reply the coordinator stopped reading, or a well-formed reply behind it -- are never read as the answer to a later request. A link comes back only by reconnecting. The hello's refusals mark the link the same way. cluster_moe_batch is unchanged in what it does: it builds the items, calls the exchange, and on -1 prints the worker by name and exits, as before. The rule is ds4's after a one-sided transport failure (antirez/ds4 f0962e3): refuse everything on the link, never consume drained bytes as a reply. tests/test_cluster_protocol.c pins it over a socketpair in two shapes. A worker whose reply header claims two items for a one-item request, with a well-formed 52-byte reply queued right behind it: the first exchange returns -1, the second returns -1 at once and the 52 bytes are still on the link, unread, and the worker saw no second request. A worker whose reply stops after the first of two rows and shuts its write side: the same, with nothing pending. Without the link guard the second exchange reads the queued reply as its own (the test fails on rc == -1); with it, 8 of 8 runs pass. 0 warnings.
…ts rows by index A v1 request copies a row into every item that routes it: at decode with top-K the same row crosses the wire K times (GLM-5.2, top-8: eight copies of a 24 KB row per layer). A v2 request now carries S and the batch's S rows once, in `act`, then per item eid, nr and nr row indices (u32, network order); the worker gathers each item's rows from the batch. Replies are per item as before. v1 requests keep their copies, so the default bytes are unchanged; the shared-expert item names every row by index instead of carrying S more copies, and the coordinator no longer copies rows per item for v2 either (cluster_item's copy_inputs). Measured on the tiny GLM oracle (32 positions, top-2, D=256), bytes the coordinator sends per position: v1 4102, v2 2080 (1.97x; the rest is indices and headers). q8 in total: 2317 -> 1758 B/position (f32 8203, 4.67x), the same 0 oracle mismatches. tests/test_cluster_protocol: the exact 104 bytes of a two-expert, top-2 v2 request where row 0 routes to both experts and row 1 to one, the shared item riding (header, S, the two rows once, three index lists); the worker gathers and the rows accumulate exactly. The q8 round trip and the shared-expert test move to the new layout. tests/test_cluster_sharding: f32 parity and q8 drift unchanged; the parity gate prints the bytes sent per position for v1 and v2 and requires the v1/v2 ratio to be at least 0.9 K.
cluster_moe_batch served the workers in turn: build worker 0's batch, send it, wait for its reply, then worker 1's. A layer therefore took the SUM of the workers' times and, with N workers, paid N round trips end to end. Now every worker's request goes on the wire first, then every reply is read, in worker order -- the order the routed rows were summed in before, so the f32 sum does not move and the parity gate stays token-exact. The bytes per worker are unchanged (protocol v1, no wire change); a layer waits for its slowest worker instead of for each of them. tests/test_cluster_protocol: two socketpair workers that will not answer until both have received their request (a 2 s barrier), each then holding its reply 200 ms. Served in turn, the barrier times out and the test fails (checked against the previous cluster_moe_batch: `bar.late == 0` fails); with fan-out the batch takes 200 ms, not 400. The accumulated rows are checked exactly. test_cluster_sharding's f32 parity passes unchanged on both fixtures.
…or any worker count The local loop adds a row's experts in uniq[] order. cluster_exchange added each worker's replies into out as they arrived, and cluster_moe_batch calls it worker by worker, so a row routed to experts on two workers got worker 0's experts first -- a different f32 summation order from two workers on (one worker's replies are already in uniq[] order, so one was exact). Same tokens, logits off in the last bits: the gate in tests/test_cluster_sharding compares logits bytes with one worker, so it never saw it (#1968's body named it for colibri.c and promised the fix). Now cluster_moe_batch builds the block's items once, in uniq[base+j] order, sends each worker the ones (eid+layer)%n assigns it, and holds every reply in its item (ClusterItem gains y; cluster_exchange no longer takes out) until the last worker has answered; one loop then weights the rows into out, item by item in that order -- the local loop's. The shared item still lands in sh_out for FASE E. Allocation stays bounded by the block's 64 items (+ the shared one); a failed exchange still marks the link failed, leaves no reply rows behind, and ends the run by name as before. Measured on the tiny GLM oracle (32 positions, top-2, D=256, 8192 logits dumped), delegated vs local, every run token-exact against the oracle -- before: 1 worker 0 logits differ; 2 workers 6860 (fmt6) / 6692 (fmt4) differ, max 7.2e-7; 3 workers 3543 / 5046; 4 workers 6932 / 6736. After: 1, 2 and 3 workers on fmt6 and 2 and 3 on fmt4, 0 of 8192 differ (the module's 9 tests OK). With top-2 a row's two terms commute unless the add is contracted into an FMA (GCC's default here, -O3 -march=native): the previous head built with -ffp-contract=off shows 0 differ with 2 and 3 workers too, so the protocol test below gives one row three experts, red on any build. tests/test_cluster_protocol: one row routed to experts 42, 43 and 45 of layer 7 over two workers ((eid+7)%2 puts 43 and 45 on worker 0, 42 on worker 1), every expert answering 1.0, gates 1, 2^-24, 2^-24: uniq[] order gives 1.0 and worker-major 1+2^-23, and the test checks its own numbers tell the orders apart. On the previous head it aborts on out[d] == expect; with the fix 3 of 3 runs ok, 0 warnings. The failed-link cases keep their contract with the new signature (items[0].y stays NULL). tests/test_cluster_sharding gains ClusterWorkerCountParityTest: the logits bytes of 1, 2 and 3 workers on fmt6 and 2 and 3 on fmt4 against the local run, counting the floats that differ. docs/cluster.md says the order.
…, bit-exact for any worker count
…e VRAM carve-out # Conflicts: # c/Makefile
From Python 3.14 the json decoder's recursion guard follows the C stack, so the test's depth parses, and the server answers the one-array batch as an invalid request instead of a parse error. The session still answers the next line, which is what #2163 fixes.
JustVugg
force-pushed
the
batch/external-2026-10-22
branch
from
October 22, 2026 07:25
38b4c76 to
9657361
Compare
JustVugg
changed the title
Owner
Author
|
Speed against dev (fae63d9) on the RTX PRO 4000 (24 GB) with 62 GB of RAM and an i5-13500:
The Qwen3.8 CPU row is this branch (9657361), measured after #2118 left it; the MTP head accepted 87.9% of its drafts on both sides. #2253's switch changes nothing measurable on this card; it stays off by default, as the PR proposes. |
…e Windows runner)
Owner
Author
|
Speed against dev, an RTX PRO 4000 (24 GB) with 62 GB of RAM, servers alternated on frozen expert histories (prompt / decode tok/s, three runs after a warm-up, two rounds):
No path moved outside the run-to-run spread. #2253's The one red job of the first run ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The external pull requests reviewed on 2026-10-22, merged into one branch so that they run one CI together. Each pull request keeps its own commits and gets its own merge commit, so GitHub closes every one of them as merged when this lands.
In this batch
Fixes with their tests (rudycelekli): #2149, #2153, #2155, #2160, #2163, #2169, #2170, #2171, #2172, #2174, #2175, #2176, #2177, #2178, #2179, #2180, #2181, #2271, #2272, #2273.
Server and cluster (Avicennasis): #2121 (listen backlog 128); the cluster stack #1963, #1964, #1970, #1967 and #2120, rebased as one stack on dev as asked on 2026-10-16 (merged through #2120's head, which contains the other four).
Engines:
COLI_KV_SHAREcopies the KV8 and KV_TQ byte rows and their scales, with a test of the adoption under both.pick_memtype,pick_memtype_cachedandpick_memtype_device).COLI_VK_CHAIN_GEMM_ROWSfor DeepSeek V4's chain prompt rows, off by default.Conflicts resolved here
c/tests/test_qwen_image_cli.py; both kept.c/Makefile,test_vk_memtypeadded to the exclusions beside dev'stest_cu_tierline.Not in this batch
coli chatwould get empty answers with the reason only in the engine's log once the history fills up (comment on the PR).DSV4_MM_WARP) by default on V4's FP8 CUDA path, not only underCOLI_CUDA_MOE_BATCH=1, which needs a measurement of the default configuration first.Changed here besides the merges
tests/test_mcp_server.py(fix(mcp): keep stdio session alive after JSON nesting errors #2163): the nested-JSON case accepts Python 3.14's answer (see below).tests/test_bench_ux_greedy.py,tests/test_bench_ux_failure.py(fix(benchmarks): force primary greedy temperature in UX runs #2181, fix(benchmarks): propagate UX engine failures #2180): POSIX only. They runbench_ux.shon a#!/bin/shfixture engine, and on the Windows UCRT64 runner both exited 1 (expected 0 and 7).Verified
make checkon Linux x86-64 (Zen 4, Python 3.14): the C suites pass; the Python suite (2863 tests) passes after one test fix in this branch: from Python 3.14 the json decoder's recursion guard follows the C stack, so fix(mcp): keep stdio session alive after JSON nesting errors #2163's nested input parses and the server answers it as an invalid request instead of a parse error. The test now accepts either; the session answering the next line is what it checks.make VK=1of qwen36, qwen38, colibri, deepseek-v4 and glm53: no warnings.scripts/quickstart_tiny.sh): the base 20/20, the trained continuation 19/20 with the adapter, 20/20 again without it.