Repository navigation
Conversation
make tracks file timestamps, not #include graphs, and c/Makefile has no
-MMD/-include dependency generation. Each engine rule names its prerequisites
by hand, and those lists have drifted behind what the sources actually
include. The result is the worst kind of build outcome: make reports success,
changes nothing, and leaves a STALE binary that looks freshly built.
Reproduce on a clean checkout:
cd c
make colibri # builds
touch edge_runtime.h && make colibri
make: 'colibri' is up to date. # but colibri.c includes it
Touching a header the rule DOES list (st.h) relinks as expected, so this is
specifically the omitted ones. Missing per engine, all of them headers the
source includes and that exist in c/:
colibri 13 abl.h backend_cuda.h backend_metal.h backend_vulkan.h
decode_batch.h edge_adapters.h edge_runtime.h
edge_tok_internal.h schema_gbnf.h segment_adapter_internal.h
segment_adapters.h segment_runtime.h tier.h
inkling 8 backend_cuda_ink.h backend_metal.h edge_adapters.h
edge_runtime.h edge_tok_internal.h
segment_adapter_internal.h segment_adapters.h
segment_runtime.h
kimi_k3 9 backend_metal.h backend_vulkan.h edge_adapters.h
edge_runtime.h edge_tok_internal.h hybrid_split.h
segment_adapter_internal.h segment_adapters.h
segment_runtime.h
olmoe 7 edge_adapters.h edge_runtime.h edge_tok_internal.h
fused_simd.h segment_adapter_internal.h segment_adapters.h
segment_runtime.h
qwen36 6 edge_adapter_internal.h edge_adapters.h edge_runtime.h
segment_adapter_internal.h segment_adapters.h
segment_runtime.h
Adds tests/test_makefile_deps.py so the two lists cannot drift apart again.
It compares each engine rule's prerequisites against the headers its source
includes, counting only headers that exist in c/ (system headers and
generated files are not prerequisites). Verified to go red against the
unfixed Makefile and green with it, so it discriminates rather than passing
vacuously. Picked up by the existing `unittest discover -p 'test_*.py'`.
make -C c test-python: 705 tests, OK (skipped=35).
Both follow-ups asked for in review of #1284, rebased from main onto dev so the two engines that only exist here are in scope. 1. glm53 and qwen38. The branch was cut from main, which has five engines; dev has seven. So the first version of this fix left glm53 and qwen38 with exactly the bug it fixed for the other five. Verified before touching anything, and without a compiler -- `make -q` answers the timestamp question the recipe would answer, so this needs no toolchain and no build: touch glm53 && sleep 1 && touch edge_runtime.h make -q glm53; echo $? 0 # "up to date", though glm53.c includes it Missing per engine, all of them headers the source includes and that exist in c/: glm53 7 backend_vulkan.h edge_adapter_internal.h edge_adapters.h edge_runtime.h segment_adapter_internal.h segment_adapters.h segment_runtime.h qwen38 7 edge_adapter_internal.h edge_adapters.h edge_runtime.h qwen38_vision.h segment_adapter_internal.h segment_adapters.h segment_runtime.h After the fix `make -q` exits 1 -- would relink -- for every engine against a header that was previously missing from its rule: glm53 (edge_runtime.h, backend_vulkan.h), qwen38 (edge_runtime.h, qwen38_vision.h), colibri, inkling, kimi_k3, olmoe, qwen36. 2. ENGINE_RULES is gone; the list comes from the Makefile. The review is right that the hand-kept dict was the same class of bug as the one being fixed. `_prereqs_by_target()` filtered on it, so `assertEqual(set(prereqs), set(ENGINE_RULES))` was load-bearing in one direction only: it caught a rule renamed or removed, and could not catch a rule that was never added. That is why it passed green while glm53 was broken. Now anything matching `NAME$(EXE):` with a `NAME.c` beside it is checked, so there is no list to keep in sync and the next engine is covered without anyone remembering. Measured, old test against new: scenario old new fixed Makefile (baseline) PASS PASS glm53+qwen38 unfixed (the real bug) PASS FAIL a new engine rule omitting st.h PASS FAIL rule syntax changed, parse matches nothing FAIL FAIL Two things fall out of deriving rather than listing: - iobench is now covered too. It is not an engine, but it is built from one translation unit by the same kind of rule, so it has the same failure mode and gets the same guarantee for free. Its list is already complete, so this adds no Makefile change. fp8_bench is correctly not picked up -- there is no fp8_bench.c, it is built from backend_cuda.cu. - deriving can go vacuous in a way listing cannot. If the rule syntax ever changes, the parse matches nothing and a header check over zero targets passes, which reads exactly like a clean tree. So the parse is now asserted non-empty and cross-checked against FAMILIES -- the registry is already the authority on what must be buildable, and test_family_registry already pins that each family has a rule, so this reuses that rather than keeping a second list. deepseek_v4 is excluded by name: it is built by Makefile.deepseek-v4 through a phony target and has no prerequisite list here to check. Line continuations are folded before parsing, with the same `re.sub(r"\\\n[ \t]*", " ", ...)` idiom test_family_registry uses: a rule wrapped across lines would otherwise present a truncated prerequisite list and report headers as missing that are listed on the next line. No engine rule wraps today; the failure would be a false positive rather than a silent pass, but it costs one line to not have it. The include regex now tolerates leading whitespace. No engine has an indented `#include "..."` today (checked, 0 across all eight sources), but one inside an `#ifdef` is exactly where a new header would land, and missing it fails in the silent direction.
#1304: `coli convert` su Qwen/Qwen3.8-Flash-Next. Il convertitore e' di GLM-5.2 e non controlla cosa gli viene dato: i tensori BF16 che non riconosce come esperti GLM passano upcastati a f32, ogni shard raddoppia (3.36 GB -> 6.71 GB nel log del reporter), e il run si e' fermato a 424 GB scritti su 360 di sorgente, a disco pieno. Ora check_model_family legge il model_type e decide PRIMA del primo shard: config.json pesa 5 KB e stabilisce se i prossimi 300 GB hanno senso. GLM-5.2 passa; ogni altra famiglia viene rifiutata con il puntatore giusto (Qwen3.8: nessuna conversione esiste, si esegue l'FP8 ufficiale; GLM-5.3: convert_glm53.py; e cosi' via); l'ignoto viene rifiutato senza indovinare. --indir senza config.json procede come sempre: le fixture sintetiche non ne hanno. --repo con config.json illeggibile procede con un avviso: il fallimento utile e' sul model_type sbagliato, non sulla rete. Gli insiemi della guardia sono copie di family_registry, e una copia che diverge e' il difetto che la guardia cura: il test li confronta col registry e fallisce alla prima famiglia aggiunta o rinominata. Se GLM-5.2 rinominasse un model_type senza aggiornare qui, il convertitore rifiuterebbe il modello che esiste per servire -- e' il caso che il contratto blocca per primo. Provato end-to-end: --indir con config qwen4_exp esce 1 col messaggio per-famiglia; --selftest GLM invariato; i test esistenti del convertitore restano verdi. Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
…mparison (#1272) Five rows for the 12900K + 990 Pro box (commit-stamped 72d3d37, July 2026): the MTP=0 result with its mechanism (600 experts/token against 1,062 with MTP on), the CUDA expert tier separated from the config change it rode in with, a two-drive file-level split, and six consecutive REPIN rounds showing where learned pins converge. One row for an M4 Pro running OLMoE fully resident, comparing cap 32 against cap 64 in interleaved pairs: cap 32 wins 7 of 8, median 19.1 against 16.3 tok/s, at half the resident footprint. Two notes. The cap comparison is paired because unpaired sweeps on that host do not agree with each other about the sign -- one five-run sweep put cap 64 ahead, a later ten-run block put cap 32 ahead by 35%, and cap 32 alone ranged 13.97 to 20.62 across sessions. Same-session alternation is the only thing that resolves it, which is the control the decode failure ledger already asks for. And the file-level split is recorded as the negative half of #1249: same class of hardware, +5.5% against +37.5%, because splitting whole shards only parallelises reads that are already concurrent and that box sat at queue depth ~1.
#1318) #1317: 141 shard su 142 presenti e una riga sola di errore, "missing tensor: model.embed_tokens.weight". La diagnostica che conta gli shard, vede i buchi nella numerazione e suggerisce il resume col flag anti-xet ESISTEVA gia' -- st_die_missing, dal 25 luglio -- ma sette punti fatali di st.h uscivano con la riga nuda senza chiamarla. Il percorso dell'embed, cioe' il primo tensore che ogni caricamento tocca, era fra questi: il fallimento piu' comune al mondo era l'unico senza spiegazione. Riprodotto sul motore attuale prima di toccare qualcosa. I sette siti fatali ora chiamano st_die_missing. L'ottavo sito, la probe che ritorna -1, resta com'era: non e' fatale e i chiamanti gestiscono il ritorno. Il test costruisce il container a mano, byte per byte -- niente numpy, niente torch, nessuna fixture esterna: config GLM tiny piu' uno shard senza embed, lo scenario esatto della segnalazione. Pretende il conteggio shard, la frase sul core tensor e il suggerimento di resume, e vieta il ritorno della riga nuda. Controllo negativo: con i sette siti ripristinati fallisce; ripristinando UN sito solo non fallisce, perche' vede solo la prima morte del percorso embed -- il contratto protegge il caso comune, non ogni sito singolarmente, ed e' detto qui perche' resti vero. Il job Python della CI ora compila anche colibri: l'ha preteso il contratto di #1297 da solo, appena il test nuovo e' comparso con la sua guardia sul binario. Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
v4_hwinfo_emit() reads /proc/cpuinfo for the CPU name and /proc/meminfo
for memory. Neither exists on macOS, both fopen() calls return NULL, and
the line goes out with zeroes and no error:
HWINFO 16 0.0 0.0 0 0.0 unknown|v4-cpu
The gateway forwards that faithfully, so the dashboard's Runtime panel
renders "unknown" with "0 GB RAM / 0 GB free" while correctly showing the
core count. Nothing looks broken enough to investigate, which is why it
survived.
Add a Darwin branch using machdep.cpu.brand_string and hw.memsize, and
delegate availability to coli_v4_os_available_memory(), which already
carries a reviewed Darwin branch (free + inactive + purgeable pages, the
MemAvailable equivalent) rather than repeating the detection. It fills
only what /proc could not supply, so the Linux path is byte-identical.
Units follow the Linux branch exactly: it reports kB/1e6, i.e. decimal
GB, which is the contract the web UI was built against, so this uses
bytes/1e9 rather than bytes/2^30.
The include sits directly above its only caller rather than with the
other platform includes near the top. This file is an amalgamation
compiled once per -DCOLI_V4_UNIT_*, and that upper region is not part of
the unit containing v4_hwinfo_emit; placing it there produces "call to
undeclared function 'sysctlbyname'".
Verified on macOS 15, Apple M3 Max, by running the serve engine and
reading the emitted line:
before: HWINFO 16 0.0 0.0 0 0.0 unknown|v4-cpu
after: HWINFO 16 137.4 105.5 0 0.0 Apple M3 Max|v4-cpu
which matches the machine: sysctl machdep.cpu.brand_string reports
"Apple M3 Max" and hw.memsize reports 137.4 GB.
Co-authored-by: Vincenzo Fornaro <vincenzofornaro89@gmail.com>
…16_order (#1314) 13b8f16 deliberately traded throughput for determinism so the reference matvec accumulates in the same order as the optimized rows16 path, recording 155 against 235 GFLOP/s and roughly 6 to 9 percent end to end. That ordering guarantee can be kept while recovering much of the speed. Processing four output rows per pass, each with its own accumulator, leaves every row's addition sequence untouched: same operands, same column order, same intermediate roundings. Only the interleaving of four independent chains differs, so results are unchanged. The gain is latency hiding plus one activation load feeding four rows, and a short tail clamps spare row indices with guarded stores. Measured on an Apple M3 Max at p256 / 40 tokens with the companion matmul_fp8 change, N=2, deterministic: decode 49.97s to 36.51s and 0.7805 to 1.0683 tok/s, plus 36.80 percent, time to first token down 18.0 percent. Output md5 is identical to the unmodified engine. Applied on its own this recovers part of that; the quoted figure is for both changes together. Co-authored-by: Vincenzo Fornaro <vincenzofornaro89@gmail.com>
The Darwin arm of the Makefile finds libomp with `brew --prefix libomp`. On a CLT-only Mac driven non-interactively (`make` from a script, over SSH, in CI) Homebrew is routinely installed but not on PATH, the probe prints nothing, and the OpenMP arm never engages. Nothing warns: the build is a valid single-threaded engine with libomp sitting right there on disk. Measured on an M4 Max with the Qwen3.6-35B qpack: 0.87 tok/s from the silent single-threaded build, 4.44 tok/s (range 3.80-4.76, thermal) once OpenMP engaged, TTFT 3.89 s to about 0.7 s, output byte-identical by cmp. Engine code is unchanged by this commit -- every omp site is row-partitioned and deterministic by construction -- so the difference is entirely the missing flag. When the brew probe comes back empty, fall back to the two standard Homebrew prefixes (/opt/homebrew/opt/libomp on Apple silicon, /usr/local/opt/libomp on Intel). The existing artifact check still runs after the fallback: both omp.h and libomp.* must exist before any flag is added, so a bare prefix directory without the library still yields the plain build, as before. (cherry picked from commit 44ddb39f98a05a332921dd47491b53d7b55fa3c7)
load_tokenizer indexed the BPE merge table from tokenizer.json by splitting each entry on a space: the legacy "a b" string spelling. tokenizers >= 0.20 (transformers 4.45+) writes each merge as a two-element array ["a","b"] instead, and that is what the Qwen3.6 checkpoints ship. Every entry failed the string test, `continue` skipped it, and the reader finished with ZERO merges and no message. Nothing downstream treats an empty merge table as an error: bpe_piece reads it as "nothing to merge", so encode_text degraded to one token per byte-symbol. Measured on a real-model run before the fix: a 24-character prompt became 24 tokens, and the model was fed a sequence it had never seen in training. Accept both spellings. A string entry splits on the first space as before; a two-element array of strings takes its halves directly; any other shape is skipped as it was. Both forms index into the same smap_put(&g_merge, key, r) table with the same 0x1F-joined key, so the rest of the tokenizer is unchanged. tests/test_qwen36_tok_merges writes two tokenizer.json files for the same five-piece vocabulary that differ only in the merge spelling and asserts that both encode "in in" to the same merged ids [in, Ġin] -- one plain merge (i+n) and one chained merge (Ġ+in) whose input the first produces. On the pre-fix reader the pair form loads zero merges and the encoding comes back as one id per byte-symbol, so the gate fails. The scratch directories the test binaries write (tests/tmp_*) are added to .gitignore. (cherry picked from commit d783505d3e0b97b0fa078d7320877f3c4bd740e1)
Moves the idea from #1324 to where it belongs: one primitive at the st.h layer instead of a per-engine table, so every engine that adopts it pays for one implementation, not N. Adds st_map_shard_range(fd, off, nbytes) next to the existing st_map_raw: maps a whole FILE once (not per tensor/range -- expert weight pieces sit at byte offsets that are not page-aligned, and mapping each of a model's ~12000+ expert pieces individually would mean thousands of small mappings for no benefit over one mapping per shard), keeps it for the process lifetime, and serves any byte range inside it with no allocation or copy. Portable via the existing compat_map_readonly (already used by st_map_raw), so Windows gets the same path as POSIX. A failed or disabled mapping returns NULL; every call site here falls back to its unmodified pread path on NULL, so this primitive never changes behavior on its own. Off by default (COLI_MAP_EXPERTS unset or 0): st_map_experts_enabled() short-circuits to NULL immediately and every engine takes the exact pread path it always has. This is deliberate per review on #1324 -- the flip to mapped-by-default should follow a measurement of the global-page-cache- scarcity regime (small box, large model), not precede it. Wires it into the two engines that had their own bespoke version of this (glm53.c's g_fmap table from #1324, qwen38's q38_shard_map table from the qwen38 port) -- both now call the shared primitive instead, removing the duplicate table/logic. Neither engine's default behavior changes. Verified on both, same prompt, --temp 0, pread vs COLI_MAP_EXPERTS=1: GLM-5.3-Flash int4-g64: teacher_forcing md5 identical, 1.103 -> 1.983 tok/s Qwen3.8-Flash-Next-FP8: generated-text md5 identical qwen38 (default, CPU-only target) and qwen38 VK=1 both still build and link with no Vulkan dependency, unaffected by this change. glm53, kimi_k3 and colibri also build clean (kimi_k3.c already used st_map_raw from the same file; this addition is purely additive). Supersedes the glm53-only version in #1324, which should close in favor of this one.
… ours (#1327) GLM-5.3 non sa non ragionare. Nel suo chat_template.jinja `effective_reasoning_effort` ha un ramo else che vale 'max', quindi non e' mai none e la riga di sistema esce a ogni render; e il prompt di generazione apre sempre <think>. Di enable_thinking, in quel template, non c'e' traccia. Il nostro --no-think produceva invece [gMASK]<sop><|user|>ciao<|assistant|><think></think> che si discosta in due punti da qualunque forma il modello abbia visto: manca la riga di effort, e il blocco e' chiuso invece che aperto. E' la causa di #1278 -- gli si chiede una posizione fuori distribuzione e lui continua a ragionare, che e' cio' per cui e' addestrato. Meta' di quella deviazione l'ho aggiunta io in #1282, togliendo la riga di effort e giustificandolo con un meccanismo ("il modello riapre un <think>") che ho poi misurato falso e ritirato su #1278. Ho reso il prompt piu' lontano dal template mentre annunciavo una correzione. Ora enable_thinking=False significa "il minimo che il modello supporta", Reasoning Effort: Low, con il blocco aperto come sempre. Nascondere il pensiero e' compito del gateway, non del prompt. Le tre modalita' rendono byte per byte come il template renderizzato con jinja2 (verificato accanto). Il test proteggeva l'errore: pretendeva le due deviazioni come contratto, col commento che spiegava la teoria ritirata. Ora confronta col template reso con effort 'low' invece che con una forma scritta a mano, e rifiuta esplicitamente il ritorno del blocco chiuso e la sparizione della riga di effort. Controllo negativo: rimessa la forma di #1282, fallisce mostrando i due prompt. Il commento nel renderer che sosteneva che <think></think> fosse "uno stato su cui il modello e' addestrato" e' corretto: e' vero per un turno passato, falso per la posizione da cui il modello scrive. Diagnosi di acedogblast in #1311. 205 test server verdi, template a 25 rese identiche. Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
…1329) #1328: test_cancels_generation_after_consumer_disconnects falliva a caso sul job Windows con "Lists differ: [] != ['x']", su PR che non toccano quel percorso. Non era Windows: era una corsa, e Windows e' solo dove la finestra e' piu' larga. cancelled tornava True da subito, quindi il ramo idle -- che esiste apposta per cancellare PRIMA del primo frame (#908) -- correva contro l'arrivo del frame stesso. Se vinceva l'idle, nessun token veniva consegnato e l'asserzione su output falliva senza che nulla fosse rotto. Fra l'altro i due test si sovrapponevano: il caso pre-primo-frame ha gia' il suo, deterministico. Ora il flag si alza dentro il sink. Nel ramo "data" decode() consegna il token PRIMA che cancelled() venga interrogato, quindi l'ordine lo garantisce il codice e non lo scheduler -- ed e' lo scenario che il nome del test promette: il consumatore se ne va dopo aver ricevuto qualcosa. Il tetto sui poll e' l'altra meta', e l'ho aggiunto dopo aver misurato che serviva: con il solo flag, un guasto che impedisce la consegna del token lo lascia basso per sempre e il test si APPENDE invece di fallire. Il ramo idle interroga cancelled() ogni 50 ms, quindi dopo un secondo si cancella comunque. Rotto decode() apposta: fallisce in 2s con "[] != ['x']", che e' la diagnosi giusta. Controlli: 20 giri su 20 verdi; decode rotto -> fallimento in 2s; suite test_openai_server verde. Disabilitare il cancel del ramo data NON fa fallire il test, ed e' corretto: il contratto e' che la cancellazione avvenga, non quale ramo la esegua. Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
#1333 di jmpmachado: dsa_score_single -- l'unica funzione che leggerebbe la cache Ic per scegliere le posizioni top-K -- non ha chiamanti. Una grep su tutto il repo trova solo la sua definizione. Confermato. Guardando meglio, il buco e' piu' largo di come e' stato segnalato: i tensori caricati per ogni layer DSA sono CINQUE (w_k, w_q, w_p, kn_w, kn_b) e di questi w_q, w_p e kn_b non sono toccati NEMMENO dal codice morto -- si caricano dal disco e si liberano a fine vita, e basta. Piu' la cache Ic da [max_t * index_hd] float per layer, piu' per ogni token un matmul, un rmsnorm e una rope. Ora tutto questo sta dietro KIMI_DSA_INDEXER, spento di default. Non cancellato: quel che manca e' collegare dsa_score_single, e chi lo fara' ha bisogno di questi pesi. L'avvertenza sta nel commento, perche' e' la parte che conta: collegarlo CAMBIA le uscite, l'attenzione diventa sparsa dove oggi e' densa, quindi va fatto contro l'oracolo token-exact e non a occhio. Prove: l'oracolo tiny passa identico con l'interruttore spento e acceso (76/76 posizioni in teacher forcing, kimi-k3-ckpt compreso) -- cioe' quel lavoro non serviva a nessuno, che e' il punto della segnalazione. Sulla fixture il risparmio e' 8 KB perche' la fixture e' 1.7 MB; sul modello vero scala con index_hd x hidden x 5 x layer piu' la cache, e non ho l'hardware per misurarlo qui. Un primo confronto l'avevo fatto sull'md5 dello stdout e dava "identici": era l'md5 della stringa vuota, entrambe le esecuzioni non stampavano niente. L'oracolo sopra e' la misura vera.
#1332 di jmpmachado: il protocollo documenta "CANCEL <id> (abort current turn)", ma in qwen36 l'unico posto che legge un CANCEL -- serve_read_req -- viene chiamato solo FRA una richiesta e l'altra. Il comando arrivava quando il turno che doveva fermare era gia' finito. Il difetto e' piu' largo di come e' stato segnalato: cinque motori su sette non guardano stdin durante il decode. colibri lo fa dal #678 (mux_ctl_poll) e kimi_k3 ha il suo; glm53, qwen36, qwen38, inkling, olmoe e deepseek_v4 no -- e glm53 lo ammette nel proprio commento, "un CANCEL arriva sempre per una che non e' piu' in volo", rispondendo NOT_FOUND. Conta perche' il gateway, dopo aver mandato CANCEL, aspetta l'ack tenendo l'ammissione dello scheduler: su quei motori un client che si disconnette non libera niente, si aspetta comunque la fine del turno. serve_poll.h estrae la primitiva di colibri -- select con timeout zero su POSIX, PeekNamedPipe su Windows -- perche' gli altri non debbano reinventarla. Guarda soltanto se c'e' qualcosa da leggere: il formato delle righe resta al chiamante, che lo conosce. Regola sui casi ignoti: se la piattaforma non sa dire se stdin e' pronto si risponde "non pronto" e si torna al comportamento di prima. Mai fingere "pronto": una lettura bloccante dentro il ciclo di decode fermerebbe la generazione per sempre in attesa di un comando che potrebbe non arrivare mai. qwen36 e' cablato come riferimento: una guardata per token, e al CANCEL si scarica l'UTF-8 parziale, si risponde ERROR <id> CANCELLED e si esce. Gli altri quattro seguiranno lo stesso schema. Il test misura il TEMPO, non solo il valore di ritorno: una versione bloccante passerebbe un controllo sul solo risultato. Controllo negativo: portato il timeout della select a 2 secondi, fallisce su due asserzioni. Il diff su qwen36 e' di sole aggiunte dentro serve_one: generate() non e' toccata, quindi l'oracolo non puo' cambiare per costruzione.
…n branch pipe/dup2 su MinGW hanno l'underscore: rosso su Windows UCRT64 mentre serve_poll.h compilava benissimo. Con gli alias il test gira anche li', ed e' quello che conta: la pipe anonima di _pipe e' esattamente cio' che PeekNamedPipe sa interrogare, quindi il ramo Windows viene esercitato invece di restare non provato.
Il test, appena ha potuto girare su Windows, ha trovato una differenza di semantica vera: PeekNamedPipe FALLISCE su una pipe rotta, mentre select() POSIX la dichiara leggibile -- e select ha ragione, perche' una lettura li' non blocca, ritorna EOF. Con la versione precedente, su Windows, un gateway che muore o chiude lasciava il motore cieco fino a fine turno: coli_serve_stdin_ready avrebbe risposto 'niente da leggere' per sempre. Ora ERROR_BROKEN_PIPE vale 'pronto' e il chiamante distingue leggendo, come su POSIX. Su Linux i due casi coincidono, quindi questa differenza sarebbe partita per la produzione senza che me ne accorgessi: e' esattamente il motivo per cui il test e' stato portato su MinGW invece che saltato li'.
…ving for nothing (#1334) * fix(qwen36): the VRAM tier now promotes int8 experts instead of reserving for nothing #1331 di jmpmachado, verificata riga per riga: su un container int8 qt_plan_fill riservava budget e metteva planned=1 senza guardare il formato, mentre la promozione esigeva i puntatori int4 -- NULL, perche' su int8 non c'e' niente da impacchettare. Il budget restava prenotato, il flag alzato, e "if(resident||queued||planned) continue" garantiva che quell'esperto non venisse mai piu' riconsiderato. Zero esperti in VRAM per tutta la vita del processo, nessun messaggio, e un tier che dall'esterno sembrava acceso e semplicemente freddo. La risposta non e' avvisare che non funziona: e' farlo funzionare. backend_cuda sa gia' leggere fmt=1 (int8) -- backend_cuda.cu:174 per la taglia, :307 per la decodifica -- mancava solo che il tier glielo offrisse. Ora il formato lo decide qt_init e lo seguono tutti: - G.wfmt: 4 = int4 impacchettato, 1 = int8 - exp_bytes conta un byte per elemento su int8, mezzo su int4: senza questo si prometterebbe il doppio degli esperti che ci stanno - stage() copia i byte int8 senza lo XOR 0x88, che serve a portare i nibble int4 da complemento a due a binario sfalsato e su byte interi sarebbe corruzione silenziosa - lo staging e l'upload usano il passo giusto fra le tre matrici Il caso che il kernel non sa esprimere -- int8 con scale raggruppate, perche' backend_cuda onora gs solo per fmt=4 -- viene rifiutato in qt_init PRIMA di riservare qualunque budget, dicendolo. Promuoverlo darebbe numeri sbagliati; riservare e non promuovere era il difetto. Il formato lo rileva il chiamante dalla TAGLIA SU DISCO del primo esperto, non da meta.ebits: esiste un container i8 che dichiara ebits=4, ed e' il motivo per cui anche il loader guarda nbytes. Il test gira senza GPU: definisce i coli_cuda_* e REGISTRA cosa riceve il backend. Un test fermo a "qt_init ritorna 1" sarebbe passato anche col difetto; questo pretende che un esperto arrivi davvero, con fmt=1, con i byte intatti. Controlli negativi: rimesso il fmt cablato a 4 fallisce su tre asserzioni; tolto lo skip dello XOR prende la corruzione dei pesi. * test(qwen36): setenv shim so the tier test builds on MinGW Il test usava setenv, che MinGW non ha: rosso su Windows UCRT64 mentre il codice di prodotto compilava benissimo. _putenv_s fa la stessa cosa. Il tier legge la sua configurazione dall'ambiente, quindi il test deve poterla impostare su ogni piattaforma dove gira. --------- Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
#1333 di jmpmachado: dsa_score_single -- l'unica funzione che leggerebbe la cache Ic per scegliere le posizioni top-K -- non ha chiamanti. Una grep su tutto il repo trova solo la sua definizione. Confermato. Guardando meglio, il buco e' piu' largo di come e' stato segnalato: i tensori caricati per ogni layer DSA sono CINQUE (w_k, w_q, w_p, kn_w, kn_b) e di questi w_q, w_p e kn_b non sono toccati NEMMENO dal codice morto -- si caricano dal disco e si liberano a fine vita, e basta. Piu' la cache Ic da [max_t * index_hd] float per layer, piu' per ogni token un matmul, un rmsnorm e una rope. Ora tutto questo sta dietro KIMI_DSA_INDEXER, spento di default. Non cancellato: quel che manca e' collegare dsa_score_single, e chi lo fara' ha bisogno di questi pesi. L'avvertenza sta nel commento, perche' e' la parte che conta: collegarlo CAMBIA le uscite, l'attenzione diventa sparsa dove oggi e' densa, quindi va fatto contro l'oracolo token-exact e non a occhio. Prove: l'oracolo tiny passa identico con l'interruttore spento e acceso (76/76 posizioni in teacher forcing, kimi-k3-ckpt compreso) -- cioe' quel lavoro non serviva a nessuno, che e' il punto della segnalazione. Sulla fixture il risparmio e' 8 KB perche' la fixture e' 1.7 MB; sul modello vero scala con index_hd x hidden x 5 x layer piu' la cache, e non ho l'hardware per misurarlo qui. Un primo confronto l'avevo fatto sull'md5 dello stdout e dava "identici": era l'md5 della stringa vuota, entrambe le esecuzioni non stampavano niente. L'oracolo sopra e' la misura vera. Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
# Conflicts: # c/Makefile
fix(serve): honor CANCEL while the turn is still running
qt_init allocated G.is_x as 32*D floats total (one device's worth), but qt_issue strides each device's block by 8*D floats and then writes up to 32 rows into it. Device di's block starts at 8*di*D but can span 32*D floats, so any di>0 with a full 32-row issue overruns its own slice and the end of the allocation -- silent heap corruption on multi-GPU setups. Add G.is_x_floats, size G.is_x as ndev*32*D floats, and stride each device's block by 32*D (its true max row count) instead of 8*D. Also factors the fake CUDA backend shared by the tier's tests out of test_qwen36_tier_int8.c into tests/qwen36_fake_cuda.h (unchanged behaviour, verified by rerunning it), with fake_ndev and fake_issue_hook hooks the new multi-device test needs. test_qwen36_tier_multidev.c fails to compile against the pre-fix tier (G.is_x_floats does not exist yet) and passes once the fix lands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019BacNGNxAJ1M57UdYE2M3N
qt_shutdown set G.th_stop and signalled only G.cv, then pthread_join()d the uploader. But the uploader's LFRU victim path -- and qt_note_block, qt_note_planned, qt_fill_wait -- all wait on G.cv_take, which nothing in qt_shutdown ever broadcasts. If a caller issued a group (qt_issue sets G.issue_open=1) and never called qt_take (the only thing that clears it and broadcasts cv_take), a queued LFRU swap parks the uploader on that wait forever and qt_shutdown hangs. Broadcast G.cv_take alongside G.cv when th_stop is set. That alone unparks the wait loop, but the uploader would then fall through to freeing the victim's tensors and uploading into it while a group may still reference them; instead, when th_stop is set and issue_open is still set, abandon the swap: free the staged upload, clear the hot slot's queued flag, and restore the victim's resident flag (it was already cleared by qt_lfru_tick_locked before enqueue) so it stays consistent with the tensor it still holds. test_qwen36_tier_shutdown.c reproduces the sequence with the fake CUDA backend and an alarm(10) watchdog: it prints "FAIL: qt_shutdown hung" and hits the 10s alarm before the fix, returns in single-digit milliseconds after. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019BacNGNxAJ1M57UdYE2M3N
The warmstart hands the tier the live RAM weights for an int8 container (`wg = expert_is_int4 ? e->g4 : (const uint8_t *)e->g`), qt_note_planned parks that pointer in the tier's slot, and the next statement freed it unconditionally. The comment justified that with "on LFRU eviction slot_ensure_int8() rematerializes from g4" -- true only for int4. On an int8 container there is no second copy: e->g4 is NULL, so slot_ensure_int8() returns early (`if (s->g || !s->g4) return;`) and the CPU fallback in the decode loop dereferences NULL, while the pointer the tier kept dangles for any later stage(). Free the int8 block only when the container is int4, where g4/u4/d4 remain the source of truth and the peak-RSS win still applies; COLI_KEEP_INT8 keeps its meaning there. The warmstart body moves out of main into tier_warmstart(Model *, int expert_is_int4) -- same code, same messages, the QT_NO_WARMSTART check stays at the call site -- so a test can drive it without main. tests/test_qwen36_tier_int8_engine.c is that test: it includes qwen36.c (the test_qwen36_ctx.c pattern), the shared fake CUDA backend and qwen36_tier.c in one TU, builds an in-memory int8 model with no container behind it (slots pre-populated via slot_ensure_allocated and published in the layer index, so expert_get takes its hit path), runs the warmstart against the fake backend and checks that every planned expert is VRAM-resident, that slot_ensure_int8 still leaves g/u/d with the exact bytes the loader wrote, and that qs(0,eid)->g4 still points at the live block. A second in-memory model with packed int4 slots pins the behaviour the old line existed for: e->g is still dropped after staging and rebuilt bit-exact from g4/u4/d4. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019BacNGNxAJ1M57UdYE2M3N
In mm_gemv's fmt=8 FP8 E4M3 dequantization branch, normal values previously computed 2^(exp - 7) via exp2(float(int(exp) - 7)). When the Metal shader library compiles with default fast-math enabled, exp2() on Apple7 GPUs (Apple M1) introduces a 1-2 ULP underestimation for negative arguments (exp in 1..6, representing 2^-6 .. 2^-1). This caused 96 out of 256 byte codes in tests/test_backend_metal.mm:run_fp8_lut() to fail the strict bit-exactness check against the CPU reference decoder on M1 hardware. Synthesize the exact power-of-two float directly by shifting the biased exponent (exp - 7 + 127 = exp + 120) into the IEEE-754 binary32 exponent field with as_type<float>((uint(exp) + 120u) << 23). This guarantees 0 ULP error across all GPU architectures without disabling fast-math globally, resolving the test failure on M1.
_image_bytes_from_url reads a local path with the server process's own permissions. On a server that binds beyond loopback (which already requires an API key) an authenticated client could send "file:///etc/passwd" and read any file the process can reach, and the OSError message echoed the full path back, confirming a file's existence and permissions. Refuse '..' outright (never needed for a real image path), let COLI_IMAGE_ROOT confine reads to one directory the same way serve_static uses relative_to(), and make the error generic. The documented loopback single-user case (a plain path or file:// URI) is unchanged. Covered by ImageUrlPathGuard.
Third-party actions were referenced by mutable tags (msys2/setup-msys2@v2, hadolint/hadolint-action@v3.1.0, Jimver/cuda-toolkit@v0.2.19, ilammy/msvc-dev-cmd@v1). A tag can be moved by whoever controls the action's account, so a compromised maintainer would run arbitrary code with the job's GITHUB_TOKEN, including the contents:write token in release.yml. Pin each to the commit SHA the tag currently points at, keeping the version in a comment. ci.yml had no permissions block; it only reads the tree, so pin its token to contents:read as defense in depth. Official actions/* are left on tags.
The invite in the READMEs and the site returns "Invite is expired" (Discord API code 50270), so every Discord link we publish is currently dead: 4 READMEs plus site/index.html, 2 occurrences each. Replaces MAaKtQRc with FkyrEeJR across all 10.
docs: replace the expired Discord invite (all 4 READMEs + site)
pack_python.py reaches a file two ways: by following imports out from coli, and by spotting scripts that are launched as subprocesses. Only the first kind had its own imports followed, so a module that only a subprocess-launched script imports was left out of the release archive while --check still reported the archive complete. On dev this happens twice: tools/iq3_pack.py, which tools/convert_fp8_to_int4.py imports inside quant_e8(), and tools/engine_evidence.py, which this branch's eval_glm.py imports. Both are now reached. The test suite grows a real-tree assertion as well as fixture cases, because every existing case builds a disposable tree and so could not have caught either omission. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Following the imports of a subprocess-launched script (previous commit) packages tools/iq3_pack.py, but --xbits e8 still dies from an archive: iq3_pack.py opens tools/iq3xxs_grid.json next to itself at line 50, and the packer copied only .py files. The import edge was closed; the data edge was not. needed() now collects, along the same walk, files named by a string literal that exist next to the module and carry a data suffix. Verified end to end from a packed archive: iq3_pack.encode() runs and loads its 256x4 grid; delete the JSON and it is FileNotFoundError again. The suffix allowlist is deliberate and not "anything that is not .py": coli names colibri.c, family_registry.py names five engine sources, and none of those belong in a release archive. tools/iq3_encode.c stays out for the same reason it is optional in iq3_pack.py's own docstring -- the numpy path works without it, and compiling it needs a toolchain on the user's machine. That is a decision, not an heuristic's side effect. The summary line now counts the two kinds separately. Saying "N Python files" while shipping data files is how a display starts agreeing with a bug instead of catching it. Tests: five new cases, and the real-tree assertion now names the two files this was about instead of an invented one. Both defects restored individually make the suite fail (3 and 2 failures).
fix(release): close both edges the packer was missing (#1359)
coli doctor failed model.required with "2 required core tensor(s) are missing" on every GLM-5.3-Flash checkpoint. Nothing was missing. The check held three literal names taken from GLM-5.2, and Flash's root is the vision wrapper, so its language model is nested: model.language_model.embed_tokens.weight model.language_model.norm.weight Two of three names miss, hence exactly two. Reported on Discord by a user who re-downloaded 195 GB and re-ran the converter to test the theory, because a count does not say what was looked for. The check now matches roles, prefix-agnostically: a token embedding, a final norm outside the layer stack, an output head. That is what makes it hold for families nobody has written yet, which is the real defect -- every engine already discovers its prefix at load time (qwen38.c probes the nested name and falls back to the flat one), and doctor was the one place that treated the prefix as a constant. Two exclusions the tail match needs, both from real names in the tree: .layers. so a block norm cannot stand in for the final one, and layernorm so the vision tower's post_layernorm cannot either. Tied embeddings satisfy the head role without an lm_head tensor, which avoids trading this bug for the same bug on a different checkpoint. The summary now names the roles it could not fill instead of counting them. Verified against three real checkpoints: both GLM-5.3-Flash containers go from 2 missing to none, and the 116,915-tensor GLM-5.3 full container stays complete. 46 doctor tests green; restoring the literal list turns 7 of the 8 new ones red.
fix(doctor): require core tensor roles, not GLM-5.2's literal names (#1365)
A GLM-5.3 container was announced as "GLM-5.2 744B". The engine loaded the right weights and ran them correctly; the banner named a different model. This is not fixable by detection, and that is the point of the change rather than an excuse for it. Z.ai state on GLM-5.3's own card that it "uses the same base model as GLM-5.2 -- every gain comes from post-training", and the checkpoints agree: diffing the two real config.json leaves one extra key (moe_router_dtype) and the transformers_version that wrote the file. Same 78 layers, 256 experts, hidden 6144, moe_intermediate 2048. #1326's rule for Qwen -- name a checkpoint by its geometry -- cannot apply, because the geometry is identical. No rule over the configuration can name one and not the other, now or later. So the family says what it actually loads: GLM-5.2/5.3, at the 744B scale both share. Verified by resolving both real configs through the registry. The docs move with it wherever the family is ENUMERATED: the "families run today" line in all four READMEs, the model table and the RAM table in README.md, the model card and the nav entry on the site. Mentions of GLM-5.2 as a fact -- DSA came from its lightning indexer, its native MTP head, benchmarks measured on it, the pre-converted 5.2 container on Hugging Face -- are still true and are left alone. Three tests pin the reason, because the obvious later "improvement" is to add detection: both configs resolve to one family, neither differing key may be read by the registry, and the name must state both models. Reverting the name fails the third and nothing else. 751 Python tests green.
fix(registry): the GLM family names both models it loads (GLM-5.2/5.3)
coli convert ran tools/convert_fp8_to_int4.py on whatever it was given.
That converter is GLM-5.2's and classifies tensors by FLAT name:
if name in ("model.embed_tokens.weight", "lm_head.weight"): return "io"
GLM-5.3-Flash nests its language model under the vision wrapper, so its
embedding is model.language_model.embed_tokens.weight. It matched no
rule, fell into the final endswith(".weight") fallback, and was
quantized. Hours later glm53.c read it with load_f32, refused the U8,
and the engine died inside coli web. Nothing on that path was wrong
except the first choice.
The family now declares its converter and which coli options that script
actually takes, and coli reads config.json -- 5 KB, the same hook the
converter's own guard uses -- to choose. GLM-5.2's command is unchanged,
including the int8 MTP second pass. GLM-5.3-Flash reaches
convert_glm53.py, with no precision flags and one pass, because that
script has neither. Families coli cannot drive keep today's path, where
the #1305 guard refuses with a per-family pointer; keeping a second copy
of that map here is the defect this commit is closing.
An option the target does not take is REFUSED, not dropped: --ebits on a
converter with no such flag would otherwise give a precision the user
never chose, silently. When the family cannot be determined -- no
network, no huggingface_hub -- the old command runs unchanged, so a
working conversion never fails because a metadata fetch did.
Two things this surfaced.
tools/convert_glm53.py was never in any release archive, on dev or in
1.10.1. The documented fix for #1368 could not be run from a download.
It is packaged now: naming a script from the registry is a third way to
reach a file, and pack_python.py follows it like the other two. The
existing release test caught this as a regression from moving the name
out of a literal call site, which is exactly the job it was written for.
The generic banner said "GLM-5.2 · 744B" when no model was given: an
example that aged, and since #1367 that family is not even called that.
It now counts the families from the registry.
Negative controls: reverting the dispatch fails the routing test,
reverting the packer rule fails the release test.
The refusal of an option the target converter does not take scanned sys.argv for the literal flag. That misses --ebits=3 and every abbreviation argparse accepts (--eb 3), so those forms were dropped silently -- the exact failure the refusal exists to prevent. The four precision options now default to None in the parser, so "written" is "not None" whatever the spelling, and the real defaults are applied in one place afterwards. Verified through the real parser with --ebits 3, --ebits=3, --eb 3 and --xbits=4 against a glm5_next checkpoint: all four refused before any converter is launched; an accepted --group-size=128 passes through as 128.
fix(convert): coli convert picks the converter from the checkpoint (#1368)
#1310 (merged 2026-09-02, not in 1.10.1) made the REAP-pruned DeepSeek V4 Flash load: puwaer/DeepSeek-V4-Flash-0731-reap-150b, 85 GB, 132 of the 256 routed experts, same engine, no conversion. A second user confirmed it runs. Nothing in README or docs said so. Documented in the model table, the RAM table and the V4 download section, with the layout detail that made #1310 necessary. And one label fixed on the way. The banner announced that checkpoint as "DeepSeek V4 Flash 284B": display_scale is the reference checkpoint's number, printed for every checkpoint of the family. That is #1367 again, except that here the config can tell the two apart -- 43 layers in both, 256 experts against 132 -- so the banner must. The family now records the reference expert count (from the vendor config.json) and, when a checkpoint's count differs, the banner prints the measured geometry the existing unknown-model path already produces: 43L x 132E MoE. Checked on the real config.json of both checkpoints; the official one still says 284B. Only deepseek_v4 declares a reference, so no other family's banner changes. Reverting the branch fails exactly the new REAP test.
alarm()/SIGALRM exist on POSIX and not on MinGW, and the Windows UCRT64 job is what proved it: the test could not build there. The test already uses pthread and nanosleep, so a detached watchdog thread does the same job everywhere it builds: sleep up to 10 s in 100 ms steps, and if qt_shutdown() has not returned by then, print the same line and _exit(1). Verified on Linux: the test passes; a probe with the flag never set exits 1 after 10 s.
Version in the six places it lives (version.py, four README badges, the site's shipping line); Cargo.lock's 1.10.1 is httparse/hyper, not ours. CHANGELOG had stopped at 1.7.0: the [Unreleased] block was Qwen3.8, which shipped as 1.10.0 on 2026-08-31 per that release's notes, so it is labelled as such, 1.10.1 is recorded as the packaging repair it was, and 1.10.2 lists what dev accumulated since.
docs+banner: DeepSeek V4 Flash REAP-150B is supported, and is not 284B
Resolves the conflict on the glm53 FamilyLimits line: dev had raised the interactive output cap to 16384 (#1302) on the same line this PR changes. Kept dev's arguments and this PR's max_kv_slots=16, which matches GLM53_MAX_SLOTS in glm53.c.
docs: lead Windows DeepSeek users to release launcher
No conflicts. Pushed by a maintainer so that CI runs on this PR: as a first-time contribution it had never been allowed to run.
fix(glm53): expose supported KV slots
Security: confine image_url local reads and pin CI actions by SHA
release: 1.10.2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patch release. Three of these fixes answer reports made against 1.10.1 in the days after it shipped. No new engine; one behaviour change, opt-in.
If you hit one of these on 1.10.1, this is the fix
coli convertproduced a GLM-5.3-Flash container the engine refused (#1368). It ran GLM-5.2's converter on everything; Flash nests its language model under the vision wrapper, so the embedding matched no rule, fell through to the generic fallback, and was quantized. The engine was right to refuse it, hours later, insidecoli web.coli convertnow reads the checkpoint'sconfig.jsonand picks the converter from it. GLM-5.2's path is unchanged. An option the target converter does not take is refused, not silently dropped (#1369). Andtools/convert_glm53.py, the right tool for Flash, is in the archive for the first time (#1364).coli doctorsaid two core tensors were missing from a healthy GLM-5.3-Flash (#1365). The check held three literal tensor names taken from GLM-5.2. It now matches roles, prefix-agnostically, so it holds for any family that nests, including ones not written yet (#1366). Nothing was ever missing from those downloads.The archive was missing files
colireaches (#1359):iq3_pack.pyand its grid, reachable only through a subprocess-launched script. The packer follows that edge now, and data files next to a module (#1364).Also fixed
RSS guard counts the memory the process owns, not page cache the kernel can reclaim, so a mapped container no longer evicts experts to free memory it was not using (fix(rss): count the memory we own, not what the kernel can reclaim #1350)
servehonors CANCEL mid-turn (fix(serve): honor CANCEL while the turn is still running #1336); the disconnect scenario is built, not hoped for (test(serve): build the disconnect scenario instead of hoping for it #1329)Metal: bit-exact fp8-e4m3 decode (fix(metal): bit-exact fp8-e4m3 decode via ieee754 exponent bitcast #1346)
Qwen3.6: tokenizer merges in both spellings (fix(qwen36): read tokenizer.json merges in string and pair spellings #1319)
macOS: Homebrew prefixes found when
brewis off the PATH (fix(build): fall back to the Homebrew prefixes when brew is off the PATH #1320); DeepSeek V4 reports real CPU and memory on macOS (fix(dsv4): report real CPU and memory in HWINFO on macOS #1308)doctoromits the GPU plan for a CPU-only engine (fix(doctor): omit GPU plan for CPU-only engine #1322); a missing core tensor explains itself on every fatal path (fix(loader): a missing core tensor explains itself on every fatal path #1318)GLM-5.3-Flash
--no-thinkis the template's lowest effort level, not a prompt shape of ours (fix(glm53): --no-think is the template's lowest level, not a shape of ours #1327)The converter refuses a checkpoint it cannot serve, with a per-family pointer (fix(convert): refuse a checkpoint this converter cannot serve #1305)
GLM-5.3-Flash
servecan use the 16 KV slots the engine has; the registry declared 1 (fix(glm53): expose supported KV slots #1283)Security:
image_urllocal reads refuse.., and withCOLI_IMAGE_ROOTset the path must resolve inside it, so an authenticated client of a non-loopback server cannot read arbitrary files through the image API; errors no longer confirm a path exists (Security: confine image_url local reads and pin CI actions by SHA #1354, dmoraesrs). CI runs withcontents: readand third-party actions are pinned by SHASupported checkpoints, stated plainly
colibriengine. It is the same base model as GLM-5.2 (Z.ai say so; the configs differ only by the transformers version that wrote them), so the family is now namedGLM-5.2/5.3rather than announcing one as the other (fix(registry): the GLM family names both models it loads (GLM-5.2/5.3) #1367).puwaer/DeepSeek-V4-Flash-0731-reap-150b, 85 GB, 132 of 256 experts) loads with the same engine and no conversion (Enable support for quantized DeepSeek V4 Flash #1310), confirmed by a second user. It is documented, and the banner reports it by its measured geometry,43L x 132E, not as 284B (docs+banner: DeepSeek V4 Flash REAP-150B is supported, and is not 284B #1371).Changed
is_xoverrun with two or more GPUs, a shutdown that could hang, and a use-after-free on int8 containers (qwen36 CUDA tier: qt_issue overruns G.is_x with two or more GPUs (stride 8*D into a 32*D buffer) #1339, qwen36 CUDA tier: qt_shutdown never signals cv_take, so pthread_join can hang if a group is open #1340, qwen36 CUDA tier: int8 containers free the weights the tier still points at (regression from #1334) #1341; fix fix(qwen36): three CUDA expert-tier bugs (#1339, #1340, #1341) #1344 by crichalchemist)COLI_MAP_EXPERTS=1: experts served through a per-shard file mapping (st.h: shared per-shard mapping primitive, opt-in via COLI_MAP_EXPERTS=1 #1325). Off by default; output byte-identical either wayDocs
coli.cmd, instead of a source build (docs: lead Windows DeepSeek users to release launcher #1291)Build
Merge with Create a merge commit, not squash: main already has two commits dev does not (the #1363 Discord link merge), and a squash would break ancestry again as it did for 1.10.0. Then tag
v1.10.2and paste the text above into the release.