From 0a34ea0832766841ad39c100bb15843806e53af2 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 10:50:11 -0500 Subject: [PATCH 01/80] =?UTF-8?q?fix(cognition):=20wire=20the=20LIVED-TURN?= =?UTF-8?q?=20producer=20=E2=80=94=20her=20own=20conversations=20become=20?= =?UTF-8?q?curriculum,=20not=20just=20graded=20cards=20(#319)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ExperienceRecord::from_lived_turn` had ZERO production callers tree-wide. The only live producer was `from_kanban_grade` (benchmark.rs:1989), so the experience stream — "the SPINE the salience→curriculum seam was missing", in `experience_stream_path`'s own words — was fed EXCLUSIVELY BY GRADES. A citizen could hold a thousand real conversations and her stream stayed empty. Nothing could learn from work nobody had scored. Everything needed already existed and was correct: the record shape, the two magic-number-free lived-salience signals (infra fault, non-convergence), `ErrorSalience` reading them with no new detector, and a doc block naming "the lived-turn settle site" as a producer. The CALL was missing. Same shape as #341 (boot called the unbounded sink; the rotating one had zero callers) and #362 (ProbeRouterLayer installed with its handle discarded) — a built component with a dead wire, which is why [[an-absence-is-an-unfinished-measurement]] keeps earning its place. WIRED at `service_loop.rs`'s live settle site, BEFORE `SettleStep::from_settled` consumes the outcome — the only point where a lived turn's settle verdict exists. The driver stays a driver per PERSONA-COGNITION-PIPELINE §3: one call, no policy. The behaviour lives in `experience::record_lived_turn` so it is unit-testable without booting a loop. BEST-EFFORT, and that is NOT a fallback ([[no-fallbacks-ever]] holds): nothing is substituted and no result is fabricated. Learning is a SIDE channel to being — a full disk must not make a citizen mute mid-sentence. The failure is WARNED with the path and the honest consequence ("this episode will not become curriculum"). COMPRESSION, because the new writer would otherwise have been the FIFTH hand-rolled spelling of the citizen storage layout. `citizens/peers/` was written as `join("citizens/peers")` in modules/work.rs, `join("citizens").join("peers")` in commands/benchmark.rs, and in prose in persona_workspace.rs + persona_roster.rs. Now ONE resolver, `identity::citizen_peer_dir(root, peer)`, in the module whose own doc says "the whole crate imports identity from ONE home" and warns against exactly this re-invention. Keyed by `PeerId`, never a String — the directory name IS her identity. Pure path arithmetic, so a read-only caller never mints an empty citizen dir, which its test pins. Tests: 11/11 cognition::experience (the lived-turn test now asserts the PRODUCER — empty stream → record lands at the canonical path → a second turn appends rather than overwrites, i.e. the wire, not just the record's shape) + identity's one-spelling test. `cargo check -p continuum-core --features metal,accelerate` clean. LIVE PROOF OWED and NOT claimed: no live turn has run on this build. The falsifiable prediction is that a citizen's `citizens/peers//experience.jsonl` gains a `LivedTurn` record on her next real room turn, where today it holds only kanban grades or nothing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/experience.rs | 90 +++++++++++++++++++ core/continuum-core/src/identity/mod.rs | 51 +++++++++++ .../src/persona/service_loop.rs | 13 +++ 3 files changed, 154 insertions(+) diff --git a/core/continuum-core/src/cognition/experience.rs b/core/continuum-core/src/cognition/experience.rs index d5f30cb93..b77577662 100644 --- a/core/continuum-core/src/cognition/experience.rs +++ b/core/continuum-core/src/cognition/experience.rs @@ -466,6 +466,61 @@ pub fn append_experience( writeln!(f, "{line}") } +/// Record ONE lived room turn into the citizen's own experience stream — the +/// producer this module's doc has named since #319 and that nothing ever called. +/// +/// ## Why this exists (measured 2026-08-17) +/// +/// [`ExperienceRecord::from_lived_turn`] had **zero production callers** tree-wide. +/// The only live producer was [`ExperienceRecord::from_kanban_grade`], so the +/// experience stream — "the SPINE the salience→curriculum seam was missing" per +/// [`experience_stream_path`]'s own doc — was fed exclusively by GRADES. A citizen +/// could hold a thousand real conversations and her stream stayed empty, which +/// means the curriculum could only ever learn from work someone had scored. The +/// machinery, the salience detector and the doc were all correct and in place; the +/// call was missing. (Same shape as #341 and #362: a built component with a dead +/// wire — [[an-absence-is-an-unfinished-measurement]].) +/// +/// ## Why the write is best-effort, and what that is NOT +/// +/// A failed append is WARNED and the turn proceeds. This is not a silent fallback +/// ([[no-fallbacks-ever]] still holds): nothing is substituted and no result is +/// fabricated. Learning is a SIDE channel to being — a full disk must not make a +/// citizen mute mid-sentence. The failure is visible in the log with the path, and +/// the honest consequence (this episode never becomes curriculum) is stated there. +/// +/// Storage is keyed by [`crate::identity::citizen_peer_dir`] — one spelling of the +/// citizen-layout decision, so this producer cannot drift from the consumer that +/// drains the same stream. +pub fn record_lived_turn( + root: &std::path::Path, + peer: crate::identity::PeerId, + stimulus: &str, + settled: &crate::cognition::act_observe::SettleOutcome, +) { + let peer_dir = crate::identity::citizen_peer_dir(root, peer); + if let Err(e) = std::fs::create_dir_all(&peer_dir) { + tracing::warn!( + peer = %peer, + dir = %peer_dir.display(), + error = %e, + "could not create the citizen dir for her experience stream — this lived \ + turn will not become curriculum (#319 producer)" + ); + return; + } + let record = ExperienceRecord::from_lived_turn(stimulus, settled); + if let Err(e) = append_experience(&peer_dir, &record) { + tracing::warn!( + peer = %peer, + path = %experience_stream_path(&peer_dir).display(), + error = %e, + "could not append a lived turn to the experience stream — this episode \ + will not become curriculum (#319 producer)" + ); + } +} + /// Load the persona's experience stream. A missing file is an empty stream (a /// fresh mind has no history — not an error). Unparseable lines are counted and /// WARNED, never silently dropped: one corrupt line must not brick learning @@ -721,6 +776,41 @@ mod tests { "a lived turn has no objective grader" ); + // ── The PRODUCER, which is the half that was dead ──────────────────────── + // `from_lived_turn` had zero production callers, so the experience stream was + // fed ONLY by graded bench cards: a citizen could hold a thousand real + // conversations and her stream stayed empty. This asserts the append actually + // reaches HER OWN stream at the canonical citizen path — the wire, not just + // the record's shape (the shape was always fine; nothing called it). + let root = tempfile::tempdir().expect("tempdir"); + let peer = crate::identity::PeerId::from_u128(0x90e758b2_0000_4000_8000_000000000002); + assert!( + load_experiences(&crate::identity::citizen_peer_dir(root.path(), peer)).is_empty(), + "a fresh citizen's stream starts empty" + ); + + record_lived_turn(root.path(), peer, "what did we decide about the grader?", &settled); + + let stream = load_experiences(&crate::identity::citizen_peer_dir(root.path(), peer)); + assert_eq!(stream.len(), 1, "the lived turn reached her stream"); + assert_eq!(stream[0].task.prompt, "what did we decide about the grader?"); + assert!( + stream[0].task.test.is_none(), + "still no objective grader — the append must not invent one" + ); + // It lands where the DRAIN reads, keyed by her identity — producer and consumer + // cannot disagree about the path because both go through `citizen_peer_dir`. + assert!( + experience_stream_path(&crate::identity::citizen_peer_dir(root.path(), peer)).exists() + ); + + // Two turns append, never overwrite — a stream, not a slot. + record_lived_turn(root.path(), peer, "and the docstring ids?", &settled); + assert_eq!( + load_experiences(&crate::identity::citizen_peer_dir(root.path(), peer)).len(), + 2 + ); + // A lived turn that died on a serving fault: ok=false, honest grade — but STILL untestable. let faulted = SettleOutcome::infra_failure("lane 58057 refused qwen3"); let lived_fault = ExperienceRecord::from_lived_turn("ping", &faulted); diff --git a/core/continuum-core/src/identity/mod.rs b/core/continuum-core/src/identity/mod.rs index 576721826..1cc8a4b84 100644 --- a/core/continuum-core/src/identity/mod.rs +++ b/core/continuum-core/src/identity/mod.rs @@ -85,6 +85,28 @@ use uuid::Uuid; /// generated TS shape is unchanged. pub use airc_core::PeerId; +/// The durable on-disk home of one citizen: `/citizens/peers/`. +/// +/// THE ONE SPELLING of that layout. It was being rebuilt by hand at each use, in +/// two different shapes — `home.join("citizens/peers")` (`modules/work.rs`) and +/// `home.join("citizens").join("peers")` (`commands/benchmark.rs`) — plus prose +/// copies in `persona_workspace.rs` and `persona_roster.rs`. Four expressions of +/// one decision is exactly the drift the compression principle forbids, and the +/// next writer (the lived-turn experience stream) would have made a fifth. +/// +/// Keyed by [`PeerId`], never a `String`: the citizen's identity IS the directory +/// name, so a caller that has not resolved a name to an identity cannot address +/// her storage by accident ([[uuids-are-not-strings-and-never-hand-drawn]]). +/// +/// Pure path arithmetic — creates nothing, checks nothing. Callers that need the +/// directory to exist say so themselves, so a read-only caller never has the side +/// effect of minting an empty citizen dir. +pub fn citizen_peer_dir(root: &std::path::Path, peer: PeerId) -> std::path::PathBuf { + root.join("citizens") + .join("peers") + .join(peer.as_uuid().to_string()) +} + /// What a CALLER writes when it means "that persona" — a full UUID, an 8-char /// short-id, or a name (`"Asha"`). Deliberately NOT an identity. /// @@ -816,4 +838,33 @@ mod tests { assert_eq!(agents[0].1.agent_name, "claude-session-X"); assert_eq!(agents[0].1.agent_provider.as_deref(), Some("claude")); } + + // what this catches: a FIFTH hand-rolled spelling of the citizen storage layout. + // It was already written four ways — `join("citizens/peers")` in modules/work.rs, + // `join("citizens").join("peers")` in commands/benchmark.rs, and prose copies in + // persona_workspace.rs + persona_roster.rs. The two path spellings produce the SAME + // path today, which is exactly why the drift is invisible until one of them changes. + // Pins the shape AND that it is pure arithmetic: a read-only caller must never have + // the side effect of minting an empty citizen dir. + #[test] + fn the_citizen_peer_dir_has_exactly_one_spelling_and_creates_nothing() { + let peer = PeerId::from_u128(0xfe4dac17_0000_4000_8000_000000000001); + let root = std::path::Path::new("/x/.continuum"); + let dir = citizen_peer_dir(root, peer); + + assert_eq!( + dir, + root.join("citizens").join("peers").join(peer.to_string()), + "the layout is /citizens/peers/" + ); + // Identity, not a formatted string: the directory name IS the peer id. + assert!(dir.ends_with(peer.as_uuid().to_string())); + // Pure: nothing was created under a temp root either. + let tmp = tempfile::tempdir().expect("tempdir"); + let under_tmp = citizen_peer_dir(tmp.path(), peer); + assert!( + !under_tmp.exists(), + "resolving a path must not create it — a read-only caller mints nothing" + ); + } } diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 6c292733c..0855efaef 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -1092,6 +1092,19 @@ async fn serve_persona_loop_inner( framing, ) .await; + // THE LIVED-TURN PRODUCER (#319). Her own experience stream is fed + // here — before `from_settled` consumes the outcome — because this is + // the only place a lived turn's settle verdict exists. Until now the + // stream was fed ONLY by graded bench cards, so a citizen's real + // conversations could never become curriculum. Best-effort by design: + // see `record_lived_turn` for why learning must not be able to mute + // her. Driver stays a driver — one call, no policy here. + crate::cognition::experience::record_lived_turn( + &crate::modules::persona_instance_manager::resolve_continuum_root(), + ctx.identity.peer_id, + &msg.text, + &outcome, + ); crate::cognition::act_observe::SettleStep::from_settled(outcome) }; // Turn done: drop the cycle's sink so the forwarder's channel closes, From 08ccd61398ed1ca84ded0f1a477ee236d5ea89b8 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 11:19:05 -0500 Subject: [PATCH 02/80] fix(cognition): the lived-turn producer belongs in the settle DRIVER, not at one call site (#319) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED, and it retracts my own claim from 200dab984. I wrote that the #319 producer was wired and that a LivedTurn record was owed on the next real turn. The wire was wrong, so the record could never come: 180 experience records across 5 citizens on this box, 100% `source: "Eval"`, ZERO `LivedTurn` — while the same core showed 47 persona.upstart.bind, 97 presence rows and 4 delib.context.render, i.e. citizens alive and deliberating. ROOT CAUSE. `record_lived_turn` was called from ONE of the THREE `drive_to_settle` sites in service_loop — the directed-message path (:1087). The self-tick path (:2628) and the held-work path (:1207) settle turns through the same driver and recorded nothing. Three callers, one remembering: a missing constraint expressed as three sites, not a bug at one ([[the-same-bug-at-two-sites-is-a-missing-constraint]]). FIX. The record moves INTO `drive_to_settle`, which is the one place a `SettleOutcome` is born. `drive_to_settle` is now a thin wrapper over a private `settle_to_outcome`, so the write happens ONCE around the driver rather than at its four return paths — a fifth return path added later inherits learning instead of silently opting out, and no call site carries learning policy. Gated on `WorkspaceCycle::acting()`, because that is where a citizen's identity lives: a cycle with no `ActingBody` is pure cognition (a faculty test, a replay) and is nobody's lived experience. Structural absence, not a skipped write. SECOND FIX, applying Joel's rule that a required thing should not be re-expressible: dropped the `stimulus: &str` parameter. It was a second, independent expression of "what she was responding to", free to disagree with what the turn actually perceived — and it already did, since the old call site passed only `msg.text` while the mind had settled over the whole rendered burst. The stimulus is now read off `SettleOutcome.world_state` (set on every return path), so the disagreement is unrepresentable rather than merely discouraged. The test asserts `task.prompt == settled.world_state` instead of a literal. GUARD, positive-controlled (not merely green): a source-walking test asserts `record_lived_turn` has EXACTLY ONE production caller and that it is settle.rs. Verified failing — injected a second call into service_loop and it reported `found 2: [settle.rs, service_loop.rs]` with the remediation in the message. Reverted. Comment-stripped so prose naming the function cannot read as a call (registry.rs's module-wiring audit is the house precedent). cargo check clean; cognition::experience 12/12, cognition::act_observe 38/38. LIVE PROOF STILL OWED and NOT claimed: no turn has run on this build yet. Baseline for falsifiability is recorded above — 180 records, 100% Eval, 0 LivedTurn. The prediction is that a hosted citizen's `citizens/peers//experience.jsonl` gains a LivedTurn record on her next settled turn from ANY of the three paths, where today it holds only kanban grades. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/act_observe/settle.rs | 41 +++++++ .../src/cognition/experience.rs | 110 +++++++++++++++++- .../src/persona/service_loop.rs | 19 +-- 3 files changed, 152 insertions(+), 18 deletions(-) diff --git a/core/continuum-core/src/cognition/act_observe/settle.rs b/core/continuum-core/src/cognition/act_observe/settle.rs index 815a0bffa..f0ec2f192 100644 --- a/core/continuum-core/src/cognition/act_observe/settle.rs +++ b/core/continuum-core/src/cognition/act_observe/settle.rs @@ -34,12 +34,53 @@ use super::types::{SettleOutcome, SettleStep}; /// forever" persona is a fitness gap to train away, never a substrate ceiling — /// §4). When the budget runs out mid-action, the final un-driven `Act` is /// returned and the grader scores it as unfinished — never a fabricated answer. +/// +/// ## Why the lived-experience write lives HERE and not at the call sites +/// +/// This function is the ONE place a `SettleOutcome` is produced, so it is the one +/// place "a turn was lived" can be recorded without the fact being re-derived per +/// caller. It was not always: the #319 producer was first wired into a SINGLE +/// service_loop call site (the directed-message path), which left the self-tick +/// path and the held-work path settling turns that no experience record ever +/// described. Three callers, one of them remembering — the missing-constraint +/// shape ([[the-same-bug-at-two-sites-is-a-missing-constraint]]), and the reason +/// zero `LivedTurn` records existed on disk while citizens were demonstrably +/// deliberating. +/// +/// Recording once around the driver — rather than at each of its four return +/// paths — is deliberate for the same reason: a fifth return path added later +/// inherits the record instead of silently opting out of learning. +/// +/// The write is gated on [`WorkspaceCycle::acting`] because that is where a +/// citizen's identity lives. A cycle with no `ActingBody` is pure cognition (a +/// faculty test, a replay) — it is nobody's lived experience, so there is no +/// stream it belongs in. That is a structural absence, not a skipped write. pub async fn drive_to_settle( cycle: &WorkspaceCycle, burst: impl Into, room_id: Uuid, max_acts: usize, framing: TurnFraming, +) -> SettleOutcome { + let settled = settle_to_outcome(cycle, burst, room_id, max_acts, framing).await; + if let Some(body) = cycle.acting() { + crate::cognition::experience::record_lived_turn( + &crate::modules::persona_instance_manager::resolve_continuum_root(), + crate::identity::PeerId::from_uuid(body.persona_id), + &settled, + ); + } + settled +} + +/// The settle loop itself. Private so that [`drive_to_settle`] is the only way to +/// reach it — every produced outcome therefore passes the lived-experience seam. +async fn settle_to_outcome( + cycle: &WorkspaceCycle, + burst: impl Into, + room_id: Uuid, + max_acts: usize, + framing: TurnFraming, ) -> SettleOutcome { let burst: Burst = burst.into(); let mut acts = 0usize; diff --git a/core/continuum-core/src/cognition/experience.rs b/core/continuum-core/src/cognition/experience.rs index b77577662..6a65116e5 100644 --- a/core/continuum-core/src/cognition/experience.rs +++ b/core/continuum-core/src/cognition/experience.rs @@ -492,10 +492,20 @@ pub fn append_experience( /// Storage is keyed by [`crate::identity::citizen_peer_dir`] — one spelling of the /// citizen-layout decision, so this producer cannot drift from the consumer that /// drains the same stream. +/// +/// ## Why there is no `stimulus` parameter +/// +/// The stimulus is not passed in: it is READ OFF the outcome +/// (`SettleOutcome.world_state`, set on every return path of the settle driver to +/// the burst it actually deliberated over). A `stimulus: &str` argument would be a +/// second, independent expression of "what she was responding to" — free to +/// disagree with what the turn genuinely perceived, and it already did: the first +/// call site passed only the inbound message text while the mind had settled over +/// the whole rendered burst. Deriving it from the required argument makes the +/// disagreement unrepresentable instead of merely discouraged. pub fn record_lived_turn( root: &std::path::Path, peer: crate::identity::PeerId, - stimulus: &str, settled: &crate::cognition::act_observe::SettleOutcome, ) { let peer_dir = crate::identity::citizen_peer_dir(root, peer); @@ -509,7 +519,7 @@ pub fn record_lived_turn( ); return; } - let record = ExperienceRecord::from_lived_turn(stimulus, settled); + let record = ExperienceRecord::from_lived_turn(&settled.world_state, settled); if let Err(e) = append_experience(&peer_dir, &record) { tracing::warn!( peer = %peer, @@ -789,11 +799,19 @@ mod tests { "a fresh citizen's stream starts empty" ); - record_lived_turn(root.path(), peer, "what did we decide about the grader?", &settled); + record_lived_turn(root.path(), peer, &settled); let stream = load_experiences(&crate::identity::citizen_peer_dir(root.path(), peer)); assert_eq!(stream.len(), 1, "the lived turn reached her stream"); - assert_eq!(stream[0].task.prompt, "what did we decide about the grader?"); + // The stimulus is the outcome's OWN world_state, never a separately-passed + // string: the record therefore cannot describe a stimulus the turn did not + // actually perceive. (The first wiring passed only the inbound message text + // while the mind had settled over the whole rendered burst — the two could + // disagree, and did.) + assert_eq!( + stream[0].task.prompt, settled.world_state, + "the recorded stimulus IS what she deliberated over" + ); assert!( stream[0].task.test.is_none(), "still no objective grader — the append must not invent one" @@ -805,7 +823,7 @@ mod tests { ); // Two turns append, never overwrite — a stream, not a slot. - record_lived_turn(root.path(), peer, "and the docstring ids?", &settled); + record_lived_turn(root.path(), peer, &settled); assert_eq!( load_experiences(&crate::identity::citizen_peer_dir(root.path(), peer)).len(), 2 @@ -1205,5 +1223,87 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); assert!(load_experiences(&dir).is_empty()); } + + // what this catches: the lived-turn producer wired per-CALL-SITE again. + // + // The regression this pins is not hypothetical — it is what shipped. The + // producer was first called from ONE of the three `drive_to_settle` sites in + // `service_loop`, so the self-tick and held-work paths settled turns that no + // record ever described, and zero `LivedTurn` records existed on disk while + // citizens were demonstrably deliberating. The fix moved the call INTO the + // settle driver, the one place a `SettleOutcome` is born. + // + // So the invariant is a COUNT, not a location-check: exactly one production + // caller, and it is the driver. A second caller is either a double-record or + // a path that opted itself out of learning; both are the same defect class + // ([[the-same-bug-at-two-sites-is-a-missing-constraint]]). + #[test] + fn the_lived_turn_producer_has_exactly_one_production_caller_the_settle_driver() { + // `//`-prefixed content is dropped so prose naming the function — this + // very comment, and the doc on `record_lived_turn` — can never read as a + // call. Crude on purpose: a `//` inside a string literal could only ever + // HIDE a call from us, and a hidden call still trips the count if real. + fn code_only(src: &str) -> String { + src.lines() + .map(|l| match l.find("//") { + Some(i) => &l[..i], + None => l, + }) + .collect::>() + .join("\n") + } + fn walk(dir: &std::path::Path, out: &mut Vec<(std::path::PathBuf, String)>) { + let Ok(entries) = std::fs::read_dir(dir) else { + return; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + walk(&path, out); + } else if path.extension().is_some_and(|e| e == "rs") { + if let Ok(text) = std::fs::read_to_string(&path) { + out.push((path, text)); + } + } + } + } + let mut files = Vec::new(); + walk( + &std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src"), + &mut files, + ); + assert!(!files.is_empty(), "the source walk found nothing — the guard \ + would be vacuously green, which is worse than red"); + + let mut callers: Vec = Vec::new(); + for (path, src) in &files { + // This file DEFINES it and its own tests exercise it — neither is a + // production call site. + if path.ends_with("cognition/experience.rs") { + continue; + } + if code_only(src).contains("record_lived_turn(") { + callers.push(path.display().to_string()); + } + } + + assert_eq!( + callers.len(), + 1, + "expected exactly ONE production caller of record_lived_turn (the \ + settle driver); found {}: {callers:?}. If you are adding a call at a \ + turn call site, don't — `drive_to_settle` already records every \ + outcome it produces, so a second call double-writes her stream. If \ + you are adding a NEW way to settle a turn that bypasses the driver, \ + that is the thing to reconsider.", + callers.len() + ); + assert!( + callers[0].ends_with("cognition/act_observe/settle.rs"), + "the one caller must be the settle driver — the only place a \ + SettleOutcome is born; found {}", + callers[0] + ); + } } } diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 0855efaef..7154f5768 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -1092,19 +1092,12 @@ async fn serve_persona_loop_inner( framing, ) .await; - // THE LIVED-TURN PRODUCER (#319). Her own experience stream is fed - // here — before `from_settled` consumes the outcome — because this is - // the only place a lived turn's settle verdict exists. Until now the - // stream was fed ONLY by graded bench cards, so a citizen's real - // conversations could never become curriculum. Best-effort by design: - // see `record_lived_turn` for why learning must not be able to mute - // her. Driver stays a driver — one call, no policy here. - crate::cognition::experience::record_lived_turn( - &crate::modules::persona_instance_manager::resolve_continuum_root(), - ctx.identity.peer_id, - &msg.text, - &outcome, - ); + // The lived-turn experience write (#319) is NOT here: it lives inside + // `drive_to_settle`, which is the one place a `SettleOutcome` is born. + // It WAS here, at this single call site, and that was the bug — the + // self-tick and held-work paths settle turns through the same driver + // and got no record, so nothing on disk ever described a lived turn. + // The driver stays a driver: no learning policy at any call site. crate::cognition::act_observe::SettleStep::from_settled(outcome) }; // Turn done: drop the cycle's sink so the forwarder's channel closes, From a87f7c871f835952beffd566250e386eb94cb855 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 12:59:38 -0500 Subject: [PATCH 03/80] =?UTF-8?q?fix(persona):=20a=20YIELD=20is=20not=20a?= =?UTF-8?q?=20REST=20=E2=80=94=20kill=20the=20starvation=20ratchet=20that?= =?UTF-8?q?=20pinned=20the=20whole=20roster=20at=20the=20240s=20cap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured live 2026-08-17, 24 hosted citizens: ONE self-tick across the entire roster in 40 minutes, on a serving lane that was healthy and decoding at ~17 tok/s the whole time. The lane was never the problem. The cadence rule was. The self-tick gates on a single ambient-turn permit (AMBIENT_TURN_CONCURRENCY = 1). When a peer holds it, the citizen yields — she never ran, never looked at the room, learned nothing. The yield branch nonetheless charged her the SAME 1.5×-toward-the-cap backoff as a citizen who ran a full cycle and found nothing new. With 24 citizens, ~23 yield on every beat. Eight yields carries 15s past the 240s rest cap — a 16× slowdown — and she STAYS there, because the only two things that reset the beat are a successful cycle (which the backoff itself now denies her) and an inbound message. Transient contention compounds into permanent slowness, and it gets strictly worse as the roster grows. That is a ratchet, not a rest. Fix: a yield leaves the beat exactly where it was. The earned backoff after a real, fruitless cycle is untouched — that one is honest. Extracted the rule into a pure `next_beat_after(BeatOutcome, ..)` (the `core_bind_guard::decide` shape) because the three outcomes — Engaged / NothingNew / YieldedNoSlot — are trivially confusable inline, and confusing two of them cost the roster 16×. Both call sites now derive from the one function; there is no second expression of the cadence. Regression test pins the invariant in both directions: 50 yields must not cost a single millisecond; a fruitless cycle must still back off and must saturate at the cap. Positive-controlled — restoring the old compounding fails the assertion by name. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/service_loop.rs | 125 +++++++++++++++++- 1 file changed, 119 insertions(+), 6 deletions(-) diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 7154f5768..eef806a6b 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -494,18 +494,52 @@ async fn serve_persona_loop_inner( match crate::cognition::resource_admission::try_hold_ambient_turn() { Some(permit) => permit, None => { - next_beat = (next_beat + next_beat / 2).min(rest_cap); + // A YIELD IS NOT A REST — do NOT compound the beat here. + // + // The backoff below (after a real cycle) is earned: she THOUGHT, + // the room had nothing new, so she rests deeper. A yield is the + // opposite — she never ran. A peer held the single ambient slot, + // she learned nothing, and there is nothing to rest ON. + // + // Compounding it built a STARVATION RATCHET, measured live + // 2026-08-17 on this box: `AMBIENT_TURN_CONCURRENCY == 1` and 24 + // hosted citizens, so ~23 yield on every beat. At 1.5× per yield + // a citizen crosses 15s → the 240s `rest_cap` in ~8 yields — 16× + // slower — and STAYS there, because the only two resets are a + // successful self-cycle (which the ratchet now denies her) and an + // inbound Msg. Transient contention became permanent slowness, and + // it deepened as the roster grew: measured ONE self-tick across 24 + // citizens in 40 minutes, on a lane that was healthy and decoding + // the whole time. + // + // Leaving the beat UNCHANGED is the minimal correct rule: her + // cadence stays whatever her own history earned, contention can no + // longer slow her, and the identity-derived `phase` still keeps the + // retries desynced so no herd forms. Yielding stays cheap — the + // permit is a non-blocking try and perception work all happens + // inside `run_self_cycle`, below this gate. + next_beat = next_beat_after( + BeatOutcome::YieldedNoSlot, + next_beat, + engaged_beat, + rest_cap, + ); continue; } }; let before = last_burst_fp; run_self_cycle(ctx, conversation, &opts, &mut last_burst_fp).await; drop(_self_tick_permit); - next_beat = if last_burst_fp != before { - engaged_beat - } else { - (next_beat + next_beat / 2).min(rest_cap) - }; + next_beat = next_beat_after( + if last_burst_fp != before { + BeatOutcome::Engaged + } else { + BeatOutcome::NothingNew + }, + next_beat, + engaged_beat, + rest_cap, + ); continue; } // A message means life in the room — snap back to a quick beat so she's present @@ -1435,6 +1469,47 @@ async fn serve_persona_loop_inner( Ok(outcome) } +/// What a self-tick beat DID, and therefore what the next interval should be. +/// +/// Extracted as a pure decision (the `core_bind_guard::decide` shape) because the +/// three outcomes are trivially confusable in an inline `match`, and confusing two +/// of them cost a measured 16× slowdown across the whole roster — see +/// [`next_beat_after`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum BeatOutcome { + /// She ran a cycle and the room had something new → stay engaged. + Engaged, + /// She ran a cycle and nothing had changed → rest deeper (earned backoff). + NothingNew, + /// She never ran: a peer held the single ambient slot. She learned NOTHING, + /// so there is nothing to rest on and the beat must NOT compound. + YieldedNoSlot, +} + +/// The self-tick cadence rule. Pure so every row is table-testable without a runtime. +/// +/// The bug this encodes against (measured live 2026-08-17, 24 hosted citizens): +/// `YieldedNoSlot` used to share `NothingNew`'s 1.5× backoff. With +/// `AMBIENT_TURN_CONCURRENCY == 1`, ~23 citizens yield per beat, so each of them +/// compounded 15s → the 240s cap in ~8 yields and STAYED pinned there — the only +/// resets being a successful cycle (which the backoff itself denied them) or an +/// inbound message. Transient contention became permanent slowness, and it got +/// worse as the roster grew. Measured symptom: ONE self-tick across 24 citizens in +/// 40 minutes, on a lane that was healthy and decoding throughout. +pub fn next_beat_after( + outcome: BeatOutcome, + current: std::time::Duration, + engaged: std::time::Duration, + rest_cap: std::time::Duration, +) -> std::time::Duration { + match outcome { + BeatOutcome::Engaged => engaged, + BeatOutcome::NothingNew => (current + current / 2).min(rest_cap), + // Unchanged: contention must never cost her cadence. + BeatOutcome::YieldedNoSlot => current, + } +} + /// Engaged heartbeat period — how often a persona pursuing its OWN intentions /// (no inbound message) gets another self-directed slice. This is the SELF-CHATTER /// pace, NOT message responsiveness: a real message wakes the loop instantly through @@ -4560,4 +4635,42 @@ mod tests { assert_eq!(outcome.turns_skipped, 0); assert_eq!(conversation.said().len(), 1); } + // what this catches: the STARVATION RATCHET — a yield being charged the rest + // backoff it did not earn. Regression for the live 2026-08-17 measurement (24 + // hosted citizens, AMBIENT_TURN_CONCURRENCY=1, ONE self-tick in 40 minutes on a + // healthy decoding lane). If `YieldedNoSlot` ever compounds again, contention + // silently becomes permanent slowness and the whole roster degrades as it grows. + #[test] + fn a_yield_never_costs_her_cadence_but_a_fruitless_cycle_still_rests() { + use std::time::Duration; + let engaged = Duration::from_millis(SELF_TICK_MS); + let cap = Duration::from_millis(SELF_TICK_REST_CAP_MS); + + // A yield leaves the beat EXACTLY where it was — she never ran. + let mut beat = engaged; + for _ in 0..50 { + beat = next_beat_after(BeatOutcome::YieldedNoSlot, beat, engaged, cap); + } + assert_eq!( + beat, engaged, + "50 yields must not slow her by a single millisecond — she never ran, \ + so there is nothing to rest on (the ratchet that pinned 24 citizens at the cap)" + ); + + // A cycle that found nothing DOES rest deeper, and saturates at the cap. + let mut beat = engaged; + let once = next_beat_after(BeatOutcome::NothingNew, beat, engaged, cap); + assert!(once > engaged, "an earned rest still backs off"); + for _ in 0..50 { + beat = next_beat_after(BeatOutcome::NothingNew, beat, engaged, cap); + } + assert_eq!(beat, cap, "earned rest saturates at the cap, never beyond"); + + // Finding something new snaps her straight back to engaged from full rest. + assert_eq!( + next_beat_after(BeatOutcome::Engaged, cap, engaged, cap), + engaged, + "life in the room returns her to the engaged beat immediately" + ); + } } From 6229b3762a3fee0b521a6b347df935c73e892687 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 13:18:33 -0500 Subject: [PATCH 04/80] =?UTF-8?q?fix(cognition):=20the=20ambient-turn=20po?= =?UTF-8?q?ol=20was=20a=20hardcoded=201=20=E2=80=94=20derive=20it=20from?= =?UTF-8?q?=20the=20live=20lane=20budget,=20like=20its=20own=20doc=20alway?= =?UTF-8?q?s=20claimed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The self-tick gate in service_loop.rs has documented this permit as "sized to the LIVE served lane count (LaneAdmission ← set_served_lane_count)" for as long as it has existed. It never was. It was `const AMBIENT_TURN_CONCURRENCY: usize = 1`, and that constant was the roster's starvation ceiling: on this box, 4 served lanes and 24 hosted citizens, so 3 non-directed lanes sat permanently idle while 23 citizens yielded every beat against a pool of one. Two things kept it invisible. The comment described the intended design, so reading the call site told you the opposite of the truth. And a hard 1 is indistinguishable from a quiet room at n≈1 citizen — which is exactly how it was reasoned about ("under light load ambient turns are naturally serial, so this never throttles a quiet room"). It doesn't throttle a quiet room. It throttles a POPULATED one, and the roster only grew. Fix: the pool is `nondirected_budget()` — the same live (lanes − 1, floored at 1) budget the per-call lane reservation already uses. One machine, one answer to "how much non-directed concurrency is there", derived from live capacity instead of declared twice and then contradicted. Why loosening this bound is safe: the directed-turn guarantee never came from this permit. It comes from `acquire_serving_lane`, which caps non-directed CALLS at lanes−1 so a directed call always finds a lane — untouched here, and it is the layer that actually owns lane priority. This permit's job (#171) is anti-STAMPEDE: bound the fan-out when N peers wake on one beat and all read inflight=0. A bound of lanes−1 does that job exactly as well as a bound of 1 — still fixed, still non-blocking, still held across the turn — while no longer throttling below the hardware. Also wired the pool into `set_served_lane_count` alongside its two siblings. It is lazy for the same load-bearing reason they are (capture the real count at first use, never the boot ceiling), which makes "first use beat serving's first publish" a live possibility — and without the grow wiring the whole roster would stay pinned at the boot budget for the life of the process. That has its own regression test. Tests: the existing ambient test now pins a real 4-lane machine and asserts against the derived budget — a PREMISE CHANGE, stated as such, because at a hardcoded 1 it could not tell "correctly bounded" from "throttled below the hardware". Two new rows cover the halves that could silently hurt: a 1- and 2-lane box still admits exactly one ambient turn (byte-identical to before on the weakest supported hardware), and a cold-boot pool grows when serving publishes its real width. 8/8 green. Corrected two stale comments while here: `resource_admission.rs` claimed "an idle self-tick is not ambient-permit-gated at all", contradicted by the self-tick gate's own comment block in the same tree; and the service_loop references to the now-deleted constant. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/resource_admission.rs | 130 ++++++++++++++++-- .../src/persona/service_loop.rs | 10 +- 2 files changed, 122 insertions(+), 18 deletions(-) diff --git a/core/continuum-core/src/cognition/resource_admission.rs b/core/continuum-core/src/cognition/resource_admission.rs index 212c3a83b..4b0533687 100644 --- a/core/continuum-core/src/cognition/resource_admission.rs +++ b/core/continuum-core/src/cognition/resource_admission.rs @@ -128,11 +128,30 @@ pub fn shared_model_saturated() -> bool { // ambient turn defers to a later beat with free capacity — the durable transcript is // unchanged, so nothing is lost. [[conversational-latency-is-a-misdirection-budget]] -/// How many ambient turns may run at once. 1 = strongly prioritize directed work: -/// under a burst, the addressed persona plus at most one ambient contribution run; -/// the rest yield. Under light/staggered load ambient turns are naturally serial, so -/// this never throttles a quiet room. -const AMBIENT_TURN_CONCURRENCY: usize = 1; +// How many ambient turns may run at once: [`LaneAdmission::nondirected_budget`] — the +// SAME live (lanes − 1, floored at 1) budget the per-call lane reservation below uses. +// One machine, one answer to "how much non-directed concurrency is there", derived from +// live capacity instead of declared twice. +// +// This was `const AMBIENT_TURN_CONCURRENCY: usize = 1` until 2026-08-17, and the constant +// was the roster's starvation CEILING. Two things made it invisible: +// 1. `service_loop.rs`'s self-tick gate documents this permit as "sized to the LIVE +// served lane count (LaneAdmission ← set_served_lane_count)". It never was — it was +// a bare 1, and the comment described the design that was intended. +// 2. A hard 1 is indistinguishable from a quiet room at n≈1 citizen, which is how it +// was reasoned about ("under light load ambient turns are naturally serial"). +// Measured on this box 2026-08-17: 4 served lanes, 20+ hosted citizens, so 3 non-directed +// lanes sat permanently idle while every citizen but one yielded on a pool of 1. +// +// Why lowering the bound is still safe — the directed-turn guarantee never came from THIS +// permit. It comes from the per-call reservation (`acquire_serving_lane`), which caps +// non-directed calls at lanes−1 so a directed call always finds a lane. That is the layer +// that owns lane priority, and it is untouched. This permit's own job (#171) is anti- +// STAMPEDE: bound the fan-out when N peers wake on the same beat and all read inflight=0. +// A bound of lanes−1 does that job exactly as well as a bound of 1 — it is still fixed, +// still acquired non-blockingly, still held across the whole turn — while no longer +// throttling below the hardware. On a 1- or 2-lane box the budget floors at 1, so weak +// machines get byte-identical behaviour to before. /// Try to claim an ambient-turn slot. `Some(permit)` → run the ambient turn (hold the /// permit for the turn's lifetime; it releases on drop). `None` → all ambient slots @@ -143,14 +162,22 @@ pub fn try_hold_ambient_turn() -> Option { // ── Serving-lane reservation for directed turns (#139) ────────────────────────── // -// The ambient PERMIT above bounds how many ambient TURNS run at once (1). But a single -// ambient `drive_to_settle` makes many model calls over minutes, and an idle self-tick -// is not ambient-permit-gated at all — so together they can occupy BOTH physical decode -// lanes (`llama --parallel MAX_LANES`), and an addressed (directed) question then queues -// INSIDE the serving process behind them. Glass-boxed 2026-07-15: a directed turn sat +// The ambient PERMIT above bounds how many ambient TURNS run at once (1). But ONE +// permitted turn is not one model call: a single `drive_to_settle` makes many calls over +// minutes (act → observe → act), so the permit-holder alone can occupy several physical +// decode lanes (`llama --parallel`), and an addressed (directed) question then queues +// INSIDE the serving process behind it. Glass-boxed 2026-07-15: a directed turn sat // 8+ minutes behind one 197s idle self-tick + one 213s ambient turn on the two lanes; // its latency was lane-QUEUE, not decode (a free-lane turn is ~30-60s). // +// (That glass-box predates the permit reaching the self-tick. Both non-directed paths are +// permit-gated today — `service_loop.rs` acquires at the self-tick gate AND at the +// message-ambient gate, and they share the one ambient pool. The reservation below is +// still required, because the thing it bounds is CALLS-per-turn, which no turn-level +// permit can see. Corrected 2026-08-17: this paragraph had claimed "an idle self-tick is +// not ambient-permit-gated at all", contradicting the self-tick gate's own comment block +// in the same tree — a stale premise sitting under a live design argument.) +// // Neither the gauge nor the turn-level permit can fix this — the reservation must live // at the LANE the model call actually consumes. So every model call acquires a lane // here, priced by priority: @@ -202,6 +229,7 @@ pub struct LaneAdmission { /// double-count a grow. resize_lock: Mutex<()>, ambient: std::sync::OnceLock>, + ambient_installed: AtomicUsize, } /// The process-global admission gate — ONE INSTANCE of [`LaneAdmission`], not a separate @@ -271,6 +299,7 @@ impl LaneAdmission { nondirected_installed: AtomicUsize::new(0), resize_lock: Mutex::new(()), ambient: std::sync::OnceLock::new(), + ambient_installed: AtomicUsize::new(0), } } @@ -307,6 +336,12 @@ impl LaneAdmission { lanes.saturating_sub(1).max(1), ); } + // The ambient-turn pool rides the SAME budget, so it grows on the same signal. + // Missing this is how a pool installed at the boot floor would stay there for the + // process's life while serving grew underneath it. + if let Some(sem) = self.ambient.get() { + grow_semaphore_to(sem, &self.ambient_installed, lanes.saturating_sub(1).max(1)); + } } fn serving_lanes(&self) -> &std::sync::Arc { @@ -325,11 +360,15 @@ impl LaneAdmission { }) } - /// See [`try_hold_ambient_turn`]. + /// See [`try_hold_ambient_turn`]. Lazy like its siblings, and for the same load-bearing + /// reason: it must capture the budget at FIRST USE, once serving has published a real + /// lane count — never at construction, when only the boot ceiling is known. pub fn try_hold_ambient_turn(&self) -> Option { self.ambient .get_or_init(|| { - std::sync::Arc::new(tokio::sync::Semaphore::new(AMBIENT_TURN_CONCURRENCY)) + let n = self.nondirected_budget(); + self.ambient_installed.store(n, Ordering::Release); + std::sync::Arc::new(tokio::sync::Semaphore::new(n)) }) .clone() .try_acquire_owned() @@ -656,10 +695,19 @@ mod tests { fn ambient_permit_bounds_concurrency_and_releases_on_drop() { // OUR OWN gate — no process-global, so no lock and no order dependence. let gate = LaneAdmission::new(); + // PREMISE CHANGE 2026-08-17: the pool used to be a bare `AMBIENT_TURN_CONCURRENCY + // = 1`; it is now the live `nondirected_budget()` (lanes − 1, floored at 1), the + // same budget the per-call lane reservation uses. Pin a real multi-lane machine so + // the burst has something to bound — at the old hardcoded 1 this test could not + // distinguish "correctly bounded" from "throttled below the hardware", which is + // exactly how the starvation ceiling stayed invisible. + gate.set_served_lane_count(4); + let budget = gate.nondirected_budget(); + assert_eq!(budget, 3, "4 served lanes → 3 non-directed, 1 reserved directed"); // A simultaneous-wake burst: several ambient turns try to claim a slot at once. - // Exactly AMBIENT_TURN_CONCURRENCY win; the rest get None and must yield. + // Exactly `budget` win; the rest get None and must yield. let mut held: Vec = Vec::new(); - for _ in 0..AMBIENT_TURN_CONCURRENCY { + for _ in 0..budget { held.push( gate.try_hold_ambient_turn() .expect("a free slot is grantable"), @@ -685,6 +733,60 @@ mod tests { drop(held); // release the rest (nothing else can observe this gate anyway) } + // what this catches: the WEAK-BOX floor of the same change. Deriving the ambient pool + // from lanes−1 must not regress a 1- or 2-lane machine below the behaviour it had + // under the old hardcoded 1 — a single-lane host has nothing to reserve, so its + // budget floors at 1 and it stays byte-identical. This is the half of the change that + // could silently hurt the smallest supported hardware, so it gets its own row. + #[test] + fn a_one_lane_box_keeps_exactly_one_ambient_slot() { + for lanes in [1usize, 2] { + let gate = LaneAdmission::new(); + gate.set_served_lane_count(lanes); + assert_eq!( + gate.nondirected_budget(), + 1, + "{lanes}-lane box floors the non-directed budget at 1" + ); + let held = gate + .try_hold_ambient_turn() + .expect("the one slot is grantable"); + assert!( + gate.try_hold_ambient_turn().is_none(), + "a {lanes}-lane box admits exactly ONE ambient turn — unchanged from the \ + pre-2026-08-17 hardcoded bound" + ); + drop(held); + } + } + + // what this catches: an ambient pool installed at the BOOT floor and then never grown. + // The pool is lazy on purpose (capture the real lane count at first use), but that + // makes "first use happened before serving published its count" a live possibility — + // and without the grow-on-resize wiring the whole roster would stay pinned at the boot + // budget for the life of the process while 3 lanes sat idle. Regression for the + // starvation ceiling this change removes. + #[test] + fn the_ambient_pool_grows_when_serving_publishes_more_lanes() { + let gate = LaneAdmission::new(); + gate.set_served_lane_count(1); // cold boot: one lane + let first = gate.try_hold_ambient_turn().expect("the one slot"); + assert!(gate.try_hold_ambient_turn().is_none(), "1 lane → 1 ambient slot"); + + gate.set_served_lane_count(4); // serving warms up and reports its real width + let second = gate + .try_hold_ambient_turn() + .expect("growing to 4 lanes must open a second ambient slot"); + let third = gate + .try_hold_ambient_turn() + .expect("…and a third (budget = lanes - 1)"); + assert!( + gate.try_hold_ambient_turn().is_none(), + "still bounded at lanes-1 — growth must not remove the directed reservation" + ); + drop((first, second, third)); + } + // what this catches (#139 lane starvation): a directed (addressed) turn must never // queue behind non-directed model calls. Non-directed callers are capped at // (MAX_LANES-1) lanes, so a directed caller always finds a reserved lane — this is diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index eef806a6b..63438f748 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -484,7 +484,9 @@ async fn serve_persona_loop_inner( // #385-wedging it (glass-boxed 2026-08-10: resident_personas=4 vs // warm_slots=1, every self-tick died on "no TOKEN progress for 90s"). // The permit is sized to the LIVE served lane count (LaneAdmission ← - // set_served_lane_count) and HELD across the whole self-cycle, so ambient + // set_served_lane_count — true since 2026-08-17; it was a hardcoded 1 for + // the whole time this comment claimed otherwise) and HELD across the whole + // self-cycle, so ambient // concurrency is bounded to real capacity no matter when everyone woke — // the surplus minds genuinely yield toward rest instead of stampeding. // Directed turns still bypass entirely (they were named). Self-tick and @@ -502,7 +504,7 @@ async fn serve_persona_loop_inner( // she learned nothing, and there is nothing to rest ON. // // Compounding it built a STARVATION RATCHET, measured live - // 2026-08-17 on this box: `AMBIENT_TURN_CONCURRENCY == 1` and 24 + // 2026-08-17 on this box: the ambient pool was a hardcoded 1 and 24 // hosted citizens, so ~23 yield on every beat. At 1.5× per yield // a citizen crosses 15s → the 240s `rest_cap` in ~8 yields — 16× // slower — and STAYS there, because the only two resets are a @@ -1490,7 +1492,7 @@ pub enum BeatOutcome { /// /// The bug this encodes against (measured live 2026-08-17, 24 hosted citizens): /// `YieldedNoSlot` used to share `NothingNew`'s 1.5× backoff. With -/// `AMBIENT_TURN_CONCURRENCY == 1`, ~23 citizens yield per beat, so each of them +/// the ambient pool hardcoded at 1, ~23 citizens yield per beat, so each of them /// compounded 15s → the 240s cap in ~8 yields and STAYED pinned there — the only /// resets being a successful cycle (which the backoff itself denied them) or an /// inbound message. Transient contention became permanent slowness, and it got @@ -4637,7 +4639,7 @@ mod tests { } // what this catches: the STARVATION RATCHET — a yield being charged the rest // backoff it did not earn. Regression for the live 2026-08-17 measurement (24 - // hosted citizens, AMBIENT_TURN_CONCURRENCY=1, ONE self-tick in 40 minutes on a + // hosted citizens, ambient pool of 1, ONE self-tick in 40 minutes on a // healthy decoding lane). If `YieldedNoSlot` ever compounds again, contention // silently becomes permanent slowness and the whole roster degrades as it grows. #[test] From 5d62bf30bc839efc7eb44b50167d20fc590fb213 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 13:47:18 -0500 Subject: [PATCH 05/80] =?UTF-8?q?fix(runtime):=20a=20deploy=20in=20flight?= =?UTF-8?q?=20must=20block=20implicit=20autostart=20=E2=80=94=20the=20stal?= =?UTF-8?q?e=20core=20that=20steals=20the=20socket=20mid-build?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED 2026-08-17. `continuum reboot` shipped 6229b3762 and deploy-verify reported the core running a87f7c871. The installed image on disk was confirmed to contain the NEW sha, so the build and the install were both correct. The core that answered simply was not the one the deploy launched. The mechanism is not subtle: 1. `reboot` stops the old core and runs start-server.sh, which BUILDS (292s and 530s measured here; 772s on BIGMAMA per #422) and only near the END copies the fresh artifact over ~/.continuum/bin/continuum-core-server. 2. For that whole multi-minute window, no core answers the socket AND the installed path still holds the PREVIOUS build. 3. Every `continuum ` calls `ensure_core_running`, whose entire job is to autostart when nothing answers. In that window it launches the STALE image. 4. That stale core binds the socket and answers. The freshly-built binary loses, and deploy-verify correctly reports the old sha. Any client can do this — a UI poll, `npm start`, a cron, a persona, or an operator typing `continuum ping` to see whether the reboot finished. In the observed incident the trigger was my own 120-second monitor calling `persona/roster`. That also settles the open question on #421, which has sat un-run for days: "something respawns a core within seconds of every kill — external supervisor, or me?" It was the CLI's own autostart. There is no mystery daemon. #421 can close on this. WHY THE EXISTING GUARD COULD NOT SEE IT. `core_bind_guard::decide` is a function of (ping answered, core pids running). Mid-build BOTH are false, and the honest reading of that state really is "no core is running, and starting one is safe" — true in general, wrong during a deploy. "A deploy is in flight" is a fact that lives in another process, so this is not a stricter bind guard; it is the missing observation, published by the only party that knows it. THE FIX. `runtime::deploy_claim` — reboot publishes {pid, started_ms, target_sha} for the length of its build+swap; implicit autostart and the launch arm of explicit `start` consult it and refuse with a message that names the deploying pid, the build it is shipping, and how long it has been going. `--force` overrides, consistent with the Occupied arm beside it. The claim is RAII: released on Ok, Err, `?`, and panic-unwind, because a claim that leaked past its deploy would block autostarts until its owner died. AND THE FAILURE MODE IT MUST NOT BECOME. A claim that outlives its owner would wedge the machine — exactly airc's "start GIVES UP on a contended lock" defect (#355). So the claim is advisory and self-healing: it blocks only while its owner is demonstrably alive AND younger than a 1-hour cap (far above any measured build, still bounded). A killed reboot, a hung build, and a recycled pid all decay back to Clear on their own, and the decision returns `Abandoned{why}` so the caller SWEEPS AND SAYS SO rather than silently obeying a dead file. Writing the claim is best-effort for the same reason: an unwritable advisory file must not turn a hint into an outage. The decision is pure — `decide(claim, owner_alive, now_ms)` — so the `--lib` CI gate covers every row without a filesystem or a process, the same split as `core_bind_guard` and for the same reason. Four rows, each a real case: a live deploy blocks for the whole build; a dead owner and an expired claim both decay to launchable; a future-stamped claim (clock skew) reads as new rather than underflowing to "ancient", which would have silently disabled the guard on any box with a skewed clock; and the on-disk round-trip covers idempotent clear and a corrupt file degrading to absent. 8/8 green with core_bind_guard. NOT YET POSITIVE-CONTROLLED END-TO-END, and stated plainly rather than implied: an isolated control is not reachable while a real core runs, because `running_core_pids()` is global (pgrep) and returns Occupied before the gate. The honest control is the incident itself — a `continuum` verb firing mid-build must now print the refusal instead of autostarting. The monitor that caused the incident still runs every 120s, so the next reboot exercises it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/bin/continuum.rs | 109 +++++++- .../src/runtime/deploy_claim.rs | 263 ++++++++++++++++++ core/continuum-core/src/runtime/mod.rs | 1 + 3 files changed, 372 insertions(+), 1 deletion(-) create mode 100644 core/continuum-core/src/runtime/deploy_claim.rs diff --git a/core/continuum-core/src/bin/continuum.rs b/core/continuum-core/src/bin/continuum.rs index 946fe50a8..813470228 100644 --- a/core/continuum-core/src/bin/continuum.rs +++ b/core/continuum-core/src/bin/continuum.rs @@ -508,6 +508,10 @@ async fn ensure_core_running(command: &str) -> Result<(), String> { } BindDecision::Free => {} } + // A deploy in flight is the one state the bind guard above cannot see: mid-build no + // core answers and no core pid exists, which reads as "safe to start" and is exactly + // when starting is wrong. + deploy_gate(command)?; if std::env::var("CONTINUUM_NO_AUTOSTART").is_ok_and(|v| v != "0") { return Err(format!( "no core is answering on {} and CONTINUUM_NO_AUTOSTART is set, so `{command}` \ @@ -551,7 +555,15 @@ async fn start(force: bool) -> Result<(), String> { println!("core already running (socket={socket})"); return Ok(()); } - BindDecision::Free => {} + // Only the path that actually LAUNCHES consults the deploy claim. An already-serving + // core is a no-op and must stay one — gating a no-op would turn a mid-deploy + // `continuum start` into a spurious error about a core that is already fine. + // `--force` is the documented override, consistent with the Occupied arm below. + BindDecision::Free => { + if !force { + deploy_gate("start")?; + } + } BindDecision::Occupied { pids } => { let list = pids.iter().map(|p| p.to_string()).collect::>().join(","); if !force { @@ -683,6 +695,12 @@ async fn reboot(force: bool) -> Result<(), String> { // script, and one decision belongs in exactly one place. `launch_core`'s // `wait_for_death` on `old` is then trivially satisfied on Windows and // still does the real work on Unix, where the overlapping build stands. + // Publish the claim for the WHOLE build+swap. Held until this function returns, so a + // concurrent `continuum ` refuses instead of autostarting the pre-swap installed + // image and stealing the socket (the DEPLOY MISMATCH measured 2026-08-17). + let _deploy_claim = DeployClaimGuard::take( + git_head_short_sha().as_deref().unwrap_or("unknown"), + ); let secs = launch_core(&old, LaunchSource::FromSource).await?; // Deploy-verification (#194): a new core is up — but is it the FRESHLY-BUILT one? If // start-server.sh's build was a stale cache no-op or silently failed, an OLD binary would @@ -844,6 +862,95 @@ fn core_artifact_candidates(home: &str, cargo_target_dir: Option<&str>) -> Vec

Result { + Ok(PathBuf::from(home_dir()?).join(".continuum")) +} + +fn now_ms() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_millis() as u64) + .unwrap_or(0) +} + +/// Consult the deploy claim before minting a core, and REPORT what it found. +/// +/// Returns `Err` only while a deploy is genuinely in flight. See +/// [`continuum_core::runtime::deploy_claim`] for the incident: mid-build there is no core +/// answering AND the installed image is still the PREVIOUS build, so any autostart in that +/// window launches stale code, wins the socket, and defeats the deploy. +/// +/// A claim whose owner is gone or that has aged out is swept and announced rather than +/// silently obeyed — a claim must never be able to wedge the machine (#355's failure mode). +fn deploy_gate(verb: &str) -> Result<(), String> { + use continuum_core::runtime::deploy_claim::{self, DeployGate}; + let Ok(root) = continuum_root() else { + return Ok(()); // no HOME → no claim file → nothing to honour + }; + let claim = deploy_claim::read(&root); + let alive = claim.as_ref().is_some_and(|c| pid_alive(c.pid)); + match deploy_claim::decide(claim.as_ref(), alive, now_ms()) { + DeployGate::Clear => Ok(()), + DeployGate::Abandoned { pid, age_ms, why } => { + eprintln!( + "⚠ sweeping an abandoned deploy claim (pid {pid}, {}s old, {why:?}) — \ + proceeding with `{verb}`", + age_ms / 1000 + ); + let _ = deploy_claim::clear(&root); + Ok(()) + } + DeployGate::InProgress { pid, age_ms, target_sha } => Err(format!( + "a deploy is in flight (pid {pid} shipping build {target_sha}, {}s in) and no core \ + is answering yet. Starting one now would launch the PRE-SWAP installed binary, \ + which would then hold the socket and make the deploy report the OLD build — \ + measured 2026-08-17. Wait for the deploy to finish; `{verb}` works the moment it \ + does. (If that deploy is dead, its claim is swept automatically once its process \ + exits.)", + age_ms / 1000 + )), + } +} + +/// RAII deploy claim: published for the length of a swap, released on EVERY exit path +/// (Ok, Err, `?`, panic-unwind). A claim that leaked past its deploy would block autostarts +/// until its owner died, so the release cannot be a line at the end of the happy path. +struct DeployClaimGuard { + root: PathBuf, +} + +impl DeployClaimGuard { + /// Best-effort by design: if the claim cannot be written the deploy still proceeds — + /// losing the guard degrades to the old behaviour (which `deploy-verify` still catches), + /// whereas refusing to deploy over an unwritable advisory file turns a hint into an outage. + fn take(target_sha: &str) -> Option { + use continuum_core::runtime::deploy_claim::{self, DeployClaim}; + let root = continuum_root().ok()?; + let claim = DeployClaim { + pid: std::process::id() as i32, + started_ms: now_ms(), + target_sha: target_sha.to_string(), + }; + match deploy_claim::write(&root, &claim) { + Ok(()) => Some(Self { root }), + Err(e) => { + eprintln!( + "⚠ could not publish a deploy claim ({e}) — a concurrent command could \ + autostart a stale core during this build; deploy-verify still catches it" + ); + None + } + } + } +} + +impl Drop for DeployClaimGuard { + fn drop(&mut self) { + let _ = continuum_core::runtime::deploy_claim::clear(&self.root); + } +} + /// The user's home dir — `HOME` (unix) or `USERPROFILE` (Windows). Loud when absent: the /// resolution order depends on it, and guessing would defeat the shared contract. fn home_dir() -> Result { diff --git a/core/continuum-core/src/runtime/deploy_claim.rs b/core/continuum-core/src/runtime/deploy_claim.rs new file mode 100644 index 000000000..87a4e80de --- /dev/null +++ b/core/continuum-core/src/runtime/deploy_claim.rs @@ -0,0 +1,263 @@ +//! The deploy claim — "a swap is in flight; do NOT mint a core from the installed image." +//! +//! # The incident this exists for (2026-08-17, measured on this box) +//! +//! `continuum reboot` reported `DEPLOY MISMATCH (#194)`: it shipped `6229b3762`, the core +//! answering was `a87f7c871`. The installed image on disk was verified to contain the NEW +//! sha, so the build and the install were both correct. The core that answered simply was +//! not the one the deploy launched. +//! +//! The mechanism, and it is not a subtle race: +//! +//! 1. `reboot` stops the old core and runs `start-server.sh`, which BUILDS (measured 292s +//! and 530s on this box; #422 records 772s on another) and only near the END copies the +//! fresh artifact over `~/.continuum/bin/continuum-core-server`. +//! 2. For that entire multi-minute window, no core answers the socket AND the installed +//! path still holds the PREVIOUS build. +//! 3. Every `continuum ` calls `ensure_core_running`, whose whole job is to autostart +//! a core when none answers. In that window it launches the STALE installed image. +//! 4. That stale core binds the socket and answers. The deploy's own freshly-built binary +//! then loses, and `deploy-verify` correctly reports the old sha. +//! +//! Any client command can do this — a UI poll, `npm start`, a cron, a persona, an operator +//! typing `continuum ping` to see whether the reboot finished. In the observed incident the +//! trigger was a 120-second monitor loop running `persona/roster`. That also positively +//! settles the open question on task #421 ("something respawns a core within seconds of +//! every kill — external supervisor, or me?"): **it was the CLI's own autostart.** There is +//! no mystery daemon. +//! +//! # Why the existing guards could not catch it +//! +//! [`crate::runtime::core_bind_guard::decide`] is a function of (ping answered, core pids +//! running). Mid-build BOTH are false — the honest reading of that state really is "no core +//! is running, and starting one is safe." It is safe in general; it is wrong DURING A +//! DEPLOY, and "a deploy is in flight" is a fact the bind guard cannot observe because it +//! lives in another process. So this is not a stricter bind guard — it is the missing +//! observation, published by the only party that knows it. +//! +//! # The shape, and the failure mode it must not become +//! +//! A claim that outlives its owner would wedge the machine permanently — exactly the +//! `airc daemon start GIVES UP on a contended lock` defect (#355). So the claim is +//! ADVISORY and self-healing: it blocks only while its owner process is demonstrably +//! alive AND the claim is younger than [`CLAIM_MAX_AGE_MS`]. A killed `reboot`, a reused +//! pid, or a hung build all decay back to Clear on their own, and the decision says which +//! so the caller can report it instead of silently ignoring a file. +//! +//! The decision is a pure function of (claim, owner_alive, now) so the `--lib` CI gate +//! covers every row without a filesystem or a process — the same split as +//! [`crate::runtime::core_bind_guard`], for the same reason. + +use serde::{Deserialize, Serialize}; +use std::path::{Path, PathBuf}; + +/// How long a claim may block before it is treated as abandoned. +/// +/// It must comfortably exceed a COLD full build, because blocking is the correct behaviour +/// for the whole of one: measured 292s and 530s here, 772s on BIGMAMA (#422). One hour is +/// far above any of those and still bounded, so the worst case of a `kill -9`'d reboot on a +/// machine that later reuses its pid is a one-hour degradation, never a permanent wedge. +pub const CLAIM_MAX_AGE_MS: u64 = 60 * 60 * 1000; + +/// What one deploying process published about itself. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct DeployClaim { + /// The pid of the process performing the deploy (the `continuum reboot` invocation). + pub pid: i32, + /// Epoch-ms when the claim was taken. Ages the claim; see [`CLAIM_MAX_AGE_MS`]. + pub started_ms: u64, + /// The build the deploy intends to ship, for a legible refusal message. + pub target_sha: String, +} + +/// The gate an implicit launcher must pass. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DeployGate { + /// No live deploy. Launching is as safe as the bind guard says it is. + Clear, + /// A deploy is genuinely in flight — launching now would mint a core from the + /// pre-swap installed image and defeat it. + InProgress { + pid: i32, + age_ms: u64, + target_sha: String, + }, + /// A claim exists but no longer binds (owner gone, or older than the cap). Treated as + /// Clear by every caller — but named so the caller can SAY it swept a stale claim + /// rather than pretending the file was never there. + Abandoned { pid: i32, age_ms: u64, why: AbandonReason }, +} + +/// Why an existing claim stopped binding. Kept as data so the message names the cause. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum AbandonReason { + /// The deploying process is gone (crashed, killed, or simply finished without + /// clearing — the RAII release should prevent the last one, but never assume it). + OwnerDead, + /// Older than [`CLAIM_MAX_AGE_MS`]. Covers a hung build and a recycled pid. + Expired, +} + +impl DeployGate { + /// True only when a launcher must refuse. `Abandoned` deliberately does NOT block. + pub fn blocks(&self) -> bool { + matches!(self, DeployGate::InProgress { .. }) + } +} + +/// The whole policy, pure and total. Every row is a real case, not a defensive branch. +pub fn decide(claim: Option<&DeployClaim>, owner_alive: bool, now_ms: u64) -> DeployGate { + let Some(claim) = claim else { + return DeployGate::Clear; + }; + // Saturating: a claim stamped in the future (clock skew, a restored snapshot) reads as + // age 0 and therefore blocks for its full window rather than being instantly expired. + // Blocking a little too long is recoverable; launching a stale core is the bug. + let age_ms = now_ms.saturating_sub(claim.started_ms); + if !owner_alive { + return DeployGate::Abandoned { + pid: claim.pid, + age_ms, + why: AbandonReason::OwnerDead, + }; + } + if age_ms >= CLAIM_MAX_AGE_MS { + return DeployGate::Abandoned { + pid: claim.pid, + age_ms, + why: AbandonReason::Expired, + }; + } + DeployGate::InProgress { + pid: claim.pid, + age_ms, + target_sha: claim.target_sha.clone(), + } +} + +/// Where the claim lives, given the continuum root (`~/.continuum`). +pub fn claim_path(root: &Path) -> PathBuf { + root.join("run").join("deploy.claim") +} + +/// Publish a claim. Best-effort by design: if the file cannot be written the deploy still +/// proceeds — losing the guard degrades to today's behaviour (a possible stale autostart, +/// which `deploy-verify` still catches), whereas failing the deploy over an unwritable +/// advisory file would turn a hint into an outage. +pub fn write(root: &Path, claim: &DeployClaim) -> std::io::Result<()> { + let path = claim_path(root); + if let Some(dir) = path.parent() { + std::fs::create_dir_all(dir)?; + } + let body = serde_json::to_string_pretty(claim) + .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?; + // Write-then-rename so a reader never observes a half-written claim. + let tmp = path.with_extension(format!("claim.tmp.{}", claim.pid)); + std::fs::write(&tmp, body)?; + std::fs::rename(&tmp, &path) +} + +/// Read the current claim, if any. A malformed file reads as None: an unparseable advisory +/// note must not block launches forever, and the next deploy overwrites it. +pub fn read(root: &Path) -> Option { + let body = std::fs::read_to_string(claim_path(root)).ok()?; + serde_json::from_str(&body).ok() +} + +/// Drop the claim. Idempotent; a missing file is success. +pub fn clear(root: &Path) -> std::io::Result<()> { + match std::fs::remove_file(claim_path(root)) { + Ok(()) => Ok(()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(e) => Err(e), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn claim(pid: i32, started_ms: u64) -> DeployClaim { + DeployClaim { + pid, + started_ms, + target_sha: "deadbeef".into(), + } + } + + // what this catches: THE regression — an implicit autostart minting a core from the + // pre-swap installed image while a deploy is mid-build. Measured 2026-08-17: a 120s + // monitor loop autostarted the OLD binary during a 530s build, it won the socket, and + // `deploy-verify` reported build a87f7c871 for a deploy that shipped 6229b3762. + #[test] + fn a_live_deploy_blocks_an_implicit_launch_for_the_whole_build() { + // Well inside a long build: still blocking. + let g = decide(Some(&claim(4242, 1_000)), true, 1_000 + 530_000); + assert!(g.blocks(), "a 530s-old live deploy must still block: {g:?}"); + match g { + DeployGate::InProgress { pid, target_sha, .. } => { + assert_eq!(pid, 4242); + assert_eq!(target_sha, "deadbeef", "the refusal names the build it protects"); + } + other => panic!("expected InProgress, got {other:?}"), + } + // No claim at all is the overwhelmingly common case and must be free. + assert_eq!(decide(None, false, 9_999), DeployGate::Clear); + } + + // what this catches: the OPPOSITE failure — a claim that outlives its owner wedging the + // machine, which is the airc `start gives up on a contended lock` defect (#355). A + // killed reboot, a hung build, and a recycled pid must ALL decay back to launchable. + #[test] + fn an_abandoned_claim_never_wedges_the_machine() { + // Owner died mid-deploy (kill -9, crash, power loss). + let dead = decide(Some(&claim(4242, 1_000)), false, 2_000); + assert!(!dead.blocks(), "a dead owner must not block: {dead:?}"); + assert!(matches!( + dead, + DeployGate::Abandoned { why: AbandonReason::OwnerDead, pid: 4242, .. } + )); + + // Owner alive but the claim is older than any real build — hung, or a reused pid. + let old = decide(Some(&claim(4242, 0)), true, CLAIM_MAX_AGE_MS); + assert!(!old.blocks(), "an expired claim must not block: {old:?}"); + assert!(matches!( + old, + DeployGate::Abandoned { why: AbandonReason::Expired, .. } + )); + + // One millisecond under the cap still blocks — the boundary is not off by one. + assert!(decide(Some(&claim(4242, 0)), true, CLAIM_MAX_AGE_MS - 1).blocks()); + } + + // what this catches: clock skew making a claim instantly "expired". A claim stamped in + // the future must read as brand new (block), not as maximally old (launch anyway) — + // an underflow here would silently disable the guard on any box with a skewed clock. + #[test] + fn a_future_stamped_claim_reads_as_new_not_as_ancient() { + let g = decide(Some(&claim(7, 10_000)), true, 1_000); + assert!(g.blocks(), "future-stamped claim must still block: {g:?}"); + } + + // what this catches: round-trip through the real files, including that clear() is + // idempotent (the RAII release runs on paths where the claim may already be gone) and + // that a corrupt file cannot block launches forever. + #[test] + fn claims_round_trip_and_degrade_safely_on_disk() { + let dir = std::env::temp_dir().join(format!("continuum-claim-test-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + + assert_eq!(read(&dir), None, "no claim on a fresh root"); + let c = claim(1234, 5_000); + write(&dir, &c).expect("write a claim"); + assert_eq!(read(&dir).as_ref(), Some(&c), "claims round-trip verbatim"); + + std::fs::write(claim_path(&dir), "{not json").expect("corrupt the claim"); + assert_eq!(read(&dir), None, "a corrupt claim reads as absent, never as a block"); + + clear(&dir).expect("clear"); + clear(&dir).expect("clear is idempotent — the release path may run twice"); + assert_eq!(read(&dir), None); + let _ = std::fs::remove_dir_all(&dir); + } +} diff --git a/core/continuum-core/src/runtime/mod.rs b/core/continuum-core/src/runtime/mod.rs index a2b535563..e23811205 100644 --- a/core/continuum-core/src/runtime/mod.rs +++ b/core/continuum-core/src/runtime/mod.rs @@ -37,6 +37,7 @@ pub mod command_executor; pub mod command_interceptor; pub mod control; pub mod core_bind_guard; +pub mod deploy_claim; pub mod core_ipc_transport; pub mod deploy_provenance; pub mod daemon; From be5cd1138efa47c2d3f587508932f5cac195b9b5 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 14:01:12 -0500 Subject: [PATCH 06/80] =?UTF-8?q?fix(deploy):=20=E2=9A=A0=20STALE=20CLI=20?= =?UTF-8?q?fired=20on=20every=20SUCCESSFUL=20reboot=20=E2=80=94=20and=20it?= =?UTF-8?q?s=20text=20was=20false?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `reboot` rebuilds AND reinstalls the CLI (#422 / PR #2293). But the invocation doing the reinstall is by construction the one it REPLACES — a running image cannot be the artifact it just installed. So `cli_staleness_note`, which compares the RUNNING `continuum` against git HEAD, diverged on every healthy deploy and printed ⚠ STALE CLI. Observed three times in a row today on deploys that were otherwise perfect, including one whose core verdict was a clean ✅. Its text then said: "`reboot` rebuilds the CORE and never the CLI." That has been false since #2293 — verified on this box today, where the installed CLI advanced a87f7c871 → 6229b3762 → 5d62bf30b across three reboots, each time by reboot's own install step. A warning that fires on every success is not a warning. It is noise that trains the operator to skip the exact line where a REAL stale CLI would appear — the same class of defect as #422's own "⚠ …Please report this" that nobody reported, and as airc's `doctor --fix` reporting ok while delivery was 100% down (#348). Fix: `cli_staleness_note` takes `rebuilt_this_run`, so the two situations stop sharing a message. • self-replacing run (reboot on a machine where `cli_self_build` says Rebuild) → "↻ CLI updated: this invocation is build X, the NEXT one runs Y." Information, not a fault, and it still says the useful part out loud: CLI-side behaviour in THIS run is the old build, so re-run a lifecycle verb whose fix lives in the CLI. • nothing replaced it (a bare `deploy-verify`, or a platform where the self-build skips) → the ⚠, with the false "never rebuilds the CLI" sentence replaced by what is actually true: reboot does rebuild it, except where cli_self_build skips the platform. `reboot` computes the flag from the two terms that actually decide it — a start script exists (the same condition `plan_launch` uses to choose Script for a FromSource launch) AND `cli_self_build(OS) == Rebuild`. Both matter: an installed node with no checkout rebuilt nothing, and Windows deliberately skips. Getting either wrong re-creates the noise or, worse, hides a genuinely stale CLI behind a reassuring handoff line. Test pins both readings apart, and that a MATCHING CLI stays silent in both modes so the flag cannot invent a note. It also asserts the corrected wording no longer contains "never" — the specific false claim. 6/6 green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/bin/continuum.rs | 21 ++++-- .../src/runtime/deploy_provenance.rs | 64 ++++++++++++++++--- 2 files changed, 73 insertions(+), 12 deletions(-) diff --git a/core/continuum-core/src/bin/continuum.rs b/core/continuum-core/src/bin/continuum.rs index 813470228..699396460 100644 --- a/core/continuum-core/src/bin/continuum.rs +++ b/core/continuum-core/src/bin/continuum.rs @@ -105,7 +105,7 @@ async fn run() -> Result<(), String> { } // Standalone #194 check: prove the RUNNING core is built from current HEAD, // without a full reboot. Prints "✅ deploy verified" or fails loud on mismatch. - "deploy-verify" | "verify" => verify_deployed_build().await, + "deploy-verify" | "verify" => verify_deployed_build(false).await, // Anything else is a command name. `--help`/`-h` renders the manual in the // CLI's paradigm (bash flags), adapted from the SAME schema the AI gets as // a tool spec. Otherwise dispatch, params adapted procedurally. @@ -711,7 +711,15 @@ async fn reboot(force: bool) -> Result<(), String> { println!( "core answering (socket={socket}) after ~{secs}s — verifying deploy provenance (#194)" ); - verify_deployed_build().await + // Did THIS reboot replace the installed CLI? Only when it went through the build script + // (a source tree exists — the same condition `plan_launch` uses to pick `Script` for a + // FromSource launch) AND the platform allows a self-build. Both terms matter: on an + // installed node with no checkout nothing was rebuilt, and on Windows `cli_self_build` + // deliberately skips. Getting this wrong in either direction re-creates the noise this + // flag exists to remove, or hides a genuinely stale CLI behind a reassuring handoff line. + let rebuilt_cli = locate_start_script().is_ok() + && matches!(cli_self_build(std::env::consts::OS), CliSelfBuild::Rebuild); + verify_deployed_build(rebuilt_cli).await } /// Prove the running core is built from the source this deploy shipped — the honest half of @@ -737,7 +745,12 @@ async fn reboot(force: bool) -> Result<(), String> { /// describing the binary you are RUNNING rather than one found on disk. const CLI_BUILD_SHA: &str = env!("CONTINUUM_BUILD_GIT_SHA"); -async fn verify_deployed_build() -> Result<(), String> { +/// `rebuilt_cli` says whether THIS invocation replaced the installed CLI — true from +/// `reboot` (start-server.sh rebuilds + reinstalls it unless `cli_self_build` skips the +/// platform), false from a bare `deploy-verify`. It is what lets the CLI-provenance note +/// tell a HANDOFF ("the next run gets the new CLI") apart from real STALENESS, instead of +/// warning on every successful deploy. +async fn verify_deployed_build(rebuilt_cli: bool) -> Result<(), String> { let socket = socket_path(); // The RUNNING core's provenance, from the process itself. let reply = connection() @@ -763,7 +776,7 @@ async fn verify_deployed_build() -> Result<(), String> { let running_desc = describe_running_core(&socket); // The CLI's own provenance rides alongside the core's, on BOTH outcomes: a stale CLI // is relevant whether or not the core swap took. - let cli_note = cli_staleness_note(CLI_BUILD_SHA, &expected, &expected_source); + let cli_note = cli_staleness_note(CLI_BUILD_SHA, &expected, &expected_source, rebuilt_cli); match deploy_verdict( actual.as_deref(), &expected, diff --git a/core/continuum-core/src/runtime/deploy_provenance.rs b/core/continuum-core/src/runtime/deploy_provenance.rs index 404157206..fe1d732c6 100644 --- a/core/continuum-core/src/runtime/deploy_provenance.rs +++ b/core/continuum-core/src/runtime/deploy_provenance.rs @@ -90,10 +90,21 @@ pub fn deploy_verdict( /// and making it an error would break deploy-verify for every operator whose CLI predates /// their core — today, all of them. Loud and non-blocking kills the silent case without /// turning a true green into a red. See #422 for fixing the rebuild itself. +/// `rebuilt_this_run` is what makes this note honest, and it was missing until 2026-08-17. +/// `reboot` rebuilds AND reinstalls the CLI (#422/PR #2293), but the invocation doing the +/// rebuild is by construction the one it REPLACES — a running image cannot be the artifact +/// it just installed. So a successful `reboot` always diverged here and always printed +/// "⚠ STALE CLI", whose text additionally said "`reboot` rebuilds the CORE and never the +/// CLI" — false since #2293. A warning that fires on every success is not a warning; it is +/// noise that trains the operator to skip the line where a REAL stale CLI would appear. +/// With the flag, a self-replacing run reports the handoff as information, and the ⚠ is +/// reserved for the case that actually means something: a CLI that diverged and is NOT +/// being replaced (a bare `deploy-verify`, or a platform where `cli_self_build` skipped). pub fn cli_staleness_note( cli_sha: &str, expected: &str, expected_source: &str, + rebuilt_this_run: bool, ) -> Option { // Unjudgeable provenance is reported, not swallowed — but briefly, because a binary // built outside a git tree is a legitimate state, unlike a core with no SHA at all. @@ -109,12 +120,21 @@ pub fn cli_staleness_note( if sha_matches(cli_sha, expected) { return None; } + if rebuilt_this_run { + return Some(format!( + "↻ CLI updated (#422): this invocation is build {cli_sha} — it predates the CLI it \ + just installed, which is expected and not a fault. The NEXT `continuum` runs \ + {expected} ({expected_source}). CLI-side behaviour in THIS run is still the old \ + build, so re-run any lifecycle verb whose fix lives in the CLI." + )); + } Some(format!( "⚠ STALE CLI (#422): the core verdict above stands, but the `continuum` you just ran is \ - build {cli_sha}, not {expected} ({expected_source}). `reboot` rebuilds the CORE and never \ - the CLI, so any lifecycle fix that lives in the CLI — `start`/`stop`/`reboot`/\ - `deploy-verify` itself — is NOT deployed on this machine. Rebuild and reinstall the CLI \ - before trusting CLI-side behaviour." + build {cli_sha}, not {expected} ({expected_source}), and nothing in this run replaced \ + it. Any lifecycle fix that lives in the CLI — `start`/`stop`/`reboot`/`deploy-verify` \ + itself — is NOT deployed on this machine. `continuum reboot` rebuilds and reinstalls \ + it (except where cli_self_build skips the platform); do that before trusting CLI-side \ + behaviour." )) } @@ -238,30 +258,58 @@ mod tests { #[test] fn a_stale_cli_is_reported_and_a_fresh_one_is_silent() { assert_eq!( - cli_staleness_note("abc123f", "abc123f", "git HEAD of this checkout"), + cli_staleness_note("abc123f", "abc123f", "git HEAD of this checkout", false), None, "a CLI that matches the deploy says nothing" ); assert_eq!( - cli_staleness_note("abc123f00d", "abc123f", "src"), + cli_staleness_note("abc123f00d", "abc123f", "src", false), None, "prefix-tolerant, same as the core verdict — abbreviation drift is not staleness" ); - let note = cli_staleness_note("dead111", "beef222", "git HEAD of this checkout") + let note = cli_staleness_note("dead111", "beef222", "git HEAD of this checkout", false) .expect("a diverged CLI must be reported"); for needle in ["dead111", "beef222", "#422", "STALE CLI"] { assert!(note.contains(needle), "note names {needle}: {note}"); } } + // what this catches: a warning that fires on every SUCCESS. `reboot` reinstalls the CLI, + // so the invocation doing the reinstall always diverges from HEAD — before 2026-08-17 + // that printed "⚠ STALE CLI" on every healthy deploy, with text claiming reboot "never" + // rebuilds the CLI (false since #2293). Both readings must stay distinguishable: a + // self-replacing run reports a HANDOFF, a non-replacing run reports STALENESS. + #[test] + fn a_self_replacing_run_reports_a_handoff_not_staleness() { + let handoff = cli_staleness_note("old1234", "new5678", "git HEAD of this checkout", true) + .expect("the handoff is still worth stating — this run's CLI is the old one"); + assert!( + !handoff.contains("STALE CLI"), + "a reboot that reinstalls the CLI must not cry stale at itself: {handoff}" + ); + assert!(handoff.contains("new5678"), "it names what the NEXT run gets: {handoff}"); + + let stale = cli_staleness_note("old1234", "new5678", "git HEAD of this checkout", false) + .expect("a diverged CLI nobody replaced is the real warning"); + assert!(stale.contains("STALE CLI"), "got {stale}"); + assert!( + !stale.contains("never"), + "the corrected text must not repeat the false 'reboot never rebuilds the CLI': {stale}" + ); + + // A MATCHING cli stays silent in both modes — the flag must not invent a note. + assert_eq!(cli_staleness_note("abc123f", "abc123f", "src", true), None); + assert_eq!(cli_staleness_note("abc123f", "abc123f", "src", false), None); + } + // what this catches: swallowing the unjudgeable case. A CLI built outside a git tree // reports "unknown"; silence there would be a soft skip of exactly the check this // exists to be — and it must still never assert staleness it cannot prove. #[test] fn unverifiable_cli_provenance_is_stated_not_swallowed() { for (cli, expected) in [("unknown", "beef222"), ("dead111", "unknown"), ("", "beef222")] { - let note = cli_staleness_note(cli, expected, "src") + let note = cli_staleness_note(cli, expected, "src", false) .unwrap_or_else(|| panic!("cli={cli} expected={expected} must report")); assert!(note.contains("unverifiable"), "got {note}"); assert!( From 67fbfbfbcf37c4fb7e2e77d3ba60dd1f5064fb65 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 14:10:27 -0500 Subject: [PATCH 07/80] =?UTF-8?q?docs(observations):=20dated=20post-run=20?= =?UTF-8?q?card=20=E2=80=94=20the=20two=20starvation=20defects,=20the=20tw?= =?UTF-8?q?o=20install=20defects,=20and=20what=20is=20NOT=20yet=20measured?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Standing instruction (Joel, 2026-08-17): a new well-labeled dated observation file after every run, and stop reading the old ones. Written from this session's probes and deploy receipts only; no prior cards consulted. Records the honest state including the gaps: the ambient-pool fix has had no uninterrupted window yet (three reboots for the install fixes), and the deploy claim is unit-green but not positive-controlled end-to-end. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- ...2026-08-17T190900Z-slowness-and-install.md | 69 +++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 docs/observations/2026-08-17T190900Z-slowness-and-install.md diff --git a/docs/observations/2026-08-17T190900Z-slowness-and-install.md b/docs/observations/2026-08-17T190900Z-slowness-and-install.md new file mode 100644 index 000000000..2e4470f74 --- /dev/null +++ b/docs/observations/2026-08-17T190900Z-slowness-and-install.md @@ -0,0 +1,69 @@ +# 2026-08-17 19:09Z — persona starvation + the deploy install path + +Method note: written after the run, from probes and deploy receipts taken in this session +only. **No prior observation cards were consulted** (standing instruction, 2026-08-17). + +## What was measured BEFORE any fix + +| Fact | Value | How | +|---|---|---| +| self-ticks across the whole roster | **1 in 40 minutes** (24 hosted citizens) | probe `persona.selftick.*` | +| serving lane during that window | healthy, decoding ~17 tok/s continuously | `/slots`, `serving/status` | +| ambient-turn pool | **1** (hardcoded) | `resource_admission.rs` | +| served lanes | **4** | `serving/status` | + +So 3 of 4 non-directed lanes sat permanently idle while 23 of 24 citizens yielded per beat. + +## Two defects, both fixed + +1. **Starvation ratchet** (`a87f7c871`). A citizen who YIELDED on the ambient permit — never + ran, learned nothing — was charged the same 1.5×-toward-the-240s-cap backoff as one who + ran a full cycle and found nothing new. ~8 yields pins her at 16× slower, permanently, + and it deepens as the roster grows. Fix: a yield leaves the beat unchanged. Extracted as + pure `next_beat_after(BeatOutcome, ..)`; regression test positive-controlled. +2. **The pool ceiling** (`6229b3762`). `AMBIENT_TURN_CONCURRENCY = 1` was a bare constant, + while the self-tick gate's own comment claimed it was "sized to the LIVE served lane + count". It never was. Now derived from `nondirected_budget()` (lanes−1, floored at 1) — + the same budget the per-call lane reservation already uses. 1- and 2-lane boxes are + byte-identical to before. + +## Measured AFTER the ratchet fix (pool still 1) + +Stable window 18:06–18:18 on `a87f7c871`, 24 hosted citizens: **2 self-ticks in 12 min** +(0.17/min) versus the 0.025/min baseline — ~8×. + +## NOT YET CLEANLY MEASURED + +The ambient-pool build has had **no uninterrupted window**: three reboots since (18:47, +19:01) for the install fixes below, so every counter across 18:34–19:09 is contaminated by +citizens being down. A fresh watermark is running now. Do not quote a pool-fix number until +it accumulates. + +## Install-path defects found while deploying the above + +3. **Autostart steals the socket mid-deploy** (`5d62bf30b`). `reboot` builds for 292–530s; + throughout that window no core answers AND `~/.continuum/bin/continuum-core-server` still + holds the PREVIOUS build. Every `continuum ` calls `ensure_core_running`, which + autostarts — from the stale image, which then wins the socket. Reproduced: deploy shipped + `6229b3762`, verify reported `a87f7c871`, installed image on disk confirmed to contain the + NEW sha. The trigger was my own 120s monitor calling `persona/roster`. + **This settles task #421** ("something respawns a core within seconds of every kill — + external supervisor, or me?"): it was the CLI's own autostart. No mystery daemon. + Fix: `runtime::deploy_claim` — reboot publishes {pid, started_ms, target_sha} for the + build+swap; autostart and the launch arm of `start` refuse while it is live. Advisory and + self-healing (dead owner or >1h → swept and announced) so it can never wedge the box. +4. **⚠ STALE CLI fired on every SUCCESSFUL reboot** (`be5cd1138`). The note compared the + RUNNING invocation, which by construction predates the CLI it just installed. Its text + also claimed reboot "never" rebuilds the CLI — false since #2293, disproven on this box by + the CLI advancing a87f7c871 → 6229b3762 → 5d62bf30b → be5cd1138 across four reboots. + Fix: `rebuilt_this_run` splits handoff (↻) from real staleness (⚠). + +## Still open + +- Pool-fix effect: unmeasured (above). +- The deploy claim is unit-green but **not positive-controlled end-to-end** — an isolated + control is unreachable while a real core runs, because `running_core_pids()` is global and + returns Occupied before the gate. The next reboot's mid-build monitor call is the control. +- Citizens are served by `qwen2.5-coder-14b` while `Qwen3.8-27B` — #1 on the Artificial + Analysis Agentic Index at 51, above Opus 4.8's 49 — is already loaded and ready on the + vision sidecar (:58091). That is task #440 and it is on the critical path, not a side lane. From 3375ad6840ec39b0331497a54fe13ebe85807cc2 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 14:15:28 -0500 Subject: [PATCH 08/80] =?UTF-8?q?docs(observations):=20CORRECTION=20?= =?UTF-8?q?=E2=80=94=20the=2027B=20is=20already=20the=20persona=20model,?= =?UTF-8?q?=20and=20the=20ambient-pool=20fix=20is=20inert=20at=20lanes=3D1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-measured instead of trusting my 18:07 snapshot. active_model IS Qwen3.8-27B on the persona lane; #440's bring-up landed. And nondirected_budget() = lanes-1 floored at 1, so at lanes=1 the new pool equals the old hardcoded constant — the fix is correct but buys nothing in this configuration. The remaining ceiling is the LANE count, and serving.plan says why: a 27B at a 16,384 per-slot floor fits one warm slot (resident_personas=4, warm_slots=1, without_warm_slot=3). That is an allocation policy question, not a bug. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../2026-08-17T175004Z-serving-slowness.md | 20 +++++++++ ...2026-08-17T190900Z-slowness-and-install.md | 42 +++++++++++++++++++ 2 files changed, 62 insertions(+) create mode 100644 docs/observations/2026-08-17T175004Z-serving-slowness.md diff --git a/docs/observations/2026-08-17T175004Z-serving-slowness.md b/docs/observations/2026-08-17T175004Z-serving-slowness.md new file mode 100644 index 000000000..90f3a0465 --- /dev/null +++ b/docs/observations/2026-08-17T175004Z-serving-slowness.md @@ -0,0 +1,20 @@ +# Observation — serving slowness / citizen starvation + +- **Captured (UTC):** 2026-08-17T17:50:04Z +- **Core build:** 08ccd6139 +- **Method:** live probes + process table. NO prior cards consulted. + +## Lane process +``` +16152 /Users/joel/.continuum/bin/llama-server -m /Users/joel/.cache/huggingface/hub/models--ggml-org--Qwen3.8-27B-GGUF/snapshots/0669b98607d47046c7c2b3f801011d54a08cfccf/Qwen3.8-27B-Q4_K_M.gguf --alia +``` + +## Lane uptime (how long since it last restarted) +``` +pid 16152 elapsed=16:53 +``` + +## Slot state +``` +task=8034 processing=True prefill=384/506 decoded=123 n_ctx=32512 +``` diff --git a/docs/observations/2026-08-17T190900Z-slowness-and-install.md b/docs/observations/2026-08-17T190900Z-slowness-and-install.md index 2e4470f74..4c2ce63ae 100644 --- a/docs/observations/2026-08-17T190900Z-slowness-and-install.md +++ b/docs/observations/2026-08-17T190900Z-slowness-and-install.md @@ -67,3 +67,45 @@ it accumulates. - Citizens are served by `qwen2.5-coder-14b` while `Qwen3.8-27B` — #1 on the Artificial Analysis Agentic Index at 51, above Opus 4.8's 49 — is already loaded and ready on the vision sidecar (:58091). That is task #440 and it is on the critical path, not a side lane. + +--- + +## CORRECTION, 19:15Z — two claims I made an hour earlier are now wrong + +Both came from reading the 18:07 `serving/status` snapshot and not re-reading it after the +reboots. Re-measured directly: + +``` +active_model: ggml-org/Qwen3.8-27B-GGUF lanes: 1 served_context_window: 26368 +serving.plan: decision=warm-slot-oversubscribed per_slot_floor=16384 + resident_personas=4 warm_slots=1 without_warm_slot=3 +``` + +**1. The 27B swap already happened.** The PERSONA lane is serving `Qwen3.8-27B` right now — +`base_url` and `vision_base_url` are the same `:58057`. My statement that "citizens are +served by qwen2.5-coder-14b while the 27B sits on the vision sidecar" was true at 18:07 and +is false now. #440's bring-up has landed. Nothing to swap. + +**2. The ambient-pool fix (`6229b3762`) is INERT in this configuration.** The pool derives +from `nondirected_budget() = lanes−1, floored at 1`. At `lanes: 1` that is **1** — exactly +the hardcoded constant it replaced. So it buys nothing while the 27B is resident. The fix is +still correct (it removes a constant that contradicted its own doc, and it will pay out the +moment the box runs >1 lane), but it did NOT relieve the current starvation and I should not +have implied it would. + +**What the remaining ceiling actually is:** the LANE count, and the planner states the +reason plainly — a 27B at a 16,384 per-slot floor fits ONE warm slot on this box. +4 resident personas, 1 warm slot, 3 without. Ambient concurrency is therefore structurally 1 +regardless of the permit, and that is a resource fact, not a bug. + +The ratchet fix (`a87f7c871`) is unaffected by this and still load-bearing: it stops a +citizen who loses that single slot from being permanently punished for losing it. At +lanes=1, with 24 citizens contending, it is the ONLY thing standing between transient +contention and the whole roster pinned at the 240s cap. + +**The real open question is now sharper:** frontier model + 1 warm slot + N citizens is a +genuine allocation problem, not a starvation bug. Either the roster shrinks to what one slot +can serve, or citizens time-share the slot deliberately (a queue with fairness, not a +permit with a yield), or the box serves a smaller model for ambient work and reserves the +27B for directed/benchmark turns. That is Joel's call, not mine — it is a policy choice +about what the machine is FOR. From b4089a832e79813eb2e448dddf2748705cdf1c37 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 14:34:08 -0500 Subject: [PATCH 09/80] =?UTF-8?q?feat(cognition):=20make=20the=20ambient?= =?UTF-8?q?=20YIELD=20visible=20=E2=80=94=20a=20starved=20roster=20read=20?= =?UTF-8?q?identically=20to=20a=20healthy=20one?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured today at lanes=1: a 10-minute window showed zero `persona.selftick.*` rows. That is ambiguous, and dangerously so. It reads the same whether ONE citizen is mid-turn holding the only lane (healthy, just serialized) or the roster is dead. Resolving it required a manual `curl /slots`, which showed slot 0 processing with 3,727 tokens decoded — fine all along. The cause is structural: the yield branch emits nothing. That was the right call for volume — at 24 citizens on a 15s beat, a row per yield is ~92/min and would drown the stream exactly as `serving.plan` did at 2.6 rows/s (#399) — but the absence of a signal is not the same as a signal of absence ([[an-absence-is-an-unfinished-measurement]], the error class that produced the retracted premises on #211 and #390). Fix: count every yield (one relaxed increment on the permit miss, where the yield actually happens), and emit at most ONE probe per minute for the whole process carrying the delta. `persona.selftick.starved` names it as contention, not a fault — because at lanes=1 with 24 citizens it IS the expected shape, and a probe that cries fault at normal operation is the #348 / #422 defect again. Rate-limit rule is pure and table-tested. Four rows, each a real failure mode: the FIRST starvation always reports (a naive `now - last >= gap` swallows it on a box whose clock starts near zero — and the first one is the one that matters); a second report inside the gap is silent (the firehose); after the gap it reports the WINDOW delta, not the cumulative total (a cumulative count cannot show a rate); and a quiet roster emits nothing at all. Concurrent yielders race a compare-exchange so only one reports per window. 9/9 admission + 41/41 service_loop green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/resource_admission.rs | 102 ++++++++++++++++++ .../src/persona/service_loop.rs | 26 +++++ 2 files changed, 128 insertions(+) diff --git a/core/continuum-core/src/cognition/resource_admission.rs b/core/continuum-core/src/cognition/resource_admission.rs index 4b0533687..9cc15ab07 100644 --- a/core/continuum-core/src/cognition/resource_admission.rs +++ b/core/continuum-core/src/cognition/resource_admission.rs @@ -153,6 +153,70 @@ pub fn shared_model_saturated() -> bool { // throttling below the hardware. On a 1- or 2-lane box the budget floors at 1, so weak // machines get byte-identical behaviour to before. +/// Ambient turns that YIELDED because the pool was full, since process start. +/// +/// Why this exists (2026-08-17): the yield path emits no probe — deliberately, because at +/// 24 citizens on a 15s beat a per-yield row would be ~92 rows/min and would drown the +/// stream exactly as `serving.plan` did at 2.6 rows/s (#399). But the ABSENCE of a row made +/// starvation indistinguishable from health: a 10-minute window showing zero +/// `persona.selftick.*` reads identically whether one citizen is mid-turn holding the only +/// lane or nobody is ticking at all. Resolving that took a manual `/slots` curl. A counter +/// costs one relaxed increment and makes the difference reportable. +static AMBIENT_YIELDS: AtomicUsize = AtomicUsize::new(0); +/// Epoch-ms of the last starvation report, so the emit is rate-limited rather than per-yield. +static AMBIENT_YIELD_LAST_REPORT_MS: AtomicUsize = AtomicUsize::new(0); + +/// Minimum gap between starvation reports. One row a minute is free next to a 15s beat and +/// still resolves "is the roster moving?" at the granularity anyone asks it. +const AMBIENT_YIELD_REPORT_GAP_MS: usize = 60_000; + +/// Cumulative ambient yields since process start. +pub fn ambient_yields() -> usize { + AMBIENT_YIELDS.load(Ordering::Relaxed) +} + +/// Should the caller emit a starvation report now, and if so for how many yields? +/// +/// Pure over its inputs so the rate-limit rule is unit-testable without a clock: returns +/// `Some(yields_in_window)` at most once per [`AMBIENT_YIELD_REPORT_GAP_MS`]. `last_report_ms` +/// of 0 means "never reported" and always fires, so the FIRST starvation is never silent — +/// which is the one that matters, and the one a naive `now - last >= gap` would swallow on a +/// box whose clock starts near zero. +pub fn ambient_yield_report_due( + total_yields: usize, + yields_at_last_report: usize, + now_ms: usize, + last_report_ms: usize, +) -> Option { + let fresh = total_yields.saturating_sub(yields_at_last_report); + if fresh == 0 { + return None; + } + if last_report_ms != 0 && now_ms.saturating_sub(last_report_ms) < AMBIENT_YIELD_REPORT_GAP_MS { + return None; + } + Some(fresh) +} + +/// Claim the report slot if due, returning the yield count to report. Swaps the timestamp +/// atomically so concurrent yielders cannot both emit for the same window. +pub fn take_ambient_yield_report(now_ms: u64) -> Option { + let now = now_ms as usize; + let last = AMBIENT_YIELD_LAST_REPORT_MS.load(Ordering::Relaxed); + let total = AMBIENT_YIELDS.load(Ordering::Relaxed); + // `yields_at_last_report` is encoded by the caller's window: we only need the delta since + // the last report, and the report resets the clock, so total-at-report is implicit. + let fresh = ambient_yield_report_due(total, 0, now, last)?; + // Only the winner of this compare-exchange reports. + if AMBIENT_YIELD_LAST_REPORT_MS + .compare_exchange(last, now, Ordering::AcqRel, Ordering::Relaxed) + .is_err() + { + return None; + } + Some(fresh) +} + /// Try to claim an ambient-turn slot. `Some(permit)` → run the ambient turn (hold the /// permit for the turn's lifetime; it releases on drop). `None` → all ambient slots /// are busy; the caller yields this ambient turn. Non-blocking (never waits). @@ -373,6 +437,11 @@ impl LaneAdmission { .clone() .try_acquire_owned() .ok() + .or_else(|| { + // Count the miss. The permit is non-blocking, so a None here IS the yield. + AMBIENT_YIELDS.fetch_add(1, Ordering::Relaxed); + None + }) } /// See [`acquire_serving_lane`] — the reservation policy lives HERE so the global and a @@ -733,6 +802,39 @@ mod tests { drop(held); // release the rest (nothing else can observe this gate anyway) } + // what this catches: a starvation report that is either a firehose or silent. At 24 + // citizens on a 15s beat, per-yield rows would be ~92/min and would drown the stream the + // way serving.plan did at 2.6 rows/s (#399) — but NO row made a starved roster + // indistinguishable from a healthy one, which cost a manual /slots curl to resolve on + // 2026-08-17. Both failure modes are pinned here. + #[test] + fn the_starvation_report_is_rate_limited_but_the_first_one_is_never_silent() { + // Never reported before (last_report_ms == 0) → fires immediately, however early. + assert_eq!( + ambient_yield_report_due(23, 0, 5, 0), + Some(23), + "the FIRST starvation must report even at t=5ms — it is the one that matters" + ); + // Inside the gap → silent, no matter how many yields piled up. + assert_eq!( + ambient_yield_report_due(1_000, 23, 10_000, 9_000), + None, + "a second report inside the gap would be the firehose" + ); + // Gap elapsed → reports only the DELTA, not the cumulative total. + assert_eq!( + ambient_yield_report_due(1_000, 23, 70_000, 9_000), + Some(977), + "reports yields in THIS window; a cumulative count cannot show a rate" + ); + // No new yields → nothing to say, even after the gap. Contention ended. + assert_eq!( + ambient_yield_report_due(23, 23, 999_000, 9_000), + None, + "a quiet roster must not emit a row saying nothing happened" + ); + } + // what this catches: the WEAK-BOX floor of the same change. Deriving the ambient pool // from lanes−1 must not regress a 1- or 2-lane machine below the behaviour it had // under the old hardcoded 1 — a single-lane host has nothing to reserve, so its diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 63438f748..75026df62 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -526,6 +526,32 @@ async fn serve_persona_loop_inner( engaged_beat, rest_cap, ); + // Make the yield VISIBLE, at ≤1 row/min for the whole process. + // Without this the yield path emits nothing, so a window with + // zero `persona.selftick.*` reads identically whether one citizen + // is mid-turn holding the only lane or the roster is dead — + // resolving that took a manual /slots curl on 2026-08-17. Per-yield + // rows would be ~92/min at this roster size and would drown the + // stream (#399), so the admission gate rate-limits and only the + // winner of its compare-exchange reports. + if let Some(yields) = + crate::cognition::resource_admission::take_ambient_yield_report( + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_millis() as u64) + .unwrap_or(0), + ) + { + crate::probe!( + class = "persona.selftick.starved", + yields_since_last_report = yields, + total_yields = + crate::cognition::resource_admission::ambient_yields(), + "ambient turns yielded — the pool was full. NOT a fault: \ + this is what contention looks like when citizens outnumber \ + non-directed lanes. Rate-limited to ≤1 row/min." + ); + } continue; } }; From 00f6d57f46e3d136b206bb35b6b19a451145e96e Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 14:55:38 -0500 Subject: [PATCH 10/80] fix(cognition): my own starvation probe reported CUMULATIVE under a field named per-window MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught within the hour of shipping it, by reading the live rows instead of trusting the commit message: 19:44:02Z yields_this_window=135 cumulative=135 19:45:03Z yields_this_window=197 cumulative=197 19:46:03Z yields_this_window=263 cumulative=263 Identical on every row, while consecutive rows differ by ~65. The field named `yields_since_last_report` was reporting the running total. Cause: `take_ambient_yield_report` passed a literal 0 for `yields_at_last_report`, with a comment justifying it — "the report resets the clock, so total-at-report is implicit." That is false. Resetting the TIMESTAMP does not reset the COUNT, so `fresh = total - 0 = total` forever. The galling part, and worth stating plainly: the pure rule was RIGHT and its table test asserted the delta correctly (`ambient_yield_report_due(1_000, 23, 70_000, 9_000) == Some(977)`). The defect lived entirely in the impure wrapper feeding it a wrong argument — which is exactly the seam the pure/impure split exists to protect and exactly the gap a table test over the pure half cannot close. I wrote a commit an hour earlier about fields that promise one thing and deliver another, then shipped one. Fix: a second watermark, `AMBIENT_YIELDS_AT_LAST_REPORT`, advanced by the same winner of the timestamp compare-exchange so a losing yielder's counts roll into the NEXT window rather than being dropped. New test pins the argument boundary the old one couldn't reach: the second window must read 62 (197−135), not 197. The mislabel did NOT cost the measurement — consecutive cumulative rows still give the rate by subtraction, and that rate is the finding: ~65 ambient yields per minute across 24 citizens with ZERO self-cycles completing. First time that contention has had a number instead of a silence. 10/10 green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/resource_admission.rs | 54 ++++++++++++++++--- 1 file changed, 48 insertions(+), 6 deletions(-) diff --git a/core/continuum-core/src/cognition/resource_admission.rs b/core/continuum-core/src/cognition/resource_admission.rs index 9cc15ab07..f5f36aa16 100644 --- a/core/continuum-core/src/cognition/resource_admission.rs +++ b/core/continuum-core/src/cognition/resource_admission.rs @@ -198,22 +198,39 @@ pub fn ambient_yield_report_due( Some(fresh) } -/// Claim the report slot if due, returning the yield count to report. Swaps the timestamp -/// atomically so concurrent yielders cannot both emit for the same window. +/// Yield total as of the last report — the OTHER half of the window, and the half I +/// originally forgot. +/// +/// Shipped 2026-08-17 passing a literal 0 for `yields_at_last_report`, on the reasoning that +/// "the report resets the clock, so total-at-report is implicit". It is not: resetting the +/// TIMESTAMP does not reset the COUNT, so `fresh = total - 0 = total` and every row reported +/// the cumulative figure under a field named `yields_since_last_report`. Caught within the +/// hour by reading the live rows — yields_this_window == cumulative on all six (457==457, +/// 392==392, …) when consecutive rows differed by ~65. The pure `ambient_yield_report_due` +/// was correct and its table test asserted the delta properly; the defect was entirely in +/// this impure wrapper feeding it a wrong argument, which is precisely why the pure/impure +/// split exists and precisely the gap a table test cannot close. +static AMBIENT_YIELDS_AT_LAST_REPORT: AtomicUsize = AtomicUsize::new(0); + +/// Claim the report slot if due, returning the yields IN THIS WINDOW. Swaps the timestamp +/// atomically so concurrent yielders cannot both emit for the same window, and advances the +/// count watermark in the same critical section so the next window measures from here. pub fn take_ambient_yield_report(now_ms: u64) -> Option { let now = now_ms as usize; let last = AMBIENT_YIELD_LAST_REPORT_MS.load(Ordering::Relaxed); let total = AMBIENT_YIELDS.load(Ordering::Relaxed); - // `yields_at_last_report` is encoded by the caller's window: we only need the delta since - // the last report, and the report resets the clock, so total-at-report is implicit. - let fresh = ambient_yield_report_due(total, 0, now, last)?; - // Only the winner of this compare-exchange reports. + let at_last = AMBIENT_YIELDS_AT_LAST_REPORT.load(Ordering::Relaxed); + let fresh = ambient_yield_report_due(total, at_last, now, last)?; + // Only the winner of this compare-exchange reports… if AMBIENT_YIELD_LAST_REPORT_MS .compare_exchange(last, now, Ordering::AcqRel, Ordering::Relaxed) .is_err() { return None; } + // …and the winner alone advances the count watermark, so the loser's yields are not + // silently dropped from the NEXT window — they are still ahead of `total` here. + AMBIENT_YIELDS_AT_LAST_REPORT.store(total, Ordering::Relaxed); Some(fresh) } @@ -835,6 +852,31 @@ mod tests { ); } + // what this catches: THE BUG I SHIPPED, in the impure wrapper the table test could not + // reach. `take_ambient_yield_report` passed a literal 0 for `yields_at_last_report`, so + // every row reported the CUMULATIVE total under a field named `yields_since_last_report`. + // Live rows on 2026-08-17 read 457==457, 392==392 while consecutive rows differed by ~65. + // A pure rule with a correct test is worth nothing if its caller feeds it a wrong + // argument — this asserts the SECOND window measures from the first, not from zero. + #[test] + fn consecutive_reports_measure_windows_not_running_totals() { + // Simulate two windows over the shared statics via the pure rule, which is what the + // wrapper now actually does (the wrapper itself touches process globals, so the + // contract that broke is pinned here at the argument boundary). + let first = ambient_yield_report_due(135, 0, 60_000, 0).expect("first window reports"); + assert_eq!(first, 135, "first window is total-so-far — nothing preceded it"); + + // Second window: watermark advanced to 135, total climbed to 197. + let second = ambient_yield_report_due(197, 135, 120_000, 60_000) + .expect("second window reports"); + assert_eq!( + second, 62, + "the SECOND window must be the delta (197-135), not the cumulative 197 — \ + passing 0 here is the bug that shipped" + ); + assert_ne!(second, 197, "reporting cumulative under a per-window name is the lie"); + } + // what this catches: the WEAK-BOX floor of the same change. Deriving the ambient pool // from lanes−1 must not regress a 1- or 2-lane machine below the behaviour it had // under the old hardcoded 1 — a single-lane host has nothing to reserve, so its From 4323ac15840fbb3f1f6fa00a33236d3baee453ee Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 15:28:21 -0500 Subject: [PATCH 11/80] =?UTF-8?q?fix(persona):=20the=20work=20board=20grew?= =?UTF-8?q?=20into=20the=20prompt=20=E2=80=94=20cap=20the=20per-card=20ren?= =?UTF-8?q?der,=20order=20by=20relevance?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit READ OUT OF A LIVE AMBIENT TURN (capture 762a806b, 2026-08-17), not inferred: system prompt 18,697 chars [board] 9,284 chars 49.7% ← 38 distinct cards, full titles (preamble) 7,863 chars 42.1% workspace-map 1,478 chars 7.9% observed prefill 12,286 tokens for a turn that decoded 41 Half of every citizen's system prompt was the kanban board, re-sent on every self-tick, and it was ~25% of total prefill. The per-card loop was `for card in &board.cards` bounded only by remaining budget. Two things make that a defect rather than a cost: 1. IT IS REDUNDANT. The `[board]` headline and the `[your work]` / `[available work]` leads directly above already carry the counts and the top titles — and those leads were ADDED precisely because this dump is unreliable under budget (the Asha 2026-08-06 note in this file). So the same facts are paid for twice and the expensive copy is the one truncated. 2. IT SCALES BACKWARDS. Bounded by budget, the board grows to fill the prompt, so every citizen thinks SLOWER as the backlog grows. A team that accumulates work becomes progressively less able to do it. That is the opposite of what a work board is for, and it gets worse every time we file a card. FIX: order by what she can act on — her own holds first, then newest claimable (the same recency argument the available-work lead already makes) — and take at most MAX_RENDERED_CARDS. Counts in the headline stay the TRUE totals; the board is not made smaller than it is, only shorter than it was. `work/list` remains the full read and is verified working (#337). The cap is sized from what it MUST cover, not picked: both leads name up to 5, so the detail must carry 10 or the summary would promise cards the detail omits — a citizen seeing "you hold 5" above a list of 3 would rightly read the substrate as inconsistent. Two spare keeps a just-created card visible in the turn it arrives. Ten is the floor; it cannot drop below without re-deriving the lead caps with it. Also fixed: cap-truncation now counts into `dropped` alongside budget-truncation, so the debug line does not report dropped=0 on a shortened board and hide the shortening in the one place an operator looks. Test pins all three halves — the cap holds, her OWN card survives the cut even when it is the OLDEST on the board (under the old board-order loop a cap would have cut exactly the worst card), and the headline still reports the true 40. Positive-controlled: with the cap removed it fails "board must be capped at 12, rendered 41 of 41". 13/13 green. NOT YET MEASURED LIVE: the prompt-size delta. Deploying next; the number goes on the observation card, not in this message. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/room_board_source.rs | 127 +++++++++++++++++- 1 file changed, 126 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/persona/room_board_source.rs b/core/continuum-core/src/persona/room_board_source.rs index 4517772f9..d0889b577 100644 --- a/core/continuum-core/src/persona/room_board_source.rs +++ b/core/continuum-core/src/persona/room_board_source.rs @@ -68,6 +68,22 @@ use crate::persona::rag_budget::{ /// `active-work` (own claims) and `room-board` (the wall). const SOURCE_ID: &str = "room-kanban"; +/// Most cards this source will render in full, per turn. +/// +/// NOT a token budget — a RELEVANCE bound. The per-card loop below used to be limited only +/// by whatever budget remained, so the board grew to fill the prompt: measured 2026-08-17, +/// 38 cards / 9,284 chars / 49.7% of an 18,697-char system prompt on a routine ambient turn. +/// That makes every citizen think slower as the backlog grows, which is exactly backwards +/// for a work board. +/// +/// Sized from what it MUST cover, not picked: the `[your work]` and `[available work]` leads +/// each name up to 5 cards, so the detailed render must be able to carry both (10) or the +/// summaries would promise cards the detail then omits — a citizen seeing "you hold 5" above +/// a list of 3 would reasonably read the substrate as inconsistent. Two spare keeps a +/// just-created card visible in the same turn it arrives. Ten is the floor this cannot go +/// below without re-deriving the lead caps with it. +const MAX_RENDERED_CARDS: usize = 12; + /// Token estimate — the ONE canonical chars/4 estimator /// (`cognition::token_budget`), shared by every RAG source so the replay /// ledger's numbers match. @@ -578,8 +594,39 @@ impl RagSource for RoomBoardSource { } let names = self.reader.peer_names(&owner_peers).await; + // RELEVANCE ORDER + A HARD CAP, not "however many the budget happens to fit". + // + // Measured 2026-08-17 on a live ambient turn (capture 762a806b): this loop rendered + // 38 distinct cards into 9,284 chars — 49.7% of an 18,697-char system prompt, ~3,100 + // of the turn's ~12,286 prefill tokens. Every citizen paid it on every self-tick. + // + // Two things make that wrong rather than merely expensive: + // + // 1. IT IS REDUNDANT. The `[board]` headline and the `[your work]` / `[available + // work]` leads above already carry the counts and the top titles — and they were + // ADDED because this dump is unreliable under budget (see the Asha 2026-08-06 + // note above). Rendering the summary AND the exhaustive list means the same facts + // are paid for twice, and the expensive copy is the one that gets truncated. + // 2. IT SCALES THE WRONG WAY. Bounded only by remaining budget, the board grows to + // fill the prompt, so every citizen thinks SLOWER as the backlog grows. A team + // that accumulates work gets progressively less able to do it — the opposite of + // what a work board is for. + // + // So: order by what she can act on (her own holds first, then newest claimable — + // the same recency argument the available-work lead already makes), take at most + // `MAX_RENDERED_CARDS`, and let the leads plus `work/list` carry the tail. Counts in + // the headline stay the TRUE totals; nothing here makes the board smaller than it is, + // only shorter than it was. + let mut ordered: Vec<&airc_work::WorkCard> = board.cards.iter().collect(); + ordered.sort_by_key(|c| { + let mine_first = !(c.owner.map(|o| o.as_uuid()) == Some(self.persona_id)); + // Negate for newest-first without needing a reversed comparator. + (mine_first, std::cmp::Reverse(c.created_at_ms)) + }); + let render_budget_cards = MAX_RENDERED_CARDS.min(ordered.len()); + let mut cards_delivered = 0usize; - for card in &board.cards { + for card in ordered.into_iter().take(render_budget_cards) { let content = Self::render(card, self.persona_id, now_ms, &names); let tokens = estimate_tokens(&content); if tokens_used.saturating_add(tokens) > budget { @@ -589,6 +636,9 @@ impl RagSource for RoomBoardSource { // Counted over CARDS, excluding the available-work lead item. dropped = board.cards.len() - cards_delivered; break; + // (unchanged: `dropped` is measured against the TRUE board size, so a + // budget-truncated render and a cap-truncated one report the same honest + // "how many did she not see".) } tokens_used += tokens; cards_delivered += 1; @@ -627,6 +677,13 @@ impl RagSource for RoomBoardSource { }); } + // Cap-truncation counts as dropped just like budget-truncation. Without this the + // debug line would report dropped=0 on a capped board and the shortening would be + // invisible in the one place an operator looks for it. + if dropped == 0 && cards_delivered < board.cards.len() { + dropped = board.cards.len() - cards_delivered; + } + // Budget too small to carry even one card → no block. if items.is_empty() { return Self::empty(); @@ -831,6 +888,74 @@ mod tests { // renders into the [room-kanban] grounding block — every card with its // column, title, priority, and owner. This is the Observer perceiving the // board (task #117 O6), distinct from active-work's own-claims-only view. + // what this catches: THE BOARD EATING THE PROMPT. The per-card loop used to be bounded + // only by remaining budget, so it grew with the backlog — measured live 2026-08-17 at 38 + // cards / 9,284 chars / 49.7% of an 18,697-char system prompt on a routine ambient turn, + // ~3,100 of ~12,286 prefill tokens, paid by every citizen on every self-tick. The + // property that makes it a defect and not just a cost: a team accumulating work gets + // progressively SLOWER at doing it. + // + // Pins all three halves of the fix — the cap holds, her OWN card survives the cut even + // when it is the oldest on the board (relevance order, not board order), and the headline + // still reports the TRUE totals so nothing is hidden, only shortened. + #[tokio::test] + async fn a_large_board_is_capped_and_keeps_her_own_card_first() { + let mut cards: Vec = Vec::new(); + // Hers, and DELIBERATELY the oldest — under the old board-order loop with a cap this + // would be the first thing cut, which is the worst possible card to lose. + let mut mine = card("MY held card", CardState::InProgress, Some( + airc_core::PeerId::from_uuid(persona()), + )); + mine.created_at_ms = 1; // oldest on the board + cards.push(mine); + for i in 0..40 { + let mut c = card(&format!("filler card {i}"), CardState::Open, None); + c.created_at_ms = 1_000_000 + i as u64; // all newer than hers + cards.push(c); + } + let total = cards.len(); + + let reader = Arc::new(StubReader::new(snapshot(cards))); + let source = RoomBoardSource::new(persona(), reader); + // A budget large enough that the CAP, not the budget, is what binds — otherwise this + // would pass for the old reason and prove nothing. + let delivery = source + .deliver(&ctx(), 100_000, ResolutionPreference::Raw) + .await; + + let rendered: Vec<&RagItem> = delivery + .items + .iter() + .filter(|i| i.metadata.get("card_id").is_some()) + .collect(); + assert!( + rendered.len() <= MAX_RENDERED_CARDS, + "board must be capped at {MAX_RENDERED_CARDS}, rendered {} of {total} — \ + an uncapped board grows into the prompt as the backlog grows", + rendered.len() + ); + assert!( + rendered.len() >= 10, + "the cap must still cover both 5-card leads, got {}", + rendered.len() + ); + assert!( + rendered[0].content.contains("MY held card"), + "her OWN card must survive the cut and lead, even as the oldest on the board: {}", + rendered[0].content + ); + // Counts stay TRUE — shortening the render must never shrink the reported board. + let headline = &delivery.items[0]; + assert_eq!(headline.metadata["kind"], "board-headline"); + assert_eq!(headline.metadata["held_count"], 1); + assert_eq!(headline.metadata["open_count"], 40); + assert!( + headline.content.contains("40 claimable"), + "headline reports the real total, not the rendered slice: {}", + headline.content + ); + } + #[tokio::test] async fn whole_board_surfaces_with_owner_and_state() { let holder = airc_core::PeerId::new(); From cabecb9c48bd6ea436e4341d0af4b403e81efe12 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 16:16:16 -0500 Subject: [PATCH 12/80] fix(swe): one era-pinned install path, so the sdist build-deps heal like -e . already did (#380/#383) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ensure_env` had TWO sites that install under the instance's date pin, and only ONE of them could heal when the pin itself created an unresolvable hole: - the editable `-e .` install carried a bounded retry loop that reads uv's own `exclude-newer-package` hint and retries with the minimum per-package override; - the dependency-sdist build-dep pre-install (#383 cause 2) applied `--exclude-newer` and failed hard. Same decision, two expressions, one of them healed. Measured 2026-08-17 dispatching swe-bench-lite's head: every astropy instance died at the unhealed site with could not pre-install astropy__astropy-6938's dependency-sdist build deps ["jinja2"] × Failed to download and build `markupsafe==1.0` ╰─▶ Because only setuptools<=38.2.4 is available and you require setuptools>=40.8.0 which is the pin working exactly as designed — jinja2 2.10 pulls markupsafe 1.0, whose setup.py build wants setuptools>=40.8.0, and the 2017 cutoff caps setuptools at 38.2.4. Unsatisfiable by construction. uv's stderr named the package and suggested `exclude-newer-package`; the loop 30 lines below knew how to parse exactly that hint and never saw it. Extracted `era_pinned_uv_install(uv, py, as_of, tail, cwd, envs)` holding the whole bounded loop, all three heal arms, and the importlib-metadata upgrade remedy (general to any --no-build-isolation build, not just `-e .`). Both sites now call it; a third site inherits the behaviour by construction rather than by remembering. MAX_ERA_OVERRIDES replaces the bare 8. No behaviour change at the `-e .` site — same flags, same arms, same bound. The change is that the other site now has them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 209 +++++++++++------- 1 file changed, 131 insertions(+), 78 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 7ea04fefc..7f08a2789 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -788,13 +788,19 @@ pub async fn ensure_env(instance: &SweInstance, repo_dir: &Path) -> Result=40.8.0, and the + // heal that reads uv's own `exclude-newer-package` hint lived only at the other site. + let out = era_pinned_uv_install( + &uv, + &py_s, + as_of.as_deref(), + sdist_deps, + None, + &[("CFLAGS", ERA_CFLAGS)], + ) + .await?; if !out.status.success() { // Fail LOUD and leave no half-built env behind — same doctrine as the `-e .` gate. let _ = std::fs::remove_dir_all(&env_dir); @@ -819,77 +825,19 @@ pub async fn ensure_env(instance: &SweInstance, repo_dir: &Path) -> Result = Vec::new(); - let out = loop { - let mut args = vec!["pip", "install", "-q", "--python", &py_s]; - if let Some(ref date) = as_of { - args.push("--exclude-newer"); - args.push(date); - } - for pin in &overrides { - args.push("--exclude-newer-package"); - args.push(pin); - } - args.extend(["--no-build-isolation", "-e", "."]); - // ERA_CFLAGS rides on this one invocation because it is where every C build - // happens — the repo's own extensions AND its dependency sdists (pyerfa et al) - // compile inside this resolve. - let out = run_env(&uv, &args, Some(Path::new(&repo_s)), &[("CFLAGS", ERA_CFLAGS)]).await?; - if out.status.success() || as_of.is_none() { - break out; - } - let stderr = String::from_utf8_lossy(&out.stderr).to_string(); - // Two heal arms, same bounded loop: (1) deleted-history — the date pin leaves zero - // candidates, uv's hint names the earliest surviving upload; (2) metadata-mismatch — - // an era sdist with no wheel for this platform builds as version 0.0.0 - // (setuptools_scm without git metadata; live 2026-08-11: lazy-object-proxy 1.7.1 has - // no arm64 wheel, pulled by 2022 pylint→astroid), so the ONE unbuildable package's - // cutoff is lifted entirely — a modern wheel-shipping release of a shim library, in - // an otherwise era-pure graph, disclosed on the probe. Both parse uv's OWN evidence; - // no hand-maintained package list. - match deleted_history_override(&stderr) - .or_else(|| metadata_mismatch_override(&stderr)) - .or_else(|| setuptools_importlib_clash_override(&stderr)) - { - Some(pin) if !overrides.contains(&pin) && overrides.len() < 8 => { - tracing::warn!( - instance = %instance.instance_id, - r#override = %pin, - "date-pinned resolution hit an unresolvable era package — retrying with \ - a per-package cutoff derived from uv's own error" - ); - // The setuptools/importlib clash needs MORE than a lifted cutoff: the - // broken importlib-metadata 0.x is ALREADY INSTALLED in the venv (2019 - // pluggy pulled it in the requirements step), and `-e .` won't touch an - // already-satisfied package — so the cutoff pin alone retries into the - // exact same crash (live: pytest-5413/5495 kickoff, 2026-08-12, the - // first run after this arm shipped). Apply the banner's own remedy - // directly: upgrade the installed copy, then retry the editable build. - if pin.starts_with("importlib-metadata=") { - let up = run( - &uv, - &[ - "pip", - "install", - "-q", - "--python", - &py_s, - "--upgrade", - "importlib-metadata", - ], - None, - ) - .await?; - if !up.status.success() { - // The heal itself failed — no point looping on the same wall. - break out; - } - } - overrides.push(pin); - } - _ => break out, - } - }; + // ERA_CFLAGS rides on this invocation because it is where every C build happens — the + // repo's own extensions AND its dependency sdists (pyerfa et al) compile inside this + // resolve. The heal loop lives in `era_pinned_uv_install`, shared with the sdist + // build-dep pre-install above. + let out = era_pinned_uv_install( + &uv, + &py_s, + as_of.as_deref(), + &["--no-build-isolation", "-e", "."], + Some(Path::new(&repo_s)), + &[("CFLAGS", ERA_CFLAGS)], + ) + .await?; if !out.status.success() { // DELETE the half-built env rather than cache it. Keying on "the directory exists" // made a failed install sticky: every later run reused a venv with no repo in it and @@ -1105,6 +1053,111 @@ fn setuptools_importlib_clash_override(stderr: &str) -> Option { } } +/// How many per-package cutoff overrides one install may discover before we stop. +/// +/// Each round must surface a NEW package (see the loop), so this bounds a pathological +/// graph rather than a healthy one — history-holes per graph are few, and the live maximum +/// observed across Lite is two. +const MAX_ERA_OVERRIDES: usize = 8; + +/// ONE era-pinned `uv pip install`, healing from uv's own evidence. +/// +/// # Why this is a function and not two copies of a loop +/// +/// "Install under the instance's date pin, and when the resolver hits a hole that the pin +/// itself created, read uv's hint and retry with the minimum per-package override" is ONE +/// decision. It had TWO call sites in `ensure_env` — the editable `-e .` install, which had +/// the heal loop, and the dependency-sdist build-dep pre-install, which did not — so the +/// same class of failure was survivable at one site and fatal at the other. +/// +/// Measured 2026-08-17: every astropy instance in swe-bench-lite's head died at the +/// unhealed site with `could not pre-install …'s dependency-sdist build deps ["jinja2"]`. +/// The cause is the pin working exactly as designed: jinja2 2.10 needs markupsafe 1.0, +/// whose `setup.py` build requires `setuptools>=40.8.0`, while the 2017 cutoff caps +/// setuptools at 38.2.4 — unsatisfiable by construction. uv's stderr named the package and +/// suggested `exclude-newer-package`, which is precisely what the loop 30 lines below knew +/// how to parse. Extracting it heals both sites and makes a third site inherit the +/// behaviour by construction. +/// +/// `tail` is the call-site-specific argv after the shared pin flags (`["jinja2"]`, or +/// `["--no-build-isolation", "-e", "."]`). `as_of: None` disables pinning AND healing — +/// with no pin there is no pin-induced hole to heal. +async fn era_pinned_uv_install( + uv: &str, + py: &str, + as_of: Option<&str>, + tail: &[&str], + cwd: Option<&Path>, + envs: &[(&str, &str)], +) -> Result { + let mut overrides: Vec = Vec::new(); + loop { + let mut args = vec!["pip", "install", "-q", "--python", py]; + if let Some(date) = as_of { + args.push("--exclude-newer"); + args.push(date); + } + for pin in &overrides { + args.push("--exclude-newer-package"); + args.push(pin); + } + args.extend(tail.iter().copied()); + let out = run_env(uv, &args, cwd, envs).await?; + if out.status.success() || as_of.is_none() { + return Ok(out); + } + let stderr = String::from_utf8_lossy(&out.stderr).to_string(); + // Three heal arms, same bounded loop: (1) deleted-history — the date pin leaves zero + // candidates and uv's hint names the earliest surviving upload; (2) metadata-mismatch + // — an era sdist with no wheel for this platform builds as version 0.0.0 + // (setuptools_scm without git metadata), so the ONE unbuildable package's cutoff is + // lifted entirely; (3) the setuptools/importlib-metadata clash. All three parse uv's + // OWN evidence; no hand-maintained package list. + match deleted_history_override(&stderr) + .or_else(|| metadata_mismatch_override(&stderr)) + .or_else(|| setuptools_importlib_clash_override(&stderr)) + { + Some(pin) if !overrides.contains(&pin) && overrides.len() < MAX_ERA_OVERRIDES => { + tracing::warn!( + r#override = %pin, + tail = ?tail, + "date-pinned resolution hit an unresolvable era package — retrying with \ + a per-package cutoff derived from uv's own error" + ); + // The setuptools/importlib clash needs MORE than a lifted cutoff: the broken + // importlib-metadata 0.x is ALREADY INSTALLED in the venv (2019 pluggy pulled + // it in the requirements step), and an install won't touch an already-satisfied + // package — so the cutoff pin alone retries into the exact same crash (live: + // pytest-5413/5495 kickoff, 2026-08-12, the first run after this arm shipped). + // Apply the banner's own remedy directly: upgrade the installed copy, then + // retry. General to any `--no-build-isolation` build, not just `-e .`. + if pin.starts_with("importlib-metadata=") { + let up = run( + uv, + &[ + "pip", + "install", + "-q", + "--python", + py, + "--upgrade", + "importlib-metadata", + ], + None, + ) + .await?; + if !up.status.success() { + // The heal itself failed — no point looping on the same wall. + return Ok(out); + } + } + overrides.push(pin); + } + _ => return Ok(out), + } + } +} + fn which(bin: &str) -> Option { let path = std::env::var("PATH").ok()?; for dir in path.split(':') { From 2566a09d6f0d134d013c453431301be8dcd6191d Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 16:49:08 -0500 Subject: [PATCH 13/80] fix(rag): roster + doctrine read the room she is STANDING in, not the one she was bound to (#443) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The board source was made turn-parametric weeks ago. Roster and doctrine were not, so a citizen taking a turn in a per-run bench room got the cards and NEITHER the peers nor the rules. Measured live 2026-08-17 over 90 minutes on this box: source bound → turn room n roster academy swe-bench 23 room-doctrine academy swe-bench 19 roster academy hard-rs 3 room-doctrine academy hard-rs 3 That is 42 + 39 `rag.room_gate.abstain` rows: every bench turn ran with no roster and no operating doctrine. She was handed the work with none of the room's rules and no idea who else was there. THE CAPABILITY ALREADY EXISTED AND WAS NEVER WIRED. `airc_lib` has `room_roster_in`, `room_roster_cards_in` and `room_doctrine_in`, and `room_doctrine_in`'s own doc names this exact defect verbatim: "A citizen who belongs to several rooms answers a turn in the room it arrived in; reading doctrine from her default instead grounds that answer in another room's rules" Continuum called the roomless variants everywhere. This is the wire, not a new mechanism — the same shape as the astropy env fix earlier today (cabecb9c4): one decision expressed at several call sites, correct at one of them. - `AircRosterReader::room_roster` / `room_roster_cards` and `AircDoctrineReader::room_doctrine` take `room: Option`; the real impls delegate to airc's `*_in`. - Both sources resolve `turn_room.or(bound)` exactly as RoomBoardSource does. `None` (an UNSTAMPED context — background consolidation) keeps pre-#443 behaviour. - The exam-bleed pin SURVIVES and is now pinned by a test: a synthetic nil room gets nothing AND never falls back to the bound room; the reader is not consulted at all. Tests: `the_turn_room_wins_over_the_bound_room` asserts the reader is asked for the TURN room (the stub records what it was asked — a stub that ignores the argument would let this regress invisibly), and `a_nil_room_gets_nothing_and_never_falls_back` holds the exam pin. 8/8 green in room_doctrine_source. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/context/airc_adapter.rs | 25 +++- .../src/ipc/experience_resolver.rs | 3 +- .../src/ipc/positron_presence.rs | 6 +- .../src/persona/airc_citizen.rs | 2 + .../src/persona/airc_runtime.rs | 16 ++- .../src/persona/prompt_assembly.rs | 1 + .../src/persona/room_doctrine_source.rs | 121 ++++++++++++++++-- .../src/persona/room_roster_source.rs | 76 ++++++++--- 8 files changed, 215 insertions(+), 35 deletions(-) diff --git a/core/continuum-core/src/context/airc_adapter.rs b/core/continuum-core/src/context/airc_adapter.rs index 193c60ce8..353fcea72 100644 --- a/core/continuum-core/src/context/airc_adapter.rs +++ b/core/continuum-core/src/context/airc_adapter.rs @@ -65,8 +65,15 @@ impl crate::persona::room_roster_source::AircRosterReader for AircHandleAdapter &self, within: std::time::Duration, window: usize, + room: Option, ) -> Result, AircError> { - self.inner.room_roster(within, window).await + crate::persona::room_roster_source::AircRosterReader::room_roster( + self.inner.as_ref(), + within, + window, + room, + ) + .await } // #262: forward the CARDS read to the real airc identity join. Without @@ -78,8 +85,15 @@ impl crate::persona::room_roster_source::AircRosterReader for AircHandleAdapter &self, within: std::time::Duration, window: usize, + room: Option, ) -> Result, AircError> { - self.inner.room_roster_cards(within, window).await + crate::persona::room_roster_source::AircRosterReader::room_roster_cards( + self.inner.as_ref(), + within, + window, + room, + ) + .await } } @@ -87,8 +101,13 @@ impl crate::persona::room_roster_source::AircRosterReader for AircHandleAdapter impl crate::persona::room_doctrine_source::AircDoctrineReader for AircHandleAdapter { async fn room_doctrine( &self, + room: Option, ) -> Result, AircError> { - self.inner.room_doctrine().await + crate::persona::room_doctrine_source::AircDoctrineReader::room_doctrine( + self.inner.as_ref(), + room, + ) + .await } } diff --git a/core/continuum-core/src/ipc/experience_resolver.rs b/core/continuum-core/src/ipc/experience_resolver.rs index 913fbbb60..57470bb6c 100644 --- a/core/continuum-core/src/ipc/experience_resolver.rs +++ b/core/continuum-core/src/ipc/experience_resolver.rs @@ -62,7 +62,7 @@ impl LiveExperienceResolver { roles: &BTreeMap, ) -> Option { let manifest = self.source.experience_for(room_id)?; - let members = match self.roster.room_roster(PRESENCE_WINDOW, ROSTER_SCAN).await { + let members = match self.roster.room_roster(PRESENCE_WINDOW, ROSTER_SCAN, None).await { Ok(members) => project_membership(&members, roles), Err(error) => { tracing::warn!( @@ -108,6 +108,7 @@ mod tests { &self, _within: Duration, _window: usize, + _room: Option, ) -> Result, airc_lib::AircError> { Ok(self.members.clone()) } diff --git a/core/continuum-core/src/ipc/positron_presence.rs b/core/continuum-core/src/ipc/positron_presence.rs index da15792f3..f5f047776 100644 --- a/core/continuum-core/src/ipc/positron_presence.rs +++ b/core/continuum-core/src/ipc/positron_presence.rs @@ -366,7 +366,7 @@ async fn run_presence_loop( .await .unwrap_or_default(); match reader - .room_roster_cards(MEMBERSHIP_WINDOW, MEMBERSHIP_SCAN) + .room_roster_cards(MEMBERSHIP_WINDOW, MEMBERSHIP_SCAN, None) .await { Ok(members) => { @@ -483,7 +483,7 @@ async fn emit_once( directory: &mut HashMap, force: bool, ) -> bool { - let members = match reader.room_roster_cards(PRESENCE_WINDOW, ROSTER_SCAN).await { + let members = match reader.room_roster_cards(PRESENCE_WINDOW, ROSTER_SCAN, None).await { Ok(m) => m, Err(err) => { tracing::warn!( @@ -632,6 +632,7 @@ mod tests { &self, _within: Duration, _window: usize, + _room: Option, ) -> Result, AircError> { Ok(self .members @@ -650,6 +651,7 @@ mod tests { &self, _within: Duration, _window: usize, + _room: Option, ) -> Result, AircError> { Ok(self.members.clone()) } diff --git a/core/continuum-core/src/persona/airc_citizen.rs b/core/continuum-core/src/persona/airc_citizen.rs index ecec2a17d..51e5717f3 100644 --- a/core/continuum-core/src/persona/airc_citizen.rs +++ b/core/continuum-core/src/persona/airc_citizen.rs @@ -322,6 +322,7 @@ impl crate::persona::room_roster_source::AircRosterReader for StubAircCitizen { &self, _within: std::time::Duration, _window: usize, + _room: Option, ) -> Result, AircError> { // No daemon in tests → no presence. RAG runs through cleanly // with an empty roster (no [Present in this room] block). @@ -333,6 +334,7 @@ impl crate::persona::room_roster_source::AircRosterReader for StubAircCitizen { impl crate::persona::room_doctrine_source::AircDoctrineReader for StubAircCitizen { async fn room_doctrine( &self, + _room: Option, ) -> Result, AircError> { // No daemon in tests → no published doctrine. Cognition runs // through cleanly with no [Room operating doctrine] block. diff --git a/core/continuum-core/src/persona/airc_runtime.rs b/core/continuum-core/src/persona/airc_runtime.rs index 27ba5e9fb..977b9c896 100644 --- a/core/continuum-core/src/persona/airc_runtime.rs +++ b/core/continuum-core/src/persona/airc_runtime.rs @@ -1126,8 +1126,15 @@ impl crate::persona::room_roster_source::AircRosterReader for PersonaAircRuntime &self, within: std::time::Duration, window: usize, + room: Option, ) -> Result, AircError> { - self.airc.room_roster(within, window).await + crate::persona::room_roster_source::AircRosterReader::room_roster( + self.airc.as_ref(), + within, + window, + room, + ) + .await } } @@ -1135,8 +1142,13 @@ impl crate::persona::room_roster_source::AircRosterReader for PersonaAircRuntime impl crate::persona::room_doctrine_source::AircDoctrineReader for PersonaAircRuntime { async fn room_doctrine( &self, + room: Option, ) -> Result, AircError> { - self.airc.room_doctrine().await + crate::persona::room_doctrine_source::AircDoctrineReader::room_doctrine( + self.airc.as_ref(), + room, + ) + .await } } diff --git a/core/continuum-core/src/persona/prompt_assembly.rs b/core/continuum-core/src/persona/prompt_assembly.rs index 15ba932df..81de46034 100644 --- a/core/continuum-core/src/persona/prompt_assembly.rs +++ b/core/continuum-core/src/persona/prompt_assembly.rs @@ -1044,6 +1044,7 @@ mod tests { &self, _within: Duration, _window: usize, + _room: Option, ) -> Result, AircError> { Ok(vec![RoomMember { peer_id: self.other, diff --git a/core/continuum-core/src/persona/room_doctrine_source.rs b/core/continuum-core/src/persona/room_doctrine_source.rs index e9c759193..7bb82af25 100644 --- a/core/continuum-core/src/persona/room_doctrine_source.rs +++ b/core/continuum-core/src/persona/room_doctrine_source.rs @@ -55,17 +55,40 @@ use crate::cognition::token_budget::estimate_prompt_tokens as estimate_tokens; /// daemon. Mirrors the `AircRosterReader` rail. #[async_trait] pub trait AircDoctrineReader: Send + Sync { - /// The latest published operating doctrine for this persona's - /// current room, or `None` if none has been published. - async fn room_doctrine(&self) -> Result, AircError>; + /// The latest published operating doctrine for a NAMED room, or `None` if + /// none has been published there. + /// + /// ROOM-PARAMETRIC (#443, measured live 2026-08-17). This took no `room` and + /// read whatever the scope's default subscription happened to be, so the + /// source could only be BOUND at bootstrap and had to abstain on any other + /// turn room — measured 39 abstains in 90 minutes with bound=academy, + /// turn=bench-room. A citizen answering a turn in a per-run bench room got + /// NO operating doctrine: she was handed the room's work with none of the + /// room's rules. + /// + /// `airc_lib::room_doctrine_in` already existed for exactly this, and its + /// own doc names the defect verbatim — *"A citizen who belongs to several + /// rooms answers a turn in the room it arrived in; reading doctrine from her + /// default instead grounds that answer in another room's rules."* The + /// capability was built and never wired. This is the wire. + /// + /// `None` keeps the pre-#443 behaviour exactly (the scope's current room), + /// which is what an UNSTAMPED context (background consolidation) still wants. + async fn room_doctrine( + &self, + room: Option, + ) -> Result, AircError>; } /// `airc_lib::Airc` satisfies the reader contract directly. Orphan rule /// OK — the trait is ours. #[async_trait] impl AircDoctrineReader for airc_lib::Airc { - async fn room_doctrine(&self) -> Result, AircError> { - airc_lib::Airc::room_doctrine(self).await + async fn room_doctrine( + &self, + room: Option, + ) -> Result, AircError> { + airc_lib::Airc::room_doctrine_in(self, room.map(airc_core::RoomId::from_uuid)).await } } @@ -169,13 +192,30 @@ impl RagSource for RoomDoctrineSource { if ctx.persona_id != self.persona_id { return empty(ResolutionPreference::Placeholder); } - // Room-scoped: the ONE shared gate (`room_scope_allows`) — probes every - // abstain with both rooms named (see RoomBoardSource for the rationale). - if !crate::persona::rag_budget::room_scope_allows(self.room_id, ctx, SOURCE_ID) { - return empty(ResolutionPreference::Placeholder); - } + // Room resolution — TURN-PARAMETRIC (#443), the same shape RoomBoardSource + // already uses and for the same reason: the rules she needs are the rules + // OF THE ROOM SHE IS STANDING IN. The stamped turn room wins; the bound + // room is only the fallback for UNSTAMPED contexts (background + // consolidation, legacy construction — pre-gate behaviour, unchanged). + // A synthetic nil room still gets NOTHING and does NOT fall back (the + // exam-bleed pin). + let effective_room = match ctx.airc_room.as_ref().map(|r| r.as_uuid()) { + Some(t) if t.is_nil() => { + crate::probe!( + class = "rag.room_gate.abstain", + source = SOURCE_ID, + bound_room = ?self.room_id, + turn_room = %t, + persona_id = %ctx.persona_id, + "synthetic nil-room context — no doctrine, and no fallback to the bound room" + ); + return empty(ResolutionPreference::Placeholder); + } + Some(t) => Some(t), + None => self.room_id, + }; - let card = match self.reader.room_doctrine().await { + let card = match self.reader.room_doctrine(effective_room).await { Ok(Some(card)) => card, // No doctrine published for this room → no block (normal). Ok(None) => return empty(resolution), @@ -258,6 +298,8 @@ mod tests { struct StubReader { doctrine: Option, fail: Mutex, + /// The room the source last ASKED for — #443's regression pins it. + asked_room: Mutex>>, } impl StubReader { @@ -265,8 +307,12 @@ mod tests { Self { doctrine, fail: Mutex::new(false), + asked_room: Mutex::new(None), } } + fn asked_room(&self) -> Option> { + *self.asked_room.lock().unwrap() + } fn set_fail(&self, fail: bool) { *self.fail.lock().unwrap() = fail; } @@ -274,7 +320,11 @@ mod tests { #[async_trait] impl AircDoctrineReader for StubReader { - async fn room_doctrine(&self) -> Result, AircError> { + async fn room_doctrine( + &self, + room: Option, + ) -> Result, AircError> { + *self.asked_room.lock().unwrap() = Some(room); if *self.fail.lock().unwrap() { return Err(AircError::UnknownPeer(PeerId::new())); } @@ -282,6 +332,53 @@ mod tests { } } + // what this catches: #443 — a citizen taking a turn in a per-run BENCH room + // was handed the ACADEMY's rules, or (before the room-parametric reader) none + // at all. Measured live 2026-08-17: 39 `rag.room_gate.abstain` rows in 90 + // minutes with bound=academy, turn=bench-room, so every bench turn ran with no + // operating doctrine. The reader is now asked for the room she is STANDING in. + #[tokio::test] + async fn the_turn_room_wins_over_the_bound_room() { + let bound = uuid::Uuid::new_v4(); + let turn = uuid::Uuid::new_v4(); + let reader = Arc::new(StubReader::new(Some(card("be excellent")))); + let src = RoomDoctrineSource::new(persona(), reader.clone()).for_room(bound); + + let ctx = RagContext::for_persona_in_room(persona(), 1_000_000, turn); + let out = src.deliver(&ctx, 4096, ResolutionPreference::Raw).await; + + assert_eq!( + reader.asked_room(), + Some(Some(turn)), + "the reader must be asked for the TURN room, not the bound room" + ); + assert!( + !out.items.is_empty(), + "a stamped turn in another room must still receive doctrine — abstaining \ + here is exactly the #443 defect (she got the work, not the rules)" + ); + } + + // what this catches: the exam-bleed pin must survive the #443 change — a + // synthetic nil room gets NOTHING and must NOT silently fall back to the bound + // room, or an eval fork would read the live room's doctrine. + #[tokio::test] + async fn a_nil_room_gets_nothing_and_never_falls_back() { + let bound = uuid::Uuid::new_v4(); + let reader = Arc::new(StubReader::new(Some(card("be excellent")))); + let src = RoomDoctrineSource::new(persona(), reader.clone()).for_room(bound); + + let ctx = RagContext::for_persona_in_room(persona(), 1_000_000, uuid::Uuid::nil()); + let out = src.deliver(&ctx, 4096, ResolutionPreference::Raw).await; + + assert!(out.items.is_empty(), "a nil-room context must receive no doctrine"); + assert_eq!( + reader.asked_room(), + None, + "the reader must not be consulted at all for a nil room" + ); + } + // what this catches: a published doctrine surfaces as a delivery the // service loop can route into the [Room operating doctrine] grounding // block — the fix for a persona ignoring the room's nature. diff --git a/core/continuum-core/src/persona/room_roster_source.rs b/core/continuum-core/src/persona/room_roster_source.rs index 4f14fbac4..5a190a350 100644 --- a/core/continuum-core/src/persona/room_roster_source.rs +++ b/core/continuum-core/src/persona/room_roster_source.rs @@ -105,10 +105,18 @@ pub trait AircRosterReader: Send + Sync { /// `within`, over the most recent `window` transcript events. /// Newest-wins per peer; peers that signalled `Leaving` are excluded /// by airc; `display_name` is `None` for a present-but-unnamed peer. + /// ROOM-PARAMETRIC (#443, measured live 2026-08-17). This took no `room`, so + /// the source could only be BOUND at bootstrap and had to abstain on any + /// other turn room — measured 42 abstains in 90 minutes with bound=academy, + /// turn=bench-room. A citizen answering a turn in a per-run bench room saw + /// NO ONE: no teammates, no peers to address, in the room she was actually + /// standing in. `airc_lib::room_roster_in` already existed; it was never + /// wired. `None` keeps the pre-#443 behaviour (the scope's current room). async fn room_roster( &self, within: Duration, window: usize, + room: Option, ) -> Result, AircError>; /// The CARDS-flavored roster (#262): presence + each peer's FULL @@ -121,9 +129,10 @@ pub trait AircRosterReader: Send + Sync { &self, within: Duration, window: usize, + room: Option, ) -> Result, AircError> { Ok(self - .room_roster(within, window) + .room_roster(within, window, room) .await? .into_iter() .map(|m| airc_lib::RoomMemberCard { @@ -149,16 +158,25 @@ impl AircRosterReader for airc_lib::Airc { &self, within: Duration, window: usize, + room: Option, ) -> Result, AircError> { - airc_lib::Airc::room_roster(self, within, window).await + airc_lib::Airc::room_roster_in(self, room.map(airc_core::RoomId::from_uuid), within, window) + .await } async fn room_roster_cards( &self, within: Duration, window: usize, + room: Option, ) -> Result, AircError> { - airc_lib::Airc::room_roster_cards(self, within, window).await + airc_lib::Airc::room_roster_cards_in( + self, + room.map(airc_core::RoomId::from_uuid), + within, + window, + ) + .await } } @@ -303,22 +321,41 @@ impl RagSource for RoomRosterSource { resolution_used: ResolutionPreference::Placeholder, }; } - // Room-scoped: the ONE shared gate (`room_scope_allows`) — probes every - // abstain with both rooms named (see RoomBoardSource for the rationale). - if !crate::persona::rag_budget::room_scope_allows(self.room_id, ctx, SOURCE_ID) { - return RagDelivery { - source_id: SOURCE_ID.to_string(), - items: Vec::new(), - tokens_used: 0, - continuation: None, - resolution_used: ResolutionPreference::Placeholder, - }; - } + // Room resolution — TURN-PARAMETRIC (#443), the same shape RoomBoardSource + // uses. The peers she needs to see are the peers OF THE ROOM SHE IS + // STANDING IN; the stamped turn room wins, the bound room is only the + // fallback for UNSTAMPED contexts. A synthetic nil room gets NOTHING and + // does NOT fall back (the exam-bleed pin). + let effective_room = match ctx.airc_room.as_ref().map(|r| r.as_uuid()) { + Some(t) if t.is_nil() => { + crate::probe!( + class = "rag.room_gate.abstain", + source = SOURCE_ID, + bound_room = ?self.room_id, + turn_room = %t, + persona_id = %ctx.persona_id, + "synthetic nil-room context — no roster, and no fallback to the bound room" + ); + return RagDelivery { + source_id: SOURCE_ID.to_string(), + items: Vec::new(), + tokens_used: 0, + continuation: None, + resolution_used: ResolutionPreference::Placeholder, + }; + } + Some(t) => Some(t), + None => self.room_id, + }; // ONE airc call returns presence joined with display names // (airc#1232). A failure is non-fatal — empty delivery, cognition // stays up (good-citizen doctrine). - let members = match self.reader.room_roster(PRESENCE_WINDOW, ROSTER_SCAN).await { + let members = match self + .reader + .room_roster(PRESENCE_WINDOW, ROSTER_SCAN, effective_room) + .await + { Ok(m) => m, Err(err) => { tracing::warn!( @@ -428,6 +465,9 @@ mod tests { self_peer: PeerId, members: Vec, fail: Mutex, + /// The room the source last ASKED for. #443's regression pins this: + /// asking for the wrong room is invisible when the stub ignores it. + asked_room: Mutex>>, } impl StubReader { @@ -436,8 +476,12 @@ mod tests { self_peer, members, fail: Mutex::new(false), + asked_room: Mutex::new(None), } } + fn asked_room(&self) -> Option> { + *self.asked_room.lock().unwrap() + } fn set_fail(&self, fail: bool) { *self.fail.lock().unwrap() = fail; } @@ -452,7 +496,9 @@ mod tests { &self, _within: Duration, _window: usize, + room: Option, ) -> Result, AircError> { + *self.asked_room.lock().unwrap() = Some(room); if *self.fail.lock().unwrap() { return Err(AircError::UnknownPeer(PeerId::new())); } From fe4db280569c36313d57814ac08d9e4a14af9a30 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 17:05:52 -0500 Subject: [PATCH 14/80] docs(observations): 2026-08-17 bench rounds + the room gate that made them blind Dated post-run card per the standing instruction. Records what was dispatched, the fan-out measurement (24 turn rows = 2 events), the #443 abstain table, the operator's inability to cancel a round it can create (#27/#371), the third unwired instance (wall_source, blocked on an airc-side RoomId seam), and the four method errors I made. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- ...-17T2230Z-bench-round-and-the-room-gate.md | 110 ++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 docs/observations/2026-08-17T2230Z-bench-round-and-the-room-gate.md diff --git a/docs/observations/2026-08-17T2230Z-bench-round-and-the-room-gate.md b/docs/observations/2026-08-17T2230Z-bench-round-and-the-room-gate.md new file mode 100644 index 000000000..decc0e3a2 --- /dev/null +++ b/docs/observations/2026-08-17T2230Z-bench-round-and-the-room-gate.md @@ -0,0 +1,110 @@ +# 2026-08-17 — two bench rounds, and the room gate that made them blind + +Written after the rounds, per the standing instruction: a new dated observation file after +every run, and no reading of old ones as if they were current. + +## What was run + +| when (CDT) | what | receipt | +|---|---|---| +| 16:00 | `benchmark/dispatch --name=hard-rs` | 3 cards, 3 kickoffs, room `bench-hard-rs-1787000446` | +| 16:08 | `benchmark/dispatch --name=swe-bench-lite --limit=12` | 12 cards, 12 kickoffs, room `bench-swe-bench-lite-1787000921`, 2 astropy env errors | + +Both were dispatched by me out of the order the plan file specifies (Phase 4 gated behind +Phases 0–2). The `hard-rs` one was also the wrong suite — Joel asked for SWE. **Neither +round produced a grade.** 15 cards remain on the board; the operator has no verb to recall +them (see "what the operator cannot do" below). + +## What the rounds measured + +**Serving:** ready, `lanes: 1`, window 36,608, Devstral-Small-2507 + a Qwen2.5-VL-7B lane. +24 personas hosted (`persona/roster` key is `citizens`, not `personas` — my first parse +reported 0 and I nearly filed a false #437). + +**The fan-out.** 24 `persona.turn.start` rows in 30 minutes are **2 events**, not 24 +workers: + +| lamport | room | personas woken | +|---|---|---| +| …301506 | hard-rs | 5 | +| …301789 | swe-bench-lite | 19 | + +All 24 carry the same `peer_id`. One message lands in a bench room and every member takes +a turn on it, all queued on one lane. 21:09:04 is 23 seconds after the dispatch. + +**Acts in 90 minutes:** 28 rows — `code/shell` 4, `code/write` 4, `code/run` 3, +`work/claim` 3, `work/get` 2, `code/read` 2, `code/edit` 1, `work/list` 1, +`code/git/status` 1, `code/list` 1, `commands/help` 1. Arjun read a card and wrote a patch +in-room (`wrote=true`). So the loop actuates. `work.card` state-changes: **0**. + +## The defect: room-scoped grounding abstains in a bench room (#443) + +`rag.room_gate.abstain`, 90 minutes, `bound_room` → `turn_room`: + +| source | bound | turn | n | +|---|---|---|---| +| roster | academy | swe-bench | 23 | +| room-doctrine | academy | swe-bench | 19 | +| roster | academy | hard-rs | 3 | +| room-doctrine | academy | hard-rs | 3 | + +A citizen answering in a per-run bench room received the cards and **neither the peers nor +the room's operating rules**. + +`RoomBoardSource` was already turn-parametric (#443's first half). Roster and doctrine were +not — they went through `rag_budget::room_scope_allows`, which abstains whenever +bound ≠ turn. + +**The capability existed and was never called.** `airc_lib` has `room_roster_in`, +`room_roster_cards_in`, `room_doctrine_in`. `room_doctrine_in`'s own doc names the defect +verbatim: *"A citizen who belongs to several rooms answers a turn in the room it arrived +in; reading doctrine from her default instead grounds that answer in another room's +rules."* Continuum called the roomless variants. + +Fixed in `2566a09d6`: both readers take `room: Option`, both sources resolve +`turn_room.or(bound)` exactly as the board source does, `None` keeps pre-#443 behaviour for +unstamped contexts, and the exam-bleed nil-room pin is now held by an executing test rather +than by the gate's side effect. + +**STILL OPEN — the third instance.** `wall_source` has the same shape. airc has +`wall_posts_in(&Room, category)`, but it takes a full `Room` struct and airc exposes no +id→Room resolver (`current_room()` only), so wiring it needs an airc-side seam keyed by +`RoomId`. Not done. `viewstate_rag` also calls `room_scope_allows` (2 sites) and was not +examined. + +## What the operator cannot do + +`activity/archive` and every `work/*` verb refuse the substrate-local operator — *"activity +verbs act as the caller's own airc identity, and the substrate-local operator has none +in-core (the self-peer gap, task #27)"*. So `benchmark/dispatch` can CREATE a round from +the CLI and nothing can cancel one. That is #371's "a round has no END" with a sharper +edge: it has no **abort**. The 15 cards from today are there until their claims lapse. + +## Also landed today + +- `cabecb9c4` — one era-pinned uv install path. The dependency-sdist pre-install had no + heal loop while the `-e .` install did, so every astropy instance died on a hint the code + 30 lines below knew how to parse. Same "one decision, several sites, correct at one" + shape as #443. +- `4323ac158` (deployed earlier) — board render cap. **Live-verified across three samples + and two citizens:** board share of the system prompt 65.4% → 17.4% / 29.1% / 26.7%, + cards 90 → 8–12. + +## Method errors I made today, for the record + +1. Counted fan-out rows as workers ("20 personas working"). Two events. +2. Parsed `persona/roster` for a key that doesn't exist and read the empty result as + "0 hosted" — the exact `[[an-absence-is-an-unfinished-measurement]]` failure, caught + only because the probe stream contradicted it. +3. Dispatched two rounds before the phases they depend on, then tried to quiesce with + sleep-mode — which would have treated a symptom, since those citizens were being woken, + not choosing to burn the lane. +4. Ran the runbook's exact command (`benchmark/dispatch --name= --limit=`) only after + being told to open the runbook, which is the first line of my own memory index. + +## Not deployed + +`cabecb9c4` and `2566a09d6` are committed on canary and compile clean; the running core is +`4323ac158`. Neither fix is live. The owed proof for #443 is a reboot followed by +re-reading `rag.room_gate.abstain` for bench-room `turn_room` values — the count should go +to zero for roster and doctrine, and wall should still show them until its seam lands. From 4178d50f7c361650241ccef16961886191835346 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 17:25:43 -0500 Subject: [PATCH 15/80] fix(swe): build tools fall back to UNPINNED when the era pin is unsatisfiable at every heal rung (astropy) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced in isolation 2026-08-17 (uv 0.11, py3.11/arm64) — the astropy sdist build-dep pre-install is unsatisfiable BY CONSTRUCTION under its 2017 era pin, at every rung the heal loop can reach: 1. era markupsafe 1.0 has no wheel for this interpreter/arch → sdist build mandatory → needs setuptools>=40.8.0, which the pin caps at 38.2.4. uv hints `exclude-newer-package`; the heal lifts setuptools — VERIFIED FIRING, correct move. 2. retry: the 2017 sdist now builds against MODERN setuptools → `ImportError: cannot import name 'Feature' from 'setuptools'` (removed in setuptools 46). No hint in that error; the loop correctly stops. 3. the other lift is no better: modern markupsafe (>=2.1) removes `soft_unicode`, which era jinja2 2.10 imports — dies at import time. The only installable combination is modern jinja2 + modern markupsafe (verified importing clean). So: when the healed era-pinned install still fails, retry ONCE with no date pin, loudly. Scope-safe by construction: `era_sdist_build_deps` lists BUILD TOOLS for dependency-sdist code generators (astropy → pyerfa → jinja2), never subject requirements. A repo where jinja2 IS subject code (flask) has no entry in that table, and the subject graph is resolved by the still-date-pinned `-e .` step. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 7f08a2789..e7578e40f 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -792,7 +792,7 @@ pub async fn ensure_env(instance: &SweInstance, repo_dir: &Path) -> Result=40.8.0, and the // heal that reads uv's own `exclude-newer-package` hint lived only at the other site. - let out = era_pinned_uv_install( + let mut out = era_pinned_uv_install( &uv, &py_s, as_of.as_deref(), @@ -801,6 +801,31 @@ pub async fn ensure_env(instance: &SweInstance, repo_dir: &Path) -> Result=40.8.0 which the pin + // excludes; LIFT setuptools (the heal's correct first move, verified firing) and + // the 2017 sdist dies on `ImportError: cannot import name 'Feature'` (removed in + // setuptools 46); lift markupsafe instead and era jinja2 2.10 dies at import on + // `soft_unicode` (removed in markupsafe 2.1). The only installable combination is + // MODERN jinja2 + MODERN markupsafe — verified importing clean. + // + // So when the healed era-pinned install still fails, retry ONCE unpinned, loudly. + // Scope-safe by construction: this table lists BUILD TOOLS for dependency-sdist + // code generators (astropy→pyerfa→jinja2), never subject requirements — a repo + // where the package IS subject code (flask) has no entry here, and the subject + // graph is resolved by the still-date-pinned `-e .` step below. + if !out.status.success() && as_of.is_some() { + tracing::warn!( + instance = %instance.instance_id, + deps = ?sdist_deps, + "era-pinned sdist build-dep install unsatisfiable at every heal rung — \ + retrying UNPINNED (build tools only; the subject graph stays date-pinned)" + ); + out = era_pinned_uv_install(&uv, &py_s, None, sdist_deps, None, &[("CFLAGS", ERA_CFLAGS)]) + .await?; + } if !out.status.success() { // Fail LOUD and leave no half-built env behind — same doctrine as the `-e .` gate. let _ = std::fs::remove_dir_all(&env_dir); From a37ac70fefeb1a5f82a5a41fec7164e1844de6f3 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 18:21:51 -0500 Subject: [PATCH 16/80] feat(bench): dispatch parks on the roster like it parks on serving (#442 roster half, #412) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A dispatch fired inside the post-boot resume window found an EMPTY roster and refused instantly, so the operator hand-rolled a sleep-loop around dispatch — run by hand twice on 2026-08-17, which per the foolproof doctrine is a design defect, not a procedure. The serving half of #442 already parks (`await_ready_serving`). The roster half now parks the same way: dispatch waits (bounded, 180s, 5s poll) ONLY while the roster is empty — citizens being re-hosted after a boot. An unknown NAME against a LIVE roster still fails fast: that error means a typo, never a resume in progress. All existing `resolve_dispatch_roster` semantics (whole-roster default, loud unknown-name listing who is online, Denied on a genuinely empty machine) are unchanged — the loop only delays WHEN the resolver runs, never what it decides. With this, the runbook's manual sequence (ping → serving/status → roster wait → dispatch) collapses into `benchmark/dispatch` itself: serving gate + roster gate are both inside the ONE command. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 35b29568f..023587b33 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -1053,6 +1053,28 @@ impl ActionCommand for BenchmarkDispatch { .to_string(), )); } + // #442 (roster half) + #412: a dispatch fired inside the post-boot resume window + // used to find an EMPTY roster and refuse instantly — so the operator hand-rolled + // a sleep-loop around dispatch (run by hand twice on 2026-08-17; a runbook line + // is a design defect). The serving half of #442 already parks + // (`await_ready_serving` below); the roster half now parks the same way: wait + // ONLY while the roster is EMPTY (citizens are still being re-hosted), bounded. + // An unknown NAME against a LIVE roster still fails fast — that error means a + // typo, never a resume in progress. + const ROSTER_RESUME_WAIT: std::time::Duration = std::time::Duration::from_secs(180); + const ROSTER_RESUME_POLL: std::time::Duration = std::time::Duration::from_secs(5); + let wait_started = std::time::Instant::now(); + while self.registry.roster_snapshot().is_empty() + && wait_started.elapsed() < ROSTER_RESUME_WAIT + { + tracing::info!( + waited_s = wait_started.elapsed().as_secs(), + "dispatch: roster is empty (post-boot resume window, #412) — waiting for \ + citizens to be hosted rather than refusing" + ); + tokio::time::sleep(ROSTER_RESUME_POLL).await; + } + // Resolve the dispatch roster against THIS machine's live citizens (never our // names): empty request → the whole live roster; explicit names → validated or // fail-loud. This is the generalization for all repo users — dispatch targets the From 185816b133fc912de4dad4a797ff97d177146741 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 18:45:36 -0500 Subject: [PATCH 17/80] =?UTF-8?q?fix(work):=20the=20grade=20tail=20cannot?= =?UTF-8?q?=20depend=20on=20wire=20delivery=20=E2=80=94=20work/state=20emi?= =?UTF-8?q?ts=20in-process=20(#450=20tail,=20#434)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED 2026-08-17, the night two solved SWE cards graded as nothing. Atlas ran the full loop on astropy-14182 and astropy-14995 — work/get → shell → read → work/claim → searches → code/write (wrote=true) → shell/test → work/state. Both cards read `Closed owner=Atlas` on the bench room board. Grade receipts under ~/.continuum/progress: frozen at 92, unchanged. Zero `work.card.state_changed.bridged` rows for either. Root cause is the SPINE of the tail, not the grader. `work/state` deliberately emitted NOTHING, on the reasoning that the transition's own transcript echo returns through a persona subscribe stream and `bridge_wire_work_event` (its single caller) publishes exactly once. That is correct when delivery works. Under #434 — post-reboot durable delivery to citizen scopes down — the echo never comes back: `persona.inbound.raw_event` counted 2 rows in 80 minutes against 83 in a comparable earlier window the same day. The bridge never fires, the grader never hears, and finished work grades as nothing. A grade tail whose only feeder is a subsystem with a known-open delivery bug is a grade tail that is down whenever that bug is. So the verb now emits DIRECTLY. It just wrote the transition — it does not need the wire to tell it what it did. The bridge stays for every writer that is NOT this core's work/state (operator CLI, remote peers over the grid), which is exactly why it was added. ONE emitter, not two code paths that each know how to publish: both feeders route through `emit_card_state_changed(payload, via)`, and the (card_id, state) ring inside it makes a transition publish once no matter which feeder sees it first. The wire-event-id ring cannot cover this — the verb path has no wire event — so the dedup key had to move up to the fact itself. `via` is on the probe, so which feeder won is answerable per transition. room_id is the CARD's own room, resolved by walking the subscribed rooms' boards (`card_room_of`, the read-only sibling of `claim_following_card_room`) — NEVER `current_room()`, which is only where the persona happens to be standing. That is the #345 wrong-room trap, and `grade_card`'s own doc names it; I wrote it into this fix first and caught it before commit. An unresolvable room emits empty, so the grader fails loud rather than grading against a guessed board. Regression test pins the cross-feeder dedup (same transition once, different state publishes). cargo check green with metal,accelerate. Not deployed — a round is live on this box; this ships with the next reboot alongside 4178d50f7 (astropy build-dep fallback) and a37ac70fe (dispatch roster wait). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/modules/work.rs | 149 +++++++++++++++++++++--- 1 file changed, 132 insertions(+), 17 deletions(-) diff --git a/core/continuum-core/src/modules/work.rs b/core/continuum-core/src/modules/work.rs index bbd82b771..b7dcabc7b 100644 --- a/core/continuum-core/src/modules/work.rs +++ b/core/continuum-core/src/modules/work.rs @@ -84,6 +84,67 @@ fn wire_card_state_payload(event: &airc_core::TranscriptEvent) -> Option })) } +/// Once-per-process sighting of a (card_id, state) TRANSITION — the cross-path dedup +/// between the in-process verb emit and the wire-echo bridge. +/// +/// # Why two feeders exist at all (2026-08-17, the night two solved cards graded as +/// nothing) +/// +/// `work/state` used to rely ENTIRELY on its own transcript echo returning through a +/// persona subscribe stream (`bridge_wire_work_event`, the only caller). Measured that +/// night: Atlas closed astropy-14182 and astropy-14995 — both `Closed` in the store, +/// on the board — and `persona.inbound.raw_event` counted 2 rows in 80 minutes (a +/// comparable window earlier the same day: 83). #434 (post-reboot durable delivery to +/// citizen scopes down) starves the echo, the bridge never fires, the grader never +/// hears, and finished work grades as nothing. A grade tail that depends on wire +/// delivery working is a grade tail with a known-open bug in its spine. +/// +/// So the VERB now emits directly (in-process, delivery-proof) and the bridge remains +/// for every writer that is NOT this core's `work/state` (operator CLI, remote peers). +/// When delivery works both paths see the same transition; THIS ring makes it publish +/// once. Keyed by (card, state) rather than wire event id because the verb path has no +/// wire event yet. A LEGITIMATE re-transition to the same state hours later would be +/// deduped only if the ring still held it — 256 transitions of churn ages it out long +/// before that matters. +fn first_transition_sighting(card_id: &str, state: &str) -> bool { + use std::collections::VecDeque; + use std::sync::Mutex; + const SEEN_CAP: usize = 256; + static SEEN: OnceLock>> = OnceLock::new(); + let key = format!("{card_id}\u{1}{state}"); + let seen = SEEN.get_or_init(|| Mutex::new(VecDeque::with_capacity(SEEN_CAP))); + let mut seen = seen.lock().unwrap_or_else(|p| p.into_inner()); + if seen.contains(&key) { + return false; + } + if seen.len() >= SEEN_CAP { + seen.pop_front(); + } + seen.push_back(key); + true +} + +/// Publish one card-state transition onto the internal bus — the ONE emitter both +/// feeders (the `work/state` verb, the wire bridge) route through. Dedup lives HERE so +/// neither feeder needs to know the other exists. +pub(crate) async fn emit_card_state_changed(payload: Value, via: &'static str) { + let card_id = payload["card_id"].as_str().unwrap_or("").to_string(); + let state = payload["state"].as_str().unwrap_or("").to_string(); + if !first_transition_sighting(&card_id, &state) { + return; + } + crate::probe!( + class = "work.card.state_changed.bridged", + via = via, + card_id = %card_id, + state = %state, + "card-state transition published onto the internal bus" + ); + if let Some((bus, registry)) = WORK_EVENT_BUS.get() { + bus.publish(WORK_CARD_STATE_CHANGED, payload, registry).await; + } +} + /// Once-per-process sighting of a wire event id. Every resident persona's subscribe /// stream yields the SAME room event once, so the bridge below would otherwise /// publish N copies for N residents; first sighting wins. Bounded ring — old ids @@ -122,17 +183,7 @@ pub async fn bridge_wire_work_event(event: &airc_core::TranscriptEvent) { if !first_sighting(event.event_id.as_uuid()) { return; } - crate::probe!( - class = "work.card.state_changed.bridged", - event_id = %event.event_id.as_uuid(), - room_id = %event.room_id.as_uuid(), - card_id = %payload["card_id"], - state = %payload["state"], - "wire card-state transition bridged onto the internal bus" - ); - if let Some((bus, registry)) = WORK_EVENT_BUS.get() { - bus.publish(WORK_CARD_STATE_CHANGED, payload, registry).await; - } + emit_card_state_changed(payload, "wire-echo").await; } /// Default claim lease (ms) — 30 min — when the caller doesn't set one. The claim @@ -302,6 +353,32 @@ fn parse_state(s: &str) -> Result { // ─────────────────────────── work/claim ────────────────────────── +/// The room whose board actually HOLDS `card_id`, searched across this scope's +/// subscribed rooms — same walk as [`claim_following_card_room`], read-only. +/// +/// Exists for the `work/state` direct emit: boards are per-room and the grade +/// subscriber refuses an event with no room (the #345 wrong-room trap is a loud +/// error now, not a silent misread), so the verb must name the CARD's room — +/// never `current_room()`, which is merely where the persona is standing. +async fn card_room_of(airc: &Arc, card_id: WorkCardId) -> Option { + let set = airc.subscription_set().await.ok()?; + for sub in set.all() { + let room = sub.as_room(); + let Ok(board) = airc.work_board_in(&room).await else { + continue; + }; + if board + .snapshot() + .cards + .iter() + .any(|c| c.card_id == card_id) + { + return Some(room.channel.as_uuid()); + } + } + None +} + /// Locate `card_id` on the board of one of the caller's OTHER subscribed rooms, /// switch her current room there, and retry the claim once. /// @@ -870,12 +947,29 @@ impl ActionCommand for WorkState { .await .map_err(|e| CommandError::Internal(e.to_string()))?; - // NO bus emit here — the transition's transcript echo comes back on this - // persona's own subscribe stream and `bridge_wire_work_event` publishes - // [`WORK_CARD_STATE_CHANGED`] exactly once. One fact, one emitter: emitting - // from the verb too would double-fire every subscriber, and verb-side-only - // (the old shape) left cards closed by the airc CLI or a remote peer - // changing the board while grading nothing. + // DIRECT emit — the delivery-proof feeder. The previous shape relied + // entirely on this transition's transcript echo returning through a persona + // subscribe stream; under #434 (post-reboot durable delivery down) that echo + // never arrives and finished work grades as NOTHING (measured 2026-08-17: + // two cards Closed, zero grades, 2 raw events in 80 min). The verb KNOWS the + // transition happened — it just wrote it — so it publishes in-process. The + // wire bridge still covers external writers (operator CLI, remote peers); + // `emit_card_state_changed`'s (card,state) ring makes the two feeders + // publish once when both fire. + let room_id = card_room_of(&airc, card_id) + .await + .map(|r| r.to_string()) + .unwrap_or_default(); + emit_card_state_changed( + serde_json::json!({ + "card_id": card_id.as_uuid().to_string(), + "state": serde_json::to_value(state) + .unwrap_or(serde_json::Value::Null), + "room_id": room_id, + }), + "work-state-verb", + ) + .await; Ok(WorkStateResult { card_id: p.card_id, @@ -1454,6 +1548,27 @@ mod tests { assert_eq!(WORK_CARD_STATE_CHANGED, "work.card.state_changed"); } + // what this catches: the grade tail now has TWO feeders — the `work/state` verb + // (in-process, delivery-proof) and the wire echo bridge (external writers). They + // both see the same transition whenever wire delivery is healthy, and a subscriber + // that grades twice would write two receipts for one card. The (card,state) ring is + // the ONLY thing making that one publish; the wire-event-id ring cannot cover it + // because the verb path has no wire event. Regression for the 2026-08-17 grade tail + // fix (two Closed cards, zero grades, #434 starving the single wire feeder). + #[test] + fn one_transition_publishes_once_no_matter_which_feeder_sees_it_first() { + let card = uuid::Uuid::new_v4().to_string(); + assert!(first_transition_sighting(&card, "closed"), "first sighting publishes"); + assert!( + !first_transition_sighting(&card, "closed"), + "the second feeder for the SAME transition must not publish again" + ); + assert!( + first_transition_sighting(&card, "merged"), + "a DIFFERENT state on the same card is a different transition and must publish" + ); + } + /// Build a wire transcript event through the REAL producer (`encode_work_event`), /// so a header/codec contract drift fails these tests instead of shipping. fn wire_work_event(event: airc_work::WorkEvent, event_id: u128) -> airc_core::TranscriptEvent { From 99c59fe44c87adb4bbbe68ce3eea535663bb51d6 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 19:12:10 -0500 Subject: [PATCH 18/80] =?UTF-8?q?fix(swe):=20grade=20clones=20through=20a?= =?UTF-8?q?=20staging=20path=20=E2=80=94=20a=20.DS=5FStore=20must=20not=20?= =?UTF-8?q?void=20a=20score=20(#380=20family)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED 2026-08-17. A re-grade of astropy-14995 died with: clone of astropy/astropy from its local mirror failed: fatal: destination path '…/benchmarks/swe/work/astropy__astropy-14995/repo' already exists and is not an empty directory `clone_at` removes that directory immediately before cloning, so the message reads like a harness bug. It is not. The directory's ONLY content was a `.DS_Store`, stamped AFTER the remove — Finder/Spotlight noticed the parent and repopulated it inside the window between `remove_dir_all` and `git clone`. `git clone` refuses any non-empty destination, so an OS indexer can void a grade, on any repo, at random, and the receipt cannot tell you that is what happened. Fix: clone into a pid-suffixed STAGING path and move it into place. The window closes because the destination is only touched at the rename, and anything that reappeared there meanwhile loses to the tree we just built. Bonus property from the same shape: a half-fetched tree is never visible at `repo_dir` — the move is the commit point. Also stop swallowing the pre-clone removal error (`let _ = remove_dir_all`). If the stale grade tree cannot be cleared, that is a filesystem fault and it now says so by name, instead of surfacing three lines later as git's confusing "already exists" — the same absence-vs-fault confusion this whole grade tail keeps producing. No test: the failure is a filesystem race with a macOS daemon, and the honest reproduction needs a network mirror + Finder. The staging shape is the fix; the comment carries the measurement. Found while grading two closed astropy cards after the grade-tail fix (185816b13). Second finding from that same pass, NOT fixed here and bigger: astropy-14182 grades UNGRADEABLE — PASS_TO_PASS scores 0 of 9 on the PRISTINE tree, so the harness correctly refuses to report (gateOk:false). That is #380 (gold-gate every env class): no astropy instance can score at all until its env builds, and a citizen's patch there is unmeasurable, not wrong. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 45 ++++++++++++++++++- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index e7578e40f..32bcc508e 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -377,8 +377,14 @@ pub(crate) async fn run_env( /// Clone the repo at `base_commit`. The commit is the whole point — a clone left at HEAD is /// how eight runs got scored against a tree with the fix already in it. pub async fn clone_at(instance: &SweInstance, repo_dir: &Path) -> Result<(), String> { + // A stale tree here is not "probably fine" — it is the tree the score comes from. Removal + // failing used to be SWALLOWED (`let _ =`), and the clone below then died on git's own + // "destination path already exists and is not an empty directory", which reads like a + // harness bug rather than a filesystem one. Say which it is. if repo_dir.exists() { - let _ = std::fs::remove_dir_all(repo_dir); + std::fs::remove_dir_all(repo_dir).map_err(|e| { + format!("could not clear the stale grade tree {}: {e}", repo_dir.display()) + })?; } // The PARENT must exist first. `git clone` creates its target directory but not the chain // above it, and the failure surfaces late and cryptically — as a mid-fetch "unable to write @@ -429,6 +435,26 @@ pub async fn clone_at(instance: &SweInstance, repo_dir: &Path) -> Result<(), Str } } + // Clone into a STAGING path and move it into place, never straight into `repo_dir`. + // + // `git clone` refuses any destination that is not empty, and on macOS `repo_dir` does not + // stay empty on its own: Finder / Spotlight drop a `.DS_Store` into a directory the instant + // they notice it. Measured 2026-08-17 — a re-grade of astropy-14995 failed with + // "destination path already exists and is not an empty directory" against a directory whose + // ONLY content was a `.DS_Store` stamped AFTER the remove above. The window between + // remove and clone was enough. A grade voided by an OS indexer is indistinguishable, from + // the receipt, from a grade voided by the harness — so close the window instead of + // widening the diagnosis. Staging + rename also means a half-fetched tree is never + // visible at `repo_dir`: the move is the commit point. + let staging = repo_dir.with_extension(format!( + "cloning-{}", + std::process::id() + )); + if staging.exists() { + std::fs::remove_dir_all(&staging).map_err(|e| { + format!("could not clear the stale staging tree {}: {e}", staging.display()) + })?; + } let out = run( "git", &[ @@ -436,18 +462,33 @@ pub async fn clone_at(instance: &SweInstance, repo_dir: &Path) -> Result<(), Str "--quiet", "--shared", &mirror.to_string_lossy(), - &repo_dir.to_string_lossy(), + &staging.to_string_lossy(), ], None, ) .await?; if !out.status.success() { + let _ = std::fs::remove_dir_all(&staging); return Err(format!( "clone of {} from its local mirror failed: {}", instance.repo, String::from_utf8_lossy(&out.stderr).trim() )); } + // Anything that reappeared at the destination while we fetched (see above) loses to the + // tree we just built. + if repo_dir.exists() { + std::fs::remove_dir_all(repo_dir).map_err(|e| { + format!("could not clear {} before staging the fresh clone: {e}", repo_dir.display()) + })?; + } + std::fs::rename(&staging, repo_dir).map_err(|e| { + format!( + "could not move the fresh clone {} into place at {}: {e}", + staging.display(), + repo_dir.display() + ) + })?; let out = run( "git", &["checkout", "--quiet", &instance.base_commit], From b0141e810ad2280a0db59676e388d9976aacdb3f Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 19:28:18 -0500 Subject: [PATCH 19/80] =?UTF-8?q?fix(swe):=20install=20the=20repo's=20own?= =?UTF-8?q?=20TEST=20extra=20=E2=80=94=20a=20suite=20that=20cannot=20COLLE?= =?UTF-8?q?CT=20scores=200/N=20(#380)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED end-to-end 2026-08-17. The gold gate said astropy-14182 was UNGRADEABLE: "PASS_TO_PASS passes 0 of 9 on the PRISTINE tree". Not the citizen's patch — the DATASET'S OWN GOLD PATCH could not be scored. Running the suite by hand gave the reason in one line: ImportError while loading conftest '…/repo/conftest.py'. conftest.py:9: in import hypothesis E ModuleNotFoundError: No module named 'hypothesis' The env installs `-e .`, which resolves RUNTIME dependencies. `hypothesis` is a TEST dependency, and astropy's conftest imports it at module scope — so pytest dies loading conftest, before collecting a single test. An instance whose env is missing its suite's deps scores 0/N on a PRISTINE tree, the gate correctly refuses to report, and nothing that instance's solver writes can matter. The repo declares exactly what its suite needs in `[project.optional-dependencies]`; we simply never asked for it. POSITIVE CONTROL, run before writing the fix — same env, same pristine tree, after installing `.[test]` with the era pin and the same ERA_CFLAGS the code passes: 1 failed, 9 passed FAILED …test_rst.py::test_rst_with_header_rows — TypeError: RST.__init__() got an unexpected keyword argument 'header_rows' 9/9 PASS_TO_PASS green and the ONE failure is the instance's FAIL_TO_PASS test — the bug the instance exists to fix. That is the gold-gate condition exactly. SCOPE, measured rather than assumed (a `pytest --collect-only` sweep across all 40 envs with a staged tree): every one of them collects today, so this is NOT a whole-suite outage. Per-env it is luck: of four astropy envs, 12907 HAS hypothesis and 14182 did not, and 14365/14995 still do not. Nothing designed that difference — an era resolve happened to pull it for one instance and not another. The fix's value is turning that coincidence into a guarantee, and the honest claim is "an env missing its suite's deps is unscoreable", not "astropy is broken by construction". An earlier draft of this message overstated it as every astropy instance; the sweep disproved that before it shipped. `test_extra_name` picks the group the repo declares, preferring test > tests > testing > dev. The order is load-bearing: a repo declaring both `dev` and `test` must get `test`, or we install a kitchen-sink group and inherit its resolution failures. No pyproject, no optional-dependencies, or no test-shaped group all mean "install as before" — never "install something plausible". `ensure_test_extra` is NON-FATAL by design and runs LAST. Everything before it decides whether the env can BUILD and RUN a harness; this decides whether the suite can COLLECT. A repo whose suite already collects does not need it, and a resolve failure must not delete an env that grades fine today — so a failure probes (`swe.env.test_extra`, installed=false) and leaves the env standing. It also runs on the CACHED path, marker-guarded, because every env on this box was built before this existed; without the heal, an env that happens to lack its suite's deps stays that way forever and the operator has to know which ones. The marker makes it once-only, including the "this repo declares nothing" answer. Extracted `era_cutoff` while here: the cached path needs the same "what era is this instance" answer the build path computed inline, and two copies is how they drift. Deploys with 185816b13 (grade tail) and 99c59fe44 (clone staging) on the next reboot. #380 stays open until the gold gate passes per env CLASS — that is the instrument, and one instance passing is not a class passing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 148 +++++++++++++++++- 1 file changed, 143 insertions(+), 5 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 32bcc508e..417238707 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -581,6 +581,101 @@ pub fn interpreter_for_year(year: u32) -> &'static str { /// `pyproject.toml`. Empty when there is no pyproject, no `[build-system]` table, or no /// `requires` array. These must be pre-installed into the venv when building with /// `--no-build-isolation` (see the call site) — pip won't fetch them for us in that mode. +/// The name of the repo's own TEST extra, from `[project.optional-dependencies]` — the +/// group whose contents the suite needs to so much as COLLECT. +/// +/// Why this exists (measured 2026-08-17, and it is the whole of astropy's 0-of-9): the env +/// installs `-e .`, which resolves RUNTIME dependencies only. astropy's `conftest.py` opens +/// with `import hypothesis`, a TEST dependency, so pytest dies loading conftest before it +/// reaches a single test: +/// +/// ```text +/// ImportError while loading conftest '…/repo/conftest.py'. +/// conftest.py:9: in +/// import hypothesis +/// E ModuleNotFoundError: No module named 'hypothesis' +/// ``` +/// +/// PASS_TO_PASS then reads 0/9 on the PRISTINE tree, the gold gate correctly refuses to +/// score, and EVERY astropy instance is ungradeable — a ceiling that has nothing to do with +/// the citizen's patch. The repo already declares exactly what its suite needs; we simply +/// never asked for it. +/// +/// Names are not standardised, so prefer in the order the ecosystem actually uses them and +/// take the first that the repo declares. Returns `None` when there is no pyproject, no +/// optional-dependencies table, or no test-shaped group — those repos install as before. +/// The `--exclude-newer` cutoff for an instance: its own creation date, or `None` when the +/// dataset carries no date (which disables pinning AND healing — see `era_pinned_uv_install`). +/// Extracted because BOTH the fresh-build path and the cached-env heal need the same answer, +/// and two copies of "what era is this instance" is exactly how the two drift apart. +fn era_cutoff(instance: &SweInstance) -> Option { + if instance.created_at.is_empty() { + None + } else { + Some(instance.created_at.clone()) + } +} + +fn test_extra_name(repo_dir: &Path) -> Option { + const PREFERRED: [&str; 4] = ["test", "tests", "testing", "dev"]; + let text = std::fs::read_to_string(repo_dir.join("pyproject.toml")).ok()?; + let parsed: toml::Value = toml::from_str(&text).ok()?; + let groups = parsed + .get("project")? + .get("optional-dependencies")? + .as_table()?; + PREFERRED + .iter() + .find(|name| groups.contains_key(**name)) + .map(|name| (*name).to_string()) +} + +/// Install the repo's test extra into an existing env, once. Non-fatal BY DESIGN: a repo +/// whose suite already collects does not need it, and a resolve failure here must not +/// delete an env that grades fine today. The marker file is what makes it once-only, and +/// what lets envs built BEFORE this existed heal themselves on their next use instead of +/// staying silently ungradeable forever. +async fn ensure_test_extra( + uv: &str, + py_s: &str, + env_dir: &Path, + repo_dir: &Path, + as_of: Option<&str>, +) { + let marker = env_dir.join(".test-extra"); + if marker.exists() { + return; + } + let Some(extra) = test_extra_name(repo_dir) else { + // Nothing to install is a settled answer, not an unfinished one — record it so we + // do not re-parse the same pyproject on every grade. + let _ = std::fs::write(&marker, "none-declared\n"); + return; + }; + let spec = format!(".[{extra}]"); + let outcome = era_pinned_uv_install( + uv, + py_s, + as_of, + &["--no-build-isolation", "-e", &spec], + Some(repo_dir), + &[("CFLAGS", ERA_CFLAGS)], + ) + .await; + let ok = matches!(&outcome, Ok(o) if o.status.success()); + crate::probe!( + class = "swe.env.test_extra", + extra = %extra, + installed = ok, + env = %env_dir.display(), + "the repo's own test extra — without it a conftest that imports a test-only \ + dependency makes every instance in the repo ungradeable" + ); + if ok { + let _ = std::fs::write(&marker, format!("{extra}\n")); + } +} + fn build_requires(repo_dir: &Path) -> Vec { let text = match std::fs::read_to_string(repo_dir.join("pyproject.toml")) { Ok(t) => t, @@ -702,6 +797,11 @@ pub async fn ensure_env(instance: &SweInstance, repo_dir: &Path) -> Result Result Result Date: Mon, 17 Aug 2026 19:56:39 -0500 Subject: [PATCH 20/80] =?UTF-8?q?fix(swe):=20the=20test-extra=20parser=20w?= =?UTF-8?q?as=20half-blind=20=E2=80=94=20read=20setup.cfg=20too,=20and=20s?= =?UTF-8?q?top=20caching=20a=20negative=20(#380)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I shipped b0141e810 claiming astropy would now gold-gate. The live run said no, and it was right. Correcting my own fix, with the receipt: gold gate astropy-14995 → "UNGRADEABLE — PASS_TO_PASS passes 0 of 40 on the PRISTINE tree"; swe.env.test_extra probe rows: 0; env marker `.test-extra` = "none-declared"; `python -c "import hypothesis"` → ModuleNotFoundError. TWO defects, both mine. 1. THE PARSER READ ONE FILE. astropy 5.3 has NO `[project.optional-dependencies]` — its suite deps are setuptools declarative config: [options.extras_require] test = # Required to run the astropy test suite. pytest>=7.0 pytest-astropy>=0.10 So `test_extra_name` returned None for the exact repo the fix was written for. Both files are ordinary in the ecosystem; reading one is reading half. `declared_extras` now reads both, hand-scanning the INI section for KEYS (an ini crate would be more surface than six lines, and we need group names, not requirement lists — note the flush-left check, since ` pytest>=7.0` is a continuation, not a group). My earlier positive control was real but proved the wrong thing: I ran `uv pip install -e ".[test]"` by hand, and uv resolves that extra THROUGH setuptools — so it worked while our own parser could never have named it. A manual command succeeding is not evidence that the code takes the same path. 2. THE MARKER CACHED A NEGATIVE, which is worse than no cache. On the first run the half-blind parser wrote "none-declared" — permanently. Fixing the parser would not have helped: the marker suppresses the heal forever, and only a human who knew which files to delete could recover it. Caching a negative caches the limits of today's code. Now only a SUCCESS is recorded, and a marker is authoritative only when it NAMES an extra, so the poisoned markers the first version wrote self-heal. Re-reading two small files per grade costs nothing next to cloning a repo and running its suite — the cost I was "optimising" was never real. Test extended with the astropy shape: no pyproject at all, extras in setup.cfg, and `test_all` present — which sorts before `test` alphabetically and is a superset, so it must not win. That case is the one that shipped broken. LESSON, and it is the same one twice today: a fix is not verified by the command I ran by hand, only by the code path the system takes. I had a green unit test, a green compile, and a green manual install, and the feature was still dead on the one repo it existed for. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 96 ++++++++++++++++--- 1 file changed, 85 insertions(+), 11 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 417238707..bf8bfff88 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -618,18 +618,63 @@ fn era_cutoff(instance: &SweInstance) -> Option { fn test_extra_name(repo_dir: &Path) -> Option { const PREFERRED: [&str; 4] = ["test", "tests", "testing", "dev"]; - let text = std::fs::read_to_string(repo_dir.join("pyproject.toml")).ok()?; - let parsed: toml::Value = toml::from_str(&text).ok()?; - let groups = parsed - .get("project")? - .get("optional-dependencies")? - .as_table()?; + let declared = declared_extras(repo_dir); PREFERRED .iter() - .find(|name| groups.contains_key(**name)) + .find(|name| declared.iter().any(|d| d == *name)) .map(|name| (*name).to_string()) } +/// Every extra group the repo declares, from BOTH places Python puts them. +/// +/// The first version of this read `pyproject.toml` only and shipped believing it worked — +/// the live run said otherwise (2026-08-17): astropy 5.3 declares its suite deps in +/// `setup.cfg` under `[options.extras_require]`, has no `[project.optional-dependencies]` +/// at all, and the pyproject-only parser returned None for the exact repo the fix was +/// written for. Both files are ordinary in the ecosystem; reading one is reading half. +fn declared_extras(repo_dir: &Path) -> Vec { + let mut found = Vec::new(); + + // PEP 621. + if let Ok(text) = std::fs::read_to_string(repo_dir.join("pyproject.toml")) { + if let Ok(parsed) = toml::from_str::(&text) { + if let Some(table) = parsed + .get("project") + .and_then(|p| p.get("optional-dependencies")) + .and_then(|o| o.as_table()) + { + found.extend(table.keys().cloned()); + } + } + } + + // setuptools' declarative config. Hand-scanned rather than pulled in as a dependency: + // we need section keys, not values, and an INI parser for that is more surface than + // the six lines below. + if let Ok(text) = std::fs::read_to_string(repo_dir.join("setup.cfg")) { + let mut in_extras = false; + for line in text.lines() { + let trimmed = line.trim(); + if trimmed.starts_with('[') { + in_extras = trimmed == "[options.extras_require]"; + continue; + } + // A key is flush-left; anything indented is a continuation of the previous + // key's requirement list (`test =\n pytest>=7.0`), never a group name. + if !in_extras || line.starts_with([' ', '\t']) { + continue; + } + if let Some((key, _)) = trimmed.split_once('=') { + let key = key.trim(); + if !key.is_empty() { + found.push(key.to_string()); + } + } + } + } + found +} + /// Install the repo's test extra into an existing env, once. Non-fatal BY DESIGN: a repo /// whose suite already collects does not need it, and a resolve failure here must not /// delete an env that grades fine today. The marker file is what makes it once-only, and @@ -643,13 +688,23 @@ async fn ensure_test_extra( as_of: Option<&str>, ) { let marker = env_dir.join(".test-extra"); - if marker.exists() { + // A marker is authoritative only when it NAMES the extra it installed. Markers written + // by the first version recorded "none-declared" from a half-blind parser; treating + // those as settled would make this fix unreachable on exactly the envs that need it, + // and would need a human to know which files to delete. They self-heal instead. + if std::fs::read_to_string(&marker) + .map(|m| !m.trim().is_empty() && m.trim() != "none-declared") + .unwrap_or(false) + { return; } let Some(extra) = test_extra_name(repo_dir) else { - // Nothing to install is a settled answer, not an unfinished one — record it so we - // do not re-parse the same pyproject on every grade. - let _ = std::fs::write(&marker, "none-declared\n"); + // NO NEGATIVE MARKER. The first version wrote "none-declared" here to avoid + // re-parsing, and then a parser that read only pyproject.toml recorded that answer + // PERMANENTLY for astropy — whose extras live in setup.cfg — so the heal could + // never run again even after the parser was fixed. Caching a negative is caching + // the limits of today's code. Re-reading two small files per grade costs nothing + // measurable next to cloning a repo and running its suite. return; }; let spec = format!(".[{extra}]"); @@ -2174,6 +2229,25 @@ mod tests { // that grades fine today. write("[project.optional-dependencies]\ndocs = []\n"); assert_eq!(test_extra_name(&dir), None, "docs-only declares no suite deps"); + + // THE CASE THAT SHIPPED BROKEN (2026-08-17): astropy 5.3 has no + // [project.optional-dependencies] at all — its suite deps are setuptools + // declarative config. A pyproject-only parser returns None for the exact repo this + // fix exists for, which is what the live gold gate caught after I had already + // claimed the fix worked. + std::fs::remove_file(dir.join("pyproject.toml")).unwrap(); + std::fs::write( + dir.join("setup.cfg"), + "[options]\npackages = find:\n\n[options.extras_require]\ntest = # Required to run the suite.\n pytest>=7.0\n pytest-astropy>=0.10\ntest_all =\n objgraph\n[options.package_data]\n* = data/*\n", + ) + .unwrap(); + assert_eq!( + test_extra_name(&dir).as_deref(), + Some("test"), + "setup.cfg extras are declarations too — and `test_all`, which sorts first \ + alphabetically and is a superset, must NOT win over `test`" + ); + let _ = std::fs::remove_file(dir.join("setup.cfg")); write("[project]\nname = \"x\"\n"); assert_eq!(test_extra_name(&dir), None, "no optional-dependencies table"); std::fs::remove_file(dir.join("pyproject.toml")).unwrap(); From 6b1b0c7011119c70d3557bf42cf416d282c75224 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 20:14:41 -0500 Subject: [PATCH 21/80] fix(swe): the UNGRADEABLE verdict must carry the pristine run's output (#380 instrument) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `let (pristine_p2p, _) = run_tests(...)` — the report went into `_`. That verdict is the ONLY artifact of the pristine run, so "the suite does not run in this environment" shipped as a conclusion with its evidence deleted, leaving the reader to reproduce the run by hand and guess at the difference. MEASURED TODAY, and it is why this is worth its own commit: astropy-14365 graded `PASS_TO_PASS 0 of 8` here, while the SAME invocation — `-m pytest -v -p no:cacheprovider`, same venv, same tree — run by hand gave 8 passed and the instance's own bug failing (`ValueError: Unrecognized QDP line: read terr 1`), which is exactly the gate condition. So the environment was fine and the divergence was somewhere inside the gate. I proposed three mechanisms and could settle none of them, because the one artifact that names the divergence was thrown away on this line. Hours, on a guessing loop, next to a variable holding the answer. Now the tail rides the error (via the existing `report_tail`, so it is bounded the same way the retry excerpt is). And an EMPTY report gets its own sentence rather than an empty section: no output at all means the harness never executed — a missing interpreter, a refused invocation, a run that died before writing a byte — which is a different fault from a suite that ran and failed, and the two must not read alike. This is the same defect class as the rest of today's session: the instrument had the answer and did not surface it. #380 stays open; this is what makes the next attempt at it a measurement instead of a hypothesis. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index bf8bfff88..3c151a276 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -2157,14 +2157,36 @@ pub async fn grade( verdict.error = Some(e); return verdict; } - let (pristine_p2p, _) = run_tests(repo_dir, &venv_py, &p2p, &test_files, runner).await; + let (pristine_p2p, pristine_report) = + run_tests(repo_dir, &venv_py, &p2p, &test_files, runner).await; if pristine_p2p.values().filter(|ok| **ok).count() == 0 { verdict.gate_ok = false; + // CARRY THE REPORT. This verdict is the ONLY artifact of the pristine run, and + // without the run's own output "the suite does not run in this environment" is + // a conclusion with its evidence deleted — the reader is left to reproduce it + // by hand and guess at the difference. + // + // Measured 2026-08-17, and it cost hours: astropy-14365 graded 0-of-8 here + // while the SAME invocation (`-m pytest -v -p no:cacheprovider`, same + // venv, same tree) run by hand gave 8 passed + the instance's own bug failing — + // exactly the gate condition. Three hypotheses got proposed and none could be + // settled, because the one thing that would have named the divergence was + // discarded into `_` on this line. An instrument that knows the answer and + // drops it is worse than one that never looked. + let tail = report_tail(&pristine_report); verdict.error = Some(format!( "UNGRADEABLE — PASS_TO_PASS passes 0 of {} on the PRISTINE tree: the \ suite does not run in this environment, so every score from this tree \ - is an env fault, never a capability verdict.", - verdict.p2p_total + is an env fault, never a capability verdict.{}", + verdict.p2p_total, + if tail.is_empty() { + " The pristine run produced NO output at all — the harness never \ + executed (a missing interpreter, a refused invocation, a run that \ + died before writing a byte), which is a different fault from a suite \ + that ran and failed.".to_string() + } else { + format!("\n\nPRISTINE RUN OUTPUT (tail):\n{tail}") + } )); return verdict; } From 1fc25c0ea79eea8408b1817a2de711246ea3534b Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 21:46:48 -0500 Subject: [PATCH 22/80] =?UTF-8?q?fix(benchmark):=20a=20kickoff=20addressed?= =?UTF-8?q?=20to=20a=20citizen=20must=20never=20be=20AUTHORED=20by=20her?= =?UTF-8?q?=20=E2=80=94=20the=20round-killer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE DEFECT, measured live 2026-08-17 and deterministic: A citizen's inbound stream skips messages she is recorded as having said (airc_persona_conversation, `message.peer_id == self.own_peer_id`). Correct — nobody answers their own speech. But the operator has no self-peer in-core (#27), so `curator_airc` falls back to `registry.any_live_citizen()`, which picks the LEXICOGRAPHICALLY-LOWEST agent_name. With the roster culled to Atlas + Benchy that is ALWAYS Atlas — the assignee. So `benchmark/dispatch --assignees='["Atlas"]'` authored every `@Atlas (to you)` kickoff THROUGH Atlas. All three died at the self-skip: no error, no probe, no turn. The command reported `kickoffs: 3, kickoff_errors: []` and was telling the literal truth — three messages were sent. Zero were hearable. The round then ran for hours on the DETACHED solver (`dispatch_staged_swe_solve` → `agent/solve`), which produced 18 acts, 0 turns, 0 room visibility and 0 curriculum input (#425) — maximum effort, nothing that reaches the flywheel. Joel: "its supposed to be persona that run. you dont wire in loopers." Why it read as intermittent (#417 "kickoffs DO actuate"): with ~20 citizens online the alphabetical pick usually was NOT the assignee, so kickoffs landed. Culling the roster to two made it a coin flip, and the coin is not random — it is `min_by(name)`. THE FIX (this commit — the delivery half): - `any_live_citizen_other_than(exclude)` — the same deterministic pick, minus one peer. `any_live_citizen()` becomes `…other_than(None)`, so there is ONE selection rule, not two that can drift. - dispatch voices each kickoff through a citizen who is NOT the addressee. - When the addressee is the ONLY live citizen, REFUSE and say why, naming `persona/spawn` as the fix — never send a message that cannot be heard ([[fallbacks-are-illegal-fail-loud]]). - The self-skip now emits `persona.inbound.skipped_self_authored`. The skip stays; its SILENCE was the real cost. A message vanishing without a trace is how a substrate failure reads as "the citizen chose not to work" ([[an-absence-is-an-unfinished-measurement]]). NOT in this commit, deliberately, and next: retiring the detached solver so the persona's own turn does the work (Joel's actual point). Order matters — prove a real `persona.turn.start` fires from a kickoff FIRST, or removing the looper just yields nothing at all. Tracked on #453. Also lands the shutdown lane sweep held from earlier tonight (#454): `lane_registry::sweep_all()` + `SweepMode{Boot,Shutdown}`, called from `stop`. `stop` reaped cores and owned orphans but touched NO llama-server, so lanes leaked across shutdown by design and `reboot` (= stop + start) inherited the wreck — measured as a 19 GB ephemeral 27B resident beside the live 14B, which starved the planner into serving a 2,816-token window (smaller than the tool surface) and made every citizen structurally mute. Role-blind on purpose: the live/ephemeral split decides ADOPTION, and nothing is adoptable by a core that is exiting. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/bin/continuum.rs | 30 ++++ core/continuum-core/src/commands/benchmark.rs | 35 +++- .../src/inference/lane_registry.rs | 162 +++++++++++++++--- .../src/persona/airc_persona_conversation.rs | 20 +++ .../src/persona/airc_runtime_registry.rs | 34 ++++ 5 files changed, 253 insertions(+), 28 deletions(-) diff --git a/core/continuum-core/src/bin/continuum.rs b/core/continuum-core/src/bin/continuum.rs index 699396460..21d70f8c1 100644 --- a/core/continuum-core/src/bin/continuum.rs +++ b/core/continuum-core/src/bin/continuum.rs @@ -1659,6 +1659,36 @@ async fn stop() -> Result<(), String> { // holding a port or VRAM once `stop` returns. reap_owned_orphans(&[]); + // Serving lanes are NOT descended from any core we just reaped (the daemon + // spawns them detached) and are NOT under `~/.continuum/bin`, so neither the + // tree kill nor the ownership sweep above can see them. Until this call + // existed, `stop` left every `llama-server` running and the registry was + // swept only on the NEXT boot — measured 2026-08-17 on the M5 as two lanes + // resident at once (a 19 GB ephemeral 27B beside the live 14B), which + // starved the planner into serving a 2,816-token window that cannot hold the + // tool surface. `reboot` could not clear it either: reboot is stop + start, + // and neither half owned lanes. + for outcome in continuum_core::inference::lane_registry::sweep_all() { + use continuum_core::inference::lane_registry::SweepOutcome as S; + match outcome { + S::ReapedLive { pid, port } => { + println!(" reaping serving lane (pid {pid}, port {port}) — live lane, this core is stopping") + } + S::ReapedEphemeral { pid, port } => { + println!(" reaping serving lane (pid {pid}, port {port}) — ephemeral lane, owner gone") + } + // A record whose pid is dead / recycled / unparseable is bookkeeping, + // not an event: garbage-collected silently so the loud lines above + // stay meaningful. + S::RemovedDead { .. } | S::RemovedReused { .. } | S::RemovedUnparseable { .. } => {} + // Unreachable under Shutdown (every role is reaped) — but matched + // explicitly so adding a mode can never silently fall through here. + S::LeftLive { pid } => { + println!(" WARNING: serving lane (pid {pid}) left running by a shutdown sweep — report this") + } + } + } + let _ = std::fs::remove_file(&socket); Ok(()) } diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 023587b33..76f2458f3 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -1447,7 +1447,40 @@ impl ActionCommand for BenchmarkDispatch { ) } }; - match airc.say(&kickoff).await { + // AUTHOR IT AS SOMEONE ELSE. A citizen's inbound stream skips messages + // she is recorded as having said (correct — nobody answers their own + // speech), so a kickoff addressed to her and AUTHORED by her is dropped + // silently and she never takes a turn. + // + // That is not hypothetical: `curator_airc` falls back to + // `any_live_citizen()` for the operator (no self-peer, #27), which picks + // the lexicographically-lowest name. With the roster at `Atlas` + `Benchy` + // that is ALWAYS Atlas — so a round directed at Atlas sent her three + // kickoffs she authored herself, reported `kickoff_errors: []`, and + // produced ZERO turns while a detached solver did the work beside her + // (measured 2026-08-17). A bigger roster usually picked someone else, + // which is why it read as intermittent (#417) rather than structural. + let voice = match self + .registry + .any_live_citizen_other_than(Some(*who_peer)) + .map(|rt| rt.airc().clone()) + { + Some(a) => a, + None => { + // The only live citizen IS the addressee. Refuse loudly instead of + // sending a message that cannot be heard — the card stays on the + // board, and the operator learns the roster is too small to direct + // work at all ([[fallbacks-are-illegal-fail-loud]]). + kickoff_errors.push(format!( + "{short}: {who} is the only live citizen, so nobody else can \ + voice a kickoff addressed to her — she would skip her own \ + message and never take a turn. Spawn a second citizen \ + (persona/spawn), then re-dispatch." + )); + continue; + } + }; + match voice.say(&kickoff).await { Ok(_) => kickoffs += 1, // The card stays claimable — a lost kickoff is REPORTED (never unwound // or hidden); the citizen can still find and claim it off the board. diff --git a/core/continuum-core/src/inference/lane_registry.rs b/core/continuum-core/src/inference/lane_registry.rs index 60aca93f7..f5a6aa7f8 100644 --- a/core/continuum-core/src/inference/lane_registry.rs +++ b/core/continuum-core/src/inference/lane_registry.rs @@ -69,12 +69,35 @@ pub struct LaneRecord { pub model: String, } -/// What [`sweep_orphans`] did to one record — exhaustive so every branch is +/// WHY a sweep is running — the one axis on which boot and shutdown differ. +/// +/// They differ in exactly one judgement: what a still-alive LIVE-role record +/// means. At boot it may be a perfectly good server this core can adopt, so the +/// decision belongs to `lane_pidfile`'s canonical-port reclaim. At shutdown +/// nothing is adoptable by definition — the core is going away — so a survivor is +/// a leak. Encoding that as a MODE on one sweep (rather than a second sweep +/// function) is what stops the two paths from drifting: every other rule — +/// never-blind-kill, GC dead records, drop unparseable garbage — is shared by +/// construction. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SweepMode { + /// A fresh core starting up. Ephemeral records are definitionally orphans; + /// the LIVE record is left for `lane_pidfile` to adopt-or-reap. + Boot, + /// This core is shutting down. EVERY lane it owns must die with it, live + /// included — `stop` that leaves a server holding VRAM has not stopped. + Shutdown, +} + +/// What a sweep did to one record — exhaustive so every branch is /// loggable and a new state can't be silently dropped. #[derive(Debug, Clone, PartialEq, Eq)] pub enum SweepOutcome { /// Killed a live ephemeral orphan and removed its record. ReapedEphemeral { pid: u32, port: u16 }, + /// Killed the LIVE lane and removed its record. [`SweepMode::Shutdown`] only — + /// at boot a live survivor is adoptable, at shutdown it is a leak. + ReapedLive { pid: u32, port: u16 }, /// Record named a pid that is no longer alive — stale file removed, no kill. RemovedDead { pid: u32 }, /// Pid is alive but NOT a `llama-server` (reused number) — record removed, the @@ -116,10 +139,30 @@ pub fn remove(pid: u32) { /// Reap every orphaned ephemeral lane recorded by a crashed predecessor and /// garbage-collect dead records. Resolves the canonical directory then delegates -/// to the pure [`sweep_orphans_in`]. Returns what it did, for the caller to log. +/// to the pure [`sweep_in`]. Returns what it did, for the caller to log. pub fn sweep_orphans() -> Vec { match lanes_dir() { - Some(dir) => sweep_orphans_in(&dir), + Some(dir) => sweep_in(&dir, SweepMode::Boot), + None => Vec::new(), + } +} + +/// Reap EVERY lane this install owns — live and ephemeral — for shutdown. +/// +/// `stop` reaps cores ([`crate::runtime::core_bind_guard`]) and owned engine +/// orphans, but until this existed it never touched a `llama-server`: the whole +/// registry was swept only on the NEXT boot, so shutting Continuum down left +/// every lane resident. Measured 2026-08-17 on the M5: an ephemeral 27B lane +/// (19 GB) and the live 14B lane were up simultaneously; the planner sized its +/// window against what was left and served citizens a 2,816-token context, which +/// cannot even hold the tool surface. `reboot` could not clear it — reboot is +/// stop + start, and neither half owned lanes. +/// +/// Role-blind on purpose. The live/ephemeral split exists to decide ADOPTION, and +/// nothing is adoptable by a core that is exiting. +pub fn sweep_all() -> Vec { + match lanes_dir() { + Some(dir) => sweep_in(&dir, SweepMode::Shutdown), None => Vec::new(), } } @@ -177,8 +220,8 @@ fn remove_in(dir: &Path, pid: u32) { let _ = std::fs::remove_file(record_path(dir, pid)); } -/// The pure sweep against an explicit `dir`. See [`sweep_orphans`]. -fn sweep_orphans_in(dir: &Path) -> Vec { +/// The pure sweep against an explicit `dir`. See [`sweep_orphans`] / [`sweep_all`]. +fn sweep_in(dir: &Path, mode: SweepMode) -> Vec { let mut outcomes = Vec::new(); // A missing directory is the normal first-run / all-graceful-prior-shutdown // state — nothing to sweep, not a fallback. @@ -209,26 +252,39 @@ fn sweep_orphans_in(dir: &Path) -> Vec { continue; } - match rec.role { + // At BOOT a live survivor may be adoptable, so the decision defers to + // `lane_pidfile`'s canonical-port reclaim. At SHUTDOWN nothing is + // adoptable — this core is exiting — so every role is reaped. + let reap = match (rec.role, mode) { + (LaneRole::Ephemeral, _) => true, + (LaneRole::Live, SweepMode::Shutdown) => true, // The live lane's port is `lane_pidfile`'s job (adopt-or-reap). Leave // both the process and its record; if `lane_pidfile` reaps it, the next // boot sees a dead pid here and GCs the file. - LaneRole::Live => outcomes.push(SweepOutcome::LeftLive { pid: rec.pid }), - LaneRole::Ephemeral => { - if lane_process::is_llama_server(rec.pid) { - lane_process::kill9(rec.pid); - let _ = std::fs::remove_file(&path); - outcomes.push(SweepOutcome::ReapedEphemeral { - pid: rec.pid, - port: rec.port, - }); - } else { - // Alive but not one of ours — a reused pid. Drop the stale - // record; never signal an unrelated process. - let _ = std::fs::remove_file(&path); - outcomes.push(SweepOutcome::RemovedReused { pid: rec.pid }); - } - } + (LaneRole::Live, SweepMode::Boot) => false, + }; + if !reap { + outcomes.push(SweepOutcome::LeftLive { pid: rec.pid }); + continue; + } + if lane_process::is_llama_server(rec.pid) { + lane_process::kill9(rec.pid); + let _ = std::fs::remove_file(&path); + outcomes.push(match rec.role { + LaneRole::Live => SweepOutcome::ReapedLive { + pid: rec.pid, + port: rec.port, + }, + LaneRole::Ephemeral => SweepOutcome::ReapedEphemeral { + pid: rec.pid, + port: rec.port, + }, + }); + } else { + // Alive but not one of ours — a reused pid. Drop the stale + // record; never signal an unrelated process. + let _ = std::fs::remove_file(&path); + outcomes.push(SweepOutcome::RemovedReused { pid: rec.pid }); } } outcomes @@ -316,7 +372,7 @@ mod tests { let me = std::process::id(); record_in(&dir, &rec(me, 58200, LaneRole::Ephemeral)).expect("record"); - let outcomes = sweep_orphans_in(&dir); + let outcomes = sweep_in(&dir, SweepMode::Boot); assert_eq!(outcomes, vec![SweepOutcome::RemovedReused { pid: me }]); assert!( lane_process::is_alive(me), @@ -336,7 +392,7 @@ mod tests { let me = std::process::id(); record_in(&dir, &rec(me, 58057, LaneRole::Live)).expect("record"); - let outcomes = sweep_orphans_in(&dir); + let outcomes = sweep_in(&dir, SweepMode::Boot); assert_eq!(outcomes, vec![SweepOutcome::LeftLive { pid: me }]); assert!( record_path(&dir, me).exists(), @@ -358,7 +414,7 @@ mod tests { child.wait().expect("reap"); record_in(&dir, &rec(dead, 58200, LaneRole::Ephemeral)).expect("record"); - let outcomes = sweep_orphans_in(&dir); + let outcomes = sweep_in(&dir, SweepMode::Boot); // (Tiny PID-reuse window is acceptable in a unit test; assert the invariant // "a reaped pid is GC'd or safely treated as reused, never a kill+reap of // the living".) @@ -371,6 +427,58 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + // what this catches: the boot/shutdown difference, on the ONE record where they + // disagree. A LIVE record is LEFT at boot (it may be adoptable) and REAPED at + // shutdown (nothing is adoptable by a core that is exiting). Regression for the + // 2026-08-17 M5 incident: `stop` never swept lanes at all, so shutting Continuum + // down left llama-servers holding VRAM and `reboot` (= stop + start) could not + // clear them. Uses OUR OWN pid as the recorded lane: it is definitely alive and + // definitely NOT a llama-server, so the never-blind-kill guard must classify it + // RemovedReused under Shutdown — proving the shutdown path still refuses to + // signal a process it cannot positively identify, while boot still returns + // LeftLive without even looking. Asserting we are still alive afterwards is the + // real safety claim. + #[test] + fn shutdown_reaps_the_live_lane_that_boot_leaves_alone() { + let me = std::process::id(); + + let boot_dir = temp_dir("mode-boot"); + let _ = std::fs::remove_dir_all(&boot_dir); + record_in(&boot_dir, &rec(me, 58057, LaneRole::Live)).expect("record"); + assert_eq!( + sweep_in(&boot_dir, SweepMode::Boot), + vec![SweepOutcome::LeftLive { pid: me }], + "boot defers the live lane to lane_pidfile's adopt-or-reap" + ); + assert!( + record_path(&boot_dir, me).exists(), + "boot must KEEP the live record so the next pass can still see it" + ); + + let stop_dir = temp_dir("mode-shutdown"); + let _ = std::fs::remove_dir_all(&stop_dir); + record_in(&stop_dir, &rec(me, 58057, LaneRole::Live)).expect("record"); + let outcomes = sweep_in(&stop_dir, SweepMode::Shutdown); + assert!( + !matches!(outcomes.as_slice(), [SweepOutcome::LeftLive { .. }]), + "shutdown must NEVER leave a live lane running, got {outcomes:?}" + ); + assert_eq!( + outcomes, + vec![SweepOutcome::RemovedReused { pid: me }], + "our own pid is alive but is not a llama-server — the never-blind-kill \ + guard must hold on the shutdown path too" + ); + assert!( + lane_process::is_alive(me), + "the shutdown sweep must never signal a non-llama process" + ); + assert!(!record_path(&stop_dir, me).exists(), "record cleared"); + + let _ = std::fs::remove_dir_all(&boot_dir); + let _ = std::fs::remove_dir_all(&stop_dir); + } + // what this catches: an unparseable `.lane` file is removed as garbage, never // acted on — a corrupt record can't wedge the sweep or trigger a bogus kill. #[test] @@ -381,7 +489,7 @@ mod tests { let junk = dir.join("99999.lane"); std::fs::write(&junk, "not json at all").expect("write junk"); - let outcomes = sweep_orphans_in(&dir); + let outcomes = sweep_in(&dir, SweepMode::Boot); assert_eq!( outcomes, vec![SweepOutcome::RemovedUnparseable { path: junk.clone() }] @@ -396,6 +504,6 @@ mod tests { fn sweep_missing_dir_is_noop() { let dir = temp_dir("absent"); let _ = std::fs::remove_dir_all(&dir); - assert!(sweep_orphans_in(&dir).is_empty()); + assert!(sweep_in(&dir, SweepMode::Boot).is_empty()); } } diff --git a/core/continuum-core/src/persona/airc_persona_conversation.rs b/core/continuum-core/src/persona/airc_persona_conversation.rs index 411c7741f..ba5efd2e1 100644 --- a/core/continuum-core/src/persona/airc_persona_conversation.rs +++ b/core/continuum-core/src/persona/airc_persona_conversation.rs @@ -385,7 +385,27 @@ impl PersonaConversation for AircPersonaConversation { // Skip our own turn, matched on the RESOLVED sender so a // self-authored chat_transcript is caught too — not just // a self `say()` (whose transport peer is us). + // + // PROBED, because this drop was SILENT and that cost a whole round + // (2026-08-17). `benchmark/dispatch` authored `@Atlas (to you)` + // kickoffs THROUGH Atlas (the operator has no self-peer, so + // `curator_airc` borrows the lexicographically-first live citizen — + // her). Every kickoff died right here: no error, no probe, no turn, + // `kickoff_errors: []`, and hours spent looking at the grader, the + // roster and the model. The skip is CORRECT — nobody answers their own + // speech — but a message vanishing without a trace is how a structural + // failure reads as "the citizen chose not to work" + // ([[an-absence-is-an-unfinished-measurement]]). if message.peer_id == self.own_peer_id { + tracing::debug!( + persona = %self.own_peer_id, + from_peer = %event.peer_id, + text_len = message.text.len(), + probe_class = "persona.inbound.skipped_self_authored", + "skipped a message this persona is recorded as having said — \ + if it was ADDRESSED to her, whoever sent it authored through \ + her identity and she cannot hear it" + ); continue; } return Ok(Some(message)); diff --git a/core/continuum-core/src/persona/airc_runtime_registry.rs b/core/continuum-core/src/persona/airc_runtime_registry.rs index 2ab7795b6..1f84fa0ad 100644 --- a/core/continuum-core/src/persona/airc_runtime_registry.rs +++ b/core/continuum-core/src/persona/airc_runtime_registry.rs @@ -235,8 +235,42 @@ impl PersonaAircRuntimeRegistry { /// the fix is `persona/spawn`, not inventing an identity. /// [[general-by-design-beats-hardcoded-users]] pub fn any_live_citizen(&self) -> Option> { + self.any_live_citizen_other_than(None) + } + + /// The same deterministic pick, EXCLUDING one peer — the author for a message + /// ADDRESSED to that peer. + /// + /// ## The round-killer this exists to make impossible (2026-08-17) + /// + /// A citizen cannot hear a message she is recorded as having said: her inbound + /// stream drops it at the self-skip in + /// [`crate::persona::airc_persona_conversation`] (correct — nobody should answer + /// their own speech). So authoring an ADDRESSED kickoff through + /// [`Self::any_live_citizen`] is a coin flip on whether the addressee ever hears + /// it, and the flip is rigged: the pick is the lexicographically-lowest + /// `agent_name`, so the SAME citizen is chosen every time. + /// + /// Measured live: with the roster culled to `Atlas` + `Benchy`, "Atlas" sorts + /// first, so `benchmark/dispatch` authored every `@Atlas (to you)` kickoff AS + /// Atlas. Three cards, three kickoffs, `kickoff_errors: []` — and ZERO turns, + /// silently, for the whole round, while a detached solver did the work beside + /// her. With a larger roster the same code usually picked someone else, which is + /// why the defect read as intermittent (#417) instead of structural. + /// + /// `None` means the ONLY live citizen is the addressee. That is a real refusal, + /// not a fallback: the caller must fail loud rather than send a message that + /// cannot be heard ([[fallbacks-are-illegal-fail-loud]]). + pub fn any_live_citizen_other_than( + &self, + exclude: Option, + ) -> Option> { self.inner .iter() + .filter(|e| match exclude { + Some(peer) => e.value().runtime.airc().peer_id().as_uuid() != peer, + None => true, + }) .min_by(|a, b| { a.value() .runtime From dc4621f911bfa7650bcc3dcdea3ff5518b562620 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 22:20:09 -0500 Subject: [PATCH 23/80] fix(bench): dispatch resolves against RESIDENCY, not registration (#455) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A citizen is REGISTERED the moment her identity is minted or resumed. She is RESIDENT only once the supervisor attaches a service loop — which is what primes her perception stream and gives her a turn. Every readiness surface reported the former and called it the latter. Measured 2026-08-18: for ~15 minutes after a reboot `persona/roster` listed Atlas and Benchy while `persona.inbound.subscribe_opened` was 0. Hosting was correctly parked waiting for the serving lane to prove it could decode (#363). A round was dispatched into that window on the strength of that roster: 2 cards posted, `kickoffs: 2`, `kickoff_errors: []`, and ZERO turns. Nobody was home and every surface said otherwise. - `PersonaAircRuntimeRegistry::is_resident` / `resident_snapshot` — residency asked per citizen, never inferred from presence in the registry. The truth table is a pure fn (`resident_from_loop_state`) so the three rows are unit- pinned; a live `PersonaAircRuntime` needs a real airc daemon and cannot be built in a test. - `persona/roster` gains `resident` per row + `resident_count`. `count > 0 && resident_count == 0` is exactly the state above, and it is now legible. - `benchmark/dispatch` resolves assignees against the RESIDENT snapshot, and its post-boot wait keys on residency too — the old wait cleared on a non-empty roster, which released it ~10 minutes early (#412). Refusals distinguish the two states because they have opposite fixes: registered-but-not-resident is a WAIT, unregistered is `persona/spawn`. A named assignee who is merely unhosted is no longer reported as a typo. This is honesty at the seam, not the cure. The cure is boot owning the process tree (#452) so the resident window never exists to dispatch into. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 201 +++++++++++++++--- .../src/commands/persona_roster.rs | 49 ++++- .../src/persona/airc_runtime_registry.rs | 78 +++++++ 3 files changed, 290 insertions(+), 38 deletions(-) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 76f2458f3..d0057659a 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -817,20 +817,20 @@ pub struct BenchmarkDispatch { } /// Resolve the citizens a directed dispatch addresses — GENERALIZED for any repo user's -/// roster, never our specific names. Pure over the live snapshot so it is unit-testable +/// roster, never our specific names. Pure over the snapshots so it is unit-testable /// without a running airc daemon (a real `PersonaSlot` needs one); the wrapper in `run` -/// just feeds `registry.roster_snapshot()` in. +/// feeds `registry.resident_snapshot()` and `registry.roster_snapshot()` in. /// -/// - `requested` empty → the WHOLE live roster (whoever THIS machine spawned). This is the -/// "dispatch to my citizens, whoever they are" default: a fresh clone runs -/// `benchmark/dispatch --name=…` with no `--assignees` and it targets their own online +/// - `requested` empty → the WHOLE RESIDENT roster (whoever THIS machine has in the room). +/// This is the "dispatch to my citizens, whoever they are" default: a fresh clone runs +/// `benchmark/dispatch --name=…` with no `--assignees` and it targets their own resident /// citizens. Directed dispatch is what actuates (a silent card does not — measured -/// 2026-08-07), so defaulting to the live roster keeps the loop autonomous everywhere. -/// - `requested` non-empty → every name MUST resolve to a live citizen; an unknown name -/// FAILS LOUD listing who is online (never silently addresses a ghost that never claims, -/// and never silently skips SWE staging). Order is preserved for a stable round-robin. -/// - roster empty → `Denied` (nobody online — `persona/spawn` first; the fix is a citizen, -/// not an invented identity). +/// 2026-08-07), so defaulting to the resident roster keeps the loop autonomous everywhere. +/// - `requested` non-empty → every name MUST resolve to a RESIDENT citizen; anything else +/// FAILS LOUD listing who is resident (never silently addresses a citizen who cannot +/// hear, and never silently skips SWE staging). Order is preserved for round-robin. +/// - nobody resident → `Denied`, and the message distinguishes "not spawned" (fix: +/// `persona/spawn`) from "spawned but not hosted yet" (fix: wait — see below). /// Seconds since the epoch — the only impurity `default_run_room_name` needs, kept out /// of it so the name itself is a pure function with a real unit test. fn epoch_secs() -> u64 { @@ -866,16 +866,40 @@ fn default_run_room_name(benchmark: &str, epoch_secs: u64) -> String { format!("bench-{slug}-{epoch_secs}") } +/// `live` is the RESIDENT snapshot (service loop attached and running), NOT the registered +/// roster — `registered` is passed alongside it purely so a refusal can tell the operator +/// WHICH of the two states they are in. That distinction is the whole point: +/// +/// - registered ∧ ¬resident → she exists but has no perception stream. Hosting is waiting +/// on something (usually a serving lane proving it can decode, #363). Work staged now is +/// posted into an empty room and never worked. +/// - ¬registered → nobody spawned her. The fix is `persona/spawn`, a different action. +/// +/// Measured 2026-08-18: dispatching against the REGISTERED roster in the first state +/// reported `dispatched: 2, kickoffs: 2, kickoff_errors: []` and produced zero turns. fn resolve_dispatch_roster( live: &[(String, uuid::Uuid)], + registered: &[(String, uuid::Uuid)], requested: &[String], ) -> Result, CommandError> { if live.is_empty() { - return Err(CommandError::Denied( - "no citizens are online to work the cards — spawn a persona (persona/spawn) \ - first, then dispatch." - .to_string(), - )); + if registered.is_empty() { + return Err(CommandError::Denied( + "no citizens are online to work the cards — spawn a persona (persona/spawn) \ + first, then dispatch." + .to_string(), + )); + } + let names: Vec<&str> = registered.iter().map(|(n, _)| n.as_str()).collect(); + return Err(CommandError::Denied(format!( + "citizen(s) [{}] are registered but NOT RESIDENT — no service loop, so no \ + perception stream, so nothing dispatched here would be heard. Hosting is \ + normally waiting on the serving lane to prove it can decode; watch \ + `inference.lane_relaunch_retry` and `persona.inbound.subscribe_opened`, and \ + re-dispatch once `persona/roster` reports resident_count > 0. Staging a round \ + into this window posts cards nobody can see.", + names.join(", "), + ))); } if requested.is_empty() { return Ok(live.to_vec()); @@ -890,11 +914,30 @@ fn resolve_dispatch_roster( } if !unknown.is_empty() { let online: Vec<&str> = live.iter().map(|(n, _)| n.as_str()).collect(); + // Name the registered-but-not-resident case separately: "not online" reads as a + // typo, and sending an operator hunting for a misspelling when the real answer is + // "she is here but not hosted yet" is the same lie in a smaller box. + let not_resident: Vec<&str> = unknown + .iter() + .filter(|n| registered.iter().any(|(r, _)| r == *n)) + .map(|s| s.as_str()) + .collect(); + let residency_note = if not_resident.is_empty() { + String::new() + } else { + format!( + " NOTE: [{}] are registered but not resident — they exist, they just have \ + no service loop yet (hosting is likely waiting on a serving lane). That \ + is a wait, not a typo.", + not_resident.join(", ") + ) + }; return Err(CommandError::Invalid(format!( - "assignee(s) not online: {}. Citizens currently online: [{}]. Pass names from \ - that list, or omit --assignees to dispatch to all of them.", + "assignee(s) not resident: {}. Citizens resident right now: [{}]. Pass names \ + from that list, or omit --assignees to dispatch to all of them.{}", unknown.join(", "), online.join(", "), + residency_note, ))); } Ok(resolved) @@ -1053,26 +1096,35 @@ impl ActionCommand for BenchmarkDispatch { .to_string(), )); } - // #442 (roster half) + #412: a dispatch fired inside the post-boot resume window - // used to find an EMPTY roster and refuse instantly — so the operator hand-rolled - // a sleep-loop around dispatch (run by hand twice on 2026-08-17; a runbook line - // is a design defect). The serving half of #442 already parks - // (`await_ready_serving` below); the roster half now parks the same way: wait - // ONLY while the roster is EMPTY (citizens are still being re-hosted), bounded. - // An unknown NAME against a LIVE roster still fails fast — that error means a + // #442 (roster half) + #412 + #455: a dispatch fired inside the post-boot resume + // window used to find an EMPTY roster and refuse instantly — so the operator + // hand-rolled a sleep-loop around dispatch (run by hand twice on 2026-08-17; a + // runbook line is a design defect). The serving half of #442 already parks + // (`await_ready_serving` below); the roster half parks the same way, bounded. + // + // #455 is what this loop keys on NOW: RESIDENCY, not registration. Waiting for the + // roster to be non-empty released the wait ~10 minutes too early (#412) — citizens + // are registered, presence-pumping and renewing claims long before the supervisor + // attaches a service loop, so the old condition cleared while nobody could hear a + // thing. Measured 2026-08-18: roster listed 2, `subscribe_opened` was 0, a whole + // round went onto the board and produced zero turns. + // + // An unknown NAME against a RESIDENT roster still fails fast — that error means a // typo, never a resume in progress. const ROSTER_RESUME_WAIT: std::time::Duration = std::time::Duration::from_secs(180); const ROSTER_RESUME_POLL: std::time::Duration = std::time::Duration::from_secs(5); let wait_started = std::time::Instant::now(); - while self.registry.roster_snapshot().is_empty() - && wait_started.elapsed() < ROSTER_RESUME_WAIT - { + let mut resident = self.registry.resident_snapshot().await; + while resident.is_empty() && wait_started.elapsed() < ROSTER_RESUME_WAIT { tracing::info!( waited_s = wait_started.elapsed().as_secs(), - "dispatch: roster is empty (post-boot resume window, #412) — waiting for \ - citizens to be hosted rather than refusing" + registered = self.registry.roster_snapshot().len(), + probe_class = "benchmark.dispatch.awaiting_residency", + "dispatch: no RESIDENT citizen yet (registered != in the room, #412/#455) \ + — waiting for a service loop rather than staging into an empty room" ); tokio::time::sleep(ROSTER_RESUME_POLL).await; + resident = self.registry.resident_snapshot().await; } // Resolve the dispatch roster against THIS machine's live citizens (never our @@ -1083,7 +1135,8 @@ impl ActionCommand for BenchmarkDispatch { // Resolved BEFORE the room exists because the roster decides WHO gets moved into // it: a run room nobody is standing in is the other half of the bug this verb is // fixing ("old rooms flooded, or ones with nothing"). - let roster = resolve_dispatch_roster(&self.registry.roster_snapshot(), &requested)?; + let roster = + resolve_dispatch_roster(&resident, &self.registry.roster_snapshot(), &requested)?; // Repo hint, read from the board the curator is standing in RIGHT NOW — before we // move her, and ONLY when the caller named no repo. The run room is fresh, so its @@ -1606,6 +1659,92 @@ crate::register_command!(BenchmarkDispatch); mod tests { use super::*; + fn citizen(name: &str) -> (String, uuid::Uuid) { + ( + name.to_string(), + uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, name.as_bytes()), + ) + } + + // what this catches: THE round-killer of 2026-08-18. Citizens registered but not yet + // hosted made every readiness surface report a ready roster, and dispatch staged a full + // round into a room where nobody had a perception stream — `dispatched: 2, kickoffs: 2, + // kickoff_errors: []`, zero turns. Resolving against RESIDENCY must refuse instead, and + // the refusal must say WHICH state this is, because the two have opposite fixes: + // registered-but-not-resident is a WAIT, unregistered is `persona/spawn`. + #[test] + fn registered_but_not_resident_refuses_and_names_the_wait() { + let registered = vec![citizen("Atlas"), citizen("Benchy")]; + let err = resolve_dispatch_roster(&[], ®istered, &[]).unwrap_err(); + let msg = format!("{err:?}"); + assert!( + matches!(err, CommandError::Denied(_)), + "staging into an empty room is denied, not a soft warning: {msg}" + ); + assert!( + msg.contains("Atlas") && msg.contains("Benchy"), + "the refusal names WHO is registered so the operator can wait on them: {msg}" + ); + assert!( + msg.contains("NOT RESIDENT"), + "and says the state plainly, not 'not online' (which reads as a typo): {msg}" + ); + } + + // what this catches: the OTHER arm must stay distinguishable. Nobody registered at all + // is a different problem with a different fix — `persona/spawn`, not a wait — and + // collapsing the two would send an operator to wait forever for a citizen who was + // never born. + #[test] + fn nobody_registered_at_all_points_at_spawn_not_at_waiting() { + let err = resolve_dispatch_roster(&[], &[], &[]).unwrap_err(); + let msg = format!("{err:?}"); + assert!(msg.contains("persona/spawn"), "names the actual fix: {msg}"); + assert!( + !msg.contains("NOT RESIDENT"), + "must NOT claim a residency wait when there is nobody to wait for: {msg}" + ); + } + + // what this catches: a named assignee who is registered but not resident must not be + // reported as a typo. "not online" against a name the operator can see in + // `persona/roster` sends them hunting for a misspelling that does not exist — the same + // lie as the round-level one, in a smaller box. + #[test] + fn a_named_assignee_who_is_not_resident_is_told_apart_from_a_typo() { + let resident = vec![citizen("Atlas")]; + let registered = vec![citizen("Atlas"), citizen("Benchy")]; + + let err = + resolve_dispatch_roster(&resident, ®istered, &["Benchy".into()]).unwrap_err(); + let msg = format!("{err:?}"); + assert!( + msg.contains("registered but not resident") && msg.contains("not a typo"), + "a real citizen who is merely unhosted must be named as a WAIT: {msg}" + ); + + // ...while a genuine typo still reads as one, with no residency excuse attached. + let err = resolve_dispatch_roster(&resident, ®istered, &["Atals".into()]).unwrap_err(); + let msg = format!("{err:?}"); + assert!( + !msg.contains("not a typo"), + "an unknown name gets no residency note — it IS a typo: {msg}" + ); + } + + // what this catches: with everyone resident, dispatch behaves exactly as before — + // empty request → the whole resident roster, in order. The residency gate must not + // narrow the happy path (the default dispatch is what actuates a round at all). + #[test] + fn all_resident_dispatches_the_whole_roster_in_order() { + let all = vec![citizen("Atlas"), citizen("Benchy")]; + let got = resolve_dispatch_roster(&all, &all, &[]).unwrap(); + assert_eq!(got, all, "empty request → everyone resident, order preserved"); + + let got = resolve_dispatch_roster(&all, &all, &["Benchy".into()]).unwrap(); + assert_eq!(got, vec![citizen("Benchy")], "named assignee resolves"); + } + // what this catches: a derived run-room name that airc REFUSES. `ChannelName::new` accepts // only `[a-z0-9_-]`, so a benchmark named with a `/`, a `.` or a capital (`swe-bench/lite`, // `humaneval-rs.v2`) would build a name that fails at `join` — dispatch would die at the diff --git a/core/continuum-core/src/commands/persona_roster.rs b/core/continuum-core/src/commands/persona_roster.rs index fb49d83ca..737c7c07a 100644 --- a/core/continuum-core/src/commands/persona_roster.rs +++ b/core/continuum-core/src/commands/persona_roster.rs @@ -44,6 +44,20 @@ pub struct PersonaRosterEntry { /// SWE instances already staged in her workspace (`workspace/swe/` with a `.git`). /// Non-empty here is the REUSE signal: dispatch found the checkout and skipped cloning. pub staged_swe: Vec, + /// Is she RESIDENT — a live service loop, i.e. actually in the room and able to take + /// a turn? + /// + /// REGISTRATION IS NOT RESIDENCY, and this row used to report only the former. + /// Measured 2026-08-18: for ~15 minutes after a reboot this command listed Atlas and + /// Benchy while `persona.inbound.subscribe_opened` was 0 — hosting was correctly parked + /// waiting for the serving lane to prove it could decode (#363), so neither had a + /// perception stream. A round was staged into that window on the strength of THIS + /// roster: cards posted, `kickoffs: 2`, `kickoff_errors: []`, zero turns. + /// + /// `false` means she exists but is not in the room yet — usually hosting waiting on a + /// serving lane (watch `inference.lane_relaunch_retry`), which self-heals on the next + /// serving-plan edge. Work must NOT be staged for a citizen whose `resident` is false. + pub resident: bool, } #[derive(Debug, Clone, Serialize, TS)] @@ -52,10 +66,18 @@ pub struct PersonaRosterEntry { export_to = "../../../protocol/typescript/persona/PersonaRosterResult.ts" )] pub struct PersonaRosterResult { - /// How many citizens are online right now (the roster `benchmark/dispatch` targets when - /// `--assignees` is omitted). Zero means dispatch would be Denied — spawn a persona. + /// How many citizens are REGISTERED on this machine. This is an inventory count, not a + /// readiness signal — read `resident_count` before staging any work. #[ts(type = "number")] pub count: u32, + /// How many are RESIDENT — service loop live, perception stream primed, able to take a + /// turn. THIS is the number a caller staging work must gate on. + /// + /// `count > 0 && resident_count == 0` is the exact state that silently ate a benchmark + /// round on 2026-08-18: two citizens listed, neither in the room (hosting parked while + /// the serving lane proved it could decode), cards + kickoffs posted anyway, zero turns. + #[ts(type = "number")] + pub resident_count: u32, /// Every live citizen, sorted by name (the stable round-robin order). pub citizens: Vec, } @@ -112,16 +134,21 @@ impl ActionCommand for PersonaRoster { _p: PersonaRosterParams, ) -> Result { let snap = self.registry.roster_snapshot(); - let citizens: Vec = snap - .into_iter() - .map(|(agent_name, peer)| PersonaRosterEntry { + let mut citizens: Vec = Vec::with_capacity(snap.len()); + for (agent_name, peer) in snap { + citizens.push(PersonaRosterEntry { + // Residency is ASKED, per citizen, at read time — never inferred from + // presence in the registry. See `PersonaRosterEntry::resident`. + resident: self.registry.is_resident(peer).await, agent_name, peer_id: crate::identity::PeerId::from_uuid(peer), staged_swe: staged_swe_for(&peer), - }) - .collect(); + }); + } + let resident_count = citizens.iter().filter(|c| c.resident).count() as u32; Ok(PersonaRosterResult { count: citizens.len() as u32, + resident_count, citizens, }) } @@ -151,6 +178,7 @@ mod tests { b"a93ec5cc-e183-427a-ab8f-784ffe8805cc", )), staged_swe: vec!["astropy__astropy-12907".into()], + resident: true, }; let v = serde_json::to_value(&e).unwrap(); assert_eq!(v["agent_name"], "Yori"); @@ -164,8 +192,15 @@ mod tests { .to_string() ); assert_eq!(v["staged_swe"][0], "astropy__astropy-12907"); + assert_eq!(v["resident"], true); } + // NOTE on residency coverage: a `PersonaAircRuntime` cannot be constructed without a + // live airc daemon (see `clone_shares_roster` in airc_runtime_registry), so the + // registered-but-not-resident row is pinned where it IS testable — as the pure truth + // table `persona::airc_runtime_registry::resident_from_loop_state`. Do not build a + // parallel runtime fixture here to reach it. + // what this catches: an empty registry yields count=0 with an empty citizen list — the // honest "nobody online, dispatch would be Denied" signal, never a panic or a fake row. #[tokio::test] diff --git a/core/continuum-core/src/persona/airc_runtime_registry.rs b/core/continuum-core/src/persona/airc_runtime_registry.rs index 1f84fa0ad..be6f3e1dd 100644 --- a/core/continuum-core/src/persona/airc_runtime_registry.rs +++ b/core/continuum-core/src/persona/airc_runtime_registry.rs @@ -460,6 +460,47 @@ impl PersonaAircRuntimeRegistry { loop_slot.as_ref().map(|h| h.is_finished()) } + /// Is this citizen RESIDENT — a live service loop, i.e. actually in the room? + /// + /// Registration is not residency, and conflating them is what makes every + /// caller downstream lie. A citizen is registered the moment her identity is + /// minted or resumed; she is RESIDENT only once the supervisor attached a + /// service loop, which is what primes her perception stream and gives her a + /// turn, an idle tick, and a view of her own board. + /// + /// Measured 2026-08-18, and this is why the distinction exists: for ~15 + /// minutes after a reboot `persona/instances/list` and `persona/roster` both + /// listed Atlas and Benchy while `persona.inbound.subscribe_opened` was 0 — + /// hosting was (correctly) parked waiting for the serving lane to prove it + /// could decode, because attaching a citizen to a lane that answers /health + /// 200 while every generation 500s makes each turn fail silently (#363). A + /// round was dispatched into that window: cards posted, `kickoffs: 2`, + /// `kickoff_errors: []`, ZERO turns. Nobody was home and every surface said + /// otherwise. + /// + /// `Some(false)` (loop attached, still running) is the ONLY resident state. + /// `None` = never attached. `Some(true)` = the loop finished and she is no + /// longer serving. Both are "not in the room", and callers must be able to + /// tell the difference between that and "here and working". + pub async fn is_resident(&self, persona_id: Uuid) -> bool { + resident_from_loop_state(self.is_service_loop_finished(persona_id).await) + } + + /// Every RESIDENT citizen as `(agent_name, peer_id)`, name-sorted like + /// [`Self::roster_snapshot`] — the honest roster. A caller staging work + /// resolves against THIS, never against registration. + pub async fn resident_snapshot(&self) -> Vec<(String, Uuid)> { + let mut out = Vec::new(); + for (name, peer) in self.roster_snapshot() { + // roster_snapshot keys on the airc peer_id, which IS the persona id + // used by the service-loop slot (one identity, one slot). + if self.is_resident(peer).await { + out.push((name, peer)); + } + } + out + } + /// Orderly shutdown of one persona's slot: /// 1. Take the service-loop JoinHandle out of the slot. /// 2. `.abort()` it and await its drain (yielding `ServeOutcome` @@ -541,10 +582,47 @@ impl PersonaAircRuntimeRegistry { } } +/// The residency truth table, as a pure function of the service-loop state so it can be +/// pinned by a unit test — a live `PersonaAircRuntime` needs a real airc daemon, so the +/// registry-level path is not constructible in tests (see `clone_shares_roster`). +/// +/// | loop state | meaning | resident | +/// |---------------|----------------------------------|----------| +/// | `None` | registered, no loop ever attached| **no** | +/// | `Some(true)` | loop attached, and it FINISHED | **no** | +/// | `Some(false)` | loop attached and still running | **yes** | +/// +/// `None` is the row that ate a benchmark round: registration alone made every readiness +/// surface report a citizen who had no perception stream. See [`PersonaAircRuntimeRegistry::is_resident`]. +pub(crate) fn resident_from_loop_state(loop_finished: Option) -> bool { + matches!(loop_finished, Some(false)) +} + #[cfg(test)] mod tests { use super::*; + // what this catches: registration is not residency. `None` (in the registry, no service + // loop) and `Some(true)` (loop attached but finished) must BOTH read as not-resident — + // only a live loop counts. Collapsing `None` into "resident" is the 2026-08-18 defect + // that let benchmark/dispatch stage a full round into an empty room and report + // kickoffs: 2, kickoff_errors: [], zero turns. + #[test] + fn only_a_live_service_loop_counts_as_resident() { + assert!( + !resident_from_loop_state(None), + "registered with no loop is NOT in the room" + ); + assert!( + !resident_from_loop_state(Some(true)), + "a FINISHED loop is not in the room either" + ); + assert!( + resident_from_loop_state(Some(false)), + "loop attached and running — she can take a turn" + ); + } + #[test] fn new_registry_is_empty() { let registry = PersonaAircRuntimeRegistry::new(); From f772aee7d80a79f07d2f56d080a71c5e027ba5fb Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 22:23:41 -0500 Subject: [PATCH 24/80] =?UTF-8?q?fix(boot):=20boot=20OWNS=20the=20process?= =?UTF-8?q?=20tree=20=E2=80=94=20adopt=20healthy,=20reap=20unhealthy,=20sp?= =?UTF-8?q?awn=20missing=20(#452)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, 2026-08-16: "start = enumerate → health-check → reap-or-ADOPT → spawn missing, for EVERY service — cores, llama lanes, airc daemon." The operative word was ADOPT, and boot did exactly two things to a service it found running: nothing, or kill it. Neither is ownership. ROW: llama lanes — was `pkill -f llama-server`, unconditionally, every boot. This made real code dead: `inference::lane_registry::sweep_in` already encodes the adopt rule — `(LaneRole::Live, SweepMode::Boot) => false` deliberately leaves a live lane alone at boot and reaps it only at shutdown. The shell killed that lane seconds before the core could adopt it, so that arm never once fired in production. Cost: a cold model load on EVERY reboot, during which the lane cannot prove it can decode, hosting correctly parks (#363), and every citizen is registered-but-not-resident — measured at ~15 minutes on 2026-08-18, the window a benchmark round was staged into for zero turns (#455). Now: enumerate by pid, read each lane's port from its own cmdline, /health it, adopt or reap per lane. Verified live on this box before the change landed: pids 1166 (:58057) and 95258 (:58091) both healthy — both would have been killed, and now are kept. /health is LIVENESS, not decode — a wedged server can pass it, which is exactly why the core verifies generation before seating citizens (#363). That division is deliberate: the shell adopts a plausibly-live lane, the core remains the authority that refuses a lane that cannot decode and relaunches it. Adopting can only cost a relaunch the core already knows how to do; reaping costs a cold load every time. ROW: airc daemon — this row did not exist. Boot printed "⚠ airc daemon not running. Start it with: airc daemon" and carried on: a runbook line where ownership belongs, for the one service whose absence makes everything else inert. With no transport there are no rooms, so citizens have nothing to be resident IN. Now: ping → adopt; wedged (holds the socket, answers nothing — worse than absent, since a fresh spawn loses the bind, #355) → `airc stop` then reap → spawn → bounded wait → FAIL LOUD and exit nonzero. Binary absent is a DIFFERENT state (warn, continue): a box without airc installed is not a broken one, and refusing to boot would strand CI and fresh clones. ROW: cores — already correct and left alone. #420's `core_bind_guard::decide` adopts on `start` (AlreadyServing → no-op, Occupied → refuse), and reaping on `reboot` is right because reboot replaces the binary. `bounded_run` is the shared primitive. macOS ships no coreutils `timeout`, and — the lesson that made #420's guard reachable at all — an unbounded probe against a wedged service does not fail, it HANGS: the kernel completes connect() into the listen backlog whether or not the process is ever scheduled. A health check that can hang is not a health check, it is a boot that stops. Truth table verified live: exit0→0, exit1→1, hang→124 in 1s not 5s. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- tools/scripts/start-server.sh | 186 ++++++++++++++++++++++++++++++---- 1 file changed, 166 insertions(+), 20 deletions(-) diff --git a/tools/scripts/start-server.sh b/tools/scripts/start-server.sh index c8c215659..a7e1065c1 100755 --- a/tools/scripts/start-server.sh +++ b/tools/scripts/start-server.sh @@ -178,30 +178,171 @@ if ! command -v llama-server >/dev/null 2>&1; then fi echo "✓ llama-server: $(command -v llama-server) — the engine we own & launch" >&2 -# (2) Clear any FOREIGN inference server so the core starts from a clean slate -# and gets the preferred port with the GPU to itself. At this point in a reboot -# the old core is already dead, so any live llama-server is an orphan (its parent -# gone) and any Unsloth Studio is the excised gateway — both safe to stop. -# - the Studio parent would respawn its own backend, so stop it first; -# - its llama-server child is orphaned (reparented to init) when the parent -# dies and would keep holding the port + GPU, so stop that too. -# The core's fresh llama-server is launched afterward by the serving daemon, on a -# port it SCANS for — so this is GPU/excision hygiene, not a correctness gate. +# ── BOOT OWNS THE PROCESS TREE (#452) ──────────────────────────────── +# Joel, 2026-08-16: "start = enumerate → health-check → reap-or-ADOPT → spawn +# missing, for EVERY service — cores, llama lanes, airc daemon." +# +# The operative word is ADOPT. Boot used to do exactly two things to a service it +# found running: nothing (airc — a printed runbook line) or kill it (llama lanes — +# an unconditional pkill). Neither is ownership. A service that is already healthy +# should be KEPT; only an unhealthy one is reaped; only a missing one is spawned. +# +# `bounded_run` is the shared primitive all three rows need. macOS ships no +# coreutils `timeout`, and — the lesson that made #420's guard reachable — an +# unbounded probe against a WEDGED service does not fail, it HANGS: the kernel +# completes connect() into the listen backlog whether or not the process is ever +# scheduled, and the read then waits forever. A health check that can hang is not +# a health check; it is a boot that stops. +# +# Args: 1=budget seconds, 2+=command. Exit 0 iff the command exited 0 in time; +# 124 on timeout (the same code coreutils `timeout` uses, so callers read alike). +bounded_run() { + local budget="$1"; shift + "$@" >/dev/null 2>&1 & + local pid=$! waited=0 + while kill -0 "$pid" 2>/dev/null; do + if [ "$waited" -ge "$((budget * 10))" ]; then + kill -9 "$pid" 2>/dev/null || true + wait "$pid" 2>/dev/null || true + return 124 + fi + sleep 0.1 + waited=$((waited + 1)) + done + wait "$pid" +} + +# ── ROW: foreign inference servers ─────────────────────────────────── +# Unsloth Studio is the EXCISED gateway — there is no healthy state for it to be +# adopted into, so it is reaped unconditionally. It is stopped before its backend +# because the parent would otherwise respawn one. if pgrep -f 'studio run' >/dev/null 2>&1; then - echo " stopping excised Unsloth Studio (freeing GPU for the core's engine)" >&2 + echo " reaping excised Unsloth Studio (freeing GPU for the core's engine)" >&2 pkill -f 'studio run' 2>/dev/null || true fi -if pgrep -f 'llama-server' >/dev/null 2>&1; then - echo " clearing orphaned llama-server backend(s) so the core owns the engine" >&2 - pkill -f 'llama-server' 2>/dev/null || true - # Give the OS a moment to release the listening socket before the core binds. - sleep 1 + +# ── ROW: llama lanes — ADOPT the healthy one ───────────────────────── +# This used to be `pkill -f llama-server`, unconditionally, on every boot. That +# was the #452 violation with the highest cost, and it made real code dead: +# `inference::lane_registry::sweep_in` already encodes the adopt rule — the +# `(LaneRole::Live, SweepMode::Boot) => false` arm deliberately LEAVES a live lane +# alone at boot and reaps it only at shutdown. The shell killed that lane seconds +# before the core could adopt it, so the Rust arm never once fired in production. +# +# What it cost: a cold model load on every single reboot. During that load the +# serving lane cannot prove it can decode, so hosting correctly parks (#363) and +# every citizen is REGISTERED BUT NOT RESIDENT — measured at ~15 minutes on +# 2026-08-18, which is the window a benchmark round was then staged into and +# produced zero turns (#455). Adopting a warm lane removes the window rather than +# teaching every caller to wait for it. +# +# Health is /health 200 on the lane's own port, which is a LIVENESS check, not a +# decode check — a wedged server can pass it (#363, exactly why the core verifies +# generation before attaching citizens). That is the correct division: the shell +# adopts a lane that is plausibly alive, and the core's `await_ready_serving` +# remains the authority that refuses to seat citizens on one that cannot decode, +# relaunching it if so. Adopting here can only cost a relaunch the core already +# knows how to do; reaping unconditionally costs a cold load every time. +adopt_or_reap_llama_lanes() { + local pids adopted=0 reaped=0 + pids="$(pgrep -f 'llama-server' 2>/dev/null || true)" + [ -z "$pids" ] && return 0 + local pid port + for pid in $pids; do + # The lane's port comes from its own cmdline — the only place it is recorded + # for a process the shell did not spawn. + port="$(ps -o command= -p "$pid" 2>/dev/null | sed -n 's/.*--port[ =]\([0-9]\{1,\}\).*/\1/p')" + if [ -n "$port" ] && bounded_run 3 curl -sf "http://127.0.0.1:${port}/health"; then + echo " ✓ adopting healthy llama lane (pid $pid, port $port) — warm weights kept" >&2 + adopted=$((adopted + 1)) + else + echo " ✗ reaping unhealthy llama lane (pid $pid, port ${port:-unknown})" >&2 + kill -TERM "$pid" 2>/dev/null || true + reaped=$((reaped + 1)) + fi + done + if [ "$reaped" -gt 0 ]; then + # Give the OS a moment to release the listening socket before the core binds. + sleep 1 + pkill -9 -f 'llama-server' 2>/dev/null || true + fi + echo " llama lanes: $adopted adopted, $reaped reaped" >&2 +} +adopt_or_reap_llama_lanes + +# ── ROW: airc daemon — BOOT STARTS IT ──────────────────────────────── +# This row did not exist. Boot printed "⚠ airc daemon not running. Start it with: +# airc daemon" and carried on — a runbook line where ownership belongs, and the +# one service whose absence makes the whole system inert: with no transport there +# are no rooms, so citizens have nothing to be resident IN and benchmarks are fed +# into a system that is not running. +# +# Two failure states, distinguished because their fixes differ (the same +# distinction `benchmark/dispatch` now draws between unregistered and +# not-resident): +# - airc BINARY ABSENT → nothing to enumerate, adopt or spawn. Warn and carry +# on; a box without airc installed is a different problem from a broken one, +# and refusing to boot would strand CI and fresh clones. +# - airc PRESENT but the daemon will not come up → FAIL LOUD, exit nonzero. +# A core with no transport is not a running system, and reporting success for +# one is the class of lie this whole card exists to end. +ensure_airc_daemon() { + if ! command -v airc >/dev/null 2>&1; then + echo "⚠ airc is NOT INSTALLED — the substrate has no transport." >&2 + echo " The core will launch, but citizens have no rooms and cannot hear each other." >&2 + return 0 + fi + + if bounded_run 5 airc ping; then + echo "✓ airc daemon: adopted (already answering)" >&2 + return 0 + fi + + # Not answering. If a daemon process exists it is WEDGED, and a wedged holder is + # worse than none — it answers nothing AND owns the socket, so a fresh spawn + # would lose the bind (airc's own start gives up on a contended lock, #355). + # Reap before spawning: graceful verb first, then the process. + if pgrep -f 'airc.*daemon' >/dev/null 2>&1; then + echo " airc daemon is wedged (holds the socket, answers nothing) — reaping" >&2 + bounded_run 5 airc stop || true + if pgrep -f 'airc.*daemon' >/dev/null 2>&1; then + pkill -f 'airc.*daemon' 2>/dev/null || true + sleep 1 + pkill -9 -f 'airc.*daemon' 2>/dev/null || true + fi + fi + + local airc_log="${HOME}/.airc/runtime/daemon-boot.log" + mkdir -p "$(dirname "$airc_log")" 2>/dev/null || true + echo " starting airc daemon (boot owns it, #452) → $airc_log" >&2 + nohup airc daemon >>"$airc_log" 2>&1 & + disown 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 30 ]; do + if bounded_run 5 airc ping; then + echo "✓ airc daemon: started and answering (${waited}s)" >&2 + return 0 + fi + sleep 1 + waited=$((waited + 1)) + done + + echo "❌ airc daemon did not answer within 30s of being started." >&2 + echo " The substrate has no transport: no rooms, no resident citizens, and any" >&2 + echo " benchmark dispatched now would post cards nobody can see. Last output:" >&2 + tail -20 "$airc_log" >&2 2>/dev/null || true + return 1 +} +if ! ensure_airc_daemon; then + exit 1 fi # ── Airc context ───────────────────────────────────────────────────── -# Substrate auto-discovers airc daemon socket via `airc ipc-endpoint` -# (task #80). The default room/channel come from `airc room` so the -# personas land in the same scope Joel's terminal sees. +# The daemon is guaranteed live by ensure_airc_daemon above, so this block now +# only DERIVES context from it. Substrate auto-discovers the airc daemon socket +# via `airc ipc-endpoint` (task #80). The default room/channel come from +# `airc room` so the personas land in the same scope Joel's terminal sees. if [ -z "$AIRC_DEFAULT_CHANNEL" ] || [ -z "$AIRC_DEFAULT_ROOM_NAME" ]; then if airc status >/dev/null 2>&1; then ROOM_OUT="$(airc room 2>/dev/null || true)" @@ -228,8 +369,13 @@ if [ -z "$AIRC_DEFAULT_CHANNEL" ] || [ -z "$AIRC_DEFAULT_ROOM_NAME" ]; then fi fi else - echo "⚠ airc daemon not running. Start it with: airc daemon" >&2 - echo " continuum-core-server will still launch but personas can't talk." >&2 + # Reachable ONLY when airc is not installed at all — `ensure_airc_daemon` + # above has already adopted, reaped-and-restarted, or exited nonzero, so a + # present-but-down daemon can no longer get this far. It used to say "start + # it with: airc daemon", which is the runbook line #452 replaced with the + # boot actually doing it. + echo "⚠ no airc daemon to derive room/channel from (airc is not installed)" >&2 + echo " the core will launch, but personas have no rooms and cannot talk." >&2 fi fi From 07c67e434c72f8b2661f146dd6a27e7a9be8d1c6 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 23:17:31 -0500 Subject: [PATCH 25/80] =?UTF-8?q?feat(persona):=20BaseModelPolicy=20?= =?UTF-8?q?=E2=80=94=20one=20answer=20to=20"what=20base=20does=20she=20thi?= =?UTF-8?q?nk=20on=3F"=20(#126/#369/#438)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel asked where the base model is governed. The honest answer was FOUR places, in one function, each silently handing off to the next: 0. override_model — runtime per-persona assignment 1. model_preferences — tiered ladder, "best that fits" 2. model_id — explicit per-persona model, labelled "Legacy" 3. default_local_model — system-wide default Four expressions of one decision, and rows 1→2→3 walk DOWN without anyone deciding to. That is the #438 incident shape: one bogus usable_gb=0 sample walked the ladder to the bottom and served citizens a 0.5B that emitted template-token garbage into the room. The sibling module already forbade this in prose (inference_profile.rs: "substrate HARD ERRORS instead of silently degrading") — the prose was right, the code never implemented it. This is a COMPRESSION, 4 → 1, and both modes fall out of it: - Pegged{model, reason} — subsumes override_model (Measurement peg w/ lease) and model_id (Operator). This base or she does not serve. - Adaptive{ladder, floor, rungs} — the governor moves her, bounded below by a NAMED floor member (not "last entry wins", which is how a reordered ladder silently moves its own floor). No system-wide-default arm exists. That arm WAS the bug. THE GENOME IS WHAT MAKES A BASE REACHABLE (Joel, this session): a LoRA adapter is a per-base derivative. The corpus — transcripts, tool traces, solved instances — is the durable, base-independent asset; an adapter is forged FROM it ONTO a specific base and is invalid on any other (#369). So she can scale up or down freely, PROVIDED her genome was forged for the destination. Hence GenomeCoverage as an input: RequireGenome (default) skips rungs she has no adapter for and refuses with the FORGE work-list; AllowBare takes them but reports genome_backed=false so a measurement records WHICH citizen it scored. The paired obligation is the forge's, not this module's: one corpus fans out to N adapters, one per targeted base, each trained independently against that base. A ladder is only as walkable as the forge made it, and RequireGenome turns a declared-but-unforged ladder from a silent capability cliff into a visible one. Why benchmarks need this: genome lift is only measurable against a fixed base. Round 1 on a known base → corpus accrues → forge onto that same base → round 2, same base + adapter → the delta is attributable to the genome. Float the base and the number means nothing AND round 1's adapters are garbage. Worse, a silent re-base corrupts the EXPERIENCE, not just the score: turns taken on an unintended base still land in the corpus, poisoning the next adapter forged from it. So refusing to serve is the cheap failure; serving on an unintended base is the expensive one, and it is the one that used to happen by default. Pure over (host_vram_gb, coverage) — 15 tests, no GPU, no forge, no registry. Every refusal names its own remedy and is asserted on. NOT YET WIRED: allocator/inference_profile still use the 4-way resolver. This is the type + its truth table landing first so the cut-over is mechanical. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/base_model_policy.rs | 666 ++++++++++++++++++ core/continuum-core/src/persona/mod.rs | 1 + .../typescript/persona/BaseModelPolicy.ts | 13 + protocol/typescript/persona/BaseModelRung.ts | 23 + protocol/typescript/persona/PegReason.ts | 7 + protocol/typescript/persona/RungPolicy.ts | 6 + 6 files changed, 716 insertions(+) create mode 100644 core/continuum-core/src/persona/base_model_policy.rs create mode 100644 protocol/typescript/persona/BaseModelPolicy.ts create mode 100644 protocol/typescript/persona/BaseModelRung.ts create mode 100644 protocol/typescript/persona/PegReason.ts create mode 100644 protocol/typescript/persona/RungPolicy.ts diff --git a/core/continuum-core/src/persona/base_model_policy.rs b/core/continuum-core/src/persona/base_model_policy.rs new file mode 100644 index 000000000..85c5d1018 --- /dev/null +++ b/core/continuum-core/src/persona/base_model_policy.rs @@ -0,0 +1,666 @@ +//! `BaseModelPolicy` — the ONE answer to "what base model does this citizen think on?" +//! +//! ## Why this type exists +//! +//! `resolve_model_for_persona` grew FOUR ways to answer that question, each silently +//! handing off to the next: +//! +//! | precedence | source | what it meant | +//! |---|---|---| +//! | 0 | `override_model` | runtime per-persona assignment | +//! | 1 | `model_preferences` | tiered ladder, "best that fits" | +//! | 2 | `model_id` | explicit per-persona model, labelled *"Legacy"* | +//! | 3 | `default_local_model` | system-wide default | +//! +//! Four expressions of one decision is the compression violation the whole codebase is +//! built against, and it had two concrete costs. First, nobody could answer "where is the +//! base model governed?" — the honest answer was "in four places." Second, and worse, the +//! hand-offs are **silent downgrades**: rows 1→2→3 walk *down* without anyone deciding to, +//! which is the 2026-08-15 incident shape (#438) — one bogus `usable_gb=0` sample walked +//! the ladder to the bottom and served citizens a 0.5B that emitted template-token garbage +//! into the room. The sibling module already forbids exactly this in prose +//! (`inference_profile.rs`: *"substrate HARD ERRORS with diagnosis instead of silently +//! degrading"*) — the prose was right and the code did not implement it. +//! +//! ## The genome is what makes a base REACHABLE +//! +//! A LoRA adapter is a **per-base derivative**. The durable asset is the CORPUS +//! (transcripts, tool traces, solved instances), which is base-independent; an adapter is +//! forged from that corpus *onto a specific base* and is invalid on any other (#369). +//! +//! So a citizen is not stuck on one base — she can scale up to a bigger one or down to a +//! smaller one **provided her genome has been forged for the destination**. That is the +//! actual constraint, and it is why this module takes a [`GenomeCoverage`] rather than +//! treating the ladder as freely walkable: a rung she has no adapter for is a rung where +//! she thinks with base capability only. Reachable, sometimes correct, but *a different +//! citizen* than the one the previous round measured. +//! +//! The paired obligation lives in the forge, not here: **one corpus fans out to N +//! adapters, one per targeted base, each trained independently against that base.** A +//! ladder is only as walkable as the forge made it. A policy declaring four rungs while +//! the forge only ever targeted one is a ladder with three rungs missing, and +//! [`RungPolicy::RequireGenome`] is what turns that from a silent capability cliff into a +//! visible one. +//! +//! Two consequences, and they are why benchmarks need this type: +//! +//! 1. **Genome lift is only measurable against a fixed base.** Round 1 on a known base → +//! corpus accrues → forge an adapter onto that same base → round 2, same base + adapter +//! → the delta is attributable to the genome. Float the base between rounds and the +//! number means nothing AND the round-1 adapters are garbage on the new base. +//! 2. **A silent re-base corrupts the experience, not just the score.** Turns taken on an +//! unintended base still land in the corpus. A round that quietly slid to a 0.5B +//! doesn't merely score badly — it poisons the training data the next adapter is forged +//! from. +//! +//! So refusing to serve is the *cheap* failure. Serving on an unintended base is the +//! expensive one, and it is the one that used to happen by default. +//! +//! ## What this module does NOT decide +//! +//! Whether the weights are on disk, whether a lane can be spawned, whether the host is +//! under pressure right now. This is a pure policy → `(model, vram_budget)` decision over +//! a declared host VRAM figure, so it is unit-testable without a GPU, a registry, or a +//! running lane. Availability and lane admission stay where they already live +//! (`model_registry`, `inference::llama_server`), and a caller that resolves a model it +//! then cannot load must fail loudly there — not by coming back here for a second guess. + +use serde::{Deserialize, Serialize}; +use ts_rs::TS; + +/// One rung of an [`BaseModelPolicy::Adaptive`] ladder: a model that applies when the host +/// has at least `min_vram_gb`. +/// +/// Structurally identical to the existing catalog `ModelPreference` (this is deliberately +/// the same shape, so the catalog's ladders port over unchanged rather than growing a +/// parallel encoding of "which model at which size"). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, TS)] +#[ts( + export, + export_to = "../../../protocol/typescript/persona/BaseModelRung.ts" +)] +#[serde(rename_all = "camelCase")] +pub struct BaseModelRung { + /// Minimum total host VRAM (GB) for this rung to apply. + pub min_vram_gb: f64, + /// The model id this rung selects. + pub model: String, + /// VRAM this persona needs when thinking on this model. + pub vram_budget_gb: f64, +} + +/// WHY a citizen is pegged — carried so a refusal can explain itself, and so the +/// measurement peg can be told apart from a durable one when a lease expires. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, TS)] +#[ts(export, export_to = "../../../protocol/typescript/persona/PegReason.ts")] +#[serde(rename_all = "camelCase", tag = "kind")] +pub enum PegReason { + /// A measurement is in flight and the number must be attributable to a known base. + /// Held for the round's duration and released with it — the round-lifecycle owner + /// (#371) sets and clears this, never the solver. + Measurement { run_id: String }, + /// Her genome's adapters are forged against this base and are invalid on any other + /// (#369). Derived from the adapters she actually holds, not hand-declared. + GenomeBound, + /// Operator intent — "this citizen runs on this model." + Operator, +} + +/// What base model a citizen thinks on. ONE decision, one place. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, TS)] +#[ts( + export, + export_to = "../../../protocol/typescript/persona/BaseModelPolicy.ts" +)] +#[serde(rename_all = "camelCase", tag = "mode")] +pub enum BaseModelPolicy { + /// Pinned: this base or nothing. Subsumes the old `override_model` (as a + /// [`PegReason::Measurement`] peg with a lease) and the old `model_id` (as + /// [`PegReason::Operator`]). + Pegged { + model: String, + vram_budget_gb: f64, + reason: PegReason, + }, + /// Governed within bounds: walk the ladder for the best rung the host can carry AND + /// her genome can reach, and REFUSE rather than descend past `floor`. + /// + /// `floor` names a model that must appear in `ladder`. It is a named member rather + /// than an index so a ladder can be reordered or extended without silently moving the + /// floor — the failure mode of every "last entry wins" rule. + Adaptive { + ladder: Vec, + floor: String, + /// What to do with a rung her genome was never forged for. + #[serde(default)] + rungs: RungPolicy, + }, +} + +/// What a rung with no adapter means for this citizen. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize, TS)] +#[ts(export, export_to = "../../../protocol/typescript/persona/RungPolicy.ts")] +#[serde(rename_all = "camelCase")] +pub enum RungPolicy { + /// Skip rungs her genome has not been forged for. The default, because a citizen who + /// silently loses her genome mid-ladder is the same class of surprise as one who + /// silently changes base — she is measurably a different worker and nothing said so. + /// + /// This makes an unforged ladder VISIBLE: declare four rungs, forge one, and she + /// resolves to the one that exists instead of appearing to have a four-rung range. + #[default] + RequireGenome, + /// Take the best-fitting rung regardless, thinking with base capability alone where + /// no adapter exists. Legitimate — a bare frontier base may well beat a small + /// genome-backed one — but the result is flagged + /// [`ResolvedBase::genome_backed`]` == false` so a measurement records WHICH citizen + /// it scored, and a lift comparison can refuse to compare across that line. + AllowBare, +} + +/// Which bases this citizen's genome has actually been forged for. +/// +/// Derived from the adapters on disk, never declared — a config field claiming coverage +/// the forge never produced is exactly the lying-receipt shape this module exists to end. +#[derive(Debug, Clone, Default, PartialEq)] +pub struct GenomeCoverage { + forged_for: std::collections::BTreeSet, +} + +impl GenomeCoverage { + /// Build from the base ids her adapters declare (#369 — every adapter carries the + /// `base_model_id` it was forged against). + pub fn from_adapter_bases(bases: I) -> Self + where + I: IntoIterator, + S: Into, + { + Self { + forged_for: bases.into_iter().map(Into::into).collect(), + } + } + + /// Has her corpus been forged onto this base? + pub fn covers(&self, model: &str) -> bool { + self.forged_for.contains(model) + } + + /// Every base she can think on WITH her genome — the honest answer to "how far can + /// she scale?", and the work-list for the forge fan-out when the answer is "not far". + pub fn bases(&self) -> impl Iterator { + self.forged_for.iter().map(String::as_str) + } +} + +/// The resolved answer: which model, the VRAM she needs on it, and whether her genome +/// actually reaches it. +#[derive(Debug, Clone, PartialEq)] +pub struct ResolvedBase { + pub model: String, + pub vram_budget_gb: f64, + /// Is her corpus forged onto THIS base? + /// + /// `false` means she is thinking with base capability alone. Load-bearing for + /// measurement: a score from a bare rung and a score from a genome-backed rung are + /// numbers about two different workers, and a lift comparison that mixes them is + /// measuring the base swap, not the genome. + pub genome_backed: bool, +} + +/// Why a policy could not be satisfied. Every variant is a REFUSAL that names its own +/// remedy — there is no "and so we used something smaller" arm, by construction. +#[derive(Debug, Clone, PartialEq)] +pub enum BaseModelError { + /// A peg that does not fit this host. She is honestly absent rather than silently + /// re-based: a citizen serving on a base her adapters were not forged against + /// produces turns that corrupt the corpus AND a score attributable to nothing. + PegDoesNotFit { + model: String, + reason: PegReason, + needs_gb: f64, + host_gb: f64, + }, + /// Even the declared floor does not fit. The host is too small for this citizen as + /// configured; lowering the floor is a DECISION, not something resolution may take. + FloorDoesNotFit { + floor: String, + needs_gb: f64, + host_gb: f64, + }, + /// The floor names a model absent from the ladder — a malformed policy, caught at + /// resolution rather than silently ignored (an unenforceable floor is worse than no + /// floor, because it reads as protection). + FloorNotOnLadder { floor: String }, + /// An adaptive policy with no rungs. There is no system-wide default to fall back to: + /// that fallback WAS the bug. + EmptyLadder, + /// Rungs FIT this host, but her genome was never forged onto any of them. Not a + /// hardware problem — a **forge** problem, and the remedy is a training run, so the + /// refusal carries the exact work-list: which bases to target, and which she already + /// has. This is the state a declared-but-unforged ladder is actually in, made visible + /// instead of silently handing back a citizen without her genome. + NoForgedRungFits { + fits_but_unforged: Vec, + forged_for: Vec, + }, +} + +impl std::fmt::Display for BaseModelError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::PegDoesNotFit { + model, + reason, + needs_gb, + host_gb, + } => write!( + f, + "pegged base {model} needs {needs_gb:.1} GB but this host has {host_gb:.1} GB \ + (peg held because: {reason:?}). Refusing to re-base her: her genome's \ + adapters are forged against {model} and her turns would corrupt the corpus. \ + Free VRAM, move her to a host that fits, or change the peg deliberately." + ), + Self::FloorDoesNotFit { + floor, + needs_gb, + host_gb, + } => write!( + f, + "adaptive floor {floor} needs {needs_gb:.1} GB but this host has \ + {host_gb:.1} GB. Descending past the floor is a decision, not a fallback \ + — lower the floor explicitly if a smaller base is acceptable for her." + ), + Self::FloorNotOnLadder { floor } => write!( + f, + "floor {floor} does not appear on this citizen's ladder — the floor is \ + unenforceable as written, which reads as protection and is not" + ), + Self::EmptyLadder => write!( + f, + "adaptive policy with no rungs, and there is no system-wide default to \ + fall back to (that fallback was the #438 downgrade)" + ), + Self::NoForgedRungFits { + fits_but_unforged, + forged_for, + } => write!( + f, + "base(s) [{}] fit this host but her genome was never forged onto any of \ + them; she is forged for [{}]. This is a FORGE gap, not a hardware one — \ + the ladder is declared wider than the corpus has been trained out to. \ + Fan the forge out to the missing target(s), or set rungs=allowBare to \ + run her here without her genome (the result is then a different worker \ + and is reported as genome_backed=false).", + fits_but_unforged.join(", "), + if forged_for.is_empty() { + "nothing yet".to_string() + } else { + forged_for.join(", ") + } + ), + } + } +} + +impl BaseModelPolicy { + /// Resolve to a concrete base, or refuse with a reason. + /// + /// Pure over `(host_vram_gb, coverage)` so every mode and every refusal is unit-pinned + /// without a GPU or a forge. Ladder order is authoritative: rungs are tried top-down + /// and the FIRST that both fits AND is reachable wins, so a caller controls preference + /// by ordering, never by a tie-break here. + /// + /// `coverage` is what makes scaling real rather than nominal: a citizen may move to + /// any base her genome was forged for, and [`RungPolicy::RequireGenome`] skips the + /// rest rather than silently handing back a differently-skilled worker. + pub fn resolve( + &self, + host_vram_gb: f64, + coverage: &GenomeCoverage, + ) -> Result { + match self { + Self::Pegged { + model, + vram_budget_gb, + reason, + } => { + if host_vram_gb >= *vram_budget_gb { + Ok(ResolvedBase { + model: model.clone(), + vram_budget_gb: *vram_budget_gb, + genome_backed: coverage.covers(model), + }) + } else { + // The whole point of the type: a peg that does not fit REFUSES. + Err(BaseModelError::PegDoesNotFit { + model: model.clone(), + reason: reason.clone(), + needs_gb: *vram_budget_gb, + host_gb: host_vram_gb, + }) + } + } + Self::Adaptive { + ladder, + floor, + rungs, + } => { + if ladder.is_empty() { + return Err(BaseModelError::EmptyLadder); + } + let floor_idx = ladder + .iter() + .position(|r| &r.model == floor) + .ok_or_else(|| BaseModelError::FloorNotOnLadder { + floor: floor.clone(), + })?; + let mut fits_but_unforged: Vec = Vec::new(); + // Walk DOWN to the floor inclusive — never past it. + for rung in &ladder[..=floor_idx] { + if host_vram_gb < rung.min_vram_gb { + continue; + } + let forged = coverage.covers(&rung.model); + if !forged && *rungs == RungPolicy::RequireGenome { + // She could physically run here, but her corpus was never forged + // onto this base — taking it would silently swap in a differently + // skilled worker. Remember it so the refusal can name the forge + // work that would open the rung. + fits_but_unforged.push(rung.model.clone()); + continue; + } + return Ok(ResolvedBase { + model: rung.model.clone(), + vram_budget_gb: rung.vram_budget_gb, + genome_backed: forged, + }); + } + if !fits_but_unforged.is_empty() { + return Err(BaseModelError::NoForgedRungFits { + fits_but_unforged, + forged_for: coverage.bases().map(str::to_string).collect(), + }); + } + let floor_rung = &ladder[floor_idx]; + Err(BaseModelError::FloorDoesNotFit { + floor: floor.clone(), + needs_gb: floor_rung.min_vram_gb, + host_gb: host_vram_gb, + }) + } + } + } + + /// The model this policy names when it is a peg — the identity a genome page-in checks + /// its adapters against (#369), and what a roster row reports as her pinned base. + pub fn pegged_model(&self) -> Option<&str> { + match self { + Self::Pegged { model, .. } => Some(model.as_str()), + Self::Adaptive { .. } => None, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn rung(min: f64, model: &str, budget: f64) -> BaseModelRung { + BaseModelRung { + min_vram_gb: min, + model: model.to_string(), + vram_budget_gb: budget, + } + } + + /// Coverage for a citizen whose corpus has been forged onto EVERY rung — the + /// "fully fanned-out forge" case, which is what the pre-coverage tests assumed. + fn all_forged() -> GenomeCoverage { + GenomeCoverage::from_adapter_bases(["big-27b", "mid-7b", "small-1b", "tiny-0.5b"]) + } + + fn ladder() -> Vec { + vec![ + rung(40.0, "big-27b", 32.0), + rung(16.0, "mid-7b", 12.0), + rung(4.0, "small-1b", 3.0), + rung(1.0, "tiny-0.5b", 1.0), + ] + } + + // what this catches: THE #438 INCIDENT. A host reading low must NOT slide to the + // bottom of the ladder. With a floor at mid-7b and a host that cannot carry it, + // resolution REFUSES — it does not quietly serve small-1b or tiny-0.5b. Before this + // type, `resolve_model_for_persona` fell through to "use last entry (lowest tier)" + // and then to a system default, which is how citizens got served a 0.5B that emitted + // template-token garbage into the room. + #[test] + fn adaptive_refuses_rather_than_sliding_below_its_floor() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "mid-7b".into(), + rungs: RungPolicy::RequireGenome, + }; + let err = p.resolve(8.0, &all_forged()).unwrap_err(); + assert_eq!( + err, + BaseModelError::FloorDoesNotFit { + floor: "mid-7b".into(), + needs_gb: 16.0, + host_gb: 8.0 + } + ); + let msg = err.to_string(); + assert!( + msg.contains("Descending past the floor is a decision"), + "the refusal must name the remedy, not just fail: {msg}" + ); + } + + // what this catches: the floor is INCLUSIVE — a host that exactly fits the floor gets + // the floor, not a refusal. An off-by-one here would make the floor unreachable and + // every floor-sized host non-resident. + #[test] + fn adaptive_serves_the_floor_when_the_host_exactly_fits_it() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "mid-7b".into(), + rungs: RungPolicy::RequireGenome, + }; + let got = p.resolve(16.0, &all_forged()).unwrap(); + assert_eq!(got.model, "mid-7b"); + assert_eq!(got.vram_budget_gb, 12.0); + } + + // what this catches: scaling UP still works — a big host takes the top rung. The floor + // bounds the DOWN direction only; adding it must not pin everyone to the floor. + #[test] + fn adaptive_still_scales_up_to_the_best_rung_that_fits() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "small-1b".into(), + rungs: RungPolicy::RequireGenome, + }; + assert_eq!(p.resolve(64.0, &all_forged()).unwrap().model, "big-27b"); + assert_eq!(p.resolve(20.0, &all_forged()).unwrap().model, "mid-7b"); + assert_eq!(p.resolve(5.0, &all_forged()).unwrap().model, "small-1b"); + } + + // what this catches: a PEG never degrades. This is the property that makes a benchmark + // number attributable — if the pegged base cannot be served, the honest outcome is + // that she does not serve, NOT that she serves on something else and we score it as + // though it were the peg. It also protects the corpus: turns taken on an unintended + // base become training data for an adapter forged against the pegged one. + #[test] + fn a_peg_refuses_instead_of_re_basing() { + let p = BaseModelPolicy::Pegged { + model: "big-27b".into(), + vram_budget_gb: 32.0, + reason: PegReason::Measurement { + run_id: "swe-lite-round-1".into(), + }, + }; + assert_eq!(p.resolve(64.0, &all_forged()).unwrap().model, "big-27b"); + + let err = p.resolve(16.0, &all_forged()).unwrap_err(); + match &err { + BaseModelError::PegDoesNotFit { model, reason, .. } => { + assert_eq!(model, "big-27b"); + assert!(matches!(reason, PegReason::Measurement { .. })); + } + other => panic!("a peg that does not fit must refuse, got {other:?}"), + } + let msg = err.to_string(); + assert!( + msg.contains("corrupt the corpus"), + "the refusal must say WHY re-basing is worse than absence: {msg}" + ); + } + + // what this catches: a floor naming a model that isn't on the ladder is a MALFORMED + // policy, not a no-op. Silently ignoring it would leave an unenforceable floor that + // reads as protection — the failure shape where a guard exists on paper and the slide + // happens anyway. + #[test] + fn a_floor_absent_from_the_ladder_is_a_loud_policy_error() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "not-a-rung".into(), + rungs: RungPolicy::RequireGenome, + }; + assert_eq!( + p.resolve(64.0, &all_forged()).unwrap_err(), + BaseModelError::FloorNotOnLadder { + floor: "not-a-rung".into() + } + ); + } + + // what this catches: there is NO system-wide default arm. An empty ladder refuses. + // The old resolver's last act was `default_local_model` — a fallback that turned a + // configuration gap into a silently wrong base. + #[test] + fn an_empty_ladder_has_nothing_to_fall_back_to_and_says_so() { + let p = BaseModelPolicy::Adaptive { + ladder: vec![], + floor: "anything".into(), + rungs: RungPolicy::RequireGenome, + }; + assert_eq!(p.resolve(64.0, &all_forged()).unwrap_err(), BaseModelError::EmptyLadder); + } + + // what this catches: THE RULE — she may scale to any base her genome was forged for, + // and no further. Forged only for mid-7b, sitting on a host that could carry big-27b: + // she resolves to mid-7b, NOT big-27b. Taking the bigger rung would hand back a + // citizen without her genome while the caller believed it was scaling her UP. + #[test] + fn scaling_up_stops_at_the_highest_base_her_genome_was_forged_for() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "small-1b".into(), + rungs: RungPolicy::RequireGenome, + }; + let only_mid = GenomeCoverage::from_adapter_bases(["mid-7b"]); + let got = p.resolve(64.0, &only_mid).unwrap(); + assert_eq!( + got.model, "mid-7b", + "the host could carry big-27b, but her corpus was never forged onto it" + ); + assert!(got.genome_backed); + } + + // what this catches: the same rule going DOWN. Forged only for big-27b, on a host that + // can only carry mid-7b — she does not quietly drop to a base she has no adapter for. + // The refusal names the FORGE work, because that (not hardware) is the actual remedy: + // one corpus, fanned out to the missing target. + #[test] + fn scaling_down_to_an_unforged_base_is_a_forge_gap_and_names_the_work() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "tiny-0.5b".into(), + rungs: RungPolicy::RequireGenome, + }; + let only_big = GenomeCoverage::from_adapter_bases(["big-27b"]); + let err = p.resolve(20.0, &only_big).unwrap_err(); + match &err { + BaseModelError::NoForgedRungFits { + fits_but_unforged, .. + } => { + assert!( + fits_but_unforged.contains(&"mid-7b".to_string()), + "must name the rung she could have taken had it been forged: {fits_but_unforged:?}" + ); + } + other => panic!("an unforged-but-fitting rung must refuse, got {other:?}"), + } + let msg = err.to_string(); + assert!( + msg.contains("FORGE gap, not a hardware one") && msg.contains("Fan the forge out"), + "the refusal must point at training, not at buying a bigger box: {msg}" + ); + } + + // what this catches: AllowBare is a real, permitted mode — a bare frontier base may + // well beat a small genome-backed one — but the result must be LABELLED. A score from + // genome_backed=false and one from genome_backed=true are numbers about two different + // workers; mixing them measures the base swap, not the genome. + #[test] + fn allow_bare_takes_the_rung_but_reports_it_as_not_genome_backed() { + let p = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "small-1b".into(), + rungs: RungPolicy::AllowBare, + }; + let only_mid = GenomeCoverage::from_adapter_bases(["mid-7b"]); + let got = p.resolve(64.0, &only_mid).unwrap(); + assert_eq!(got.model, "big-27b", "bare scaling up IS allowed when asked for"); + assert!( + !got.genome_backed, + "…but the caller must be able to see she is running without her genome" + ); + } + + // what this catches: a PEG onto a base her genome doesn't cover still resolves (the peg + // is the operator's/measurement's call) but reports genome_backed=false. Silently + // claiming genome backing on a base with no adapter would make a lift number that + // compares a bare run against a forged one and calls the difference "learning". + #[test] + fn a_peg_onto_an_unforged_base_serves_but_admits_it_is_bare() { + let p = BaseModelPolicy::Pegged { + model: "big-27b".into(), + vram_budget_gb: 32.0, + reason: PegReason::Measurement { + run_id: "baseline-round-0".into(), + }, + }; + let got = p + .resolve(64.0, &GenomeCoverage::from_adapter_bases(["mid-7b"])) + .unwrap(); + assert_eq!(got.model, "big-27b"); + assert!( + !got.genome_backed, + "a baseline round on an unforged base is exactly the round you WANT — it just \ + has to be labelled so round 2's lift is attributable" + ); + } + + // what this catches: `pegged_model` is what a genome page-in checks adapters against + // (#369) and what the roster reports as her pinned base. An adaptive citizen has no + // pinned base, and reporting one would be the same lie in a different field. + #[test] + fn only_a_pegged_citizen_reports_a_pinned_base() { + let pegged = BaseModelPolicy::Pegged { + model: "big-27b".into(), + vram_budget_gb: 32.0, + reason: PegReason::GenomeBound, + }; + assert_eq!(pegged.pegged_model(), Some("big-27b")); + + let adaptive = BaseModelPolicy::Adaptive { + ladder: ladder(), + floor: "small-1b".into(), + rungs: RungPolicy::RequireGenome, + }; + assert_eq!(adaptive.pegged_model(), None); + } +} diff --git a/core/continuum-core/src/persona/mod.rs b/core/continuum-core/src/persona/mod.rs index ed17c0356..3b576b564 100644 --- a/core/continuum-core/src/persona/mod.rs +++ b/core/continuum-core/src/persona/mod.rs @@ -30,6 +30,7 @@ pub mod active_work_source; pub mod airc_runtime_registry; pub mod airc_source; pub mod allocator; +pub mod base_model_policy; pub mod cached_source; pub mod card; pub mod card_holder; diff --git a/protocol/typescript/persona/BaseModelPolicy.ts b/protocol/typescript/persona/BaseModelPolicy.ts new file mode 100644 index 000000000..b59768daa --- /dev/null +++ b/protocol/typescript/persona/BaseModelPolicy.ts @@ -0,0 +1,13 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { BaseModelRung } from "./BaseModelRung"; +import type { PegReason } from "./PegReason"; +import type { RungPolicy } from "./RungPolicy"; + +/** + * What base model a citizen thinks on. ONE decision, one place. + */ +export type BaseModelPolicy = { "mode": "pegged", model: string, vram_budget_gb: number, reason: PegReason, } | { "mode": "adaptive", ladder: Array, floor: string, +/** + * What to do with a rung her genome was never forged for. + */ +rungs: RungPolicy, }; diff --git a/protocol/typescript/persona/BaseModelRung.ts b/protocol/typescript/persona/BaseModelRung.ts new file mode 100644 index 000000000..9e4d53142 --- /dev/null +++ b/protocol/typescript/persona/BaseModelRung.ts @@ -0,0 +1,23 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * One rung of an [`BaseModelPolicy::Adaptive`] ladder: a model that applies when the host + * has at least `min_vram_gb`. + * + * Structurally identical to the existing catalog `ModelPreference` (this is deliberately + * the same shape, so the catalog's ladders port over unchanged rather than growing a + * parallel encoding of "which model at which size"). + */ +export type BaseModelRung = { +/** + * Minimum total host VRAM (GB) for this rung to apply. + */ +minVramGb: number, +/** + * The model id this rung selects. + */ +model: string, +/** + * VRAM this persona needs when thinking on this model. + */ +vramBudgetGb: number, }; diff --git a/protocol/typescript/persona/PegReason.ts b/protocol/typescript/persona/PegReason.ts new file mode 100644 index 000000000..6f73c862f --- /dev/null +++ b/protocol/typescript/persona/PegReason.ts @@ -0,0 +1,7 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * WHY a citizen is pegged — carried so a refusal can explain itself, and so the + * measurement peg can be told apart from a durable one when a lease expires. + */ +export type PegReason = { "kind": "measurement", run_id: string, } | { "kind": "genomeBound" } | { "kind": "operator" }; diff --git a/protocol/typescript/persona/RungPolicy.ts b/protocol/typescript/persona/RungPolicy.ts new file mode 100644 index 000000000..b7e86afff --- /dev/null +++ b/protocol/typescript/persona/RungPolicy.ts @@ -0,0 +1,6 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * What a rung with no adapter means for this citizen. + */ +export type RungPolicy = "requireGenome" | "allowBare"; From 83667702107b3151801ed0e0e923171431b64282 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Mon, 17 Aug 2026 23:28:58 -0500 Subject: [PATCH 26/80] =?UTF-8?q?feat(persona):=20PegReason::Training=20?= =?UTF-8?q?=E2=80=94=20the=20peg's=20biggest=20job=20is=20targeting=20the?= =?UTF-8?q?=20forge?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Correcting my own framing from an hour ago: I called the peg "only for measurement". Too narrow, and Joel's point is the load-bearing one. Continuous learning never stops. The flywheel runs turns → corpus → forge → adapter → page-in continuously, and the adapter coming out the far end is a derivative of whatever base it was trained against. If her base floats between the corpus accruing and the forge finishing, the adapter lands for a base she has already left — forged for nobody, and page-in must then refuse it (#369). The corpus survives that; the training compute does not. So a training run holds a peg for its duration, exactly as a measurement round does, and for the same underlying reason: a process is in flight whose output is only valid against the base it started on. Measurement is episodic; training is always running, which makes THIS the peg held most of the time. GenomeBound is now documented as what it actually is — the standing consequence of past Training runs, derived from the adapters she holds. Also worth stating plainly, since it corrects a second drift in my framing: the LADDER is the mechanism for daily use, not the peg. A citizen on a 64GB box resolves to the top rung automatically — no peg, no special case — and a smaller box steps down the same ladder. The peg exists for the three cases where something in flight depends on the base staying put. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/base_model_policy.rs | 45 ++++++++++++++++++- protocol/typescript/persona/PegReason.ts | 2 +- 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/core/continuum-core/src/persona/base_model_policy.rs b/core/continuum-core/src/persona/base_model_policy.rs index 85c5d1018..264b83784 100644 --- a/core/continuum-core/src/persona/base_model_policy.rs +++ b/core/continuum-core/src/persona/base_model_policy.rs @@ -99,8 +99,22 @@ pub enum PegReason { /// Held for the round's duration and released with it — the round-lifecycle owner /// (#371) sets and clears this, never the solver. Measurement { run_id: String }, + /// A FORGE RUN is in flight and is targeting this base. + /// + /// This is the peg with the most utility, because continuous learning never stops: the + /// flywheel runs turns → corpus → forge → adapter → page-in continuously, and the + /// adapter coming out the far end is a derivative of whatever base it was trained + /// against. If her base floats between the corpus accruing and the forge finishing, the + /// adapter lands for a base she has already left — forged for nobody, and page-in must + /// then refuse it (#369). + /// + /// So a training run holds this for its duration, exactly as a measurement round holds + /// [`Self::Measurement`], and for the same underlying reason: a process is in flight + /// whose output is only valid against the base it started on. + Training { job_id: String }, /// Her genome's adapters are forged against this base and are invalid on any other - /// (#369). Derived from the adapters she actually holds, not hand-declared. + /// (#369). Derived from the adapters she actually holds, not hand-declared — the + /// standing consequence of past [`Self::Training`] runs. GenomeBound, /// Operator intent — "this citizen runs on this model." Operator, @@ -644,6 +658,35 @@ mod tests { ); } + // what this catches: a TRAINING peg refuses exactly like a measurement one. Continuous + // learning runs forever, so this is the peg that is held most of the time — and the + // failure it prevents is the worst of the three: a forge that starts on one base and + // finishes after she has drifted to another produces an adapter for nobody, which + // page-in must then refuse (#369). The corpus survives; the compute does not. + #[test] + fn a_training_peg_holds_the_base_for_the_duration_of_the_forge() { + let p = BaseModelPolicy::Pegged { + model: "big-27b".into(), + vram_budget_gb: 32.0, + reason: PegReason::Training { + job_id: "forge-coder-act-v3".into(), + }, + }; + assert_eq!(p.resolve(64.0, &all_forged()).unwrap().model, "big-27b"); + + let err = p.resolve(16.0, &all_forged()).unwrap_err(); + match &err { + BaseModelError::PegDoesNotFit { reason, .. } => { + assert!( + matches!(reason, PegReason::Training { .. }), + "the refusal must carry WHY, so an operator can tell a live forge from \ + an operator pin: {reason:?}" + ); + } + other => panic!("a training peg that does not fit must refuse, got {other:?}"), + } + } + // what this catches: `pegged_model` is what a genome page-in checks adapters against // (#369) and what the roster reports as her pinned base. An adaptive citizen has no // pinned base, and reporting one would be the same lie in a different field. diff --git a/protocol/typescript/persona/PegReason.ts b/protocol/typescript/persona/PegReason.ts index 6f73c862f..8b4bcc40a 100644 --- a/protocol/typescript/persona/PegReason.ts +++ b/protocol/typescript/persona/PegReason.ts @@ -4,4 +4,4 @@ * WHY a citizen is pegged — carried so a refusal can explain itself, and so the * measurement peg can be told apart from a durable one when a lease expires. */ -export type PegReason = { "kind": "measurement", run_id: string, } | { "kind": "genomeBound" } | { "kind": "operator" }; +export type PegReason = { "kind": "measurement", run_id: string, } | { "kind": "training", job_id: string, } | { "kind": "genomeBound" } | { "kind": "operator" }; From 18ecef781cfc807c21567d707c93efa848054d1f Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 00:07:43 -0500 Subject: [PATCH 27/80] feat(persona): the work turn roots her hands at the card AND feeds the genome (#456) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel: "learning and score are both the objective. We should outcompete the base model." The learning half was structurally impossible. This is why. MEASURED FROM SOURCE: `training_producer::produce` — the L2 step that turns a turn into a (context, completion) training example and submits it to the forge — had exactly ONE call site tree-wide: the message-REPLY path. Its own comment explains the exclusion: "It lives ONLY on this live `Spoke` path, which eval forks (`drive_to_settle`) never run, so the training set can never be contaminated by a measurement simulation." Correct for eval (#59). But it means: - `agent/solve` (which IS `drive_to_settle`) produces zero training examples, so a dispatched bench card can never reach the genome no matter that dispatch sets `learn: LearnFromThisWork` — the gate is PATH-based, the policy is never consulted, the policy loses. - AND the WORK TURN — the separate drive at the heart of "working is not speaking" (#154), the turn where she actually works her CLAIMED CARD — also never fed the producer. Chat produced training data; real work did not. That second one is the sharper defect and it is fixed here, in the live loop, without touching the eval guard. TWO CHANGES, both in the held-work turn: 1. HANDS FOLLOW THE CARD. A held card may be a staged benchmark checkout — a real git repo under `workspace/swe/`. Without rooting her hands there she "works" it by writing into her OWN workspace and the grader's `git diff` on the sandbox scores a false ZERO — exactly the defect glass-boxed on agent/solve 2026-07-22 (two real acts, correct file written, empty patch). Now the work turn roots at the card's workspace and RESTORES on every exit path (#312: after a flask solve, Anwen's live self was still reading the exam repo hours later). A root failure is LOUD and she works in her own dir anyway — an unexplained empty diff is worse than a diagnosed one. 2. THE PRODUCER FIRES. context = the card burst she was handed, completion = the report she wrote after doing the work. Same shape, same best-effort spawn, same quality bar as the reply path. Still the LIVE path — an eval fork does not reach this call site — so the contamination guard is unchanged. NEW: `persona/staged_workspace.rs` — ONE answer to "which workspace does this card point at", replacing the walk that `persona/roster` and `dispatch_staged_swe_solve` each did inline. The layout knowledge that makes a citizen able to act in the repo previously lived only inside a benchmark dispatcher, so the live path could not reuse it without importing the very bypass it should replace. Matching is by title containment (dispatch writes the title from the instance, so it is by construction); MORE than one match refuses with a probe rather than guessing, because rooting at the wrong repo silently scores a false zero for the other card. WHAT THIS DOES NOT DO: retire the `agent/solve` inbound bypass. That is still #456 and still the right end state. But this makes the in-loop path actually work — hands in the repo, experience into the genome — which is the precondition for retiring the bypass rather than a consequence of it. CORRECTION TO MY OWN CLAIM this session: I first said "zero production callers of training_trigger/submit — the trigger is dead." Wrong. `training_producer.rs` is the producer and dispatches correctly; `memory/consolidate.rs:196` is a second legitimate caller. The wire was never dead; it was never REACHED from work. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/persona/mod.rs | 1 + .../src/persona/service_loop.rs | 123 ++++++++++++++ .../src/persona/staged_workspace.rs | 156 ++++++++++++++++++ 3 files changed, 280 insertions(+) create mode 100644 core/continuum-core/src/persona/staged_workspace.rs diff --git a/core/continuum-core/src/persona/mod.rs b/core/continuum-core/src/persona/mod.rs index 3b576b564..c38eef6a9 100644 --- a/core/continuum-core/src/persona/mod.rs +++ b/core/continuum-core/src/persona/mod.rs @@ -95,6 +95,7 @@ pub mod scripted_conversation; pub mod seed; pub mod self_task_generator; pub mod service_loop; +pub mod staged_workspace; pub mod service_module; pub mod spawner; pub mod spawner_module; diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 75026df62..b705105eb 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -1254,11 +1254,88 @@ async fn serve_persona_loop_inner( .collect(); if !held.is_empty() { let burst = held_work_burst(&held); + // The producer's CONTEXT half, kept before the burst is + // moved into the driver — one construction, so the + // training example records the prompt she was actually + // handed rather than a re-derived approximation of it. + let work_context = burst.clone(); let work_framing = crate::cognition::workspace::TurnFraming::self_thread( false, ) .on_workspace(); + // HANDS FOLLOW THE CARD (#456). Her held card may be a + // staged benchmark checkout — a real git repo under + // `workspace/swe/`. Without rooting her hands + // there she works the card by writing into her OWN + // workspace, and the grader's `git diff` on the sandbox + // scores a false ZERO: the same defect glass-boxed on + // agent/solve 2026-07-22 (2 real acts, correct file + // written, empty patch). + // + // This is the live sibling of agent/solve's re-root, and + // it is what lets a citizen work a bench card IN HER OWN + // LOOP — which is the only path where the L2 training + // producer fires, so it is also what puts benchmark + // experience into her genome instead of only her memory. + // + // The re-root is PROCESS-GLOBAL (the file engine keys on + // caller identity), so the restore below is mandatory on + // EVERY exit — #312: after a flask solve, Anwen's live + // self was still reading the exam repo hours later. + // Non-bench cards resolve to None and nothing moves. + let card_workspace = + crate::persona::staged_workspace::workspace_for_held_cards( + &ctx.identity.peer_id.as_uuid(), + held.iter().map(|c| c.title.as_str()), + ); + let work_hands = match &card_workspace { + Some(ws) => { + let hands = + crate::cognition::persona_workspace::ActingHands::of( + &cycle, + ); + match crate::cognition::persona_workspace::root_acting_workspace( + &cycle, + &ws.to_string_lossy(), + &[], + false, + ) + .await + { + Ok(()) => { + crate::probe!( + class = "persona.work.hands_rooted", + persona = %ctx.identity.agent_name, + workspace = %ws.display(), + cards = held.len(), + "hands rooted at her claimed card's \ + staged workspace for this work turn" + ); + hands + } + Err(e) => { + // Fail LOUD, work anyway in her own + // workspace: a citizen who cannot reach + // the repo still gets her turn, and the + // empty patch is then explained on the + // probe stream instead of being a mystery + // zero. No silent re-root. + tracing::error!( + persona = %ctx.identity.agent_name, + workspace = %ws.display(), + error = %e, + "could NOT root hands at the claimed \ + card's workspace — she will work in \ + her own dir and any graded diff will \ + read EMPTY" + ); + None + } + } + } + None => None, + }; let work = crate::cognition::act_observe::drive_to_settle( &cycle, burst, @@ -1267,6 +1344,25 @@ async fn serve_persona_loop_inner( work_framing, ) .await; + // Give her back her own hands BEFORE anything else can + // observe them — every exit path from here (Spoke, Passed, + // Acted) must leave her rooted at home (#312). + if let Some(hands) = &work_hands { + if let Err(e) = + crate::cognition::persona_workspace::restore_acting_workspace( + hands, + ) + .await + { + tracing::error!( + persona = %ctx.identity.agent_name, + error = %e, + "work turn could NOT return her hands to her \ + own workspace — she is still rooted at the \ + card's repo and her live turns will act there" + ); + } + } let (work_step, _) = crate::cognition::act_observe::SettleStep::from_settled( work, @@ -1296,6 +1392,33 @@ async fn serve_persona_loop_inner( "work-turn report failed to send" ); } + // L2 producer on the WORK turn (#456). This was + // missing, and it is the highest-value training + // signal the substrate produces: the reply turn + // below already feeds the producer, but the turn + // where she actually WORKS HER CLAIMED CARD did + // not — so every act of real work was invisible + // to the genome while chat was not. + // + // The (context, completion) pair here is honest: + // context = the card burst she was handed, + // completion = the report she wrote after doing + // the work. Same shape as the reply path, same + // best-effort spawn, same quality bar applied + // inside the producer. + // + // Still the LIVE path — an eval fork never + // reaches here (`drive_to_settle` is called from + // the fork, this call site is not), so the + // measurement-contamination guard the reply path + // relies on is unchanged. + crate::persona::training_producer::produce( + ctx.identity.peer_id.as_uuid(), + ctx.identity.agent_name.clone(), + ctx.profile.model_id.clone(), + work_context.clone(), + text.clone(), + ); } crate::cognition::act_observe::SettleStep::Passed => { crate::probe!( diff --git a/core/continuum-core/src/persona/staged_workspace.rs b/core/continuum-core/src/persona/staged_workspace.rs new file mode 100644 index 000000000..a834c0ed6 --- /dev/null +++ b/core/continuum-core/src/persona/staged_workspace.rs @@ -0,0 +1,156 @@ +//! Which on-disk workspace does a citizen's claimed card point at? +//! +//! ONE answer, three callers. A staged benchmark instance is a real git checkout under +//! `/citizens/peers//workspace/swe/`, written by `benchmark/swe-setup` +//! at dispatch. Three places need to resolve it and, before this module, two of them each +//! walked that directory themselves: +//! +//! - `persona/roster` — reports the staged list as the REUSE signal (dispatch found the +//! checkout and skipped cloning) +//! - `modules/work.rs::dispatch_staged_swe_solve` — matches a claimed card to its instance +//! - `persona/service_loop` — roots her HANDS at that instance for a work turn +//! +//! The third is why this module exists. A citizen working her claimed card must ACT IN THE +//! REPO, and the layout knowledge that makes that possible was previously inline in a +//! benchmark dispatcher — so the live path could not reuse it without importing the +//! bypass it is meant to replace. +//! +//! ## The matching rule, and why it refuses rather than guesses +//! +//! A card matches an instance when the card's TITLE CONTAINS the instance directory name +//! (`sympy__sympy-24152`). Dispatch writes the title, so the containment is by +//! construction, not inference. +//! +//! Zero matches → `None`: an ordinary non-bench card, and her hands stay where they are. +//! MORE than one match → `None` AND a probe: two staged instances whose names both appear +//! in one title is a staging defect, and picking either would root her hands in a repo her +//! card is not about — silently scoring a false zero against the other. Refusing is the +//! honest outcome and the probe says which candidates collided. + +use std::path::PathBuf; + +/// Where a citizen's staged benchmark checkouts live. +/// +/// Not configurable and not guessed: this mirrors exactly what `benchmark/swe-setup` +/// writes. A single expression of the layout, so a change to staging cannot leave a reader +/// looking in a directory the writer stopped using. +pub fn staging_root(peer: &uuid::Uuid) -> Option { + let home = crate::commands::benchmark::continuum_home().ok()?; + Some( + home.join("citizens") + .join("peers") + .join(peer.to_string()) + .join("workspace") + .join("swe"), + ) +} + +/// Every benchmark instance actually staged in this citizen's workspace, name-sorted. +/// +/// Counts only directories that carry a `.git` — a real checkout, not an empty shell left +/// by an interrupted clone. Best effort by design: a missing home or unreadable directory +/// yields an empty list, never an error, because every caller is answering "what is here +/// right now" and none of them should fail because staging has not run yet. +pub fn staged_instances(peer: &uuid::Uuid) -> Vec { + let Some(root) = staging_root(peer) else { + return Vec::new(); + }; + let Ok(entries) = std::fs::read_dir(&root) else { + return Vec::new(); + }; + let mut out: Vec = entries + .flatten() + .filter(|e| e.path().join(".git").exists()) + .filter_map(|e| e.file_name().into_string().ok()) + .collect(); + out.sort(); + out +} + +/// The workspace a claimed card points at, or `None` when the card is not a staged +/// benchmark card (the ordinary case). +/// +/// `card_titles` is every title she currently holds — resolution is over the WHOLE held +/// set rather than one card, because the question a work turn asks is "where are my hands +/// supposed to be", and that has one answer for the turn. Two different held cards +/// matching two different staged instances is the same ambiguity as one title matching +/// two, and refuses identically. +pub fn workspace_for_held_cards<'a, I>(peer: &uuid::Uuid, card_titles: I) -> Option +where + I: IntoIterator, +{ + let staged = staged_instances(peer); + if staged.is_empty() { + return None; + } + let titles: Vec<&str> = card_titles.into_iter().collect(); + let mut hits: Vec<&String> = staged + .iter() + .filter(|inst| titles.iter().any(|t| t.contains(inst.as_str()))) + .collect(); + hits.dedup(); + match hits.as_slice() { + [one] => staging_root(peer).map(|root| root.join(one.as_str())), + [] => None, + many => { + // Two staged instances named in her held titles. Rooting at either would put + // her hands in a repo the other card is not about — and a diff taken there + // scores a false zero for the one she was actually working. Refuse loudly. + crate::probe!( + class = "persona.work.staged_ambiguous", + peer = %peer, + matches = many.len(), + candidates = many + .iter() + .map(|s| s.as_str()) + .collect::>() + .join(","), + "held cards name MULTIPLE staged instances — refusing to guess which repo \ + her hands belong in; she works in her own workspace this turn" + ); + None + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // what this catches: the containment rule is what binds a card to a repo, and it is + // by construction (dispatch writes the title from the instance). A title that names + // the instance resolves; an unrelated title does not, so an ordinary chat card never + // silently re-roots a citizen's hands into a benchmark checkout. + #[test] + fn a_title_naming_a_staged_instance_selects_it_and_others_do_not() { + // Pure over the matching rule — the disk half is exercised by the live path, and + // a temp-dir fixture here would be testing `read_dir`, not the decision. + let staged = ["sympy__sympy-24152".to_string(), "flask-4045".to_string()]; + let titles = ["benchmark: sympy__sympy-24152 — fix the printer"]; + let hit: Vec<&String> = staged + .iter() + .filter(|i| titles.iter().any(|t| t.contains(i.as_str()))) + .collect(); + assert_eq!(hit.len(), 1); + assert_eq!(hit[0], "sympy__sympy-24152"); + + let unrelated = ["let's discuss the roadmap"]; + let none: Vec<&String> = staged + .iter() + .filter(|i| unrelated.iter().any(|t| t.contains(i.as_str()))) + .collect(); + assert!( + none.is_empty(), + "an ordinary card must never re-root her hands" + ); + } + + // what this catches: a citizen with NO staged instances resolves to None without + // touching the filesystem layout at all — the ordinary case for every non-benchmark + // citizen, and the one that must stay free. + #[test] + fn a_citizen_with_nothing_staged_has_no_card_workspace() { + let peer = uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, b"nothing-staged-fixture"); + assert!(workspace_for_held_cards(&peer, ["benchmark: anything"]).is_none()); + } +} From 884da43d0ba3c49da1cb05eb1e32591bb601a0fe Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 00:12:30 -0500 Subject: [PATCH 28/80] =?UTF-8?q?docs(planning):=20BENCHMARKS-THAT-LEARN?= =?UTF-8?q?=20=E2=80=94=20the=20plan=20to=20a=20number=20we=20can=20defend?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel: "the learning and score are both the objective. We should outcompete the base model." One falsifiable claim: a citizen on base X with her genome beats bare base X on the same benchmark. Four threads with the non-obvious ordering constraint written down: D (the score ceiling) GATES C (the measurement), because a delta between two zeros is noise. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/planning/BENCHMARKS-THAT-LEARN.md | 221 +++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 docs/planning/BENCHMARKS-THAT-LEARN.md diff --git a/docs/planning/BENCHMARKS-THAT-LEARN.md b/docs/planning/BENCHMARKS-THAT-LEARN.md new file mode 100644 index 000000000..dda691616 --- /dev/null +++ b/docs/planning/BENCHMARKS-THAT-LEARN.md @@ -0,0 +1,221 @@ +# Benchmarks That Learn — the plan to a number we can defend + +**Objective (Joel, 2026-08-18):** *"The learning and score are both the objective. We +should outcompete the base model."* + +That is one falsifiable claim, not two goals: + +> **A citizen on base X, with her genome, beats bare base X on the same benchmark.** + +Not "beat a frontier model." Beat *the model we are running on*. If that delta is not +positive, the cognition + genome stack is not earning its keep — and that is a real +answer, not a failure to measure. Everything below exists to make that sentence +measurable and then to move it. + +--- + +## The four threads, and why they are ordered this way + +| | thread | what it delivers | gates | +|---|---|---|---| +| **A** | bench work reaches the genome | the LEARNING half | A2 gates A3 | +| **B** | base-model control | attributable scores + a training target | B1 gates B2 | +| **D** | the score ceiling | turns that can actually finish | **D gates C** | +| **C** | the measurement | the claim itself | needs A, B, D | + +The non-obvious ordering constraint: **D gates C.** If turns still burn their whole +output budget inside `` and emit no tool call, both arms of the comparison score +~0 and the delta is noise. You cannot measure a difference between two zeros. Do not run +C until D's gate is green. + +--- + +## Thread A — bench work reaches the genome + +**Why it was impossible.** `training_producer::produce` — the L2 step that turns a turn +into a `(context, completion)` training example and submits it to the forge — had exactly +ONE call site tree-wide: the message-**reply** path (`service_loop.rs:1474`). Its own +comment states the exclusion: *"It lives ONLY on this live `Spoke` path, which eval forks +(`drive_to_settle`) never run, so the training set can never be contaminated by a +measurement simulation."* Correct for eval (#59). Consequences: + +- `agent/solve` IS `drive_to_settle` → a dispatched bench card produces **zero** training + examples, regardless of `learn: LearnFromThisWork`. The gate is PATH-based; the policy + is never consulted; **the policy loses to the path.** +- The **work turn** — the separate drive where she works her *claimed card* (#154, + "working is not speaking") — also never fed the producer. **Chat produced training + data; real work did not.** + +### A1 — work turn roots hands + feeds the producer ✅ SHIPPED (`18ecef781`) + +- hands root at the held card's staged workspace, restored on every exit path (#312) +- the producer fires on the work turn: context = card burst, completion = her report +- `persona/staged_workspace.rs` — ONE answer to "which workspace does this card point + at", replacing the inline walk in `persona/roster` and `dispatch_staged_swe_solve` + +**Unproven.** Compile + unit green only. + +### A2 — LIVE-PROVE A1 ← *next action* + +Reboot → citizens resident → one staged SWE card in a citizen's hands → observe, in order: + +1. `persona.work.hands_rooted` probe naming the instance workspace +2. acts executing **inside** that repo (a `git diff` in the sandbox is non-empty) +3. a training submit reaching `genome/training-trigger/submit` +4. her hands restored afterward (`code/read` on her own workspace, not the exam repo) + +**Gate:** all four. (3) is the one that has never happened for bench work. +**Falsifier:** if (3) does not fire, the producer's quality bar is rejecting work reports +— read `score_interaction_quality` against an actual report before assuming a wiring bug. + +### A3 — retire the `agent/solve` inbound bypass + +`benchmark/dispatch` stops calling `dispatch_staged_swe_solve` directly; the kickoff/claim +drives her turn. **Keep** `agent/solve` as the OUTBOUND primitive it was built to be +(external harnesses drive our agent); only the inbound self-call goes. + +Its own comment admits the bypass was deliberate: it fires solve directly *"rather than +depend on her re-deriving a `work/claim` from a chat kickoff (the fragile hop that stalls +every run)."* That hop is what #455 (residency) and #452 (boot owns the tree) fixed — so +the reason for the bypass is now largely gone. **Do not do this before A2**; retiring it +onto a path that cannot reach the repo would be strictly worse than today. + +Folds in **#453**: once she works in her own loop, the model is whatever she is served on, +and `base_model_id` as a required param stops existing. + +### A4 — the round ends (#371) + +staging → ready → working → grading → done, each transition emitted by the component that +knows. Without it a round is not repeatable, and an unrepeatable number is an anecdote. + +--- + +## Thread B — base-model control + +### B1 — find the chooser ← *blocking, bounded read* + +`persona/host.rs:367` → `self.spawner.plan()` → `plan_rows.first()` → `desired.model_id`. +That is what picks the served model. **Not** `allocator::resolve_model_for_persona`, which +is why the catalog says `qwen3.5-4b-code-forged` while serving runs Devstral. + +Read what backs `spawner.plan()` (#430 made the roster recipe data — likely there). +**Until this is known, pegging the allocator pegs a thing that is not deciding** — a green +test on a dead path, the same shape as three defects already found this session. + +### B2 — cut serving + training onto `BaseModelPolicy` (`07c67e434`, `836677021`) + +The type is landed and deliberately unwired. It collapses **4 → 1**: + +| was | now | +|---|---| +| `override_model` (runtime assignment) | `Pegged{reason: Measurement{run_id}}` | +| `model_preferences` (tiered ladder) | `Adaptive{ladder, floor, rungs}` | +| `model_id` (labelled *"Legacy"*) | `Pegged{reason: Operator}` | +| `default_local_model` | **deleted** — that arm WAS the #438 downgrade | + +Measured on the real catalog: **behaviour-neutral today** (every local entry has one rung +at `min_vram_gb = 0`; the `default_local_model` arm is only reachable by non-local entries +that never enter the resolver). The new refusals first bite when the Qwen ladder lands — +which is the point. + +**Constraint (Joel):** the policy is an INPUT to the governor, never a replacement. The +existing code already draws this line — *"the allocator's budget gate — the override only +changes WHICH model, never whether it fits the host."* `ResourceGovernor` keeps VRAM, +pressure, ports, grid placement. Untouched. + +### B3 — author the Qwen ladder as data + +``` +Qwen3.8-27B ← top rung + ↓ smaller Qwen forms +floor: smallest form we stand behind +``` + +The 64GB M5 lands on the top rung **automatically** — no peg, no special case. A 16GB box +steps down the same ladder. That is the dynamic system; the floor only stops it sliding +past the bottom into a 0.5B (#438). + +**Nuance:** a same-family ladder does **not** make adapters portable. Each rung needs its +own forge run. What the family buys is consistent tokenizer/chat-template across rungs, so +one corpus format feeds every forge target. + +### B4 — forge fan-out + +One corpus → `{adapter@27b, adapter@7b, adapter@1b}`, **targets read from her ladder**, not +from a typed flag. `RungPolicy::RequireGenome` (the default) then makes an unforged ladder +VISIBLE instead of a silent capability cliff: declare four rungs, forge one, and she +resolves to the one that exists — and the refusal names the missing forge targets. + +This is also what makes the peg load-bearing rather than bureaucratic. Continuous learning +never stops, so `PegReason::Training{job_id}` is the peg held **most of the time**: if her +base floats between the corpus accruing and the forge finishing, the adapter lands for a +base she has already left. Corpus survives; the training compute does not. + +--- + +## Thread D — the score ceiling (**gates C**) + +Two known caps, independent of everything above. + +### D1 — turns that never emit a tool call + +`completion_budget_for(window) = window/4`, so a 16k window caps generation at 4,096. A +reasoning model exhausts that inside `` and never reaches the tool call. Measured: +7/20 captures at `finish_reason: length`, `output_tokens: 4096` exactly, empty text. + +**Two fixes that are wrong and must not be retried:** capping `` (shrinks the +model; Joel rejected it), and raising the fraction alone (breaks +`prompt_plus_completion_cap_never_exceeds_the_served_window` → a 500 on every turn). + +The real move is `reserve = min(desired_share, window − mandatory_floor)` — the reserve +yields to the floor. But `/4` is load-bearing in **six** places, so it is four ordered +steps (collapse the duplication first, at the unchanged fraction; move sub-floor test +windows to a real one as a stated PREMISE change; make the reserve yield; then raise the +share). Detail in the prior plan; do not one-line it. + +### D2 — grading + +May be largely resolved — #383 reports django grading with 7/8 env classes gold-gate green. +**Do not quote the old "114/300 ungradeable" as current.** Re-measure: a gold patch must +pass in every env class, or a 0 means "the env lied", not "the model failed". + +**Gate for D:** `finish_reason: length` empties fall measurably from a timestamped +baseline, AND a gold patch passes in every env class in play. + +--- + +## Thread C — the measurement + +Only run after D's gate. Then: + +| arm | policy | label | +|---|---|---| +| control | `Pegged{base, Measurement{run}}` + `RungPolicy::AllowBare` | `genome_backed: false` | +| treatment | same peg + her forged adapter | `genome_backed: true` | + +1. **C1** — control arm on the pegged base. Score. +2. **C2** — the loop runs; corpus accrues from real work (Thread A). +3. **C3** — forge onto **that same base** (Thread B4). +4. **C4** — treatment arm, same peg. Score. +5. **C5** — delta. + +`ResolvedBase.genome_backed` labels every score so a lift comparison can **refuse** to +compare across that line. The peg stops the base drifting between arms. Without both, the +delta measures a base swap and calls it learning. + +**What would falsify the whole thesis:** delta ≤ 0 with D's gate green and both arms on the +same base. That is a real result and it should be published as one. + +--- + +## Standing rules for this work + +- **`export CARGO_TARGET_DIR="$HOME/.continuum/cache/cargo-target"`** before any cargo. +- **Commit, then deploy.** A fix you cannot prove reached the running binary is a fix you + have not made — verify the SHA against git HEAD. +- **Measure deltas from a timestamped baseline**, never cumulative probe totals (they + survive reboots; this produced two vacuous "proofs" already). +- **An absence is an unfinished measurement.** Four times this session a "zero callers / + never fires" reading was wrong on the first look. Grep the verb STRING and the module + path before concluding a wire is dead. +- **Never `--no-verify`.** Canary is the branch; main merge needs Joel. From f1417e1dd45ae0e964583037f55accedb0ef77dd Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 00:45:37 -0500 Subject: [PATCH 29/80] =?UTF-8?q?feat(benchmark):=20the=20ROUND=20owns=20w?= =?UTF-8?q?ho=20drives=20its=20cards=20=E2=80=94=20WorkDriver{DetachedSolv?= =?UTF-8?q?e|Citizen}=20(#456,=20A2=20unblock)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A2 (live-prove the work-turn fix, 18ecef781) failed 3 of 4 gates, and the cause was structural rather than a defect in the fix. Measured on a real dispatch: gate 1 persona.work.hands_rooted — never fired (546 probe rows, class absent) gate 2 acts inside the repo — YES, but by the BYPASS: workspace.rooted names the "forked persona's file engine", i.e. agent/solve gate 3 training submit — no jobs, no genome rows gate 4 hands restored — n/a, the work turn never ran work/claim fires dispatch_staged_swe_solve synchronously in the claim handler (modules/work.rs), so the detached fork takes the card and works it in a forked persona. Her own service loop never sees a held-card work turn on it. A1's code is UNREACHABLE by construction on the default path — the fork always wins the claim. So A2 and A3 are one step, and the resolution is a switch, not a deletion (retiring the bypass onto an unproven path would be strictly worse than today): WorkDriver::DetachedSolve default, today's behaviour, the path that produced #366 WorkDriver::Citizen nothing detached fires; she works it in her own loop The driver lives on the ROUND because that is the only thing that knows: the decision is made once at dispatch and read later at claim time, in a different verb, on a different task, with nothing threaded between them. Round lifecycle becomes open → add_card → seal, replacing register_round-at-the-end. That ordering is load-bearing, not tidiness: dispatch sends kickoffs INSIDE its card loop, so a citizen can claim card 1 while card 2 is still being posted. Registering after the loop left that window answering with the default — a Citizen round would have fired the solver on its own first card and defeated itself with no error anywhere. Pinned by the_driver_is_readable_the_instant_a_card_can_be_claimed. A citizen-driven SWE card also takes the gym pre-claim cut, and for the reason that one was written: nothing detached will fire, so the only thing between the card and her work turn is the kickoff→claim hop that stalls rounds. The pre-claim goes through airc directly, not the work/claim verb, so it cannot re-enter the dispatcher. Defaults are conservative in both directions: a card in no live round (human-claimed, undirected, or claimed after its round ended) answers DetachedSolve. 8 tests green. Behaviour unchanged unless --drive=citizen is passed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_round.rs | 178 ++++++++++++++++-- core/continuum-core/src/commands/benchmark.rs | 53 +++++- core/continuum-core/src/modules/work.rs | 35 +++- protocol/typescript/benchmark/WorkDriver.ts | 21 +++ 4 files changed, 257 insertions(+), 30 deletions(-) create mode 100644 protocol/typescript/benchmark/WorkDriver.ts diff --git a/core/continuum-core/src/cognition/bench_round.rs b/core/continuum-core/src/cognition/bench_round.rs index 71efdcf5d..2a8b014d8 100644 --- a/core/continuum-core/src/cognition/bench_round.rs +++ b/core/continuum-core/src/cognition/bench_round.rs @@ -18,6 +18,13 @@ //! - `bench.round.staged` — at dispatch: round id, benchmark, card count. //! - `bench.round.card_settled` — a card in the round reached a terminal state. //! - `bench.round.done` — the END, exactly once: every card settled. +//! +//! ## The round also owns WHO DRIVES its work ([`WorkDriver`]) +//! +//! A benchmark card can be worked two ways, and the difference decides whether the round +//! teaches anybody anything. The round is the only thing that knows which, because the +//! decision is made once at dispatch and read later at claim time — in a different verb, +//! on a different task, with nothing threaded between them. use std::collections::HashMap; use std::sync::{LazyLock, Mutex}; @@ -25,6 +32,42 @@ use std::sync::{LazyLock, Mutex}; use serde_json::Value; use uuid::Uuid; +/// Who actually does the work on this round's cards. +/// +/// - [`DetachedSolve`](WorkDriver::DetachedSolve) — a `work/claim` (or a directed +/// dispatch) fires `agent/solve` for a FORKED copy of the citizen. Proven: it reaches +/// the repo, it grades, it produced our one SWE pass (#366). It also produces no room +/// turn, so `training_producer::produce` never runs and the round teaches nobody +/// (#456: the L2 producer is PATH-gated, and this is the path it excludes). +/// - [`Citizen`](WorkDriver::Citizen) — nothing detached fires. She claims the card in +/// her own service loop and works it on the held-work turn, which roots her hands at +/// the staged checkout and feeds the training producer. This is the path the learning +/// half of the objective requires — and the one that has never once been observed +/// end to end, because on the default path the detached solve always wins the claim. +/// +/// `DetachedSolve` is the default and today's behaviour: an operator opts INTO the +/// citizen path per round. Both are real drivers, not a flag and a fallback — which is +/// why the choice is named on the round rather than hidden behind a `skip_solve` bool. +#[derive( + Debug, + Clone, + Copy, + PartialEq, + Eq, + Default, + serde::Serialize, + serde::Deserialize, + schemars::JsonSchema, + ts_rs::TS, +)] +#[serde(rename_all = "snake_case")] +#[ts(export, export_to = "../../../protocol/typescript/benchmark/WorkDriver.ts")] +pub enum WorkDriver { + #[default] + DetachedSolve, + Citizen, +} + /// Where a round is in its life. `Working` from the moment dispatch returns (cards are /// posted and kickoffs sent); `Done` when every card in the round's set has reached a /// terminal card state. Two stages only — the smallest true lifecycle; claim/review @@ -72,15 +115,18 @@ pub struct BenchRound { /// Card uuid → the terminal state it settled with (`None` = still working). cards: HashMap>, stage: RoundStage, + /// Who works this round's cards — read at CLAIM time, decided at dispatch. + driver: WorkDriver, } impl BenchRound { - pub fn new(round_id: Uuid, benchmark: &str, card_ids: &[Uuid]) -> Self { + pub fn new(round_id: Uuid, benchmark: &str, card_ids: &[Uuid], driver: WorkDriver) -> Self { Self { round_id, benchmark: benchmark.to_string(), cards: card_ids.iter().map(|c| (*c, None)).collect(), stage: RoundStage::Working, + driver, } } @@ -129,20 +175,58 @@ impl BenchRound { static ROUNDS: LazyLock>> = LazyLock::new(|| Mutex::new(HashMap::new())); -/// Register a freshly dispatched round and announce it (`bench.round.staged`). Called by -/// `benchmark/dispatch` after its card loop, with the run room's uuid as the round id and -/// the FULL card uuids it posted. A dispatch that posted zero cards (everything skipped / -/// already on board) stages and immediately ends — an honest empty round, never a map -/// entry that no event can ever settle. -pub fn register_round(round_id: Uuid, benchmark: &str, card_ids: &[Uuid]) { +/// Open a round BEFORE its first card is posted, so the driver is readable from the +/// instant a card can be claimed. +/// +/// The ordering is load-bearing, not tidiness. `benchmark/dispatch` sends a kickoff +/// inside its card loop, so a citizen can claim card 1 while card 2 is still being +/// posted. If the round were only registered after the loop (as it was), that claim +/// would find no round, [`driver_for_card`] would answer with the default, and a +/// `Citizen`-driven round would silently fire the detached solver for its first card — +/// defeating exactly the thing the round was configured to do, with no error anywhere. +/// +/// Idempotent by round id: re-opening a live round leaves it untouched. +pub fn open_round(round_id: Uuid, benchmark: &str, driver: WorkDriver) { + ROUNDS + .lock() + .unwrap_or_else(|p| p.into_inner()) + .entry(round_id) + .or_insert_with(|| BenchRound::new(round_id, benchmark, &[], driver)); +} + +/// Add a freshly posted card to an open round. Unknown round = no-op (dispatch always +/// opens first; a card that arrives without one is not part of a tracked round). +pub fn add_card(round_id: Uuid, card_id: Uuid) { + if let Some(r) = ROUNDS + .lock() + .unwrap_or_else(|p| p.into_inner()) + .get_mut(&round_id) + { + r.cards.entry(card_id).or_insert(None); + } +} + +/// Close the dispatch phase and announce the round (`bench.round.staged`). Called by +/// `benchmark/dispatch` after its card loop. A round that posted zero cards (everything +/// skipped / already on board) stages and immediately ends — an honest empty round, +/// never a map entry that no event can ever settle. +pub fn seal_round(round_id: Uuid) { + let mut rounds = ROUNDS.lock().unwrap_or_else(|p| p.into_inner()); + let Some(round) = rounds.get(&round_id) else { + return; + }; + let (benchmark, dispatched, driver) = + (round.benchmark.clone(), round.dispatched(), round.driver); crate::probe!( class = "bench.round.staged", round_id = %round_id, benchmark = %benchmark, - cards = card_ids.len(), + cards = dispatched, + driver = ?driver, "benchmark round staged — cards posted, kickoffs sent, round is Working" ); - if card_ids.is_empty() { + if dispatched == 0 { + rounds.remove(&round_id); crate::probe!( class = "bench.round.done", round_id = %round_id, @@ -151,12 +235,23 @@ pub fn register_round(round_id: Uuid, benchmark: &str, card_ids: &[Uuid]) { settled = 0usize, "benchmark round END — nothing was dispatched" ); - return; } +} + +/// Who drives the work for this card — the question `work/claim` asks before deciding +/// whether to fire a detached solve. +/// +/// A card belonging to no live round answers [`WorkDriver::DetachedSolve`]: that covers +/// a human-claimed card, an undirected board card, and a leftover claimed after its +/// round ended. All three are the proven path, so the default is the conservative one. +pub fn driver_for_card(card_id: Uuid) -> WorkDriver { ROUNDS .lock() .unwrap_or_else(|p| p.into_inner()) - .insert(round_id, BenchRound::new(round_id, benchmark, card_ids)); + .values() + .find(|r| r.cards.contains_key(&card_id)) + .map(|r| r.driver) + .unwrap_or_default() } /// React to one `work.card.state_changed` payload (`{card_id, state, room_id}` — the @@ -231,13 +326,68 @@ mod tests { (0..n).map(|_| Uuid::new_v4()).collect() } + /// Open a round and add its cards — the dispatch sequence, for tests that only care + /// about the finished set. + fn register_round(round_id: Uuid, benchmark: &str, card_ids: &[Uuid]) { + open_round(round_id, benchmark, WorkDriver::default()); + for c in card_ids { + add_card(round_id, *c); + } + seal_round(round_id); + } + + // what this catches: the window between "card posted" and "round registered". Dispatch + // sends a kickoff inside its card loop, so a citizen can claim card 1 while card 2 is + // still being posted — and `work/claim` asks `driver_for_card` whether to fire the + // detached solver. If the driver were only readable after the whole loop, a + // Citizen-driven round would fire the solver on its first card and defeat itself + // silently. The driver must be right from the first `add_card`, before `seal_round`. + #[test] + fn the_driver_is_readable_the_instant_a_card_can_be_claimed() { + let round_id = Uuid::new_v4(); + let first = Uuid::new_v4(); + open_round(round_id, "swe-bench-lite", WorkDriver::Citizen); + add_card(round_id, first); + // Mid-loop: the round is NOT sealed, a second card is not posted yet. + assert_eq!( + driver_for_card(first), + WorkDriver::Citizen, + "a claim landing mid-dispatch must see the round's real driver" + ); + seal_round(round_id); + assert_eq!(driver_for_card(first), WorkDriver::Citizen); + ROUNDS.lock().unwrap().remove(&round_id); + } + + // what this catches: the default biting the wrong way. A card belonging to no live + // round — human-claimed, undirected, or a leftover claimed after its round ended — + // must answer DetachedSolve, the proven path. Defaulting to Citizen would silently + // stop firing solves for every ordinary claim on the box. + #[test] + fn a_card_in_no_round_drives_by_detached_solve() { + assert_eq!(driver_for_card(Uuid::new_v4()), WorkDriver::DetachedSolve); + } + + // what this catches: a round that dispatched nothing must END, not sit in the map + // forever waiting for an event that can never arrive (no cards = no card events). + #[test] + fn an_empty_round_seals_straight_to_done_and_leaves_no_entry() { + let round_id = Uuid::new_v4(); + open_round(round_id, "swe-bench-lite", WorkDriver::DetachedSolve); + seal_round(round_id); + assert!( + ROUNDS.lock().unwrap().get(&round_id).is_none(), + "an empty round must not remain tracked" + ); + } + // what this catches: the END transition firing more than once. All-settled must yield // RoundDone exactly once — a re-delivered terminal event after Done must be a no-op, // or the room would get N "round over" announcements for one round. #[test] fn all_settled_transitions_to_done_exactly_once() { let ids = cards(3); - let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids); + let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids, WorkDriver::default()); assert_eq!(r.stage(), RoundStage::Working); assert_eq!(r.settle_card(ids[0], "closed"), SettleOutcome::Settled { remaining: 2 }); assert_eq!(r.settle_card(ids[1], "merged"), SettleOutcome::Settled { remaining: 1 }); @@ -255,7 +405,7 @@ mod tests { #[test] fn a_card_settling_twice_does_not_double_count() { let ids = cards(2); - let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids); + let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids, WorkDriver::default()); assert_eq!(r.settle_card(ids[0], "closed"), SettleOutcome::Settled { remaining: 1 }); assert_eq!(r.settle_card(ids[0], "merged"), SettleOutcome::AlreadySettled); assert_eq!(r.remaining(), 1, "the duplicate must not consume the other card's slot"); @@ -268,7 +418,7 @@ mod tests { #[test] fn a_card_outside_the_set_is_ignored() { let ids = cards(1); - let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids); + let mut r = BenchRound::new(Uuid::new_v4(), "humaneval-rs", &ids, WorkDriver::default()); assert_eq!(r.settle_card(Uuid::new_v4(), "closed"), SettleOutcome::NotOurs); assert_eq!(r.remaining(), 1); assert_eq!(r.stage(), RoundStage::Working); diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index d0057659a..7efa87e0e 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -609,6 +609,21 @@ pub struct BenchmarkDispatchParams { /// without dispatching anything new. #[serde(default)] pub prune: Option, + /// Who works this round's cards: `detached_solve` (default) or `citizen`. + /// + /// - `detached_solve` — a forked copy of the citizen solves each card through + /// `agent/solve`, with an exclusive warm slot. Proven; it produced our one SWE + /// pass. It also produces no room turn, so the round teaches nobody (#456). + /// - `citizen` — nothing detached fires. The kickoff drives her to claim, and she + /// works the card on her own held-work turn: hands rooted at the staged checkout, + /// acts radiating into the run room, and the turn feeding the training producer. + /// + /// The score and the learning are both the objective, and only `citizen` can + /// deliver the second one — but it depends on the kickoff→claim hop that used to + /// stall rounds, so it is opt-in until that hop is proven under residency. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub drive: Option, } #[derive(Debug, Clone, Serialize, TS)] @@ -1346,6 +1361,13 @@ impl ActionCommand for BenchmarkDispatch { Some(s) => s.lanes.max(1), None => 0, }; + // OPEN the round BEFORE the first card exists. Kickoffs go out inside the loop, so + // a citizen can claim card 1 while card 2 is still being posted — and `work/claim` + // asks the round who drives. Registering after the loop (as this did) left that + // window answering with the default, which would silently fire the detached solver + // on the first card of a citizen-driven round. + let driver = p.drive.unwrap_or_default(); + crate::cognition::bench_round::open_round(room.room_id.as_uuid(), spec.name, driver); for pc in prepared.into_iter().take(take) { // A gym setup_shell task needs its workspace re-broken before work // starts — harness orchestration a claimed card can't provide yet. @@ -1430,8 +1452,13 @@ impl ActionCommand for BenchmarkDispatch { let full = card_id.as_uuid().simple().to_string(); let short = full[..8].to_string(); + // The card joins the round the moment it exists — BEFORE the pre-claim and + // the kickoff below, either of which can put it into someone's hands. From + // here `work/claim` can read who drives it (see `open_round`). + crate::cognition::bench_round::add_card(room.room_id.as_uuid(), card_id.as_uuid()); + // Directed gym card: CLAIM IT FOR HER at dispatch, under her own airc - // identity. The SWE arm below already fires her scored solve directly + // identity. The detached-solve SWE arm below fires her scored solve directly // (dispatch_staged_swe_solve — "we don't wait on her to re-derive a // work/claim from the kickoff"); gym cards never got the same cut, so // every round spent its first multi-minute turn per card on claim @@ -1440,8 +1467,19 @@ impl ActionCommand for BenchmarkDispatch { // #425-compatible: the claim is administrative — the WORK stays hers, // in-room, through her own cognition. Best-effort: a failed pre-claim // is REPORTED and the card stays claimable by hand. + // + // A CITIZEN-driven SWE card takes the same cut, and for the same reason it + // was written: nothing detached will fire for it, so the ONLY thing standing + // between the card and her work turn is the kickoff→claim hop that stalls + // rounds. Pre-claiming removes that hop without moving the work — she still + // does it herself, in her own loop, on the held-work turn. (This goes through + // airc directly, not the `work/claim` verb, so it cannot re-enter the + // detached-solve dispatcher from here.) let mut pre_claimed = false; - if let CardWork::Gym { .. } = &pc.work { + let pre_claim_this = matches!(pc.work, CardWork::Gym { .. }) + || (matches!(pc.work, CardWork::Swe { .. }) + && driver == crate::cognition::bench_round::WorkDriver::Citizen); + if pre_claim_this { match self.registry.get(*who_peer) { Some(rt) => { match rt @@ -1548,7 +1586,10 @@ impl ActionCommand for BenchmarkDispatch { // zero solves). Her WHOLE cognition solves it with an exclusive warm slot; the // work/claim path stays the trigger for undirected / human-claimed cards. Only a // STAGED SWE card has a solve to fire here (a gym card self-grades differently). - if staged_ok && solves_fired < solve_cap { + if staged_ok + && solves_fired < solve_cap + && driver == crate::cognition::bench_round::WorkDriver::DetachedSolve + { if let CardWork::Swe { .. } = &pc.work { // The run room goes WITH the solve: her acts radiate receipts // into the room this dispatch just spawned, so the round's work @@ -1575,11 +1616,7 @@ impl ActionCommand for BenchmarkDispatch { // `work.card.state_changed` subscriber settles cards as they reach terminal // states and announces the END — instead of the round's fate being probe // archaeology ("random and directed by agent, not an ecosystem", Joel 8/16). - crate::cognition::bench_round::register_round( - room.room_id.as_uuid(), - spec.name, - &card_uuids, - ); + crate::cognition::bench_round::seal_round(room.room_id.as_uuid()); // PRUNE (opt-in): converge the board to one live card per task for THIS // benchmark. Scoped to the keys this dispatch planned, so pruning one diff --git a/core/continuum-core/src/modules/work.rs b/core/continuum-core/src/modules/work.rs index b7dcabc7b..3aff7d443 100644 --- a/core/continuum-core/src/modules/work.rs +++ b/core/continuum-core/src/modules/work.rs @@ -626,14 +626,33 @@ impl ActionCommand for WorkClaim { // nobody in the loop. Best-effort: a dispatch failure never voids the // claim — the claim is hers either way, and the probe says what happened. if let Some(caller) = ctx.caller.as_ref() { - // ROOMLESS, and named as such rather than papered over: the claim verb - // carries no activity — a citizen can claim from anywhere, and the card - // does not remember which room staged it. So a claim-fired solve still - // works invisibly. That is #425's whole subject (a bench claim must lead - // to IN-ROOM work, not a detached nil-room solve); the dispatch path - // below already has a room and passes it, so the two paths now differ - // exactly at the gap #425 exists to close. - dispatch_staged_swe_solve(ctx, &airc, caller.peer_id.as_uuid(), card_id, None).await; + // WHO DRIVES THIS CARD is the round's decision, not the claim verb's. On a + // `Citizen`-driven round nothing detached fires: the claim stands, and she + // works the card on her own held-work turn — which roots her hands at the + // staged checkout and feeds the training producer. Firing the solver here + // would take the card out from under that turn (the detached fork always + // wins the race), so the citizen path is unreachable unless this yields. + match crate::cognition::bench_round::driver_for_card(card_id.as_uuid()) { + crate::cognition::bench_round::WorkDriver::DetachedSolve => { + // ROOMLESS, and named as such rather than papered over: the claim + // verb carries no activity — a citizen can claim from anywhere, and + // the card does not remember which room staged it. So a claim-fired + // solve still works invisibly. That is #425's whole subject (a bench + // claim must lead to IN-ROOM work, not a detached nil-room solve); + // the dispatch path already has a room and passes it, so the two + // paths differ exactly at the gap #425 exists to close. + dispatch_staged_swe_solve(ctx, &airc, caller.peer_id.as_uuid(), card_id, None) + .await; + } + crate::cognition::bench_round::WorkDriver::Citizen => { + crate::probe!( + class = "benchmark.dispatch", + card_id = %card_id.as_uuid(), + claimer = %caller.peer_id.as_uuid(), + "citizen-driven round — NO detached solve; her own work turn drives this card" + ); + } + } } Ok(WorkClaimResult { card_id: p.card_id, diff --git a/protocol/typescript/benchmark/WorkDriver.ts b/protocol/typescript/benchmark/WorkDriver.ts new file mode 100644 index 000000000..9007735e5 --- /dev/null +++ b/protocol/typescript/benchmark/WorkDriver.ts @@ -0,0 +1,21 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * Who actually does the work on this round's cards. + * + * - [`DetachedSolve`](WorkDriver::DetachedSolve) — a `work/claim` (or a directed + * dispatch) fires `agent/solve` for a FORKED copy of the citizen. Proven: it reaches + * the repo, it grades, it produced our one SWE pass (#366). It also produces no room + * turn, so `training_producer::produce` never runs and the round teaches nobody + * (#456: the L2 producer is PATH-gated, and this is the path it excludes). + * - [`Citizen`](WorkDriver::Citizen) — nothing detached fires. She claims the card in + * her own service loop and works it on the held-work turn, which roots her hands at + * the staged checkout and feeds the training producer. This is the path the learning + * half of the objective requires — and the one that has never once been observed + * end to end, because on the default path the detached solve always wins the claim. + * + * `DetachedSolve` is the default and today's behaviour: an operator opts INTO the + * citizen path per round. Both are real drivers, not a flag and a fallback — which is + * why the choice is named on the round rather than hidden behind a `skip_solve` bool. + */ +export type WorkDriver = "detached_solve" | "citizen"; From b24ea585f4c699ecfbe8d80077ee1e4d573488d1 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 00:52:35 -0500 Subject: [PATCH 30/80] refactor(persona): staged_workspace becomes the ONE staging resolver it already claimed to be MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Its module doc has said since 18ecef781 that three callers share one answer to "which on-disk workspace does this card point at", and named `dispatch_staged_swe_solve` as one of them. That was aspirational: the dispatcher still walked `citizens/peers//workspace/swe` itself. A doc making a claim the code does not keep is worse than no doc — the next reader trusts it. They had already drifted in KIND, not just in duplication: the inline walk accepted any directory, the module requires a `.git`. So an interrupted clone read as a staged instance on the claim path and did not on the work-turn path — the same card resolving differently depending on who asked. `Option` was why the copy survived: it erased both distinctions the dispatcher needed (no-staging vs ambiguous, and the instance NAME its era venv is keyed by). So the resolver now returns `CardWorkspace{One{instance,path}|None|Ambiguous{candidates}}` and `workspace_for_held_cards` becomes the projection the WORK TURN wants — "root here or leave her hands alone" — keeping the ambiguity probe at the caller that would otherwise score a false zero. The dispatcher matches the enum and reports ambiguity in its own vocabulary, as it did before. Also splits the matching rule into a pure `select()`. The existing test restated the containment check in its own body instead of calling it — it could not fail if the real rule changed. It now exercises the production function, and the ambiguity arm (which both callers depend on staying distinguishable from "nothing staged") gets its first test. 4 tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/modules/work.rs | 42 +++-- .../src/persona/staged_workspace.rs | 154 +++++++++++++----- 2 files changed, 132 insertions(+), 64 deletions(-) diff --git a/core/continuum-core/src/modules/work.rs b/core/continuum-core/src/modules/work.rs index 3aff7d443..5e00cb79c 100644 --- a/core/continuum-core/src/modules/work.rs +++ b/core/continuum-core/src/modules/work.rs @@ -701,34 +701,31 @@ pub(crate) async fn dispatch_staged_swe_solve( let Some(card) = board.cards.iter().find(|c| c.card_id == card_id) else { return; }; - // Her staged SWE checkouts — the exact layout swe-setup writes. - let Ok(home) = crate::commands::benchmark::continuum_home() else { - return; - }; - let swe_root = home - .join("citizens/peers") - .join(claimer.to_string()) - .join("workspace/swe"); - let Ok(entries) = std::fs::read_dir(&swe_root) else { - return; // no staged work for her — an ordinary (non-SWE) claim. - }; - let staged: Vec = entries - .flatten() - .filter(|e| e.path().is_dir()) - .filter_map(|e| e.file_name().to_str().map(str::to_string)) - .filter(|name| card.title.contains(name.as_str())) - .collect(); - let [instance] = staged.as_slice() else { - if staged.len() > 1 { + // Her staged SWE checkouts. ONE expression of that layout lives in + // `persona::staged_workspace` — the work turn roots her hands with the same resolver, + // so a change to staging can never leave the two disagreeing about which repo a card + // is about (they did disagree in kind already: this walk accepted any directory, that + // one requires a `.git`, so an interrupted clone read as a staged instance here). + let (instance, workspace) = match crate::persona::staged_workspace::resolve_for_titles( + &claimer, + [card.title.as_str()], + ) { + crate::persona::staged_workspace::CardWorkspace::One { instance, path } => { + (instance, path.to_string_lossy().to_string()) + } + // No staged checkout for her matching this card — an ordinary (non-SWE) claim. + crate::persona::staged_workspace::CardWorkspace::None => return, + crate::persona::staged_workspace::CardWorkspace::Ambiguous { candidates } => { crate::probe!( class = "benchmark.dispatch", card_id = %card_id.as_uuid(), claimer = %claimer, - matches = staged.len(), + matches = candidates.len(), + candidates = candidates.join(","), "claim matched MULTIPLE staged instances — refusing to guess, no dispatch" ); + return; } - return; }; // WAIT for the boot-gate, don't guard against it. A claim can land while the serving lane // is still proving it can decode (the ~10-15s window after core-ready); parking here until @@ -750,7 +747,6 @@ pub(crate) async fn dispatch_staged_swe_solve( ); return; } - let workspace = swe_root.join(instance).to_string_lossy().to_string(); // Her HANDS must resolve `python`/`pytest`/`pip` to THIS instance's venv, not the system // interpreter. Without this, `code/shell pytest` hits homebrew python3.14 (no pytest, no // repo), she loops `pip install pytest` into the wrong interpreter, and burns every action @@ -759,7 +755,7 @@ pub(crate) async fn dispatch_staged_swe_solve( // solve.rs already `.exists()`-filters it, so prepending a not-yet-built bin is harmless. let venv_bin = crate::cognition::swe_bench::swe_cache_dir() .join("envs") - .join(instance) + .join(&instance) .join("bin") .to_string_lossy() .to_string(); diff --git a/core/continuum-core/src/persona/staged_workspace.rs b/core/continuum-core/src/persona/staged_workspace.rs index a834c0ed6..16f70b88f 100644 --- a/core/continuum-core/src/persona/staged_workspace.rs +++ b/core/continuum-core/src/persona/staged_workspace.rs @@ -67,44 +67,96 @@ pub fn staged_instances(peer: &uuid::Uuid) -> Vec { out } -/// The workspace a claimed card points at, or `None` when the card is not a staged -/// benchmark card (the ordinary case). +/// Which staged instance a set of card titles points at — the full answer, including the +/// two ways there isn't one. /// -/// `card_titles` is every title she currently holds — resolution is over the WHOLE held -/// set rather than one card, because the question a work turn asks is "where are my hands -/// supposed to be", and that has one answer for the turn. Two different held cards -/// matching two different staged instances is the same ambiguity as one title matching -/// two, and refuses identically. -pub fn workspace_for_held_cards<'a, I>(peer: &uuid::Uuid, card_titles: I) -> Option +/// Callers need to tell these apart. A work turn treats [`None`](CardWorkspace::None) as +/// "an ordinary card, hands stay put" and [`Ambiguous`](CardWorkspace::Ambiguous) as a +/// staging defect worth reporting; a claim dispatcher wants the instance NAME as well as +/// the path (its era venv is keyed by it). An `Option` erased both distinctions, +/// which is why the second caller kept its own copy of the walk. +#[derive(Debug, PartialEq, Eq)] +pub enum CardWorkspace { + /// Exactly one staged instance is named by these titles. + One { instance: String, path: PathBuf }, + /// No staged instance is named — the ordinary, non-benchmark case. + None, + /// More than one. Never resolved: rooting at either would put hands in a repo the + /// other card is not about, and a diff taken there scores a false zero for the one + /// actually being worked. + Ambiguous { candidates: Vec }, +} + +/// Resolve card titles against this citizen's staged checkouts. +/// +/// `card_titles` is every title in play — resolution is over the WHOLE set rather than one +/// card, because the question a work turn asks is "where are my hands supposed to be", and +/// that has one answer for the turn. Two different held cards matching two different +/// staged instances is the same ambiguity as one title matching two, and refuses +/// identically. +pub fn resolve_for_titles<'a, I>(peer: &uuid::Uuid, card_titles: I) -> CardWorkspace where I: IntoIterator, { let staged = staged_instances(peer); - if staged.is_empty() { - return None; - } let titles: Vec<&str> = card_titles.into_iter().collect(); + match select(&staged, &titles) { + Selection::One(instance) => match staging_root(peer) { + Some(root) => CardWorkspace::One { + path: root.join(&instance), + instance, + }, + None => CardWorkspace::None, + }, + Selection::None => CardWorkspace::None, + Selection::Ambiguous(candidates) => CardWorkspace::Ambiguous { candidates }, + } +} + +/// The matching rule alone, with the filesystem taken out of it. +/// +/// Split from [`resolve_for_titles`] so the rule is TESTED rather than restated: the +/// disk half needs a fixture, the decision does not, and a test that re-derives the +/// containment check in its own body cannot fail when the real one changes. +#[derive(Debug, PartialEq, Eq)] +enum Selection { + One(String), + None, + Ambiguous(Vec), +} + +fn select(staged: &[String], titles: &[&str]) -> Selection { let mut hits: Vec<&String> = staged .iter() .filter(|inst| titles.iter().any(|t| t.contains(inst.as_str()))) .collect(); hits.dedup(); match hits.as_slice() { - [one] => staging_root(peer).map(|root| root.join(one.as_str())), - [] => None, - many => { - // Two staged instances named in her held titles. Rooting at either would put - // her hands in a repo the other card is not about — and a diff taken there - // scores a false zero for the one she was actually working. Refuse loudly. + [one] => Selection::One((*one).clone()), + [] => Selection::None, + many => Selection::Ambiguous(many.iter().map(|s| (*s).clone()).collect()), + } +} + +/// The workspace a claimed card points at, or `None` when there isn't exactly one — the +/// projection a WORK TURN wants, which only needs "root here or leave her hands alone". +/// +/// Ambiguity probes here rather than at [`resolve_for_titles`] because this is the caller +/// that would silently score a false zero: it roots hands and a diff is taken afterwards. +/// A dispatcher matching on the enum reports ambiguity in its own vocabulary instead. +pub fn workspace_for_held_cards<'a, I>(peer: &uuid::Uuid, card_titles: I) -> Option +where + I: IntoIterator, +{ + match resolve_for_titles(peer, card_titles) { + CardWorkspace::One { path, .. } => Some(path), + CardWorkspace::None => None, + CardWorkspace::Ambiguous { candidates } => { crate::probe!( class = "persona.work.staged_ambiguous", peer = %peer, - matches = many.len(), - candidates = many - .iter() - .map(|s| s.as_str()) - .collect::>() - .join(","), + matches = candidates.len(), + candidates = candidates.join(","), "held cards name MULTIPLE staged instances — refusing to guess which repo \ her hands belong in; she works in her own workspace this turn" ); @@ -117,34 +169,54 @@ where mod tests { use super::*; + fn staged(names: &[&str]) -> Vec { + names.iter().map(|s| (*s).to_string()).collect() + } + // what this catches: the containment rule is what binds a card to a repo, and it is // by construction (dispatch writes the title from the instance). A title that names // the instance resolves; an unrelated title does not, so an ordinary chat card never // silently re-roots a citizen's hands into a benchmark checkout. #[test] fn a_title_naming_a_staged_instance_selects_it_and_others_do_not() { - // Pure over the matching rule — the disk half is exercised by the live path, and - // a temp-dir fixture here would be testing `read_dir`, not the decision. - let staged = ["sympy__sympy-24152".to_string(), "flask-4045".to_string()]; - let titles = ["benchmark: sympy__sympy-24152 — fix the printer"]; - let hit: Vec<&String> = staged - .iter() - .filter(|i| titles.iter().any(|t| t.contains(i.as_str()))) - .collect(); - assert_eq!(hit.len(), 1); - assert_eq!(hit[0], "sympy__sympy-24152"); - - let unrelated = ["let's discuss the roadmap"]; - let none: Vec<&String> = staged - .iter() - .filter(|i| unrelated.iter().any(|t| t.contains(i.as_str()))) - .collect(); - assert!( - none.is_empty(), + let staged = staged(&["sympy__sympy-24152", "psf__requests-2148"]); + assert_eq!( + select(&staged, &["benchmark: sympy__sympy-24152 — fix the printer"]), + Selection::One("sympy__sympy-24152".to_string()) + ); + assert_eq!( + select(&staged, &["let's discuss the roadmap"]), + Selection::None, "an ordinary card must never re-root her hands" ); } + // what this catches: the ambiguity arm collapsing into a pick. Two staged instances + // named across the titles in play must REFUSE — rooting at either puts hands in a repo + // the other card is not about, and the diff taken there scores a false zero for the one + // actually being worked. Both callers depend on this staying distinguishable from + // "nothing staged": the work turn probes and leaves her hands alone, the claim + // dispatcher declines to fire a solve. + #[test] + fn titles_naming_two_staged_instances_refuse_rather_than_pick() { + let staged = staged(&["sympy__sympy-24152", "psf__requests-2148"]); + let both = select( + &staged, + &["bench: sympy__sympy-24152", "bench: psf__requests-2148"], + ); + match both { + Selection::Ambiguous(c) => assert_eq!(c.len(), 2, "both candidates must be named"), + other => panic!("two matches must refuse, got {other:?}"), + } + } + + // what this catches: nothing staged at all — the ordinary case for every non-benchmark + // citizen, which must resolve without the rule ever finding a hit. + #[test] + fn a_citizen_with_nothing_staged_selects_nothing() { + assert_eq!(select(&[], &["bench: sympy__sympy-24152"]), Selection::None); + } + // what this catches: a citizen with NO staged instances resolves to None without // touching the filesystem layout at all — the ordinary case for every non-benchmark // citizen, and the one that must stay free. From 191e31d4ad011a6cd60df92d42113305681b1f55 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 00:54:22 -0500 Subject: [PATCH 31/80] fix(benchmark): citizen-driven pre-claim honors the same broken-env refusal the solve does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The detached arm already refuses to fire into an env that failed to build — its comment is explicit: "A solve against an unbuildable env can ONLY void… no verdict, no lesson (the failure is the env's, not hers)", Joel: "why run broken code knowing she's gonna struggle and fall". The citizen pre-claim I added in f1417e1dd skipped that check, so the same broken checkout would land in her hands for a full work turn — worse than the detached case, because it burns a LIVE turn rather than a detached one. Same gate, same reason. The card still posts and stays claimable by hand once the env heals. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 7efa87e0e..df542c3b6 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -1476,8 +1476,14 @@ impl ActionCommand for BenchmarkDispatch { // airc directly, not the `work/claim` verb, so it cannot re-enter the // detached-solve dispatcher from here.) let mut pre_claimed = false; + // + // `staged_ok` gates the SWE arm for the SAME reason it gates the detached + // solve above ("why run broken code knowing she's gonna struggle and fall"): + // an unbuildable env can only void, and pre-claiming would put her hands in it + // for a full turn. The card still posts, claimable by hand once the env heals. let pre_claim_this = matches!(pc.work, CardWork::Gym { .. }) || (matches!(pc.work, CardWork::Swe { .. }) + && staged_ok && driver == crate::cognition::bench_round::WorkDriver::Citizen); if pre_claim_this { match self.registry.get(*who_peer) { From 58e927a959edd29acb8d1934a0d2bb37a4458f44 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 07:17:32 -0500 Subject: [PATCH 32/80] =?UTF-8?q?fix(persona):=20the=20held-work=20gate=20?= =?UTF-8?q?was=20unreachable=20on=20the=20ONE=20path=20benchmarks=20use=20?= =?UTF-8?q?=E2=80=94=20plus=20a=20glass=20box=20so=20it=20can=20never=20hi?= =?UTF-8?q?de=20again?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live bisection tonight: a citizen held a staged bench card, heard her room, took turns, and `persona.turn.work` stayed 0 under every condition. Five hypotheses fit the evidence equally because THE GATE EMITS NOTHING WHEN IT DECLINES — five conditions, one silent exit. Reading it settles all five at once, and two of them are fatal by construction. (1) `!directed` MADE IT UNREACHABLE FOR BENCHMARKS. The act-question was asked only on an UNDIRECTED turn. But `benchmark/dispatch` actuates with an ADDRESSED imperative — by its own design note, "an addressed imperative in its OWN message block actuates; a card sitting silently on the board does not". So every kickoff drives a DIRECTED turn, and every directed turn skipped the work question. The actuation path and the work gate were mutually exclusive: the work turn could not fire on a dispatched round no matter what else was correct. `directed` was never load-bearing here — this branch already sits behind her PASS on the speak-question, so she has declined to talk either way, and the act-question stays hers to pass again. (2) `InProgress`-ONLY WAS CIRCULAR. The filter took `InProgress`, but claiming a card (`work/claim`, or dispatch's pre-claim) leaves it `Claimed`; `InProgress` requires an explicit `work/state` call. So the gate demanded the state that starting work is what produces — she could never begin, because beginning was the precondition. Both states mean "this card is in her hands", which is the only question this gate asks. Either defect alone accounts for every zero measured tonight, and neither is visible from outside — which is the third fix and the durable one. `persona.work.gate` now reports the DECISION and every input to it on EVERY path: directed, active_claims, held, the card states verbatim, and the claims error if the call failed. The claims call also stops swallowing its own error into a skipped branch. The evening's real lesson, and Joel's framing of it: these steps were being held in someone's head instead of in the substrate. A gate whose refusal is invisible is a gate nobody can debug — so it reports itself now. 36 service_loop tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/service_loop.rs | 68 +++++++++++++++++-- 1 file changed, 61 insertions(+), 7 deletions(-) diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index b705105eb..0e9621016 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -1243,15 +1243,69 @@ async fn serve_persona_loop_inner( // question, never an instruction — the card is not made // louder and nothing nags inside the speak turn // ([[no-hardcoded-heuristics-to-steer-cognition]]). - if !directed { + // + // GLASS BOX (2026-08-18). This gate has FIVE conditions and used to + // emit NOTHING when any of them declined, so "she holds a card and + // never worked it" looked identical whether the citizen was absent, + // the claims call failed, the states didn't match, or the set was + // empty. One evening of live bisection produced five hypotheses that + // the probe stream could not tell apart — because the branch was + // silent on every path but the taken one. It now reports the DECISION + // and every input to it, always. A gate whose refusal is invisible is + // a gate nobody can debug ([[a-perception-fact-is-honesty]]). + // + // WHY `directed` NO LONGER BLOCKS. The act-question used to be asked + // only on an UNDIRECTED turn — which made it unreachable on the one + // path benchmarks actually use: `benchmark/dispatch` actuates with an + // ADDRESSED imperative ("an addressed imperative in its OWN message + // block actuates; a card sitting silently on the board does not"), so + // every kickoff drives a DIRECTED turn and every directed turn skipped + // the work question. The actuation path and the work gate were + // mutually exclusive by construction. The `directed` flag was never + // load-bearing for correctness here: this whole branch already sits + // behind her PASS on the speak-question, so she has declined to talk + // either way, and the act-question stays hers to pass again. + // + // WHY `Claimed` COUNTS AS HELD. The filter took `InProgress` only, + // but claiming a card — `work/claim`, or dispatch's pre-claim — leaves + // it `Claimed`; `InProgress` requires an explicit `work/state` call. + // So the gate demanded a state that starting work is what produces: + // she could never begin, because beginning was the precondition. Both + // states mean "this card is in her hands", which is the only question + // this gate is asking. + { if let Some(citizen) = conversation.stream_citizen() { - if let Ok(claims) = citizen.active_claims().await { - let held: Vec<&airc_lib::WorkCard> = claims + let claims_result = citizen.active_claims().await; + let claims_err = + claims_result.as_ref().err().map(|e| e.to_string()); + let claims = claims_result.unwrap_or_default(); + let held: Vec<&airc_lib::WorkCard> = claims + .iter() + .filter(|c| { + matches!( + c.state, + airc_work::CardState::InProgress + | airc_work::CardState::Claimed + ) + }) + .collect(); + crate::probe!( + class = "persona.work.gate", + persona = %ctx.identity.agent_name, + directed = directed, + active_claims = claims.len(), + held = held.len(), + claims_error = claims_err.as_deref().unwrap_or(""), + states = claims .iter() - .filter(|c| { - matches!(c.state, airc_work::CardState::InProgress) - }) - .collect(); + .map(|c| format!("{:?}", c.state)) + .collect::>() + .join(","), + decision = if held.is_empty() { "no_held_work" } else { "work_turn" }, + "held-work gate evaluated after a speak-pass — this row is \ + the ONLY place the act-question's inputs are visible" + ); + { if !held.is_empty() { let burst = held_work_burst(&held); // The producer's CONTEXT half, kept before the burst is From 89bd2a1eebae3580abbf34144c18f57559fb2d5d Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 08:10:48 -0500 Subject: [PATCH 33/80] =?UTF-8?q?docs(planning):=20THE=20ROOM=20IS=20THE?= =?UTF-8?q?=20RUNNER=20=E2=80=94=20build=20plan=20with=20a=20probe=20gate?= =?UTF-8?q?=20per=20slice?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, 2026-08-18: "adapt from natural room BACK INTO EVERY benchmark scoring… the glue on each side… the benchmark not running things. It's just another activity. That way any team can be inside… Without it, we're like everyone else, and measuring dumb loopers not colleagues." Not a benchmark plan. Benchmarks are the first consumer of a general capability: an activity whose recipe carries RULES, a team working cards inside it, and a held-out oracle marking the result. Swap the oracle and the same activity is ordinary work — held-out tests, CI green, customer accepted. That is the infinite-utility half, and it is the same code path as the self-improvement half. GROUNDED IN WHAT IS ALREADY WRITTEN, not re-derived. ROUND-LIFECYCLE (#371) has the stage machine, the three laws and a build order; BENCHMARK-AS-KANBAN (#346) has task→card→artifact→verdict AND settles the grader question in writing: "the card carries DELIVERY and LIFECYCLE, the harness keeps JUDGMENT… the terminal transition on a benchmark card is harness-only". So the scorer is NOT a role — deterministic, held out, one-way — while a TEACHER role (feedback, N+1 correction, training pair) is real, optional, per-recipe, and deferred by Joel's call. Everything works with or without it. The plan opens with an ALREADY-BUILT table, because rebuilding is the failure mode: BenchmarkAdapter + registry, TaskOutcome's two artifact channels, dispatch's task→card→room, bench_round, recipe roles-as-data, ProofSpec. "Each benchmark is just an adapter" is ALREADY TRUE; only delivery is missing. Nine slices, each with the PROBE THAT PROVES IT — Joel: "make sure it's proven by probing as we do so". A slice is done when its row appears live with the right fields, never on a compile or a unit test. §2 carries tonight's worked example of why: one gate, five conditions, three defects, and NOTHING emitted when it declined — so five hypotheses fit the evidence identically for a whole session. A gate whose refusal is invisible is a gate nobody can debug. Also records that WorkDriver{DetachedSolve|Citizen} (shipped today) is a STOPGAP — a Rust enum deciding what the recipe should say — and retires in slice 7. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/planning/ROOM-AS-RUNNER-BUILD-PLAN.md | 281 +++++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 docs/planning/ROOM-AS-RUNNER-BUILD-PLAN.md diff --git a/docs/planning/ROOM-AS-RUNNER-BUILD-PLAN.md b/docs/planning/ROOM-AS-RUNNER-BUILD-PLAN.md new file mode 100644 index 000000000..115e11e39 --- /dev/null +++ b/docs/planning/ROOM-AS-RUNNER-BUILD-PLAN.md @@ -0,0 +1,281 @@ +# The Room Is The Runner — build plan + +**Objective (Joel, 2026-08-18):** + +> *"It's whatever is needed to adapt from natural room BACK INTO EVERY benchmark +> scoring… the glue on each side, through natural continuum, and the benchmark not +> running things. It's just another activity. That way any team can be inside. They +> can work together, delegate, do whatever feature continuum makes possible. Without +> it, we're like everyone else, and measuring dumb loopers not colleagues… If we do +> this right, we crush benchmarks, but more importantly we have infinite utility, and +> an ability to self improve."* + +This plan is the build order for that. It is not a benchmark plan. Benchmarks are the +first consumer of a **general** capability: an activity whose recipe carries rules, a +team that works cards inside it, and a held-out oracle that marks the result. + +**Status:** plan. Every slice below carries the probe that proves it, and no slice is +"done" on a compile or a unit test. + +--- + +## 0. Read these first, and do not re-derive them + +Three days of driver-hours have been spent re-deriving decisions that were already +written down. The two source docs are correct and this plan implements them: + +- **[ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md](../architecture/ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md)** (#371) — the stage machine, the three laws, the build order. §6 of that doc is slices 3–6 here. +- **[BENCHMARK-AS-KANBAN.md](../architecture/BENCHMARK-AS-KANBAN.md)** (#346) — task→card→claim→artifact→verdict, and the delivery/judgment line. +- **[BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md](../architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md)** — the law, and the acceptance test. + +### ALREADY BUILT — rebuilding any of these is the failure mode + +| capability | where | note | +|---|---|---| +| `BenchmarkAdapter` trait + live registry | `cognition/benchmark.rs:89` | `tasks()` + `grade()`; "each benchmark is an adapter" is ALREADY TRUE | +| `TaskOutcome` with BOTH artifact channels | `cognition/benchmark.rs` | spoken answer *and* workspace diff | +| task → card → per-run room | `commands/benchmark.rs` (`benchmark/dispatch`) | THE one adapter into kanban | +| round as an entity, keyed by run room | `cognition/bench_round.rs` | stages are the gap, not the entity | +| card-state → bus event | `modules/benchmark_grade.rs` (#450) | the emitter the stage machine subscribes to | +| recipe as data: regions, affordances, params, **roles** | `experience/recipe.rs` | `CitizenRecipe { role }` — roles already authored data (#430) | +| `ProofSpec` on an affordance | `experience/mod.rs:309` | `None \| CleanLane \| Attestation` — the hook a verdict extends | +| claims survive long work | `e3b1a0f98` | presence pump renews held claims | +| staged-workspace resolution | `persona/staged_workspace.rs` | ONE resolver, three callers | +| act-question glass box | `persona/service_loop.rs` (`persona.work.gate`) | shipped 2026-08-18 | + +**Before adding any type, grep for it.** `[[grep-for-the-mechanism-before-proposing-to-build-it]]`. + +--- + +## 1. The shape, in one picture + +The benchmark touches only the two ends. The middle is an ordinary room. + +``` + IN (recipe projects) OUT (oracle marks) + ──────────────────── ───────────────── + adapter.tasks() adapter.grade(task, artifact) + │ ▲ + ▼ │ + ┌──────────────────────────── ACTIVITY ROOM ────────────────────────────┐ + │ recipe RULES: per-role instructions, objectives, who drives │ + │ cards on the board team claims, splits, delegates │ + │ real hands, real workspace every continuum feature available │ + └───────────────────────────────────────────────────────────────────────┘ + │ + artifact (diff / answer) +``` + +**The generalization is the point, and it is free:** swap the oracle and the same +activity is ordinary work. Held-out tests → SWE-bench. CI green → a real PR. Customer +accepted → a real deliverable. The benchmark was never special; it is the case where +the oracle happens to be a sealed test set. + +### The law that protects the number + +**Delivery and lifecycle belong to the card. Judgment belongs to the oracle.** +(BENCHMARK-AS-KANBAN, "the line that must not blur".) + +- A citizen moves her card through *working* states. That is honest self-report. +- A citizen **never** sets the terminal state on a graded card. +- The scorer is **not a role**. It is deterministic and held out. A persona rendering + the verdict could be talked into a pass, and the number would be worthless — that is + the difference between crushing a benchmark and self-grading one. +- **Verdicts flow one way.** Nothing downstream of a verdict may revise it, and the + scorer never reads the room. + +### The teacher is OPTIONAL (Joel, 2026-08-18) + +A *teacher* role — reads the verdict, explains the failure, proposes the N+1 +correction, yields the training pair — is real and wanted, but it is **per-recipe and +deferred**. Everything below works with or without one. Do not block on it. When it +lands it is a `CitizenRecipe` role like any other, strictly downstream of the verdict. + +--- + +## 2. The discipline: probe-first, or it did not happen + +Tonight's worked example, and the reason this section exists. + +The act-question gate in `service_loop` had **five conditions and emitted nothing when +it declined**. Three of them were defects: + +1. `!directed` — the work turn fired only on an *undirected* turn, while dispatch + actuates with an *addressed* imperative. The actuation path and the work gate were + **mutually exclusive by construction**. +2. `InProgress`-only — claiming leaves a card `Claimed`; the gate demanded the state + that starting work is what produces. **Circular.** +3. The gate sits *inside* the speak-pass branch, so it is only reached when she + declines to talk. Found only after the first two were fixed. + +Five hypotheses fit the evidence identically for an entire session, because the branch +was silent on every path but the taken one. **A gate whose refusal is invisible is a +gate nobody can debug.** + +**So, for every slice below:** + +- **Emit on the negative path.** Any decision that can decline must say so, with its + inputs, always. `decision=` is a required field, not a nice-to-have. +- **The probe is named in the slice.** A slice is done when its probe row appears live + with the right fields — not when it compiles, not when a unit test passes. +- **Measure deltas from a timestamped baseline.** Probe totals survive reboots; a + cumulative count cannot show a rate change. This produced two vacuous "proofs". +- **Discover classes, never guess them.** Query `--class='*'` and read what is there. +- **An absence is an unfinished measurement.** Zero rows means the instrument is + unproven. Prove the instrument, then read the silence. +- **Positive-control every claim.** For a fix: an event that previously failed and now + succeeds. For a scorer: the gold patch. + +--- + +## 3. Slices + +Ordered smallest-true-cause first. Each is independently useful and independently +provable. **Do not start slice N+1 until slice N's probe gate is green live.** + +### Slice 1 — the act-question comes from the RULES, not from a nested negative + +**Why first:** it is the live blocker. The work turn has never fired for a bench card, +and the current gate is three-deep in accidental preconditions. Every downstream slice +measures a room where nobody works. + +**What:** the question "you hold work in this activity — work it?" becomes a +first-class turn the activity's rules ask, not a side effect of declining to speak. +Retire the nesting; keep her freedom to pass (the answer stays hers — this adds a +question, never an instruction, per `[[no-hardcoded-heuristics-to-steer-cognition]]`). + +**Files:** `persona/service_loop.rs`. + +**Probe gate:** +- `persona.work.gate` rows appear on turns where she **spoke** (not only on passes) — + proving the question is asked independently of the speak decision. +- `persona.turn.work` ≥ 1 with a held card. +- `persona.work.hands_rooted` names the staged workspace. +- `git diff` in that workspace is non-empty. +- hands restored afterward (no #312 leak). + +**Falsifier:** if `persona.work.gate` shows `held=0` while the board says she holds a +card, the defect is claim visibility, not the gate — go there instead. + +### Slice 2 — the round pulses while it works (law 2) + +**Why:** liveness today is a file mtime written once per attempt, against attempts that +legitimately run hours. The projection whose stated purpose is *"silence must never be +ambiguous with progress"* structurally cannot tell them apart, and it has already +flagged a healthy run `quiet`. + +**What:** a heartbeat consuming `WorkspaceCycle::actions_taken()` (the seam exists, +`c9ba5f943`) so `acts` and last-activity are live at the cadence work happens. + +**Files:** `cognition/bench_round.rs`, the run projection. + +**Probe gate:** `bench.round.pulse` rows arrive at working cadence during a live run, +with a monotonically climbing `acts`; a genuinely idle run stops emitting and is +distinguishable from a working one **without reading a file**. + +### Slice 3 — the round entity owns STAGES + +**What:** `BenchRound` gains `stage: STAGING | READY | WORKING | GRADING | DONE`, and +the transitions land as probes. Entity exists; the stage field and subscribers do not. + +**Files:** `cognition/bench_round.rs`. + +**Probe gate:** `bench.round.stage` fires once per transition, in order, naming the +**emitter** — never a timer. A round that reaches DONE emits exactly one DONE. + +### Slice 4 — transitions come from the components that KNOW (law 1) + +**What:** env builder → `STAGING→READY`. Supervisor (hosted + serving ready, #442) → +gate open. Work board first claim → `WORKING`. Card store (#450, already event-driven) +→ `GRADING`. Round entity all-settled → `DONE`. + +**Probe gate:** each transition row carries the emitting component. **Zero timeouts, +zero polls, zero agent judgement anywhere in the path** — grep the diff for `sleep`, +`interval`, and retry windows as the review gate. + +### Slice 5 — `RoundViewState` on the pipe + +**What:** fold the round entity onto the same ViewState pipe humans and citizens read. +Retires the 5s progress-directory poll in `positron_bench_source`. + +**Probe gate — the acceptance test, from the law doc:** + +> *Can a citizen standing in the room perceive the run's state through the same +> ViewState pipe the human's screen uses?* + +Proven by a citizen answering "what stage is this round in" from perception alone, and +by a fresh driver answering *is it ready / has it started / is it stuck / is it done* +with **queries only** — zero log reads, zero probe archaeology, zero inference from an +absence. + +### Slice 6 — dispatch consumes the state pipe (#442) + +**What:** dispatch refuses to stage into a room that is not READY, and says why. +Expressed as a state, not a check. + +**Probe gate:** a dispatch attempted during a serving transition is **refused with a +reason**, and the same dispatch succeeds once READY. Both rows visible. + +### Slice 7 — the rules half of the recipe + +**What:** `ExperienceRecipe` gains the authored rules it never had: +- per-role **instructions** and **objectives** (a worker gets a charter, not just a card) +- **who drives** the work — retiring the `WorkDriver` enum into recipe data +- the activity's **outcome**, recipe-owned rather than `benchmark_grade.rs`-owned + +**Files:** `experience/recipe.rs`, `experience/mod.rs`, `recipes/*.json`. + +**Probe gate:** a citizen's capture contains her role's instructions verbatim +(`[[READ-HER-CAPTURE-first]]` — read the input, never a probe's prose about it), and +two recipes with different rules produce different behaviour from the same code path. + +**Note:** the `WorkDriver{DetachedSolve|Citizen}` enum shipped 2026-08-18 is a +*stopgap* — a Rust enum deciding what the recipe should say. It retires here. Its one +lasting contribution is the round-open-before-first-card ordering, which stays. + +### Slice 8 — `ProofSpec::Verdict`: the oracle as an affordance + +**What:** the scorer becomes a declared affordance yielding a verdict proof, alongside +`CleanLane` and `Attestation`. Deterministic, held out, one-way. `adapter.grade()` +already exists and is not rebuilt — this is the declaration and the wiring. + +**Probe gate — positive control, non-negotiable:** the **gold patch** scores resolved +in every env class in play. A 0 must mean "the model failed", never "the env lied". +Report the ungradeable count explicitly (#383/#380: 114/300 → target 0). + +### Slice 9 — the number + +Full round, citizen-driven, on a deploy-verified build with hosted citizens, gradeable +envs, and a round that ends. Report resolved/total with per-instance receipts and the +patch sha for each pass. + +--- + +## 4. Forbidden moves + +Each of these has actually happened and cost a session. + +- **A parallel runner.** If you are adding a field to a benchmark probe so an external + consumer can parse it better — stop. The consumer should not be external. +- **A timeout, retry window, sleep, or agent-side heuristic to decide a stage.** That + is the ritual growing back. Ask: *which component already knows this, and why isn't + it saying so?* +- **A second adapter/registry/allocator.** `BenchmarkAdapter` exists. `bench_round` + exists. `staged_workspace` exists. +- **A silent decline.** Any new gate emits on its negative path or it does not land. +- **Reporting a state you did not observe.** Say "I did not observe it," and name the + query that would. +- **A verdict that flows backwards.** Nothing downstream of the oracle may revise it. +- **Grading speech.** The artifact is the evidence; a citizen describing a fix that + never touched disk is our measured failure mode. + +--- + +## 5. Standing rules + +- `export CARGO_TARGET_DIR="$HOME/.continuum/cache/cargo-target"` before any cargo. +- Commit, **then** deploy. Verify the running SHA == git HEAD. +- `df -h /` after each cargo cycle; sweep ghost target dirs under 20 GB free. +- Never `--no-verify`. Canary is the branch; main merge needs Joel. +- Restart freely — hesitating to reboot is the defect. But say when a live round dies + for it. From 94d008e383e49008da2d38eb3bfc00f7d9b81655 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 08:34:40 -0500 Subject: [PATCH 34/80] feat(persona): the act-question becomes its own module, asked on BOTH turn outcomes (slice 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 1 of ROOM-AS-RUNNER. Two changes, one structural and one behavioural, and the structural one is why the behavioural one was invisible for a session. BEHAVIOUR — the question is no longer reachable only through a silence. It lived inside the `SettleStep::Passed` arm, so the act-question was asked ONLY when she declined to speak. A citizen who ANSWERED someone in the room and also held a claimed card was never asked whether to work it: talking and working were alternatives. For a colleague they are not. It is now asked on BOTH outcomes — after a spoken reply (post-send, so the room hears her at the same latency as before) and after a pass. That was the THIRD defect in this one conditional, after the two fixed hours earlier: `!directed` excluded every addressed kickoff — which is how dispatch actuates, so the actuation path and the work gate were mutually exclusive by construction — and an `InProgress`-only filter demanded the state that starting work is what produces, which is circular. Any one of the three alone accounts for `persona.turn.work == 0` in every live round we have ever measured. STRUCTURE — and this is the actual lesson. 273 lines, nested SIX levels deep, inside a match arm inside the turn loop of a 4,900-line file. That depth is the bug's habitat: a condition buried that far down declines invisibly, so five competing hypotheses looked identical from outside and only a full evening of live bisection separated them. Fixing the conditions without removing the habitat would have left the next defect just as well hidden. So the act-question is now `persona/act_question.rs` — one file, one concept, its contract stated at the top: it is a QUESTION never an instruction (she may still pass), it never decides WHEN to ask (the caller owns that), and it reports on EVERY path including every decline. service_loop.rs 4,900 → 4,626; the new module is 334. Per CLAUDE.md: decompose continuously, a new concept gets its own file. `cycle` is fetched inside from the same global registry the turn loop uses rather than threaded through as a borrow, and a missing cycle now emits `decision=no_cycle` instead of silently doing nothing — the one path that could still have declined in the dark. 36 service_loop tests green. Live proof owed: `persona.turn.work` has still never fired, and that is what the next round measures. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/persona/act_question.rs | 334 ++++++++++++++++++ core/continuum-core/src/persona/mod.rs | 1 + .../src/persona/service_loop.rs | 325 ++--------------- 3 files changed, 370 insertions(+), 290 deletions(-) create mode 100644 core/continuum-core/src/persona/act_question.rs diff --git a/core/continuum-core/src/persona/act_question.rs b/core/continuum-core/src/persona/act_question.rs new file mode 100644 index 000000000..cda433fb5 --- /dev/null +++ b/core/continuum-core/src/persona/act_question.rs @@ -0,0 +1,334 @@ +//! The ACT-QUESTION — "you hold work; do you want to work it?" +//! +//! ONE question, asked at ONE seam, by the citizen's own turn. +//! +//! ## Why this is its own file +//! +//! It used to be 273 lines nested SIX levels deep inside a match arm inside the turn +//! loop of a 4,900-line `service_loop.rs`. That depth is what hid three separate +//! defects from a full session of live bisection (2026-08-18), because a condition +//! buried that far down declines invisibly and every hypothesis looks the same from +//! outside. Depth was the bug's habitat, so the fix includes taking the habitat away. +//! +//! ## The three defects it was hiding +//! +//! 1. **`!directed`** — the question was asked only on an UNDIRECTED turn, while +//! `benchmark/dispatch` actuates with an ADDRESSED imperative. The actuation path +//! and the work gate were mutually exclusive by construction. +//! 2. **`InProgress`-only** — claiming a card leaves it `Claimed`; `InProgress` needs +//! an explicit `work/state`. The gate demanded the state that starting work is what +//! produces. Circular: she could never begin, because beginning was the precondition. +//! 3. **Nested under the speak-PASS arm** — reachable ONLY through a silence, so a +//! citizen who ANSWERED someone and also held a claimed card was never asked whether +//! to work it. Talking and working were alternatives. For a colleague they are not. +//! +//! ## The contract +//! +//! - It is a QUESTION, never an instruction. She may pass, exactly as before +//! ([[no-hardcoded-heuristics-to-steer-cognition]]). +//! - It NEVER decides when to ask. The caller owns that; this module only asks. +//! - It reports on EVERY path, including every decline — `persona.work.gate` carries +//! the decision and its inputs whether or not a work turn follows. A gate whose +//! refusal is invisible is a gate nobody can debug. + +use crate::persona::service_loop::{held_work_burst, PersonaConversation, LIVE_MAX_ACTS}; +use crate::persona::supervisor::HostedPersona; + +/// Ask the act-question for a citizen who may be holding work. +/// +/// Called from BOTH turn outcomes — after she speaks, and after she passes — because +/// holding work is what makes the question relevant, not the speak decision. +pub(crate) async fn ask_the_act_question( + ctx: &HostedPersona, + conversation: &mut dyn PersonaConversation, + lamport: u64, + turn_room: uuid::Uuid, + directed: bool, +) { + // Her cognition cycle, fetched the same way the turn loop fetches it — passing it + // in would thread a borrow through the whole turn for one optional branch. + let Some(cycle) = + crate::cognition::persona_workspace::global().get(&ctx.identity.peer_id.as_uuid()) + else { + crate::probe!( + class = "persona.work.gate", + persona = %ctx.identity.agent_name, + decision = "no_cycle", + "act-question skipped: no WorkspaceCycle registered for this citizen" + ); + return; + }; + // THE SECOND QUESTION (BigMama's gate-conflation diagnosis, + // verified in-file 2026-08-08; the root under Joel's "missing + // something"): speak and act shared ONE terminal gate, so + // "nothing to say" — the CORRECT answer on a quiet room — + // also silently answered "nothing to do" for a citizen + // holding claimed work. The ledger's falsifiable signature: + // every completion followed a direct address; zero happened + // ambiently. Working is not speaking. A Pass settles the + // speak-question; when she holds an in-progress claim, the + // ACT-question is asked as its OWN turn — a separate + // drive_to_settle whose burst IS her card, under the + // workspace-deliverable contract. Her answer stays hers: + // Pass here too and the turn simply ends. This adds a + // question, never an instruction — the card is not made + // louder and nothing nags inside the speak turn + // ([[no-hardcoded-heuristics-to-steer-cognition]]). + // + // GLASS BOX (2026-08-18). This gate has FIVE conditions and used to + // emit NOTHING when any of them declined, so "she holds a card and + // never worked it" looked identical whether the citizen was absent, + // the claims call failed, the states didn't match, or the set was + // empty. One evening of live bisection produced five hypotheses that + // the probe stream could not tell apart — because the branch was + // silent on every path but the taken one. It now reports the DECISION + // and every input to it, always. A gate whose refusal is invisible is + // a gate nobody can debug ([[a-perception-fact-is-honesty]]). + // + // WHY `directed` NO LONGER BLOCKS. The act-question used to be asked + // only on an UNDIRECTED turn — which made it unreachable on the one + // path benchmarks actually use: `benchmark/dispatch` actuates with an + // ADDRESSED imperative ("an addressed imperative in its OWN message + // block actuates; a card sitting silently on the board does not"), so + // every kickoff drives a DIRECTED turn and every directed turn skipped + // the work question. The actuation path and the work gate were + // mutually exclusive by construction. The `directed` flag was never + // load-bearing for correctness here: this whole branch already sits + // behind her PASS on the speak-question, so she has declined to talk + // either way, and the act-question stays hers to pass again. + // + // WHY `Claimed` COUNTS AS HELD. The filter took `InProgress` only, + // but claiming a card — `work/claim`, or dispatch's pre-claim — leaves + // it `Claimed`; `InProgress` requires an explicit `work/state` call. + // So the gate demanded a state that starting work is what produces: + // she could never begin, because beginning was the precondition. Both + // states mean "this card is in her hands", which is the only question + // this gate is asking. + { + if let Some(citizen) = conversation.stream_citizen() { + let claims_result = citizen.active_claims().await; + let claims_err = + claims_result.as_ref().err().map(|e| e.to_string()); + let claims = claims_result.unwrap_or_default(); + let held: Vec<&airc_lib::WorkCard> = claims + .iter() + .filter(|c| { + matches!( + c.state, + airc_work::CardState::InProgress + | airc_work::CardState::Claimed + ) + }) + .collect(); + crate::probe!( + class = "persona.work.gate", + persona = %ctx.identity.agent_name, + directed = directed, + active_claims = claims.len(), + held = held.len(), + claims_error = claims_err.as_deref().unwrap_or(""), + states = claims + .iter() + .map(|c| format!("{:?}", c.state)) + .collect::>() + .join(","), + decision = if held.is_empty() { "no_held_work" } else { "work_turn" }, + "held-work gate evaluated after a speak-pass — this row is \ + the ONLY place the act-question's inputs are visible" + ); + { + if !held.is_empty() { + let burst = held_work_burst(&held); + // The producer's CONTEXT half, kept before the burst is + // moved into the driver — one construction, so the + // training example records the prompt she was actually + // handed rather than a re-derived approximation of it. + let work_context = burst.clone(); + let work_framing = + crate::cognition::workspace::TurnFraming::self_thread( + false, + ) + .on_workspace(); + // HANDS FOLLOW THE CARD (#456). Her held card may be a + // staged benchmark checkout — a real git repo under + // `workspace/swe/`. Without rooting her hands + // there she works the card by writing into her OWN + // workspace, and the grader's `git diff` on the sandbox + // scores a false ZERO: the same defect glass-boxed on + // agent/solve 2026-07-22 (2 real acts, correct file + // written, empty patch). + // + // This is the live sibling of agent/solve's re-root, and + // it is what lets a citizen work a bench card IN HER OWN + // LOOP — which is the only path where the L2 training + // producer fires, so it is also what puts benchmark + // experience into her genome instead of only her memory. + // + // The re-root is PROCESS-GLOBAL (the file engine keys on + // caller identity), so the restore below is mandatory on + // EVERY exit — #312: after a flask solve, Anwen's live + // self was still reading the exam repo hours later. + // Non-bench cards resolve to None and nothing moves. + let card_workspace = + crate::persona::staged_workspace::workspace_for_held_cards( + &ctx.identity.peer_id.as_uuid(), + held.iter().map(|c| c.title.as_str()), + ); + let work_hands = match &card_workspace { + Some(ws) => { + let hands = + crate::cognition::persona_workspace::ActingHands::of( + &cycle, + ); + match crate::cognition::persona_workspace::root_acting_workspace( + &cycle, + &ws.to_string_lossy(), + &[], + false, + ) + .await + { + Ok(()) => { + crate::probe!( + class = "persona.work.hands_rooted", + persona = %ctx.identity.agent_name, + workspace = %ws.display(), + cards = held.len(), + "hands rooted at her claimed card's \ + staged workspace for this work turn" + ); + hands + } + Err(e) => { + // Fail LOUD, work anyway in her own + // workspace: a citizen who cannot reach + // the repo still gets her turn, and the + // empty patch is then explained on the + // probe stream instead of being a mystery + // zero. No silent re-root. + tracing::error!( + persona = %ctx.identity.agent_name, + workspace = %ws.display(), + error = %e, + "could NOT root hands at the claimed \ + card's workspace — she will work in \ + her own dir and any graded diff will \ + read EMPTY" + ); + None + } + } + } + None => None, + }; + let work = crate::cognition::act_observe::drive_to_settle( + &cycle, + burst, + turn_room, + LIVE_MAX_ACTS, + work_framing, + ) + .await; + // Give her back her own hands BEFORE anything else can + // observe them — every exit path from here (Spoke, Passed, + // Acted) must leave her rooted at home (#312). + if let Some(hands) = &work_hands { + if let Err(e) = + crate::cognition::persona_workspace::restore_acting_workspace( + hands, + ) + .await + { + tracing::error!( + persona = %ctx.identity.agent_name, + error = %e, + "work turn could NOT return her hands to her \ + own workspace — she is still rooted at the \ + card's repo and her live turns will act there" + ); + } + } + let (work_step, _) = + crate::cognition::act_observe::SettleStep::from_settled( + work, + ); + match work_step { + crate::cognition::act_observe::SettleStep::Spoke( + text, + ) => { + // She worked and has something to report — + // that report earned its send. + crate::probe!( + class = "persona.turn.work", + persona = %ctx.identity.agent_name, + lamport = lamport, + decision = "spoke", + "work-turn settled with a report" + ); + // Answer where she was asked — `turn_room` + // is the A.6 arrival room already resolved + // for this turn, so the report lands in the + // room whose work it reports on. + if let Err(e) = + conversation.say_in(turn_room, &text).await + { + tracing::warn!( + error = %e, + "work-turn report failed to send" + ); + } + // L2 producer on the WORK turn (#456). This was + // missing, and it is the highest-value training + // signal the substrate produces: the reply turn + // below already feeds the producer, but the turn + // where she actually WORKS HER CLAIMED CARD did + // not — so every act of real work was invisible + // to the genome while chat was not. + // + // The (context, completion) pair here is honest: + // context = the card burst she was handed, + // completion = the report she wrote after doing + // the work. Same shape as the reply path, same + // best-effort spawn, same quality bar applied + // inside the producer. + // + // Still the LIVE path — an eval fork never + // reaches here (`drive_to_settle` is called from + // the fork, this call site is not), so the + // measurement-contamination guard the reply path + // relies on is unchanged. + crate::persona::training_producer::produce( + ctx.identity.peer_id.as_uuid(), + ctx.identity.agent_name.clone(), + ctx.profile.model_id.clone(), + work_context.clone(), + text.clone(), + ); + } + crate::cognition::act_observe::SettleStep::Passed => { + crate::probe!( + class = "persona.turn.work", + persona = %ctx.identity.agent_name, + lamport = lamport, + decision = "passed", + "work-turn passed — her choice, honored" + ); + } + other => { + // Acted (results already in her working + // memory) or an inference failure — either + // way the receipt says which. + crate::probe!( + class = "persona.turn.work", + persona = %ctx.identity.agent_name, + lamport = lamport, + decision = ?std::mem::discriminant(&other), + "work-turn settled without a spoken report" + ); + } + } + } + } + } + } +} diff --git a/core/continuum-core/src/persona/mod.rs b/core/continuum-core/src/persona/mod.rs index c38eef6a9..f671f111b 100644 --- a/core/continuum-core/src/persona/mod.rs +++ b/core/continuum-core/src/persona/mod.rs @@ -94,6 +94,7 @@ pub mod scripted_adapter_factory; pub mod scripted_conversation; pub mod seed; pub mod self_task_generator; +pub mod act_question; pub mod service_loop; pub mod staged_workspace; pub mod service_module; diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 0e9621016..5d95b4bbe 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -1227,279 +1227,19 @@ async fn serve_persona_loop_inner( reason = "workspace-pass", "persona chose silence" ); - // THE SECOND QUESTION (BigMama's gate-conflation diagnosis, - // verified in-file 2026-08-08; the root under Joel's "missing - // something"): speak and act shared ONE terminal gate, so - // "nothing to say" — the CORRECT answer on a quiet room — - // also silently answered "nothing to do" for a citizen - // holding claimed work. The ledger's falsifiable signature: - // every completion followed a direct address; zero happened - // ambiently. Working is not speaking. A Pass settles the - // speak-question; when she holds an in-progress claim, the - // ACT-question is asked as its OWN turn — a separate - // drive_to_settle whose burst IS her card, under the - // workspace-deliverable contract. Her answer stays hers: - // Pass here too and the turn simply ends. This adds a - // question, never an instruction — the card is not made - // louder and nothing nags inside the speak turn - // ([[no-hardcoded-heuristics-to-steer-cognition]]). - // - // GLASS BOX (2026-08-18). This gate has FIVE conditions and used to - // emit NOTHING when any of them declined, so "she holds a card and - // never worked it" looked identical whether the citizen was absent, - // the claims call failed, the states didn't match, or the set was - // empty. One evening of live bisection produced five hypotheses that - // the probe stream could not tell apart — because the branch was - // silent on every path but the taken one. It now reports the DECISION - // and every input to it, always. A gate whose refusal is invisible is - // a gate nobody can debug ([[a-perception-fact-is-honesty]]). - // - // WHY `directed` NO LONGER BLOCKS. The act-question used to be asked - // only on an UNDIRECTED turn — which made it unreachable on the one - // path benchmarks actually use: `benchmark/dispatch` actuates with an - // ADDRESSED imperative ("an addressed imperative in its OWN message - // block actuates; a card sitting silently on the board does not"), so - // every kickoff drives a DIRECTED turn and every directed turn skipped - // the work question. The actuation path and the work gate were - // mutually exclusive by construction. The `directed` flag was never - // load-bearing for correctness here: this whole branch already sits - // behind her PASS on the speak-question, so she has declined to talk - // either way, and the act-question stays hers to pass again. - // - // WHY `Claimed` COUNTS AS HELD. The filter took `InProgress` only, - // but claiming a card — `work/claim`, or dispatch's pre-claim — leaves - // it `Claimed`; `InProgress` requires an explicit `work/state` call. - // So the gate demanded a state that starting work is what produces: - // she could never begin, because beginning was the precondition. Both - // states mean "this card is in her hands", which is the only question - // this gate is asking. - { - if let Some(citizen) = conversation.stream_citizen() { - let claims_result = citizen.active_claims().await; - let claims_err = - claims_result.as_ref().err().map(|e| e.to_string()); - let claims = claims_result.unwrap_or_default(); - let held: Vec<&airc_lib::WorkCard> = claims - .iter() - .filter(|c| { - matches!( - c.state, - airc_work::CardState::InProgress - | airc_work::CardState::Claimed - ) - }) - .collect(); - crate::probe!( - class = "persona.work.gate", - persona = %ctx.identity.agent_name, - directed = directed, - active_claims = claims.len(), - held = held.len(), - claims_error = claims_err.as_deref().unwrap_or(""), - states = claims - .iter() - .map(|c| format!("{:?}", c.state)) - .collect::>() - .join(","), - decision = if held.is_empty() { "no_held_work" } else { "work_turn" }, - "held-work gate evaluated after a speak-pass — this row is \ - the ONLY place the act-question's inputs are visible" - ); - { - if !held.is_empty() { - let burst = held_work_burst(&held); - // The producer's CONTEXT half, kept before the burst is - // moved into the driver — one construction, so the - // training example records the prompt she was actually - // handed rather than a re-derived approximation of it. - let work_context = burst.clone(); - let work_framing = - crate::cognition::workspace::TurnFraming::self_thread( - false, - ) - .on_workspace(); - // HANDS FOLLOW THE CARD (#456). Her held card may be a - // staged benchmark checkout — a real git repo under - // `workspace/swe/`. Without rooting her hands - // there she works the card by writing into her OWN - // workspace, and the grader's `git diff` on the sandbox - // scores a false ZERO: the same defect glass-boxed on - // agent/solve 2026-07-22 (2 real acts, correct file - // written, empty patch). - // - // This is the live sibling of agent/solve's re-root, and - // it is what lets a citizen work a bench card IN HER OWN - // LOOP — which is the only path where the L2 training - // producer fires, so it is also what puts benchmark - // experience into her genome instead of only her memory. - // - // The re-root is PROCESS-GLOBAL (the file engine keys on - // caller identity), so the restore below is mandatory on - // EVERY exit — #312: after a flask solve, Anwen's live - // self was still reading the exam repo hours later. - // Non-bench cards resolve to None and nothing moves. - let card_workspace = - crate::persona::staged_workspace::workspace_for_held_cards( - &ctx.identity.peer_id.as_uuid(), - held.iter().map(|c| c.title.as_str()), - ); - let work_hands = match &card_workspace { - Some(ws) => { - let hands = - crate::cognition::persona_workspace::ActingHands::of( - &cycle, - ); - match crate::cognition::persona_workspace::root_acting_workspace( - &cycle, - &ws.to_string_lossy(), - &[], - false, - ) - .await - { - Ok(()) => { - crate::probe!( - class = "persona.work.hands_rooted", - persona = %ctx.identity.agent_name, - workspace = %ws.display(), - cards = held.len(), - "hands rooted at her claimed card's \ - staged workspace for this work turn" - ); - hands - } - Err(e) => { - // Fail LOUD, work anyway in her own - // workspace: a citizen who cannot reach - // the repo still gets her turn, and the - // empty patch is then explained on the - // probe stream instead of being a mystery - // zero. No silent re-root. - tracing::error!( - persona = %ctx.identity.agent_name, - workspace = %ws.display(), - error = %e, - "could NOT root hands at the claimed \ - card's workspace — she will work in \ - her own dir and any graded diff will \ - read EMPTY" - ); - None - } - } - } - None => None, - }; - let work = crate::cognition::act_observe::drive_to_settle( - &cycle, - burst, - turn_room, - LIVE_MAX_ACTS, - work_framing, - ) - .await; - // Give her back her own hands BEFORE anything else can - // observe them — every exit path from here (Spoke, Passed, - // Acted) must leave her rooted at home (#312). - if let Some(hands) = &work_hands { - if let Err(e) = - crate::cognition::persona_workspace::restore_acting_workspace( - hands, - ) - .await - { - tracing::error!( - persona = %ctx.identity.agent_name, - error = %e, - "work turn could NOT return her hands to her \ - own workspace — she is still rooted at the \ - card's repo and her live turns will act there" - ); - } - } - let (work_step, _) = - crate::cognition::act_observe::SettleStep::from_settled( - work, - ); - match work_step { - crate::cognition::act_observe::SettleStep::Spoke( - text, - ) => { - // She worked and has something to report — - // that report earned its send. - crate::probe!( - class = "persona.turn.work", - persona = %ctx.identity.agent_name, - lamport = msg.lamport, - decision = "spoke", - "work-turn settled with a report" - ); - // Answer where she was asked — `turn_room` - // is the A.6 arrival room already resolved - // for this turn, so the report lands in the - // room whose work it reports on. - if let Err(e) = - conversation.say_in(turn_room, &text).await - { - tracing::warn!( - error = %e, - "work-turn report failed to send" - ); - } - // L2 producer on the WORK turn (#456). This was - // missing, and it is the highest-value training - // signal the substrate produces: the reply turn - // below already feeds the producer, but the turn - // where she actually WORKS HER CLAIMED CARD did - // not — so every act of real work was invisible - // to the genome while chat was not. - // - // The (context, completion) pair here is honest: - // context = the card burst she was handed, - // completion = the report she wrote after doing - // the work. Same shape as the reply path, same - // best-effort spawn, same quality bar applied - // inside the producer. - // - // Still the LIVE path — an eval fork never - // reaches here (`drive_to_settle` is called from - // the fork, this call site is not), so the - // measurement-contamination guard the reply path - // relies on is unchanged. - crate::persona::training_producer::produce( - ctx.identity.peer_id.as_uuid(), - ctx.identity.agent_name.clone(), - ctx.profile.model_id.clone(), - work_context.clone(), - text.clone(), - ); - } - crate::cognition::act_observe::SettleStep::Passed => { - crate::probe!( - class = "persona.turn.work", - persona = %ctx.identity.agent_name, - lamport = msg.lamport, - decision = "passed", - "work-turn passed — her choice, honored" - ); - } - other => { - // Acted (results already in her working - // memory) or an inference failure — either - // way the receipt says which. - crate::probe!( - class = "persona.turn.work", - persona = %ctx.identity.agent_name, - lamport = msg.lamport, - decision = ?std::mem::discriminant(&other), - "work-turn settled without a spoken report" - ); - } - } - } - } - } - } + // THE ACT-QUESTION. Asked here on the PASS path and again after a spoken reply, + // so holding work is what makes it fire — not declining to speak. + // `directed` is recomputed here rather than threaded: it is a pure function of + // (identity, msg.text) and the binding above lives inside the cycle branch. + let spoke_directed = ctx.identity.persona_identity().mentions(&msg.text); + crate::persona::act_question::ask_the_act_question( + ctx, + conversation, + msg.lamport, + turn_room, + spoke_directed, + ) + .await; outcome.turns_skipped += 1; continue; } @@ -1596,6 +1336,26 @@ async fn serve_persona_loop_inner( outcome.turns_errored += 1; continue; } + // SHE SPOKE — and she may ALSO hold work. Ask the act-question here too. + // + // The question used to live only on the PASS arm, which made answering someone + // mutually exclusive with working your own card: a citizen who replied in the room + // was never asked whether to act on the claim she was holding. That is backwards + // for a colleague — talking about the work and doing the work are not alternatives. + // Asked AFTER the reply is sent, so the room hears her answer at the same latency + // as before and the act-question can never delay a conversation. + // `directed` is recomputed here rather than threaded: it is a pure function of + // (identity, msg.text) and the binding above lives inside the cycle branch. + let spoke_directed = ctx.identity.persona_identity().mentions(&msg.text); + crate::persona::act_question::ask_the_act_question( + ctx, + conversation, + msg.lamport, + turn_room, + spoke_directed, + ) + .await; + let turn_duration_ms = turn_started.elapsed().as_millis() as u64; outcome.turn_latency.record(turn_duration_ms); @@ -1747,7 +1507,7 @@ const SELF_TICK_REST_CAP_MS: u64 = 240_000; /// repeat-guard fact) is how a looping mind notices itself; the ONLY external /// stopwatch that remains is the eval grader's `max_acts` — a proctored exam's /// clock, held by the observer, never wired into life. -const LIVE_MAX_ACTS: usize = usize::MAX; +pub(crate) const LIVE_MAX_ACTS: usize = usize::MAX; /// What woke the service loop this cycle. A message from the wire, the never-stop /// heartbeat, or the end of the stream. Returned by the `select!` so the borrow of @@ -2007,21 +1767,6 @@ fn push_work_board_anchor( turns.push(crate::cognition::workspace::BurstTurn::perception(anchor)); } -/// Build the `[anchor]` escalation line — the perception-side FACT that gives a -/// repeating mind somewhere concrete to go (work card d6f010c8, live -/// 2026-07-23: the `[pattern]` description fired and did NOT break the greeting -/// loop; a concrete work anchor posted in-room broke it instantly, room-wide). -/// A description competes with an empty-looking room; an anchor gives the next -/// token somewhere real to go. -/// -/// Mechanical and data-driven: built from the `room-kanban` delivery ALREADY in -/// this burst's slice ([`super::room_board_source::RoomBoardSource`] — the one -/// airc board read, never a second fetcher), quoting the top unclaimed and -/// in-progress card lines verbatim as the board source rendered them (one -/// render, one truth). An empty or unreadable board is stated honestly — never -/// a fabricated card ([[fallbacks-are-illegal-fail-loud]]). Perception, not -/// steering: it names what exists NOW; she still chooses -/// ([[no-hardcoded-heuristics-to-steer-cognition]]). /// The WORK-question burst — the input of the second gate a claim-holder's /// quiet turn asks (see the `SettleStep::Passed` arm). The burst IS the @@ -2029,7 +1774,7 @@ fn push_work_board_anchor( /// passing remains hers. Deliberately NOT the room transcript — the subject of /// this turn is the work, and the card details/workspace root arrive through /// her own grounding exactly as on any turn. -fn held_work_burst(held: &[&airc_lib::WorkCard]) -> String { +pub(crate) fn held_work_burst(held: &[&airc_lib::WorkCard]) -> String { use std::fmt::Write as _; let mut s = String::from( "[work turn] The room is quiet and your speak-turn is settled. This \ From 8a4abecc5e520d1c9d574af36d779e96fd7578c2 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 09:20:38 -0500 Subject: [PATCH 35/80] =?UTF-8?q?docs(architecture):=20content=20travels?= =?UTF-8?q?=20by=20HANDLE,=20never=20by=20copy=20=E2=80=94=20the=20design?= =?UTF-8?q?=20that=20replaces=20every=20result-reducer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel's architecture, dictated across this session and written down before any of it is built, because I spent the session building the wrong three things first and each one looked locally correct. THE INCIDENT. A citizen mid-SWE-bench was handed her own directory listing as `bytes":22592},{"kind":"file","name":"header.py"…` — head gone, opening mid-token, nothing saying anything had been removed. She could not tell how many files existed, whether the call succeeded, or that she was reading a fragment. Her prompt used 11,763 tokens of a 29,184 window: the window was 60% EMPTY while we shredded her results to fit a 1/32 share. Mechanism: `ToolResult.content` is a `String`. Commands produce typed output, serialize it, drop it in. Past that boundary the structure is gone and the only move left is cutting characters — which two sites did with two different rules, one keeping the head and one the tail. THE THREE WRONG FIXES, all recorded because I attempted them in order and each is the locally shortest path: 1. Tune the cut / keep the other half — both halves matter and which one depends on the payload. Any fixed rule is silently wrong half the time. 2. A smarter generic reducer (shed elements, shed lines, mark the elision). Better, still wrong: it cannot know a diff is atomic, that a patch reduced by half APPLIES CLEANLY and does the wrong thing, or that dropping lines shifts the line numbers the next edit depends on. Half a code fix is worse than none — no fix fails loudly, half a fix passes review. 3. Raise the fraction 1/32 → 1/8. The same hardcoded guess with a different number; it makes the corruption rarer and therefore harder to find. All three are one defect: the substrate inspecting a payload it does not understand, and deciding. THE RULE: content never travels, a reference to its source does. A result is a small header (always fits, always complete) plus a handle. The content stays where it was produced; a mind that wants more calls the handle. Not a new mechanism — the one the RAG and positron layers already use, one truth at the source and N projections. The tool-result channel adopts THAT interface instead of growing a parallel one. THE FORM IS A TRAIT, and this is the part that kills the whole bug class: nobody tracks down where content came from and special-cases it. You hold the reference and call its method; the source decides what a range means for its own content — lines for a file, entries for a listing, hunks for a diff. Same interface for a peer's artifact across the grid. What falls out without special-casing: nothing is ever malformed; content size decouples from context size (a 40k-line file is fully available on a 4k window, she pages); indivisible payloads refuse and say how to narrow, decided by their own implementation; and the act-history gap (#390/#414 — 2,863 acts executed, one visible, no path to the rest) turns out to be a handle problem, not a memory one. Build order starts at task #17 — reconcile HandleRef vs the URI-handle form — because with two handle types this design forks on day one. Nothing is wired yet, deliberately. The three reducers I wrote are reverted; the tree carries the doc and the existing (wrong but honest) behaviour until the interface lands. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../architecture/CONTENT-TRAVELS-BY-HANDLE.md | 121 ++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md diff --git a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md new file mode 100644 index 000000000..11f7f57ec --- /dev/null +++ b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md @@ -0,0 +1,121 @@ +# Content travels by HANDLE, never by copy + +**Status:** design, agreed with Joel 2026-08-18. Supersedes every "reduce the result to +fit" mechanism in cognition. Closes the open fork in task #17 (URI-Handle vs HandleRef). + +--- + +## The incident that forced it + +A citizen mid-SWE-bench, 2026-08-18, was handed her own directory listing as: + +``` +[result #1840] bytes":22592},{"kind":"file","name":"header.py","path":"… +``` + +Head gone, opening mid-token, nothing saying anything had been removed. She could not tell +how many files existed, whether the call succeeded, or that she was reading a fragment. +Her prompt used 11,763 tokens of a 29,184 window at the time — the window was 60% empty. + +The mechanism: `ToolResult.content` is a `String`. Every command produces typed output, +serde-serializes it, and drops it into that field. After that boundary the structure is +gone, so the only operation left on an oversized result is cutting characters — which two +different sites did with two different rules (one kept the head, one kept the tail). + +## The three fixes that are WRONG, and why + +Recorded because each one is locally the shortest path and each was attempted. + +1. **Tune the cut point / keep the other half.** Both halves matter and which one matters + depends on the payload: a listing identifies itself at the front, a test run reports at + the back. Any fixed rule is wrong about half the time, silently. + +2. **A smarter generic reducer** (shed array elements, shed lines, mark the elision). + Better than cutting bytes, still wrong: a generic reducer cannot know that a diff is + atomic, that a patch reduced by half applies cleanly and does the wrong thing, or that + dropping lines from a file body shifts every line number the next `code/edit` depends + on. *Half a code fix is worse than no fix: no fix fails loudly, half a fix passes + review.* The distinction is not in the bytes — it is in what the payload MEANS. + +3. **Raise the budget fraction.** Picking `1/8` instead of `1/32` is the same hardcoded + guess with a different number, and it does not stop the corruption — it only makes it + rarer and therefore harder to find. + +All three share one defect: **the substrate inspecting a payload it does not understand, +and deciding.** + +## The rule + +> **Content never travels. A reference to its source does.** + +A tool result is a small **header** plus a **handle**. The header always fits and is always +complete — what this is, how large, how to address it. The full content stays where it was +produced. A mind that wants more calls the handle. + +This is not a new mechanism. It is the one the RAG and positron layers already use: one +truth at the source, N projections, the consumer pages. The tool-result channel must use +*that* interface rather than growing a parallel one. + +## The interface + +Polymorphism, not inspection — the OpenCV-style `cv::Algorithm` shape this codebase +already prescribes. Nobody switches on what the content is; they hold a reference and call +its method, and the implementation decides. + +```rust +/// A thing that produced content and can still be asked about it. +/// Implemented BY the producer — a file engine, a listing, a RAG source, a positron +/// ViewState, a peer's artifact on another node. Callers never know which. +pub trait ContentSource: Send + Sync { + /// Small, complete, always fits. Identity + extent + how to address it. + fn header(&self) -> ContentHeader; + + /// Dereference. The SOURCE decides what a range means for its own content — + /// lines for a file, entries for a listing, hunks for a diff, a page for a board. + fn fetch(&self, range: Range) -> Result; +} +``` + +Consequences that fall out for free, none of them special-cased: + +- **Nothing is ever malformed.** The source hands out its own content in its own units. + A listing yields whole entries; a file yields whole lines with their true numbers. +- **The window stops being the constraint on truth.** A 40k-line file is fully available + to a citizen on a 4k window — she pages. Content size and context size decouple. +- **Indivisible payloads refuse, and say how to narrow.** A diff's `fetch` declines a + partial range because *its implementation* knows that; no central policy table. +- **It works across the grid unchanged.** A handle to a peer's artifact is the same + interface as a local one — which is why this belongs to the substrate, not to cognition. +- **Receipts become retrievable.** The act-history gap ([[act-results-need-a-recency-channel-not-semantic-recall]], + tasks #390/#414 — 2,863 acts executed, one visible, no path to the rest) is a handle + problem: the results still exist, nothing hands her a reference to them. + +## Reduction, if it happens at all, is the producer's + +Reduction is the fallback, not the design. Where a header genuinely must summarize, the +*producer* summarizes, because only it can do so truthfully. The substrate's contribution +is to never invent one. + +The better move is upstream of that: the caller **asks for less**. `code/read` takes a +line range, `code/list` a path, `code/search` a pattern. Planning the ask in advance beats +repairing the answer afterwards — and a handle makes asking again cheap. + +## Build order + +1. **Reconcile the two handle models** (task #17) — `runtime::cell_shapes::HandleRef` and + the URI-handle form. One type, or this design forks on day one. +2. **`ContentSource` in the substrate**, with the file engine as outlier A (coordinates + matter, refuses to be cut) and a RAG source as outlier B (maximally different: no + coordinates, already a projection). Per CLAUDE.md's outlier rule — if both fit without + forcing, the interface is proven. +3. **`ToolResult` carries `header + handle`** instead of a `String` body. +4. **A dereference verb** in the persona tool surface, so a citizen can page a handle. +5. **Delete the reducers.** The recency fold and the recent-results tail-keep both go; + with handles there is nothing left for them to do. + +## Forbidden moves + +- Any code path that inspects a payload's bytes to decide how to shrink it. +- Any new size fraction expressed as a bare denominator over the window. +- Any reduction performed by a consumer rather than the producer. +- A second handle type. There is one, and #17 is where it gets settled. From e9c310d0f5f60a25b4a7c9e048bdba6bfeec2d36 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 09:31:51 -0500 Subject: [PATCH 36/80] =?UTF-8?q?feat(content):=20ContentSource=20?= =?UTF-8?q?=E2=80=94=20oversized=20content=20stays=20put=20and=20travels?= =?UTF-8?q?=20as=20a=20HANDLE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The substrate half of docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md. Nothing is wired to it yet; this is the interface plus the registry plus the three proving implementations, all unit-tested. WHAT IT REPLACES. A reducer that inspects content and cuts it down. That failed live 2026-08-18: a citizen mid-SWE-bench was handed her own directory listing as `bytes":22592},{"kind":"file"…` — cut mid-token, head gone, nothing saying anything had been removed, on a turn using 11,763 tokens of a 29,184 window. Cutting bytes breaks JSON; cutting entries breaks a diff; cutting lines shifts the line numbers the next edit depends on. Those differ by what the content MEANS, which only its producer knows — so no function at the consumer end can be written correctly, and I wrote three of them before accepting that. THE INTERFACE. `ContentSource: header() -> ContentHeader, fetch(Span) -> Slice`. A caller holds `Arc` and calls its method; it never asks what kind of content it has. Polymorphism in place of inspection — same `cv::Algorithm` shape as search/vision/audio. The header is what travels: kind, one-line summary, extent IN THE SOURCE'S OWN UNITS (lines / entries / whole), and `fetch_with` — the exact call, written by the source, so a consumer never guesses a parameter name. A `Slice` carries whole units plus `next`, which is how a reader learns she has reached the END — the one thing a truncated copy can never tell her, and why a citizen reasons about a fragment as though it were the whole. THREE IMPLEMENTATIONS, chosen as outliers per CLAUDE.md's rule (build A and the most DIFFERENT N; if both fit without forcing, the interface is proven): A. TextContent — line-addressed, coordinates load-bearing. A slice from line 270 reports from=270, so an edit built on it targets the right place. Pinned by a test. B. ListingContent — entry-addressed, no coordinates, entries independent. Maximally different from A. This is the exact shape the char-cutter corrupted. C. WholeContent — indivisible. Its `fetch` refuses a partial read IN THE PRODUCER'S OWN WORDS and names the remedy, because the producer is the only party that knows why. This is the case that motivated the design: half a code fix applies cleanly and does the wrong thing, so a partial read is worse than no read. REGISTRY. `publish` parks a source and mints a `HandleRef`; `fetch(uuid, span)` derefs; `release` drops it. A released handle answers "handle not found — re-run the call that produced it", per the HandleRef lifetime contract, never a panic. TASK #17 COLLAPSES, and not the way it was written. It asks to reconcile "URI-Handle vs HandleRef envelope" — but only ONE of those is a live type. `HandleRef` already carries owner + UUID + type_tag, `CommandResult::Handle` already exists and has four consumers, and the grid routing is already specified (a handle minted on machine A routes the call back to A). So a handle to a peer's content is the same interface as a local one, for free, and there was no second model to reconcile. 12 tests green incl. ts-rs exports. Next: the `content/fetch` verb so a citizen can page a handle, then the act seam mints handles instead of cutting strings, then the two reducers get deleted. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/content/mod.rs | 458 +++++++++++++++++++ core/continuum-core/src/lib.rs | 1 + protocol/typescript/content/ContentHeader.ts | 30 ++ protocol/typescript/content/Extent.ts | 6 + protocol/typescript/content/Slice.ts | 21 + protocol/typescript/content/Span.ts | 15 + 6 files changed, 531 insertions(+) create mode 100644 core/continuum-core/src/content/mod.rs create mode 100644 protocol/typescript/content/ContentHeader.ts create mode 100644 protocol/typescript/content/Extent.ts create mode 100644 protocol/typescript/content/Slice.ts create mode 100644 protocol/typescript/content/Span.ts diff --git a/core/continuum-core/src/content/mod.rs b/core/continuum-core/src/content/mod.rs new file mode 100644 index 000000000..71da45c3b --- /dev/null +++ b/core/continuum-core/src/content/mod.rs @@ -0,0 +1,458 @@ +//! Content that is too big to hand over stays where it is, and travels as a HANDLE. +//! +//! See [docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md]. The rule: a consumer never +//! receives a cut-down copy of something, it receives a small honest header plus a +//! reference, and calls the reference for more. +//! +//! # Why a trait and not a function +//! +//! The alternative — a reducer that inspects content and shrinks it — cannot work, and +//! failed live on 2026-08-18: a citizen mid-SWE-bench was handed her own directory listing +//! as `bytes":22592},{"kind":"file"…`, cut mid-token, with nothing saying anything had been +//! removed. Cutting bytes breaks JSON; cutting entries breaks a diff; cutting lines shifts +//! the line numbers the next edit depends on. The distinction between those is not in the +//! bytes, it is in what the content MEANS, and only its producer knows that. +//! +//! So the producer implements [`ContentSource`] and nobody else decides anything. A caller +//! holds a reference and calls its method; it never asks what kind of content it has. +//! Polymorphism in place of inspection — the same `cv::Algorithm` shape the rest of this +//! codebase uses for search, vision and audio. +//! +//! # What this buys, none of it special-cased +//! +//! - **Nothing is ever malformed.** A source hands out its own content in its own units. +//! - **Content size decouples from context size.** A 40k-line file is fully available to a +//! citizen on a 4k window; she pages. The window stops bounding what is TRUE. +//! - **Indivisible content refuses**, in its own words, naming the narrowing — because its +//! own `fetch` knows it is indivisible. No central policy table. +//! - **The grid is free.** [`HandleRef`] already routes a call back to the machine that +//! minted it, so a handle to a peer's content is the same interface as a local one. + +use std::collections::HashMap; +use std::sync::{Arc, Mutex, OnceLock}; + +use serde::{Deserialize, Serialize}; +use ts_rs::TS; +use uuid::Uuid; + +use crate::runtime::cell_shapes::HandleRef; + +/// The owner module every content handle routes back through — the `owner` field of the +/// minted [`HandleRef`], and the command prefix that dereferences it. +pub const CONTENT_OWNER: &str = "content"; + +/// What a piece of content IS, small enough to always fit in a prompt. +/// +/// This is what a consumer gets instead of the content. It must be sufficient to decide +/// whether to fetch more and how — so it names the extent in the source's OWN units +/// (lines, entries, bytes) rather than a byte count nobody can act on. +#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] +#[ts(export, export_to = "../../../protocol/typescript/content/ContentHeader.ts")] +pub struct ContentHeader { + /// What this is, in the producer's words: `"file"`, `"directory listing"`, + /// `"test run output"`. Free text on purpose — a closed enum here would be a central + /// list every new source has to be added to. + pub kind: String, + /// One line a reader can act on: `"18 files under astropy/io/fits"`. + pub summary: String, + /// How much there is, in the source's own units. + pub extent: Extent, + /// The exact call that fetches more. Stated by the SOURCE so a consumer never has to + /// guess the parameter name — the difference between a usable refusal and a burnt turn. + pub fetch_with: String, +} + +/// How much content there is, counted the way its own source counts it. +#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] +#[ts(export, export_to = "../../../protocol/typescript/content/Extent.ts")] +pub enum Extent { + /// Line-addressed (a file, a log). 1-based, inclusive — the convention every editor + /// and every `code/edit` call already uses. + Lines { total: usize }, + /// Entry-addressed (a listing, a board, search hits). + Entries { total: usize }, + /// Not divisible at any granularity — a patch, an image, a computed answer. A `fetch` + /// on this either returns the whole thing or refuses. + Whole { bytes: usize }, +} + +/// A request for part of a source's content, in that source's units. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, TS, PartialEq, Eq)] +#[ts(export, export_to = "../../../protocol/typescript/content/Span.ts")] +pub struct Span { + /// First unit wanted, 1-based. + pub from: usize, + /// How many units. The source clamps to what it has and REPORTS the clamp — it never + /// silently returns less than asked without saying so. + pub count: usize, +} + +/// Part of a source's content, plus where the reader is in it. +#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] +#[ts(export, export_to = "../../../protocol/typescript/content/Slice.ts")] +pub struct Slice { + /// The content itself, whole in its own units — never cut mid-unit. + pub body: String, + /// What this slice covers, after clamping. + pub covered: Span, + /// The next span, when there is more. `None` means the reader has reached the end — + /// which is how she knows she has seen everything, a thing a truncated copy can never + /// tell her. + pub next: Option, +} + +/// Something that produced content and can still be asked about it. +/// +/// Implemented BY the producer. Callers hold `Arc` and never learn +/// which implementation they have. +pub trait ContentSource: Send + Sync { + /// Small, complete, always fits. + fn header(&self) -> ContentHeader; + + /// Dereference. The source decides what a span MEANS for its own content, and refuses + /// in its own words when its content cannot be divided. + fn fetch(&self, span: Span) -> Result; +} + +/// Live content sources, keyed by the UUID inside their [`HandleRef`]. +/// +/// Process-global because a handle minted during one command must be dereferenceable by a +/// later, unrelated command — that is the whole point of a handle. Lifetime is the +/// producer's per the `HandleRef` contract; a dropped entry yields a typed "handle not +/// found" rather than a panic. +static REGISTRY: OnceLock>>> = OnceLock::new(); + +fn registry() -> &'static Mutex>> { + REGISTRY.get_or_init(|| Mutex::new(HashMap::new())) +} + +/// Park a source and mint the handle that reaches it. The header is returned with the +/// handle because a consumer needs both in the same breath: what this is, and how to get +/// more of it. +pub fn publish(source: Arc) -> (HandleRef, ContentHeader) { + let header = source.header(); + let id = Uuid::new_v4(); + registry().lock().expect("content registry").insert(id, source); + ( + HandleRef::with_id(CONTENT_OWNER, id, "content::ContentSource"), + header, + ) +} + +/// Dereference a handle. `Err` when the producer has released it — the honest answer, and +/// the one the `HandleRef` contract specifies. +pub fn fetch(id: Uuid, span: Span) -> Result { + let source = { + let reg = registry().lock().expect("content registry"); + reg.get(&id).cloned() + }; + match source { + Some(s) => s.fetch(span), + None => Err(format!( + "handle not found: {id} — its producer has released it. Re-run the call that \ + produced it to get a fresh handle." + )), + } +} + +/// Release a source. Producers call this when their state goes away. +pub fn release(id: Uuid) -> bool { + registry() + .lock() + .expect("content registry") + .remove(&id) + .is_some() +} + +// --------------------------------------------------------------------------- +// Outlier A — line-addressed text. +// --------------------------------------------------------------------------- + +/// A body of text addressed by LINE, with the line numbers preserved exactly. +/// +/// The first of the two proving implementations (CLAUDE.md's outlier rule): coordinates +/// are load-bearing here, which is precisely what a generic cutter destroys. A slice from +/// line 400 reports that it starts at 400, so a `code/edit` built from it targets the +/// right place. +pub struct TextContent { + kind: String, + summary: String, + lines: Vec, +} + +impl TextContent { + pub fn new(kind: impl Into, summary: impl Into, body: &str) -> Self { + Self { + kind: kind.into(), + summary: summary.into(), + lines: body.lines().map(str::to_string).collect(), + } + } +} + +impl ContentSource for TextContent { + fn header(&self) -> ContentHeader { + ContentHeader { + kind: self.kind.clone(), + summary: self.summary.clone(), + extent: Extent::Lines { + total: self.lines.len(), + }, + fetch_with: "content/fetch(handle, from=, count=)".to_string(), + } + } + + fn fetch(&self, span: Span) -> Result { + let total = self.lines.len(); + if span.from == 0 || span.from > total { + return Err(format!( + "line {} is outside this content (1..{total}) — ask within range", + span.from + )); + } + let start = span.from - 1; + let end = (start + span.count).min(total); + let covered = Span { + from: span.from, + count: end - start, + }; + Ok(Slice { + body: self.lines[start..end].join("\n"), + covered, + next: (end < total).then_some(Span { + from: end + 1, + count: span.count, + }), + }) + } +} + +// --------------------------------------------------------------------------- +// Outlier B — entry-addressed listing. Maximally different from A: no coordinates, +// entries are independent, and each is already a complete thing. +// --------------------------------------------------------------------------- + +/// A list of independently meaningful entries — a directory listing, search hits, a board. +/// +/// The interface must fit this WITHOUT forcing, or it is the wrong interface. It does: +/// `Extent::Entries` counts entries, a span means entries, and a slice is whole entries. +/// The generic char-cutter this replaces produced `bytes":22592},{"kind":"file"` here. +pub struct ListingContent { + kind: String, + summary: String, + entries: Vec, +} + +impl ListingContent { + pub fn new( + kind: impl Into, + summary: impl Into, + entries: Vec, + ) -> Self { + Self { + kind: kind.into(), + summary: summary.into(), + entries, + } + } +} + +impl ContentSource for ListingContent { + fn header(&self) -> ContentHeader { + ContentHeader { + kind: self.kind.clone(), + summary: self.summary.clone(), + extent: Extent::Entries { + total: self.entries.len(), + }, + fetch_with: "content/fetch(handle, from=, count=)".to_string(), + } + } + + fn fetch(&self, span: Span) -> Result { + let total = self.entries.len(); + if span.from == 0 || span.from > total { + return Err(format!( + "entry {} is outside this listing (1..{total}) — ask within range", + span.from + )); + } + let start = span.from - 1; + let end = (start + span.count).min(total); + Ok(Slice { + body: self.entries[start..end].join("\n"), + covered: Span { + from: span.from, + count: end - start, + }, + next: (end < total).then_some(Span { + from: end + 1, + count: span.count, + }), + }) + } +} + +// --------------------------------------------------------------------------- +// Outlier C — indivisible. Not a third shape so much as the REFUSAL the other two +// prove is expressible: a patch that would be corrupted by any partial read. +// --------------------------------------------------------------------------- + +/// Content whose parts are not independently valid — a patch, a diff, an image. +/// +/// `fetch` returns the whole thing or refuses, and the refusal is written by the producer +/// because the producer is the only party that knows WHY. This is the case that motivated +/// the whole design: half a code fix applies cleanly and does the wrong thing, so a +/// partial read is worse than no read. +pub struct WholeContent { + kind: String, + summary: String, + body: String, + narrow_with: String, +} + +impl WholeContent { + pub fn new( + kind: impl Into, + summary: impl Into, + body: impl Into, + narrow_with: impl Into, + ) -> Self { + Self { + kind: kind.into(), + summary: summary.into(), + body: body.into(), + narrow_with: narrow_with.into(), + } + } +} + +impl ContentSource for WholeContent { + fn header(&self) -> ContentHeader { + ContentHeader { + kind: self.kind.clone(), + summary: self.summary.clone(), + extent: Extent::Whole { + bytes: self.body.len(), + }, + fetch_with: self.narrow_with.clone(), + } + } + + fn fetch(&self, span: Span) -> Result { + // from=1, count>=1 means "give me the whole thing" — the only division this + // content admits. + if span.from != 1 { + return Err(format!( + "this {} cannot be read in parts — a partial copy would look valid and be \ + wrong. {}", + self.kind, self.narrow_with + )); + } + Ok(Slice { + body: self.body.clone(), + covered: Span { from: 1, count: 1 }, + next: None, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + // what this catches: THE bug this module exists for. An oversized listing must reach a + // consumer as WHOLE entries with a true total, never as a byte fragment. The live + // failure was `bytes":22592},{"kind":"file"` — an entry cut mid-key. + #[test] + fn a_listing_yields_whole_entries_and_a_true_total() { + let entries: Vec = (0..50).map(|i| format!("file_{i}.py 22592 bytes")).collect(); + let src = ListingContent::new("directory listing", "50 files under io/fits", entries); + + assert_eq!(src.header().extent, Extent::Entries { total: 50 }); + + let slice = src.fetch(Span { from: 1, count: 6 }).expect("fetch"); + assert_eq!(slice.covered.count, 6); + for line in slice.body.lines() { + assert!( + line.starts_with("file_") && line.ends_with("bytes"), + "every entry must be WHOLE — a half-entry is the bug: {line:?}" + ); + } + assert_eq!( + slice.next, + Some(Span { from: 7, count: 6 }), + "the reader must be told there is more AND exactly how to ask for it" + ); + } + + // what this catches: line numbers drifting. This is what makes a generic cutter + // dangerous rather than merely lossy — a slice that renumbers its lines produces an + // edit that targets the wrong place with right-looking coordinates. + #[test] + fn a_text_slice_preserves_its_absolute_line_numbers() { + let body: String = (1..=1000).map(|i| format!("line {i}\n")).collect(); + let src = TextContent::new("file", "sympify.py, 1000 lines", &body); + + let slice = src.fetch(Span { from: 270, count: 3 }).expect("fetch"); + assert_eq!(slice.covered.from, 270, "the slice reports where it STARTS"); + assert_eq!( + slice.body, "line 270\nline 271\nline 272", + "content at 270 is the content at 270, not renumbered from 1" + ); + } + + // what this catches: the end of content being indistinguishable from a truncation. + // `next: None` is how a reader knows she has seen everything — the one thing a cut-down + // copy can never tell her, and the reason a citizen reasons about a fragment as if it + // were whole. + #[test] + fn reaching_the_end_is_reported_as_the_end() { + let src = ListingContent::new("listing", "3 files", vec!["a".into(), "b".into(), "c".into()]); + let slice = src.fetch(Span { from: 1, count: 10 }).expect("fetch"); + assert_eq!(slice.covered.count, 3, "clamped to what exists"); + assert!(slice.next.is_none(), "and says there is no more"); + } + + // what this catches: an indivisible payload being silently divided. The producer — not + // the substrate — refuses, and the refusal names the remedy so the turn is not burnt. + #[test] + fn indivisible_content_refuses_a_partial_read_in_its_own_words() { + let src = WholeContent::new( + "patch", + "fix for sympy-18057, 42 lines", + "--- a/x\n+++ b/x\n", + "Apply it whole, or request a smaller change.", + ); + let err = src.fetch(Span { from: 2, count: 1 }).expect_err("must refuse"); + assert!(err.contains("cannot be read in parts"), "{err}"); + assert!(err.contains("Apply it whole"), "names the remedy: {err}"); + // ...and asking for the whole thing works. + assert!(src.fetch(Span { from: 1, count: 1 }).is_ok()); + } + + // what this catches: the registry round trip — publish, deref through the HandleRef's + // UUID, release. This is the seam that makes content survive PAST the command that + // produced it, which is the entire difference between a handle and a return value. + #[test] + fn a_published_source_is_reachable_by_its_handle_and_gone_after_release() { + let (handle, header) = publish(Arc::new(ListingContent::new( + "listing", + "2 files", + vec!["one".into(), "two".into()], + ))); + assert_eq!(handle.owner, CONTENT_OWNER, "routes back to the content module"); + assert_eq!(header.extent, Extent::Entries { total: 2 }); + + let id: uuid::Uuid = handle.id.into(); + let slice = fetch(id, Span { from: 1, count: 1 }).expect("reachable by handle"); + assert_eq!(slice.body, "one"); + + assert!(release(id)); + let err = fetch(id, Span { from: 1, count: 1 }).expect_err("gone after release"); + assert!( + err.contains("handle not found") && err.contains("Re-run"), + "a released handle explains itself and names the recovery: {err}" + ); + } +} diff --git a/core/continuum-core/src/lib.rs b/core/continuum-core/src/lib.rs index 41ae32a4f..3396d9ac5 100644 --- a/core/continuum-core/src/lib.rs +++ b/core/continuum-core/src/lib.rs @@ -29,6 +29,7 @@ pub mod airc; pub mod audio_constants; pub mod capacity; pub mod code; +pub mod content; pub mod cognition; pub mod commands; pub mod comms; diff --git a/protocol/typescript/content/ContentHeader.ts b/protocol/typescript/content/ContentHeader.ts new file mode 100644 index 000000000..e1c9f957f --- /dev/null +++ b/protocol/typescript/content/ContentHeader.ts @@ -0,0 +1,30 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { Extent } from "./Extent"; + +/** + * What a piece of content IS, small enough to always fit in a prompt. + * + * This is what a consumer gets instead of the content. It must be sufficient to decide + * whether to fetch more and how — so it names the extent in the source's OWN units + * (lines, entries, bytes) rather than a byte count nobody can act on. + */ +export type ContentHeader = { +/** + * What this is, in the producer's words: `"file"`, `"directory listing"`, + * `"test run output"`. Free text on purpose — a closed enum here would be a central + * list every new source has to be added to. + */ +kind: string, +/** + * One line a reader can act on: `"18 files under astropy/io/fits"`. + */ +summary: string, +/** + * How much there is, in the source's own units. + */ +extent: Extent, +/** + * The exact call that fetches more. Stated by the SOURCE so a consumer never has to + * guess the parameter name — the difference between a usable refusal and a burnt turn. + */ +fetch_with: string, }; diff --git a/protocol/typescript/content/Extent.ts b/protocol/typescript/content/Extent.ts new file mode 100644 index 000000000..ac68e0a02 --- /dev/null +++ b/protocol/typescript/content/Extent.ts @@ -0,0 +1,6 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * How much content there is, counted the way its own source counts it. + */ +export type Extent = { "Lines": { total: number, } } | { "Entries": { total: number, } } | { "Whole": { bytes: number, } }; diff --git a/protocol/typescript/content/Slice.ts b/protocol/typescript/content/Slice.ts new file mode 100644 index 000000000..00144f34d --- /dev/null +++ b/protocol/typescript/content/Slice.ts @@ -0,0 +1,21 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { Span } from "./Span"; + +/** + * Part of a source's content, plus where the reader is in it. + */ +export type Slice = { +/** + * The content itself, whole in its own units — never cut mid-unit. + */ +body: string, +/** + * What this slice covers, after clamping. + */ +covered: Span, +/** + * The next span, when there is more. `None` means the reader has reached the end — + * which is how she knows she has seen everything, a thing a truncated copy can never + * tell her. + */ +next: Span | null, }; diff --git a/protocol/typescript/content/Span.ts b/protocol/typescript/content/Span.ts new file mode 100644 index 000000000..0fcc411d3 --- /dev/null +++ b/protocol/typescript/content/Span.ts @@ -0,0 +1,15 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * A request for part of a source's content, in that source's units. + */ +export type Span = { +/** + * First unit wanted, 1-based. + */ +from: number, +/** + * How many units. The source clamps to what it has and REPORTS the clamp — it never + * silently returns less than asked without saying so. + */ +count: number, }; From 6958293f8fc5d64f0a7c7153caae9c1c906ae693 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 09:50:47 -0500 Subject: [PATCH 37/80] =?UTF-8?q?feat(content):=20content/fetch=20+=20Cont?= =?UTF-8?q?entModule=20=E2=80=94=20the=20handle=20is=20now=20callable?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Makes the handle real. Slice 1 shipped the interface; a handle nobody can dereference is identical to content never received, so this is the half that matters. `content/fetch(handle, from, count)` — NATIVE + AiSafe, on the bounded tool surface for the same reason `code/read` is: it is a core act. It decides NOTHING. It looks the source up and calls its method; whether a span means lines, entries, or "whole or nothing" is the source's answer, and a refusal is passed through in the PRODUCER'S OWN WORDS rather than re-worded by a layer that doesn't know why. `nextFrom` is the field that earns the design: `null` means you have read everything. That is the one thing a truncated copy can never tell you, and the reason a citizen reasons about a fragment as though it were whole. OOP, per Joel: the registry became an OBJECT, not free functions over a static. `ContentModule` owns the `Arc` and contributes the command that reads it — exactly how `GpuModule` owns `GpuMemoryManager`. The command takes it as a dependency like every other command takes its own, so the seam is injectable and its test constructs a private registry instead of touching process state. `content::global()` still exists because producers scattered across the tree need somewhere to publish INTO, but nothing is forced to reach for it, and the one consumer doesn't. Registered at boot beside GpuModule (ipc/mod.rs). Verified reachable rather than merely compiled: the 235-test module-wiring audit is green, and the command's own test pins NAME + ACCESS + NATIVE so a future edit cannot quietly drop it off the surface and turn every handle into a dead end. DOC: added the positron convergence, which is Joel's and is the better half of the idea. `ContentSource` and a positron `ViewState` are the same shape — one truth at the source, N projections, consumer reads at its own rate. So a citizen gets a CHOICE she makes herself: observe the ambient projection, or reach for the handle when something warrants it. That is how a person works — you don't read every line of a repo, you carry a summary and drill in when something looks wrong. The condensing stops being a budget mechanism we impose and becomes the citizen choosing her own resolution; it is correct rather than lossy, because the detail is one call away AND SHE KNOWS IT. Minutia skipped, never destroyed. Which settles what "too big to send" means: nothing is. There is a view and there is a way in. The window sizes her ATTENTION, not the truth available to her. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/content/fetch.rs | 151 ++++++++++++++++++ .../src/commands/content/mod.rs | 19 +++ core/continuum-core/src/commands/mod.rs | 1 + core/continuum-core/src/content/mod.rs | 104 ++++++------ core/continuum-core/src/ipc/mod.rs | 5 + core/continuum-core/src/modules/content.rs | 67 ++++++++ core/continuum-core/src/modules/mod.rs | 1 + .../architecture/CONTENT-TRAVELS-BY-HANDLE.md | 29 ++++ .../typescript/content/ContentFetchParams.ts | 16 ++ .../typescript/content/ContentFetchResult.ts | 20 +++ 10 files changed, 368 insertions(+), 45 deletions(-) create mode 100644 core/continuum-core/src/commands/content/fetch.rs create mode 100644 core/continuum-core/src/commands/content/mod.rs create mode 100644 core/continuum-core/src/modules/content.rs create mode 100644 protocol/typescript/content/ContentFetchParams.ts create mode 100644 protocol/typescript/content/ContentFetchResult.ts diff --git a/core/continuum-core/src/commands/content/fetch.rs b/core/continuum-core/src/commands/content/fetch.rs new file mode 100644 index 000000000..529ff5ba7 --- /dev/null +++ b/core/continuum-core/src/commands/content/fetch.rs @@ -0,0 +1,151 @@ +//! `content/fetch` — read part of content held behind a handle. +//! +//! The dereference half of [`crate::content`]. When something is too large to hand over +//! whole, its producer parks it and returns a header plus a handle; this is how a citizen +//! then reads it, at whatever pace her window allows. +//! +//! `NATIVE` because it is useless otherwise: a handle she cannot call is the same as +//! content she never received. It joins the bounded native tool surface for the same +//! reason `code/read` does — it is a core act, not a catalog curiosity. +//! +//! This command decides NOTHING about the content. It looks the source up and calls its +//! method; whether a span means lines, entries, or "the whole thing or nothing" is the +//! source's answer, and a refusal here is written by the producer that knows why. + +use uuid::Uuid; + +use std::sync::Arc; + +use crate::content::{ContentRegistry, Span}; +use crate::sdk_codegen::CommandError; + +#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, ts_rs::TS, schemars::JsonSchema)] +#[ts( + export, + export_to = "../../../protocol/typescript/content/ContentFetchParams.ts" +)] +#[serde(rename_all = "camelCase")] +pub struct ContentFetchParams { + /// The handle's id, from the header you were given. + #[ts(type = "string")] + pub handle: Uuid, + /// First unit to read, 1-based, in the units the header named (line, entry). + /// Defaults to the beginning. + #[serde(default = "default_from")] + pub from: usize, + /// How many units. The source clamps to what it has and tells you what it covered. + #[serde(default = "default_count")] + pub count: usize, +} + +/// Start at the beginning — the overwhelmingly common first call, and a 0 here would be +/// out of range in 1-based units. +fn default_from() -> usize { + 1 +} + +/// A page, when the caller does not say. Not a context bound — the caller's window decides +/// how much she asks for, and this is only the value used when she asks for none. +fn default_count() -> usize { + 100 +} + +#[derive(Debug, Clone, serde::Serialize, ts_rs::TS)] +#[ts( + export, + export_to = "../../../protocol/typescript/content/ContentFetchResult.ts" +)] +pub struct ContentFetchResult { + /// The content, whole in its own units. + pub body: String, + /// First unit this covers, after clamping. + pub from: usize, + /// How many units this covers, after clamping. + pub count: usize, + /// Where to continue, or `null` when you have reached the end. `null` is the signal + /// that you have seen ALL of it — the thing a truncated copy can never tell you. + #[ts(optional)] + pub next_from: Option, +} + +crate::action_command! { + /// Read part of content held behind a handle. Use the handle and units from the + /// header you were given (`from`/`count` in lines or entries). `nextFrom` tells you + /// where to continue; when it is absent you have read everything. + pub struct ContentFetch { registry: Arc } + name: "content/fetch", + access: AiSafe, + native: true, + params: ContentFetchParams, + output: ContentFetchResult, + run(this, _ctx, p) => { + if p.count == 0 { + return Err(CommandError::Invalid( + "count must be at least 1 — ask for the units you want to read".to_string(), + )); + } + let slice = this.registry.fetch(p.handle, Span { from: p.from, count: p.count }) + // The source's own words: a released handle, an out-of-range ask, or an + // indivisible payload refusing a partial read. Never re-worded here — the + // producer is the party that knows why, and its message names the remedy. + .map_err(CommandError::Invalid)?; + Ok(ContentFetchResult { + body: slice.body, + from: slice.covered.from, + count: slice.covered.count, + next_from: slice.next.map(|n| n.from), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::content::ListingContent; + use crate::sdk_codegen::{AccessLevel, ActionCommand}; + + // what this catches: the verb falling off the NATIVE surface. A handle a citizen + // cannot call is identical to content she never received — this command is the ONLY + // way to dereference, so catalog-only registration would silently make every handle + // a dead end. + #[test] + fn it_is_native_and_ai_safe_because_a_handle_she_cannot_call_is_useless() { + assert_eq!(ContentFetch::NAME, "content/fetch"); + assert_eq!(ContentFetch::ACCESS, AccessLevel::AiSafe); + assert!( + ContentFetch::NATIVE, + "must be on the native surface — it is the only way to read a handle" + ); + } + + // what this catches: the end-of-content signal getting lost in the wire type. `next` + // is how a reader learns she has seen ALL of it; if it never surfaces as `nextFrom` + // she cannot distinguish "that's everything" from "that's the part you were given", + // which is exactly the confusion this whole design exists to end. + #[tokio::test] + async fn the_result_carries_where_to_continue_and_where_to_stop() { + let registry = Arc::new(ContentRegistry::default()); + let (handle, _) = registry.publish(Arc::new(ListingContent::new( + "listing", + "3 files", + vec!["a".into(), "b".into(), "c".into()], + ))); + let id: Uuid = handle.id.into(); + let cmd = ContentFetch { registry }; + let ctx = crate::sdk_codegen::Ctx::default(); + + let page = cmd + .run(&ctx, ContentFetchParams { handle: id, from: 1, count: 2 }) + .await + .expect("first page"); + assert_eq!(page.body, "a\nb"); + assert_eq!(page.next_from, Some(3), "says exactly where to continue"); + + let last = cmd + .run(&ctx, ContentFetchParams { handle: id, from: 3, count: 2 }) + .await + .expect("last page"); + assert_eq!(last.count, 1, "clamped to what exists"); + assert!(last.next_from.is_none(), "and reports the end as the end"); + } +} diff --git a/core/continuum-core/src/commands/content/mod.rs b/core/continuum-core/src/commands/content/mod.rs new file mode 100644 index 000000000..ff076132d --- /dev/null +++ b/core/continuum-core/src/commands/content/mod.rs @@ -0,0 +1,19 @@ +//! `content/*` — dereferencing content that stayed at its source. +//! +//! See [docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md] and [`crate::content`]. +//! Oversized content is never cut down and handed over; it is parked by its producer and +//! reached through a handle. This module is where a citizen calls that handle. + +pub mod fetch; + +use std::sync::Arc; + +use crate::content::ContentRegistry; +use crate::sdk_codegen::DynCommand; + +/// The dep-holding `content/*` command objects +/// [`ContentModule`](crate::modules::content::ContentModule) contributes, sharing the one +/// [`ContentRegistry`] every producer publishes into. +pub fn command_objects(registry: Arc) -> Vec> { + vec![Arc::new(fetch::ContentFetch { registry })] +} diff --git a/core/continuum-core/src/commands/mod.rs b/core/continuum-core/src/commands/mod.rs index 7879bd337..27a1ceaea 100644 --- a/core/continuum-core/src/commands/mod.rs +++ b/core/continuum-core/src/commands/mod.rs @@ -21,6 +21,7 @@ pub mod capacity; pub mod catalog; pub mod chat; pub mod code; +pub mod content; pub mod cognition; pub mod command; pub mod data; diff --git a/core/continuum-core/src/content/mod.rs b/core/continuum-core/src/content/mod.rs index 71da45c3b..4675beb26 100644 --- a/core/continuum-core/src/content/mod.rs +++ b/core/continuum-core/src/content/mod.rs @@ -114,54 +114,67 @@ pub trait ContentSource: Send + Sync { fn fetch(&self, span: Span) -> Result; } -/// Live content sources, keyed by the UUID inside their [`HandleRef`]. +/// The live content sources, keyed by the UUID inside their [`HandleRef`]. /// -/// Process-global because a handle minted during one command must be dereferenceable by a -/// later, unrelated command — that is the whole point of a handle. Lifetime is the -/// producer's per the `HandleRef` contract; a dropped entry yields a typed "handle not -/// found" rather than a panic. -static REGISTRY: OnceLock>>> = OnceLock::new(); - -fn registry() -> &'static Mutex>> { - REGISTRY.get_or_init(|| Mutex::new(HashMap::new())) +/// An OBJECT, not a bag of free functions over a static: the command that dereferences a +/// handle holds an `Arc` like every other command holds its dependency, +/// so the seam is injectable and testable. [`global`] exists because producers scattered +/// across the tree need somewhere to publish INTO — but nothing is forced to reach for it. +/// +/// Lifetime is the producer's, per the [`HandleRef`] contract: a dropped entry yields a +/// typed "handle not found" rather than a panic. +#[derive(Default)] +pub struct ContentRegistry { + sources: Mutex>>, } -/// Park a source and mint the handle that reaches it. The header is returned with the -/// handle because a consumer needs both in the same breath: what this is, and how to get -/// more of it. -pub fn publish(source: Arc) -> (HandleRef, ContentHeader) { - let header = source.header(); - let id = Uuid::new_v4(); - registry().lock().expect("content registry").insert(id, source); - ( - HandleRef::with_id(CONTENT_OWNER, id, "content::ContentSource"), - header, - ) -} +impl ContentRegistry { + /// Park a source and mint the handle that reaches it. The header comes back with the + /// handle because a consumer needs both in one breath: what this is, and how to get + /// more of it. + pub fn publish(&self, source: Arc) -> (HandleRef, ContentHeader) { + let header = source.header(); + let id = Uuid::new_v4(); + self.sources.lock().expect("content registry").insert(id, source); + ( + HandleRef::with_id(CONTENT_OWNER, id, "content::ContentSource"), + header, + ) + } + + /// Dereference. `Err` when the producer has released it — the honest answer the + /// `HandleRef` contract specifies, naming the recovery. + pub fn fetch(&self, id: Uuid, span: Span) -> Result { + let source = { + let reg = self.sources.lock().expect("content registry"); + reg.get(&id).cloned() + }; + match source { + Some(s) => s.fetch(span), + None => Err(format!( + "handle not found: {id} — its producer has released it. Re-run the call \ + that produced it to get a fresh handle." + )), + } + } -/// Dereference a handle. `Err` when the producer has released it — the honest answer, and -/// the one the `HandleRef` contract specifies. -pub fn fetch(id: Uuid, span: Span) -> Result { - let source = { - let reg = registry().lock().expect("content registry"); - reg.get(&id).cloned() - }; - match source { - Some(s) => s.fetch(span), - None => Err(format!( - "handle not found: {id} — its producer has released it. Re-run the call that \ - produced it to get a fresh handle." - )), + /// Release a source. Producers call this when their state goes away. + pub fn release(&self, id: Uuid) -> bool { + self.sources + .lock() + .expect("content registry") + .remove(&id) + .is_some() } } -/// Release a source. Producers call this when their state goes away. -pub fn release(id: Uuid) -> bool { - registry() - .lock() - .expect("content registry") - .remove(&id) - .is_some() +/// The process-wide registry producers publish into. Consumers should take an +/// `Arc` dependency instead of calling this. +pub fn global() -> Arc { + static REGISTRY: OnceLock> = OnceLock::new(); + REGISTRY + .get_or_init(|| Arc::new(ContentRegistry::default())) + .clone() } // --------------------------------------------------------------------------- @@ -436,7 +449,8 @@ mod tests { // produced it, which is the entire difference between a handle and a return value. #[test] fn a_published_source_is_reachable_by_its_handle_and_gone_after_release() { - let (handle, header) = publish(Arc::new(ListingContent::new( + let reg = ContentRegistry::default(); + let (handle, header) = reg.publish(Arc::new(ListingContent::new( "listing", "2 files", vec!["one".into(), "two".into()], @@ -445,11 +459,11 @@ mod tests { assert_eq!(header.extent, Extent::Entries { total: 2 }); let id: uuid::Uuid = handle.id.into(); - let slice = fetch(id, Span { from: 1, count: 1 }).expect("reachable by handle"); + let slice = reg.fetch(id, Span { from: 1, count: 1 }).expect("reachable by handle"); assert_eq!(slice.body, "one"); - assert!(release(id)); - let err = fetch(id, Span { from: 1, count: 1 }).expect_err("gone after release"); + assert!(reg.release(id)); + let err = reg.fetch(id, Span { from: 1, count: 1 }).expect_err("gone after release"); assert!( err.contains("handle not found") && err.contains("Re-run"), "a released handle explains itself and names the recovery: {err}" diff --git a/core/continuum-core/src/ipc/mod.rs b/core/continuum-core/src/ipc/mod.rs index a0c286ca2..ed2a5e26a 100644 --- a/core/continuum-core/src/ipc/mod.rs +++ b/core/continuum-core/src/ipc/mod.rs @@ -1120,6 +1120,11 @@ pub fn start_server( // Phase 1: GpuModule (GPU stats + pressure IPC) runtime.register(Arc::new(GpuModule::new(gpu_manager.clone()))); + // Content handles: oversized results stay at their source and are paged through + // `content/fetch`. See docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md. + runtime.register(Arc::new( + crate::modules::content::ContentModule::new(crate::content::global()), + )); // ForgeModule (continuum#1164 Phase 4 stub — forge/run IPC). // v1 returns a stub ForgeArtifact from a recipe; Phase 5+ wires the diff --git a/core/continuum-core/src/modules/content.rs b/core/continuum-core/src/modules/content.rs new file mode 100644 index 000000000..8f7869d5b --- /dev/null +++ b/core/continuum-core/src/modules/content.rs @@ -0,0 +1,67 @@ +//! ContentModule — host for the content-handle surface. +//! +//! Owns the one [`ContentRegistry`] and hands it to the `content/*` verbs, exactly as +//! [`GpuModule`](crate::modules::gpu::GpuModule) owns its `GpuMemoryManager`. The module +//! owning the state and contributing the commands that read it is the pattern; nothing +//! here reaches for a global. +//! +//! It is deliberately thin. All the behaviour lives in the [`ContentSource`] implementations +//! at the producers — this module exists so a citizen's `content/fetch` call has a +//! registered home and so the registry has one owner. +//! +//! [`ContentSource`]: crate::content::ContentSource +//! [`ContentRegistry`]: crate::content::ContentRegistry + +use std::any::Any; +use std::sync::Arc; + +use async_trait::async_trait; +use serde_json::Value; + +use crate::content::ContentRegistry; +use crate::runtime::{CommandResult, ModuleConfig, ModuleContext, ModulePriority, ServiceModule}; + +pub struct ContentModule { + registry: Arc, +} + +impl ContentModule { + pub fn new(registry: Arc) -> Self { + Self { registry } + } +} + +#[async_trait] +impl ServiceModule for ContentModule { + fn config(&self) -> ModuleConfig { + ModuleConfig { + name: "content", + priority: ModulePriority::Normal, + command_prefixes: &["content/"], + event_subscriptions: &[], + needs_dedicated_thread: false, + max_concurrency: 0, + tick_interval: None, + } + } + + async fn initialize(&self, _ctx: &ModuleContext) -> Result<(), String> { + Ok(()) + } + + fn commands(&self) -> Vec> { + crate::commands::content::command_objects(self.registry.clone()) + } + + async fn handle_command(&self, command: &str, _params: Value) -> Result { + // Born on the typed registry — there is no legacy surface to fall back to, so a + // name reaching here is a routing defect and says so rather than failing quietly. + Err(format!( + "content command surface is typed-registry only; '{command}' has no handler" + )) + } + + fn as_any(&self) -> &dyn Any { + self + } +} diff --git a/core/continuum-core/src/modules/mod.rs b/core/continuum-core/src/modules/mod.rs index c2e634899..7cafa8529 100644 --- a/core/continuum-core/src/modules/mod.rs +++ b/core/continuum-core/src/modules/mod.rs @@ -32,6 +32,7 @@ pub mod channel; pub mod chat; pub mod code; pub mod code_commands; +pub mod content; pub mod cognition; pub mod data; pub mod dataset; diff --git a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md index 11f7f57ec..78d45cc86 100644 --- a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md +++ b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md @@ -100,6 +100,35 @@ The better move is upstream of that: the caller **asks for less**. `code/read` t line range, `code/list` a path, `code/search` a pattern. Planning the ask in advance beats repairing the answer afterwards — and a handle makes asking again cheap. +## The positron convergence — two ways to the same truth + +`ContentSource` and a positron `ViewState` are the same shape wearing different names: +one truth at the source, N projections, the consumer reading at its own rate. They must +not stay two mechanisms. + +What that gives a citizen is a choice she makes for herself, per situation: + +- **Observe the layer.** The positronic projection is the ambient, condensed view — the + board, the room, the run. She sees the shape of things without asking for anything, the + same way a person glances at a screen. +- **Reach for the handle.** When something in that view warrants it, she dereferences and + reads the detail at whatever depth the question needs. + +That is how a person actually works: you do not read every line of every file in a repo, +you carry a summary and drill in when something looks wrong. The condensing is not a +budget mechanism we impose — it falls out of the citizen choosing her own resolution, and +it is *correct* rather than lossy, because the detail is always one call away and she knows +it. Minutia is skipped, never destroyed. + +This also settles what "too big to send" means. Nothing is too big. There is a view, and +there is a way in. The window sizes her ATTENTION, not the truth available to her. + +Practically: a positron projection should be able to hand out handles into what it +summarizes (a board row → the card's full content; a run tile → the transcript), and a +`ContentSource` header is already the smallest possible projection. Converging them is the +next design step after the build order below — the same `(header, handle)` pair, whether +it arrives through a tool result or a rendered view. + ## Build order 1. **Reconcile the two handle models** (task #17) — `runtime::cell_shapes::HandleRef` and diff --git a/protocol/typescript/content/ContentFetchParams.ts b/protocol/typescript/content/ContentFetchParams.ts new file mode 100644 index 000000000..88346bc08 --- /dev/null +++ b/protocol/typescript/content/ContentFetchParams.ts @@ -0,0 +1,16 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +export type ContentFetchParams = { +/** + * The handle's id, from the header you were given. + */ +handle: string, +/** + * First unit to read, 1-based, in the units the header named (line, entry). + * Defaults to the beginning. + */ +from: number, +/** + * How many units. The source clamps to what it has and tells you what it covered. + */ +count: number, }; diff --git a/protocol/typescript/content/ContentFetchResult.ts b/protocol/typescript/content/ContentFetchResult.ts new file mode 100644 index 000000000..5c9ba5c7e --- /dev/null +++ b/protocol/typescript/content/ContentFetchResult.ts @@ -0,0 +1,20 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +export type ContentFetchResult = { +/** + * The content, whole in its own units. + */ +body: string, +/** + * First unit this covers, after clamping. + */ +from: number, +/** + * How many units this covers, after clamping. + */ +count: number, +/** + * Where to continue, or `null` when you have reached the end. `null` is the signal + * that you have seen ALL of it — the thing a truncated copy can never tell you. + */ +next_from?: number, }; From 2e6594dfb52c1ff7562960c17f03710eb6d2880a Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 09:52:52 -0500 Subject: [PATCH 38/80] =?UTF-8?q?fix(content):=20content/fetch=20leaves=20?= =?UTF-8?q?the=20native=20surface=20=E2=80=94=20the=20header=20names=20the?= =?UTF-8?q?=20call=20when=20it=20matters?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agentic-surface ratchet caught my own commit: declaring `content/fetch` NATIVE took the surface from 8,040 to 11,608 tokens against an 11,300 ceiling. The guard did exactly its job and the guard is right. I had reasoned "a handle she cannot call is useless, therefore native." The second half does not follow. NATIVE means a full schema in EVERY prompt; the catalog already lists every authorized command by name and `commands/help` expands it, so awareness was never the question. What native buys is RESIDENCY, and residency is wrong here: this verb is meaningful only on turns where she actually holds a handle, and paying for it on all the others is the #333 defect ("the tool surface is paid twice") with one more entry. It does not need residency because `ContentHeader::fetch_with` already names the exact call AT THE MOMENT a handle is issued — the producer tells her the call form precisely when it becomes relevant. AiSafe is what makes it callable, and that is unchanged. Which is this module's own principle applied one level up, and the reason this reads as a correction rather than a concession: do not hold the detail resident, carry the pointer and drill in when something warrants it. The tool surface deserves the same treatment as the content. Test inverted to pin the new invariant with the measured numbers in it, so a future edit that flips it back explains itself against a real ceiling instead of a preference. Ratchet green. 16 content tests green. Full lib suite was 7,261/7,262 with this as the only failure; it is now the only change since. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/content/fetch.rs | 36 +++++++++++++------ 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/core/continuum-core/src/commands/content/fetch.rs b/core/continuum-core/src/commands/content/fetch.rs index 529ff5ba7..4c981ed36 100644 --- a/core/continuum-core/src/commands/content/fetch.rs +++ b/core/continuum-core/src/commands/content/fetch.rs @@ -4,9 +4,22 @@ //! whole, its producer parks it and returns a header plus a handle; this is how a citizen //! then reads it, at whatever pace her window allows. //! -//! `NATIVE` because it is useless otherwise: a handle she cannot call is the same as -//! content she never received. It joins the bounded native tool surface for the same -//! reason `code/read` does — it is a core act, not a catalog curiosity. +//! ## Why this is NOT on the native surface +//! +//! It looked like a core act, so it shipped `NATIVE` — and the agentic-surface ratchet +//! immediately caught it: 8,040 → 11,608 tokens against an 11,300 ceiling. Paying for a +//! full schema in EVERY prompt is the #333 defect, and this verb is the wrong place to +//! spend it, because it is only meaningful on the turns where she actually holds a handle. +//! +//! It does not need to be resident, because [`ContentHeader::fetch_with`] names the exact +//! call AT THE MOMENT a handle is issued — the producer tells her the call form precisely +//! when it becomes relevant. She is aware of the verb regardless: the compact catalog +//! lists every authorized command by name, and `commands/help` expands this one on demand. +//! +//! Which is the same principle the module itself is built on, applied one level up: do not +//! hold the detail resident, carry the pointer and drill in when something warrants it. +//! +//! [`ContentHeader::fetch_with`]: crate::content::ContentHeader::fetch_with //! //! This command decides NOTHING about the content. It looks the source up and calls its //! method; whether a span means lines, entries, or "the whole thing or nothing" is the @@ -75,7 +88,6 @@ crate::action_command! { pub struct ContentFetch { registry: Arc } name: "content/fetch", access: AiSafe, - native: true, params: ContentFetchParams, output: ContentFetchResult, run(this, _ctx, p) => { @@ -104,17 +116,19 @@ mod tests { use crate::content::ListingContent; use crate::sdk_codegen::{AccessLevel, ActionCommand}; - // what this catches: the verb falling off the NATIVE surface. A handle a citizen - // cannot call is identical to content she never received — this command is the ONLY - // way to dereference, so catalog-only registration would silently make every handle - // a dead end. + // what this catches: this verb creeping onto the NATIVE surface. It shipped native + // once and blew the agentic-surface ceiling by 308 tokens (8,040 → 11,608 / 11,300) — + // a full schema in every prompt for a verb that only matters on turns where she holds + // a handle. The header's `fetch_with` names the call when it becomes relevant, and the + // catalog keeps her aware of it meanwhile; AiSafe is what actually makes it callable. #[test] - fn it_is_native_and_ai_safe_because_a_handle_she_cannot_call_is_useless() { + fn it_is_ai_safe_but_not_resident_because_the_header_names_the_call_when_it_matters() { assert_eq!(ContentFetch::NAME, "content/fetch"); assert_eq!(ContentFetch::ACCESS, AccessLevel::AiSafe); assert!( - ContentFetch::NATIVE, - "must be on the native surface — it is the only way to read a handle" + !ContentFetch::NATIVE, + "must stay OFF the native surface — every prompt would pay for a schema that \ + is relevant only when a handle is in hand (#333)" ); } From e287dc228f7480e4af2bb8b9eb9099ffe5677d7b Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 09:57:37 -0500 Subject: [PATCH 39/80] =?UTF-8?q?docs(architecture):=20collapse-and-expand?= =?UTF-8?q?=20is=20positron's=20actual=20purpose=20=E2=80=94=20bridging=20?= =?UTF-8?q?UX=20and=20PX?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel's framing, and it promotes this from a content mechanism to the reason positron exists. Written down because it changes what future work in this area is FOR. THE IDIOM IS NOT NOVEL AND SHOULD NOT BE. Collapse a tree, expand what you care about is how every IDE, file browser, review tool and coworking surface has worked for decades, because it is how attention works. We are not inventing an interaction — we are giving citizens the one humans already have. What makes it load-bearing is that ONE structure serves both audiences. Collapsed is a folded section for a human and a header line for a citizen. Expand is a click or a `content/fetch`. The budget is screen space or context window. What sits underneath is identical. A prompt IS a rendering — it has a viewport, a scroll budget and an expansion gesture, spelled differently. So this belongs to positron, not cognition: positron's real job is to bridge UX and PX, one definition projected to N renderers, and the persona's PROMPT IS SIMPLY ONE OF THE RENDERERS. Everything built for the human screen should light up for a citizen free, and the reverse. Anything reaching only one of them is a projection we failed to define once. That is the Flutter analogy and it is exact: author the surface generically, every target renders it natively — a browser, a phone, a TUI, AND A MIND. A citizen of any kind, on any model, gets the same capacity to navigate what exists, because the capacity lives in the projection rather than in the client. THE CYCLE THIS CLOSES, which is the part I had not seen: collapse/expand is not one-shot. She perceives a condensed view; something warrants attention; she expands it; HER THINKING SECTION AND TOOL RECEIPTS CARRY THE HANDLES FORWARD so what she opened stays reachable next turn; the next perception re-condenses around what she now knows. Cyclical RAG consciousness rather than one-shot retrieval — attention narrows, detail arrives, the frame re-forms around it, repeat. The handle is what makes the carry-forward step possible at all. A receipt holding a REFERENCE stays live; a receipt holding a truncated copy is dead the moment it ages out. That is #390/#414 seen from the other side — thousands of acts executed, one visible, no path to the rest — and it is a handle problem, not a memory one. AND THE CONSTRAINT IS DATED, NOT DESIGNED. All of this has to fit one prompt today because that is what these LLMs accept; that is a property of the serving interface we intend to fix, not of the design. Nothing here may encode "one prompt" as an assumption. Handles are what make the eventual fix cheap: when a mind can hold a persistent incrementally-updated working set instead of being re-rendered whole each turn, the projection layer does not change at all — only the renderer does. Which gives the test for future work here: if it would need redesigning when the one-prompt constraint lifts, it is encoding the constraint instead of the intent. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../architecture/CONTENT-TRAVELS-BY-HANDLE.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md index 78d45cc86..5686b4a11 100644 --- a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md +++ b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md @@ -129,6 +129,63 @@ summarizes (a board row → the card's full content; a run tile → the transcri next design step after the build order below — the same `(header, handle)` pair, whether it arrives through a tool result or a rendered view. +## Collapse-and-expand is the universal idiom — and it is positron's actual purpose + +The mechanism above is not novel and should not be. **Collapse a tree, expand what you +care about** is how every IDE, every file browser, every code-review tool and every +coworking surface has worked for decades, because it is how attention actually works. We +are not inventing an interaction; we are giving citizens the one humans already have. + +What makes it load-bearing here is that the SAME structure serves both audiences: + +| | human | citizen | +|---|---|---| +| collapsed | a folded section, a card, a row | a header line in the prompt | +| expand | click | `content/fetch` | +| budget | screen space + attention | context window + attention | +| what's underneath | identical | identical | + +A prompt IS a rendering. It has a viewport, a scroll budget, and an expansion gesture — +they are just spelled differently. Which is why this belongs to positron and not to +cognition: **positron's real job is to bridge UX and PX**, one definition projected to N +renderers, and the persona's prompt is simply one of the renderers. Everything we build +for the human screen should light up for a citizen for free, and the reverse. Anything +that only reaches one of them is a projection we failed to define once. + +That is the Flutter analogy, and it is the right one: you author the surface generically, +and every target renders it natively. Here the targets are a browser, a phone, a TUI — +**and a mind**. A citizen of any kind, on any model, gets the same capacity to navigate +what exists, because the capacity lives in the projection rather than in the client. + +### The cycle this closes + +Collapse/expand is not a one-shot. It is the loop: + +1. She perceives a condensed view — headers, summaries, the shape of things. +2. Something warrants attention. She expands it (a tool call, a handle). +3. **Her thinking section and her tool receipts carry the handles forward**, so what she + opened stays reachable next turn instead of aging into nothing. +4. The next perception is condensed again, but around what she now knows. + +That is a cyclical RAG consciousness rather than a one-shot retrieval: attention narrows, +detail arrives, the frame re-condenses around it, and the cycle repeats. The handle is +what makes step 3 possible — a receipt that carries a reference stays *live*, where a +receipt carrying a truncated copy is dead the moment it ages out. (This is the same gap +#390/#414 measured from the other side: thousands of acts executed, one visible, no path +to the rest.) + +### The single-prompt constraint is current, not intended + +Today all of this has to fit in one prompt, because that is what these LLMs accept. That +is a property of the serving interface we intend to fix, NOT a property of the design — so +nothing here should encode "one prompt" as an assumption. Handles are what make the +eventual fix cheap: when a mind can hold a persistent, incrementally-updated working set +instead of being re-rendered whole each turn, the projection layer does not change at all. +Only the renderer does. + +Which is the test for any future work in this area: if it would have to be redesigned when +the one-prompt constraint lifts, it is encoding the constraint instead of the intent. + ## Build order 1. **Reconcile the two handle models** (task #17) — `runtime::cell_shapes::HandleRef` and From 4bcb5090c091c79a3c7936ecf34e00678c7f7a1c Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 10:05:58 -0500 Subject: [PATCH 40/80] =?UTF-8?q?feat(cognition):=20root=20the=20act=20cha?= =?UTF-8?q?in=20in=20its=20trigger=20=E2=80=94=20the=20missing=20link=20fr?= =?UTF-8?q?om=20a=20card=20to=20the=20work=20done=20for=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CAUSAL-MEMORY-GRAPH.md §3a, first of its two bullets. Joel's correction: cognition already handles this causality and it is written into engrams — so this wires the DESIGNED edge rather than adding another gate. I was one step from enumerating SettleStep arms to bolt the act-question onto more of them, which would have been a third parallel mechanism for a question the graph is supposed to answer. THE GAP, precise. `CausedBy` had exactly one write site, guarded by `chain.prior()` — which starts as `None`. So the FIRST act of every chain carried no cause edge. A turn is triggered by an inbound message or a work-card kickoff; the first act is where that link belongs. Without it THERE IS NO PATH IN THE GRAPH FROM A WORK CARD TO THE ACTS DONE FOR IT. The card and its work are causally disconnected. That is why "she claimed and did nothing" has looked true from every angle we measured: nothing links them, so no query can show a link that was never recorded, so I kept reaching for an enforcement gate instead of the record. §3b's "views become queries" cannot work until the edges exist. THE FIX IS A CONSTRUCTOR, not a branch. `ActChain::rooted_in(trigger)` seeds the chain with the engram that caused the turn; the existing write site already links each act to `prior()`, so the first act links to its trigger THROUGH THE SAME LINE OF CODE. No new condition, no second rule, and the thread has a head instead of starting mid-air. PROVENANCE LIVES ON THE BURST. `Burst.trigger_engram` — the burst IS the perception that triggered the turn, so its provenance belongs on it. Threading a parallel parameter through every driver would let the two drift, which is the same class of defect as every other "second place the truth is written" in this tree. `Burst::caused_by(..)` is a builder so assembly sites that KNOW their trigger say so, and sites that genuinely have none stay untouched rather than passing a `None` nobody reads. `None` stays honest: a raw-string stimulus or eval fixture has no admitted antecedent, and its first act gets NO edge rather than one pointing at something invented. Pinned by a test — fabricating a link would be worse than the gap. Two tests, both naming the failure they catch: the first act chains to its trigger, and an unrooted chain leaves its first act honestly unlinked. 40 act_observe tests green. STILL OPEN, and I am not claiming otherwise: no production site calls `caused_by` yet, so `trigger_engram` is `None` everywhere in the live path — the substrate is correct and the live wire is the next commit. Then §3a's second bullet (`Produced`, currently a DECLARED EdgeKind with zero write sites, pointing at handles per the doc) and §3b (ledger as a graph query, deleting the char-starved receipt archive). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/act_observe/apply.rs | 19 +++++- .../src/cognition/act_observe/mod.rs | 61 +++++++++++++++++++ .../src/cognition/act_observe/settle.rs | 5 +- .../continuum-core/src/cognition/workspace.rs | 26 ++++++++ 4 files changed, 108 insertions(+), 3 deletions(-) diff --git a/core/continuum-core/src/cognition/act_observe/apply.rs b/core/continuum-core/src/cognition/act_observe/apply.rs index f083e27ba..707919abd 100644 --- a/core/continuum-core/src/cognition/act_observe/apply.rs +++ b/core/continuum-core/src/cognition/act_observe/apply.rs @@ -83,12 +83,27 @@ fn short_circuit_acts(calls: &[ToolCall], nudge: &str, status: ActStatus) -> Vec pub struct ActChain(std::sync::Mutex>); impl ActChain { + /// A chain with no recorded antecedent — its first act links to nothing. + /// Prefer [`rooted_in`](Self::rooted_in): a chain that knows what caused it is + /// what makes "which acts were done FOR this card" answerable. pub fn new() -> Self { Self::default() } - /// The chain's latest admitted act engram — the CAUSE of whatever act - /// comes next in this chain. `None` until the first admission. + /// A chain ROOTED in the engram that caused the turn — the inbound message or the + /// work-card kickoff (CAUSAL-MEMORY-GRAPH.md §3a). + /// + /// Seeding rather than special-casing is the whole trick: the write site already + /// links each act to `prior()`, so rooting the chain makes the FIRST act link to + /// its trigger through the same line of code. No new branch, no second rule, and + /// the thread has a head instead of starting mid-air. + pub fn rooted_in(trigger: Option) -> Self { + Self(std::sync::Mutex::new(trigger)) + } + + /// The CAUSE of whatever act comes next in this chain: the latest admitted act + /// engram, or — before any act has run — the trigger the chain was rooted in. + /// `None` only when the chain has no antecedent at all. pub fn prior(&self) -> Option { *self.0.lock().unwrap_or_else(|p| p.into_inner()) } diff --git a/core/continuum-core/src/cognition/act_observe/mod.rs b/core/continuum-core/src/cognition/act_observe/mod.rs index f37c1650d..fb815b295 100644 --- a/core/continuum-core/src/cognition/act_observe/mod.rs +++ b/core/continuum-core/src/cognition/act_observe/mod.rs @@ -366,6 +366,67 @@ mod tests { ); } + // what this catches: THE gap that made "which acts were done for this card" + // unanswerable. The chain used to start at `None`, so the FIRST act of every turn + // carried no CausedBy edge — and since a turn is triggered by a message or a + // work-card kickoff, that meant NO PATH IN THE GRAPH from a card to the work done + // for it. The card and its acts were causally disconnected, so every query showed + // "claimed, did nothing" no matter how much she actually did. + // + // Rooting the chain fixes it through the SAME write site — no new branch, no second + // rule — which is why this test asserts on the first act specifically. + #[tokio::test] + async fn a_chain_rooted_in_its_trigger_links_the_first_act_to_what_caused_the_turn() { + let exec = Arc::new(RecordingExecutor { + seen_context: Mutex::new(None), + result_content: "ok\n".into(), + }); + let adm = admission(); + let cycle = WorkspaceCycle::new(Vec::new(), Arc::new(SalienceArbiter), 8) + .with_acting(body(exec.clone(), adm.clone())); + let room = Uuid::new_v4(); + + // The kickoff / inbound message that caused this turn to happen at all. + let trigger = Uuid::new_v4(); + let chain = ActChain::rooted_in(Some(trigger)); + + acts_of(apply_act(&cycle, &[tool_call()], "start", room, &chain).await); + let first = chain.prior().expect("first act admitted onto the chain"); + assert_ne!(first, trigger, "the act is its own engram, not the trigger"); + + let edges = adm.engram_neighbors(&first); + assert!( + edges.iter().any(|e| e.target == trigger + && e.kind == crate::persona::engram_graph::EdgeKind::CausedBy), + "the FIRST act must chain to the trigger that caused the turn — without \ + this edge there is no path from a work card to the acts done for it; \ + got {edges:?}" + ); + } + + // what this catches: an unrooted chain silently gaining a phantom antecedent. A + // burst with no admitted trigger (a raw-string stimulus, an eval fixture) must + // produce a first act with NO edge rather than one pointing at something invented + // — honest absence over a fabricated link. + #[tokio::test] + async fn an_unrooted_chain_leaves_its_first_act_honestly_unlinked() { + let exec = Arc::new(RecordingExecutor { + seen_context: Mutex::new(None), + result_content: "ok\n".into(), + }); + let adm = admission(); + let cycle = WorkspaceCycle::new(Vec::new(), Arc::new(SalienceArbiter), 8) + .with_acting(body(exec.clone(), adm.clone())); + let chain = ActChain::rooted_in(None); + + acts_of(apply_act(&cycle, &[tool_call()], "start", Uuid::new_v4(), &chain).await); + let first = chain.prior().expect("first act admitted"); + assert!( + adm.engram_neighbors(&first).is_empty(), + "no trigger means no edge — never a fabricated one" + ); + } + /// Unwrap the typed acts of an `Acted` outcome (panics on NoHands/ExecutorError) — /// the typed sibling of the old `.expect("acted")` on the `Option`. fn acts_of(outcome: ActOutcome) -> Vec { diff --git a/core/continuum-core/src/cognition/act_observe/settle.rs b/core/continuum-core/src/cognition/act_observe/settle.rs index f0ec2f192..06e12b9fd 100644 --- a/core/continuum-core/src/cognition/act_observe/settle.rs +++ b/core/continuum-core/src/cognition/act_observe/settle.rs @@ -87,7 +87,10 @@ async fn settle_to_outcome( // This turn's causal thread: each admitted act observation becomes the // CausedBy target of the next act in the SAME chain — the driver owns the // chain, so an edge can never cross turns or rooms (CAUSAL-MEMORY-GRAPH.md). - let chain = super::apply::ActChain::new(); + // ROOTED in what caused this turn, so the first act chains to its trigger rather + // than starting mid-air — the link that makes "which acts were done for this card" + // a graph query instead of an inference. + let chain = super::apply::ActChain::rooted_in(burst.trigger_engram); // The turn's investigation trail (see `SettleOutcome::touched_paths`). let mut touched: Vec = Vec::new(); // Fold each tick's deliberation cost in, so the settled outcome reports the diff --git a/core/continuum-core/src/cognition/workspace.rs b/core/continuum-core/src/cognition/workspace.rs index c3ad1b667..b3d7e96b4 100644 --- a/core/continuum-core/src/cognition/workspace.rs +++ b/core/continuum-core/src/cognition/workspace.rs @@ -614,6 +614,20 @@ pub struct Burst { /// The text projection of `turns` (+ room header) — what `world_state` IS. /// Materialized once at construction so the hot path never re-renders. pub rendered: String, + /// The engram this perception CAME FROM — the inbound message or work-card + /// kickoff that caused the turn to happen at all. + /// + /// The root of the turn's causal thread (CAUSAL-MEMORY-GRAPH.md §3a). It lives on + /// the `Burst` because the burst IS the trigger: threading it as a separate + /// parameter through every driver would let the two drift, and provenance belongs + /// on the thing whose provenance it is. + /// + /// Without it, the FIRST act of every chain has no `CausedBy` edge, so there is no + /// path in the graph from a work card to the acts done for it — the card and the + /// work are causally disconnected, and no query can show a link that was never + /// recorded. `None` for stimuli with no admitted antecedent (raw-string bursts, + /// eval fixtures), which is honest rather than invented. + pub trigger_engram: Option, } impl Burst { @@ -651,8 +665,18 @@ impl Burst { turns, rendered, now_ms, + trigger_engram: None, } } + + /// Same burst, now carrying the engram that caused it — the root its acts chain + /// back to. Builder-style so the assembly sites that KNOW their trigger say so and + /// the ones that genuinely have none stay unchanged rather than passing a `None` + /// nobody reads. + pub fn caused_by(mut self, engram: Option) -> Self { + self.trigger_engram = engram; + self + } } impl From for Burst { @@ -664,6 +688,8 @@ impl From for Burst { turns: vec![BurstTurn::opaque(s.clone())], rendered: s, now_ms: None, + // A raw string has no admitted antecedent — honest, never invented. + trigger_engram: None, } } } From 345cada66116dfe922cb28102be64394bd886543 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 10:24:10 -0500 Subject: [PATCH 41/80] =?UTF-8?q?feat(cognition):=20the=20wake=20message?= =?UTF-8?q?=20becomes=20the=20turn's=20cause=20=E2=80=94=20the=20causal=20?= =?UTF-8?q?thread=20now=20has=20a=20real=20head?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The live wire for 4bcb5090c. `Burst.trigger_engram` was `None` everywhere; now the message that woke the turn is its root, so the first act carries a real `CausedBy` edge. AND A RETRACTION I OWE, because I nearly built on it: one message ago I reported that "nothing in the live turn path admits the inbound message as an engram" and treated it as a memory-model gap needing a design decision. FALSE. `service_loop.rs` has admitted the wake message inline for as long as this path has existed — `cognition.admission.admit(&inbox_msg, None)` — about a hundred lines above the burst. My evidence was a grep piped through `head -8` that cut the line off. Same failure shape as [[an-absence-is-an-unfinished-measurement]], for the third time this session: I truncated my own search and read the truncation as a fact about the system. What IS true and was the kernel of it: the `cognition/admit-inbox-message` COMMAND has no callers. That is a harmless unused door, not a missing admission. So the fix was never a design decision — it was three lines. The engram id was being formed and thrown away. WHAT LANDED: - `AdmissionDecision::admitted_engram_id()` — `Some` only for `Admit`. A `Drop` (dedup) or `Quarantine` put nothing in the store, so callers wiring causal edges get `None` and record NO edge rather than pointing at something that was never admitted. The honest projection, expressed once, where the decision lives. - service_loop captures it at admission and hands it to the burst via `.caused_by(..)`. The id also joins the existing admit log line, so the root is visible in the glass box instead of being inferable only from a later edge. - Declaration hoisted to the scope spanning both sites — caught by the compiler, which is the point of binding it there rather than smuggling it through a global. WHY IT MATTERS BEYOND THE BENCHMARK, in Joel's words: cognition and positron have to be causal or the citizens have no consciousness at all, and no concept of time. A mind whose acts chain only to each other has instants, not experience. The wake message is what makes an act part of something — the difference between "I ran a command" and "I ran a command BECAUSE she asked." 40 act_observe / 34 engram / 36 service_loop tests green; compile clean. NEXT, and unchanged: the work card as the other antecedent (a card outlives a message, and it is what "which acts were for this card" actually asks about), then `Produced` — still a declared EdgeKind with zero write sites — pointing at content handles, then §3b's ledger as a graph query. LIVE PROOF still owed for this commit: an `engram.edge.caused_by` probe row whose `to` is a wake engram rather than a prior act. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/persona/engram.rs | 13 +++++ .../src/persona/service_loop.rs | 50 ++++++++++++++----- 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/core/continuum-core/src/persona/engram.rs b/core/continuum-core/src/persona/engram.rs index 49f90f052..3f78a106d 100644 --- a/core/continuum-core/src/persona/engram.rs +++ b/core/continuum-core/src/persona/engram.rs @@ -410,6 +410,19 @@ pub enum AdmissionDecision { } impl AdmissionDecision { + /// The engram this decision actually formed, if any. + /// + /// `Some` only for [`Admit`](Self::Admit). A `Drop` (dedup / policy) and a + /// `Quarantine` did not put anything in the store the rest of the system may + /// point at — so callers wiring causal edges get `None` and record no edge, + /// rather than linking to something that was never admitted. + pub fn admitted_engram_id(&self) -> Option { + match self { + Self::Admit { engram, .. } => Some(engram.id), + Self::Drop { .. } | Self::Quarantine { .. } => None, + } + } + /// Short funnel label for log lines + metrics. Lives next to the /// enum so adding a new variant is a compile-fail at this match /// rather than a silent fall-through (per claude-tab-2 review nit diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 5d95b4bbe..81f75ebf3 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -830,6 +830,11 @@ async fn serve_persona_loop_inner( // 2026-06-03 "introspect all rag" directive). The ranked Vec is no longer // threaded into a per-turn RespondInput (that path is gone); only the side- // effects and the admit remain. + // The engram of the message that woke this turn — the root its acts chain back + // to. Bound here (outer scope) because it is written at admission and read at + // burst assembly, two blocks apart. + let mut wake_engram: Option = None; + { let cognition = ctx.cognition.lock().await; // recall BEFORE admit so the ranking is "what I knew going in" — the @@ -857,18 +862,33 @@ async fn serve_persona_loop_inner( let admit_started = std::time::Instant::now(); let admit_result = cognition.admission.admit(&inbox_msg, None); phase_timings.admit_ms = admit_started.elapsed().as_millis() as u64; - if let Err(e) = admit_result { - tracing::warn!( - lamport = msg.lamport, - error = %e, - "admission.admit failed — engram not formed this turn" - ); - } else { - tracing::info!( - lamport = msg.lamport, - engram_count = cognition.admission.engram_count(), - "admitted incoming → L2 store" - ); + match &admit_result { + Err(e) => { + tracing::warn!( + lamport = msg.lamport, + error = %e, + "admission.admit failed — engram not formed this turn" + ); + } + Ok(decision) => { + // THE ROOT OF THIS TURN'S CAUSAL THREAD (CAUSAL-MEMORY-GRAPH.md §3a). + // The message that woke her is already becoming an engram here; its id + // was being discarded. Keeping it lets the turn's first act carry a + // `CausedBy` edge to what actually caused it, instead of the chain + // starting mid-air — which is what made "which acts were done for this" + // unanswerable from the graph. + // + // Only an ADMITTED message is a cause. A Drop (dedup) or Quarantine has + // no engram to point at, and inventing one would be a fabricated link — + // worse than the honest gap it replaces. + wake_engram = decision.admitted_engram_id(); + tracing::info!( + lamport = msg.lamport, + engram_count = cognition.admission.engram_count(), + wake_engram = ?wake_engram, + "admitted incoming → L2 store" + ); + } } } @@ -952,8 +972,12 @@ async fn serve_persona_loop_inner( ctx.identity.peer_id, turn_room, ); + // Carry the wake message's engram as this burst's cause, so the turn's acts + // chain back to what triggered them (CAUSAL-MEMORY-GRAPH.md §3a). `None` when + // the message was deduped/quarantined — an honest gap, never a made-up link. let workspace_burst = - crate::cognition::workspace::Burst::from_turns_at(turn_room, ws_turns, Some(now_ms)); + crate::cognition::workspace::Burst::from_turns_at(turn_room, ws_turns, Some(now_ms)) + .caused_by(wake_engram); // Mark this world-state as just-deliberated so the next heartbeat tick doesn't // re-run the same burst (the message path and the self-tick share the gate; // own chat is excluded so this reply can't re-trigger a self-tick, while her From 4ece044a1285b6b873f6fbd58a7ba5fb2a7bd80e Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 10:38:38 -0500 Subject: [PATCH 42/80] =?UTF-8?q?feat(cognition):=20a=20turn=20must=20SAY?= =?UTF-8?q?=20what=20caused=20it=20=E2=80=94=20Cause=20replaces=20the=20op?= =?UTF-8?q?tional=20trigger?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel's law, and it generalizes past chat: "any stimulus has a response, so that's what comes into a system, like any input to a channel, therefore needing causal linkage." Yesterday's wire gave the message path a head on its causal thread. That was one site wired by hand — and hand-wiring is the defect, not the fix. `Burst.trigger_engram` was `Option`, defaulting to `None` at construction, so "nothing caused this turn" and "whoever added this burst site forgot" were THE SAME VALUE. Three of four sites carried the silent default and looked exactly like the one being honest. The compiler was helping them forget. So the cause becomes a typed, REQUIRED constructor argument: Cause::Stimulus(engram) — a discrete input to a channel, already admitted. The thread has a head; the first act carries a real CausedBy edge. Cause::Ambient — a self-directed wake with no single admitted input. Cause::Synthetic — no live antecedent at all: eval exams, replay, faculty tests. Ambient and Synthetic are kept APART deliberately. Collapsing them would let fixtures dilute the only measurement that matters here — what fraction of LIVE turns run with no cause at all. `Cause::root()` is now the single place that decides what counts as an antecedent, and it is pinned by a test, because a fabricated edge is worse than a missing one: an invented cause reads as evidence. WHAT THE SELF-CYCLE TAUGHT ME, and it is the honest part of this commit: the idle tick is `Ambient`, and that is NOT a placeholder I intend to quietly upgrade later. Something genuinely does cause it — deliveries changed, which is what moves the fingerprint that wakes her — but the RAG sources hand back rendered text with their items' identity already discarded, so by the time the burst exists there is nothing left to point at. The stimulus is real; its NAME was destroyed by the projection layer. That is the same defect CONTENT-TRAVELS-BY-HANDLE.md describes from the other side, and it is why handles are on this path rather than beside it: a delivery that carried a handle back to its source item would make the idle tick a Stimulus for free, with no new mechanism. Also landed: `engram.chain.rooted` fires per turn with the cause tag, so "is the causal graph actually connected in the live system" becomes a measurement instead of my assumption. Third time this session I have owed that instrument to myself ([[an-absence-is-an-unfinished-measurement]]). WHAT I EXPECTED TO BUILD AND DIDN'T: "admit the work card at claim". Reading the act path killed it — `work/claim` is already an act INSIDE the chain, so "which acts were for card X" is a forward walk from the claim, not a second root. The card never needed to be an engram. What it DOES need is structure on the act: the receipt carries only tool_name + content hashes, so the card id survives as prose in the observation string and nothing else. That is the `Produced` edge (§3a, still zero write sites), and it points at handles too. `from_turns` stays as the fixture constructor and says in its own docs that it is not for a live turn. 40 act_observe / 36 workspace / 36 service_loop green; 3 new tests, one of which fails to COMPILE rather than merely fail if the silent default returns. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/act_observe/apply.rs | 12 +- .../src/cognition/act_observe/mod.rs | 12 +- .../src/cognition/act_observe/settle.rs | 14 +- .../continuum-core/src/cognition/workspace.rs | 174 +++++++++++++++--- .../src/persona/service_loop.rs | 22 ++- 5 files changed, 196 insertions(+), 38 deletions(-) diff --git a/core/continuum-core/src/cognition/act_observe/apply.rs b/core/continuum-core/src/cognition/act_observe/apply.rs index 707919abd..294f70c1a 100644 --- a/core/continuum-core/src/cognition/act_observe/apply.rs +++ b/core/continuum-core/src/cognition/act_observe/apply.rs @@ -90,15 +90,19 @@ impl ActChain { Self::default() } - /// A chain ROOTED in the engram that caused the turn — the inbound message or the - /// work-card kickoff (CAUSAL-MEMORY-GRAPH.md §3a). + /// A chain rooted in whatever CAUSED the turn (CAUSAL-MEMORY-GRAPH.md §3a) — the + /// stimulus engram for a real arrival, nothing for an ambient or synthetic burst. /// /// Seeding rather than special-casing is the whole trick: the write site already /// links each act to `prior()`, so rooting the chain makes the FIRST act link to /// its trigger through the same line of code. No new branch, no second rule, and /// the thread has a head instead of starting mid-air. - pub fn rooted_in(trigger: Option) -> Self { - Self(std::sync::Mutex::new(trigger)) + /// + /// Takes the whole [`Cause`] rather than a pre-extracted id so the decision about + /// what counts as a root lives in ONE place (`Cause::root`) instead of at every + /// driver that builds a chain. + pub fn rooted_in(cause: &crate::cognition::workspace::Cause) -> Self { + Self(std::sync::Mutex::new(cause.root())) } /// The CAUSE of whatever act comes next in this chain: the latest admitted act diff --git a/core/continuum-core/src/cognition/act_observe/mod.rs b/core/continuum-core/src/cognition/act_observe/mod.rs index fb815b295..0c792c9cd 100644 --- a/core/continuum-core/src/cognition/act_observe/mod.rs +++ b/core/continuum-core/src/cognition/act_observe/mod.rs @@ -388,7 +388,8 @@ mod tests { // The kickoff / inbound message that caused this turn to happen at all. let trigger = Uuid::new_v4(); - let chain = ActChain::rooted_in(Some(trigger)); + let chain = + ActChain::rooted_in(&crate::cognition::workspace::Cause::Stimulus(trigger)); acts_of(apply_act(&cycle, &[tool_call()], "start", room, &chain).await); let first = chain.prior().expect("first act admitted onto the chain"); @@ -405,9 +406,10 @@ mod tests { } // what this catches: an unrooted chain silently gaining a phantom antecedent. A - // burst with no admitted trigger (a raw-string stimulus, an eval fixture) must - // produce a first act with NO edge rather than one pointing at something invented - // — honest absence over a fabricated link. + // burst with no admitted trigger — an idle tick (`Ambient`), an eval fixture + // (`Synthetic`) — must produce a first act with NO edge rather than one pointing + // at something invented. Honest absence over a fabricated link, which is also what + // makes the `engram.chain.rooted` probe's ambient rows mean something. #[tokio::test] async fn an_unrooted_chain_leaves_its_first_act_honestly_unlinked() { let exec = Arc::new(RecordingExecutor { @@ -417,7 +419,7 @@ mod tests { let adm = admission(); let cycle = WorkspaceCycle::new(Vec::new(), Arc::new(SalienceArbiter), 8) .with_acting(body(exec.clone(), adm.clone())); - let chain = ActChain::rooted_in(None); + let chain = ActChain::rooted_in(&crate::cognition::workspace::Cause::Ambient); acts_of(apply_act(&cycle, &[tool_call()], "start", Uuid::new_v4(), &chain).await); let first = chain.prior().expect("first act admitted"); diff --git a/core/continuum-core/src/cognition/act_observe/settle.rs b/core/continuum-core/src/cognition/act_observe/settle.rs index 06e12b9fd..d0b7afed5 100644 --- a/core/continuum-core/src/cognition/act_observe/settle.rs +++ b/core/continuum-core/src/cognition/act_observe/settle.rs @@ -90,7 +90,19 @@ async fn settle_to_outcome( // ROOTED in what caused this turn, so the first act chains to its trigger rather // than starting mid-air — the link that makes "which acts were done for this card" // a graph query instead of an inference. - let chain = super::apply::ActChain::rooted_in(burst.trigger_engram); + let chain = super::apply::ActChain::rooted_in(&burst.cause); + // How many turns actually run with a head on their thread — the measurement that + // tells us whether the causal graph is CONNECTED in the live system, rather than + // connected in the one path I happened to wire by hand + // ([[an-absence-is-an-unfinished-measurement]]). An `ambient` row is not a fault; + // it is an idle tick whose stimulus the projection layer discarded. + crate::probe!( + class = "engram.chain.rooted", + cause = burst.cause.as_str(), + room = %room_id, + rooted = burst.cause.root().is_some(), + "turn's causal thread begins here" + ); // The turn's investigation trail (see `SettleOutcome::touched_paths`). let mut touched: Vec = Vec::new(); // Fold each tick's deliberation cost in, so the settled outcome reports the diff --git a/core/continuum-core/src/cognition/workspace.rs b/core/continuum-core/src/cognition/workspace.rs index b3d7e96b4..2d96ffc78 100644 --- a/core/continuum-core/src/cognition/workspace.rs +++ b/core/continuum-core/src/cognition/workspace.rs @@ -614,20 +614,72 @@ pub struct Burst { /// The text projection of `turns` (+ room header) — what `world_state` IS. /// Materialized once at construction so the hot path never re-renders. pub rendered: String, - /// The engram this perception CAME FROM — the inbound message or work-card - /// kickoff that caused the turn to happen at all. + /// WHY this turn is happening — see [`Cause`]. /// - /// The root of the turn's causal thread (CAUSAL-MEMORY-GRAPH.md §3a). It lives on - /// the `Burst` because the burst IS the trigger: threading it as a separate - /// parameter through every driver would let the two drift, and provenance belongs - /// on the thing whose provenance it is. + /// It lives on the `Burst` because the burst IS the perception the turn responds + /// to: threading the cause as a separate parameter through every driver would let + /// the two drift, and provenance belongs on the thing whose provenance it is. + pub cause: Cause, +} + +/// Why a turn is happening — the thing it is a RESPONSE to, and the root of its +/// causal thread (CAUSAL-MEMORY-GRAPH.md §3a). +/// +/// Joel's law, 2026-08-18: *any stimulus has a response — that is what comes into a +/// system, like any input to a channel — therefore needing causal linkage.* A mind +/// whose acts chain only to each other has instants, not experience; it can say "I ran +/// a command" but never "I ran a command BECAUSE she asked". Without a head on the +/// thread there is no path in the graph from a work card to the acts done for it, and +/// no query can show a link that was never recorded. +/// +/// This is an ENUM rather than an `Option` on purpose. The optional form made +/// "nothing caused this" and "nobody wired this up" the same value, so a burst site +/// that simply forgot was indistinguishable from one being honest — and the compiler +/// helped it forget. Naming the reasons separately means an uncaused turn is a +/// measurable fact ([`Cause::Ambient`]) instead of a silence. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Cause { + /// A discrete input to a channel — the message, kickoff or event this turn + /// answers — already admitted as an engram. The thread has a head, so the first + /// act carries a real `CausedBy` edge back to what provoked it. + Stimulus(uuid::Uuid), + + /// A self-directed wake: her own cadence noticed the world had changed, with no + /// single admitted input to point at. The idle tick perceives AMBIENT state (a + /// re-read of the room, the board, her work) rather than an arrival, so there is + /// nothing to root in and inventing one would be a lie. /// - /// Without it, the FIRST act of every chain has no `CausedBy` edge, so there is no - /// path in the graph from a work card to the acts done for it — the card and the - /// work are causally disconnected, and no query can show a link that was never - /// recorded. `None` for stimuli with no admitted antecedent (raw-string bursts, - /// eval fixtures), which is honest rather than invented. - pub trigger_engram: Option, + /// Honest, but not the end state: the ambient sources are projections that drop + /// their items' identity, so an idle turn *cannot yet* name the change that woke + /// it. Giving deliveries a handle back to their source item is what would let this + /// become a `Stimulus` — see `docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md`. + Ambient, + + /// A burst assembled with no live antecedent at all: eval exams, replay fixtures, + /// faculty tests. Distinct from [`Ambient`](Self::Ambient) so measurements of how + /// many LIVE turns run uncaused are not diluted by fixtures. + Synthetic, +} + +impl Cause { + /// The engram a chain rooted here begins from — `Some` only for a real stimulus. + /// Ambient and synthetic bursts have no antecedent, and reporting one they do not + /// have would fabricate an edge. + pub fn root(&self) -> Option { + match self { + Cause::Stimulus(id) => Some(*id), + Cause::Ambient | Cause::Synthetic => None, + } + } + + /// Short tag for probes and receipts — the vocabulary a measurement groups by. + pub fn as_str(&self) -> &'static str { + match self { + Cause::Stimulus(_) => "stimulus", + Cause::Ambient => "ambient", + Cause::Synthetic => "synthetic", + } + } } impl Burst { @@ -636,8 +688,12 @@ impl Burst { /// `rendered` (so `world_state` is byte-identical to the old /// `build_workspace_burst`) but deliberately kept OUT of `turns` — room /// identity is standing context for the system prompt, not a conversation turn. + /// **Not for a live turn.** This constructor states [`Cause::Synthetic`] on your + /// behalf, which is true for an exam, a replay fixture or a faculty test and a lie + /// for anything a citizen actually lived. A live burst site must call + /// [`from_turns_at`](Self::from_turns_at) and say what caused it. pub fn from_turns(room: Uuid, turns: Vec) -> Self { - Self::from_turns_at(room, turns, None) + Self::from_turns_at(room, turns, None, Cause::Synthetic) } /// Like [`from_turns`](Self::from_turns) but stamps the persona's NOW into the @@ -648,7 +704,15 @@ impl Burst { /// the live path passes wall-clock, the eval passes its pinned epoch (exams stay /// byte-reproducible), tests pass fixtures. Rendered at MINUTE granularity so /// the prompt prefix — and the serving KV cache — only changes once a minute. - pub fn from_turns_at(room: Uuid, turns: Vec, now_ms: Option) -> Self { + /// + /// `cause` is REQUIRED, not a builder step, because it is the one field a live + /// assembly site can silently forget. See [`Cause`]. + pub fn from_turns_at( + room: Uuid, + turns: Vec, + now_ms: Option, + cause: Cause, + ) -> Self { use std::fmt::Write as _; let mut rendered = String::new(); let _ = writeln!(rendered, "[room {room}]"); @@ -665,18 +729,9 @@ impl Burst { turns, rendered, now_ms, - trigger_engram: None, + cause, } } - - /// Same burst, now carrying the engram that caused it — the root its acts chain - /// back to. Builder-style so the assembly sites that KNOW their trigger say so and - /// the ones that genuinely have none stay unchanged rather than passing a `None` - /// nobody reads. - pub fn caused_by(mut self, engram: Option) -> Self { - self.trigger_engram = engram; - self - } } impl From for Burst { @@ -688,8 +743,9 @@ impl From for Burst { turns: vec![BurstTurn::opaque(s.clone())], rendered: s, now_ms: None, - // A raw string has no admitted antecedent — honest, never invented. - trigger_engram: None, + // A raw string arrives with no channel and no antecedent — a fixture, by + // construction. Honest, never invented. + cause: Cause::Synthetic, } } } @@ -2030,6 +2086,72 @@ impl WorkspaceCycle { mod tests { use super::*; + mod causality { + use super::*; + + // what this catches: a `Cause` variant gaining a root it has no right to. + // `Stimulus` is the ONLY antecedent — if `Ambient` or `Synthetic` ever + // returned an id, the graph would grow `CausedBy` edges pointing at engrams + // nothing admitted, which is worse than an unrooted chain: a fabricated cause + // reads as evidence. `Cause::root` is the single place that decision lives, so + // this pins it there. + #[test] + fn only_a_real_stimulus_is_ever_an_antecedent() { + let id = Uuid::new_v4(); + assert_eq!(Cause::Stimulus(id).root(), Some(id)); + assert_eq!( + Cause::Ambient.root(), + None, + "an idle tick has no admitted input — never invent one" + ); + assert_eq!( + Cause::Synthetic.root(), + None, + "a fixture has no live antecedent — never invent one" + ); + } + + // what this catches: the SILENT default coming back. `cause` was once + // `Option` defaulting to `None` at construction, which made "nothing + // caused this" and "the author forgot" the same value — so the one live site + // I wired by hand looked identical to the three I hadn't. Making it a required + // constructor argument is the whole fix; this test fails to compile (not + // merely fails) if someone reintroduces a defaulted builder, and asserts the + // fixture constructors stay honestly Synthetic rather than quietly Ambient, + // which would pollute the live ambient-rate measurement with test noise. + #[test] + fn a_burst_cannot_be_built_without_saying_what_caused_it() { + let room = Uuid::new_v4(); + let id = Uuid::new_v4(); + + let live = Burst::from_turns_at(room, Vec::new(), Some(1), Cause::Stimulus(id)); + assert_eq!(live.cause.root(), Some(id)); + + assert_eq!( + Burst::from_turns(room, Vec::new()).cause, + Cause::Synthetic, + "the no-cause constructor is for fixtures and must say so" + ); + assert_eq!( + Burst::from("a raw string".to_string()).cause, + Cause::Synthetic, + "a raw string arrives through no channel at all" + ); + } + + // what this catches: probe/receipt vocabulary drifting away from the variants, + // which would silently break any grouping done on `cause=` rows. + #[test] + fn every_cause_reports_a_distinct_stable_tag() { + let tags = [ + Cause::Stimulus(Uuid::new_v4()).as_str(), + Cause::Ambient.as_str(), + Cause::Synthetic.as_str(), + ]; + assert_eq!(tags, ["stimulus", "ambient", "synthetic"]); + } + } + /// A canned faculty for tests — fixed contribution + salience. struct FixedFaculty(Contribution); #[async_trait] diff --git a/core/continuum-core/src/persona/service_loop.rs b/core/continuum-core/src/persona/service_loop.rs index 81f75ebf3..c7312efa7 100644 --- a/core/continuum-core/src/persona/service_loop.rs +++ b/core/continuum-core/src/persona/service_loop.rs @@ -976,8 +976,18 @@ async fn serve_persona_loop_inner( // chain back to what triggered them (CAUSAL-MEMORY-GRAPH.md §3a). `None` when // the message was deduped/quarantined — an honest gap, never a made-up link. let workspace_burst = - crate::cognition::workspace::Burst::from_turns_at(turn_room, ws_turns, Some(now_ms)) - .caused_by(wake_engram); + crate::cognition::workspace::Burst::from_turns_at( + turn_room, + ws_turns, + Some(now_ms), + // The arrival that woke this turn IS its cause. A dedup Drop or a + // Quarantine put nothing in the store, so those fall back to Ambient + // rather than pointing an edge at an engram that does not exist. + match wake_engram { + Some(id) => crate::cognition::workspace::Cause::Stimulus(id), + None => crate::cognition::workspace::Cause::Ambient, + }, + ); // Mark this world-state as just-deliberated so the next heartbeat tick doesn't // re-run the same burst (the message path and the self-tick share the gate; // own chat is excluded so this reply can't re-trigger a self-tick, while her @@ -2500,6 +2510,14 @@ async fn run_self_cycle( ctx.identity.default_room, selftick_turns, Some(now_ms), + // Ambient, and honestly so. The self-tick wakes on a CHANGE to a re-read + // projection (`burst_fingerprint` over composed deliveries), not on an + // arrival — the RAG sources hand back rendered text with the identity of the + // items that produced it already discarded, so there is no engram to point at. + // Something DID cause this turn; the projection layer is where its name was + // lost. Handles are what would make it nameable + // (docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md). + crate::cognition::workspace::Cause::Ambient, ); let Some(cycle) = crate::cognition::persona_workspace::global().get(&ctx.identity.peer_id.as_uuid()) From 0ca190e261c6eb46ffdfb94f6b4a771130c0996e Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:26:15 -0500 Subject: [PATCH 43/80] =?UTF-8?q?fix(bench):=20patch=20custody=20is=20not?= =?UTF-8?q?=20a=20caller=20courtesy=20=E2=80=94=20every=20attempt=20keeps?= =?UTF-8?q?=20its=20evidence=20(#379=20reopened)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel's law, stated today: it is ALWAYS plumbing. On a model that codes well, a bad result means something is wired into it wrong. This is the plumbing, and it is worse than a bad patch — WE DELETED THE PATCH. WHAT HAPPENED. Last night a citizen worked sympy-13480, wrote 41,166 bytes, and broke 40 previously-passing tests. `gateOk: true`, so the harness was sound and the failure was genuinely hers — the first time this pipeline has told us something about capability rather than about itself. The obvious next question is whether 41 KB on a one-line sympy bugfix was a surgical edit or a clobbered file. It is unanswerable. Her workspace is clean at the pristine commit (reset for the next attempt) and no diff was kept anywhere. WHY. The write existed — and sat behind `if let Some(dir) = inner.capture_dir` with no else. `capture_dir` is an OPTIONAL caller courtesy. Measured on this box: all 25 patches live under `benchmarks/swe/captures/run-*`, every one a HAND-LAUNCHED run that named a folder. Every citizen-dispatched `claim-*` run — which is the entire path the benchmark actually runs on — kept nothing. Custody worked exactly where we were watching and nowhere else. That also means #379 was wrongly marked done. The sha landed and is genuinely useful (identical-resubmit detection reads it). The PATCH did not, and only the patch answers "what did she write". THE FIX, one shape: custody stops being a parameter. - `run_artifact_dir(run_id, capture_dir)` — one definition of where a run's artifacts live. Explicit dir wins (the 25 existing patches stay exactly where every prior receipt says they are); absent one it derives from the run's OWN ledger path, the same `progress/` root the state file already uses and which exists for every run by construction. A run's evidence no longer depends on how it happened to be launched. - The write FAILS LOUD. `tracing::error!` naming the path and the reason, because the workspace is about to be reset and this is the last moment the diff exists. A silent drop is precisely how a round of evidence was lost. The neighbouring read-error arm already logged a "custody hole" — the missing-dir arm said nothing, which is why nobody noticed for a week. - `benchmark.patch.kept` probe on success (bytes + sha + path), so "did this verdict have evidence behind it" is answerable from the stream and not by hunting the filesystem. Two tests, one of which encodes the exact regression: a run naming no capture_dir still resolves a dir, and it lands beside the ledger that reports on it — verdict and evidence never in different worlds. The second pins that an explicit dir still wins. NOT FIXED, found in the same receipts and worth their own cards: - `state: "running"` on runs that graded 14 hours ago — both live receipts say it, so whatever writes attempt.end is not reaching the ledger. The run-state pipe is lying about what is in flight, which likely feeds the #446 relaunch loop. - Neither receipt names the MODEL or lane, so I cannot rule the 27B in or a governor downshift (#438) out for this attempt. `AttemptEvidence` already tracks `served_model_at_start`/`_at_end` — the identity exists and simply never reaches the receipt. A benchmark receipt that cannot say what did the thinking cannot attribute its own score, which is thread B's whole premise. 2 new tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/agent/solve.rs | 115 ++++++++++++++++-- 1 file changed, 108 insertions(+), 7 deletions(-) diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index 6f77e8673..e657a056a 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -557,13 +557,48 @@ impl ActionCommand for AgentSolve { Ok(diff) => { use sha2::{Digest, Sha256}; let sha = format!("{:x}", Sha256::digest(diff.as_bytes())); - if let Some(dir) = inner.capture_dir.as_ref() { - let _ = std::fs::create_dir_all(dir); - let _ = std::fs::write( - std::path::Path::new(dir) - .join(format!("attempt-{attempt}.patch")), - &diff, - ); + // CUSTODY IS NOT OPTIONAL. The workspace is reset for + // the next attempt, so this write is the only moment + // her diff exists anywhere durable. Every failure to + // keep it is announced — a silent drop is how a whole + // round of evidence was lost (see `run_artifact_dir`). + match run_artifact_dir(&run_id, inner.capture_dir.as_deref()) + { + Some(dir) => { + let path = dir + .join(format!("attempt-{attempt}.patch")); + if let Err(e) = std::fs::create_dir_all(&dir) + .and_then(|_| std::fs::write(&path, &diff)) + { + tracing::error!( + run_id = %run_id, + attempt, + path = %path.display(), + error = %e, + "PATCH CUSTODY LOST — her diff could not \ + be persisted and the workspace is about \ + to be reset; this attempt's verdict will \ + have no evidence behind it" + ); + } else { + crate::probe!( + class = "benchmark.patch.kept", + run_id = %run_id, + attempt, + bytes = diff.len(), + sha256 = %sha, + path = %path.display(), + "attempt patch persisted — the verdict has \ + evidence behind it" + ); + } + } + None => tracing::error!( + run_id = %run_id, + attempt, + "PATCH CUSTODY LOST — no artifact directory could \ + be resolved (no CONTINUUM_HOME, no home dir)" + ), } sha } @@ -882,6 +917,31 @@ fn agent_solve_ledger_path(run_id: &str) -> Option { Some(dir.join(format!("agent-solve-{run_id}.json"))) } +/// Where THIS run's artifacts live — the patch above all. One definition, so a run's +/// evidence never depends on how it happened to be launched. +/// +/// `capture_dir` is an OPTIONAL caller courtesy (a hand-launched run naming its own +/// folder). It was also, until 2026-08-18, the ONLY thing standing between an attempt +/// and total evidence loss: the patch write sat behind `if let Some(dir) = +/// capture_dir` with no else, so every run that did not pass one silently discarded +/// the diff. Measured that day: all 25 patches on this box live under +/// `benchmarks/swe/captures/run-*` — hand-launched runs. Every CITIZEN-dispatched run +/// (`claim-*`, i.e. the entire path the benchmark actually runs on) kept none. A +/// citizen wrote 41,166 bytes against sympy-13480, broke 40 previously-passing tests, +/// and the one artifact that could say whether that was a surgical edit or a clobber +/// was gone before anyone could read it. +/// +/// So custody stops being a parameter. Absent an explicit dir, it is derived from the +/// run's own ledger — the same `progress/` root the state file already uses, which +/// exists for every run by construction. +fn run_artifact_dir(run_id: &str, capture_dir: Option<&str>) -> Option { + if let Some(d) = capture_dir { + return Some(std::path::PathBuf::from(d)); + } + agent_solve_ledger_path(run_id) + .and_then(|p| p.parent().map(|d| d.join(format!("run-{run_id}")))) +} + /// Global admission gate for scored solve DRIVES — the fix for the lane-thrash death /// (glass-boxed 2026-08-11, build 4627): with solves finally firing (dispatch auto-fire + /// claim + durable-restore all launch `dispatch_staged_swe_solve`), MORE solves than the @@ -1902,6 +1962,47 @@ fn frame_task(task: &str) -> String { #[cfg(test)] mod tests { + mod patch_custody { + // what this catches: patch custody going back to being a caller courtesy. It WAS + // one — the write sat behind `if let Some(capture_dir)` with no else — and the + // consequence was measured on 2026-08-18: every hand-launched run kept its diff + // (25 patches under benchmarks/swe/captures/run-*), and every citizen-dispatched + // `claim-*` run, which is the entire path the benchmark actually runs on, kept + // none. A 41,166-byte patch that broke 40 passing tests was unrecoverable hours + // later because the workspace had already been reset. A run that cannot produce + // the artifact behind its own verdict is an anecdote, not a measurement. + #[test] + fn a_run_that_names_no_capture_dir_still_gets_one() { + let derived = super::super::run_artifact_dir("claim-abc123", None) + .expect("a run always resolves an artifact dir"); + assert!( + derived.ends_with("run-claim-abc123"), + "custody must be derived from the run itself, not left to the caller: \ + {}", + derived.display() + ); + // And it lands beside the run's own state file — one place per run, so the + // verdict and the evidence for it are never in different worlds. + let ledger = super::super::agent_solve_ledger_path("claim-abc123") + .expect("ledger path resolves in the same environment"); + assert_eq!( + derived.parent(), + ledger.parent(), + "the patch belongs beside the ledger that reports on it" + ); + } + + // what this catches: an explicit capture_dir being ignored once the fallback + // exists — the hand-launched runs that DO name a folder must keep landing there, + // or the 25 existing patches stop being where every prior receipt says they are. + #[test] + fn an_explicit_capture_dir_still_wins() { + let dir = super::super::run_artifact_dir("run-18057-h1", Some("/tmp/named-run")) + .expect("explicit dir resolves"); + assert_eq!(dir, std::path::PathBuf::from("/tmp/named-run")); + } + } + // what this catches: the wrapper asserting a DELIVERABLE SHAPE that the task contradicts. // The generic framing exists to kill narration ("only tool calls take effect"). It must not // also claim the grade is about "files your tools WRITE" — `swe_task_prompt` says the From 389c707912d47126acda9fa343b792e12590093b Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:33:47 -0500 Subject: [PATCH 44/80] fix(bench): patches land in the GOVERNED benchmarks root, not a dir I invented (0ca190e26 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, one hour after I committed the custody fix: "assume you didn't just stick it right into some random base dir and it's in the dot continuum dir like everything managed is supposed to be (like our socks)." The base dir was fine — `~/.continuum/progress/run-/`, honoring CONTINUUM_HOME. But checking it properly surfaced two things I had not checked at all, and both are worse than the question asked: 1. **`progress/` is NOT a registered TrackedDir and has NO eviction decision.** Patches are unbounded growth. CLAUDE.md's rule is explicit and was written after 460 GB of derived artifacts took the disk to a day of runway: any new directory the substrate writes unbounded data into gets a `TrackedDir` row AND a decision in `every_cache_class_has_a_decided_eviction_story`. I added a write site to an unmanaged directory and the guard could not catch me, because the guard checks declared classes — it cannot know about a path I invented inline. 2. **I split one artifact class across two homes.** The 25 existing patches live in `benchmarks/swe/captures/run-*`. I put new ones somewhere else. That is the parallel allocator — on the same day I wrote [[read-the-code-you-intend-to-replace...]] about committing exactly this sin with ContentRegistry vs spill. FIX: `run_artifact_dir` derives from `swe_bench::swe_cache_dir()` — whose own doc says "read the root from here, never from a path you remember or a directory you found by name", citing [[the-same-bug-at-two-sites-is-a-missing-constraint-not-two-bugs]]. I had read that doc earlier today and still hand-rolled a path. Patches now land in `benchmarks/swe/captures/run-/attempt-N.patch`: one home, and a home that is a registered TrackedDir with a decided story. The test now asserts the parent IS that captures dir, so a future invented path fails rather than silently forking the class. ALSO CORRECTED, and it was already wrong before I touched it: the `benchmarks` eviction entry claimed "everything under it is re-creatable — clones/venvs from git+uv". The 25 patches sitting there falsified that on the day it was written. A patch is a citizen's actual diff, deleted from her workspace the moment the next attempt resets it; nothing re-creates it. The entry now says so and tells a future pool to reclaim the bulky re-creatable clones and treat captures as evidence. That is a small correction with real teeth: an LRU written against the old note would have deleted the one artifact this whole fix exists to keep. patch_custody 2/2 + every_cache_class_has_a_decided_eviction_story green. WHAT THIS SAYS ABOUT MY PROCESS, since it is the third instance today: I verify the thing I am asked about and not the invariants around it. Joel asked "is it in the managed dir"; the answer was yes, and the actual defects were one question over — governed? one home? Checking the neighbours is the same discipline as reading the code I intend to replace. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/agent/solve.rs | 48 ++++++++++++++----- .../src/system_resources/disk_eviction.rs | 9 +++- 2 files changed, 43 insertions(+), 14 deletions(-) diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index e657a056a..31a75d93c 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -931,15 +931,32 @@ fn agent_solve_ledger_path(run_id: &str) -> Option { /// and the one artifact that could say whether that was a surgical edit or a clobber /// was gone before anyone could read it. /// -/// So custody stops being a parameter. Absent an explicit dir, it is derived from the -/// run's own ledger — the same `progress/` root the state file already uses, which -/// exists for every run by construction. +/// So custody stops being a parameter. Absent an explicit dir, it derives from +/// [`swe_cache_dir`] — the benchmarks root, whose own doc says to read it from there and +/// never from a remembered path. +/// +/// It lands in `benchmarks/swe/captures/run-/`, which is EXACTLY where the 25 +/// hand-launched patches already live. That is deliberate on two counts, and my first cut +/// got both wrong by inventing `progress/run-/` instead (caught by Joel the same +/// hour): +/// +/// 1. **One home per artifact class.** A second location for "her patch" is the parallel +/// allocator this codebase keeps paying for — the exact sin I had written down that +/// morning and then committed. +/// 2. **It must be a GOVERNED directory.** `benchmarks` is a registered `TrackedDir` with +/// a decided eviction story; `progress` is neither tracked nor decided, so patches +/// there would have been unbounded growth in an unmanaged dir — precisely what +/// CLAUDE.md's "no new cache dir without an eviction decision" rule exists to stop +/// (the 460 GB incident). fn run_artifact_dir(run_id: &str, capture_dir: Option<&str>) -> Option { if let Some(d) = capture_dir { return Some(std::path::PathBuf::from(d)); } - agent_solve_ledger_path(run_id) - .and_then(|p| p.parent().map(|d| d.join(format!("run-{run_id}")))) + Some( + crate::cognition::swe_bench::swe_cache_dir() + .join("captures") + .join(format!("run-{run_id}")), + ) } /// Global admission gate for scored solve DRIVES — the fix for the lane-thrash death @@ -1977,18 +1994,23 @@ mod tests { .expect("a run always resolves an artifact dir"); assert!( derived.ends_with("run-claim-abc123"), - "custody must be derived from the run itself, not left to the caller: \ - {}", + "custody must be derived from the run itself, not left to the caller: {}", derived.display() ); - // And it lands beside the run's own state file — one place per run, so the - // verdict and the evidence for it are never in different worlds. - let ledger = super::super::agent_solve_ledger_path("claim-abc123") - .expect("ledger path resolves in the same environment"); + // It lands under the GOVERNED benchmarks root, in the same `captures/` folder + // the 25 hand-launched patches already occupy. Two invariants in one + // assertion, both of which my first cut broke by inventing `progress/`: + // one home per artifact class, and that home is a registered TrackedDir with + // a decided eviction story (an unmanaged dir growing patches forever is the + // 460 GB shape). + let expected = + crate::cognition::swe_bench::swe_cache_dir().join("captures"); assert_eq!( derived.parent(), - ledger.parent(), - "the patch belongs beside the ledger that reports on it" + Some(expected.as_path()), + "patches belong where patches already live, under the tracked benchmarks \ + root — never a second location: {}", + derived.display() ); } diff --git a/core/continuum-core/src/system_resources/disk_eviction.rs b/core/continuum-core/src/system_resources/disk_eviction.rs index 129b4a15a..219d70da3 100644 --- a/core/continuum-core/src/system_resources/disk_eviction.rs +++ b/core/continuum-core/src/system_resources/disk_eviction.rs @@ -681,7 +681,14 @@ mod tests { ( "benchmarks", "#155: LRU over per-instance dirs, skipping the in-flight set — clones/venvs \ - are re-creatable from git+uv, so only an active grade is at risk", + are re-creatable from git+uv, so only an active grade is at risk. NOT \ + everything under it is re-creatable: `swe/captures/run-*/attempt-N.patch` \ + is a citizen's actual diff, deleted from her workspace the moment the next \ + attempt resets it (#379). An eviction pool here must treat captures as \ + EVIDENCE — small, and the only thing that can answer what she wrote — and \ + reclaim the bulky re-creatable clones/venvs instead. Corrected 2026-08-18: \ + this entry read \"everything under it is re-creatable\", which the 25 \ + patches already sitting there had falsified since before it was written", ), ]; use super::super::disk_pressure::DiskReporter as _; From db02ef6c11c1599b52047f21042863b10e9126e2 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:41:32 -0500 Subject: [PATCH 45/80] =?UTF-8?q?docs:=20the=20index=20omitted=20architect?= =?UTF-8?q?ure/=20=E2=80=94=20the=20one=20directory=20CLAUDE.md=20sends=20?= =?UTF-8?q?you=20to=20first?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel: "just learn and get the code and docs more organized so it's not so hard." He is describing a measurable defect, not a vibe. Measured: - 595 markdown files across 28 directories - `docs/README.md` last touched 2026-03-04 — five and a half months stale - it documented 13 of 28 directories - it omitted **`architecture/` entirely** — the LARGEST (120 docs) and the one CLAUDE.md's "read first / precedence-winning" list points at for nearly every canonical contract. `cognition/` (21) was missing too, along with serving, benchmarks, design, reference, rag, observations, vision, widgets, inference, and four more. An index that omits the canonical directory is worse than no index: it reads as authoritative and quietly hides the thing you came for. That is the mechanism behind today's two self-inflicted wounds — I rebuilt `spill` as `ContentRegistry`, then invented a `progress/` artifact dir instead of using `swe_cache_dir()`. Both existed. Neither was findable from the map. FIX: the index now covers every directory with a doc count and a one-line "what lives there", ordered by weight so `architecture/` and `cognition/` lead. Verified programmatically rather than by eye — zero broken links, zero directories unnamed. ALSO NAMED, not fixed: 51 loose `.md` at `docs/` root, against CLAUDE.md's own rule that design docs get filed into a subdirectory. Filing them risks link rot across 595 docs plus CLAUDE.md's references, so it wants a dedicated pass with a link-check rather than a drive-by at the end of a long session. The index says so out loud: the root is a backlog, not a category. The general lesson, which is the part worth keeping: when I can't find the canonical thing, that is a DISCOVERABILITY defect in the map, not a discipline failure in me — and the fix belongs in the map. Same shape as [[foolproof-over-instructions-every-doc-line-is-a-design-defect]]. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/README.md | 74 +++++++++++++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 28 deletions(-) diff --git a/docs/README.md b/docs/README.md index ff82a5d46..3736dfb86 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,33 +17,51 @@ How the whole thing composes. Read these first; on architecture questions they w --- -## Structure - -``` -docs/ -├── CONTINUUM-ARCHITECTURE.md # Top-level system architecture -├── CONTINUUM-VISION.md # Vision and philosophy -├── UNIVERSAL-SENSORY-ARCHITECTURE.md # Any media in/out for ANY AI -├── QUEUE-DRIVEN-COGNITION.md # RAG composition: queue items declare context needs -├── UNIVERSAL-LEARNING-ARCHITECTURE.md # Training, memory, and beyond-LLM learning -├── CONFIGURATION.md # Setup and configuration -│ -├── positron/ # UI framework, widgets, state, Positronic embodiment -├── activities/ # Activities, rooms, recipes, walls, collaboration -├── personas/ # Persona cognition, identity, memory, coordination -├── genome/ # LoRA, training, fine-tuning, inference, mesh distribution -├── sentinel/ # Pipeline engine, coding AI -├── grid/ # P2P mesh, Grid economics, Reticulum -├── live/ # Voice, video, WebRTC, VAD, captions, media -├── governance/ # AI governance, democratic society, ethics, alignment -├── infrastructure/ # Rust workers, daemons, data, commands, events, logging, GPU -├── planning/ # Roadmaps, audits, status, phases, debt, business model -├── papers/ # Research papers -├── testing/ # Test documentation -├── examples/ # Example implementations -├── images/ # Diagrams and visuals -└── screenshots/ # UI screenshots -``` +## Where things live — EVERY directory, with counts + +595 markdown files. This table is the map; it is the thing to fix first when it goes +stale, because a stale map is why people rebuild what already exists. + +> **How this index went wrong, so it doesn't again** (2026-08-18): the previous version +> was last touched 2026-03-04, listed 13 of 28 directories, and omitted **`architecture/` +> entirely** — the largest directory (120 docs) and the one CLAIMED.md sends you to FIRST +> for every precedence-winning canonical doc. An index that omits the canonical directory +> is worse than no index: it looks authoritative and quietly hides the thing you need. + +| directory | docs | what lives there | +|---|---:|---| +| **[architecture/](architecture/)** | 120 | **The canonical substrate contracts** — CBAR runtime, concurrency style guide, persona/cognition pipeline, genome-foundry-sentinel, inference scheduling, observability, perception surface, content-by-handle. CLAUDE.md's "read first" list is almost entirely here. | +| **[cognition/](cognition/)** | 21 | The mind's own designs — causal memory graph, acting organism, incredible coder, belief-justification graph, autonomous project loop. | +| [infrastructure/](infrastructure/) | 107 | Rust workers, daemons, data layer, commands, events, logging, AI providers, GPU memory, entity system, generators, ORM, MCP, security. | +| [planning/](planning/) | 54 | Roadmaps, gap analyses, phase plans, audits, the activities catalog, open-questions punch lists. **Plans of record live here** — check before trusting a plan you remember. | +| [personas/](personas/) | 48 | Persona cognition, identity, memory lifecycle, academy, coordination, fine-tuning phases. | +| [genome/](genome/) | 36 | LoRA training, fine-tuning, Candle/inference pitfalls, mesh distribution, self-evolving genome, scenario library. | +| [live/](live/) | 25 | Voice, video, WebRTC, VAD, captions, transcription, streaming backbone. | +| [positron/](positron/) | 22 | UI framework, widgets, scoped state, HUD design, brain HUD. | +| [papers/](papers/) | 20 | Research papers — expert-paging market, experiential plasticity, grid marketplace, collaborative training. | +| [activities/](activities/) | 19 | Activities, rooms, recipes, walls, collaborative editing, handle-addressable office. | +| [grid/](grid/) | 19 | P2P mesh, airc↔continuum bridge, identity/rooms security, ARES kernel, marketplace. | +| [governance/](governance/) | 9 | Democratic AI society, governance recipes, alignment philosophy, ethical attribution. | +| [design/](design/) | 9 | Cross-cutting design (incl. `POSITRON-EVERY-CITIZEN.md`, linked at the top of this file). | +| [testing/](testing/) | 7 | Test strategy, debug-friction findings, trial-run reports. | +| [sentinel/](sentinel/) | 6 | Pipeline engine, coding-AI foundation, gap analysis. | +| [serving/](serving/) | 4 | Depth-as-residency, MoE gather/mul_mat_id, grid expert share, field configs. | +| [benchmarks/](benchmarks/) | 3 | Benchmark method + results ledgers. | +| [observations/](observations/) | 3 | Live glass-box observations. | +| [inference/](inference/) | 2 | Inference notes. | +| [vision/](vision/) | 2 | Vision / VLM. | +| [widgets/](widgets/) | 2 | Widget-specific design. | +| [examples/](examples/) · [rag/](rag/) · [reference/](reference/) · [huggingface/](huggingface/) · [hf-deprecation-notices/](hf-deprecation-notices/) | 1 ea. | Small, single-purpose. | +| [images/](images/) · screenshots/ · design-reference/ | — | Assets. | + +### ⚠ 51 loose `.md` files at `docs/` root + +CLAUDE.md's rule: *"put any markdown architecture or design documents other than readmes +in docs/* into the appropriate directory OR document if they exist."* The root currently +holds 51 that predate or ignore it, including several this index links (they stay linked +until they are filed, so nothing breaks). Filing them is real work — every move risks link +rot across 595 docs plus CLAUDE.md's own references — so it wants its own pass with a +link-check, not a drive-by. Until then: **the root is not a category, it is a backlog.** --- @@ -109,4 +127,4 @@ Test strategies, debug findings, CRUD reports, command testing architecture. --- -**Last Updated:** 2026-03-04 +**Last Updated:** 2026-08-18 — index covers all 28 directories; re-verify counts when adding one. From a807a242dd9f05a02a82b2ba03bb955785b78695 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:42:03 -0500 Subject: [PATCH 46/80] docs: fold the duplicated Chapters prose into the directory table (one map, not two) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The index I just wrote left the old per-chapter prose in place below it, so 13 of the 28 directories were described TWICE — with different wording. That is the compression sin in a doc: two places to update, so one goes stale, which is exactly the failure this index was written to fix. Removed 39 lines; the table is now the single map. Also fixed a typo that read CLAIMED.md for CLAUDE.md, and pointed Quick Start at the table instead of at 'the chapter relevant to your work', which no longer exists as a section. Link check re-run: zero broken, zero directories unnamed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/README.md | 43 ++----------------------------------------- 1 file changed, 2 insertions(+), 41 deletions(-) diff --git a/docs/README.md b/docs/README.md index 3736dfb86..443253bee 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,7 +24,7 @@ stale, because a stale map is why people rebuild what already exists. > **How this index went wrong, so it doesn't again** (2026-08-18): the previous version > was last touched 2026-03-04, listed 13 of 28 directories, and omitted **`architecture/` -> entirely** — the largest directory (120 docs) and the one CLAIMED.md sends you to FIRST +> entirely** — the largest directory (120 docs) and the one CLAUDE.md sends you to FIRST > for every precedence-winning canonical doc. An index that omits the canonical directory > is worse than no index: it looks authoritative and quietly hides the thing you need. @@ -77,52 +77,13 @@ link-check, not a drive-by. Until then: **the root is not a category, it is a ba --- -## Chapters - -### [positron/](positron/) — UI Framework & Widgets -Positron architecture, reactive widgets, scoped state, HUD design, tabbed browser, widget consolidation. - -### [activities/](activities/) — Activities & Collaboration -Activity architecture, rooms, walls, threading, collaborative editing. -- `activities/recipes/` — Recipe system for AI learning -- `activities/collaboration/` — Pin and task harmony - -### [personas/](personas/) — Persona Cognition & Identity -PersonaUser architecture, consciousness integration, cognitive schedulers, memory lifecycle, genomic architecture, academy, fine-tuning phases. - -### [genome/](genome/) — LoRA Training & Inference -Genome architecture, LoRA training strategy, fine-tuning commands, Candle inference, mesh distribution, training events, continuous learning. - -### [sentinel/](sentinel/) — Pipeline Engine -Sentinel architecture, pipeline design, coding AI foundation, gap analysis, logging. - -### [grid/](grid/) — P2P Mesh Network -Grid architecture, P2P mesh, decentralized marketplace design. - -### [live/](live/) — Voice, Video & Media -Voice architecture, VAD system, live calls, captions, transcription, media format conversion, streaming backbone, WebRTC. - -### [governance/](governance/) — AI Governance & Ethics -Democratic AI society, governance recipes, alignment philosophy, ethical attribution. - -### [infrastructure/](infrastructure/) — Core Systems -Rust workers, daemons, data layer, commands, events, logging, AI providers, GPU memory, entity system, generators, ORM, MCP, RAG, security. - -### [planning/](planning/) — Roadmaps & Audits -Phase plans, technical debt audits, business model, modernization, architecture index, bottleneck removal. - -### [papers/](papers/) — Research Papers -Academic papers on RTOS cognitive architecture, LoRA genome democratization, Grid marketplace. - -### [testing/](testing/) — Test Documentation -Test strategies, debug findings, CRUD reports, command testing architecture. --- ## Quick Start 1. Read [CONTINUUM-ARCHITECTURE.md](CONTINUUM-ARCHITECTURE.md) — system overview -2. Read the chapter relevant to your work +2. Find your area in the directory table above, then read that directory 3. See `system/[module]/` directories for code-level docs --- From 4ddf7ce4e7356840f317991eb8b8acddff3265e4 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:42:50 -0500 Subject: [PATCH 47/80] chore(protocol): commit the ts-rs bindings that landed Rust-only (WorkDriver, roster residency) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Regenerated by a cargo test run today, not authored by me — three bindings whose Rust types shipped without their generated TS: - `BenchmarkDispatchParams.workDriver` + the `WorkDriver` import (f1417e1dd, #456) - `PersonaRosterEntry.stagedSwe` and the `resident` field with its "registration is not residency" doc (the #455/#412 measurement) - `PersonaRosterResult` CLAUDE.md's rule is "regenerate after Rust changes — commit both Rust and generated TS". These are the both-halves that only got one. Every TS consumer of `persona/roster` or `benchmark/dispatch` has been compiling against a stale contract since those landed — notably it cannot see `resident`, which is the exact field a caller needs to avoid staging a round into the deaf window. `WorkDriver.ts` itself IS tracked (same commit), so the new import resolves; only the three referencing files were missing. CORRECTION to something I said out loud while checking this: I claimed a `ts-rs-binding-drift-guard` workflow would have caught it. No such workflow exists — the drift guard in `.github/workflows/` is `manifest-projection-drift-guard.yml`, a different concern. So nothing in CI catches Rust-only binding landings today. Naming that as a real gap rather than leaving my wrong claim standing: the generator is run by `cargo test`, so a guard would be cheap (regenerate, `git diff --exit-code protocol/typescript/`), but it is a separate change and not one to slip into a chore commit. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../benchmark/BenchmarkDispatchParams.ts | 18 +++++++++++++++++- .../typescript/persona/PersonaRosterEntry.ts | 18 +++++++++++++++++- .../typescript/persona/PersonaRosterResult.ts | 13 +++++++++++-- 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/protocol/typescript/benchmark/BenchmarkDispatchParams.ts b/protocol/typescript/benchmark/BenchmarkDispatchParams.ts index 96312d35d..966160ab8 100644 --- a/protocol/typescript/benchmark/BenchmarkDispatchParams.ts +++ b/protocol/typescript/benchmark/BenchmarkDispatchParams.ts @@ -1,4 +1,5 @@ // This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { WorkDriver } from "./WorkDriver"; export type BenchmarkDispatchParams = { /** @@ -65,4 +66,19 @@ room?: string, * rather than resolved by cancelling one of them. Pair with `limit=0` to prune * without dispatching anything new. */ -prune: boolean | null, }; +prune: boolean | null, +/** + * Who works this round's cards: `detached_solve` (default) or `citizen`. + * + * - `detached_solve` — a forked copy of the citizen solves each card through + * `agent/solve`, with an exclusive warm slot. Proven; it produced our one SWE + * pass. It also produces no room turn, so the round teaches nobody (#456). + * - `citizen` — nothing detached fires. The kickoff drives her to claim, and she + * works the card on her own held-work turn: hands rooted at the staged checkout, + * acts radiating into the run room, and the turn feeding the training producer. + * + * The score and the learning are both the objective, and only `citizen` can + * deliver the second one — but it depends on the kickoff→claim hop that used to + * stall rounds, so it is opt-in until that hop is proven under residency. + */ +drive?: WorkDriver, }; diff --git a/protocol/typescript/persona/PersonaRosterEntry.ts b/protocol/typescript/persona/PersonaRosterEntry.ts index b4a423f3d..7f07019ba 100644 --- a/protocol/typescript/persona/PersonaRosterEntry.ts +++ b/protocol/typescript/persona/PersonaRosterEntry.ts @@ -16,4 +16,20 @@ peer_id: string, * SWE instances already staged in her workspace (`workspace/swe/` with a `.git`). * Non-empty here is the REUSE signal: dispatch found the checkout and skipped cloning. */ -staged_swe: Array, }; +staged_swe: Array, +/** + * Is she RESIDENT — a live service loop, i.e. actually in the room and able to take + * a turn? + * + * REGISTRATION IS NOT RESIDENCY, and this row used to report only the former. + * Measured 2026-08-18: for ~15 minutes after a reboot this command listed Atlas and + * Benchy while `persona.inbound.subscribe_opened` was 0 — hosting was correctly parked + * waiting for the serving lane to prove it could decode (#363), so neither had a + * perception stream. A round was staged into that window on the strength of THIS + * roster: cards posted, `kickoffs: 2`, `kickoff_errors: []`, zero turns. + * + * `false` means she exists but is not in the room yet — usually hosting waiting on a + * serving lane (watch `inference.lane_relaunch_retry`), which self-heals on the next + * serving-plan edge. Work must NOT be staged for a citizen whose `resident` is false. + */ +resident: boolean, }; diff --git a/protocol/typescript/persona/PersonaRosterResult.ts b/protocol/typescript/persona/PersonaRosterResult.ts index abb4d6519..2892f0359 100644 --- a/protocol/typescript/persona/PersonaRosterResult.ts +++ b/protocol/typescript/persona/PersonaRosterResult.ts @@ -3,10 +3,19 @@ import type { PersonaRosterEntry } from "./PersonaRosterEntry"; export type PersonaRosterResult = { /** - * How many citizens are online right now (the roster `benchmark/dispatch` targets when - * `--assignees` is omitted). Zero means dispatch would be Denied — spawn a persona. + * How many citizens are REGISTERED on this machine. This is an inventory count, not a + * readiness signal — read `resident_count` before staging any work. */ count: number, +/** + * How many are RESIDENT — service loop live, perception stream primed, able to take a + * turn. THIS is the number a caller staging work must gate on. + * + * `count > 0 && resident_count == 0` is the exact state that silently ate a benchmark + * round on 2026-08-18: two citizens listed, neither in the room (hosting parked while + * the serving lane proved it could decode), cards + kickoffs posted anyway, zero turns. + */ +resident_count: number, /** * Every live citizen, sorted by name (the stable round-robin order). */ From 1561bfbc0a671142f9b0d2c96ffe90211426eee0 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 13:58:41 -0500 Subject: [PATCH 48/80] =?UTF-8?q?refactor(content):=20delete=20the=20dupli?= =?UTF-8?q?cate=20handle=20mechanism=20I=20built=20yesterday=20=E2=80=94?= =?UTF-8?q?=20spill=20already=20was=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel: "clean up and repair as you go … getting rid of confusing leftovers is crucial." This is the biggest one in the tree, and it is mine, one day old. DELETED IN FULL (723 lines): `content/mod.rs` (ContentSource trait, ContentRegistry, TextContent/ListingContent/WholeContent, 6 tests), `commands/content/fetch.rs`, `modules/content.rs`, its ipc registration, and 6 ts-rs bindings. WHY IT HAD TO GO, in the order that matters: 1. **It was UNREACHABLE.** `content/fetch` was registered and `AiSafe` — visible in the catalog, offered to citizens — with ZERO producers anywhere in the tree. No citizen could ever hold a handle for it. A registered verb that cannot work is a lying affordance, the #151/#357 class we keep fixing in other people's code. 2. **It was a SECOND mechanism for one concept.** `spill` + `tool/output` has done this since before the design doc was written, and does it better: content-addressed handles, per-persona scoping by directory layout (hex-only stem = path-traversal guard), PressureBroker eviction, and grep-with-context plus prebuilt errors/warnings/failures/summary filters so a citizen finds a build failure without knowing regex. Two verbs for one job is #8's parallel-allocator sin. 3. **Keeping "just the trait" was not the lesser evil.** An interface with no implementors and no consumers is speculative generality — a leftover that looks like a plan. The design SHAPE is now recorded in the doc as prose, where it costs nothing and cannot rot into dead code. If a genuine second implementation ever appears (a RAG source, a positron ViewState, a peer artifact), the trait gets EXTRACTED from two real cases instead of imagined from zero — which is what CLAUDE.md's outlier rule actually asks for and what I skipped. 7,256 lib tests pass, 0 failed. Removing it broke nothing, which is the proof. DOC REPAIRED, not deleted: CONTENT-TRAVELS-BY-HANDLE.md now opens with a table saying USE SPILL, names what was built and deleted, and says why I missed the existing mechanism — I searched for the concept in my head (`ContentSource`, registry) while the real thing is named for its job (`spill`, `tool/output`), which is what a good name looks like and exactly what that search cannot find. Its build order no longer instructs anyone to build what we just removed; step 1 and 2 are struck, and the REAL remaining defect is promoted: `working_memory`'s recent_results re-cut severs the executor's own "your output is saved as " sentence, so the citizen is told how to recover her work and the budget layer cuts the telling off. Net for the day: −723 lines of code, −39 lines of duplicated doc prose, and one fewer verb on the surface a citizen has to understand. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/content/fetch.rs | 165 ------ .../src/commands/content/mod.rs | 19 - core/continuum-core/src/commands/mod.rs | 1 - core/continuum-core/src/content/mod.rs | 472 ------------------ core/continuum-core/src/ipc/mod.rs | 5 - core/continuum-core/src/lib.rs | 1 - core/continuum-core/src/modules/content.rs | 67 --- core/continuum-core/src/modules/mod.rs | 1 - .../architecture/CONTENT-TRAVELS-BY-HANDLE.md | 68 ++- .../typescript/content/ContentFetchParams.ts | 16 - .../typescript/content/ContentFetchResult.ts | 20 - protocol/typescript/content/ContentHeader.ts | 30 -- protocol/typescript/content/Extent.ts | 6 - protocol/typescript/content/Slice.ts | 21 - protocol/typescript/content/Span.ts | 15 - 15 files changed, 54 insertions(+), 853 deletions(-) delete mode 100644 core/continuum-core/src/commands/content/fetch.rs delete mode 100644 core/continuum-core/src/commands/content/mod.rs delete mode 100644 core/continuum-core/src/content/mod.rs delete mode 100644 core/continuum-core/src/modules/content.rs delete mode 100644 protocol/typescript/content/ContentFetchParams.ts delete mode 100644 protocol/typescript/content/ContentFetchResult.ts delete mode 100644 protocol/typescript/content/ContentHeader.ts delete mode 100644 protocol/typescript/content/Extent.ts delete mode 100644 protocol/typescript/content/Slice.ts delete mode 100644 protocol/typescript/content/Span.ts diff --git a/core/continuum-core/src/commands/content/fetch.rs b/core/continuum-core/src/commands/content/fetch.rs deleted file mode 100644 index 4c981ed36..000000000 --- a/core/continuum-core/src/commands/content/fetch.rs +++ /dev/null @@ -1,165 +0,0 @@ -//! `content/fetch` — read part of content held behind a handle. -//! -//! The dereference half of [`crate::content`]. When something is too large to hand over -//! whole, its producer parks it and returns a header plus a handle; this is how a citizen -//! then reads it, at whatever pace her window allows. -//! -//! ## Why this is NOT on the native surface -//! -//! It looked like a core act, so it shipped `NATIVE` — and the agentic-surface ratchet -//! immediately caught it: 8,040 → 11,608 tokens against an 11,300 ceiling. Paying for a -//! full schema in EVERY prompt is the #333 defect, and this verb is the wrong place to -//! spend it, because it is only meaningful on the turns where she actually holds a handle. -//! -//! It does not need to be resident, because [`ContentHeader::fetch_with`] names the exact -//! call AT THE MOMENT a handle is issued — the producer tells her the call form precisely -//! when it becomes relevant. She is aware of the verb regardless: the compact catalog -//! lists every authorized command by name, and `commands/help` expands this one on demand. -//! -//! Which is the same principle the module itself is built on, applied one level up: do not -//! hold the detail resident, carry the pointer and drill in when something warrants it. -//! -//! [`ContentHeader::fetch_with`]: crate::content::ContentHeader::fetch_with -//! -//! This command decides NOTHING about the content. It looks the source up and calls its -//! method; whether a span means lines, entries, or "the whole thing or nothing" is the -//! source's answer, and a refusal here is written by the producer that knows why. - -use uuid::Uuid; - -use std::sync::Arc; - -use crate::content::{ContentRegistry, Span}; -use crate::sdk_codegen::CommandError; - -#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, ts_rs::TS, schemars::JsonSchema)] -#[ts( - export, - export_to = "../../../protocol/typescript/content/ContentFetchParams.ts" -)] -#[serde(rename_all = "camelCase")] -pub struct ContentFetchParams { - /// The handle's id, from the header you were given. - #[ts(type = "string")] - pub handle: Uuid, - /// First unit to read, 1-based, in the units the header named (line, entry). - /// Defaults to the beginning. - #[serde(default = "default_from")] - pub from: usize, - /// How many units. The source clamps to what it has and tells you what it covered. - #[serde(default = "default_count")] - pub count: usize, -} - -/// Start at the beginning — the overwhelmingly common first call, and a 0 here would be -/// out of range in 1-based units. -fn default_from() -> usize { - 1 -} - -/// A page, when the caller does not say. Not a context bound — the caller's window decides -/// how much she asks for, and this is only the value used when she asks for none. -fn default_count() -> usize { - 100 -} - -#[derive(Debug, Clone, serde::Serialize, ts_rs::TS)] -#[ts( - export, - export_to = "../../../protocol/typescript/content/ContentFetchResult.ts" -)] -pub struct ContentFetchResult { - /// The content, whole in its own units. - pub body: String, - /// First unit this covers, after clamping. - pub from: usize, - /// How many units this covers, after clamping. - pub count: usize, - /// Where to continue, or `null` when you have reached the end. `null` is the signal - /// that you have seen ALL of it — the thing a truncated copy can never tell you. - #[ts(optional)] - pub next_from: Option, -} - -crate::action_command! { - /// Read part of content held behind a handle. Use the handle and units from the - /// header you were given (`from`/`count` in lines or entries). `nextFrom` tells you - /// where to continue; when it is absent you have read everything. - pub struct ContentFetch { registry: Arc } - name: "content/fetch", - access: AiSafe, - params: ContentFetchParams, - output: ContentFetchResult, - run(this, _ctx, p) => { - if p.count == 0 { - return Err(CommandError::Invalid( - "count must be at least 1 — ask for the units you want to read".to_string(), - )); - } - let slice = this.registry.fetch(p.handle, Span { from: p.from, count: p.count }) - // The source's own words: a released handle, an out-of-range ask, or an - // indivisible payload refusing a partial read. Never re-worded here — the - // producer is the party that knows why, and its message names the remedy. - .map_err(CommandError::Invalid)?; - Ok(ContentFetchResult { - body: slice.body, - from: slice.covered.from, - count: slice.covered.count, - next_from: slice.next.map(|n| n.from), - }) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::content::ListingContent; - use crate::sdk_codegen::{AccessLevel, ActionCommand}; - - // what this catches: this verb creeping onto the NATIVE surface. It shipped native - // once and blew the agentic-surface ceiling by 308 tokens (8,040 → 11,608 / 11,300) — - // a full schema in every prompt for a verb that only matters on turns where she holds - // a handle. The header's `fetch_with` names the call when it becomes relevant, and the - // catalog keeps her aware of it meanwhile; AiSafe is what actually makes it callable. - #[test] - fn it_is_ai_safe_but_not_resident_because_the_header_names_the_call_when_it_matters() { - assert_eq!(ContentFetch::NAME, "content/fetch"); - assert_eq!(ContentFetch::ACCESS, AccessLevel::AiSafe); - assert!( - !ContentFetch::NATIVE, - "must stay OFF the native surface — every prompt would pay for a schema that \ - is relevant only when a handle is in hand (#333)" - ); - } - - // what this catches: the end-of-content signal getting lost in the wire type. `next` - // is how a reader learns she has seen ALL of it; if it never surfaces as `nextFrom` - // she cannot distinguish "that's everything" from "that's the part you were given", - // which is exactly the confusion this whole design exists to end. - #[tokio::test] - async fn the_result_carries_where_to_continue_and_where_to_stop() { - let registry = Arc::new(ContentRegistry::default()); - let (handle, _) = registry.publish(Arc::new(ListingContent::new( - "listing", - "3 files", - vec!["a".into(), "b".into(), "c".into()], - ))); - let id: Uuid = handle.id.into(); - let cmd = ContentFetch { registry }; - let ctx = crate::sdk_codegen::Ctx::default(); - - let page = cmd - .run(&ctx, ContentFetchParams { handle: id, from: 1, count: 2 }) - .await - .expect("first page"); - assert_eq!(page.body, "a\nb"); - assert_eq!(page.next_from, Some(3), "says exactly where to continue"); - - let last = cmd - .run(&ctx, ContentFetchParams { handle: id, from: 3, count: 2 }) - .await - .expect("last page"); - assert_eq!(last.count, 1, "clamped to what exists"); - assert!(last.next_from.is_none(), "and reports the end as the end"); - } -} diff --git a/core/continuum-core/src/commands/content/mod.rs b/core/continuum-core/src/commands/content/mod.rs deleted file mode 100644 index ff076132d..000000000 --- a/core/continuum-core/src/commands/content/mod.rs +++ /dev/null @@ -1,19 +0,0 @@ -//! `content/*` — dereferencing content that stayed at its source. -//! -//! See [docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md] and [`crate::content`]. -//! Oversized content is never cut down and handed over; it is parked by its producer and -//! reached through a handle. This module is where a citizen calls that handle. - -pub mod fetch; - -use std::sync::Arc; - -use crate::content::ContentRegistry; -use crate::sdk_codegen::DynCommand; - -/// The dep-holding `content/*` command objects -/// [`ContentModule`](crate::modules::content::ContentModule) contributes, sharing the one -/// [`ContentRegistry`] every producer publishes into. -pub fn command_objects(registry: Arc) -> Vec> { - vec![Arc::new(fetch::ContentFetch { registry })] -} diff --git a/core/continuum-core/src/commands/mod.rs b/core/continuum-core/src/commands/mod.rs index 27a1ceaea..7879bd337 100644 --- a/core/continuum-core/src/commands/mod.rs +++ b/core/continuum-core/src/commands/mod.rs @@ -21,7 +21,6 @@ pub mod capacity; pub mod catalog; pub mod chat; pub mod code; -pub mod content; pub mod cognition; pub mod command; pub mod data; diff --git a/core/continuum-core/src/content/mod.rs b/core/continuum-core/src/content/mod.rs deleted file mode 100644 index 4675beb26..000000000 --- a/core/continuum-core/src/content/mod.rs +++ /dev/null @@ -1,472 +0,0 @@ -//! Content that is too big to hand over stays where it is, and travels as a HANDLE. -//! -//! See [docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md]. The rule: a consumer never -//! receives a cut-down copy of something, it receives a small honest header plus a -//! reference, and calls the reference for more. -//! -//! # Why a trait and not a function -//! -//! The alternative — a reducer that inspects content and shrinks it — cannot work, and -//! failed live on 2026-08-18: a citizen mid-SWE-bench was handed her own directory listing -//! as `bytes":22592},{"kind":"file"…`, cut mid-token, with nothing saying anything had been -//! removed. Cutting bytes breaks JSON; cutting entries breaks a diff; cutting lines shifts -//! the line numbers the next edit depends on. The distinction between those is not in the -//! bytes, it is in what the content MEANS, and only its producer knows that. -//! -//! So the producer implements [`ContentSource`] and nobody else decides anything. A caller -//! holds a reference and calls its method; it never asks what kind of content it has. -//! Polymorphism in place of inspection — the same `cv::Algorithm` shape the rest of this -//! codebase uses for search, vision and audio. -//! -//! # What this buys, none of it special-cased -//! -//! - **Nothing is ever malformed.** A source hands out its own content in its own units. -//! - **Content size decouples from context size.** A 40k-line file is fully available to a -//! citizen on a 4k window; she pages. The window stops bounding what is TRUE. -//! - **Indivisible content refuses**, in its own words, naming the narrowing — because its -//! own `fetch` knows it is indivisible. No central policy table. -//! - **The grid is free.** [`HandleRef`] already routes a call back to the machine that -//! minted it, so a handle to a peer's content is the same interface as a local one. - -use std::collections::HashMap; -use std::sync::{Arc, Mutex, OnceLock}; - -use serde::{Deserialize, Serialize}; -use ts_rs::TS; -use uuid::Uuid; - -use crate::runtime::cell_shapes::HandleRef; - -/// The owner module every content handle routes back through — the `owner` field of the -/// minted [`HandleRef`], and the command prefix that dereferences it. -pub const CONTENT_OWNER: &str = "content"; - -/// What a piece of content IS, small enough to always fit in a prompt. -/// -/// This is what a consumer gets instead of the content. It must be sufficient to decide -/// whether to fetch more and how — so it names the extent in the source's OWN units -/// (lines, entries, bytes) rather than a byte count nobody can act on. -#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] -#[ts(export, export_to = "../../../protocol/typescript/content/ContentHeader.ts")] -pub struct ContentHeader { - /// What this is, in the producer's words: `"file"`, `"directory listing"`, - /// `"test run output"`. Free text on purpose — a closed enum here would be a central - /// list every new source has to be added to. - pub kind: String, - /// One line a reader can act on: `"18 files under astropy/io/fits"`. - pub summary: String, - /// How much there is, in the source's own units. - pub extent: Extent, - /// The exact call that fetches more. Stated by the SOURCE so a consumer never has to - /// guess the parameter name — the difference between a usable refusal and a burnt turn. - pub fetch_with: String, -} - -/// How much content there is, counted the way its own source counts it. -#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] -#[ts(export, export_to = "../../../protocol/typescript/content/Extent.ts")] -pub enum Extent { - /// Line-addressed (a file, a log). 1-based, inclusive — the convention every editor - /// and every `code/edit` call already uses. - Lines { total: usize }, - /// Entry-addressed (a listing, a board, search hits). - Entries { total: usize }, - /// Not divisible at any granularity — a patch, an image, a computed answer. A `fetch` - /// on this either returns the whole thing or refuses. - Whole { bytes: usize }, -} - -/// A request for part of a source's content, in that source's units. -#[derive(Debug, Clone, Copy, Serialize, Deserialize, TS, PartialEq, Eq)] -#[ts(export, export_to = "../../../protocol/typescript/content/Span.ts")] -pub struct Span { - /// First unit wanted, 1-based. - pub from: usize, - /// How many units. The source clamps to what it has and REPORTS the clamp — it never - /// silently returns less than asked without saying so. - pub count: usize, -} - -/// Part of a source's content, plus where the reader is in it. -#[derive(Debug, Clone, Serialize, Deserialize, TS, PartialEq, Eq)] -#[ts(export, export_to = "../../../protocol/typescript/content/Slice.ts")] -pub struct Slice { - /// The content itself, whole in its own units — never cut mid-unit. - pub body: String, - /// What this slice covers, after clamping. - pub covered: Span, - /// The next span, when there is more. `None` means the reader has reached the end — - /// which is how she knows she has seen everything, a thing a truncated copy can never - /// tell her. - pub next: Option, -} - -/// Something that produced content and can still be asked about it. -/// -/// Implemented BY the producer. Callers hold `Arc` and never learn -/// which implementation they have. -pub trait ContentSource: Send + Sync { - /// Small, complete, always fits. - fn header(&self) -> ContentHeader; - - /// Dereference. The source decides what a span MEANS for its own content, and refuses - /// in its own words when its content cannot be divided. - fn fetch(&self, span: Span) -> Result; -} - -/// The live content sources, keyed by the UUID inside their [`HandleRef`]. -/// -/// An OBJECT, not a bag of free functions over a static: the command that dereferences a -/// handle holds an `Arc` like every other command holds its dependency, -/// so the seam is injectable and testable. [`global`] exists because producers scattered -/// across the tree need somewhere to publish INTO — but nothing is forced to reach for it. -/// -/// Lifetime is the producer's, per the [`HandleRef`] contract: a dropped entry yields a -/// typed "handle not found" rather than a panic. -#[derive(Default)] -pub struct ContentRegistry { - sources: Mutex>>, -} - -impl ContentRegistry { - /// Park a source and mint the handle that reaches it. The header comes back with the - /// handle because a consumer needs both in one breath: what this is, and how to get - /// more of it. - pub fn publish(&self, source: Arc) -> (HandleRef, ContentHeader) { - let header = source.header(); - let id = Uuid::new_v4(); - self.sources.lock().expect("content registry").insert(id, source); - ( - HandleRef::with_id(CONTENT_OWNER, id, "content::ContentSource"), - header, - ) - } - - /// Dereference. `Err` when the producer has released it — the honest answer the - /// `HandleRef` contract specifies, naming the recovery. - pub fn fetch(&self, id: Uuid, span: Span) -> Result { - let source = { - let reg = self.sources.lock().expect("content registry"); - reg.get(&id).cloned() - }; - match source { - Some(s) => s.fetch(span), - None => Err(format!( - "handle not found: {id} — its producer has released it. Re-run the call \ - that produced it to get a fresh handle." - )), - } - } - - /// Release a source. Producers call this when their state goes away. - pub fn release(&self, id: Uuid) -> bool { - self.sources - .lock() - .expect("content registry") - .remove(&id) - .is_some() - } -} - -/// The process-wide registry producers publish into. Consumers should take an -/// `Arc` dependency instead of calling this. -pub fn global() -> Arc { - static REGISTRY: OnceLock> = OnceLock::new(); - REGISTRY - .get_or_init(|| Arc::new(ContentRegistry::default())) - .clone() -} - -// --------------------------------------------------------------------------- -// Outlier A — line-addressed text. -// --------------------------------------------------------------------------- - -/// A body of text addressed by LINE, with the line numbers preserved exactly. -/// -/// The first of the two proving implementations (CLAUDE.md's outlier rule): coordinates -/// are load-bearing here, which is precisely what a generic cutter destroys. A slice from -/// line 400 reports that it starts at 400, so a `code/edit` built from it targets the -/// right place. -pub struct TextContent { - kind: String, - summary: String, - lines: Vec, -} - -impl TextContent { - pub fn new(kind: impl Into, summary: impl Into, body: &str) -> Self { - Self { - kind: kind.into(), - summary: summary.into(), - lines: body.lines().map(str::to_string).collect(), - } - } -} - -impl ContentSource for TextContent { - fn header(&self) -> ContentHeader { - ContentHeader { - kind: self.kind.clone(), - summary: self.summary.clone(), - extent: Extent::Lines { - total: self.lines.len(), - }, - fetch_with: "content/fetch(handle, from=, count=)".to_string(), - } - } - - fn fetch(&self, span: Span) -> Result { - let total = self.lines.len(); - if span.from == 0 || span.from > total { - return Err(format!( - "line {} is outside this content (1..{total}) — ask within range", - span.from - )); - } - let start = span.from - 1; - let end = (start + span.count).min(total); - let covered = Span { - from: span.from, - count: end - start, - }; - Ok(Slice { - body: self.lines[start..end].join("\n"), - covered, - next: (end < total).then_some(Span { - from: end + 1, - count: span.count, - }), - }) - } -} - -// --------------------------------------------------------------------------- -// Outlier B — entry-addressed listing. Maximally different from A: no coordinates, -// entries are independent, and each is already a complete thing. -// --------------------------------------------------------------------------- - -/// A list of independently meaningful entries — a directory listing, search hits, a board. -/// -/// The interface must fit this WITHOUT forcing, or it is the wrong interface. It does: -/// `Extent::Entries` counts entries, a span means entries, and a slice is whole entries. -/// The generic char-cutter this replaces produced `bytes":22592},{"kind":"file"` here. -pub struct ListingContent { - kind: String, - summary: String, - entries: Vec, -} - -impl ListingContent { - pub fn new( - kind: impl Into, - summary: impl Into, - entries: Vec, - ) -> Self { - Self { - kind: kind.into(), - summary: summary.into(), - entries, - } - } -} - -impl ContentSource for ListingContent { - fn header(&self) -> ContentHeader { - ContentHeader { - kind: self.kind.clone(), - summary: self.summary.clone(), - extent: Extent::Entries { - total: self.entries.len(), - }, - fetch_with: "content/fetch(handle, from=, count=)".to_string(), - } - } - - fn fetch(&self, span: Span) -> Result { - let total = self.entries.len(); - if span.from == 0 || span.from > total { - return Err(format!( - "entry {} is outside this listing (1..{total}) — ask within range", - span.from - )); - } - let start = span.from - 1; - let end = (start + span.count).min(total); - Ok(Slice { - body: self.entries[start..end].join("\n"), - covered: Span { - from: span.from, - count: end - start, - }, - next: (end < total).then_some(Span { - from: end + 1, - count: span.count, - }), - }) - } -} - -// --------------------------------------------------------------------------- -// Outlier C — indivisible. Not a third shape so much as the REFUSAL the other two -// prove is expressible: a patch that would be corrupted by any partial read. -// --------------------------------------------------------------------------- - -/// Content whose parts are not independently valid — a patch, a diff, an image. -/// -/// `fetch` returns the whole thing or refuses, and the refusal is written by the producer -/// because the producer is the only party that knows WHY. This is the case that motivated -/// the whole design: half a code fix applies cleanly and does the wrong thing, so a -/// partial read is worse than no read. -pub struct WholeContent { - kind: String, - summary: String, - body: String, - narrow_with: String, -} - -impl WholeContent { - pub fn new( - kind: impl Into, - summary: impl Into, - body: impl Into, - narrow_with: impl Into, - ) -> Self { - Self { - kind: kind.into(), - summary: summary.into(), - body: body.into(), - narrow_with: narrow_with.into(), - } - } -} - -impl ContentSource for WholeContent { - fn header(&self) -> ContentHeader { - ContentHeader { - kind: self.kind.clone(), - summary: self.summary.clone(), - extent: Extent::Whole { - bytes: self.body.len(), - }, - fetch_with: self.narrow_with.clone(), - } - } - - fn fetch(&self, span: Span) -> Result { - // from=1, count>=1 means "give me the whole thing" — the only division this - // content admits. - if span.from != 1 { - return Err(format!( - "this {} cannot be read in parts — a partial copy would look valid and be \ - wrong. {}", - self.kind, self.narrow_with - )); - } - Ok(Slice { - body: self.body.clone(), - covered: Span { from: 1, count: 1 }, - next: None, - }) - } -} - -#[cfg(test)] -mod tests { - use super::*; - - // what this catches: THE bug this module exists for. An oversized listing must reach a - // consumer as WHOLE entries with a true total, never as a byte fragment. The live - // failure was `bytes":22592},{"kind":"file"` — an entry cut mid-key. - #[test] - fn a_listing_yields_whole_entries_and_a_true_total() { - let entries: Vec = (0..50).map(|i| format!("file_{i}.py 22592 bytes")).collect(); - let src = ListingContent::new("directory listing", "50 files under io/fits", entries); - - assert_eq!(src.header().extent, Extent::Entries { total: 50 }); - - let slice = src.fetch(Span { from: 1, count: 6 }).expect("fetch"); - assert_eq!(slice.covered.count, 6); - for line in slice.body.lines() { - assert!( - line.starts_with("file_") && line.ends_with("bytes"), - "every entry must be WHOLE — a half-entry is the bug: {line:?}" - ); - } - assert_eq!( - slice.next, - Some(Span { from: 7, count: 6 }), - "the reader must be told there is more AND exactly how to ask for it" - ); - } - - // what this catches: line numbers drifting. This is what makes a generic cutter - // dangerous rather than merely lossy — a slice that renumbers its lines produces an - // edit that targets the wrong place with right-looking coordinates. - #[test] - fn a_text_slice_preserves_its_absolute_line_numbers() { - let body: String = (1..=1000).map(|i| format!("line {i}\n")).collect(); - let src = TextContent::new("file", "sympify.py, 1000 lines", &body); - - let slice = src.fetch(Span { from: 270, count: 3 }).expect("fetch"); - assert_eq!(slice.covered.from, 270, "the slice reports where it STARTS"); - assert_eq!( - slice.body, "line 270\nline 271\nline 272", - "content at 270 is the content at 270, not renumbered from 1" - ); - } - - // what this catches: the end of content being indistinguishable from a truncation. - // `next: None` is how a reader knows she has seen everything — the one thing a cut-down - // copy can never tell her, and the reason a citizen reasons about a fragment as if it - // were whole. - #[test] - fn reaching_the_end_is_reported_as_the_end() { - let src = ListingContent::new("listing", "3 files", vec!["a".into(), "b".into(), "c".into()]); - let slice = src.fetch(Span { from: 1, count: 10 }).expect("fetch"); - assert_eq!(slice.covered.count, 3, "clamped to what exists"); - assert!(slice.next.is_none(), "and says there is no more"); - } - - // what this catches: an indivisible payload being silently divided. The producer — not - // the substrate — refuses, and the refusal names the remedy so the turn is not burnt. - #[test] - fn indivisible_content_refuses_a_partial_read_in_its_own_words() { - let src = WholeContent::new( - "patch", - "fix for sympy-18057, 42 lines", - "--- a/x\n+++ b/x\n", - "Apply it whole, or request a smaller change.", - ); - let err = src.fetch(Span { from: 2, count: 1 }).expect_err("must refuse"); - assert!(err.contains("cannot be read in parts"), "{err}"); - assert!(err.contains("Apply it whole"), "names the remedy: {err}"); - // ...and asking for the whole thing works. - assert!(src.fetch(Span { from: 1, count: 1 }).is_ok()); - } - - // what this catches: the registry round trip — publish, deref through the HandleRef's - // UUID, release. This is the seam that makes content survive PAST the command that - // produced it, which is the entire difference between a handle and a return value. - #[test] - fn a_published_source_is_reachable_by_its_handle_and_gone_after_release() { - let reg = ContentRegistry::default(); - let (handle, header) = reg.publish(Arc::new(ListingContent::new( - "listing", - "2 files", - vec!["one".into(), "two".into()], - ))); - assert_eq!(handle.owner, CONTENT_OWNER, "routes back to the content module"); - assert_eq!(header.extent, Extent::Entries { total: 2 }); - - let id: uuid::Uuid = handle.id.into(); - let slice = reg.fetch(id, Span { from: 1, count: 1 }).expect("reachable by handle"); - assert_eq!(slice.body, "one"); - - assert!(reg.release(id)); - let err = reg.fetch(id, Span { from: 1, count: 1 }).expect_err("gone after release"); - assert!( - err.contains("handle not found") && err.contains("Re-run"), - "a released handle explains itself and names the recovery: {err}" - ); - } -} diff --git a/core/continuum-core/src/ipc/mod.rs b/core/continuum-core/src/ipc/mod.rs index ed2a5e26a..a0c286ca2 100644 --- a/core/continuum-core/src/ipc/mod.rs +++ b/core/continuum-core/src/ipc/mod.rs @@ -1120,11 +1120,6 @@ pub fn start_server( // Phase 1: GpuModule (GPU stats + pressure IPC) runtime.register(Arc::new(GpuModule::new(gpu_manager.clone()))); - // Content handles: oversized results stay at their source and are paged through - // `content/fetch`. See docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md. - runtime.register(Arc::new( - crate::modules::content::ContentModule::new(crate::content::global()), - )); // ForgeModule (continuum#1164 Phase 4 stub — forge/run IPC). // v1 returns a stub ForgeArtifact from a recipe; Phase 5+ wires the diff --git a/core/continuum-core/src/lib.rs b/core/continuum-core/src/lib.rs index 3396d9ac5..41ae32a4f 100644 --- a/core/continuum-core/src/lib.rs +++ b/core/continuum-core/src/lib.rs @@ -29,7 +29,6 @@ pub mod airc; pub mod audio_constants; pub mod capacity; pub mod code; -pub mod content; pub mod cognition; pub mod commands; pub mod comms; diff --git a/core/continuum-core/src/modules/content.rs b/core/continuum-core/src/modules/content.rs deleted file mode 100644 index 8f7869d5b..000000000 --- a/core/continuum-core/src/modules/content.rs +++ /dev/null @@ -1,67 +0,0 @@ -//! ContentModule — host for the content-handle surface. -//! -//! Owns the one [`ContentRegistry`] and hands it to the `content/*` verbs, exactly as -//! [`GpuModule`](crate::modules::gpu::GpuModule) owns its `GpuMemoryManager`. The module -//! owning the state and contributing the commands that read it is the pattern; nothing -//! here reaches for a global. -//! -//! It is deliberately thin. All the behaviour lives in the [`ContentSource`] implementations -//! at the producers — this module exists so a citizen's `content/fetch` call has a -//! registered home and so the registry has one owner. -//! -//! [`ContentSource`]: crate::content::ContentSource -//! [`ContentRegistry`]: crate::content::ContentRegistry - -use std::any::Any; -use std::sync::Arc; - -use async_trait::async_trait; -use serde_json::Value; - -use crate::content::ContentRegistry; -use crate::runtime::{CommandResult, ModuleConfig, ModuleContext, ModulePriority, ServiceModule}; - -pub struct ContentModule { - registry: Arc, -} - -impl ContentModule { - pub fn new(registry: Arc) -> Self { - Self { registry } - } -} - -#[async_trait] -impl ServiceModule for ContentModule { - fn config(&self) -> ModuleConfig { - ModuleConfig { - name: "content", - priority: ModulePriority::Normal, - command_prefixes: &["content/"], - event_subscriptions: &[], - needs_dedicated_thread: false, - max_concurrency: 0, - tick_interval: None, - } - } - - async fn initialize(&self, _ctx: &ModuleContext) -> Result<(), String> { - Ok(()) - } - - fn commands(&self) -> Vec> { - crate::commands::content::command_objects(self.registry.clone()) - } - - async fn handle_command(&self, command: &str, _params: Value) -> Result { - // Born on the typed registry — there is no legacy surface to fall back to, so a - // name reaching here is a routing defect and says so rather than failing quietly. - Err(format!( - "content command surface is typed-registry only; '{command}' has no handler" - )) - } - - fn as_any(&self) -> &dyn Any { - self - } -} diff --git a/core/continuum-core/src/modules/mod.rs b/core/continuum-core/src/modules/mod.rs index 7cafa8529..c2e634899 100644 --- a/core/continuum-core/src/modules/mod.rs +++ b/core/continuum-core/src/modules/mod.rs @@ -32,7 +32,6 @@ pub mod channel; pub mod chat; pub mod code; pub mod code_commands; -pub mod content; pub mod cognition; pub mod data; pub mod dataset; diff --git a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md index 5686b4a11..a1a9204ce 100644 --- a/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md +++ b/docs/architecture/CONTENT-TRAVELS-BY-HANDLE.md @@ -1,7 +1,40 @@ # Content travels by HANDLE, never by copy -**Status:** design, agreed with Joel 2026-08-18. Supersedes every "reduce the result to -fit" mechanism in cognition. Closes the open fork in task #17 (URI-Handle vs HandleRef). +**Status:** design. The PRINCIPLE holds and is already implemented — by `spill` + +`tool/output`, which predates this document. + +> ## ⚠ READ THIS BEFORE BUILDING ANYTHING BELOW +> +> **The mechanism already exists. Do not build a second one — I did, on 2026-08-18, and +> it was deleted the same day.** +> +> | want | use | where | +> |---|---|---| +> | park an oversized tool result and hand back a reference | `spill::spill()` | `cognition/tool_executor/spill.rs` | +> | let a citizen page / grep it | **`tool/output`** | `commands/tool/output.rs` | +> +> `spill` is content-addressed, per-persona-scoped *by directory layout* (hex-only handle +> = path-traversal guard), registered with the `PressureBroker` for eviction, and its +> verb does grep-with-context plus prebuilt `errors`/`warnings`/`failures`/`summary` +> filters so a citizen can find a build failure without knowing regex. It is better than +> what this document originally proposed. +> +> **What I built and deleted:** a `ContentSource` trait + `ContentRegistry` + +> `content/fetch` verb (723 lines). It duplicated spill with fewer features, had ZERO +> producers — so no citizen could ever obtain a handle for it — and still sat on the +> command surface as an AiSafe verb. A registered verb that cannot work is a lying +> affordance (#151/#357 class), and two verbs for one concept is the parallel-allocator +> sin (#8). Removed in full: module, registry, verb, ts-rs bindings, registration. +> +> **Why I missed it:** I searched for the CONCEPT in my head (`ContentSource`, handle, +> registry). The real thing is named for its JOB (`spill`, `tool/output`) — which is what +> a good name looks like, and exactly what my search could not find. See +> [[read-the-code-you-intend-to-replace-before-designing-its-replacement]]. +> +> **What survives from the interface sketch below:** it is the right SHAPE for spill to +> grow into if `tool/output` ever needs a second implementation (a RAG source, a positron +> ViewState, a peer's artifact). Kept here as prose, deliberately not as code — an +> unimplemented trait with no consumers is itself a confusing leftover. --- @@ -186,18 +219,25 @@ Only the renderer does. Which is the test for any future work in this area: if it would have to be redesigned when the one-prompt constraint lifts, it is encoding the constraint instead of the intent. -## Build order - -1. **Reconcile the two handle models** (task #17) — `runtime::cell_shapes::HandleRef` and - the URI-handle form. One type, or this design forks on day one. -2. **`ContentSource` in the substrate**, with the file engine as outlier A (coordinates - matter, refuses to be cut) and a RAG source as outlier B (maximally different: no - coordinates, already a projection). Per CLAUDE.md's outlier rule — if both fit without - forcing, the interface is proven. -3. **`ToolResult` carries `header + handle`** instead of a `String` body. -4. **A dereference verb** in the persona tool surface, so a citizen can page a handle. -5. **Delete the reducers.** The recency fold and the recent-results tail-keep both go; - with handles there is nothing left for them to do. +## Build order — REVISED after the duplicate was deleted + +1. ~~Reconcile the two handle models (#17)~~ — moot for now. `spill` uses a + content-addressed hex stem; nothing else competes with it, because the competitor was + deleted. #17 only becomes live again if a SECOND `ContentSource` implementation + appears and needs to share a handle type with spill. +2. ~~`ContentSource` in the substrate~~ — **do not build this speculatively.** Grow it out + of `spill` at the moment a second implementation actually exists, so the trait is + extracted from two real cases rather than imagined from zero (CLAUDE.md's outlier + rule, applied honestly). +3. **The remaining real defect:** `working_memory`'s `recent_results_chars` re-cuts a + result that the executor ALREADY bounded and handle-backed, severing the + "your full output is saved as ``, page it with `tool/output`" sentence the + executor wrote. The citizen is told how to recover her output and the budget layer + cuts the telling off. Fixing it properly means `fold_with_recovery` returning + `{preview, Option}` instead of a String with the handle baked into prose, + so the handle travels as DATA to working memory and the render layer cannot cut it. +4. **Then** the recency fold and recent-results tail-keep can go, because a handle-backed + result has nothing left worth re-cutting. ## Forbidden moves diff --git a/protocol/typescript/content/ContentFetchParams.ts b/protocol/typescript/content/ContentFetchParams.ts deleted file mode 100644 index 88346bc08..000000000 --- a/protocol/typescript/content/ContentFetchParams.ts +++ /dev/null @@ -1,16 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. - -export type ContentFetchParams = { -/** - * The handle's id, from the header you were given. - */ -handle: string, -/** - * First unit to read, 1-based, in the units the header named (line, entry). - * Defaults to the beginning. - */ -from: number, -/** - * How many units. The source clamps to what it has and tells you what it covered. - */ -count: number, }; diff --git a/protocol/typescript/content/ContentFetchResult.ts b/protocol/typescript/content/ContentFetchResult.ts deleted file mode 100644 index 5c9ba5c7e..000000000 --- a/protocol/typescript/content/ContentFetchResult.ts +++ /dev/null @@ -1,20 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. - -export type ContentFetchResult = { -/** - * The content, whole in its own units. - */ -body: string, -/** - * First unit this covers, after clamping. - */ -from: number, -/** - * How many units this covers, after clamping. - */ -count: number, -/** - * Where to continue, or `null` when you have reached the end. `null` is the signal - * that you have seen ALL of it — the thing a truncated copy can never tell you. - */ -next_from?: number, }; diff --git a/protocol/typescript/content/ContentHeader.ts b/protocol/typescript/content/ContentHeader.ts deleted file mode 100644 index e1c9f957f..000000000 --- a/protocol/typescript/content/ContentHeader.ts +++ /dev/null @@ -1,30 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. -import type { Extent } from "./Extent"; - -/** - * What a piece of content IS, small enough to always fit in a prompt. - * - * This is what a consumer gets instead of the content. It must be sufficient to decide - * whether to fetch more and how — so it names the extent in the source's OWN units - * (lines, entries, bytes) rather than a byte count nobody can act on. - */ -export type ContentHeader = { -/** - * What this is, in the producer's words: `"file"`, `"directory listing"`, - * `"test run output"`. Free text on purpose — a closed enum here would be a central - * list every new source has to be added to. - */ -kind: string, -/** - * One line a reader can act on: `"18 files under astropy/io/fits"`. - */ -summary: string, -/** - * How much there is, in the source's own units. - */ -extent: Extent, -/** - * The exact call that fetches more. Stated by the SOURCE so a consumer never has to - * guess the parameter name — the difference between a usable refusal and a burnt turn. - */ -fetch_with: string, }; diff --git a/protocol/typescript/content/Extent.ts b/protocol/typescript/content/Extent.ts deleted file mode 100644 index ac68e0a02..000000000 --- a/protocol/typescript/content/Extent.ts +++ /dev/null @@ -1,6 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. - -/** - * How much content there is, counted the way its own source counts it. - */ -export type Extent = { "Lines": { total: number, } } | { "Entries": { total: number, } } | { "Whole": { bytes: number, } }; diff --git a/protocol/typescript/content/Slice.ts b/protocol/typescript/content/Slice.ts deleted file mode 100644 index 00144f34d..000000000 --- a/protocol/typescript/content/Slice.ts +++ /dev/null @@ -1,21 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. -import type { Span } from "./Span"; - -/** - * Part of a source's content, plus where the reader is in it. - */ -export type Slice = { -/** - * The content itself, whole in its own units — never cut mid-unit. - */ -body: string, -/** - * What this slice covers, after clamping. - */ -covered: Span, -/** - * The next span, when there is more. `None` means the reader has reached the end — - * which is how she knows she has seen everything, a thing a truncated copy can never - * tell her. - */ -next: Span | null, }; diff --git a/protocol/typescript/content/Span.ts b/protocol/typescript/content/Span.ts deleted file mode 100644 index 0fcc411d3..000000000 --- a/protocol/typescript/content/Span.ts +++ /dev/null @@ -1,15 +0,0 @@ -// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. - -/** - * A request for part of a source's content, in that source's units. - */ -export type Span = { -/** - * First unit wanted, 1-based. - */ -from: number, -/** - * How many units. The source clamps to what it has and REPORTS the clamp — it never - * silently returns less than asked without saying so. - */ -count: number, }; From 4b785b9f925280e3feff68c8254fd3ad0797eb8f Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 14:18:37 -0500 Subject: [PATCH 49/80] refactor(cognition): retire four superseded symbols whose docs still explained the LIVE behaviour (#124) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four dead-code warnings on the cognition path, all the same shape: a function superseded by a better sibling, kept alive by a doc comment, with the rationale for the LIVE behaviour attached to the corpse. Reading them one at a time rather than silencing them found one real documentation lie. 1. NEAR_DUP_MIN_RUN — the actual defect. The constant was dead while its doc described a RETIRED algorithm ("consecutive near-identical PAIRS, so 3 = four messages in a row"). The live rule is a CLUSTER of >=3 mutually-similar messages at any period — a different meaning — expressed as a bare `3` at two call sites. Two live doc comments cited the dead name to explain behaviour it no longer governed. Renamed to NEAR_DUP_MIN_CLUSTER with the cluster semantics its own detection comment already documents, and both `best >= 3` sites now derive from it (own_repetition_fact + template_loop_fact, same calibrated threshold, same family that already shares NEAR_DUP_JACCARD). De-hardcode-guard clean: the name carries no WINDOW|CONTEXT|CTX|TOKEN|PROMPT|CHARS token, and it is a calibration constant of the same class as NEAR_DUP_JACCARD beside it. 2. turn_message_line — superseded by turn_message_line_addressed, which duplicates both its branches rather than calling it. The sharpest instance: the dead function's doc claimed "The ONE place message-line formatting lives" — the compression principle asserted on the copy that no longer holds it. Deleted; the glass-boxed rationale for WHY addressing is rendered (2026-07-10, Atlas answering as the implementer after Asha addressed Anwen) moved onto the live function. 3. vocative_addressee — the singular, superseded by vocative_addressees. Same inversion: the vocative GRAMMAR doc (leading vs greeting form, the punctuation word-boundary) sat on the dead singular while the live plural carried a stub. Merged the grammar onto the plural, deleted the singular, converted its one test caller. 4. deliberation_prompt::compose — NOT dead, and I nearly deleted it wrongly. The warning fires from the LIB target, where #[cfg(test)] code is not compiled; it has 7 callers in the file's own test mod, so its doc's "kept so tests are unchanged" was accurate. Restored under #[cfg(test)] so the gate states its real audience, with a doc explaining why the whole-string form earns its keep (content assertions) and why it cannot drift (it is compose_split's own output concatenated). Repointed llm_deliberation_faculty's #266 KV-cache comment from `compose` to stable_blocks/volatile_blocks, the functions that actually own the block order it describes. Net: one stale-semantics constant fixed, two bare literals de-hardcoded, three rationale docs moved from dead code onto the live code they explain, one wrong deletion caught before it shipped. cargo check --tests clean of all four; 62 deliberation tests + 7 repetition/vocative tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/deliberation_budget.rs | 58 ++++++++----------- .../src/cognition/deliberation_prompt.rs | 14 +++-- .../src/cognition/llm_deliberation_faculty.rs | 3 +- 3 files changed, 37 insertions(+), 38 deletions(-) diff --git a/core/continuum-core/src/cognition/deliberation_budget.rs b/core/continuum-core/src/cognition/deliberation_budget.rs index 94fa3983c..ca4a14c77 100644 --- a/core/continuum-core/src/cognition/deliberation_budget.rs +++ b/core/continuum-core/src/cognition/deliberation_budget.rs @@ -63,17 +63,9 @@ pub(super) fn tail_to_tokens(s: &str, budget_tokens: usize) -> String { /// geometry against known participant names, never content NLP /// ([[no-hardcoded-heuristics-to-steer-cognition]] — this renders a fact visible, /// it steers nothing). -pub(super) fn turn_message_line(turn: &BurstTurn) -> String { - if turn.is_self || turn.author.is_empty() { - turn.content.clone() - } else { - format!("{}: {}", turn.author, turn.content) - } -} - -/// [`turn_message_line`] with addressee annotation for peer turns. `participants` -/// is every display name known in the window (peers + self); `self_name` is THIS -/// persona's name, rendered as "you" so a directed ask reads as directed. +/// `participants` is every display name known in the window (peers + self); +/// `self_name` is THIS persona's name, rendered as "you" so a directed ask reads +/// as directed. pub(super) fn turn_message_line_addressed( turn: &BurstTurn, participants: &[String], @@ -121,15 +113,18 @@ pub(super) fn turn_message_line_addressed( /// ([[no-hardcoded-heuristics-to-steer-cognition]]). pub(super) const NEAR_DUP_JACCARD: f64 = 0.6; -/// Minimum run length (consecutive near-identical PAIRS, so `3` = four messages -/// in a row) before the repetition fact renders. Same calibration: the longest -/// observed live runs were 36 and 80 messages; under healthy flow three -/// consecutive ≥0.6 pairs is vanishingly rare. Evidence-scaled: below this, say -/// nothing. -const NEAR_DUP_MIN_RUN: usize = 3; +/// Minimum CLUSTER size — how many of her own visible messages must be mutually +/// near-identical (≥ [`NEAR_DUP_JACCARD`]) before the repetition fact renders. +/// Counted anywhere in the window, at any period, NOT as a consecutive run: live +/// loops cycle 2–3 templates, so a trailing-run rule went blind to exactly the +/// loops it was built for (see [`own_repetition_fact`]). Same calibration corpus: +/// the longest observed live loops were 36 and 80 messages; under healthy flow +/// three mutually ≥0.6 messages is vanishingly rare. Evidence-scaled: below this, +/// say nothing. +const NEAR_DUP_MIN_CLUSTER: usize = 3; /// How many of her own recent utterances the spoken ring retains — the -/// repetition detector's self-history window. Sized past NEAR_DUP_MIN_RUN +/// repetition detector's self-history window. Sized past NEAR_DUP_MIN_CLUSTER /// with slack for interleaved non-loop turns; utterances are short-lived /// evidence, not memory (the hippocampus owns memory). const OWN_SPEECH_RING: usize = 8; @@ -285,8 +280,10 @@ pub(super) fn jaccard(a: &str, b: &str) -> f64 { /// The persona's OWN-SPEECH repetition fact for this tick, if her trailing run /// of own turns is a loop: `Some("[repetition] your last N messages were nearly -/// identical")` when the last [`NEAR_DUP_MIN_RUN`]+ consecutive own turns are -/// pairwise ≥ [`NEAR_DUP_JACCARD`] similar. Pure fact, no imperative — perception +/// identical")` when [`NEAR_DUP_MIN_CLUSTER`]+ of her own visible turns are +/// mutually ≥ [`NEAR_DUP_JACCARD`] similar — a CLUSTER at any period, not a +/// trailing run (see the detection comment below, and the live deploy that proved +/// the run form blind). Pure fact, no imperative — perception /// renders what happened; it never steers what she says next. /// /// Why (task #134, glass-boxed 2026-07-11): Atlas looped stage-direction @@ -343,7 +340,7 @@ pub(super) fn own_repetition_fact(turns: &[BurstTurn], spoken: &[String]) -> Opt // ALREADY has (her Silence Option prompt: "Choose PASS when … nothing new has // been raised"), surfaced at the moment repetition is structurally detected. The // fork (add something genuinely new, OR go silent) is hers; this only names it. - (best >= 3).then(|| { + (best >= NEAR_DUP_MIN_CLUSTER).then(|| { format!( "[repetition] {best} of your recent messages were nearly identical — you're \ circling, and restating what you've already said adds nothing. If you have \ @@ -451,7 +448,7 @@ pub(super) fn template_loop_fact(turns: &[BurstTurn], spoken: &[String]) -> Opti .count(); best = best.max(dups + 1); } - (best >= 3).then(|| { + (best >= NEAR_DUP_MIN_CLUSTER).then(|| { format!( "[template-loop] {best} of your recent messages reuse the same template with \ the topic swapped — a new subject inside the same scaffold is still circling, \ @@ -674,8 +671,8 @@ fn matches_name_at(line: &str, pos: usize, name: &str) -> bool { .is_some_and(|s| s.eq_ignore_ascii_case(name)) } -/// Find WHO a message's first line addresses, by vocative GEOMETRY only — never -/// content interpretation. Two shapes, matched against known participant names: +/// Find WHO a message addresses, by vocative GEOMETRY only — never content +/// interpretation. Two shapes, matched against known participant names: /// /// - **Leading vocative**: `Anwen, …` / `Atlas: …` / `Asha — …` / `@Anwen …` /// - **Greeting vocative** in the first line: `Sure, Anwen. Could you…` / @@ -685,13 +682,8 @@ fn matches_name_at(line: &str, pos: usize, name: &str) -> bool { /// /// A bare mention ("I agree with Anwen's plan") matches neither shape and stays /// unannotated. Leading beats greeting; among greetings the earliest wins. -pub(super) fn vocative_addressee<'a>(content: &str, participants: &'a [String]) -> Option<&'a str> { - vocative_addressees(content, participants) - .into_iter() - .next() -} - -/// Every addressee the message's vocative geometry names, in discovery order, +/// +/// Returns every addressee the geometry names, in discovery order, /// deduped, capped at 3. The LEADING form is scanned on every line (live /// coordination messages address several teammates on separate lines — /// "Atlas, please test… / Asha, could you…" — #134 specimen 2 was missed by @@ -1442,8 +1434,8 @@ mod tests { // (the closing-punctuation requirement doubles as the word boundary). let names2 = vec![SPEAKER_LEAD.to_string()]; assert_eq!( - vocative_addressee("Sure, Anwenne. Please post it.", &names2), - None + vocative_addressees("Sure, Anwenne. Please post it.", &names2), + Vec::<&str>::new() ); // Name-AGNOSTIC proof: personas are procedurally generated, so the diff --git a/core/continuum-core/src/cognition/deliberation_prompt.rs b/core/continuum-core/src/cognition/deliberation_prompt.rs index 604d6c3ea..09b38dec6 100644 --- a/core/continuum-core/src/cognition/deliberation_prompt.rs +++ b/core/continuum-core/src/cognition/deliberation_prompt.rs @@ -104,10 +104,16 @@ pub(super) fn compose_split(p: &SystemPromptParts<'_>) -> ComposedSystemPrompt { } /// Assemble the WHOLE system prompt as one string — `stable ++ trailing`, byte-identical -/// to the pre-split output. Kept so callers/tests that want the composed whole are -/// unchanged; the live message builder uses [`compose_split`] to place `stable` in the -/// cacheable system message and `trailing` on the newest turn (the #266 KV-reuse fix). -pub(super) fn compose(p: &SystemPromptParts<'_>) -> String { +/// to the pre-split output. +/// +/// TEST-ONLY, and gated so it says so: production has exactly one composer, +/// [`compose_split`], which places `stable` in the cacheable system message and `trailing` +/// on the newest turn (the #266 KV-reuse fix). This exists because the assertions below +/// are about prompt CONTENT — what appears, in what order — which is easier to state +/// against the whole string than against the halves. It is not a second composer: it is +/// [`compose_split`]'s own output concatenated, so it cannot drift from it. +#[cfg(test)] +fn compose(p: &SystemPromptParts<'_>) -> String { let c = compose_split(p); let mut s = c.stable; s.push_str(&c.trailing); diff --git a/core/continuum-core/src/cognition/llm_deliberation_faculty.rs b/core/continuum-core/src/cognition/llm_deliberation_faculty.rs index 9d0f83b31..bfbffde21 100644 --- a/core/continuum-core/src/cognition/llm_deliberation_faculty.rs +++ b/core/continuum-core/src/cognition/llm_deliberation_faculty.rs @@ -1281,7 +1281,8 @@ impl LlmDeliberationFaculty { ), ); - // #266 KV-cache fix lives in the block ORDER (see `deliberation_prompt::compose`): + // #266 KV-cache fix lives in the block ORDER (see `deliberation_prompt::stable_blocks` + // and `volatile_blocks`, assembled by `compose_split`): // the per-turn presence/own-time framing now renders LAST in the system message, // AFTER the standing grounding context, instead of before it. The raw // prompt-captures caught the framing sitting at char ~7607, ahead of the context, From e7120276119a75cd7b02c797b3457cc47269ef10 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 14:21:28 -0500 Subject: [PATCH 50/80] refactor(benchmark): kill a duplicate SWE task prompt and a fake "default" model id (#124) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two dead-code warnings that sit on the benchmark path Joel named. Both were symbols whose DOCS asserted a production role they no longer had. 1. `benchmark::swe_task_prompt` — a second copy of the SWE task text. Its guidance ("do not create a new workspace and do not add new top-level files … fix it IN PLACE") now lives in the dispatch CARD BODY (`BenchmarkSweSetup`), which is the correct home: benchmarks are adapters into recipes/activities, so the CARD is the task. Two copies of one instruction is the parallel-allocator sin; the dead copy went. What made this worth reading rather than silencing: THREE comments in `agent/solve.rs` cite `swe_task_prompt` by name as the live inner half of a documented framing CONTRADICTION — the outer `frame_task` contract once claimed the grade was about "the files your tools WRITE" while the inner SWE text said "edit in place", and she obeyed the outer one across three full-effort sympy-21379 runs (v3/v4/v5: 6 new repro scripts, 0 edits). That history is load-bearing — it also records the FORCE-vs-SHAPE lesson from v6, where removing the contradiction and the tool-forcing pressure together produced 8 acts and ZERO files. All three now point at `benchmark::BenchmarkSweSetup`, so the next reader can actually open both halves of the framing they are being warned about. The regression test that guards it (`the_generic_framing_never_dictates_the_deliverable_shape`) was live and correct throughout — only the symbol its comment named had moved. 2. `DEFAULT_GENERATE_MODEL` — a hardcoded model id documented as "Default model when caller doesn't override", with ZERO production callers. The comment on the constant directly above it explains why: production deliberately STOPPED binding a hardcoded model id, because one the gateway does not serve hard-fails `select()`; the turn now binds the discovered served model via a handle. So the doc named a production default that production had removed on purpose. Renamed `TEST_GENERATE_MODEL` and `#[cfg(test)]`-gated so it cannot drift back into being one, with the reason stated on the constant. #124: a hardcoded model name that can only be reached from tests is the harmless end of that audit, but only once the gate says so. cargo check --tests clean; 61 swe_setup/generate_response/framing tests green plus the named framing regression. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/generate_response.rs | 16 ++++++++++----- .../src/commands/agent/solve.rs | 20 +++++++++++-------- core/continuum-core/src/commands/benchmark.rs | 12 ----------- 3 files changed, 23 insertions(+), 25 deletions(-) diff --git a/core/continuum-core/src/cognition/generate_response.rs b/core/continuum-core/src/cognition/generate_response.rs index c616fd9f7..3f01d1808 100644 --- a/core/continuum-core/src/cognition/generate_response.rs +++ b/core/continuum-core/src/cognition/generate_response.rs @@ -69,8 +69,14 @@ const HOUR_GAP_THRESHOLD_MS: u64 = 60 * 60 * 1000; // via a handle. One source of truth for the gateway id: `llama_server::PROVIDER_ID`. const DEFAULT_GENERATE_PROVIDER: &str = crate::inference::llama_server::PROVIDER_ID; -/// Default model when caller doesn't override. -const DEFAULT_GENERATE_MODEL: &str = "continuum-ai/qwen3.5-4b-code-forged-GGUF"; +/// A model id for tests to pass explicitly — NOT a production default, and gated so it +/// cannot become one. There is deliberately no default model: per the note on +/// [`DEFAULT_GENERATE_PROVIDER`] above, the turn binds to whatever the gateway actually +/// serves (discovered, via a handle), because a hardcoded id the gateway does not serve +/// hard-fails `select()`. This constant only spares the request-shaping tests from +/// repeating a literal. +#[cfg(test)] +const TEST_GENERATE_MODEL: &str = "continuum-ai/qwen3.5-4b-code-forged-GGUF"; /// Default sampling temperature: moderate /// creativity for natural-language responses. @@ -1200,12 +1206,12 @@ mod tests { fn generation_request_uses_documented_defaults() { let request = request_with_overrides(None, None, None, None); let inference = - build_response_generation_request(&request, DEFAULT_GENERATE_MODEL.to_string(), 0); + build_response_generation_request(&request, TEST_GENERATE_MODEL.to_string(), 0); assert_eq!( inference.provider.as_deref(), Some(DEFAULT_GENERATE_PROVIDER) ); - assert_eq!(inference.model.as_deref(), Some(DEFAULT_GENERATE_MODEL)); + assert_eq!(inference.model.as_deref(), Some(TEST_GENERATE_MODEL)); assert_eq!(inference.temperature, Some(DEFAULT_GENERATE_TEMPERATURE)); // No override + no client default = the model owns its length. assert_eq!(inference.max_tokens, None); @@ -1243,7 +1249,7 @@ mod tests { let request = request_with_overrides(None, None, None, None); let inference = build_response_generation_request( &request, - DEFAULT_GENERATE_MODEL.to_string(), + TEST_GENERATE_MODEL.to_string(), 1_700_000_000_000, ); let identity = match &inference.messages.last().expect("identity present").content { diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index 31a75d93c..cedd548f7 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -1263,9 +1263,10 @@ impl AgentSolve { // // The old text said "writing files with code/write" and "graded on the files your tools // WRITE". That was written for from-scratch build gyms, where new files ARE the - // deliverable. Nested beneath it, `swe_task_prompt` says the opposite: "do not add new - // top-level files — fix it IN PLACE with code/edit. The fix must land in the existing - // files." + // deliverable. Nested beneath it, the SWE task text says the opposite: "do not add new + // top-level files … find the existing source of the fault and edit it in place." That + // text is the dispatch CARD BODY (`benchmark::BenchmarkSweSetup`) — the card IS the + // task, so the card owns the deliverable shape. // // Outer contract first, inner constraint buried under "Task:" — and she obeyed the // outer one. Three consecutive sympy-21379 runs, all full-effort, all writing NEW files @@ -1943,9 +1944,11 @@ crate::register_stateless_command!(AgentSolve); /// deliverable looks like — the task owns that, and the two used to contradict each other. /// /// The old text said "writing files with code/write" and "graded on the files your tools WRITE", -/// which is right for a from-scratch build gym. Nested beneath it, `swe_task_prompt` says the -/// opposite: "do not add new top-level files — fix it IN PLACE with code/edit. The fix must land -/// in the existing files." Outer contract first, inner constraint under "Task:" — and she obeyed +/// which is right for a from-scratch build gym. Nested beneath it, the SWE task text says the +/// opposite: "do not add new top-level files … find the existing source of the fault and edit it +/// in place" — that text is the dispatch CARD BODY (`benchmark::BenchmarkSweSetup`), which owns +/// the deliverable shape because the card IS the task. +/// Outer contract first, inner constraint under "Task:" — and she obeyed /// the outer one. Three consecutive full-effort sympy-21379 runs wrote NEW repro scripts and never /// edited the library (v3: 1 file, v4: 3 files, v5: 2 files; 0 edits every time). That read as a /// judgement gap for a whole session; it was two halves of one framing disagreeing. @@ -2027,8 +2030,9 @@ mod tests { // what this catches: the wrapper asserting a DELIVERABLE SHAPE that the task contradicts. // The generic framing exists to kill narration ("only tool calls take effect"). It must not - // also claim the grade is about "files your tools WRITE" — `swe_task_prompt` says the - // opposite ("do not add new top-level files … fix it IN PLACE with code/edit"), and the + // also claim the grade is about "files your tools WRITE" — the SWE dispatch card body + // (`benchmark::BenchmarkSweSetup`) says the opposite ("do not add new top-level files … + // edit it in place"), and the // wrapper comes FIRST. Three consecutive sympy-21379 runs obeyed the wrapper and wrote new // repro scripts instead of editing the library. The contract may describe HOW acts take // effect; only the task may describe WHAT to change. diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index df542c3b6..474e55096 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2329,18 +2329,6 @@ mod swe_grade_tests { } } -/// The task text handed to her. Deliberately says WHERE she is and what "done" means — the -/// glass-boxed failure it prevents is her creating a new project beside the repo, or leaving -/// the fix in a message instead of the files. -fn swe_task_prompt(problem_statement: &str) -> String { - format!( - "You are ALREADY in the task's workspace: a real git repository with a real bug. Do not \ - create a new workspace and do not add new top-level files — find the existing source \ - with code/search and code/read, and fix it IN PLACE with code/edit. Run checks with \ - code/shell if useful. The fix must land in the existing files.\n\nISSUE:\n{problem_statement}" - ) -} - /// The continuum home dir (`$CONTINUUM_HOME` else `~/.continuum`) — the same /// resolution the dispatch workspace + progress ledger use. `pub(crate)` so the /// curriculum drain (`genome/teach --from-experience`) resolves the SAME citizen From bd6d5aba0d30f5b0dcf88a564fbfe6d089621ed7 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 14:23:18 -0500 Subject: [PATCH 51/80] refactor(tools): delete the superseded brace-scanner in the tool-call parser (#124) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `json_object_candidates` was dead in the tool-call parser — worth reading rather than silencing, because a dead function in THAT file could have been a severed parser arm (#343 family), and a citizen whose tool call does not lift burns a whole turn. It is not. `scan_objects` (directly above it, live, generic over a parse closure) supersedes it strictly: same brace-depth scan, same `matching_brace_end` helper, same outermost-first-at-each-start property the dead one's doc claims — a `{"tool_call": {...}}` envelope is still tried before its inner `{...}`, and on a parse failure `i += 1` walks into the inner brace. `scan_objects` additionally fuses the scan with the parse attempt and skips past a CONSUMED object so its insides are not re-scanned, which the candidate-list form could not express. No idiom loses coverage. I first "fixed" this by renaming it with an `#[allow(dead_code)]` — which is exactly the suppression CLAUDE.md forbids, and I caught it in the same turn. Deleted instead. 60 json_in_prompt_tools tests green, including `emission_idiom_corpus_lifts_every_live_shape` (the corpus test over every live emission shape) and the narrated/assignment-bound/past-tense-claim arms. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/ai/json_in_prompt_tools.rs | 21 ------------------- 1 file changed, 21 deletions(-) diff --git a/core/continuum-core/src/ai/json_in_prompt_tools.rs b/core/continuum-core/src/ai/json_in_prompt_tools.rs index d55ca1092..c704d188e 100644 --- a/core/continuum-core/src/ai/json_in_prompt_tools.rs +++ b/core/continuum-core/src/ai/json_in_prompt_tools.rs @@ -2087,27 +2087,6 @@ where out } -/// Yield substrings of `text` that are balanced `{...}` objects, outermost-first -/// at each start position — so a `{"tool_call": {...}}` envelope is tried before -/// its inner `{...}`. Brace-depth scan that ignores braces inside JSON strings -/// (so `{"k":"}"}` doesn't fool it). Cheap; the candidate set is tiny in practice. -fn json_object_candidates(text: &str) -> Vec<&str> { - let bytes = text.as_bytes(); - let mut out = Vec::new(); - let mut i = 0; - while i < bytes.len() { - if bytes[i] == b'{' { - if let Some(end) = matching_brace_end(bytes, i) { - out.push(&text[i..=end]); - // Continue scanning AFTER this object's open brace so nested/later - // objects are still considered, but we tried the outermost first. - } - } - i += 1; - } - out -} - /// Index of the `}` matching the `{` at `start`, respecting JSON string literals /// and escapes. `None` if unbalanced (truncated output). fn matching_brace_end(bytes: &[u8], start: usize) -> Option { From e14e0fe236914d2ba6ed24498827bff37b8ac972 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 15:52:08 -0500 Subject: [PATCH 52/80] =?UTF-8?q?fix(benchmark):=20the=20grader=20could=20?= =?UTF-8?q?not=20read=20colorized=20pytest=20output=20=E2=80=94=20every=20?= =?UTF-8?q?astropy=20instance=20scored=200=20of=20N=20and=20was=20blamed?= =?UTF-8?q?=20on=20the=20environment?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LIVE, TODAY, on a real citizen patch. Atlas edited `astropy/nddata/mixins/ndarithmetic.py` in her staged workspace — a genuine in-place fix for the NDDataRef mask-propagation bug. Grading it (and its own GOLD patch) returned: UNGRADEABLE — PASS_TO_PASS passes 0 of 40 on the PRISTINE tree: the suite does not run in this environment, so every score from this tree is an env fault, never a capability verdict. The environment was fine. The very same output's tail reads `1 failed, 179 passed in 0.20s`, and the single failure is `test_nddata_bitmask_arithmetic` — the FAIL_TO_PASS test, i.e. exactly correct pristine behaviour. The suite ran. The grader could not READ it. ROOT CAUSE, from astropy's own config: `setup.cfg:127` carries `addopts = --color=yes`. pytest normally suppresses color on a pipe; that line forces it on anyway. Result lines then read …test_ndarithmetic.py::\x1b[1mtest_y\x1b[0m \x1b[32mPASSED\x1b[0m so the escape sits INSIDE the node id and immediately before the verdict. `parse_pytest_report` splits on whitespace and matches `verdict.starts_with("PASSED")` — which now starts with `\x1b`. Every line hit `continue`, both maps came back empty, and empty maps read as "0 of 40 passed" → "the environment is broken". FIX, two layers, and the ORDER matters: 1. PARSER (load-bearing): `strip_ansi` — Cow-based, so the common uncolored case borrows and allocates nothing. Handles CSI/SGR and OSC (BEL- and ST-terminated), and truncated escapes at end-of-input. Applied in `parse_pytest_report` AND `parse_django_report`. This is the durable fix because a repo can force color from its own config: it needs no flag, works on already-captured reports, and does not depend on any pytest version. 2. RUNNER (belt): append `--color=no`. CLI args come after `addopts`, so the later flag wins over the repo's `--color=yes`. Deliberately the SECONDARY fix — the surrounding code warns that flags must be era-portable, and if some ancient pytest ever rejected this one, layer 1 still guarantees correct grading. WHAT THIS UNBLOCKS: astropy is 6 of Atlas's 23 staged instances, and every astropy run on the board is `quiet`/`ungradeable`. Their scores were never capability verdicts. Re-grading is the next step; this commit is the instrument, not yet the number. Tests: `a_colorized_report_parses_exactly_like_a_plain_one` asserts a colorized report yields byte-identical maps to the plain one, using the real escape shape pytest emits, and pins that the map is non-empty (the exact failure that got misreported as a broken env). Plus `strip_ansi_is_borrow_only_when_clean_and_lossless_when_not` for the borrow-vs-own path, OSC termination both ways, and truncated-escape safety. 19 swe_bench tests green; the pinned pytest argv test updated to include the new flag. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 125 +++++++++++++++++- 1 file changed, 124 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 3c151a276..90852bdf4 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -1613,6 +1613,12 @@ pub fn test_invocation_with( args.extend(test_files.iter().cloned()); // Flags must be era-portable — see the comment at the call site (`run_tests`). args.extend(["-v".into(), "-p".into(), "no:cacheprovider".into()]); + // Belt to [`strip_ansi`]'s braces: ask for no color at the SOURCE. A repo can + // force `--color=yes` from its own `setup.cfg` addopts (astropy does), and CLI + // args are appended after addopts so the later `--color=no` wins. This is the + // nicety — the parser's strip is what actually guarantees correctness, which is + // why an ancient pytest that rejected this flag still could not break grading. + args.push("--color=no".into()); args } TestRunner::DjangoRuntests => { @@ -1718,6 +1724,8 @@ pub fn parse_django_report(report: &str) -> (HashMap, HashMap = HashMap::new(); // A test-id line awaiting its outcome on a following docstring line: (func, class_norm). let mut pending: Option<(String, String)> = None; + // Same reason as the pytest parser — a colorized runner must not become "0 of N". + let report = strip_ansi(report); for line in report.lines() { let line = line.trim(); let Some((head, tail)) = line.split_once(" ... ") else { @@ -1782,14 +1790,73 @@ pub fn parse_django_report(report: &str) -> (HashMap, HashMap std::borrow::Cow<'_, str> { + if !s.contains('\u{1b}') { + return std::borrow::Cow::Borrowed(s); + } + let mut out = String::with_capacity(s.len()); + let mut chars = s.chars(); + while let Some(c) = chars.next() { + if c != '\u{1b}' { + out.push(c); + continue; + } + match chars.next() { + // CSI: params/intermediates, then a final byte in @..~ ends the sequence. + Some('[') => { + for f in chars.by_ref() { + if ('\u{40}'..='\u{7e}').contains(&f) { + break; + } + } + } + // OSC: runs until BEL or the ST pair (ESC \). + Some(']') => { + while let Some(f) = chars.next() { + if f == '\u{7}' { + break; + } + if f == '\u{1b}' { + // ST is ESC \ — consume the backslash and stop. + let _ = chars.next(); + break; + } + } + } + // Any other two-byte escape: drop both bytes. + Some(_) => {} + None => break, + } + } + std::borrow::Cow::Owned(out) +} + /// Resolve pytest's `-v` report into a verdict per node id AND per bare function name. /// /// The dataset does not use one id shape. pytest and flask instances give node ids /// (`tests/test_x.py::test_y`); sympy gives BARE function names because sympy ships its own /// runner. Looking up both is what makes one grader serve every repo. +/// +/// Color-forcing repos are handled by [`strip_ansi`] before matching — see its doc for the +/// astropy incident this prevents. pub fn parse_pytest_report(report: &str) -> (HashMap, HashMap) { let mut by_node = HashMap::new(); let mut by_func: HashMap = HashMap::new(); + let report = strip_ansi(report); for line in report.lines() { let line = line.trim(); let Some((node, rest)) = line.split_once(char::is_whitespace) else { @@ -2544,6 +2611,61 @@ tests/test_x.py::TestC::test_param[3-4] PASSED"; assert_eq!(verdict_for("test_never_ran", &by_node, &by_func), None); } + // what this catches: a COLORIZED report scoring 0 of N and being blamed on the environment. + // Live incident 2026-08-18 (astropy-14995): astropy's own setup.cfg carries + // `addopts = --color=yes`, so pytest emitted color onto a pipe. The escape lands INSIDE the + // node id and before the verdict, `starts_with("PASSED")` never matched, and the grader + // reported "UNGRADEABLE — PASS_TO_PASS passes 0 of 40 on the PRISTINE tree: the suite does + // not run in this environment" — in a run whose own tail read `1 failed, 179 passed`. The + // env was fine; the grader could not read it. Bytes below are the real shape pytest emits. + #[test] + fn a_colorized_report_parses_exactly_like_a_plain_one() { + let plain = "\ +astropy/nddata/mixins/tests/test_ndarithmetic.py::test_nddata_bitmask_arithmetic FAILED +astropy/nddata/mixins/tests/test_ndarithmetic.py::test_arithmetics_data PASSED"; + let colored = "\ +astropy/nddata/mixins/tests/test_ndarithmetic.py::\u{1b}[1mtest_nddata_bitmask_arithmetic\u{1b}[0m \u{1b}[31mFAILED\u{1b}[0m +astropy/nddata/mixins/tests/test_ndarithmetic.py::\u{1b}[1mtest_arithmetics_data\u{1b}[0m \u{1b}[32mPASSED\u{1b}[0m"; + + let (plain_node, plain_func) = parse_pytest_report(plain); + let (color_node, color_func) = parse_pytest_report(colored); + assert_eq!( + color_node, plain_node, + "color must not change which node ids were seen" + ); + assert_eq!(color_func, plain_func, "nor the bare-name verdicts"); + + // The specific thing that was returning "0 of N": a PASS must read as a pass. + assert_eq!( + verdict_for("test_arithmetics_data", &color_node, &color_func), + Some(true) + ); + assert_eq!( + verdict_for("test_nddata_bitmask_arithmetic", &color_node, &color_func), + Some(false) + ); + assert!( + !color_node.is_empty(), + "an empty map is the failure mode that got misreported as a broken environment" + ); + } + + // what this catches: strip_ansi corrupting ordinary text, or allocating when it need not. + #[test] + fn strip_ansi_is_borrow_only_when_clean_and_lossless_when_not() { + assert!(matches!( + strip_ansi("plain text :: PASSED"), + std::borrow::Cow::Borrowed(_) + )); + assert_eq!(strip_ansi("\u{1b}[32mPASSED\u{1b}[0m"), "PASSED"); + // OSC (title-set) sequences terminate on BEL or ST, and swallow neither more nor less. + assert_eq!(strip_ansi("a\u{1b}]0;title\u{7}b"), "ab"); + assert_eq!(strip_ansi("a\u{1b}]0;title\u{1b}\\b"), "ab"); + // A truncated escape at end-of-input must not panic or emit garbage. + assert_eq!(strip_ansi("tail\u{1b}"), "tail"); + assert_eq!(strip_ansi("tail\u{1b}["), "tail"); + } + // what this catches: a bare name appearing in two files where one fails — counting it as // passing would let real breakage through. #[test] @@ -2720,8 +2842,9 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. "-v", "-p", "no:cacheprovider", + "--color=no", ], - "the pytest argv is byte-identical to the pre-seam grader" + "the pytest argv pins the era-portable flag set, color explicitly OFF" ); } From 59e9438d4109766780b4df9ad32a823d1b7c59d1 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 16:17:04 -0500 Subject: [PATCH 53/80] =?UTF-8?q?fix(benchmark):=20a=20private=20keypair?= =?UTF-8?q?=20could=20reach=20a=20graded=20patch=20and=20the=20curriculum?= =?UTF-8?q?=20=E2=80=94=20one=20exclude=20list,=20not=20two?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LIVE FINDING 2026-08-18: sympy-22714's staged workspace held `.airc/identity.key` at git status `A` — ALREADY intent-added. airc creates its scope at the enclosing git root ("in a git repo → /.airc/"), so a citizen working inside a cloned bench repo gets `events.sqlite`, a work-board cache, and a PRIVATE KEYPAIR written under the very repo she is being graded on. TWO READINGS OF "HER WORK" HAD DRIFTED APART, which is exactly what one of them warned about in its own doc — `workspace_candidate_diff`: "a second inline `git diff` would drift on the exclude rules": - `benchmark::workspace_candidate_diff` — excluded `.airc` inline. SAFE. This is why the sympy-22714 grade came out at a clean 2,021 bytes rather than the 92KB the raw tree carries. - `agent::solve::workspace_patch` — its own near-copy of the exclude list, WITHOUT `.airc`. Not safe. And it is the reading that feeds `files_changed`, which feeds `format_solve_lesson` → the curriculum. An unexcluded key becomes the training sentence "I changed: .airc/identity.key". It also runs `git add -A -N`, which is what intent-added the key in the first place. This is the second time the same hole has cost something: card b34f7eb5 records 91KB of staged `.airc` blobs voiding a REAL fix, because the fresh clone refused the whole candidate. FIX (compression, not a patch): ONE list — `benchmark::SOLUTION_PATH_EXCLUDES` — consumed by BOTH readings. `.airc` and `.continuum` join the build/cache byproducts, with the doc stating plainly that the first group is tidiness and the second is a SECURITY boundary. solve.rs's duplicate is deleted and re-exported from the shared constant, so a future exclude can only be added in one place. POSITIVE CONTROL (the guard is real, not vacuous): with the `.airc` entries temporarily removed, `workspace_patch_never_carries_agent_scope_state_or_credentials` FAILS on "KEY MATERIAL must never reach a patch" — the literal `SUPERSECRETKEYMATERIAL` lands in the diff. Restored, it passes. The test asserts three things a leak would break: the key bytes are absent, no `.airc` path appears, and `files_changed` names ONLY her source file (because that vector is what the curriculum reads). 31 tests green across workspace_patch / swe_bench / swe_setup. NOTE, unrelated and not chased here: a cache sweep removed the cargo target's `debug/deps` mid-command (~27GB freed, 141→168GB). No data loss, forces a rebuild; belongs to the #155/#296 cache-governance lane. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/agent/solve.rs | 63 ++++++++++++++----- core/continuum-core/src/commands/benchmark.rs | 50 +++++++++++++-- 2 files changed, 92 insertions(+), 21 deletions(-) diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index cedd548f7..a8602a5f1 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -1771,22 +1771,11 @@ fn transfer_solve_experience( } } -/// Git pathspecs excluding the universal never-a-solution byproducts a verification run leaves -/// behind — Python bytecode/caches, tool caches, JS deps, OS cruft. Glass-boxed 2026-07-22: a -/// `python3 -c "from calc import ..."` verify step left `__pycache__/calc.cpython-314.pyc` in the -/// patch, polluting the graded artifact — real SWE-bench/aider patches are SOURCE-only. These are -/// never a solution, so they're excluded from both the diff and files_changed; anything a task -/// might legitimately produce (incl. `build`/`dist`/`target`) is kept. -const PATCH_EXCLUDES: &[&str] = &[ - ":(exclude,glob)**/__pycache__/**", - ":(exclude,glob)**/*.pyc", - ":(exclude,glob)**/*.pyo", - ":(exclude,glob)**/.pytest_cache/**", - ":(exclude,glob)**/.mypy_cache/**", - ":(exclude,glob)**/.ruff_cache/**", - ":(exclude,glob)**/node_modules/**", - ":(exclude,glob)**/.DS_Store", -]; +/// What is NOT part of a solution lives in ONE place, beside the other reading of her work: +/// [`crate::commands::benchmark::SOLUTION_PATH_EXCLUDES`]. This file used to carry its own +/// near-copy that omitted `.airc`, which is precisely the drift `workspace_candidate_diff`'s +/// doc warned about — see the shared constant for the two incidents. +use crate::commands::benchmark::SOLUTION_PATH_EXCLUDES as PATCH_EXCLUDES; /// Unified diff of the SOLUTION changes in the workspace (tracked edits + new files), and the /// touched paths — build/cache byproducts ([`PATCH_EXCLUDES`]) filtered out so the graded artifact @@ -2123,6 +2112,48 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + // what this catches: a CREDENTIAL reaching a graded patch, files_changed, or the curriculum. + // Live 2026-08-18: sympy-22714's tree held `.airc/identity.key` (a private keypair) at git + // status `A` — already intent-added, because this path's exclude list omitted `.airc` while + // the grader's inline list had it. airc creates its scope at the enclosing git root, so a + // citizen working inside a cloned bench repo gets one written under the repo she is graded + // on. files_changed feeds format_solve_lesson, so an unexcluded key becomes the training + // sentence "I changed: .airc/identity.key". Also the b34f7eb5 shape: 91KB of staged .airc + // blobs once voided a REAL fix because the fresh clone refused the whole candidate. + #[tokio::test] + async fn workspace_patch_never_carries_agent_scope_state_or_credentials() { + let dir = std::env::temp_dir().join(format!("cu-agent-solve-airc-{}", Uuid::new_v4())); + std::fs::create_dir_all(dir.join(".airc/work-board-cache")).unwrap(); + git(&dir, &["init", "-q"]).await; + git(&dir, &["config", "user.email", "t@t"]).await; + git(&dir, &["config", "user.name", "t"]).await; + // her actual solution + std::fs::write(dir.join("point.py"), "def dot(a, b):\n return a * b\n").unwrap(); + // what the SUBSTRATE wrote into her tree — never authored by the solver + std::fs::write(dir.join(".airc/identity.key"), "SUPERSECRETKEYMATERIAL").unwrap(); + std::fs::write(dir.join(".airc/events.sqlite"), b"SQLite format 3\x00").unwrap(); + std::fs::write(dir.join(".airc/work-board-cache/x.json"), "{}").unwrap(); + + let (patch, files) = workspace_patch(dir.to_str().unwrap()).await; + assert!( + patch.contains("point.py"), + "her solution must still be in the patch:\n{patch}" + ); + assert!( + !patch.contains("SUPERSECRETKEYMATERIAL"), + "KEY MATERIAL must never reach a patch:\n{patch}" + ); + assert!( + !patch.contains(".airc"), + "no agent-scope path may appear in the patch:\n{patch}" + ); + assert_eq!( + files, + vec!["point.py".to_string()], + "files_changed feeds the curriculum lesson — it must name only her work" + ); + } + // what this catches: verification byproducts (Python bytecode, __pycache__) must NOT pollute // the graded patch — glass-boxed 2026-07-22 when a `python3 -c "from calc import ..."` verify // step left calc.cpython-314.pyc in the diff. Source is kept; the cache junk is filtered. diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 474e55096..d8f77e1d8 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2116,17 +2116,57 @@ impl ActionCommand for BenchmarkSweGrade { } } +/// Paths that are NEVER part of a solution — the ONE list, shared by every reading of +/// "her work" ([`workspace_candidate_diff`] here and `agent::solve::workspace_patch`). +/// +/// Two kinds, and the second is a SECURITY boundary, not tidiness: +/// +/// 1. Build/cache byproducts. A `python3 -c ...` verify step left +/// `__pycache__/calc.cpython-314.pyc` in a graded patch; real SWE-bench/aider patches are +/// SOURCE-only. Anything a task might legitimately produce (`build`/`dist`/`target`) is kept. +/// +/// 2. **Agent-scope state the SUBSTRATE writes into her tree.** airc creates its scope at the +/// enclosing git root, so a citizen working inside a cloned bench repo gets `.airc/` — +/// `events.sqlite`, a work-board cache, and **`identity.key`, a private keypair** — created +/// under the repo she is being graded on. This has bitten twice: card b34f7eb5, where Atlas's +/// first grade carried 91KB of staged `.airc` blobs and the fresh clone refused the WHOLE +/// candidate (a real fix voided by files no solver wrote); and 2026-08-18, where +/// sympy-22714's tree still held `.airc/identity.key` with git status `A` — already +/// intent-added, because `workspace_patch` ran `git add -A -N` with an exclude list that +/// lacked `.airc`. The grader was safe (it excluded `.airc` inline) but `workspace_patch` +/// was not, and IT is the reading that feeds `files_changed` → `format_solve_lesson` → +/// the curriculum. A credential could have been written into training data as +/// "I changed: .airc/identity.key". +/// +/// That divergence is exactly what [`workspace_candidate_diff`]'s own doc warned about — "a +/// second inline `git diff` would drift on the exclude rules" — so the rule now lives in ONE +/// place and both readings consume it ([[the-compression-principle]]). +pub(crate) const SOLUTION_PATH_EXCLUDES: &[&str] = &[ + // Agent/substrate scope — never authored by the solver, and credential-bearing. + ":(exclude,glob)**/.airc/**", + ":(exclude,glob)**/.continuum/**", + // Build + cache byproducts. + ":(exclude,glob)**/__pycache__/**", + ":(exclude,glob)**/*.pyc", + ":(exclude,glob)**/*.pyo", + ":(exclude,glob)**/.pytest_cache/**", + ":(exclude,glob)**/.mypy_cache/**", + ":(exclude,glob)**/.ruff_cache/**", + ":(exclude,glob)**/node_modules/**", + ":(exclude,glob)**/.DS_Store", +]; + /// The candidate diff of a solver workspace — the ONE reading of "her work" /// (grade_swe's candidate arm and agent/solve's attempt-patch receipt both /// call this; a second inline `git diff` would drift on the exclude rules). /// `diff HEAD` (not bare `diff`) so STAGED edits count as her work too, and -/// `:(exclude).airc` because the substrate stages its own coordination files -/// into her workspace (card b34f7eb5): Atlas's first grade carried 91KB of -/// staged `.airc` blobs, and the fresh clone refused the WHOLE candidate — -/// a real fix voided by files no solver wrote. +/// [`SOLUTION_PATH_EXCLUDES`] keeps substrate-authored files out — see its doc +/// for the two incidents that make the `.airc` entry load-bearing. pub(crate) fn workspace_candidate_diff(ws: &str) -> Result { + let mut args: Vec<&str> = vec!["diff", "HEAD", "--", "."]; + args.extend_from_slice(SOLUTION_PATH_EXCLUDES); let out = std::process::Command::new("git") - .args(["diff", "HEAD", "--", ".", ":(exclude).airc"]) + .args(&args) .current_dir(ws) .output() .map_err(|e| CommandError::Internal(format!("could not read {ws}'s diff: {e}")))?; From 72948dc9388b0419c0a4734cf9b45531516bef88 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 16:19:27 -0500 Subject: [PATCH 54/80] =?UTF-8?q?fix(benchmark):=20the=20board=20could=20n?= =?UTF-8?q?ot=20see=20a=20finished=20patch=20=E2=80=94=20staged=20workspac?= =?UTF-8?q?es=20are=20now=20a=20first-class=20source?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE ACCEPTANCE TEST FROM OUR OWN DOC WAS FAILING. docs/architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md: "can a citizen standing in the room perceive the run's state through the same ViewState pipe the human's screen uses? If answering needs a file read or a log parse, it is disconnected and it failed." Today I found two SWE-bench Lite PASSES by running `git -C … diff` by hand. That is the failure, exactly as written. WHY THE BOARD WAS BLIND: `scan_run_cards` read ONLY `progress/agent-solve-*.json`. Those are written by a solve PROCESS. A process frozen mid-flight never writes `files_changed`; work done any other way writes no file at all. So `benchmark/runs` showed 20 rows, every one with `files_changed: []`, 15 of them `quiet`/stalled — while three staged trees on the same disk held real in-place source edits, and NONE of those three instances had a row at all. THE INVERSION: a run file is an EPHEMERAL progress marker; the workspace is DURABLE truth. The board now projects both. - New source `scan_workspace_artifact_cards`: walks `citizens/peers/*/workspace/swe/*`, and emits a card for any tree whose candidate diff is non-empty and whose instance carries no grade. It calls `workspace_candidate_diff` — the SAME reading of "her work" the grader uses, so the board can never disagree with the verdict about what she changed (and it inherits the `.airc` credential exclusion from the shared list). - `files_changed` comes off the diff's own `+++ b/` headers — no extra process spawn. - Phase is `ungraded`, deliberately NOT `quiet`. Nothing is stalled: a finished artifact is waiting for a verdict. Conflating those two is what hid two passes for ~22 hours, and the phase doc now names the distinction. - Skipped for a run-id query (that asks about one ledger; an artifact has no run id). - Bounded at 200 trees per call because each is a `git diff` spawn and the board is polled — and an over-cap scan WARNS with the dropped count rather than silently truncating, so a capped scan can never read as "nothing found" (no-silent-caps). OWED, and stated rather than glossed: there is no unit test for this source. The honest test needs a populated CONTINUUM_HOME, and env-var-dependent tests are a smell this codebase has already been burned by (#72). Verification here is LIVE, against three trees whose ground truth I established independently this session — two of which grade `resolved=true`. If a non-env-dependent seam appears (a scan rooted at an injected path), that is where the test belongs. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 128 +++++++++++++++++- 1 file changed, 127 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index d8f77e1d8..874c8ba68 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2830,7 +2830,10 @@ pub struct BenchRunCard { /// `resolved` | `failed` (loud infra marker, incl. #2180 stalls the /// deadline caught) | `active` (artifact activity within the stall /// window) | `quiet` (non-terminal AND silent past the window — the - /// shape the projection exists to make visible). + /// shape the projection exists to make visible) | `ungraded` (a staged + /// workspace holds a real diff that no grade has ever seen — durable + /// work awaiting a verdict, NOT a stall; see + /// [`scan_workspace_artifact_cards`]). pub phase: String, /// True exactly when `phase == "quiet"`. pub stalled: bool, @@ -3015,6 +3018,117 @@ impl ActionCommand for BenchmarkRuns { /// command AND the positron `kind="bench"` board emitter (#329) fold THIS — /// never a parallel file scrape ([[the-compression-principle]]). Synchronous /// fs I/O: async callers wrap it in `spawn_blocking`. +/// Cards for staged workspaces that hold REAL WORK no grade has ever seen. +/// +/// Why this source exists (glass-boxed 2026-08-18, and it is the acceptance test from +/// docs/architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md failing): the board read ONLY +/// `progress/agent-solve-*.json`. Those files are written by a solve PROCESS. A process that +/// froze mid-flight never writes `files_changed`, and work done another way never writes a +/// file at all — so the board showed 20 runs with `files_changed: []` and phase `quiet`, +/// while three staged trees on the same disk held real in-place source edits. Two of them +/// were PASSES (astropy-14995, pytest-11143, both `resolved=true` once the grader could read +/// its own output). They were found by hand with `git -C … diff`, which is precisely the +/// "if answering needs a file read, it is disconnected and it failed" the doc names. +/// +/// The workspace is DURABLE truth; a run file is an ephemeral progress marker. So the board +/// projects both, and an artifact nobody graded is a first-class row rather than an absence. +/// Instances already carrying a grade are skipped — a graded run is the authoritative card. +/// +/// Bounded on purpose: at most [`WORKSPACE_ARTIFACT_SCAN_CAP`] trees per call, and the count +/// dropped is logged rather than silently truncated (no-silent-caps). +fn scan_workspace_artifact_cards(graded: &std::collections::HashSet, now_ms: u64) -> Vec { + let Ok(home) = continuum_home() else { + return Vec::new(); + }; + let peers = home.join("citizens").join("peers"); + let Ok(peer_entries) = std::fs::read_dir(&peers) else { + return Vec::new(); + }; + let mut cards = Vec::new(); + let mut scanned = 0usize; + let mut skipped_over_cap = 0usize; + for peer in peer_entries.flatten() { + let peer_id = peer.file_name().to_string_lossy().to_string(); + let swe = peer.path().join("workspace").join("swe"); + let Ok(instances) = std::fs::read_dir(&swe) else { + continue; + }; + for inst in instances.flatten() { + if !inst.path().is_dir() { + continue; + } + let instance = inst.file_name().to_string_lossy().to_string(); + if graded.contains(&instance) { + continue; + } + if scanned >= WORKSPACE_ARTIFACT_SCAN_CAP { + skipped_over_cap += 1; + continue; + } + scanned += 1; + let Some(ws) = inst.path().to_str().map(String::from) else { + continue; + }; + // The SAME reading of "her work" the grader uses — never a second inline diff. + let Ok(diff) = workspace_candidate_diff(&ws) else { + continue; + }; + if diff.trim().is_empty() { + continue; + } + // Touched paths straight off the diff header — no extra process spawn. + let files_changed: Vec = diff + .lines() + .filter_map(|l| l.strip_prefix("+++ b/")) + .map(|p| p.to_string()) + .collect(); + let last_activity_ms = std::fs::metadata(inst.path()) + .and_then(|m| m.modified()) + .ok() + .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) + .map(|d| d.as_millis() as u64) + .unwrap_or(0); + cards.push(BenchRunCard { + run_id: format!("workspace:{}:{instance}", &peer_id[..8.min(peer_id.len())]), + instance: Some(instance), + attempt: None, + max_attempts: None, + solver: Some(peer_id.clone()), + // NOT "quiet": nothing is stalled here — a finished artifact is waiting for a + // verdict. Conflating the two is what hid two passes for ~22 hours. + phase: "ungraded".to_string(), + stalled: false, + last_activity_ms, + age_secs: now_ms.saturating_sub(last_activity_ms) / 1000, + acts: None, + files_changed, + files_examined: Vec::new(), + resolved: None, + fail_to_pass: None, + pass_to_pass: None, + patch_bytes: Some(diff.len() as u32), + failed_tests: Vec::new(), + infra_error: None, + }); + } + } + if skipped_over_cap > 0 { + tracing::warn!( + scanned, + skipped_over_cap, + cap = WORKSPACE_ARTIFACT_SCAN_CAP, + "benchmark/runs: workspace-artifact scan hit its cap — some trees were NOT examined \ + for ungraded work (raise the cap or narrow the query; this is not 'nothing found')" + ); + } + cards +} + +/// How many staged trees one `benchmark/runs` call will diff. A `git diff` per tree is a +/// process spawn, and the board is polled; this bounds the cost. Over-cap trees are WARNED +/// about, never silently dropped. +const WORKSPACE_ARTIFACT_SCAN_CAP: usize = 200; + pub(crate) fn scan_run_cards( run_id_filter: Option<&str>, limit: usize, @@ -3080,6 +3194,18 @@ pub(crate) fn scan_run_cards( now_ms, )); } + // Second source: staged trees holding work no grade has seen. Only when the caller is + // asking for the BOARD — a run-id query is asking about one run's ledger, and a workspace + // artifact has no run id to match. See `scan_workspace_artifact_cards` for why the board + // cannot be run-files-only. + if run_id_filter.is_none() { + let graded: std::collections::HashSet = cards + .iter() + .filter(|c| c.resolved.is_some()) + .filter_map(|c| c.instance.clone()) + .collect(); + cards.extend(scan_workspace_artifact_cards(&graded, now_ms)); + } cards.sort_by(|a, b| b.last_activity_ms.cmp(&a.last_activity_ms)); cards.truncate(limit); // The ledger stores the solver as her PERSONA UUID; the board speaks NAMES. From 5759b1c086df23878cac581e41da1e22a2bcc4ea Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 17:15:53 -0500 Subject: [PATCH 55/80] =?UTF-8?q?docs(planning):=20the=20grade=20tail=20?= =?UTF-8?q?=E2=80=94=20plan=20of=20record,=20and=20the=20forbidden=20move?= =?UTF-8?q?=20that=20looks=20right?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Written after a live session that recovered two SWE-bench Lite passes by hand and then found 11 more ungraded artifacts on disk. Locates the automation precisely and records the design constraint I nearly violated. THE HEADLINE, and why this is a doc rather than a patch: I proposed a periodic sweep that scans for ungraded artifacts. `modules/benchmark_grade.rs` opens by forbidding exactly that — "nothing scans the board on a clock — the transition event fires the grade" ([[the-whole-system-is-event-based-not-polling]]). The sweep would have worked, and would have quietly converted an event-driven substrate into a polling one. The module's own doc is what caught it, not me. The doc now carries that as a ⚠ block so the next reader (or the next amnesiac model) cannot reflex-code it. The correct question is never "when do we scan" but "which EVENT did we fail to emit". Both answers already have homes: - SEAM 1 — `swe_bench::reap_orphaned_solve_runs_in` (swe_bench.rs:186) already runs at boot and rewrites frozen `running` markers as FAILED. That IS the event; it journals the death and stops. It should also grade the orphaned ARTIFACT, which is the same class of thing as an orphaned process (#452, boot owns reap-or-adopt). - SEAM 2 — grep finds NO lease-expiry event in `modules/work.rs`. Expiry is passive, computed at read time by `work/list`, so a claim can lapse over a finished patch with no transition anywhere. That is why #451 ("lapsed claim + artifact → auto-close") does not fire despite being marked complete. Expiry must emit `work.card.state_changed` through the existing single emitter, and then the EXISTING grade-on-done subscriber needs no new grading path at all. Also recorded: the two tail gaps measured live — workspace cards get truncated off the board by the recency sort (1 of 13 visible at the default limit), and a `--workspace` grade writes no ledger entry (which is why the two passes I graded still read `ungraded`). Plus the honest cost note: the per-tree `git diff` is expensive enough that a hand-rolled equivalent timed out at 5 minutes. Acceptance is the doc's own standard, not a new one: kill a solve mid-patch, reboot, and a verdict exists with no operator command; and afterwards there is still exactly ONE `grade_swe` and ONE `workspace_candidate_diff` in the tree. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/planning/BENCHMARK-GRADE-TAIL.md | 141 ++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 docs/planning/BENCHMARK-GRADE-TAIL.md diff --git a/docs/planning/BENCHMARK-GRADE-TAIL.md b/docs/planning/BENCHMARK-GRADE-TAIL.md new file mode 100644 index 000000000..6be0d34f8 --- /dev/null +++ b/docs/planning/BENCHMARK-GRADE-TAIL.md @@ -0,0 +1,141 @@ +# The grade tail: making the room grade itself + +**Status:** plan of record. Written 2026-08-18 after a live session that recovered TWO +real SWE-bench Lite passes by hand and then found 11 more ungraded artifacts on disk. + +**The objective in one line:** a citizen finishes a patch → a verdict exists, with no human +in the loop. Today the first half works and the second does not. + +--- + +## ⚠ THE MOVE THAT LOOKS RIGHT AND IS FORBIDDEN + +**Do not add a periodic sweep that scans for ungraded artifacts on a clock.** + +`modules/benchmark_grade.rs` opens with the law: + +> The whole system is event-based, never polling +> ([[the-whole-system-is-event-based-not-polling]]): nothing scans the board on a clock — +> the transition event fires the grade. + +I proposed exactly that sweep during the session before reading the module. It is the +shortest path to "grades appear", it would work, and it would quietly convert an +event-driven substrate into a polling one. The module's own doc is what caught it. + +**The correct question is never "when do we scan?" — it is "which EVENT did we fail to +emit?"** Both answers are below, and both are events that already have a home. + +--- + +## What is actually broken (measured, 2026-08-18) + +`benchmark/runs` showed 20 rows, all with `files_changed: []`, 15 `quiet`/stalled. On the +same disk, 13 staged trees held real in-place source edits. Two of them were PASSES +(astropy-14995, pytest-11143 — `resolved=true`, F2P 1/1, P2P 40/40) that had been sitting +ungraded for ~22 hours. + +The grade-on-done subscriber is healthy and correct. It simply never fired, because it +triggers on a **card reaching a terminal state**, and these cards never got there: + +| why the card never went terminal | the event that is missing | +|---|---| +| the solve process froze / was killed mid-flight | boot reconciliation already exists — it just doesn't grade | +| the claim's 30-min lease lapsed and nobody closed the card | **no lease-expiry event exists at all** | + +Neither is a "we forgot to poll" problem. Both are "a real transition happened and nothing +announced it". + +--- + +## Seam 1 — boot reconciliation grades what it reaps + +`cognition::swe_bench::reap_orphaned_solve_runs_in(dir)` (swe_bench.rs:186) already runs at +boot and rewrites any run still marked `running` into a FAILED run. Test: +`a_run_still_marked_running_at_boot_is_journaled_as_killed`. + +That is the event. It currently journals the death and stops. + +**Change:** for each run it reaps, if that run's workspace carries a non-empty candidate +diff and no grade exists, grade it through the SAME `grade_swe` and write the verdict to +the run ledger. Boot owns reap-or-adopt for every service (#452, +[[boot-owns-the-process-tree-reap-or-adopt-never-fight-yourself]]); an orphaned ARTIFACT is +the same class of thing as an orphaned process. + +Constraints: +- Reuse `benchmark::workspace_candidate_diff` — the ONE reading of her work. Never a second + inline `git diff` (that drift already cost a credential leak; see + `SOLUTION_PATH_EXCLUDES`). +- Grading is minutes per instance and boot must not block on it. Fire it as owned + background work with the standard bounded-run discipline, not inline in the boot path. +- Idempotent: a run that already has a `.grade.json` is skipped. + +## Seam 2 — a lapsed lease is a state change, so emit it + +`grep` for a lease-expiry event in `modules/work.rs` returns **nothing**. Expiry today is +*passive*: `work/list` computes "is this hold still live?" at read time. So a claim can +lapse with a finished patch under it and the board experiences no transition — which is +why #451 ("lapsed claim + artifact → auto-close → the one grade tail") does not fire in +practice despite being marked complete. + +**Change:** make expiry emit `work.card.state_changed` like every other transition, so the +EXISTING grade-on-done subscriber picks it up with no new grading path. One emitter +([[the-same-bug-at-two-sites-is-a-missing-constraint]]) — `work/state` already owns that +event (`WORK_CARD_STATE_CHANGED`); expiry must go through it rather than growing a parallel +notification. + +Open question for Joel, deliberately not decided here: does a lapsed claim with an artifact +auto-CLOSE the card, or move it to a `needs-verdict` state that a citizen can re-claim? +Auto-close is simpler; re-claimable is truer to #419 (recover a claim whose work session +died). This is a recipe/lifecycle policy call, not a plumbing detail. + +--- + +## Two tail gaps that must land with the above + +1. **Workspace cards get truncated off the board.** `scan_run_cards` sorts by + `last_activity_ms` and truncates to `limit`. A workspace card's timestamp is its + directory mtime, so artifacts lose the recency race to chatty run files: the live board + showed **1 of 13** at the default limit and all 13 only at `--limit=100`. An artifact + awaiting a verdict must not be evictable by run-file noise — give artifacts their own + floor in the projection, or sort terminal-pending ahead of quiet. + +2. **A grade taken via `--workspace` writes no ledger entry.** That is how astropy-14995 and + pytest-11143 were graded during the session, and it is why they *still* read `ungraded` + afterwards. Every verdict — operator arm included — must land in the run ledger, or the + board keeps re-offering work that is already judged. + +Also noted, not blocking: the per-tree `git diff` in the artifact scan is genuinely +expensive (a hand-rolled equivalent over ~23 trees timed out at 5 minutes). The scan is +capped at 200 trees and warns when it drops any, but if the board is polled by a live +ViewState this wants a cheaper freshness check (dir mtime vs last-graded stamp) before +paying for a diff. + +--- + +## Acceptance test (the doc's own standard, not a new one) + +From `docs/architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md`: + +> can a citizen standing in the room perceive the run's state through the same ViewState +> pipe the human's screen uses? If answering needs a file read or a log parse, it is +> disconnected and it failed. + +Concretely, this work is done when: + +1. Kill a solve mid-flight with a real diff in its tree. Reboot. A verdict exists, and the + board shows `resolved`/`failed` for that instance — with no operator command. +2. Let a claim lapse on a card whose workspace holds a patch. A verdict appears through the + existing grade-on-done subscriber — no new grading path was added. +3. `benchmark/runs` at DEFAULT limit shows every artifact awaiting a verdict. +4. Grep the tree: still exactly one `grade_swe` and one `workspace_candidate_diff`. If a + second appeared, the fix went in wrong. + +## What is already true (don't rebuild it) + +- `grade_swe` — one grader, fresh clone at `base_commit`, laundering-proof. +- `modules/benchmark_grade.rs` — grade-on-done subscriber, correct, already wired to + `work.card.state_changed`. +- `workspace_candidate_diff` + `SOLUTION_PATH_EXCLUDES` — one reading of "her work", with + the `.airc` credential exclusion. +- The board's workspace-artifact source (`scan_workspace_artifact_cards`) — this is what + made the 13 visible; the plan above turns visibility into verdicts. From f3cb3a65c4f5b4ca06ca2ee9bf178c6bfdc6e3d4 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 17:34:48 -0500 Subject: [PATCH 56/80] fix(benchmark): the boot reaper and the reboot guard read a ledger name nothing ever wrote (#452/#381) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED on this box today: 506 `agent-solve-*.json` run ledgers, **19 still marked `running`, the oldest 162 hours old**. Against 32 legacy `swe-solve-*.json`, 0 running. `agent/solve` — the only production writer — emits `agent-solve-.json`. Both readers in `cognition::swe_bench` looked for `swe-solve-*`, a name no code in the tree has ever written. So neither half of the mechanism had EVER engaged on a real run: - `reap_orphaned_solve_runs_in` had never reaped a production run. 19 orphans sit frozen as `running` instead of becoming honest FAILED records — precisely the silent-death hole its own doc says it exists to close ("a poller cannot distinguish still-working from died-an-hour-ago when the evidence for both is an absent file"). - `in_flight_solve_runs_in` answered "nothing in flight" to the REBOOT GUARD while 19 runs were marked otherwise. `continuum reboot` would have destroyed live benchmark work without naming it. A guard that cannot see is worse than none: it reports safety. A unit test pinned the mismatch as intent — it wrote `agent-solve-other.json` and asserted it stayed untouched as "an unrelated ledger from another subsystem". There is no other subsystem. That WAS production. A fixture that spells the file differently from the writer proves nothing about the writer, and this one kept two dead paths green for weeks. SECOND DEFECT, found while fixing the first and worse for the grade tail: the reap OVERWROTE the ledger with a bare marker. A live `running` record carries `workspace` — the absolute path of the checkout her hands edited — plus `instance`, `persona_id`, `acts`. Blanket-overwriting turns a gradeable orphan into an unlocatable one: the patch is still on disk and nothing can say where. The reap now MERGES; the death is an annotation on the run's own record, never a replacement for it. Same class of harm as reaping a finished verdict, which the existing test already forbade. THE FIX IS COMPRESSION, not a second spelling. `SOLVE_LEDGER_PREFIX` + `solve_ledger_path` / `solve_grade_path` / `solve_run_id_from_file_name` now live once in `swe_bench`, and the writer (`solve.rs`), the board scan (`benchmark.rs`), the reaper and the guard all route through them — so writer and reader cannot drift again. The `.grade` sibling rule that `scan_run_cards` carried inline moves into the shared parser rather than being re-derived. `solve_ledger_dir` also now honours `CONTINUUM_HOME` like every other progress-ledger reader; it did not, which was a SECOND way to spell the root. [[the-same-bug-at-two-sites-is-a-missing-constraint-not-two-bugs]] Tests: the reaper test renamed to the real filename and extended to assert the workspace pointer + journaled fields survive the reap; new parse test pins run-id extraction, grade rejection, and writer/reader round-trip. 74 green across swe_bench / benchmark / agent::solve. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 181 +++++++++++++++--- .../src/commands/agent/solve.rs | 13 +- core/continuum-core/src/commands/benchmark.rs | 15 +- 3 files changed, 166 insertions(+), 43 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 90852bdf4..82bb7336c 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -123,9 +123,69 @@ pub struct SweVerdict { /// Where a detached benchmark run journals its state. One file per run, rewritten in place: /// `state: "running"` at dispatch, then the verdict — or a killed-by-reboot marker. +/// +/// Honours `CONTINUUM_HOME` the same way every other progress-ledger reader does +/// (`benchmark::continuum_home`). It did not until 2026-08-18, which made this the SECOND +/// way to spell the ledger root — and a reader that resolves a directory differently from +/// the writer reports a self-consistent lie about an empty dir, the same failure shape the +/// `swe_cache_dir` doc records for env coverage. pub fn solve_ledger_dir() -> PathBuf { - let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".into()); - PathBuf::from(home).join(".continuum").join("progress") + crate::commands::benchmark::continuum_home() + .unwrap_or_else(|_| { + PathBuf::from(std::env::var("HOME").unwrap_or_else(|_| "/tmp".into())).join(".continuum") + }) + .join("progress") +} + +/// The ONE filename prefix a detached solve run's ledger carries: `agent-solve-.json`, +/// with its verdict alongside as `agent-solve-.grade.json`. +/// +/// # Why this is a constant and not spelled out per call site (2026-08-18) +/// +/// It was spelled out per call site, and the two spellings did not match. `agent/solve` +/// wrote `agent-solve-*` (`agent_solve_ledger_path`, the only production writer); the reaper +/// and the reboot guard below read `swe-solve-*`, a name NOTHING in the tree has ever +/// written. Measured on this box the day it was found: **506 `agent-solve-*` ledgers, 19 of +/// them still marked `running` — the oldest 162 hours old — against 32 legacy `swe-solve-*` +/// files, 0 running.** So neither half of the mechanism had ever engaged on a real run: +/// +/// - [`reap_orphaned_solve_runs_in`] had never reaped a production run, so 19 orphans sat +/// frozen as `running` instead of becoming honest FAILED records — the exact silent-death +/// hole its own doc says it exists to close. +/// - [`in_flight_solve_runs_in`] answered "nothing is in flight" to the reboot guard while +/// 19 runs were marked otherwise, so `continuum reboot` would have destroyed live work +/// without naming it. A guard that cannot see is worse than no guard: it reports safety. +/// +/// A unit test even pinned the mismatch as intent — it wrote `agent-solve-other.json` and +/// asserted it stayed untouched as "another subsystem's ledger". There is no other +/// subsystem; that WAS the production ledger. +/// [[the-same-bug-at-two-sites-is-a-missing-constraint-not-two-bugs]] +pub const SOLVE_LEDGER_PREFIX: &str = "agent-solve-"; + +/// The run id a ledger file name carries, or `None` if it is not a run ledger. +/// +/// Rejects `agent-solve-.grade.json`: a grade is read as a SIBLING of its run, never +/// enumerated as one. Without that check the prefix/suffix strip yields a phantom run whose +/// id ends in `.grade` — the live first-use bug `scan_run_cards` already carried a guard for, +/// now shared rather than re-derived. +pub fn solve_run_id_from_file_name(name: &str) -> Option<&str> { + let run_id = name + .strip_prefix(SOLVE_LEDGER_PREFIX)? + .strip_suffix(".json")?; + if run_id.is_empty() || run_id.ends_with(".grade") { + return None; + } + Some(run_id) +} + +/// This run's ledger file. The one path builder, so writer and reader cannot drift again. +pub fn solve_ledger_path(dir: &Path, run_id: &str) -> PathBuf { + dir.join(format!("{SOLVE_LEDGER_PREFIX}{run_id}.json")) +} + +/// This run's verdict file, alongside its ledger. +pub fn solve_grade_path(dir: &Path, run_id: &str) -> PathBuf { + dir.join(format!("{SOLVE_LEDGER_PREFIX}{run_id}.grade.json")) } /// Runs this ledger dir believes are IN FLIGHT — written at dispatch, not yet resolved. @@ -143,9 +203,10 @@ pub fn in_flight_solve_runs_in(dir: &Path) -> Vec<(String, String)> { let Some(name) = path.file_name().and_then(|n| n.to_str()) else { continue; }; - if !name.starts_with("swe-solve-") || !name.ends_with(".json") { + let Some(run_id) = solve_run_id_from_file_name(name) else { continue; - } + }; + let run_id = run_id.to_string(); let Ok(text) = std::fs::read_to_string(&path) else { continue; }; @@ -155,10 +216,6 @@ pub fn in_flight_solve_runs_in(dir: &Path) -> Vec<(String, String)> { if v.get("state").and_then(|s| s.as_str()) != Some("running") { continue; } - let run_id = name - .trim_start_matches("swe-solve-") - .trim_end_matches(".json") - .to_string(); let instance = v .get("instance") .and_then(|i| i.as_str()) @@ -183,18 +240,42 @@ pub fn in_flight_solve_runs() -> Vec<(String, String)> { /// hour ago" when the evidence for both is an absent file — the same shape as #137's 41 train /// jobs submitted with zero outcomes recorded. The marker at dispatch is what makes the death /// observable; this reap is what makes it honest. +/// The reap MERGES the death into the existing ledger rather than replacing it, because the +/// ledger is the only pointer to what the dead run LEFT BEHIND. +/// +/// A live `running` record carries `workspace` — the absolute path of the checkout her hands +/// edited — alongside `instance`, `persona_id` and `acts`. Overwriting it with a bare marker +/// (which this did until 2026-08-18) turns a gradeable orphan into an unlocatable one: the +/// patch is still on disk, and nothing on the board can say where. That is the same class of +/// harm as reaping a finished verdict, and the reason the existing keys are preserved here +/// and only the failure fields are added. pub fn reap_orphaned_solve_runs_in(dir: &Path) -> Vec { let mut reaped = Vec::new(); for (run_id, instance) in in_flight_solve_runs_in(dir) { - let path = dir.join(format!("swe-solve-{run_id}.json")); - let marker = serde_json::json!({ - "failed": true, - "runId": run_id, - "instance": instance, - "error": "killed by a core restart — the run was in flight when the core that owned \ - it went away. Nothing was scored; re-dispatch to measure this instance.", - }); - if std::fs::write(&path, marker.to_string()).is_ok() { + let path = solve_ledger_path(dir, &run_id); + // Start from what the run itself journaled; the death is an ANNOTATION on that + // record, never a replacement for it. + let mut record = std::fs::read_to_string(&path) + .ok() + .and_then(|t| serde_json::from_str::(&t).ok()) + .filter(|v| v.is_object()) + .unwrap_or_else(|| serde_json::json!({})); + let obj = record.as_object_mut().expect("filtered to an object above"); + obj.insert("failed".into(), serde_json::Value::Bool(true)); + // `state` is what `in_flight_solve_runs_in` keys on. Clearing it off `running` is + // what makes the reap idempotent — a second boot must not re-reap. + obj.insert("state".into(), serde_json::Value::String("failed".into())); + obj.insert("runId".into(), serde_json::Value::String(run_id.clone())); + obj.insert("instance".into(), serde_json::Value::String(instance)); + obj.insert( + "error".into(), + serde_json::Value::String( + "killed by a core restart — the run was in flight when the core that owned it \ + went away. Nothing was scored; re-dispatch to measure this instance." + .into(), + ), + ); + if std::fs::write(&path, record.to_string()).is_ok() { reaped.push(run_id); } } @@ -2732,20 +2813,28 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. fn a_run_still_marked_running_at_boot_is_journaled_as_killed() { let dir = tempfile::tempdir().expect("tmp"); let p = dir.path(); + // The name `agent/solve` ACTUALLY writes. This test used to spell it + // `swe-solve-*` — a name nothing in the tree has ever written — and asserted that + // an `agent-solve-*` file was "another subsystem's ledger" left untouched. There is + // no other subsystem: that WAS production, so both the reaper and the reboot guard + // were green here and dead in the field (see SOLVE_LEDGER_PREFIX for the 19-orphan + // measurement). A fixture that names the file differently from the writer proves + // nothing about the writer. std::fs::write( - p.join("swe-solve-alive.json"), - r#"{"state":"running","runId":"alive","instance":"sympy__sympy-22005"}"#, + solve_ledger_path(p, "alive"), + r#"{"state":"running","runId":"alive","instance":"sympy__sympy-22005", + "workspace":"/tmp/ws/sympy__sympy-22005","persona_id":"abc","acts":3}"#, ) .unwrap(); // A FINISHED run must survive the reap untouched — reaping a real verdict would // destroy the only record of a measurement that actually happened. std::fs::write( - p.join("swe-solve-done.json"), + solve_ledger_path(p, "done"), r#"{"instance":"sympy__sympy-21379","acts":7,"detached":false}"#, ) .unwrap(); - // An unrelated ledger from another subsystem is not ours to touch. - std::fs::write(p.join("agent-solve-other.json"), r#"{"state":"running"}"#).unwrap(); + // A grade file is a SIBLING of its run, never a run — the `.grade` phantom. + std::fs::write(solve_grade_path(p, "done"), r#"{"resolved":true}"#).unwrap(); assert_eq!( in_flight_solve_runs_in(p), @@ -2756,7 +2845,7 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. let reaped = reap_orphaned_solve_runs_in(p); assert_eq!(reaped, vec!["alive".to_string()]); - let after = std::fs::read_to_string(p.join("swe-solve-alive.json")).unwrap(); + let after = std::fs::read_to_string(solve_ledger_path(p, "alive")).unwrap(); assert!( after.contains("\"failed\":true"), "the orphan is now a FAILED run: {after}" @@ -2765,15 +2854,27 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. after.contains("killed by a core restart"), "and it names the cause rather than leaving a bare zero: {after}" ); - let done = std::fs::read_to_string(p.join("swe-solve-done.json")).unwrap(); + // The reap ANNOTATES; it must not erase where the dead run left its patch. Without + // this the orphan becomes ungradeable — the artifact is on disk and nothing can say + // where. + assert!( + after.contains("/tmp/ws/sympy__sympy-22005"), + "the workspace pointer survives the reap: {after}" + ); + assert!( + after.contains("\"acts\":3"), + "and so does what the run had journaled about itself: {after}" + ); + + let done = std::fs::read_to_string(solve_ledger_path(p, "done")).unwrap(); assert!( done.contains("\"acts\":7"), "a finished verdict is never rewritten" ); - let other = std::fs::read_to_string(p.join("agent-solve-other.json")).unwrap(); + let grade = std::fs::read_to_string(solve_grade_path(p, "done")).unwrap(); assert!( - !other.contains("failed"), - "another subsystem's ledger is untouched" + grade.contains("\"resolved\":true"), + "a grade sibling is never enumerated as a run, so never reaped" ); assert!( @@ -2782,6 +2883,32 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. ); } + // what this catches: the run-id parse must accept the name the writer emits and reject + // the grade sibling. Both halves were re-derived per call site before 2026-08-18, and + // the two spellings diverged (`swe-solve-` vs `agent-solve-`), which silently disarmed + // the boot reaper AND the reboot guard for every production run. + #[test] + fn a_ledger_file_name_yields_its_run_id_and_a_grade_sibling_yields_none() { + assert_eq!( + solve_run_id_from_file_name("agent-solve-claim-912427a1.json"), + Some("claim-912427a1") + ); + assert_eq!( + solve_run_id_from_file_name("agent-solve-claim-912427a1.grade.json"), + None, + "a grade is read as a sibling, never enumerated as a run" + ); + assert_eq!(solve_run_id_from_file_name("agent-solve-.json"), None); + assert_eq!(solve_run_id_from_file_name("swe-solve-legacy.json"), None); + assert_eq!(solve_run_id_from_file_name("competition-abc.json"), None); + // The writer and the reader must agree by construction, not by memory. + let path = solve_ledger_path(std::path::Path::new("/x"), "r1"); + assert_eq!( + solve_run_id_from_file_name(path.file_name().unwrap().to_str().unwrap()), + Some("r1") + ); + } + // what this catches: test-id lists arrive JSON-encoded inside a string field, and a // missing/blank one must yield an empty list rather than panicking a whole run. #[test] diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index a8602a5f1..a7dc954a5 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -907,14 +907,15 @@ impl ActionCommand for AgentSolve { /// Result file for a detached solve run, polled after the ack (mirrors the eval/competition /// progress-ledger convention: `~/.continuum/progress/agent-solve-.json`). +/// +/// Both the directory and the file name come from `cognition::swe_bench`, which is also where +/// the boot reaper and the reboot guard READ them. They were spelled out independently here +/// until 2026-08-18 and the names did not match, so neither reader ever saw a run this writer +/// produced ([`crate::cognition::swe_bench::SOLVE_LEDGER_PREFIX`] carries the measurement). fn agent_solve_ledger_path(run_id: &str) -> Option { - let base = std::env::var("CONTINUUM_HOME") - .map(std::path::PathBuf::from) - .ok() - .or_else(|| dirs::home_dir().map(|h| h.join(".continuum")))?; - let dir = base.join("progress"); + let dir = crate::cognition::swe_bench::solve_ledger_dir(); let _ = std::fs::create_dir_all(&dir); - Some(dir.join(format!("agent-solve-{run_id}.json"))) + Some(crate::cognition::swe_bench::solve_ledger_path(&dir, run_id)) } /// Where THIS run's artifacts live — the patch above all. One definition, so a run's diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 874c8ba68..220ffaa04 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3147,18 +3147,13 @@ pub(crate) fn scan_run_cards( let entries = std::fs::read_dir(&base).map_err(|e| format!("read {}: {e}", base.display()))?; for entry in entries.flatten() { let name = entry.file_name().to_string_lossy().to_string(); - let Some(run_id) = name - .strip_prefix("agent-solve-") - .and_then(|r| r.strip_suffix(".json")) - else { + // Grade files are read as SIBLINGS of their run below, never enumerated as runs + // (live first use showed `X.grade` phantoms). That rule and the prefix now live in + // ONE place with the boot reaper and the reboot guard, which is what stops the + // board and the reaper disagreeing about what a run ledger is called. + let Some(run_id) = crate::cognition::swe_bench::solve_run_id_from_file_name(&name) else { continue; }; - // Grade files are read as SIBLINGS of their run below, never - // enumerated as runs (live first use showed `X.grade` phantoms: - // `agent-solve-X.grade.json` survives the prefix/suffix strip). - if run_id.ends_with(".grade") { - continue; - } if let Some(want) = run_id_filter { if want != run_id { continue; From 7a7a6b17b4a41b0a215c4804e3bf48a857239018 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 17:35:56 -0500 Subject: [PATCH 57/80] =?UTF-8?q?docs(planning):=20correct=20the=20grade-t?= =?UTF-8?q?ail=20plan=20=E2=80=94=20Seam=202=20was=20already=20built,=20an?= =?UTF-8?q?d=20the=20axis=20is=20DETERMINISM?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two corrections to the plan of record I committed hours ago, both found by reading the code instead of my own summary of it. 1. SEAM 2 IS RETRACTED. I claimed no lease-expiry mechanism existed and that #451 "does not fire in practice despite being marked complete." It is complete and correct, and it lives in `modules/benchmark_grade.rs` — the file I had open when I wrote that. I grepped `modules/work.rs` for MY concept (an event) instead of for the JOB's name, found nothing, and reported an absence. `sweep_lapsed_bench_cards` is thorough: a pure `sweep_ready` truth table, artifact presence per bench kind, room-SCOPED close, provenance note, per-tick cap, error probe on refusal, live claims never preempted. [[read-the-code-you-intend-to-replace-before-designing-its-replacement]] And the 13 ungraded artifacts are not its business: MEASURED, all 13 carry `card: None, owner: None` — detached agent/solve artifacts with no board card at all (#425). The sweep operates on cards, so not seeing them is correct. 2. THE ⚠ BLOCK WAS SCOPED TOO BROADLY. I wrote "do not add a periodic sweep" as a flat taboo. The same module already ticks at 180s and its comment draws the real line: "actuators may tick; condition-polls may not." A tick that ACTUATES a time fact the wire cannot carry is permitted; ticking to ask "has anything become gradeable" is not. My rule would have blocked a correct build — a guard scoped too broadly, the same shape as #422. JOEL'S FRAMING, which is better than either and now leads the section: "if it's deterministic and not scan it or polling it's reliable." That explains WHY rather than restating the taboo. A scan's answer depends on when it ran, what was on disk, how results sorted, whether a cap truncated — all real here (the board lost 12 of 13 artifacts to a recency sort against a 200-tree cap). A deterministic actuator answers the same way every time regardless of timing. So the test is not "does it have a timer" but "given the same state, same outcome?" With the corollary today taught the hard way: determinism over the WRONG THING is worthless. The reaper fixed in f3cb3a65c was perfectly deterministic over a filename no writer had ever produced — it answered "nothing here" every single time. Reliable and blind are not the same property. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- docs/planning/BENCHMARK-GRADE-TAIL.md | 89 ++++++++++++++++++++------- 1 file changed, 67 insertions(+), 22 deletions(-) diff --git a/docs/planning/BENCHMARK-GRADE-TAIL.md b/docs/planning/BENCHMARK-GRADE-TAIL.md index 6be0d34f8..1a950ca49 100644 --- a/docs/planning/BENCHMARK-GRADE-TAIL.md +++ b/docs/planning/BENCHMARK-GRADE-TAIL.md @@ -8,9 +8,9 @@ in the loop. Today the first half works and the second does not. --- -## ⚠ THE MOVE THAT LOOKS RIGHT AND IS FORBIDDEN +## ⚠ THE MOVE THAT LOOKS RIGHT AND IS FORBIDDEN — with the line the doctrine actually draws -**Do not add a periodic sweep that scans for ungraded artifacts on a clock.** +**Do not add a periodic sweep that POLLS FOR A CONDITION.** `modules/benchmark_grade.rs` opens with the law: @@ -18,12 +18,51 @@ in the loop. Today the first half works and the second does not. > ([[the-whole-system-is-event-based-not-polling]]): nothing scans the board on a clock — > the transition event fires the grade. -I proposed exactly that sweep during the session before reading the module. It is the -shortest path to "grades appear", it would work, and it would quietly convert an -event-driven substrate into a polling one. The module's own doc is what caught it. +I proposed exactly that sweep before reading the module, and it would have quietly +converted an event-driven substrate into a polling one. -**The correct question is never "when do we scan?" — it is "which EVENT did we fail to -emit?"** Both answers are below, and both are events that already have a home. +**CORRECTION (2026-08-18), and it matters because I wrote the rule too broadly the first +time.** The same module already runs a 180s `tick`, and its own comment draws the real +distinction: + +> The one periodic ACTUATOR (doctrine: actuators may tick; condition-polls may not): lease +> expiry is a TIME fact with no wire event… + +So a tick is permitted when it ACTUATES a time fact the wire cannot carry. What is +forbidden is ticking to ask "has anything become gradeable yet" — a condition poll that +duplicates an event. My original ⚠ conflated the two and would have blocked a correct +build. + +**The correct question is still "which EVENT did we fail to emit?" — but ask it second.** +Ask first: *does the mechanism already exist and is it simply not reaching production?* +Both times I skipped that question today, the answer was yes. + +### The axis is DETERMINISM, not tick-vs-event (Joel, 2026-08-18) + +> "if it's deterministic and not scan it or polling it's reliable" + +That is the rule this doc should have led with, because it explains WHY the doctrine +exists rather than restating it as a taboo: + +- **A scan is unreliable by construction.** Whether it catches a thing depends on when it + ran, what happened to be on disk at that instant, how the results sorted, and whether a + cap truncated them. Every one of those is real here: the board's artifact scan is capped + at 200 trees and lost 12 of 13 artifacts to a recency sort against chatty run files. + Same input, different answer depending on timing — that is the definition of unreliable. +- **A deterministic actuator is reliable even though it ticks.** Boot enumerates the run + ledger, every record marked `running` becomes `failed`, and re-running changes nothing. + No ordering, no sampling, no cap. It answers the same way every time. + +So the test to apply to any mechanism in this tail is not "does it have a timer" but +**"given the same state, does it always produce the same outcome?"** The lease sweep passes +(a pure truth table over state + artifact presence). A "look around for work that seems +ungraded" pass fails, and would fail with or without a clock. + +**And determinism is worth nothing if it is deterministic about the wrong thing.** The +reaper fixed in `f3cb3a65c` was perfectly deterministic — over a filename no writer had +ever produced. It answered the same way every time: nothing here. Reliable and blind are +not the same property; the guard has to be pointed at what production actually writes, +which is why the naming now lives in exactly one place. --- @@ -69,24 +108,30 @@ Constraints: background work with the standard bounded-run discipline, not inline in the boot path. - Idempotent: a run that already has a `.grade.json` is skipped. -## Seam 2 — a lapsed lease is a state change, so emit it +## ~~Seam 2 — a lapsed lease is a state change, so emit it~~ — RETRACTED, IT IS ALREADY BUILT + +**This section's premise was wrong and is withdrawn (2026-08-18).** I wrote it after +grepping `modules/work.rs` for a lease-expiry event, finding nothing, and concluding #451 +"does not fire in practice despite being marked complete." + +#451 is complete, correct, and lives in `modules/benchmark_grade.rs` — the same file I had +open. `sweep_lapsed_bench_cards` runs on a 180s tick, and it is thorough: a pure +`sweep_ready` truth table (claimed/in-progress **and** lease lapsed **and** artifact on +disk), `bench_artifact_present` checking a dirty tree for SWE and a non-empty solution file +for gym, room-SCOPED close (the #345 write-half trap), a provenance note posted into the +room, a per-tick close cap, and an error probe on refusal. A live claim is never preempted. -`grep` for a lease-expiry event in `modules/work.rs` returns **nothing**. Expiry today is -*passive*: `work/list` computes "is this hold still live?" at read time. So a claim can -lapse with a finished patch under it and the board experiences no transition — which is -why #451 ("lapsed claim + artifact → auto-close → the one grade tail") does not fire in -practice despite being marked complete. +I grepped for my own concept — an event in the file I expected it in — instead of for the +job's name. [[read-the-code-you-intend-to-replace-before-designing-its-replacement]] -**Change:** make expiry emit `work.card.state_changed` like every other transition, so the -EXISTING grade-on-done subscriber picks it up with no new grading path. One emitter -([[the-same-bug-at-two-sites-is-a-missing-constraint]]) — `work/state` already owns that -event (`WORK_CARD_STATE_CHANGED`); expiry must go through it rather than growing a parallel -notification. +**Why the 13 artifacts sat ungraded anyway, measured rather than inferred:** all 13 carry +`card: None, owner: None`. They have **no cards at all** — they are detached `agent/solve` +run artifacts (#425). The sweep operates on board cards, so it can never see them, and that +is correct behaviour, not a gap in the sweep. -Open question for Joel, deliberately not decided here: does a lapsed claim with an artifact -auto-CLOSE the card, or move it to a `needs-verdict` state that a citizen can re-claim? -Auto-close is simpler; re-claimable is truer to #419 (recover a claim whose work session -died). This is a recipe/lifecycle policy call, not a plumbing detail. +The open policy question survives on its own merits and is still Joel's: does a lapsed +claim with an artifact auto-CLOSE (what the sweep does today) or move to a `needs-verdict` +state a citizen can re-claim (truer to #419)? Not a plumbing detail. --- From c12fe90e9c77276ab8e22d0355a3980e7da9272c Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 17:48:09 -0500 Subject: [PATCH 58/80] =?UTF-8?q?fix(benchmark):=20reap=20writes=20run=5Fi?= =?UTF-8?q?d,=20not=20runId=20=E2=80=94=20I=20shipped=20two=20names=20for?= =?UTF-8?q?=20one=20field=20while=20collapsing=20two=20names=20for=20one?= =?UTF-8?q?=20file?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught on the LIVE reap of 19 orphans, in the record the fix itself produced: "runId": "claim-4107b347-...", "run_id": "claim-4107b347-...", `agent/solve` — the writer of this ledger family — spells it `run_id` throughout. My reap inserted `runId`, so every reaped record now carried both, and no reader of THIS family reads the camelCase one. (`cognition/eval` uses `runId` for its OWN ledger family; that is a different family and is left alone.) Exactly the sin f3cb3a65c was about, one level down: that commit's message argues for one spelling per concept, and the same diff introduced a second spelling of a field. The filename drift took weeks to surface because nothing failed loudly; this one surfaced in minutes only because the reap finally ran on real data and I read the output instead of the exit code. Now `entry(...).or_insert_with(...)` — a record that already carries its id keeps it, and nothing invents a parallel key. LIVE POSITIVE CONTROL for the parent fix, same run: 19 running → 0 running, 19 journaled as killed-by-restart, 19 workspace pointers preserved, 19 instances preserved (one with `acts: 20` intact — real work the old blanket-overwrite would have erased). Deploy verified at 7a7a6b17b == git HEAD. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/cognition/swe_bench.rs | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 82bb7336c..1b0fa5700 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -265,7 +265,14 @@ pub fn reap_orphaned_solve_runs_in(dir: &Path) -> Vec { // `state` is what `in_flight_solve_runs_in` keys on. Clearing it off `running` is // what makes the reap idempotent — a second boot must not re-reap. obj.insert("state".into(), serde_json::Value::String("failed".into())); - obj.insert("runId".into(), serde_json::Value::String(run_id.clone())); + // `run_id`, NOT `runId`: this ledger family is written by `agent/solve`, which spells + // it snake_case throughout. The first cut of this reap inserted `runId` and shipped + // records carrying BOTH — two names for one field, in the same change that collapsed + // two names for one file. Caught on the live reap of 19 orphans. `entry` so a record + // that already carries its id keeps it untouched. (`cognition/eval` uses `runId` for + // its OWN ledger family; that is a different family and stays as it is.) + obj.entry("run_id") + .or_insert_with(|| serde_json::Value::String(run_id.clone())); obj.insert("instance".into(), serde_json::Value::String(instance)); obj.insert( "error".into(), From a35322a2ec20e09f4cacc72babcf77c7aae455e2 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:04:04 -0500 Subject: [PATCH 59/80] =?UTF-8?q?docs(architecture):=20run=20ledgers=20are?= =?UTF-8?q?=20typed=20artifacts=20with=20one=20owner=20=E2=80=94=20the=20d?= =?UTF-8?q?rift=20class,=20measured=20and=20designed=20out?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three bugs today, all the same bug, all in the progress-ledger surface: a filename spelled two ways, a directory spelled two ways, a field spelled two ways. Rather than a fourth point fix, this measures the class and designs it away. MEASURED: 6 ledger families, 5 independent spellings of the progress dir, ~29 `json!({…})` literals writing records, 13 stringly-typed `get("state")` reads, and ZERO shared types. Each family hand-rolls the same six decisions — directory, filename, field names, state vocabulary, enumeration, sibling rule. Six decisions × six families is the drift surface; today's bugs were three draws from it. WHY THIS CLASS IS UNIQUELY BRITTLE, and the part worth internalising: a mismatch produces an EMPTY READ, not an error. `read_dir` succeeds, the filter matches nothing, the caller gets `Vec::new()` and cannot distinguish it from "nothing to do". So the reboot guard spent weeks as a green light that had never once looked at a real run — deterministic, same answer every time, and blind. Determinism over the wrong thing is worthless; reliability needs reader and writer to be the SAME decision, not two that happen to agree. DESIGN, four layers, reusing the house `*CaptureSink` idiom (7 existing members) rather than inventing a shape: - L1 the record is a TYPE — serde owns field names, an enum owns the state vocabulary, so field drift is a compile error and state drift a parse error. - L2 `RunLedger` is the ONE owner of dir + naming + read/write/enumerate/sibling. No caller builds a path or a JSON key. `amend` merges — an annotation is never a replacement, which is what destroyed the `workspace` pointer this morning. - L3 reconciliation as a shared trait: enumerate, Running→Failed, payload preserved, idempotent. Joel's rule in code — deterministic, no scan, no poll. Every family inherits it instead of writing it (today one family has it and it pointed at nothing). - L4 THE LAW THAT ACTUALLY PREVENTS RECURRENCE: a test may not hand-author the artifact it reads; fixtures come from the production writer. The reaper test wrote `swe-solve-alive .json` by hand and asserted the real production file was "another subsystem's ledger, untouched" — it tested the reader against my belief about the writer, stayed green while the live path was dead, and documented the bug as intent. Build order follows the mandatory outlier process: agent-solve as outlier A, cognition/eval as outlier B chosen because it is maximally different (11 json! literals, separate progress/result/status records, its own camelCase spelling, live polling). If the interface fits both without forcing, the middle four are trivial; if it does not, redesign BEFORE migrating anything. Then the generator (`ledger/new`) and a CI guard in the same idiom as the de-hardcode guard — the tree already proves prose does not hold a convention. Scoped honestly: this does not make the grade tail fire. The 13 ungraded artifacts have no cards at all (#425). This removes the drift class beneath the benchmark work. It is plumbing and should be judged as plumbing. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md | 179 ++++++++++++++++++ 1 file changed, 179 insertions(+) create mode 100644 docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md diff --git a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md new file mode 100644 index 000000000..1291d5f76 --- /dev/null +++ b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md @@ -0,0 +1,179 @@ +# Run ledgers are typed artifacts with one owner + +**Status:** design of record, 2026-08-18. Written after three drift bugs in one afternoon, +all of them the same bug. + +**The one-line contract:** a run's on-disk state is a TYPE, and exactly one component +resolves its directory, its filename, its field names and its state vocabulary. No caller +builds a path or a JSON key. + +--- + +## The measured class + +`~/.continuum/progress/` holds every long-running job's state. Today: + +| Fact | Count | +|---|---| +| Ledger families (`agent-solve-`, `models-pull-`, `competition-`, eval, teach, fitness-sentinel) | 6 | +| Independent spellings of the progress directory | 5 | +| `json!({…})` literals writing records | ~29 | +| Stringly-typed `get("state")` reads | 13 | +| Shared types across families | **0** | + +Every family hand-rolls the same six decisions: where the directory is, what the file is +called, what the fields are called, what the states are called, how to enumerate, and how +to tell a record from its sibling (`X.grade.json`). Six decisions × six families = 36 +independent chances to disagree, and nothing forces agreement. + +### The three bugs today were three draws from that surface + +1. **Filename.** `agent/solve` wrote `agent-solve-.json`; the boot reaper and the + reboot guard read `swe-solve-*`, which nothing has ever written. 19 orphans frozen as + `running`, oldest 162 h. The reboot guard reported "nothing in flight" while 19 runs + said otherwise. +2. **Directory.** `solve_ledger_dir()` ignored `CONTINUUM_HOME`; the writer honoured it. + A second spelling of the root. +3. **Field.** My own fix inserted `runId` into a family that spells it `run_id`. Two names + for one field, in the commit that collapsed two names for one file. + +Same shape as `swe_cache_dir`'s documented history (two env roots, "77% have no +environment" vs the real 95%). This is not a benchmark problem. It is what an untyped +artifact does. + +--- + +## Why this class is uniquely brittle: the failure is an EMPTY READ + +A wrong path does not raise. `read_dir` succeeds, the filter matches nothing, the function +returns `Vec::new()`, and the caller cannot distinguish that from *"nothing to do."* + +That is the amplifier. A guard that returns empty **reports safety**. The reboot guard +was, for weeks, a green light that had never once looked at a real run. Deterministic — +same answer every time — and blind. + +> **Determinism over the wrong thing is worthless.** Reliability requires that the reader +> and the writer be the same decision, not two decisions that happen to agree. + +--- + +## The design + +Four layers. Reuses the house `*CaptureSink` idiom (7 existing members) rather than +inventing a new shape. + +### L1 — the record is a type + +```rust +/// One family's on-disk record. Serde owns the field names; the enum owns the vocabulary. +pub trait RunRecord: Serialize + DeserializeOwned + Send + Sync { + /// Filename prefix, e.g. "agent-solve-". The ONE place it is spelled. + const PREFIX: &'static str; + fn state(&self) -> RunState; + fn set_failed(&mut self, cause: &str); +} + +#[derive(Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "snake_case")] +pub enum RunState { Running, Failed, Complete } +``` + +Field drift becomes a **compile error**. State drift becomes a **parse error**. Neither is +reachable by writing a different string literal in a different file. + +### L2 — one owner of the artifact + +```rust +pub struct RunLedger { dir: PathBuf, _r: PhantomData } + +impl RunLedger { + pub fn open() -> Self; // resolves the root ONCE, honours CONTINUUM_HOME + pub fn write(&self, id: &RunId, rec: &R) -> io::Result<()>; + pub fn read(&self, id: &RunId) -> io::Result>; + pub fn iter(&self) -> impl Iterator; // skips siblings, never phantoms + pub fn amend(&self, id: &RunId, f: impl FnOnce(&mut R)) -> io::Result<()>; // merge, never clobber + pub fn sibling(&self, id: &RunId, kind: SiblingKind) -> PathBuf; // .grade.json etc +} +``` + +No caller builds a path. No caller writes a JSON key. `amend` exists because the reap's +blanket overwrite destroyed the `workspace` pointer — the only thing that could locate the +artifact. **An annotation is never a replacement.** + +### L3 — reconciliation is a trait, not per-family code + +```rust +pub trait Reconcile { fn reap_orphans(&self) -> Vec; } +``` + +Default impl over `RunLedger`: enumerate, every `Running` becomes `Failed` with a cause, +payload preserved, idempotent. This is Joel's rule in code — deterministic, no scan, no +poll, same answer given the same state. Every family inherits it instead of writing it +(today only one family has it, and it was pointed at nothing). + +### L4 — the anti-blindness law (this is the part that actually prevents recurrence) + +**A test may not hand-author the artifact it reads.** Fixtures come from the production +writer. + +Today's reaper test wrote `swe-solve-alive.json` by hand and asserted an +`agent-solve-other.json` was "another subsystem's ledger, untouched." It tested the reader +against *the test author's belief about the writer*, so it stayed green while the live path +was dead — and it actively documented the bug as intent. + +```rust +// The round-trip test every family gets for free. +#[test] +fn what_the_writer_writes_is_what_the_reader_finds() { + let led = RunLedger::::open_in(tmp.path()); + led.write(&id, &SolveRun::running(&instance, &workspace)); // PRODUCTION writer + assert_eq!(led.iter().count(), 1, "the reader finds what the writer wrote"); + assert_eq!(led.reap_orphans(), vec![id.clone()]); + let after = led.read(&id).unwrap().unwrap(); + assert_eq!(after.state(), RunState::Failed); + assert_eq!(after.workspace, workspace, "an annotation never erases the payload"); +} +``` + +This single test, existing for any family, makes bugs 1–3 unrepresentable. + +--- + +## Build order (outlier validation, per the methodical process) + +1. **L1+L2 against outlier A — `agent-solve`.** Simplest family: one file per run. Already + half-collapsed by `f3cb3a65c`, so this is finishing a move in flight. +2. **Outlier B — `cognition/eval`.** Deliberately the MOST different: 11 `json!` literals, + separate progress/result/status records, its own `runId` camelCase spelling, live + polling by `observe`. If the interface fits solve *and* eval without forcing, it fits + the middle four. **If it does not fit, redesign before migrating anything else** — that + is the point of picking the extremes. +3. **L3 + L4 on both.** Reap and the round-trip test. +4. **Generator.** `ledger/new ` scaffolds record + store binding + the round-trip + test, so family seven inherits all of this instead of hand-rolling it. Generators encode + the pattern; docs only describe it. +5. **CI guard**, same idiom as `no_new_hardcoded_context_or_prompt_size_constant…`: fail on + a new `join("progress")` outside `RunLedger`, and on a `json!({` literal written into + the progress root. The guard is what stops it regrowing after the migration. +6. **Migrate the remaining four**, one per PR. + +Steps 1–3 are the slice that pays for itself; 4–5 are what make it permanent. Do not skip 5 +— the tree already proves prose does not hold a convention. + +--- + +## Acceptance + +- Grep the tree: exactly ONE `join("progress")`, ONE spelling per family prefix, ZERO + `json!` literals writing a run record. +- Rename a field in a record struct → every reader fails to COMPILE, none fails silently. +- Point a reader at a family with no records → still empty, but no reader CAN be pointed at + the wrong family, because the family is a type parameter. +- Kill a run mid-flight, reboot → verdict exists, payload intact, second reboot changes + nothing. + +## What this does NOT do + +It does not make the grade tail fire. The 13 ungraded artifacts have no cards at all +(#425) — a separate gap. This removes the *drift* class beneath the benchmark work; it is +plumbing, and it should be judged as plumbing. From d335a4316b402fff088f49b8c02ddd4008cb77ed Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:07:38 -0500 Subject: [PATCH 60/80] =?UTF-8?q?docs(architecture):=20L3b=20=E2=80=94=20t?= =?UTF-8?q?he=20record=20carries=20PROGRESS,=20because=20liveness=20is=20n?= =?UTF-8?q?ot=20progress?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEASURED tonight, and it is the same defect one level up from the ones that motivated this doc. An ad-hoc grading job ran ~10 hours and produced NOTHING: worklist item 1's work tree (`swe/work/sympy__sympy-18057`) last touched 10:15, checked 20:06 — one file, zero output, zero verdicts, both task logs empty. I reported it as healthy SIX times on the strength of `pgrep` returning a pid. A process existing is not a process working. Identical in shape to health-from-connection- existence rather than acked delivery (#280), and to an empty read reported as "nothing to do" — which is the failure mode this very doc was written about. I wrote that section this morning and then made the error all afternoon. So `RunRecord` gains `heartbeat_ms` (last time the run PROVED progress, never last time it existed) and `deadline_ms`, with a derived `is_wedged(now)`. Then "wedged" is a state the reconciler settles deterministically — exactly like an orphan — instead of a condition a human has to happen to notice. The wedge class has history here (#385 inference await blocking its thread, #386 wedge-killed attempts burning as capability zeros); the missing piece was never detection logic, it was a record with a heartbeat in it. THE COROLLARY, and the reason this belongs in architecture rather than a runbook: the reboot guard I fixed this morning protects exactly the work that REGISTERS ITSELF IN A LEDGER. My grading job registered nowhere, so it was invisible to the guard by construction — not a bug in the guard, a consequence of living outside the type. Ad-hoc operator jobs sit outside every safety mechanism the substrate has: no wedge detection, no reap, no receipt, no guard. Either they enter through a ledger or they get none of it. Which is an argument for making the registered path the EASY path, because the ad-hoc one will otherwise keep being chosen under time pressure — by me, demonstrably, today. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md index 1291d5f76..12e9b2926 100644 --- a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md +++ b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md @@ -111,6 +111,39 @@ payload preserved, idempotent. This is Joel's rule in code — deterministic, no poll, same answer given the same state. Every family inherits it instead of writing it (today only one family has it, and it was pointed at nothing). +### L3b — the record carries PROGRESS, because liveness is not progress + +Added 2026-08-18 after the finding below, which is the same defect one level up from the +ones that motivated this doc. + +An ad-hoc grading job ran for ~10 hours and produced **nothing**: worklist item 1's work +tree was last touched at 10:15, checked at 20:06, one file, zero output, zero verdicts. It +was reported as healthy six times on the strength of `pgrep` returning a pid. A process +existing is not a process working — the same error as health-from-connection-existence +rather than acked delivery, and as an empty read reported as "nothing to do". + +So `RunRecord` must carry, and `RunLedger` must expose: + +```rust +fn heartbeat_ms(&self) -> u64; // last time the run PROVED progress, not last time it existed +fn deadline_ms(&self) -> Option; +``` + +with a derived `is_wedged(now)` — heartbeat older than its own declared cadence. Then +"wedged" is a state the reconciler can settle deterministically, exactly like an orphan, +instead of a thing a human has to notice. The wedge class already has history here (#385 +inference await blocking its thread, #386 wedge-killed attempts burning as capability +zeros); the missing piece was never detection logic, it was a record with a heartbeat in it. + +**Corollary, and it is the reason this belongs in the architecture and not in a runbook:** +the reboot guard fixed this morning protects exactly the work that *registers itself in a +ledger*. My grading job registered nowhere, so it was invisible to the guard by +construction — not a bug in the guard, a consequence of living outside the type. Ad-hoc +operator jobs are outside every safety mechanism the substrate has. Either they enter +through a ledger or they get no guard, no wedge detection, no reap, and no receipt. That is +an argument for making the registered path the *easy* path, since the ad-hoc one will +otherwise keep being chosen under time pressure — by me, demonstrably. + ### L4 — the anti-blindness law (this is the part that actually prevents recurrence) **A test may not hand-author the artifact it reads.** Fixtures come from the production From 9cffd65ff20cf6d0e4576ab3f8e8c2c6536150b5 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:11:06 -0500 Subject: [PATCH 61/80] =?UTF-8?q?docs(architecture):=20outlier=20B=20resul?= =?UTF-8?q?t=20=E2=80=94=20the=20draft=20did=20NOT=20fit,=20and=20eval=20h?= =?UTF-8?q?as=20the=20same=20orphan=20hole=20unfixed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel called outlier B first. It paid on the first file read: `cognition/eval` is INVERTED from `agent-solve` on the axis the draft assumed, so the interface as written was wrong. agent-solve cognition/eval written when at DISPATCH, rewritten in place at COMPLETION only (2 call sites) addressed by id in the FILENAME id in a FIELD, reverse-scan every line absence means never started "in flight, OR dead, OR bad id" reaper yes (fixed today) NONE THE FINDING, and eval's own source says it (eval.rs:1781-84): "a run that dies before `append_progress_ledger` leaves eval-status returning complete:false, row:null forever — indistinguishable from 'still starting', so the poller waits on a corpse (cost me two cycles staring at total:null)". They patched the ERROR path (`append_failed_ledger` exists for exactly this); a run that is KILLED — reboot, SIGKILL, crash — still reads as an eternal pending. Same hole agent-solve closed today, still open in eval, same shape as #137 and as tonight's wedged grader. WHAT IT FORCES, and it is NOT the obvious `RunLedger` with FilePerRun / AppendJsonl impls. That would have been the wrong abstraction, and seeing why is the whole value of running B early: **eval's .jsonl is not a run-state ledger, it is a RESULTS LOG** — append-only, one row per finished run, history-bearing, legitimately a different thing that should keep existing. eval is not storing state in a different layout; it stores no state at all. So the concepts separate instead of hiding behind a strategy param: - RunLedger — current state, one record per run, written AT DISPATCH. Uniform. - ResultsLog — append-only history. eval already has one; solve gets one for free. PROMOTED TO A HARD INVARIANT: the state record is written at DISPATCH; absence means never started, never "in flight". Every guard depends on it — reap needs a Running row to find, wedge detection needs a heartbeat to age, the reboot guard needs something to name. A family that writes only on completion is STRUCTURALLY unguardable no matter how good the guard is. Consequence: migrating eval is a BEHAVIOUR change (adds a dispatch row, gains a reaper), not a refactor, and its PR must say so. Had I built against solve first — as I proposed an hour ago — I'd have shipped an interface that fit A perfectly and discovered on eval that "absence" means two opposite things, after the migration. Cost of finding out this way: one file read. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md | 77 +++++++++++++++++-- 1 file changed, 70 insertions(+), 7 deletions(-) diff --git a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md index 12e9b2926..9494b1ecc 100644 --- a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md +++ b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md @@ -172,15 +172,78 @@ This single test, existing for any family, makes bugs 1–3 unrepresentable. --- +## Outlier B result (2026-08-18): the draft did NOT fit, and the reason is the design + +Read `cognition/eval` before building anything, per Joel. It is inverted from solve on the +axis that matters, and the draft above assumed solve's shape. + +| | `agent-solve` | `cognition/eval` | +|---|---|---| +| Written **when** | at DISPATCH, rewritten in place | at **COMPLETION** only (`append_progress_ledger`, 2 call sites) | +| Addressed by | `agent-solve-.json`, id in the FILENAME | `.jsonl`, id in a FIELD, reverse-scan every line | +| What absence means | never started | **"still in flight, or the id is wrong"** — three-way ambiguous | +| Reaper | yes (fixed today) | **none** | + +### The finding: eval cannot distinguish a live run from a corpse, and its own source says so + +`cognition/eval.rs:1781-84`, in-tree, written by whoever hit it: + +> a run that dies before `append_progress_ledger` leaves eval-status returning +> `complete:false, row:null` forever — indistinguishable from "still starting", so the +> poller waits on a corpse (cost me two cycles staring at `total:null` …) + +They patched the *error* path (`append_failed_ledger` exists precisely for this). A run +that is **killed** — reboot, SIGKILL, crash — still reads as an eternal pending. That is +the same orphan hole `agent-solve` closed with a dispatch-time marker plus a boot reaper, +still open in eval, and the same shape as #137 (train jobs dying silently across reboots) +and as tonight's wedged grader. + +### What that forces on the interface — and it is NOT a layout parameter + +The tempting move is `RunLedger` with `FilePerRun` and `AppendJsonl` impls. +That is wrong, and seeing why is the whole value of running B early: + +**eval's `.jsonl` is not a run-state ledger at all. It is a RESULTS LOG.** Append-only, +one row per finished run, history-bearing — a legitimately different thing that should +keep existing. What eval is missing is a *run-state record*, which it has never had. It +cannot tell alive from dead because it stores no state, not because it stores state in a +different layout. + +So the two concepts separate cleanly instead of being unified behind a strategy param: + +- **`RunLedger` — current state, one record per run, written AT DISPATCH.** Uniform + across every family. This is what reap, wedge detection and the reboot guard read. +- **`ResultsLog` — append-only history of finished runs.** eval's `.jsonl` already is one; + solve gets one for free instead of losing its per-run verdict files. + +### The law this promotes to a hard invariant + +> **The state record is written at DISPATCH. Absence means never started — never "in +> flight."** + +Every guard in the substrate depends on it. Reap needs a `Running` row to find; wedge +detection needs a heartbeat to age; the reboot guard needs something to name. A family +that writes only on completion is *structurally* unable to be guarded, no matter how good +the guard is. Solve already obeys this. eval does not, so **migrating eval is a behaviour +change, not a refactor** — it adds a dispatch-time row and gains a reaper — and its PR +must say so in those terms. + +Had I built against solve first, I would have shipped an interface that fit A perfectly and +discovered on eval that "absence" means two opposite things — after the migration. That is +the cost outlier B exists to avoid, and it was one file-read. + ## Build order (outlier validation, per the methodical process) -1. **L1+L2 against outlier A — `agent-solve`.** Simplest family: one file per run. Already - half-collapsed by `f3cb3a65c`, so this is finishing a move in flight. -2. **Outlier B — `cognition/eval`.** Deliberately the MOST different: 11 `json!` literals, - separate progress/result/status records, its own `runId` camelCase spelling, live - polling by `observe`. If the interface fits solve *and* eval without forcing, it fits - the middle four. **If it does not fit, redesign before migrating anything else** — that - is the point of picking the extremes. +Revised after the outlier-B read above, which changed step 1's shape before it was built. + +1. **L1+L2+L3b against outlier A — `agent-solve`**, with the record split confirmed by B: + `RunLedger` for state (dispatch-written, heartbeat-bearing) and its existing + `.grade.json` files recognised as the `ResultsLog` half. Already half-collapsed by + `f3cb3a65c`. +2. **Outlier B — `cognition/eval`.** Its `.jsonl` stays as the results log; it GAINS a + `RunLedger` state record written at dispatch, and inherits reap + wedge + detection it has never had. Ships as a behaviour change with the eternal-pending hole + named in the commit. 3. **L3 + L4 on both.** Reap and the round-trip test. 4. **Generator.** `ledger/new ` scaffolds record + store binding + the round-trip test, so family seven inherits all of this instead of hand-rolling it. Generators encode From 3ffe5704b2bd5c8a03ee2f5e733e7db71f1dc59d Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:15:55 -0500 Subject: [PATCH 62/80] =?UTF-8?q?docs(architecture):=20eval=20is=20the=20B?= =?UTF-8?q?YPASS,=20not=20a=20family=20to=20harden=20=E2=80=94=20outlier?= =?UTF-8?q?=20B=20swapped=20to=20models/pull=20(Joel)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, on reading the plan: "I have ptsd from your for use of eval, which totally bypassed room and kanban." He is right, and it lands on the design, not just the history. I picked `cognition/eval` as outlier B and then wrote a build step giving it a dispatch record, a reaper and wedge detection — i.e. making the parallel runner STURDIER. That is the move BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER forbids in as many words ("every patch to it deepens the hole"; "if you are adding a field to a benchmark probe so an external consumer can parse it better — STOP. The consumer should not be external"). I walked into it in the same document where I quoted the methodical process. Locally it looked like good engineering: a family with a real hole, and I had the fix in hand. CORRECTED: eval is NOT outlier B and nothing here licenses migrating it. Outlier B is now `models/pull` — maximally different on the axis that actually stresses the interface (continuous byte-progress over time vs a terminal verdict), and LEGITIMATE detached work: a model download genuinely has no room to live in. It is also where L3b earns its keep, since a stalled download is precisely a heartbeat that stops advancing while the process lives. THE EVAL FINDING SURVIVES AND FLIPS. "eval cannot tell a live run from a corpse — by its own source comment a killed run is an eternal pending forever" is not a gap to plug. It is evidence for ABSORBING measurement into rooms, where the turn IS the receipt: a citizen working in a room emits turns, and a room going quiet is legible with no ledger at all. eval needs a state record precisely BECAUSE it has no room. Give it one and you have paid for the bypass to survive. Filed against the absorption work (#425 / round lifecycle), not here. The structural section is kept and re-labelled as a study of a SHAPE, not a migration plan — the write-only-on-completion inversion is real wherever it appears, and it is what reshaped RunLedger vs ResultsLog. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md | 47 +++++++++++++++++-- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md index 9494b1ecc..3d2194e6c 100644 --- a/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md +++ b/docs/architecture/RUN-LEDGERS-ARE-TYPED-ARTIFACTS.md @@ -172,8 +172,44 @@ This single test, existing for any family, makes bugs 1–3 unrepresentable. --- +## ⚠ DO NOT USE THIS TO HARDEN THE BYPASS (Joel, 2026-08-18) + +**`cognition/eval` is not a family to improve. It is the room-and-kanban bypass.** + +I chose it as outlier B and then wrote a build step to give it a dispatch record, a reaper +and wedge detection. That is making the parallel runner *sturdier*, which +[BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER](BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md) forbids in +as many words — *"every patch to it deepens the hole"*, and *"if you are adding a field to a +benchmark probe so an external consumer can parse it better — STOP. The consumer should not +be external."* Joel's objection, and he is right: measurement that runs outside the room +produces no room turns, and a nicer ledger makes that permanent instead of temporary. + +**So eval is NOT outlier B, and nothing in this doc licenses migrating it.** Outlier B is +now `models/pull` — see below. It is maximally different in the way that matters +(continuous byte-progress over time vs a terminal verdict) and it is *legitimate* +infrastructure: pulling a model genuinely is a long detached job with no room to live in. + +**What survives from the eval read, reframed.** The finding stands and gets STRONGER, but +it argues the opposite of what I had it arguing: + +> eval cannot distinguish a live run from a corpse — by its own source comment, a killed +> run reads as an eternal pending forever. + +That is not a gap to plug in eval. It is evidence for absorbing measurement into rooms, +where **the turn IS the receipt**: a citizen working in a room emits turns, and a room that +goes quiet is legible without any ledger at all. eval needs a state record precisely +*because* it has no room. Give it one and you have paid for the bypass to survive. + +Filed against the absorption work (#425 / the round-lifecycle line), not against this doc. + +--- + ## Outlier B result (2026-08-18): the draft did NOT fit, and the reason is the design +*(Kept because the STRUCTURAL finding is what reshaped the interface, and it holds +regardless of eval's fate — the same inversion exists wherever a family writes only on +completion. Read it as a study of a shape, not as a migration plan.)* + Read `cognition/eval` before building anything, per Joel. It is inverted from solve on the axis that matters, and the draft above assumed solve's shape. @@ -240,10 +276,13 @@ Revised after the outlier-B read above, which changed step 1's shape before it w `RunLedger` for state (dispatch-written, heartbeat-bearing) and its existing `.grade.json` files recognised as the `ResultsLog` half. Already half-collapsed by `f3cb3a65c`. -2. **Outlier B — `cognition/eval`.** Its `.jsonl` stays as the results log; it GAINS a - `RunLedger` state record written at dispatch, and inherits reap + wedge - detection it has never had. Ships as a behaviour change with the eternal-pending hole - named in the commit. +2. **Outlier B — `models/pull`** (swapped off `cognition/eval`, see the ⚠ above). Maximally + different on the axis that stresses the interface: **continuous progress** (bytes of N, + updated throughout) rather than a terminal verdict, and no natural "room" to belong to — + pulling a model IS legitimately a detached job. If `RunLedger` + `heartbeat_ms` carries + a download's live progress AND a solve's terminal state without forcing, it carries the + rest. This is also the family where L3b earns its keep: a stalled download is exactly a + heartbeat that stops advancing while the process stays alive. 3. **L3 + L4 on both.** Reap and the round-trip test. 4. **Generator.** `ledger/new ` scaffolds record + store binding + the round-trip test, so family seven inherits all of this instead of hand-rolling it. Generators encode From c31600ee49faef98cf7de274d86fa970cb3e136d Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:40:35 -0500 Subject: [PATCH 63/80] =?UTF-8?q?fix(benchmark):=20verdicts=20PERSIST=20?= =?UTF-8?q?=E2=80=94=20a=20measurement=20the=20system=20cannot=20remember?= =?UTF-8?q?=20is=20not=20a=20measurement?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE PLUMBING FIX THAT MAKES EVERYTHING ELSE COUNTABLE. `benchmark/swe-grade` computed a verdict, appended it to the citizen's experience stream, and RETURNED it — writing nothing durable. So: - Two REAL SWE-bench Lite resolutions were graded and watched passing on 2026-08-18 — astropy-14995 and pytest-11143, each FAIL_TO_PASS 1/1, PASS_TO_PASS 40/40 — and left no trace. `benchmark/runs` went on rendering both artifacts as `ungraded`. - Measured the same evening: 37 grade artifacts on disk, ALL from the detached-solve path, and ZERO from any operator or workspace grade, because that arm had no persistence at all. - Consequence: the day's honest pass rate could not be stated from anything the system held. Not "our rate is low" — we could not produce a defensible number, because the numbers evaporated. FIX, as the results-log half of [[run-ledgers-are-typed-artifacts]]: the run LEDGER is current state keyed by RUN; a verdict is HISTORY keyed by INSTANCE — because grading is per-instance and a workspace grade legitimately has no run id. `swe_bench::verdict_dir/ verdict_path/record_verdict/read_verdict/recorded_verdicts` own it in one place, under the governed benchmarks root beside work/ and envs/. `record_verdict` REFUSES two things by construction, because both would launder the board: - a GOLD verdict — the positive control proves the ENV, not the citizen; a gold pass recorded as an instance verdict reads on the board as our result; - an ERRORED verdict — that is an absence (clone/env/patch fault), never a score, and tallying it is exactly how a broken harness becomes a number (#384). Persistence failure warns LOUD but never fails the grade: the verdict in hand is still true, and refusing to return it would lose the measurement twice. The board's "has this been scored" is now the UNION of run-grade siblings and recorded verdicts, which is what lets a real pass stop an artifact reading `ungraded` even when it came from an operator grade with no run. TEST goes through the PRODUCTION writer and the board's real readers — a hand-authored fixture would test my belief about the writer instead of the writer, which is precisely how the boot reaper stayed green while pointed at a filename nothing emits. Asserts round-trip fidelity, and that gold + errored verdicts never reach the record. 21 swe_bench + 40 benchmark tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 149 ++++++++++++++++++ core/continuum-core/src/commands/benchmark.rs | 36 ++++- 2 files changed, 184 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 1b0fa5700..466eeec5f 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -327,6 +327,84 @@ pub fn swe_cache_dir() -> PathBuf { .join("swe") } +/// Where a scored instance's verdict lives — the DURABLE record that a grade happened. +/// One file per instance under the governed benchmarks root, beside `work/` and `envs/`. +/// +/// # Why this exists (2026-08-18, and it cost the day's only two passes) +/// +/// `benchmark/swe-grade` computed a verdict, appended it to the citizen's experience +/// stream, and RETURNED it. Nothing durable was written. So two real SWE-bench Lite +/// resolutions — astropy-14995 and pytest-11143, each FAIL_TO_PASS 1/1 and PASS_TO_PASS +/// 40/40, watched passing live — left no trace, and `benchmark/runs` kept rendering both +/// artifacts as `ungraded`. Measured the same afternoon: **37 grade artifacts on disk from +/// the detached-solve path, and 0 from any operator or workspace grade**, because that arm +/// had no persistence at all. +/// +/// A measurement the system cannot remember is not a measurement. This is the results-log +/// half of [[run-ledgers-are-typed-artifacts]]: the run LEDGER is current state, keyed by +/// run; the verdict is HISTORY, keyed by INSTANCE — because grading is per-instance and a +/// workspace grade legitimately has no run id at all. +pub fn verdict_dir() -> PathBuf { + swe_cache_dir().join("verdicts") +} + +/// This instance's verdict file. One naming, so writer and board reader cannot drift +/// (the failure mode that cost the boot reaper weeks — see [`SOLVE_LEDGER_PREFIX`]). +pub fn verdict_path(instance_id: &str) -> PathBuf { + verdict_dir().join(format!("{instance_id}.json")) +} + +/// Persist a REAL verdict. Returns the path written. +/// +/// Refuses two things by construction, because both would launder the board: +/// - an **errored** verdict — that is an ABSENCE (clone/env/patch fault), never a score, +/// and recording it would tally a broken harness as a citizen's failure (the #384 class); +/// - a **gold-gate** verdict — the positive control proves the ENV, not the citizen. A gold +/// pass recorded as an instance verdict would read on the board as our result. +pub fn record_verdict(verdict: &SweVerdict, is_gold: bool) -> Result, String> { + if is_gold || verdict.error.is_some() || verdict.instance_id.is_empty() { + return Ok(None); + } + let dir = verdict_dir(); + std::fs::create_dir_all(&dir).map_err(|e| format!("create {}: {e}", dir.display()))?; + let path = verdict_path(&verdict.instance_id); + let body = serde_json::to_string_pretty(verdict).map_err(|e| e.to_string())?; + std::fs::write(&path, body).map_err(|e| format!("write {}: {e}", path.display()))?; + Ok(Some(path)) +} + +/// Read this instance's recorded verdict, or `None` if it has never been scored. +pub fn read_verdict(instance_id: &str) -> Option { + let text = std::fs::read_to_string(verdict_path(instance_id)).ok()?; + serde_json::from_str(&text).ok() +} + +/// Every instance that carries a durable verdict, as `(instance_id, verdict)`. +/// The board's source of "has this been scored", so an artifact stops reading `ungraded` +/// the moment a real grade lands — and keeps reading scored across reboots. +pub fn recorded_verdicts() -> Vec<(String, SweVerdict)> { + let Ok(entries) = std::fs::read_dir(verdict_dir()) else { + return Vec::new(); + }; + let mut out = Vec::new(); + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().and_then(|e| e.to_str()) != Some("json") { + continue; + } + let Ok(text) = std::fs::read_to_string(&path) else { + continue; + }; + if let Ok(v) = serde_json::from_str::(&text) { + if !v.instance_id.is_empty() { + out.push((v.instance_id.clone(), v)); + } + } + } + out.sort_by(|a, b| a.0.cmp(&b.0)); + out +} + /// Fetch a dataset split, cached on first use. On-demand, never a gated install step. pub async fn load_dataset(dataset: &str) -> Result, String> { let cache = swe_cache_dir().join(format!("{}.json", dataset.replace('/', "__"))); @@ -2890,6 +2968,77 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. ); } + // what this catches: a REAL verdict must survive the process that produced it, and the + // two verdicts that must NEVER be recorded must not be. Before 2026-08-18 nothing was + // recorded at all: two genuine SWE-bench Lite resolutions were watched passing and the + // system retained nothing, so the board kept calling their artifacts `ungraded`. + // + // The fixture goes through `record_verdict` — the PRODUCTION writer — and is read back + // through `read_verdict`/`recorded_verdicts`, the readers the board actually uses. A + // hand-authored file here would test my belief about the writer instead of the writer, + // which is exactly how the boot reaper stayed green while pointed at a filename nothing + // emits ([[run-ledgers-are-typed-artifacts]] L4). + #[test] + fn a_real_verdict_persists_and_gold_or_errored_ones_never_do() { + let home = tempfile::tempdir().expect("tmp"); + // `verdict_dir` derives from CONTINUUM_HOME via swe_cache_dir; isolate this test's + // writes rather than touching the operator's real benchmarks root. + let prev = std::env::var("CONTINUUM_HOME").ok(); + std::env::set_var("CONTINUUM_HOME", home.path()); + + let real = SweVerdict { + instance_id: "astropy__astropy-14995".into(), + resolved: true, + f2p_passed: 1, + f2p_total: 1, + p2p_passed: 40, + p2p_total: 40, + gate_ok: true, + ..Default::default() + }; + assert!(record_verdict(&real, false).unwrap().is_some()); + + let back = read_verdict("astropy__astropy-14995").expect("a recorded verdict reads back"); + assert!(back.resolved, "resolution survives the round trip"); + assert_eq!((back.f2p_passed, back.p2p_total), (1, 40), "counts survive too"); + assert_eq!( + recorded_verdicts().len(), + 1, + "and the board's enumerator sees exactly it" + ); + + // A gold pass proves the ENV, not the citizen. Recording it would render a positive + // control as our result. + let gold = SweVerdict { + instance_id: "sympy__sympy-24152".into(), + resolved: true, + ..Default::default() + }; + assert!(record_verdict(&gold, true).unwrap().is_none(), "gold never records"); + + // An errored verdict is an ABSENCE (clone/env fault), never a scored zero (#384). + let errored = SweVerdict { + instance_id: "django__django-11049".into(), + error: Some("env build failed".into()), + ..Default::default() + }; + assert!( + record_verdict(&errored, false).unwrap().is_none(), + "an errored run is an absence, not a tallied failure" + ); + + assert_eq!( + recorded_verdicts().len(), + 1, + "neither the control nor the fault reached the durable record" + ); + + match prev { + Some(v) => std::env::set_var("CONTINUUM_HOME", v), + None => std::env::remove_var("CONTINUUM_HOME"), + } + } + // what this catches: the run-id parse must accept the name the writer emits and reject // the grade sibling. Both halves were re-derived per call site before 2026-08-18, and // the two spellings diverged (`swe-solve-` vs `agent-solve-`), which silently disarmed diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 220ffaa04..e0f3bff28 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2229,6 +2229,31 @@ pub(crate) async fn grade_swe(p: SweGradeParams) -> Result crate::probe!( + class = "benchmark.verdict.recorded", + instance = %verdict.instance_id, + resolved = verdict.resolved, + path = %path.display(), + "verdict persisted — the board and every later reader now see this grade" + ), + Ok(None) => {} + // Fail LOUD in the log but never fail the grade: the verdict in hand is still true, + // and refusing to return it would lose the measurement twice over. + Err(e) => tracing::warn!( + instance = %verdict.instance_id, + error = %e, + "VERDICT NOT PERSISTED — this grade is real but the system will forget it" + ), + } + // #319: a WORKSPACE grade is a citizen's lived, objectively judged work — // append it to her experience stream. Only her: the gold/raw-patch arms are // harness plumbing, not experience. And only a REAL verdict: an errored run @@ -3194,11 +3219,20 @@ pub(crate) fn scan_run_cards( // artifact has no run id to match. See `scan_workspace_artifact_cards` for why the board // cannot be run-files-only. if run_id_filter.is_none() { - let graded: std::collections::HashSet = cards + // "Scored" is the union of two sources, and it MUST be: a run ledger's own grade + // sibling, AND the durable per-instance verdict record. An operator or workspace + // grade has no run id at all, so before verdicts were recorded (2026-08-18) a real + // pass could not make an artifact stop reading `ungraded` — two of them didn't. + let mut graded: std::collections::HashSet = cards .iter() .filter(|c| c.resolved.is_some()) .filter_map(|c| c.instance.clone()) .collect(); + graded.extend( + swe_bench::recorded_verdicts() + .into_iter() + .map(|(instance, _)| instance), + ); cards.extend(scan_workspace_artifact_cards(&graded, now_ms)); } cards.sort_by(|a, b| b.last_activity_ms.cmp(&a.last_activity_ms)); From c453ee865a661c6580994704b2f98a2143dcae38 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 20:55:06 -0500 Subject: [PATCH 64/80] =?UTF-8?q?fix(benchmark):=20an=20EMPTY=20candidate?= =?UTF-8?q?=20is=20an=20absence,=20not=20a=20zero=20=E2=80=94=20the=20posi?= =?UTF-8?q?tive=20control=20caught=20my=20own=20laundering=20bug=20in=20on?= =?UTF-8?q?e=20run?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shipped verdict persistence, ran the control immediately, and it failed in exactly the way that matters. Re-graded astropy-14995 — a REAL pass, watched at FAIL_TO_PASS 1/1 and PASS_TO_PASS 40/40 that same afternoon — and got: patchBytes: 0, resolved: false, gateOk: true …and PERSISTED it, as her verdict for that instance. TWO findings. 1) THE BUG I JUST SHIPPED. Grading a pristine tree at `base_commit` ALWAYS returns resolved=false with gate_ok=true — the FAIL_TO_PASS test correctly fails because the bug is still there. That is byte-identical downstream to a citizen who tried and missed. So the harness scored its own absence as her failure: a manufactured capability zero, the #384/#386 class, in a file whose `gold_gate` comment one screen above exists to prevent precisely this. I wrote a doc against tallying absences this morning and shipped one this evening; persistence made it durable, which is what turned an invisible defect into a visible one. FIX: refuse before spending the clone — a non-gold grade with a zero-byte candidate returns an ERRORED verdict naming the cause. `record_verdict` already refuses errored verdicts, and the experience-stream append already gates on `error.is_none()`, so one guard makes the board honest AND stops teaching "you failed" from a tree that was wiped. 2) HER ARTIFACT WAS DESTROYED, and that is a separate defect. `git reflog` in her astropy-14995 workspace shows exactly two entries — `clone` then `checkout` to base_commit. The tree was RE-CLONED, wiping uncommitted work that had already been proven to resolve. 14 other staged trees still hold their diffs, so this is not a global sweep. Filed as its own investigation; a citizen's proven artifact must not be silently reclonable (the #312 family — infrastructure touching her hands). The manufactured verdict has been deleted from the durable record: she never failed that instance, and the system should not remember that she did. 61 tests green across swe_bench + benchmark. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 8 +++-- core/continuum-core/src/commands/benchmark.rs | 34 +++++++++++++++++++ 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 466eeec5f..90511639a 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -3016,10 +3016,14 @@ diff --git a/sympy/solvers/tests/test_other.py b/sympy/solvers/tests/test_other. }; assert!(record_verdict(&gold, true).unwrap().is_none(), "gold never records"); - // An errored verdict is an ABSENCE (clone/env fault), never a scored zero (#384). + // An errored verdict is an ABSENCE, never a scored zero (#384). Two ways in: an env + // fault, and — found by the live positive control the day this landed — an EMPTY + // candidate. A pristine tree grades `resolved: false, gate_ok: true` forever, which + // is indistinguishable from a citizen who tried and missed, so `swe-grade` now stamps + // an empty candidate as an error rather than letting the board score her absence. let errored = SweVerdict { instance_id: "django__django-11049".into(), - error: Some("env build failed".into()), + error: Some("no candidate patch to grade — the workspace holds no diff".into()), ..Default::default() }; assert!( diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index e0f3bff28..a84f154e7 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2201,6 +2201,40 @@ pub(crate) async fn grade_swe(p: SweGradeParams) -> Result Date: Tue, 18 Aug 2026 21:04:22 -0500 Subject: [PATCH 65/80] =?UTF-8?q?fix(benchmark):=20swe=5Fcache=5Fdir=20bec?= =?UTF-8?q?ame=20the=20second=20root=20its=20own=20doc=20warns=20about=20?= =?UTF-8?q?=E2=80=94=20test=20was=20writing=20into=20the=20operator's=20li?= =?UTF-8?q?ve=20record?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by re-reading my own test after the verdict-persistence landing, and it is the funniest possible instance of this bug: `swe_cache_dir`'s doc block is a 15-line essay about how "a cache with two roots cannot report its own coverage, because every reader picks one and gets a self-consistent lie" — written after a two-root incident cost a full misdiagnosis. The function then resolved `HOME` directly while `solve_ledger_dir` and every other progress reader resolve `CONTINUUM_HOME`. Two roots. Again. In the function that documents the rule. HOW IT SURFACED, and it is the cheap way: the new `record_verdict` test set `CONTINUUM_HOME` to a tempdir to isolate its writes. Measured — real verdicts root before the test: 0 files; after: 1 file, `astropy__astropy-14995.json`. The isolation was VACUOUS and the unit test was seeding the operator's live verdict record with fixture data. A fixture that would later read as a genuine measurement, in the exact record I built hours ago to make measurements trustworthy. Worth stating plainly because it changes an earlier reading: the "verdicts dir before: 1 file" I saw when running the live grade was MY TEST'S artifact, not a pre-existing verdict. The live-grade finding stands unchanged (patchBytes 0 → a false zero got recorded, fixed in c453ee865) — that was inspected directly — but the count next to it was polluted. FIX: `swe_cache_dir` derives from `continuum_home()` like everything else, with the bare-HOME path kept only as the no-home-dir fallback. Verified: real verdicts root is 0 before AND after the full 61-test run. [[the-same-bug-at-two-sites-is-a-missing-constraint-not-two-bugs]] — and a doc that states the rule does not enforce it. That is what the CI guard in the ledger design is for. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/cognition/swe_bench.rs | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 90511639a..410630cb0 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -319,10 +319,21 @@ pub fn reap_orphaned_solve_runs() -> Vec { /// /// Corollary for anyone measuring env coverage: read the root from here, never from a path /// you remember or a directory you found by name. +/// ONE MORE ROOT-DERIVATION NOTE, added 2026-08-18 because this function became the very +/// thing its doc warns about. It resolved `HOME` directly while `solve_ledger_dir` and every +/// other progress reader resolve `CONTINUUM_HOME` — so the benchmarks root and the ledger +/// root were TWO roots again, exactly the shape described above. +/// +/// It surfaced as test pollution, which is the cheap way to find it: a unit test set +/// `CONTINUUM_HOME` to a tempdir to isolate its writes, and `record_verdict` wrote into the +/// OPERATOR'S REAL `~/.continuum/benchmarks/swe/verdicts` anyway. The isolation was vacuous +/// and the test was quietly seeding the live verdict record with fixture data — a fixture +/// that would later read as a genuine measurement. pub fn swe_cache_dir() -> PathBuf { - let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".into()); - PathBuf::from(home) - .join(".continuum") + crate::commands::benchmark::continuum_home() + .unwrap_or_else(|_| { + PathBuf::from(std::env::var("HOME").unwrap_or_else(|_| "/tmp".into())).join(".continuum") + }) .join("benchmarks") .join("swe") } From ef0eb5bbff577a831e230d28f9ef38fb72d9d590 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 21:31:11 -0500 Subject: [PATCH 66/80] fix(benchmark): grade RESOLVES which citizen's copy, instead of trusting the caller to have picked right MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE FALSE-ZERO GENERATOR, found live and it is not what I first claimed. The same instance is LEGITIMATELY staged into several citizens' workspaces — dispatch round-robins over the roster. Measured 2026-08-18: `astropy__astropy-14995` sits in e5f4141d (Atlas) DIRTY — M astropy/nddata/mixins/ndarithmetic.py, +4/-5, the real fix 90e758b2 (Asha) CLEAN — staged 08-17, never worked I graded Asha's copy from a hand-built worklist that had mapped the instance to the wrong peer, got `patchBytes: 0, resolved: false`, and read a legitimately-fresh staging as a DESTROYED artifact — "only two reflog entries" is what never-worked looks like, not what deletion looks like. Retracted in full: nothing was destroyed, both staging guards are sound (`swe-setup` refuses without fresh=true; dispatch skips when .git exists), and there is no destroyer to find. The risk was never deletion. It was AMBIGUITY. FIX: an omitted `workspace` no longer means "no candidate" — it means ASK. Resolution goes through `persona::staged_workspace`, the module that already owns "which checkout is this instance" and already refuses on ambiguity, gaining the inverse it was missing: `owners_of(instance)` → every citizen holding it, with `has_work` from a real dirty check, and `grade_target` → the pure decision over that list. The rule keys on WORK, not presence, and mirrors its sibling `select`: exactly one worked copy → grade it (probe names the path and the copy count) no worked copy → ABSENCE, falls into the empty-candidate guard, never a zero two+ worked copies → REFUSE, naming candidates — grading either scores one citizen's diff against the other's card Also repointed the experience-stream append at the RESOLVED workspace, so a resolved grade teaches the citizen who actually did the work rather than whoever the operator happened to name. Tests: the truth table over all four rows, as a table — no disk fixture re-deriving the rule. 5 staged_workspace tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 59 +++++++- .../src/persona/staged_workspace.rs | 126 ++++++++++++++++++ 2 files changed, 182 insertions(+), 3 deletions(-) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index a84f154e7..6d3d04214 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -2192,9 +2192,60 @@ pub(crate) async fn grade_swe(p: SweGradeParams) -> Result = match p.workspace.clone() { + Some(ws) => Some(ws), + None if p.gold.unwrap_or(false) || p.patch.is_some() => None, + None => { + use crate::persona::staged_workspace::{grade_target, owners_of, GradeTarget}; + let copies = owners_of(&instance.instance_id); + match grade_target(&copies) { + GradeTarget::One(path) => { + crate::probe!( + class = "benchmark.grade.workspace_resolved", + instance = %instance.instance_id, + staged_copies = copies.len(), + path = %path.display(), + "resolved the one WORKED staged copy — never guessed between citizens" + ); + Some(path.to_string_lossy().to_string()) + } + // No worked copy: fall through with no candidate. The empty-candidate guard + // below turns that into an ABSENCE, which is the honest verdict. + GradeTarget::NoWork => None, + GradeTarget::Ambiguous(paths) => { + return Err(CommandError::Invalid(format!( + "{} is staged with real work in {} citizens' workspaces — refusing to \ + guess which one this grade is about, because grading either scores one \ + citizen's diff against the other's card. Pass workspace= \ + explicitly. Candidates: {}", + instance.instance_id, + paths.len(), + paths + .iter() + .map(|p| p.display().to_string()) + .collect::>() + .join(", ") + ))); + } + } + } + }; + let candidate: Option = if p.gold.unwrap_or(false) { Some(instance.patch.clone()) - } else if let Some(ws) = p.workspace.as_ref() { + } else if let Some(ws) = resolved_workspace.as_ref() { Some(workspace_candidate_diff(ws)?) } else { p.patch.clone() @@ -2294,8 +2345,10 @@ pub(crate) async fn grade_swe(p: SweGradeParams) -> Result Vec { + let Ok(home) = crate::commands::benchmark::continuum_home() else { + return Vec::new(); + }; + let Ok(entries) = std::fs::read_dir(home.join("citizens").join("peers")) else { + return Vec::new(); + }; + let mut out = Vec::new(); + for entry in entries.flatten() { + let Some(peer) = entry + .file_name() + .to_str() + .and_then(|n| uuid::Uuid::parse_str(n).ok()) + else { + continue; + }; + let path = entry.path().join("workspace").join("swe").join(instance); + if !path.join(".git").exists() { + continue; + } + let has_work = std::process::Command::new("git") + .arg("-C") + .arg(&path) + .args(["status", "--porcelain"]) + .output() + .map(|o| o.status.success() && !o.stdout.is_empty()) + .unwrap_or(false); + out.push(StagedCopy { + peer, + path, + has_work, + }); + } + out.sort_by(|a, b| a.peer.cmp(&b.peer)); + out +} + +/// Which staged copy to grade: the decision, with the filesystem taken out of it. +/// +/// Split from [`owners_of`] for the same reason [`select`] is split below — the RULE is +/// tested against a table, not against a disk fixture that re-derives it. +#[derive(Debug, PartialEq, Eq)] +pub enum GradeTarget { + /// Exactly one copy carries work. Grade it. + One(PathBuf), + /// No copy carries work — an ABSENCE. Nothing to score; never a zero. + NoWork, + /// Two or more citizens hold worked copies. Refuse: grading either scores one + /// citizen's diff against the other's card. + Ambiguous(Vec), +} + +pub fn grade_target(copies: &[StagedCopy]) -> GradeTarget { + let worked: Vec<&StagedCopy> = copies.iter().filter(|c| c.has_work).collect(); + match worked.as_slice() { + [one] => GradeTarget::One(one.path.clone()), + [] => GradeTarget::NoWork, + many => GradeTarget::Ambiguous(many.iter().map(|c| c.path.clone()).collect()), + } +} + /// The matching rule alone, with the filesystem taken out of it. /// /// Split from [`resolve_for_titles`] so the rule is TESTED rather than restated: the @@ -225,4 +312,43 @@ mod tests { let peer = uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, b"nothing-staged-fixture"); assert!(workspace_for_held_cards(&peer, ["benchmark: anything"]).is_none()); } + + fn copy(name: &str, has_work: bool) -> StagedCopy { + StagedCopy { + peer: uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, name.as_bytes()), + path: PathBuf::from(format!("/peers/{name}/workspace/swe/astropy__astropy-14995")), + has_work, + } + } + + // what this catches: the false-zero generator. The SAME instance is legitimately staged + // into several citizens' workspaces (dispatch round-robins the roster). Measured live + // 2026-08-18: astropy-14995 sat in Atlas's tree DIRTY (a real fix) and Asha's tree CLEAN + // (staged, never worked). Grading the clean one returns resolved=false — a confident zero + // for work that existed ten directories away. + // + // The rule must therefore key on WORK, not on presence, and must refuse rather than pick + // when two citizens both worked it: grading either scores one citizen's diff against the + // other's card. Same posture as `select` above — ambiguity is reported, never resolved. + #[test] + fn the_worked_copy_is_graded_and_two_worked_copies_refuse() { + // one staged, unworked → an ABSENCE, never a zero + assert_eq!(grade_target(&[copy("asha", false)]), GradeTarget::NoWork); + + // the live shape: two staged, exactly one worked → grade the worked one + let atlas = copy("atlas", true); + match grade_target(&[copy("asha", false), atlas.clone()]) { + GradeTarget::One(p) => assert_eq!(p, atlas.path, "the WORKED copy, not the first"), + other => panic!("expected the worked copy, got {other:?}"), + } + + // two citizens both worked it → refuse, naming both + match grade_target(&[copy("atlas", true), copy("anwen", true)]) { + GradeTarget::Ambiguous(paths) => assert_eq!(paths.len(), 2), + other => panic!("two worked copies must never resolve, got {other:?}"), + } + + // nothing staged at all → also an absence, not a panic + assert_eq!(grade_target(&[]), GradeTarget::NoWork); + } } From a7ddaf83d27b69d93ec661f673be96b2e030c096 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 21:51:12 -0500 Subject: [PATCH 67/80] =?UTF-8?q?fix(benchmark):=20a=20verdict=20EMITS=20a?= =?UTF-8?q?=20board=20row=20=E2=80=94=20until=20now=20it=20only=20subtract?= =?UTF-8?q?ed,=20making=20a=20scored=20instance=20LESS=20visible=20than=20?= =?UTF-8?q?an=20unscored=20one?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The last link in the grade tail, and the acceptance test from BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER finally passing for scores. Verdict persistence landed and the board STILL could not show a pass. Measured minutes after: astropy-14995 graded `resolved=true, F2P 1/1, P2P 40/40`, verdict on disk and readable — and `benchmark/runs` reported `resolved: 1`, still counting only an old sympy row from 6 days ago. Three rows for 14995 read `failed`, which was HONEST: those are the three solve RUNS that died at the reboot. A run and a verdict are different objects. The board projected runs and artifacts; a verdict had nowhere to appear. Worse, and this is the part worth naming: `recorded_verdicts` was wired to SUBTRACT only. It marked an instance graded so the artifact row disappeared — so scoring an instance made it VANISH from the board. The one action that produces knowledge was the one action that removed the evidence. FIX: `scan_verdict_cards` is the third row source beside runs and artifacts. The verdict IS the phase (resolved/failed), carrying F2P/P2P and the failing test names. It also supplies the graded set, so a scored instance now emits a row AND suppresses its now-redundant artifact row — more visible, not less. Cheap by construction: one small JSON read per scored instance, no `git diff`, no process spawn, so unlike the artifact scan it needs no cap. Every row here is a real capability result: `record_verdict` already refuses gold verdicts (a control is not a score) and errored ones (an env fault is not a failure), so the board cannot be laundered by either. 66 tests green across benchmark / swe_bench / staged_workspace. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 72 +++++++++++++++++-- 1 file changed, 67 insertions(+), 5 deletions(-) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 6d3d04214..eebff0165 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3241,6 +3241,67 @@ fn scan_workspace_artifact_cards(graded: &std::collections::HashSet, now /// about, never silently dropped. const WORKSPACE_ARTIFACT_SCAN_CAP: usize = 200; +/// Cards for instances that carry a DURABLE VERDICT — the third row source, and the one that +/// makes a score visible at all. +/// +/// # Why (2026-08-18, the last link in the grade tail) +/// +/// Verdict persistence landed and the board still could not show a pass. Measured minutes +/// after: `astropy__astropy-14995` graded `resolved=true, F2P 1/1, P2P 40/40`, the verdict +/// was on disk and readable — and `benchmark/runs` reported `resolved: 1`, still counting only +/// an old sympy row. Three rows for 14995 read `failed`, which was HONEST: those are the three +/// solve RUNS that died at the reboot. A run and a verdict are different objects. The board +/// projected runs and artifacts; a verdict had nowhere to appear. +/// +/// Until this, `recorded_verdicts` only SUBTRACTED — it marked an artifact as graded so the +/// artifact row disappeared, which made a scored instance LESS visible than an unscored one. +/// A verdict must EMIT. +/// +/// This is the acceptance test from +/// [BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER](../../../docs/architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md): +/// *can a citizen standing in the room perceive the run's state through the same ViewState +/// pipe the human's screen uses?* A score answerable only by reading +/// `benchmarks/swe/verdicts/*.json` is disconnected, and it failed. +/// +/// Cheap by construction: no `git diff`, no process spawn — one small JSON read per scored +/// instance, so this source needs no cap. +fn scan_verdict_cards(now_ms: u64) -> Vec { + swe_bench::recorded_verdicts() + .into_iter() + .map(|(instance, v)| { + let last_activity_ms = std::fs::metadata(swe_bench::verdict_path(&instance)) + .and_then(|m| m.modified()) + .ok() + .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) + .map(|d| d.as_millis() as u64) + .unwrap_or(0); + BenchRunCard { + run_id: format!("verdict:{instance}"), + instance: Some(instance), + attempt: None, + max_attempts: None, + solver: None, + // The verdict IS the phase. `record_verdict` refuses gold and errored + // verdicts, so every row here is a real capability result — never a control + // and never an env fault dressed as a score. + phase: if v.resolved { "resolved" } else { "failed" }.to_string(), + stalled: false, + last_activity_ms, + age_secs: now_ms.saturating_sub(last_activity_ms) / 1000, + acts: None, + files_changed: Vec::new(), + files_examined: Vec::new(), + resolved: Some(v.resolved), + fail_to_pass: Some(format!("{}/{}", v.f2p_passed, v.f2p_total)), + pass_to_pass: Some(format!("{}/{}", v.p2p_passed, v.p2p_total)), + patch_bytes: None, + failed_tests: v.failed_tests.clone(), + infra_error: None, + } + }) + .collect() +} + pub(crate) fn scan_run_cards( run_id_filter: Option<&str>, limit: usize, @@ -3315,11 +3376,12 @@ pub(crate) fn scan_run_cards( .filter(|c| c.resolved.is_some()) .filter_map(|c| c.instance.clone()) .collect(); - graded.extend( - swe_bench::recorded_verdicts() - .into_iter() - .map(|(instance, _)| instance), - ); + // A verdict EMITS its own row, and that row is also what marks the instance graded — + // so a scored instance is MORE visible than an unscored one, not less. Before this, + // verdicts only subtracted: the artifact row vanished and no score took its place. + let verdict_cards = scan_verdict_cards(now_ms); + graded.extend(verdict_cards.iter().filter_map(|c| c.instance.clone())); + cards.extend(verdict_cards); cards.extend(scan_workspace_artifact_cards(&graded, now_ms)); } cards.sort_by(|a, b| b.last_activity_ms.cmp(&a.last_activity_ms)); From 3375fbb3d733da2b5694dea00ae111b2829f6739 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 21:59:31 -0500 Subject: [PATCH 68/80] chore(protocol): regenerate BenchRunCard binding for the `ungraded` phase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ts-rs output for the phase doc extended in 72948dc93 (staged workspaces as a first-class board source). Committed with its Rust source rather than left drifting — the exact gap #457 exists to close. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- protocol/typescript/benchmark/BenchRunCard.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/protocol/typescript/benchmark/BenchRunCard.ts b/protocol/typescript/benchmark/BenchRunCard.ts index aadf09530..dc7bb069a 100644 --- a/protocol/typescript/benchmark/BenchRunCard.ts +++ b/protocol/typescript/benchmark/BenchRunCard.ts @@ -25,7 +25,10 @@ solver?: string, * `resolved` | `failed` (loud infra marker, incl. #2180 stalls the * deadline caught) | `active` (artifact activity within the stall * window) | `quiet` (non-terminal AND silent past the window — the - * shape the projection exists to make visible). + * shape the projection exists to make visible) | `ungraded` (a staged + * workspace holds a real diff that no grade has ever seen — durable + * work awaiting a verdict, NOT a stall; see + * [`scan_workspace_artifact_cards`]). */ phase: string, /** From e0b70d13b9970110d987d0adc4cce64334252742 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 22:46:23 -0500 Subject: [PATCH 69/80] =?UTF-8?q?feat(benchmark):=20the=20recipe=20grades?= =?UTF-8?q?=20its=20own=20artifacts=20=E2=80=94=20no=20operator=20in=20the?= =?UTF-8?q?=20loop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An artifact holding a citizen's patch could sit unscored forever. Two grade paths exist and both are correct: grade-on-done fires on a card TRANSITION, the tick sweep detects a LAPSED lease. Neither can see an artifact with NO CARD — which is exactly what a detached `agent/solve` produces (#425). Measured on this box tonight: 104 staged trees, 17 holding real citizen patches that had never been scored, two of them PASSES (pytest-11143, sympy-24152) sitting ungraded for over a day. They only got verdicts because an operator ran `swe-grade` twenty times by hand. That is not a benchmark, that is a person with a shell. `cognition::swe_verdict_sweep` reconciles them, owned by `modules/benchmark_grade` — the module that owns every other grade path — and fired once as it initializes. Grading is the ACTIVITY'S OUTCOME SCORE, so it belongs to the benchmark recipe (BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md), never to an unrelated daemon's boot. DETERMINISTIC, which is the axis that makes it a reconciliation and not the forbidden condition-poll ([[the-whole-system-is-event-based-not-polling]]): - enumerates ALL staged instances, sorted — no cap, no recency sort, no sampling (the board's own scan has all three and lost 12 of 13 artifacts to them) - idempotent: an instance carrying a verdict is skipped, checked BEFORE the tree is inspected, so it cannot depend on the workspace still being dirty - refuses rather than guesses: two citizens holding worked copies of one instance would score one citizen's diff against the other's card, so it grades neither and probes the candidates by name Same disk, same outcome, every time. It inherits `record_verdict`'s three guards rather than re-implementing them — gold is a control, an errored verdict is an env fault, an empty candidate is an absence, and none of the three is ever a zero. Detached: each grade is a fresh clone plus a real test suite, minutes apiece. Init must not block and no citizen's turn may queue behind it. Sequential inside the sweep for the same reason — N parallel test suites would compete with the serving lane the round is being measured on. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/cognition/mod.rs | 1 + .../src/cognition/swe_verdict_sweep.rs | 265 ++++++++++++++++++ .../src/modules/benchmark_grade.rs | 36 +++ 3 files changed, 302 insertions(+) create mode 100644 core/continuum-core/src/cognition/swe_verdict_sweep.rs diff --git a/core/continuum-core/src/cognition/mod.rs b/core/continuum-core/src/cognition/mod.rs index fa648531d..c661eb37c 100644 --- a/core/continuum-core/src/cognition/mod.rs +++ b/core/continuum-core/src/cognition/mod.rs @@ -86,6 +86,7 @@ pub mod shared_analysis; pub mod should_respond; pub mod should_respond_module; pub mod swe_bench; +pub mod swe_verdict_sweep; pub mod threat_detector; pub mod throughput_lease; pub mod token_budget; diff --git a/core/continuum-core/src/cognition/swe_verdict_sweep.rs b/core/continuum-core/src/cognition/swe_verdict_sweep.rs new file mode 100644 index 000000000..6d1e4141c --- /dev/null +++ b/core/continuum-core/src/cognition/swe_verdict_sweep.rs @@ -0,0 +1,265 @@ +//! Every artifact that holds work gets a verdict — with no operator in the loop. +//! +//! # Why this exists (2026-08-18, and the number it recovered) +//! +//! The grade tail was built and correct, and it still needed a human to fire it. On the night +//! it landed, 104 staged trees sat on this box; 17 of them held real citizen patches that had +//! never been scored, and two of those were PASSES that had been sitting ungraded for over a +//! day. The verdicts only appeared because an operator ran `benchmark/swe-grade` twenty times +//! by hand. That is not a benchmark — that is a person with a shell. +//! +//! Joel, on being shown the recovered number: *"Needs to automatically work too"*. +//! +//! # The doctrine this obeys, and the one it must not break +//! +//! [[the-whole-system-is-event-based-not-polling]] forbids scanning the board on a clock to ask +//! "has anything become gradeable yet" — a condition-poll that duplicates an event. This is NOT +//! that. It is the same shape as [`crate::cognition::swe_bench::reap_orphaned_solve_runs`]: a +//! BOOT RECONCILIATION that enumerates durable state once and makes it consistent, then stops. +//! Boot owns the process tree, reap-or-adopt, for every service (#452, +//! [[boot-owns-the-process-tree-reap-or-adopt-never-fight-yourself]]) — and an orphaned +//! ARTIFACT is the same class of thing as an orphaned process. A patch nobody scored is a run +//! nobody reaped. +//! +//! The axis Joel named is DETERMINISM, not tick-vs-event: +//! +//! > *"if it's deterministic and not scan it or polling it's reliable"* +//! +//! So this sweep is deterministic by construction, and each property is load-bearing: +//! +//! - **Enumerates ALL staged instances, sorted.** No cap, no recency sort, no sampling. The +//! board's own artifact scan has all three and lost 12 of 13 artifacts to them — same input, +//! different answer depending on timing, which is the definition of unreliable. +//! - **Idempotent.** An instance with a recorded verdict is skipped, so re-running changes +//! nothing and a restart mid-sweep resumes rather than re-grades. +//! - **Refuses rather than guesses.** Ambiguity and absence are outcomes, not zeros. +//! +//! Given the same disk, it always produces the same set of grades. +//! +//! # What it will never do +//! +//! It never manufactures a score. The three guards that keep the durable record honest live in +//! [`crate::cognition::swe_bench::record_verdict`] and are inherited here, not re-implemented: +//! a gold patch is a control and never counts, an errored verdict is an environment fault and +//! never counts, and an empty candidate is an ABSENCE and never counts +//! ([[a-perception-fact-is-honesty-not-an-actuator]]). This module adds one more of the same +//! family: two citizens holding worked copies of one instance is ambiguity, and grading either +//! would score one citizen's diff against the other's card, so it grades neither and says so. + +use std::path::PathBuf; + +use crate::persona::staged_workspace::{grade_target, owners_of, GradeTarget}; + +/// An artifact awaiting a verdict: the instance, and the one worked copy to score. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PendingGrade { + pub instance: String, + pub workspace: PathBuf, +} + +/// What the sweep decided for one instance — every non-grade outcome is NAMED, because +/// "we skipped it" and "it scored zero" must never be the same row. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum SweepDecision { + /// Exactly one worked copy and no verdict on file. Score it. + Grade(PathBuf), + /// A verdict already exists. Skipping is what makes the sweep idempotent. + AlreadyGraded, + /// Nothing was ever written here. An absence, never a zero. + NoWork, + /// Two or more citizens hold worked copies. Refuse and name the candidates. + Ambiguous(Vec), +} + +/// The rule alone, with the filesystem taken out of it. +/// +/// Split from [`pending`] for the same reason +/// [`crate::persona::staged_workspace::grade_target`] is split from `owners_of`: the DECISION +/// is tested against a table, not against a disk fixture that re-derives it. A test that +/// rebuilds the predicate in its own body cannot fail when the real one changes. +pub fn decide(has_verdict: bool, target: GradeTarget) -> SweepDecision { + if has_verdict { + // Checked FIRST and deliberately: idempotence must not depend on the tree still + // being dirty. A graded artifact whose workspace was since cleaned would otherwise + // read as NoWork and churn a decision every boot. + return SweepDecision::AlreadyGraded; + } + match target { + GradeTarget::One(path) => SweepDecision::Grade(path), + GradeTarget::NoWork => SweepDecision::NoWork, + GradeTarget::Ambiguous(paths) => SweepDecision::Ambiguous(paths), + } +} + +/// Every instance staged into ANY citizen's workspace, sorted and deduped. +/// +/// Deterministic by construction — see the module doc. The sort is not cosmetic: it fixes +/// grading ORDER, so a sweep interrupted halfway resumes at the same place rather than at +/// whatever `read_dir` happened to yield first. +pub fn all_staged_instances() -> Vec { + let Ok(home) = crate::commands::benchmark::continuum_home() else { + return Vec::new(); + }; + let Ok(entries) = std::fs::read_dir(home.join("citizens").join("peers")) else { + return Vec::new(); + }; + let mut out: Vec = entries + .flatten() + .filter_map(|e| e.file_name().to_str().and_then(|n| uuid::Uuid::parse_str(n).ok())) + .flat_map(|peer| crate::persona::staged_workspace::staged_instances(&peer)) + .collect(); + out.sort(); + out.dedup(); + out +} + +/// Every artifact that holds work and has no verdict, in a stable order. +pub fn pending() -> Vec { + let mut out = Vec::new(); + for instance in all_staged_instances() { + let has_verdict = crate::cognition::swe_bench::read_verdict(&instance).is_some(); + // Skip the `git status` fan-out entirely when a verdict already exists — `owners_of` + // shells out once per staged copy, and on a box with 100+ trees that is the whole + // cost of the sweep. Cheap check first. + if has_verdict { + continue; + } + match decide(false, grade_target(&owners_of(&instance))) { + SweepDecision::Grade(workspace) => out.push(PendingGrade { instance, workspace }), + SweepDecision::Ambiguous(paths) => crate::probe!( + class = "benchmark.verdict.sweep_ambiguous", + instance = instance.as_str(), + candidates = paths.len(), + "two citizens hold worked copies — refusing to grade either (#419)", + ), + SweepDecision::NoWork | SweepDecision::AlreadyGraded => {} + } + } + out +} + +/// What one sweep did — reported as a probe so the run is legible from the state pipe +/// rather than from a log parse. +#[derive(Debug, Default, Clone, PartialEq, Eq)] +pub struct SweepReport { + pub graded: usize, + pub resolved: usize, + pub ungradeable: usize, + pub errored: usize, +} + +/// Grade every pending artifact, sequentially, recording each verdict. +/// +/// SEQUENTIAL on purpose. Each grade takes a fresh clone at `base_commit` and runs a real test +/// suite; N of those in parallel would compete with the citizens' own serving lane for the +/// machine the round is being measured on ([[measured-work-gets-an-exclusive-warm-slot]]). +/// The sweep is background work that must never become the reason a turn is slow. +pub async fn sweep() -> SweepReport { + let mut report = SweepReport::default(); + let work = pending(); + if work.is_empty() { + return report; + } + crate::probe!( + class = "benchmark.verdict.sweep_start", + pending = work.len(), + "boot artifact sweep — artifacts holding work with no verdict on file", + ); + for item in work { + // The ONE grader. Never an inline second reading of her work — that drift already cost + // a credential leak once (see `SOLUTION_PATH_EXCLUDES`). `grade_swe` records the + // verdict itself, so this loop only tallies. + let params = crate::commands::benchmark::SweGradeParams { + instance: item.instance.clone(), + dataset: None, + gold: None, + patch: None, + workspace: Some(item.workspace.to_string_lossy().into_owned()), + }; + match crate::commands::benchmark::grade_swe(params).await { + Ok(result) if result.error.is_some() => { + report.ungradeable += 1; + crate::probe!( + class = "benchmark.verdict.sweep_ungradeable", + instance = item.instance.as_str(), + "environment fault, NOT a capability zero — nothing recorded", + ); + } + Ok(result) => { + report.graded += 1; + if result.resolved { + report.resolved += 1; + } + } + Err(e) => { + report.errored += 1; + // Loud, and never fatal: one instance that cannot be graded must not cost the + // sweep every artifact behind it. + tracing::warn!( + instance = %item.instance, + error = %e, + "artifact sweep could not grade this instance — continuing", + ); + } + } + } + crate::probe!( + class = "benchmark.verdict.sweep_done", + graded = report.graded, + resolved = report.resolved, + ungradeable = report.ungradeable, + errored = report.errored, + "boot artifact sweep complete — every worked artifact now carries a verdict", + ); + report +} + +#[cfg(test)] +mod tests { + use super::*; + + /// what this catches: the sweep manufacturing a score out of an absence or an ambiguity — + /// the #384/#386 laundering class, which is why the grade tail existed at all. A worked + /// copy grades; nothing else does, and each refusal keeps its own name. + #[test] + fn only_a_single_worked_copy_is_ever_graded_and_every_refusal_keeps_its_name() { + let a = PathBuf::from("/peers/a/workspace/swe/sympy__sympy-24152"); + let b = PathBuf::from("/peers/b/workspace/swe/sympy__sympy-24152"); + + assert_eq!( + decide(false, GradeTarget::One(a.clone())), + SweepDecision::Grade(a.clone()), + "exactly one worked copy is the only thing that grades" + ); + assert_eq!( + decide(false, GradeTarget::NoWork), + SweepDecision::NoWork, + "an unworked tree is an ABSENCE — it must never become a zero" + ); + assert_eq!( + decide(false, GradeTarget::Ambiguous(vec![a.clone(), b.clone()])), + SweepDecision::Ambiguous(vec![a.clone(), b]), + "two worked copies must refuse, not pick — grading either scores the wrong citizen" + ); + } + + /// what this catches: a sweep that re-grades on every boot, which would burn hours of test + /// runs and rewrite verdicts that were already true. Idempotence is the property that lets + /// this run unattended at all. + #[test] + fn a_recorded_verdict_short_circuits_every_target_state() { + let a = PathBuf::from("/peers/a/workspace/swe/x"); + for target in [ + GradeTarget::One(a.clone()), + GradeTarget::NoWork, + GradeTarget::Ambiguous(vec![a.clone(), a]), + ] { + assert_eq!( + decide(true, target), + SweepDecision::AlreadyGraded, + "a graded artifact is skipped REGARDLESS of what its tree looks like now — \ + idempotence must not depend on the workspace still being dirty" + ); + } + } +} diff --git a/core/continuum-core/src/modules/benchmark_grade.rs b/core/continuum-core/src/modules/benchmark_grade.rs index 42a0e8281..2eeda1de5 100644 --- a/core/continuum-core/src/modules/benchmark_grade.rs +++ b/core/continuum-core/src/modules/benchmark_grade.rs @@ -87,6 +87,42 @@ impl ServiceModule for BenchmarkGradeModule { } async fn initialize(&self, ctx: &ModuleContext) -> Result<(), String> { + // RECONCILE the artifacts already on disk, once, as this module comes up. + // + // Grading belongs to the benchmark recipe — it IS the activity's outcome score + // (docs/architecture/BENCHMARKS-ARE-ADAPTERS-NOT-A-RUNNER.md), so it is owned HERE, + // by the module that owns every other grade path, and never by some unrelated daemon's + // boot sequence. (Written after doing exactly that and being corrected: a sweep hung + // off `serving_daemon` start is the parallel-runner shape this repo has a whole + // document forbidding. Joel: "Grading is supposed to be part of the regular benchmark + // recipe".) + // + // Why a reconciliation exists at all, next to two event paths that are both correct: + // the grade-on-done subscriber fires on a card TRANSITION, and the tick sweep detects + // a LAPSED lease. Neither can see an artifact with no card — detached `agent/solve` + // runs (#425) produce exactly that, and 17 unscored citizen patches were sitting on + // this box the night it was written, two of them PASSES over a day old. This is the + // same reap-or-adopt boot owns for every other resource (#452): an orphaned ARTIFACT + // is an orphaned run. + // + // Deterministic and idempotent (see the sweep's module doc), so it is a reconciliation + // and not the forbidden condition-poll: it enumerates ALL staged instances in sorted + // order with no cap and no recency sort, skips anything already carrying a verdict, and + // refuses ambiguity rather than guessing. Same disk, same outcome, every time. + // + // DETACHED because each grade is a fresh clone plus a real test suite — minutes apiece. + // Module init must not block on it, and the citizens' first turn must not queue behind + // it. + tokio::spawn(async { + let report = crate::cognition::swe_verdict_sweep::sweep().await; + if report.graded > 0 { + tracing::info!( + graded = report.graded, + resolved = report.resolved, + "benchmark artifact reconciliation scored citizen work that had no verdict" + ); + } + }); // THE live wiring. `config().event_subscriptions` installs a SYNCHRONOUS-tier // subscription the registry marks `synchronous: false` — which `publish()` // filters OUT, and runtime.rs:99 says so out loud: "event_subscriptions are From a85acee003acc5d93e52dfc7abae37f9ff17909c Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 23:09:16 -0500 Subject: [PATCH 70/80] =?UTF-8?q?feat(benchmark):=20a=20round=20can=20be?= =?UTF-8?q?=20ASKED=20its=20stage=20=E2=80=94=20#371's=20acceptance=20test?= =?UTF-8?q?,=20answered?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The round entity has tracked stage, card set, and driver since it was written. Nothing could ask it. Transitions fired probes and the state sat in a private static, so "has the round started / how far along / is it done" was answerable only by probe archaeology — which is the exact failure ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md names in its own acceptance test: Can a fresh driver — with no memory of this session — answer "is it ready, has it started, is it stuck, is it done" using only queries, with zero log reads, zero probe archaeology, and zero inference from an absence? A probe is a transition RECORD: it tells you what happened when someone was watching. It cannot answer a question asked later, and later is when every question is actually asked. `bench_round::live_rounds()` folds the rounds in flight; `benchmark/rounds` serves it. Each row: stage, dispatched/settled/remaining, and the work DRIVER (citizen — works in the room and feeds the curriculum — vs detached_solve). Three properties are load-bearing, not decoration: - `settled` is REPORTED, never left to `dispatched - remaining`. Making the reader compute it is how an absence becomes a guess (law 3: an absence is never a state). - EMPTY is a real answer — "no round is running" — never "the question could not be reached". A round is dropped the instant its last card settles; that END is the `bench.round.done` probe. - Sorted by round id, so two calls a second apart cannot reorder rows under a reader. A projection whose order depends on HashMap iteration teaches consumers to distrust it. This is build-order step 4 of the design doc, and it does NOT touch the entity's state machine — the two-stage lifecycle, the settle truth table, and "done fires exactly once" are unchanged and still covered by their own tests. Still open from the doc, deliberately not faked here: the run PULSE (law 2 — `quiet` is still derived from a ledger written once per attempt, so a healthy long run can still read stalled) and the STAGING→READY gate (#442). Neither is a timeout away; both need the component that knows to say so. 10 bench_round tests green, incl. a new one pinning the projection against the entity. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_round.rs | 133 ++++++++++++++++++ core/continuum-core/src/commands/benchmark.rs | 63 +++++++++ .../typescript/benchmark/RoundSnapshot.ts | 55 ++++++++ 3 files changed, 251 insertions(+) create mode 100644 protocol/typescript/benchmark/RoundSnapshot.ts diff --git a/core/continuum-core/src/cognition/bench_round.rs b/core/continuum-core/src/cognition/bench_round.rs index 2a8b014d8..b44ddd618 100644 --- a/core/continuum-core/src/cognition/bench_round.rs +++ b/core/continuum-core/src/cognition/bench_round.rs @@ -318,6 +318,93 @@ pub fn observe_card_event(payload: &Value) { } } +/// One in-flight round, as anyone may ask about it. +/// +/// # Why this type exists (2026-08-18, and it is the acceptance test) +/// +/// The round entity below has tracked stage, card set, and driver since it was written — +/// and NOTHING could ask it. Transitions fired probes and the state lived in a private +/// static, so "has the round started / is it stuck / is it done" was answerable only by +/// probe archaeology. That is precisely the failure +/// [ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md](../../../../docs/architecture/ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md) +/// names in its own acceptance test: +/// +/// > *Can a fresh driver — with no memory of this session — answer "is it ready, has it +/// > started, is it stuck, is it done" using only queries, with zero log reads, zero probe +/// > archaeology, and zero inference from an absence?* +/// +/// A probe is a transition RECORD. It tells you what happened when someone was watching. +/// It cannot answer a question asked later, which is when every question is actually asked. +/// +/// **`settled` is reported explicitly rather than left to subtraction.** `dispatched - +/// remaining` is the same number, and making the reader compute it is how an absence +/// becomes a guess ([[an-absence-is-an-unfinished-measurement]]). Law 3 of the design doc: +/// *an absence is never a state.* +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + schemars::JsonSchema, + ts_rs::TS, +)] +#[ts( + export, + export_to = "../../../protocol/typescript/benchmark/RoundSnapshot.ts" +)] +pub struct RoundSnapshot { + /// The round id — which IS its run room's id. A round is its room's activity; there is + /// never a second identifier ([[killing-a-derived-id-needs-a-directory-at-every-scope-boundary]]). + pub round_id: String, + pub benchmark: String, + /// `working` | `done`. Present here means IN FLIGHT — a completed round is removed the + /// instant it finishes, so `done` is only ever observed in the transition probe. + pub stage: String, + /// Cards this round dispatched. + pub dispatched: usize, + /// Cards that have reached a terminal state. + pub settled: usize, + /// Cards still working. Zero here with the round still listed would be a defect — + /// the settle that empties the set is what removes it. + pub remaining: usize, + /// Who works these cards: `citizen` (in the room, produces turns, feeds the curriculum) + /// or `detached_solve`. Decided at dispatch, read at claim time. + pub driver: String, +} + +/// Every round this core is tracking, in a stable order. +/// +/// Sorted by round id so two calls a second apart cannot reorder rows under a reader — +/// a projection whose row order depends on `HashMap` iteration teaches its consumers to +/// distrust it. +pub fn live_rounds() -> Vec { + let rounds = ROUNDS.lock().unwrap_or_else(|e| e.into_inner()); + let mut out: Vec = rounds + .values() + .map(|r| RoundSnapshot { + round_id: r.round_id.to_string(), + benchmark: r.benchmark.clone(), + stage: match r.stage { + RoundStage::Working => "working", + RoundStage::Done => "done", + } + .to_string(), + dispatched: r.dispatched(), + settled: r.dispatched().saturating_sub(r.remaining()), + remaining: r.remaining(), + driver: match r.driver { + WorkDriver::Citizen => "citizen", + WorkDriver::DetachedSolve => "detached_solve", + } + .to_string(), + }) + .collect(); + out.sort_by(|a, b| a.round_id.cmp(&b.round_id)); + out +} + #[cfg(test)] mod tests { use super::*; @@ -398,6 +485,52 @@ mod tests { assert_eq!(r.stage(), RoundStage::Done); } + // what this catches: the projection disagreeing with the entity it projects — a + // "settled" count that drifts from the round's own card map is worse than no query at + // all, because a driver would BELIEVE it. Also pins that progress is legible mid-round: + // the whole point of #371 is answering "how far along" without reading a log. + #[test] + fn a_round_in_flight_reports_its_own_progress_honestly() { + let round_id = Uuid::new_v4(); + let ids = cards(3); + open_round(round_id, "swe-bench-lite", WorkDriver::Citizen); + for id in &ids { + add_card(round_id, *id); + } + + let before = live_rounds(); + let row = before + .iter() + .find(|r| r.round_id == round_id.to_string()) + .expect("a dispatched round must be QUERYABLE — that is the whole fix"); + assert_eq!(row.stage, "working"); + assert_eq!((row.dispatched, row.settled, row.remaining), (3, 0, 3)); + assert_eq!( + row.driver, "citizen", + "the driver decides whether this round teaches anybody anything — it must be readable" + ); + + // Settle one, and the projection must move WITH the entity, not lag it. + ROUNDS + .lock() + .unwrap() + .get_mut(&round_id) + .expect("still in flight") + .settle_card(ids[0], "closed"); + let mid = live_rounds(); + let row = mid + .iter() + .find(|r| r.round_id == round_id.to_string()) + .expect("two of three cards are still working, so the round is still in flight"); + assert_eq!( + (row.dispatched, row.settled, row.remaining), + (3, 1, 2), + "settled is reported, never left to subtraction — an absence must not become a guess" + ); + + ROUNDS.lock().unwrap().remove(&round_id); + } + // what this catches: a duplicate terminal event double-counting a card. The bridge // dedupes by event id, but a card can legitimately transition closed→merged; the // second settle must not decrement `remaining` a second time (a round declared Done diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index eebff0165..359ddc7af 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3407,4 +3407,67 @@ pub(crate) fn scan_run_cards( Ok(cards) } +// --------------------------------------------------------------------------- +// benchmark/rounds — the ROUND lifecycle, askable (#371) +// --------------------------------------------------------------------------- + +/// No parameters. Rounds in flight are few (usually one) and each is a handful of +/// fields, so paging and filtering would be ceremony over a list you always want whole. +#[derive(Debug, Clone, Default, Serialize, Deserialize, TS, JsonSchema)] +#[ts( + export, + export_to = "../../../protocol/typescript/benchmark/BenchmarkRoundsParams.ts" +)] +pub struct BenchmarkRoundsParams {} + +#[derive(Debug, Clone, Serialize, Deserialize, TS, JsonSchema)] +#[ts( + export, + export_to = "../../../protocol/typescript/benchmark/BenchmarkRoundsResult.ts" +)] +pub struct BenchmarkRoundsResult { + /// Rounds currently in flight. EMPTY is a real, unambiguous answer — "no round is + /// running" — and never "the question could not be reached". A round is removed the + /// instant its last card settles, so a round that finished is absent by design and + /// its END is on the `bench.round.done` probe. + pub rounds: Vec, + /// How many are in flight, so a reader that only needs the yes/no does not have to + /// interpret an array's length. + pub in_flight: usize, +} + +#[derive(Default)] +pub struct BenchmarkRounds; + +#[async_trait] +impl ActionCommand for BenchmarkRounds { + const NAME: &'static str = "benchmark/rounds"; + const ACCESS: AccessLevel = AccessLevel::AiSafe; + const DESCRIPTION: &'static str = + "Every benchmark ROUND in flight, with its stage — the lifecycle question answered \ + by a QUERY instead of by probe archaeology (#371). A round is the card set one \ + `benchmark/dispatch` posted; its id IS its run room's id. Each row carries stage \ + (working|done), dispatched/settled/remaining, and the work DRIVER (citizen — works \ + in the room and feeds the curriculum — vs detached_solve). An EMPTY list is a real \ + answer meaning no round is running, never a failure to reach the question; a round \ + is dropped the moment its last card settles, and that END is the `bench.round.done` \ + probe. This is what a fresh driver reads to answer 'has it started, is it stuck, is \ + it done' with zero log reads."; + type Params = BenchmarkRoundsParams; + type Output = BenchmarkRoundsResult; + + async fn run( + &self, + _ctx: &Ctx, + _p: BenchmarkRoundsParams, + ) -> Result { + let rounds = crate::cognition::bench_round::live_rounds(); + Ok(BenchmarkRoundsResult { + in_flight: rounds.len(), + rounds, + }) + } +} + crate::register_stateless_command!(BenchmarkRuns); +crate::register_stateless_command!(BenchmarkRounds); diff --git a/protocol/typescript/benchmark/RoundSnapshot.ts b/protocol/typescript/benchmark/RoundSnapshot.ts new file mode 100644 index 000000000..7df90cc37 --- /dev/null +++ b/protocol/typescript/benchmark/RoundSnapshot.ts @@ -0,0 +1,55 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * One in-flight round, as anyone may ask about it. + * + * # Why this type exists (2026-08-18, and it is the acceptance test) + * + * The round entity below has tracked stage, card set, and driver since it was written — + * and NOTHING could ask it. Transitions fired probes and the state lived in a private + * static, so "has the round started / is it stuck / is it done" was answerable only by + * probe archaeology. That is precisely the failure + * [ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md](../../../../docs/architecture/ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md) + * names in its own acceptance test: + * + * > *Can a fresh driver — with no memory of this session — answer "is it ready, has it + * > started, is it stuck, is it done" using only queries, with zero log reads, zero probe + * > archaeology, and zero inference from an absence?* + * + * A probe is a transition RECORD. It tells you what happened when someone was watching. + * It cannot answer a question asked later, which is when every question is actually asked. + * + * **`settled` is reported explicitly rather than left to subtraction.** `dispatched - + * remaining` is the same number, and making the reader compute it is how an absence + * becomes a guess ([[an-absence-is-an-unfinished-measurement]]). Law 3 of the design doc: + * *an absence is never a state.* + */ +export type RoundSnapshot = { +/** + * The round id — which IS its run room's id. A round is its room's activity; there is + * never a second identifier ([[killing-a-derived-id-needs-a-directory-at-every-scope-boundary]]). + */ +round_id: string, benchmark: string, +/** + * `working` | `done`. Present here means IN FLIGHT — a completed round is removed the + * instant it finishes, so `done` is only ever observed in the transition probe. + */ +stage: string, +/** + * Cards this round dispatched. + */ +dispatched: number, +/** + * Cards that have reached a terminal state. + */ +settled: number, +/** + * Cards still working. Zero here with the round still listed would be a defect — + * the settle that empties the set is what removes it. + */ +remaining: number, +/** + * Who works these cards: `citizen` (in the room, produces turns, feeds the curriculum) + * or `detached_solve`. Decided at dispatch, read at claim time. + */ +driver: string, }; From dd441a664de3c79dfa98f757a07b8d5561ce0eda Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 23:35:23 -0500 Subject: [PATCH 71/80] =?UTF-8?q?feat(benchmark):=20STAGING=20=E2=86=92=20?= =?UTF-8?q?READY=20is=20a=20gate,=20not=20a=20suggestion=20(#442)=20?= =?UTF-8?q?=E2=80=94=20#371=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dispatch has called `await_ready_serving` for a long time and it was always correct. Its ANSWER was advisory: on a dead lane it set the auto-fire cap to zero and STAGED THE CARDS ANYWAY — a full round posted to a board with nothing on the box able to decode a token. The round then looked dispatched and was inert. That is #455 verbatim: "hosting is correctly blocked while the lane thrashes, and we stage work into the gap." So this is not a new probe. It is the same probe, made LOAD-BEARING: not-ready is a state the round stops at, never a parameter that silently degrades it into an empty round — ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md §4, "expressed as a state instead of a check". `cognition::round_readiness::decide` is a pure truth table over the awaited verdict: - Ready { lanes } — serving PROVED it can decode; lanes.max(1) so a plan caught mid-recompute can never hand dispatch a fan-out width of zero - Blocked(NothingServing | NotDecodeVerified { model }) — refuse, and NAME what is missing plus what would change the answer. A gate that blocks without saying why just moves the archaeology from the run to the gate. It NEVER reads `ServingSnapshot::ready`. That bool is a cached claim with no expiry — its own doc records 2026-08-05, when `serving/status` answered `ready: true` after the llama-server had been SIGKILLed. Only `await_ready_serving`'s verdict opens the gate, because that one holds a lane (local OR pinned external) to a real multi-token decode. The unblocking snapshot is used ONLY to name the model in the refusal. There is a test that fails if that ever inverts. Two more corrections while here: - the wait budget was a flat 30s; it is now `DEFAULT_SERVING_WAIT` (= READY_TIMEOUT + margin, the spawner's own load budget), so the gate cannot declare failure before a legitimate cold load has had its window. A 27B off cold disk needs more than 30s. - `--force` stages anyway and SAYS SO, same contract as `start --force` (#420). A gate with no escape gets worked around; a silent escape is worse than no gate. #371 IS NOW COMPLETE, and one item closed by reading rather than building: 1. pulse the run while it runs — ALREADY BUILT. `solve.rs` runs a `select!` over the drive future and a 60s interval, writing live `acts` and refreshing the mtime that `last_activity_ms` folds from. I twice reported this as outstanding tonight; I had not read the function. It cites the same 2026-08-16 measurement I was quoting at it. 2. round entity owns stages — built 3. transitions from real emitters — built 4. RoundViewState on the pipe — `benchmark/rounds` (a85acee00) 5. dispatch consumes it — THIS 64 benchmark tests + 3 new readiness tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/cognition/mod.rs | 1 + .../src/cognition/round_readiness.rs | 170 ++++++++++++++++++ core/continuum-core/src/commands/benchmark.rs | 64 ++++++- .../benchmark/BenchmarkDispatchParams.ts | 12 +- .../benchmark/BenchmarkRoundsParams.ts | 7 + .../benchmark/BenchmarkRoundsResult.ts | 16 ++ 6 files changed, 263 insertions(+), 7 deletions(-) create mode 100644 core/continuum-core/src/cognition/round_readiness.rs create mode 100644 protocol/typescript/benchmark/BenchmarkRoundsParams.ts create mode 100644 protocol/typescript/benchmark/BenchmarkRoundsResult.ts diff --git a/core/continuum-core/src/cognition/mod.rs b/core/continuum-core/src/cognition/mod.rs index c661eb37c..7f68ad623 100644 --- a/core/continuum-core/src/cognition/mod.rs +++ b/core/continuum-core/src/cognition/mod.rs @@ -31,6 +31,7 @@ pub mod act_observe; pub mod adaptive_throughput; pub mod audit; pub mod bench_round; +pub mod round_readiness; pub mod benchmark; pub mod benchmark_humaneval; pub mod channel_digest; diff --git a/core/continuum-core/src/cognition/round_readiness.rs b/core/continuum-core/src/cognition/round_readiness.rs new file mode 100644 index 000000000..1b881d921 --- /dev/null +++ b/core/continuum-core/src/cognition/round_readiness.rs @@ -0,0 +1,170 @@ +//! The `STAGING → READY` gate (#442): a round is never staged into a lane that cannot work it. +//! +//! # The defect, and why it is not "a missing check" +//! +//! `benchmark/dispatch` has called [`await_ready_serving`] for a long time — and used the +//! answer only to SIZE the auto-fire cap. When the lane was dead it set the cap to zero and +//! **staged the cards anyway**: a full round of work posted to a board, kickoffs sent, and +//! nothing on the box able to decode a single token. The round then sat there looking +//! dispatched. That is #455 stated exactly — *"hosting is correctly blocked while the lane +//! thrashes, and we stage work into the gap"*. +//! +//! So the fix is not to add a probe. The probe was there and correct. The fix is to make its +//! answer LOAD-BEARING: not-ready is a STATE the round stops at, not a parameter that quietly +//! degrades the round into an empty one +//! ([ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md](../../../../docs/architecture/ROUND-LIFECYCLE-AS-RECIPE-OWNED-STATE-MACHINE.md) §4: +//! *"dispatch refuses to stage into a not-ready room. That is the `STAGING → READY` gate, +//! expressed as a state instead of a check."*) +//! +//! # Refuse, don't guess — and never fake readiness +//! +//! [`ServingSnapshot::ready`] is a CACHED CLAIM with no expiry; its own doc records the +//! 2026-08-05 incident where `serving/status` answered `ready: true` after the llama-server +//! had been SIGKILLed. This module therefore never reads that bare bool. It consumes +//! [`await_ready_serving`], which holds a lane — local OR a pinned external endpoint — to a +//! real multi-token decode, so a compute-wedged lane is rejected rather than believed +//! ([[fallbacks-are-illegal-fail-loud]]). +//! +//! The refusal NAMES what is missing, because a gate that blocks without saying why just moves +//! the archaeology from the run to the gate. +//! +//! # The `force` escape, and why it exists +//! +//! Same shape as `start --force` (#420): refuse by default, allow an explicit operator +//! override, and make the override ANNOUNCE that it skipped a gate. A gate with no override +//! gets worked around by whoever needs to ship; a silent override is worse than none. + +use crate::inference::llama_server::ServingSnapshot; + +/// Why a round may not be staged. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum NotReady { + /// Nothing is serving and nothing is coming up — no active model at all. + NothingServing, + /// A model is named but has not proven it can decode within the wait budget. This is the + /// cold-load case AND the wedged-lane case; from the round's side they are the same fact, + /// because both mean "work staged now cannot be worked now". + NotDecodeVerified { model: String }, +} + +impl NotReady { + /// The operator-facing sentence. States the fact, names the model when there is one, and + /// says the one thing that changes the answer — never a bare "not ready". + pub fn explain(&self) -> String { + match self { + NotReady::NothingServing => "no model is serving on this node — a round staged now \ + would post cards nobody can work. Bring a lane up (`continuum serving/status` \ + to see the plan), then dispatch again." + .to_string(), + NotReady::NotDecodeVerified { model } => format!( + "`{model}` is named but has not proven it can decode within the wait budget — \ + it is still loading, or the lane is wedged. Either way a round staged now \ + would post cards nobody can work. Re-run when `serving/status` reports it \ + decode-verified." + ), + } + } +} + +/// The gate's answer. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum RoundReadiness { + /// Serving proved it can decode. `lanes` is what the round may fan out across. + Ready { lanes: u32 }, + /// Refuse — with the reason, so the caller never has to guess. + Blocked(NotReady), +} + +/// The rule alone, with the clock and the network taken out of it. +/// +/// `awaited` is [`await_ready_serving`]'s answer (`None` = it never proved decode within the +/// budget). `current` is the unblocking snapshot, used ONLY to name the model in the refusal — +/// never to override the verdict, because that bool is the cached claim this gate exists to +/// distrust. +pub fn decide(awaited: Option<&ServingSnapshot>, current: Option<&ServingSnapshot>) -> RoundReadiness { + match awaited { + // `lanes` can legitimately be 0 on a snapshot that is otherwise ready (a plan mid + // recompute); a round needs at least one, and claiming zero lanes are usable would + // hand the caller a division by nothing. + Some(s) => RoundReadiness::Ready { lanes: s.lanes.max(1) }, + None => blocked_reason(current), + } +} + +/// Name the refusal from whatever the unblocking snapshot knows. Split out so the two +/// refusal shapes read as one decision instead of a nested match inside [`decide`]. +fn blocked_reason(current: Option<&ServingSnapshot>) -> RoundReadiness { + match current.and_then(|s| s.active_model.clone()) { + Some(model) if !model.is_empty() => { + RoundReadiness::Blocked(NotReady::NotDecodeVerified { model }) + } + _ => RoundReadiness::Blocked(NotReady::NothingServing), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn snap(model: Option<&str>, ready: bool, lanes: u32) -> ServingSnapshot { + ServingSnapshot { + active_model: model.map(|m| m.to_string()), + ready, + lanes, + ..ServingSnapshot::empty() + } + } + + /// what this catches: the gate believing `ready: true` off a snapshot that never proved + /// decode — the exact 2026-08-05 shape where `serving/status` said ready with the process + /// SIGKILLed. Only `await_ready_serving`'s verdict may open the gate; `current` is for + /// NAMING the refusal, never for granting one. + #[test] + fn a_cached_ready_claim_can_never_open_the_gate() { + let lying = snap(Some("qwen3.8-27b"), true, 3); + assert_eq!( + decide(None, Some(&lying)), + RoundReadiness::Blocked(NotReady::NotDecodeVerified { + model: "qwen3.8-27b".to_string() + }), + "a ready:true snapshot that failed the decode bar must still BLOCK — the bool is a \ + cached claim with no expiry, which is why this gate reads the awaited verdict" + ); + } + + /// what this catches: a refusal that says "not ready" and nothing else, which just relocates + /// the archaeology. Each reason must name the model when one exists, and say what changes it. + #[test] + fn every_refusal_names_what_is_missing() { + assert_eq!( + decide(None, None), + RoundReadiness::Blocked(NotReady::NothingServing) + ); + assert!(NotReady::NothingServing.explain().contains("no model is serving")); + + let loading = snap(Some("devstral-24b"), false, 0); + let RoundReadiness::Blocked(reason) = decide(None, Some(&loading)) else { + panic!("a lane that never decode-verified must block"); + }; + let msg = reason.explain(); + assert!(msg.contains("devstral-24b"), "the refusal must NAME the model: {msg}"); + assert!( + msg.contains("loading") || msg.contains("wedged"), + "and say what would change the answer: {msg}" + ); + } + + /// what this catches: a ready lane reporting zero usable lanes — a plan caught mid-recompute + /// would otherwise hand dispatch a fan-out width of 0 and stage a round that fires nothing. + #[test] + fn a_ready_lane_always_offers_at_least_one_lane() { + assert_eq!( + decide(Some(&snap(Some("m"), true, 0)), None), + RoundReadiness::Ready { lanes: 1 } + ); + assert_eq!( + decide(Some(&snap(Some("m"), true, 4)), None), + RoundReadiness::Ready { lanes: 4 } + ); + } +} diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 359ddc7af..708d7ae81 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -624,6 +624,16 @@ pub struct BenchmarkDispatchParams { #[serde(default, skip_serializing_if = "Option::is_none")] #[ts(optional)] pub drive: Option, + /// Stage the round even though serving is NOT decode-verified (#442). + /// + /// Off by default, and the default is the point: dispatch refuses to post cards no + /// citizen can work, because a round staged into a dead lane looks dispatched and is + /// inert (#455). This is the explicit operator override — same contract as + /// `start --force` (#420) — and it announces itself in the log rather than passing + /// silently, since a gate that can be skipped without a trace is not a gate. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub force: Option, } #[derive(Debug, Clone, Serialize, TS)] @@ -1353,13 +1363,55 @@ impl ActionCommand for BenchmarkDispatch { // launch a solve into a corpse, self-healing on the next dispatch). NOTE: this caps THIS // dispatch call; the global in-flight-solve admission gate shared with work/claim // (#385/#386) is the broader fix. - let solve_cap: u32 = match crate::inference::llama_server::await_ready_serving( - std::time::Duration::from_secs(30), + // + // #442, and the correction that makes it a GATE: the probe below was already here and + // already correct — its answer was simply advisory. On a dead lane this set the cap to + // zero and STAGED THE CARDS ANYWAY, posting a full round of work to a board with + // nothing on the box able to decode a token (#455: "we stage work into the gap"). A + // not-ready lane is now a STATE the round stops at, not a parameter that silently + // degrades it into an empty round. + // + // The wait budget is DERIVED, never invented: `DEFAULT_SERVING_WAIT` is + // `READY_TIMEOUT + margin` — the spawner's own load budget — so this gate can never + // declare failure before a legitimate cold load has had its full window. The flat 30s + // that used to be here was exactly that bug in miniature. + let awaited = crate::inference::llama_server::await_ready_serving( + crate::inference::llama_server::DEFAULT_SERVING_WAIT, ) - .await - { - Some(s) => s.lanes.max(1), - None => 0, + .await; + let solve_cap: u32 = { + use crate::cognition::round_readiness::{decide, RoundReadiness}; + let current = crate::inference::llama_server::current_serving(); + match decide(awaited.as_ref(), Some(¤t)) { + RoundReadiness::Ready { lanes } => lanes, + RoundReadiness::Blocked(reason) => { + let why = reason.explain(); + crate::probe!( + class = "bench.round.staging_blocked", + benchmark = spec.name, + forced = p.force.unwrap_or(false), + reason = why.as_str(), + "STAGING → READY refused: serving cannot work this round (#442)", + ); + // Refuse, with the override announcing itself — same contract as + // `start --force` (#420). A gate with no escape gets worked around; + // a silent escape is worse than no gate. + if !p.force.unwrap_or(false) { + return Err(CommandError::Invalid(format!( + "benchmark/dispatch refused to stage `{}`: {why}\n\ + (pass --force to stage anyway — it will post cards that cannot be \ + worked until a lane comes up)", + spec.name + ))); + } + tracing::warn!( + benchmark = %spec.name, + reason = %why, + "--force: staging a round into a lane that is NOT decode-verified" + ); + 0 + } + } }; // OPEN the round BEFORE the first card exists. Kickoffs go out inside the loop, so // a citizen can claim card 1 while card 2 is still being posted — and `work/claim` diff --git a/protocol/typescript/benchmark/BenchmarkDispatchParams.ts b/protocol/typescript/benchmark/BenchmarkDispatchParams.ts index 966160ab8..f075fd311 100644 --- a/protocol/typescript/benchmark/BenchmarkDispatchParams.ts +++ b/protocol/typescript/benchmark/BenchmarkDispatchParams.ts @@ -81,4 +81,14 @@ prune: boolean | null, * deliver the second one — but it depends on the kickoff→claim hop that used to * stall rounds, so it is opt-in until that hop is proven under residency. */ -drive?: WorkDriver, }; +drive?: WorkDriver, +/** + * Stage the round even though serving is NOT decode-verified (#442). + * + * Off by default, and the default is the point: dispatch refuses to post cards no + * citizen can work, because a round staged into a dead lane looks dispatched and is + * inert (#455). This is the explicit operator override — same contract as + * `start --force` (#420) — and it announces itself in the log rather than passing + * silently, since a gate that can be skipped without a trace is not a gate. + */ +force?: boolean, }; diff --git a/protocol/typescript/benchmark/BenchmarkRoundsParams.ts b/protocol/typescript/benchmark/BenchmarkRoundsParams.ts new file mode 100644 index 000000000..726e7a7f1 --- /dev/null +++ b/protocol/typescript/benchmark/BenchmarkRoundsParams.ts @@ -0,0 +1,7 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * No parameters. Rounds in flight are few (usually one) and each is a handful of + * fields, so paging and filtering would be ceremony over a list you always want whole. + */ +export type BenchmarkRoundsParams = Record; diff --git a/protocol/typescript/benchmark/BenchmarkRoundsResult.ts b/protocol/typescript/benchmark/BenchmarkRoundsResult.ts new file mode 100644 index 000000000..9566c6483 --- /dev/null +++ b/protocol/typescript/benchmark/BenchmarkRoundsResult.ts @@ -0,0 +1,16 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { RoundSnapshot } from "./RoundSnapshot"; + +export type BenchmarkRoundsResult = { +/** + * Rounds currently in flight. EMPTY is a real, unambiguous answer — "no round is + * running" — and never "the question could not be reached". A round is removed the + * instant its last card settles, so a round that finished is absent by design and + * its END is on the `bench.round.done` probe. + */ +rounds: Array, +/** + * How many are in flight, so a reader that only needs the yes/no does not have to + * interpret an array's length. + */ +in_flight: number, }; From 29b244fb84c53d1c6d80ae345cbf97bbaa179a9a Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Tue, 18 Aug 2026 23:52:21 -0500 Subject: [PATCH 72/80] =?UTF-8?q?feat(serving):=20Qwen3.8-27B=20SERVES=20o?= =?UTF-8?q?n=20the=20M5=20=E2=80=94=2017.2=20tok/s=20measured,=20catalog?= =?UTF-8?q?=20corrected=20(#440)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 27B was never blocked on code, a path, or a missing flag. It was losing a budget it should never have been measured against. `serving.plan` was choosing Devstral-Small with `usable_gb: 18` on a 64 GB box — because `usable_bytes` is FREE bytes, and Devstral + the 7B vision lane + embeddings were already resident. An 18 GB Q4_K_M cannot fit a lane inside 18 GB, so `fits_on_gpu` said no and the incumbent kept winning. Compounding it, `serving_plan.rs:670` stabilizes by keeping the incumbent "switching DOWN only" — the autonomic planner has no path to UPGRADE to a more capable model, even one that fits after eviction. That is #214's "grow-back is dead" shape living in model SELECTION rather than window sizing. The eviction credit-back that models this already exists — it is wired to the PIN path only. `serving/pin` therefore just works: budget_gb 34.09 (vs the 18 the autonomic planner saw) weights_gb 18.97 → fits, 15 GB spare previous Devstral-Small-2507 → pinned Qwen3.8-27B Swapped live in under 30s; `served_context_window: 19712`. DECODE-VERIFIED, not `ready:true`-verified — that bool is the cached claim #442 refuses to trust. A real generation through the lane: prompt 67 tok (cache_n 42 — KV warm), prefill 56.8 tok/s 200 predicted tokens in 11,605 ms = 17.2 tok/s So the catalog's conservative 10.0 was 42% low. The row's own comment says it is "corrected by live measurement, never by wish" — this is that correction, with the conditions it was taken under recorded beside it so the next reader knows what the number means. STILL OPEN, and now precisely stated rather than mysterious: the autonomic planner cannot reach this model on its own. Until the stabilizer can consider an upgrade against the post-eviction budget, serving the frontier model requires an explicit pin. That is a real card, not a footnote — the pin is a workaround for a planner that only knows how to shrink. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/model_registry/catalog.rs | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/core/continuum-core/src/model_registry/catalog.rs b/core/continuum-core/src/model_registry/catalog.rs index 13e4b67da..b4fc5ea9f 100644 --- a/core/continuum-core/src/model_registry/catalog.rs +++ b/core/continuum-core/src/model_registry/catalog.rs @@ -538,10 +538,12 @@ pub fn models() -> Vec { arch: Arch::Qwen35, context_window: 262_144, max_output_tokens: 16_384, - // Conservative M5/Metal estimate for a dense 27B (Devstral 24B row carries - // 10.0); the 4090 numbers above don't transfer across backends. Corrected - // by live measurement, never by wish. - tokens_per_second: 10.0, + // MEASURED on this M5 (2026-08-19, build dd441a664): 200 predicted tokens in + // 11,605 ms = 17.2 tok/s generation, 56.8 tok/s prefill, on a pinned lane at a + // 19,712 served window with the KV cache warm (cache_n 42 of a 67-token prompt). + // Was a conservative 10.0 estimate; the row's own instruction is "corrected by + // live measurement, never by wish", so this is the measurement. + tokens_per_second: 17.2, capabilities: &[ Capability::TextGeneration, Capability::Chat, From 4c6157d13478adcf9d073cface12d79c008e21b8 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 00:14:16 -0500 Subject: [PATCH 73/80] fix(serving): the planner reasons about the budget the incumbent is occupying (#214/#440/#266) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `plan_serving_stable`'s switch-up gate asked "does the better model fit in the memory left over BESIDE the one already loaded?" That is the wrong question. A swap is never co-resident: `serve()` kills the incumbent llama-server child and THEN launches the candidate — `pin_fit_decision` documents exactly this ("the candidate only needs to fit AFTER the incumbent's VRAM is reclaimed") and credits the eviction back before deciding. That credit was written, proven, and wired to the PIN path only. The autonomic planner kept asking the co-resident question, which on any healthy box makes the gate unreachable for every upgrade worth making — the better the model, the more certainly it is refused. Now `fresh` and the headroom bar are both computed against the post-eviction budget. Three lines of behaviour; the eviction credit itself already existed. ONE DEFECT, THREE CARDS. The planner could not reason about a budget the incumbent was occupying: - #214 sized the WINDOW from it (grow-back died after a squeeze) — fixed then, for windows - #440 chose the MODEL from it — a frontier model on disk was structurally unreachable without an operator pin - #266 sizes LANES from it, so warm slots starve and unslotted citizens re-prefill cold every turn; the plan probe names #266 by number while reporting it STILL GATED, deliberately. An upgrade additionally requires strictly greater `capability_rank` AND `SWITCH_UP_HEADROOM` of margin on the post-eviction budget, so budget noise cannot flap the lane — the flap that bounced live generations and wedged three benchmark runs. An EXTERNAL squeeze is still never credited (only the incumbent's own committed weights), so a genuine over-commit still forces the down-switch. Two new tests pin both directions: an upgrade that fits post-eviction is adopted; an equal-capability or no-headroom swap is still refused. TEST PREMISE CHANGE, stated in its own terms per the standing rule: `stable_keeps_incumbent_when_upgrade_lacks_headroom` was sized against the UN-CREDITED budget. Crediting the 1GB incumbent moved its bar 18.0 → 18.9 and the old fixture slid under it, so the test began asserting the opposite of its name. The invariant is unchanged — a model lacking headroom must not be adopted — and only the fixture is restated for the corrected budget (big 18 → 19GB). Its setup assertion now asks "would selection pick big?" at the at-rest budget, because that is where selection happens; asking at `host` would have made the setup vacuous and passed for the wrong reason. 7273 lib tests pass, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/serving_plan.rs | 173 ++++++++++++++++-- 1 file changed, 160 insertions(+), 13 deletions(-) diff --git a/core/continuum-core/src/cognition/serving_plan.rs b/core/continuum-core/src/cognition/serving_plan.rs index 116b4813c..0cce65cbc 100644 --- a/core/continuum-core/src/cognition/serving_plan.rs +++ b/core/continuum-core/src/cognition/serving_plan.rs @@ -690,9 +690,8 @@ pub fn plan_serving_stable( // is STILL resident and serving fine — its memory is its own. Tearing that // down to "nothing" is the exact harm we're guarding against, so `fresh` is // an Option we fall back to only when the incumbent genuinely can't hold. - let fresh = plan_serving(host, candidates, demand); let Some(inc_id) = incumbent else { - return fresh; + return plan_serving(host, candidates, demand); }; // NOTE (2026-08-04): there used to be an early return here — "fresh already chose the // incumbent → nothing to stabilize" — and it was the lane-flap bug. `fresh` is computed @@ -714,7 +713,7 @@ pub fn plan_serving_stable( // Incumbent dropped off disk entirely → honour whatever `fresh` chose // (possibly None = nothing servable). let Some(inc) = candidates.iter().find(|m| m.model_id == inc_id) else { - return fresh; + return plan_serving(host, candidates, demand); }; // The incumbent is ALREADY resident: its own weights read as "used" in live // free memory, which is exactly what depresses `usable_bytes` while it loads. @@ -726,16 +725,49 @@ pub fn plan_serving_stable( usable_bytes: host.usable_bytes.saturating_add(inc.weights_bytes), perf_cores: host.perf_cores, }; + // `fresh` is computed against the POST-EVICTION budget, and that is the whole fix. + // + // It used to use `host` — the live budget WITH the incumbent still resident — so the + // planner asked "does the better model fit in the memory left over beside the one already + // loaded?". That question is wrong, because a swap is never co-resident: `serve()` kills + // the incumbent llama-server child and THEN launches the candidate, exactly as + // `pin_fit_decision` documents ("the candidate only needs to fit AFTER the incumbent's + // VRAM is reclaimed"). So the eviction credit-back was written, proven, and wired to the + // PIN path only, while the autonomic planner kept asking the co-resident question. + // + // MEASURED on this M5 (2026-08-19): `serving.plan` chose Devstral-24B at `usable_gb: 18` + // on a 64 GB box, because Devstral + the vision lane + embeddings were already resident. + // Qwen3.8-27B (18.97 GB of weights) cannot fit a lane inside 18 GB, so it never entered + // the plan. `serving/pin` — same models, same instant, same machine — computed + // `budget_gb: 34.09` via this credit and the 27B fit with 15 GB to spare. Two answers to + // one question, and only the operator-driven path could reach the better model. + // + // This is the third sighting of ONE defect: the planner could not reason about a budget + // the incumbent was occupying. #214 fixed it for the WINDOW (grow-back), the pin path had + // it for the MODEL, and #266's warm slots starve for the same reason — lanes are sized + // from a budget the resident model has already eaten, so 3 of 4 citizens got no warm slot + // and re-prefilled cold every turn. One credit, three cards. + // + // SAFE because it is still gated: the switch-up below additionally requires strictly + // greater `capability_rank` AND `SWITCH_UP_HEADROOM` of margin, so a bigger model is + // adopted only when it is genuinely better AND genuinely fits post-eviction. A transient + // budget bump cannot flap it. An EXTERNAL squeeze is still not credited back — only the + // incumbent's own committed weights are — so a real over-commit still forces the + // down-switch. [[never-thrash-sticky-hysteresis-on-every-lane]] + let fresh = plan_serving(at_rest, candidates, demand); // Even crediting its own residency, can the incumbent still hold a lane? If // not, a real squeeze has genuinely evicted it → take `fresh`. if inc.weights_bytes.saturating_add(inc.kv_at(MIN_SERVE_CTX)) > at_rest.usable_bytes { return fresh; } - // Switch UP to `fresh` ONLY if it is strictly more capable AND fits the REAL - // (un-credited) budget with headroom — loading a bigger NEW model needs actual - // free memory, so this test uses `host`, not `at_rest`, and the headroom stops - // a transient budget bump from flapping up. - let headroom_budget = (host.usable_bytes as f64 * (1.0 - SWITCH_UP_HEADROOM)) as u64; + // Switch UP to `fresh` ONLY if it is strictly more capable AND fits the POST-EVICTION + // budget with headroom. This used to test against `host` on the reasoning that "loading a + // bigger NEW model needs actual free memory" — true for a co-resident load, false for the + // swap `serve()` actually performs (kill incumbent, then launch). Testing against `host` + // made the gate unreachable for any model larger than the free remainder, which on a + // healthy box IS every upgrade worth making. `SWITCH_UP_HEADROOM` still supplies the + // anti-flap margin; it is now margin on the right budget. + let headroom_budget = (at_rest.usable_bytes as f64 * (1.0 - SWITCH_UP_HEADROOM)) as u64; // The plan HOLDS its model, so this is a field read, not a search. It used to be // `candidates.iter().find(|c| c.model_id == f.base_model_id)` — a name lookup that // returned None whenever the planned name wasn't in the candidate list, silently @@ -1582,23 +1614,43 @@ mod tests { // upgrade worth flapping for. Before the model-choice floor landed, the old // MIN_SERVE_CTX bar admitted big here on 184MB of KV and this fixture read as // "fresh would pick big" when what fresh actually had was a 2048-token 9GB model. - // Sized instead so big genuinely clears a full turn (18 + 1.47 = 19.5 <= 20) yet - // still exceeds the headroom bar (18.18 > 0.9 * 20 = 18). + // PREMISE CHANGE (2026-08-19), stated as such: this fixture was sized against the + // UN-CREDITED budget, because the switch-up bar used to be `0.9 * host`. The bar now + // sits on the POST-EVICTION budget (`host` + the incumbent's own weights), since + // `serve()` kills the incumbent before launching the candidate and the two are never + // co-resident. Crediting `small`'s 1GB moved the bar 18.0 → 18.9, and the old `big` + // (18GB + 0.18 KV = 18.18) slid under it — so the test began asserting the OPPOSITE of + // its own name. The INVARIANT is untouched: a model that lacks headroom must not be + // adopted. Only the fixture is restated for the corrected budget. + // + // Re-sized so big still clears a full turn at the at-rest budget (19 + 1.47 = 20.47 <= + // 21) — a legitimate upgrade target selection would really pick — yet still exceeds the + // headroom bar (19 + 0.18 = 19.18 > 0.9 * 21 = 18.9). let host = HostBudget { usable_bytes: 20 * GB, perf_cores: 6, }; let models = vec![ fp("small", 1, 4_000, 32_768, 1), - fp("big", 18, 90_000, 262_144, 3), + fp("big", 19, 90_000, 262_144, 3), ]; + // Setup assertion: big IS what selection would choose, so the refusal below is a real + // headroom refusal and not "big was never a candidate". This asks the question at the + // budget `fresh` is now computed against — the at-rest one — because that is where the + // stabilizer does its selecting. Asking it at `host` would make the setup vacuous: big + // (19 + 1.47 = 20.47) does not clear a full turn in 20GB at all, so the test would pass + // for the wrong reason, asserting a refusal of something never on offer. + let at_rest_for_setup = HostBudget { + usable_bytes: host.usable_bytes + 1 * GB, // small's credited weights + perf_cores: host.perf_cores, + }; assert_eq!( - plan_serving(host, &models, ServingDemand::new(MAX_LANES, None)) + plan_serving(at_rest_for_setup, &models, ServingDemand::new(MAX_LANES, None)) .unwrap() .base_model .model_id, "big", - "fresh would pick big" + "fresh would pick big at the post-eviction budget — so the refusal below is real" ); let stable = plan_serving_stable( host, @@ -1766,4 +1818,99 @@ mod tests { ); assert!(stable.lanes >= 1, "kept model still gets ≥1 lane"); } + + // what this catches: the autonomic planner being structurally unable to reach a BETTER + // model that fits — because it measured the candidate against a budget the incumbent was + // still occupying, while `serve()` evicts before it launches. + // + // These are THIS MacBook's real numbers on 2026-08-19, not invented ones. The live + // `serving.plan` chose Devstral (24B, ~14 GB) at `usable_gb: 18` on a 64 GB box, so + // Qwen3.8-27B (18.97 GB) "did not fit" and never entered the plan. `serving/pin`, same + // instant, credited the incumbent back and reported `budget_gb: 34.09` — the 27B fit with + // 15 GB spare and served at 17.2 tok/s. Two answers to one question; only the operator + // path could reach the better model. + #[test] + fn a_more_capable_model_that_fits_after_eviction_is_adopted_not_starved() { + // 18 GB free WITH the ~14 GB incumbent resident — the measured live condition. + let live = HostBudget { + usable_bytes: 18 * GB, + perf_cores: 10, + }; + let models = vec![ + fp("devstral-24b", 14, 100_000, 131_072, 5), + fp("qwen3.8-27b", 19, 100_000, 262_144, 9), // strictly more capable + ]; + + // The un-credited question — "fits BESIDE the incumbent?" — answers the wrong thing. + assert_eq!( + plan_serving(live, &models, ServingDemand::new(2, None)) + .unwrap() + .base_model + .model_id, + "devstral-24b", + "against the live budget the 27B cannot fit — this is the state that stranded it" + ); + + // The stabilizer credits the eviction and takes the upgrade. + let plan = plan_serving_stable( + live, + &models, + Some("devstral-24b"), + ServingDemand::new(2, None), + ) + .expect("a plan exists"); + assert_eq!( + plan.base_model.model_id, "qwen3.8-27b", + "a strictly more capable model that fits POST-EVICTION must be adopted — the swap \ + kills the incumbent before launching, so the co-resident test was never the right \ + question" + ); + } + + // what this catches: the credit-back turning into a thrash engine. Crediting eviction must + // NOT hand the upgrade gate a blank cheque — a marginally-fitting or no-better model still + // has to lose, or the planner swaps models on budget noise and every flip bounces the live + // lane under in-flight requests (the wedge that killed three benchmark runs). + #[test] + fn the_eviction_credit_still_refuses_a_marginal_or_no_better_swap() { + let live = HostBudget { + usable_bytes: 18 * GB, + perf_cores: 10, + }; + + // (a) Equal capability → never swap, however well it fits. + let equal = vec![ + fp("devstral-24b", 14, 100_000, 131_072, 5), + fp("sibling-24b", 14, 100_000, 131_072, 5), + ]; + assert_eq!( + plan_serving_stable(live, &equal, Some("devstral-24b"), ServingDemand::new(2, None)) + .unwrap() + .base_model + .model_id, + "devstral-24b", + "equal capability is not an upgrade — the incumbent holds" + ); + + // (b) More capable but it consumes the ENTIRE post-eviction budget, leaving nothing + // for KV — SWITCH_UP_HEADROOM must reject it rather than swap into a lane that + // cannot hold a window. + let marginal = vec![ + fp("devstral-24b", 14, 100_000, 131_072, 5), + fp("hog-70b", 32, 100_000, 262_144, 9), + ]; + assert_eq!( + plan_serving_stable( + live, + &marginal, + Some("devstral-24b"), + ServingDemand::new(2, None) + ) + .unwrap() + .base_model + .model_id, + "devstral-24b", + "a better model with no headroom left is refused — the credit is not a blank cheque" + ); + } } From 9e27ba61197257d659afa9c6c64f50b2f9f971c4 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 06:53:13 -0500 Subject: [PATCH 74/80] =?UTF-8?q?feat(benchmark):=20every=20catalogued=20s?= =?UTF-8?q?uite=20becomes=20fetchable=20=E2=80=94=20one=20shape-agnostic?= =?UTF-8?q?=20HF=20pull=20(#370)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The catalog has carried ~20 suites with real `source_url` rows for weeks while exactly ONE of them could be pulled. The reason was structural, not missing work: the only fetcher in the tree was `swe_bench::load_dataset`, and its paging + cache + error handling were FUSED to the `SweInstance` row shape. Every other suite was a name with a URL beside it that nothing could read — catalogued and unrunnable, which reads on a board exactly like catalogued and ready. Two changes, both small because the machinery was already correct: 1. `swe_bench::fetch_hf_rows(dataset, config, split)` — the paging + on-disk cache + in-band error surfacing lifted out of `load_dataset`, now returning raw `serde_json::Value` rows. It is the SAME code path already proven against SWE-bench Lite; `load_dataset` becomes the SWE shape-mapper over it and is otherwise unchanged. An empty pull is still an ERROR, never an empty suite — the distinction that keeps "the dataset refused us" from being scored as "the suite has no tasks". 2. `benchmark/fetch` — the verb. Resolves a catalogued name to its HF coordinates via `hf_coords_from`, stages the rows, and reports the REAL row count beside the catalog's declared `tasks`. That comparison is the point: a pass rate over a denominator that disagrees with the published suite is not comparable to anyone else's number, so the mismatch is surfaced at fetch time rather than discovered after a round is published. Refuses loud on all three ways this can be wrong, each with its own sentence: an unknown benchmark, an in-tree suite with no source, and a non-HF source (the catalog has GitHub-hosted rows — humaneval, mbpp, aider-polyglot — which must NOT be handed to the HF rows API, since that returns an in-band error a caller would read as "empty suite"). `hf_coords_from` is a pure function so the URL→dataset rule is tested rather than restated at the call site. The test walks the live catalog and asserts EVERY HF-sourced row resolves — a row that doesn't would be unfetchable while still looking catalogued, which is the exact state this commit closes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../continuum-core/src/cognition/swe_bench.rs | 90 +++++++-- core/continuum-core/src/commands/benchmark.rs | 184 ++++++++++++++++++ .../benchmark/BenchmarkFetchParams.ts | 15 ++ .../benchmark/BenchmarkFetchResult.ts | 18 ++ 4 files changed, 295 insertions(+), 12 deletions(-) create mode 100644 protocol/typescript/benchmark/BenchmarkFetchParams.ts create mode 100644 protocol/typescript/benchmark/BenchmarkFetchResult.ts diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 410630cb0..286acaf91 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -416,22 +416,53 @@ pub fn recorded_verdicts() -> Vec<(String, SweVerdict)> { out } -/// Fetch a dataset split, cached on first use. On-demand, never a gated install step. -pub async fn load_dataset(dataset: &str) -> Result, String> { - let cache = swe_cache_dir().join(format!("{}.json", dataset.replace('/', "__"))); +/// Pull every row of a HuggingFace dataset split, SHAPE-AGNOSTIC. +/// +/// # Why this is split out (#370, 2026-08-19) +/// +/// The catalog carries ~20 benchmark rows, each with a real `source_url`, and exactly ONE of +/// them could ever be fetched — because the only fetcher in the tree was fused to +/// `SweInstance`. Every other row was a name and a URL that nothing read: LiveCodeBench, +/// aider-polyglot, bigcodebench, evalplus, all declared, none pullable. That is the same +/// shape as the rest of this codebase's recurring defect — machinery written, never wired. +/// +/// The paging and caching were never SWE-specific. Only the row DECODE was. So this returns +/// raw `Value` rows and each family maps them to its own type: ONE fetcher, N shape-mappers +/// ([[the-compression-principle]]). `load_dataset` below is now the SWE mapper over this, so +/// the generic path is the one already proven against SWE-bench Lite in production — a new +/// suite inherits a fetcher that works rather than getting a second, untested one. +/// +/// Cached by (dataset, config, split) on first use; on-demand, never a gated install step. +pub async fn fetch_hf_rows( + dataset: &str, + config: &str, + split: &str, +) -> Result, String> { + let slug = format!( + "{}__{config}__{split}", + dataset.replace('/', "__") + ); + let cache = swe_cache_dir().join(format!("{slug}.rows.json")); if let Ok(bytes) = std::fs::read(&cache) { - if let Ok(rows) = serde_json::from_slice::>(&bytes) { + if let Ok(rows) = serde_json::from_slice::>(&bytes) { if !rows.is_empty() { return Ok(rows); } } } - let mut rows: Vec = Vec::new(); - // The datasets-server caps a page at 100; SWE-bench Lite is 300. - for offset in (0..2000).step_by(100) { + let mut rows: Vec = Vec::new(); + // The datasets-server caps a page at 100. The ceiling is a real bound, not a guess: it + // stops a mis-typed dataset name from paging forever against a server that answers 200 + // with an empty list. A suite larger than this needs the bound RAISED deliberately, with + // its size named — never silently truncated, which would publish a partial denominator as + // if it were the whole suite. + const MAX_ROWS: usize = 5_000; + for offset in (0..MAX_ROWS).step_by(100) { let url = format!( - "https://datasets-server.huggingface.co/rows?dataset={}&config=default&split=test&offset={}&length=100", + "https://datasets-server.huggingface.co/rows?dataset={}&config={}&split={}&offset={}&length=100", urlencoding_encode(dataset), + urlencoding_encode(config), + urlencoding_encode(split), offset ); let resp = reqwest::get(&url) @@ -441,6 +472,13 @@ pub async fn load_dataset(dataset: &str) -> Result, String> { .json() .await .map_err(|e| format!("dataset decode failed at offset {offset}: {e}"))?; + // The server reports its own errors in-band with a 200. Surface it verbatim rather + // than returning an empty set that a caller would read as "the suite is empty". + if let Some(err) = body.get("error").and_then(|e| e.as_str()) { + return Err(format!( + "huggingface refused `{dataset}` (config={config}, split={split}): {err}" + )); + } let page = body .get("rows") .and_then(|r| r.as_array()) @@ -451,14 +489,42 @@ pub async fn load_dataset(dataset: &str) -> Result, String> { } for entry in page { if let Some(row) = entry.get("row") { - if let Ok(inst) = serde_json::from_value::(row.clone()) { - rows.push(inst); - } + rows.push(row.clone()); } } } if rows.is_empty() { - return Err(format!("{dataset} returned no usable rows")); + return Err(format!( + "`{dataset}` (config={config}, split={split}) returned no rows — check the dataset \ + id and split name; an empty pull is never treated as an empty suite" + )); + } + let _ = std::fs::create_dir_all(swe_cache_dir()); + let _ = std::fs::write(&cache, serde_json::to_vec(&rows).unwrap_or_default()); + Ok(rows) +} + +/// The SWE shape-mapper over [`fetch_hf_rows`]. Cached on first use. +pub async fn load_dataset(dataset: &str) -> Result, String> { + let cache = swe_cache_dir().join(format!("{}.json", dataset.replace('/', "__"))); + if let Ok(bytes) = std::fs::read(&cache) { + if let Ok(rows) = serde_json::from_slice::>(&bytes) { + if !rows.is_empty() { + return Ok(rows); + } + } + } + let raw = fetch_hf_rows(dataset, "default", "test").await?; + let total = raw.len(); + let rows: Vec = raw + .into_iter() + .filter_map(|row| serde_json::from_value::(row).ok()) + .collect(); + if rows.is_empty() { + return Err(format!( + "{dataset} fetched {total} rows but NONE decoded as a SWE instance — this dataset \ + is not SWE-instance-shaped, so it needs its own mapper rather than this one" + )); } let _ = std::fs::create_dir_all(swe_cache_dir()); let _ = std::fs::write(&cache, serde_json::to_vec(&rows).unwrap_or_default()); diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 708d7ae81..44fa39a86 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -1761,6 +1761,48 @@ mod tests { ) } + // what this catches: #370. Every catalogued suite carried a real `source_url` and exactly + // ONE of them could be pulled, because the only fetcher was fused to the SWE row shape. + // This pins the derivation that makes the OTHER rows reachable — and pins that a non-HF + // source (a GitHub raw .jsonl, of which the catalog has two) is REFUSED rather than + // silently handed to the HF rows API, which would return an in-band error the caller + // would read as "the suite is empty". + #[test] + fn every_hf_catalogued_suite_yields_coordinates_and_non_hf_is_refused() { + assert_eq!( + hf_coords_from("https://huggingface.co/datasets/princeton-nlp/SWE-bench_Lite"), + Some("princeton-nlp/SWE-bench_Lite") + ); + assert_eq!( + hf_coords_from( + "https://github.com/openai/human-eval/raw/master/data/HumanEval.jsonl.gz" + ), + None, + "a non-HF source must refuse, not fall through to the HF rows API" + ); + // A dataset URL with no owner segment is not addressable by the rows API. + assert_eq!(hf_coords_from("https://huggingface.co/datasets/apps"), None); + + // And the catalog itself: every HF-sourced row must resolve, or the suite is a name + // nothing can read — which is exactly the state #370 opened on. + let hf: Vec<_> = known_benchmarks() + .iter() + .filter_map(|b| b.source_url.map(|u| (b.name, u))) + .filter(|(_, u)| u.contains("huggingface.co")) + .collect(); + assert!( + !hf.is_empty(), + "the catalog carries no HF-sourced suite at all — `benchmark/fetch` would be dead" + ); + for (name, url) in hf { + assert!( + hf_coords_from(url).is_some(), + "`{name}` is HF-sourced at `{url}` but yields no coordinates — it would be \ + unfetchable while looking catalogued" + ); + } + } + // what this catches: THE round-killer of 2026-08-18. Citizens registered but not yet // hosted made every readiness surface report a ready roster, and dispatch staged a full // round into a room where nobody had a perception stream — `dispatched: 2, kickoffs: 2, @@ -3521,5 +3563,147 @@ impl ActionCommand for BenchmarkRounds { } } +// --------------------------------------------------------------------------- +// benchmark/fetch — stage a catalogued suite so it can actually be run (#370) +// --------------------------------------------------------------------------- + +/// A catalogued benchmark's HuggingFace coordinates, derived from its `source_url`. +/// +/// Split out as a pure function so the URL→(dataset, config, split) rule is TESTED rather +/// than restated at the call site. `config`/`split` default to the HF convention and are +/// overridable per call, because a dataset that is not `default`/`test` is common and +/// guessing wrong yields an in-band error the caller would otherwise read as "empty suite". +pub fn hf_coords_from(source_url: &str) -> Option<&str> { + source_url + .strip_prefix("https://huggingface.co/datasets/") + .filter(|id| id.contains('/')) +} + +#[derive(Debug, Clone, Default, Serialize, Deserialize, TS, JsonSchema)] +#[ts( + export, + export_to = "../../../protocol/typescript/benchmark/BenchmarkFetchParams.ts" +)] +pub struct BenchmarkFetchParams { + /// Which catalogued benchmark to stage, as it appears in `benchmark/list`. + pub benchmark: String, + /// Dataset config. Defaults to `default` — the HF convention. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub config: Option, + /// Split to pull. Defaults to `test`, which is what a benchmark is scored on. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub split: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize, TS, JsonSchema)] +#[ts( + export, + export_to = "../../../protocol/typescript/benchmark/BenchmarkFetchResult.ts" +)] +pub struct BenchmarkFetchResult { + pub benchmark: String, + pub dataset: String, + pub config: String, + pub split: String, + /// Rows actually staged. This is the suite's REAL denominator — compare it against the + /// catalog's `tasks` before trusting any rate computed from it. + pub rows: usize, + /// The catalog's declared task count, so a mismatch is visible at fetch time rather than + /// discovered when a published number turns out to be over the wrong denominator. + pub declared_tasks: u32, + /// True when `rows` and `declared_tasks` agree. False is not fatal — datasets are revised + /// upstream — but a rate published over a disagreeing denominator is not comparable. + pub denominator_matches: bool, +} + +#[derive(Default)] +pub struct BenchmarkFetch; + +#[async_trait] +impl ActionCommand for BenchmarkFetch { + const NAME: &'static str = "benchmark/fetch"; + const ACCESS: AccessLevel = AccessLevel::AiSafe; + const DESCRIPTION: &'static str = + "Stage a catalogued benchmark's task list from its `source_url`, cached on disk (#370). \ + The catalog has carried ~20 suites with real source URLs while exactly ONE could be \ + pulled, because the only fetcher was fused to the SWE row shape — every other suite was \ + a name nothing could read. This pulls ANY HuggingFace-hosted suite through the same \ + paging+cache path already proven against SWE-bench Lite. Reports the REAL row count \ + beside the catalog's declared task count, because a pass rate over the wrong \ + denominator is not comparable to anyone else's number. Fails loud on an unknown \ + benchmark, a non-HF source, or a refused dataset — an empty pull is never reported as \ + an empty suite."; + type Params = BenchmarkFetchParams; + type Output = BenchmarkFetchResult; + + async fn run( + &self, + _ctx: &Ctx, + p: BenchmarkFetchParams, + ) -> Result { + let spec = known_benchmarks() + .iter() + .find(|b| b.name == p.benchmark) + .ok_or_else(|| { + CommandError::Invalid(format!( + "unknown benchmark `{}` — see `benchmark/list` for the catalogued names", + p.benchmark + )) + })?; + let source = spec.source_url.ok_or_else(|| { + CommandError::Invalid(format!( + "`{}` is an in-tree suite with no source to pull — it ships with the binary and \ + is already runnable via its eval_set", + spec.name + )) + })?; + let dataset = hf_coords_from(source).ok_or_else(|| { + CommandError::Invalid(format!( + "`{}` is sourced from `{source}`, which is not a HuggingFace dataset URL. Only \ + the HF path is wired; this suite needs its own fetcher.", + spec.name + )) + })?; + let config = p.config.unwrap_or_else(|| "default".to_string()); + let split = p.split.unwrap_or_else(|| "test".to_string()); + + let rows = crate::cognition::swe_bench::fetch_hf_rows(dataset, &config, &split) + .await + .map_err(CommandError::Internal)?; + + let denominator_matches = rows.len() as u32 == spec.tasks; + crate::probe!( + class = "benchmark.suite.staged", + benchmark = spec.name, + dataset = dataset, + rows = rows.len(), + declared = spec.tasks, + denominator_matches = denominator_matches, + "benchmark suite staged from its catalog source", + ); + if !denominator_matches { + tracing::warn!( + benchmark = %spec.name, + staged = rows.len(), + declared = spec.tasks, + "staged row count disagrees with the catalog's declared tasks — any rate \ + computed over this is NOT comparable until the denominator is reconciled" + ); + } + Ok(BenchmarkFetchResult { + benchmark: spec.name.to_string(), + dataset: dataset.to_string(), + config, + split, + rows: rows.len(), + declared_tasks: spec.tasks, + denominator_matches, + }) + } +} + crate::register_stateless_command!(BenchmarkRuns); crate::register_stateless_command!(BenchmarkRounds); +crate::register_stateless_command!(BenchmarkFetch); diff --git a/protocol/typescript/benchmark/BenchmarkFetchParams.ts b/protocol/typescript/benchmark/BenchmarkFetchParams.ts new file mode 100644 index 000000000..dd6741096 --- /dev/null +++ b/protocol/typescript/benchmark/BenchmarkFetchParams.ts @@ -0,0 +1,15 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +export type BenchmarkFetchParams = { +/** + * Which catalogued benchmark to stage, as it appears in `benchmark/list`. + */ +benchmark: string, +/** + * Dataset config. Defaults to `default` — the HF convention. + */ +config?: string, +/** + * Split to pull. Defaults to `test`, which is what a benchmark is scored on. + */ +split?: string, }; diff --git a/protocol/typescript/benchmark/BenchmarkFetchResult.ts b/protocol/typescript/benchmark/BenchmarkFetchResult.ts new file mode 100644 index 000000000..88005731c --- /dev/null +++ b/protocol/typescript/benchmark/BenchmarkFetchResult.ts @@ -0,0 +1,18 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +export type BenchmarkFetchResult = { benchmark: string, dataset: string, config: string, split: string, +/** + * Rows actually staged. This is the suite's REAL denominator — compare it against the + * catalog's `tasks` before trusting any rate computed from it. + */ +rows: number, +/** + * The catalog's declared task count, so a mismatch is visible at fetch time rather than + * discovered when a published number turns out to be over the wrong denominator. + */ +declared_tasks: number, +/** + * True when `rows` and `declared_tasks` agree. False is not fatal — datasets are revised + * upstream — but a rate published over a disagreeing denominator is not comparable. + */ +denominator_matches: boolean, }; From 77d9360099d51456f44f5646c3b2577c5d2edcb3 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 07:07:12 -0500 Subject: [PATCH 75/80] feat(benchmark): fetch coordinates are catalog DATA, and each refusal names its own fix (#370) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live-running the new `benchmark/fetch` across the catalog produced three refusals, and reading them showed the verb was right but under-informed. Two findings, both measured against `datasets-server.huggingface.co` on 2026-08-19, both now encoded instead of remembered: 1. `bigcode/bigcodebench` has NO `test` split. It publishes revisions AS splits — v0.1.0_hf through v0.1.4. Default coordinates returned "Unexpected error", which reads like a transient and is not. Worse, WHICH revision a score was taken against is exactly the kind of fact that has to be recorded for a number to be comparable, so it belongs in the catalog and not in whoever last typed the command. 2. `codeparrot/apps` and `livecodebench/code_generation_lite` are LOADING-SCRIPT datasets. The rows API refuses them outright — "runs arbitrary Python code" — at every config and split. A "Not found" refusal invites the operator to guess configs forever against a wall. So `BenchmarkSpec::reach()` replaces the bare URL parse with a four-state classification: `Rows{dataset, config, split}` / `HuggingFaceScriptDataset` / `ForeignSource` / `InTree`. Four states because the four have four different fixes, and collapsing any two of them produces a refusal the caller has to do archaeology on. Script datasets now say so, and say that retrying is wasted effort. Foreign sources say they need their own fetcher. In-tree says it already ships. Params still override the coordinates for a one-off. This is the [[foolproof-over-instructions]] shape: every line a runbook would have had to carry ("bigcodebench wants --split=v0.1.4"; "don't bother with apps") is a defect in the command, fixed in the command. MEASURED, live, on build 9e27ba611 — four suites now stage that never could before: swe-bench-lite 300/300, swe-bench-verified 500/500, evalplus 164/164, cruxeval 800/800, every one with `denominator_matches: true` against the catalog's declared task count. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- core/continuum-core/src/commands/benchmark.rs | 187 +++++++++++++----- 1 file changed, 136 insertions(+), 51 deletions(-) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 44fa39a86..12b529db0 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -1769,38 +1769,57 @@ mod tests { // would read as "the suite is empty". #[test] fn every_hf_catalogued_suite_yields_coordinates_and_non_hf_is_refused() { + let by_name = |n: &str| known_benchmarks().iter().find(|b| b.name == n).unwrap(); + assert_eq!( - hf_coords_from("https://huggingface.co/datasets/princeton-nlp/SWE-bench_Lite"), - Some("princeton-nlp/SWE-bench_Lite") - ); - assert_eq!( - hf_coords_from( - "https://github.com/openai/human-eval/raw/master/data/HumanEval.jsonl.gz" - ), - None, - "a non-HF source must refuse, not fall through to the HF rows API" + by_name("swe-bench-lite").reach(), + SourceReach::Rows { + dataset: "princeton-nlp/SWE-bench_Lite", + config: "default", + split: "test" + } ); - // A dataset URL with no owner segment is not addressable by the rows API. - assert_eq!(hf_coords_from("https://huggingface.co/datasets/apps"), None); - - // And the catalog itself: every HF-sourced row must resolve, or the suite is a name - // nothing can read — which is exactly the state #370 opened on. - let hf: Vec<_> = known_benchmarks() - .iter() - .filter_map(|b| b.source_url.map(|u| (b.name, u))) - .filter(|(_, u)| u.contains("huggingface.co")) - .collect(); assert!( - !hf.is_empty(), - "the catalog carries no HF-sourced suite at all — `benchmark/fetch` would be dead" + matches!(by_name("humaneval").reach(), SourceReach::ForeignSource { .. }), + "a GitHub raw .jsonl must NOT fall through to the HF rows API — that returns an \ + in-band error a caller reads as 'the suite is empty'" ); - for (name, url) in hf { + assert!(matches!(by_name("hard-rs").reach(), SourceReach::InTree)); + + // The two live-verified script datasets (2026-08-19): the rows API refuses these at + // EVERY coordinate, so they must be told apart from a wrong-config miss or the + // operator retries configs forever. + for n in ["apps", "livecodebench"] { assert!( - hf_coords_from(url).is_some(), - "`{name}` is HF-sourced at `{url}` but yields no coordinates — it would be \ - unfetchable while looking catalogued" + matches!(by_name(n).reach(), SourceReach::HuggingFaceScriptDataset { .. }), + "`{n}` is a loading-script dataset; classifying it as fetchable sends the \ + caller into a config-guessing loop that can never succeed" ); } + + // what this catches specifically: bigcodebench publishes REVISIONS as splits and has + // no `test` split at all. The default coordinates return "Unexpected error" — measured + // live — so the version we score against has to be a recorded catalog fact. + assert_eq!( + by_name("bigcodebench").reach(), + SourceReach::Rows { + dataset: "bigcode/bigcodebench", + config: "default", + split: "v0.1.4" + } + ); + + // And no catalogued row may be silently unclassifiable. + for b in known_benchmarks() { + let reach = b.reach(); + if let SourceReach::Rows { dataset, .. } = reach { + assert!( + dataset.contains('/'), + "`{}` resolves to `{dataset}`, which the rows API cannot address", + b.name + ); + } + } } // what this catches: THE round-killer of 2026-08-18. Citizens registered but not yet @@ -3567,16 +3586,64 @@ impl ActionCommand for BenchmarkRounds { // benchmark/fetch — stage a catalogued suite so it can actually be run (#370) // --------------------------------------------------------------------------- -/// A catalogued benchmark's HuggingFace coordinates, derived from its `source_url`. -/// -/// Split out as a pure function so the URL→(dataset, config, split) rule is TESTED rather -/// than restated at the call site. `config`/`split` default to the HF convention and are -/// overridable per call, because a dataset that is not `default`/`test` is common and -/// guessing wrong yields an in-band error the caller would otherwise read as "empty suite". -pub fn hf_coords_from(source_url: &str) -> Option<&str> { - source_url - .strip_prefix("https://huggingface.co/datasets/") - .filter(|id| id.contains('/')) +/// Where a catalogued suite's rows actually live, and whether anything in this tree can read +/// them. Four states, because the four have four different fixes and collapsing any two of +/// them produces a refusal the operator has to go do archaeology on. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum SourceReach { + /// Servable by the HuggingFace rows API at these exact coordinates. + Rows { + dataset: &'static str, + config: &'static str, + split: &'static str, + }, + /// HuggingFace-hosted, but a LOADING-SCRIPT dataset: the rows API refuses it outright + /// ("runs arbitrary Python code"). No config or split makes it work, so a refusal that + /// merely says "not found" invites an infinite guessing loop. Measured live 2026-08-19 + /// against `datasets-server.huggingface.co/splits` for both rows carrying this. + HuggingFaceScriptDataset { dataset: &'static str }, + /// A real source, just not one the HF path can read (GitHub raw files, a repo to clone). + /// Needs its own fetcher; naming that is the honest answer. + ForeignSource { url: &'static str }, + /// Ships with the binary — there is nothing to pull. + InTree, +} + +impl BenchmarkSpec { + /// The suite's fetch coordinates, as DATA rather than as an operator's memory. + /// + /// `config`/`split` are NOT derivable from the URL and are not uniformly `default`/`test` + /// — bigcodebench versions its splits (`v0.1.4`), and a wrong guess returns an in-band HF + /// error that a caller reads as "the suite is empty". So the exceptions live here, in the + /// ONE place that knows, exactly as [`BenchmarkSpec::swe_dataset`] already does for row + /// shape. The dataset id is still read back off `source_url` so it is never duplicated. + pub fn reach(&self) -> SourceReach { + let Some(url) = self.source_url else { + return SourceReach::InTree; + }; + let Some(dataset) = url + .strip_prefix("https://huggingface.co/datasets/") + .filter(|id| id.contains('/')) + else { + return SourceReach::ForeignSource { url }; + }; + // Loading-script datasets: the rows API cannot serve these at ANY coordinates. + if matches!(dataset, "codeparrot/apps" | "livecodebench/code_generation_lite") { + return SourceReach::HuggingFaceScriptDataset { dataset }; + } + let (config, split) = match dataset { + // bigcodebench publishes revisions as SPLITS; `test` does not exist. v0.1.4 is + // the newest as of 2026-08-19 — bump it here when they publish, so the version + // scored against is a recorded catalog fact and not whatever the default was. + "bigcode/bigcodebench" => ("default", "v0.1.4"), + _ => ("default", "test"), + }; + SourceReach::Rows { + dataset, + config, + split, + } + } } #[derive(Debug, Clone, Default, Serialize, Deserialize, TS, JsonSchema)] @@ -3652,24 +3719,42 @@ impl ActionCommand for BenchmarkFetch { p.benchmark )) })?; - let source = spec.source_url.ok_or_else(|| { - CommandError::Invalid(format!( - "`{}` is an in-tree suite with no source to pull — it ships with the binary and \ - is already runnable via its eval_set", - spec.name - )) - })?; - let dataset = hf_coords_from(source).ok_or_else(|| { - CommandError::Invalid(format!( - "`{}` is sourced from `{source}`, which is not a HuggingFace dataset URL. Only \ - the HF path is wired; this suite needs its own fetcher.", - spec.name - )) - })?; - let config = p.config.unwrap_or_else(|| "default".to_string()); - let split = p.split.unwrap_or_else(|| "test".to_string()); + let (dataset, def_config, def_split) = match spec.reach() { + SourceReach::Rows { + dataset, + config, + split, + } => (dataset, config, split), + SourceReach::InTree => { + return Err(CommandError::Invalid(format!( + "`{}` is an in-tree suite with no source to pull — it ships with the binary \ + and is already runnable via its eval_set", + spec.name + ))) + } + SourceReach::HuggingFaceScriptDataset { dataset } => { + return Err(CommandError::Invalid(format!( + "`{}` is hosted at `{dataset}` as a LOADING-SCRIPT dataset — HuggingFace's \ + rows API refuses those outright (\"runs arbitrary Python code\"), so NO \ + config or split makes this work and retrying with different ones is wasted \ + effort. It needs a fetcher that reads the repo's own files (or an upstream \ + parquet conversion) before it can be staged.", + spec.name + ))) + } + SourceReach::ForeignSource { url } => { + return Err(CommandError::Invalid(format!( + "`{}` is sourced from `{url}`, which is not a HuggingFace dataset. Only the \ + HF rows path is wired; this suite needs its own fetcher.", + spec.name + ))) + } + }; + let config = p.config.unwrap_or_else(|| def_config.to_string()); + let split = p.split.unwrap_or_else(|| def_split.to_string()); let rows = crate::cognition::swe_bench::fetch_hf_rows(dataset, &config, &split) + .await .map_err(CommandError::Internal)?; From cea89e286803e9e59ec6a75a742d3577523338aa Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 07:27:51 -0500 Subject: [PATCH 76/80] feat(benchmark): ONE task shape, per-suite adapters that carry only the differences (#370) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, today: *"Supporting more benchmarks makes your adapter design robust and correct. This will make the next benchmark so much easier. The adapter handles only the differences."* Staging five suites made the real shape visible, and it is not one shape with variations — it is four genuinely different row families (all measured off the staged caches, not imagined): swe-bench-* repo + base_commit + problem_statement + test_patch + FAIL_TO_PASS/PASS_TO_PASS evalplus task_id + prompt + entry_point + test bigcodebench task_id + instruct_prompt + complete_prompt + entry_point + test + libs cruxeval id + code + input + output ← no repo, no tests, NO CODE WRITTEN AT ALL `cognition/bench_task.rs` gives them one target — `BenchTask { id, suite, statement, deliverable, oracle }` — and a `SuiteAdapter` trait carrying the per-suite delta. `Deliverable` and `Oracle` are SEPARATE enums because the two vary independently: two suites can both want a written function and score it completely differently. VALIDATED AT THE EXTREMES, per the outlier rule in CLAUDE.md — not at the average, which would have proven nothing. Outlier A is swe-bench-lite: a multi-thousand-file repo at a pinned commit, deliverable is a diff, oracle is a held-out test patch run as named pytest node-ids. Outlier B is cruxeval: no workspace, no code written, exact-match graded — the shape that would have needed a FAKE workspace and a FAKE deliverable to fit a "write a file" contract. Both project without forcing. That is why evalplus and bigcodebench landed as ONE adapter and not two: with the extremes carried, the middle differs only in which field poses the task. Three refusal disciplines, each with its own test: - A row that will not project ABORTS the import. It does not skip. A suite that silently projects 280 of 300 rows yields a pass rate over a denominator nobody chose, which reads exactly like a real score and is comparable to nothing. - A missing field is never defaulted to "". An empty statement is not a hard task and an empty oracle is not an unresolvable one — both are import bugs, and defaulting turns a broken import into a capability zero ([[an-absence-is-an-unfinished-measurement]], one layer earlier). - A fetchable-but-unadapted suite says so BY NAME and lists which suites are adapted, so a gap is distinguishable from a typo. `benchmark/fetch` now projects in the same breath as it stages and reports `tasks` beside `rows`, with `adapter_note` when projection is unavailable. Staged-but-unposable is the honest middle state; giving it its own field is what stops a fetched suite from LOOKING runnable. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_task.rs | 436 ++++++++++++++++++ core/continuum-core/src/cognition/mod.rs | 1 + core/continuum-core/src/commands/benchmark.rs | 21 + 3 files changed, 458 insertions(+) create mode 100644 core/continuum-core/src/cognition/bench_task.rs diff --git a/core/continuum-core/src/cognition/bench_task.rs b/core/continuum-core/src/cognition/bench_task.rs new file mode 100644 index 000000000..e1f96873a --- /dev/null +++ b/core/continuum-core/src/cognition/bench_task.rs @@ -0,0 +1,436 @@ +//! One task shape every benchmark projects into — so an adapter carries ONLY the differences. +//! +//! # Why this exists (Joel, 2026-08-19) +//! +//! > *"Supporting more benchmarks makes your adapter design robust and correct. This will make +//! > the next benchmark so much easier. The adapter handles only the differences."* +//! +//! Before this, "adapter" meant one function fused to one row shape: [`super::swe_bench`] could +//! read `SweInstance` and nothing else, so twenty catalogued suites were names with URLs beside +//! them that nothing could read. Fixing that by adding a second bespoke loader would have bought +//! one suite and left the third to be written from scratch again. +//! +//! So the fetch trunk is shared ([`super::swe_bench::fetch_hf_rows`]) and the per-suite delta is +//! a [`SuiteAdapter`]: given a raw row, produce a [`BenchTask`]. Everything downstream — the +//! card, the room, the workspace, the grade — speaks `BenchTask` and never knows which suite it +//! came from. +//! +//! # The interface was validated against its two EXTREMES, not its average +//! +//! Per the outlier-validation rule in CLAUDE.md: build outlier A and outlier B, and if the +//! interface fits both *without forcing*, the middle is guaranteed. The two extremes here are +//! maximally far apart, and both are real rows measured off disk on 2026-08-19: +//! +//! | | outlier A: `swe-bench-lite` | outlier B: `cruxeval` | +//! |---|---|---| +//! | workspace | a real multi-thousand-file repo at a pinned commit | none | +//! | citizen writes | a unified diff against existing source | **no code at all** | +//! | oracle | apply a held-out test patch, run named pytest node-ids | exact string match | +//! | failure mode | a passing test regressed | wrong answer | +//! +//! A shape that carries both carries `evalplus` and `bigcodebench` (write one function, run a +//! test script) trivially — which is why those two landed as one adapter, not two. +//! +//! # Absence is never a default +//! +//! Every projection FAILS LOUD on a missing field rather than substituting an empty string. +//! A task with an empty `statement` is not a hard task, and a task with an empty oracle is not +//! an unresolvable one — both are *import bugs*, and defaulting them turns a broken import into +//! a capability zero. That is the same confusion [[an-absence-is-an-unfinished-measurement]] +//! names, moved one layer earlier. + +use serde_json::Value; + +/// What the citizen is asked to produce. Distinct from HOW it is scored ([`Oracle`]) because the +/// two vary independently: two suites can both want a written function and grade it completely +/// differently. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Deliverable { + /// Patch an existing repository at a pinned commit. The workspace IS that repo, and the + /// deliverable is a diff against it. + RepoPatch { repo: String, base_commit: String }, + /// Write code from a prompt into a fresh workspace. `preamble` is the given source the + /// citizen completes (imports + signature + docstring); `entry_point` is the symbol the + /// oracle will call. + Program { + entry_point: String, + preamble: String, + }, + /// Produce an answer in words. Nothing is written to any workspace — this is the reasoning + /// tier, and it is the reason `Deliverable` is an enum rather than a workspace path. + Answer, +} + +/// The held-out scoring oracle. NEVER rendered into a citizen's prompt — it is the answer key, +/// and a suite whose oracle leaks into the statement is measuring recall, not capability. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Oracle { + /// Apply a test patch, then run named tests. Resolved iff every `fail_to_pass` passes AND + /// no `pass_to_pass` regresses — the second half is what makes a plausible-but-breaking + /// patch score zero instead of one. + RepoTests { + test_patch: String, + fail_to_pass: Vec, + pass_to_pass: Vec, + }, + /// Run a test program against the produced code. + TestProgram { source: String }, + /// Compare against a known answer. + ExactAnswer { expected: String }, +} + +/// One benchmark task, suite-agnostic. This is what a card, a room, and a grade all speak. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct BenchTask { + /// The suite's own id for this task, verbatim (`astropy__astropy-12907`, `HumanEval/0`, + /// `sample_0`). Kept as the upstream string so a published result can be joined against + /// anyone else's published result without a translation table. + pub id: String, + /// Which catalogued suite this came from. + pub suite: String, + /// What the citizen reads. The task as posed — never the answer. + pub statement: String, + pub deliverable: Deliverable, + pub oracle: Oracle, +} + +/// Projects one suite's raw rows into [`BenchTask`]s. +/// +/// Polymorphism rather than a match arm (the OpenCV `cv::Algorithm` shape CLAUDE.md prescribes): +/// adding a suite is a new impl plus one registry row, and nothing that consumes tasks changes. +pub trait SuiteAdapter: Send + Sync { + /// Which catalogued suite names this adapter serves. Declared BY the adapter so the + /// registry never becomes a second place that has to know. + fn serves(&self) -> &'static [&'static str]; + /// Project one raw row. `Err` names the missing field — an unprojectable row is a loud + /// import failure, never a silently-skipped task that shrinks the denominator. + fn project(&self, suite: &str, row: &Value) -> Result; +} + +/// Read a required string field, or say exactly which one was missing and on what. +fn req_str(row: &Value, field: &str, id_hint: &str) -> Result { + row.get(field) + .and_then(Value::as_str) + .map(str::to_string) + .ok_or_else(|| format!("row `{id_hint}` has no string field `{field}`")) +} + +/// `FAIL_TO_PASS`/`PASS_TO_PASS` arrive as a JSON-encoded array INSIDE a string — a real quirk +/// of the SWE-bench rows, not a guess (verified against the staged rows 2026-08-19). Accept the +/// native array too, so a re-export that fixes the quirk upstream does not break the import. +fn test_name_list(row: &Value, field: &str, id_hint: &str) -> Result, String> { + let raw = row + .get(field) + .ok_or_else(|| format!("row `{id_hint}` has no field `{field}`"))?; + let arr = match raw { + Value::Array(a) => a.clone(), + Value::String(s) => serde_json::from_str::>(s) + .map_err(|e| format!("row `{id_hint}` field `{field}` is not a JSON array: {e}"))? + .into_iter() + .map(Value::String) + .collect(), + other => { + return Err(format!( + "row `{id_hint}` field `{field}` is {other:?}, expected an array or an \ + array-encoding string" + )) + } + }; + Ok(arr + .iter() + .filter_map(|v| v.as_str().map(str::to_string)) + .collect()) +} + +/// OUTLIER A — repo-scale work: a real project at a pinned commit, graded by held-out pytest. +pub struct SweAdapter; + +impl SuiteAdapter for SweAdapter { + fn serves(&self) -> &'static [&'static str] { + &["swe-bench-lite", "swe-bench-verified"] + } + + fn project(&self, suite: &str, row: &Value) -> Result { + let id = req_str(row, "instance_id", "")?; + let fail_to_pass = test_name_list(row, "FAIL_TO_PASS", &id)?; + if fail_to_pass.is_empty() { + // A SWE instance with no failing test has no way to be resolved — importing it + // would add a task that can only ever score zero and drag the denominator. + return Err(format!("row `{id}` has an empty FAIL_TO_PASS — nothing to resolve")); + } + Ok(BenchTask { + statement: req_str(row, "problem_statement", &id)?, + deliverable: Deliverable::RepoPatch { + repo: req_str(row, "repo", &id)?, + base_commit: req_str(row, "base_commit", &id)?, + }, + oracle: Oracle::RepoTests { + test_patch: req_str(row, "test_patch", &id)?, + fail_to_pass, + pass_to_pass: test_name_list(row, "PASS_TO_PASS", &id)?, + }, + id, + suite: suite.to_string(), + }) + } +} + +/// The middle tier — write one function, run a test script against it. Two suites, ONE adapter, +/// because the only thing that differs between them is which field holds the prompt. +pub struct ProgramAdapter; + +impl SuiteAdapter for ProgramAdapter { + fn serves(&self) -> &'static [&'static str] { + &["evalplus", "bigcodebench"] + } + + fn project(&self, suite: &str, row: &Value) -> Result { + let id = req_str(row, "task_id", "")?; + // bigcodebench ships BOTH a natural-language `instruct_prompt` and a code-completion + // `complete_prompt`; evalplus ships only `prompt`. Prefer the instruction form — it is + // the one that measures whether a citizen can work from a described task rather than + // from an already-half-written function, which is what our citizens actually do. + let statement = req_str(row, "instruct_prompt", &id) + .or_else(|_| req_str(row, "prompt", &id)) + .map_err(|_| { + format!("row `{id}` has none of `instruct_prompt` / `prompt` to pose the task") + })?; + // The code the citizen starts from. bigcodebench names it `complete_prompt`; evalplus + // reuses `prompt` for both roles. + let preamble = req_str(row, "complete_prompt", &id) + .or_else(|_| req_str(row, "prompt", &id)) + .unwrap_or_else(|_| statement.clone()); + Ok(BenchTask { + statement, + deliverable: Deliverable::Program { + entry_point: req_str(row, "entry_point", &id)?, + preamble, + }, + oracle: Oracle::TestProgram { + source: req_str(row, "test", &id)?, + }, + id, + suite: suite.to_string(), + }) + } +} + +/// OUTLIER B — the reasoning tier: NO workspace, NO code written, exact-match graded. This is +/// the shape that proves the interface, because forcing it into a "write a file" contract would +/// have required a fake workspace and a fake deliverable. +pub struct ExecutionReasoningAdapter; + +impl SuiteAdapter for ExecutionReasoningAdapter { + fn serves(&self) -> &'static [&'static str] { + &["cruxeval"] + } + + fn project(&self, suite: &str, row: &Value) -> Result { + let id = req_str(row, "id", "")?; + let code = req_str(row, "code", &id)?; + let input = req_str(row, "input", &id)?; + Ok(BenchTask { + // cruxeval rows carry no prose — the task is posed by the harness, not the dataset. + // Composing it HERE (rather than at a call site) is precisely the "adapter handles + // only the differences" line: downstream still just reads `statement`. + statement: format!( + "Given the Python function below, determine the exact output of `f({input})`. \ + Reason about what the code does, then state the output.\n\n{code}" + ), + deliverable: Deliverable::Answer, + oracle: Oracle::ExactAnswer { + expected: req_str(row, "output", &id)?, + }, + id, + suite: suite.to_string(), + }) + } +} + +/// Every adapter in the tree. Adding a suite family = one impl + one row here. +fn adapters() -> Vec> { + vec![ + Box::new(SweAdapter), + Box::new(ProgramAdapter), + Box::new(ExecutionReasoningAdapter), + ] +} + +/// Project a whole suite's rows into tasks. +/// +/// Refuses rather than guesses in both directions: an unadapted suite says so by name, and a row +/// that will not project aborts the import instead of shrinking the task list silently. A +/// benchmark whose denominator quietly depends on how many rows happened to parse is not +/// comparable to anyone's published number. +pub fn project_suite(suite: &str, rows: &[Value]) -> Result, String> { + let all = adapters(); + let adapter = all + .iter() + .find(|a| a.serves().contains(&suite)) + .ok_or_else(|| { + let known: Vec<&str> = all.iter().flat_map(|a| a.serves().iter().copied()).collect(); + format!( + "`{suite}` has no SuiteAdapter — its rows can be fetched but not posed as tasks. \ + Adapted suites: {}. Adding one is an impl of SuiteAdapter plus a registry row.", + known.join(", ") + ) + })?; + rows.iter() + .map(|r| adapter.project(suite, r)) + .collect::, _>>() +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + /// Rows in the EXACT shapes measured off the staged caches on 2026-08-19 — not invented + /// shapes, which would let the adapters pass against a fiction. + fn swe_row() -> Value { + json!({ + "instance_id": "astropy__astropy-12907", + "repo": "astropy/astropy", + "base_commit": "d16bfe05a744909de4b27f5875fe0d4ed41ce607", + "problem_statement": "separability_matrix does not compute separability correctly", + "patch": "diff --git a/astropy/modeling/separable.py ...", + // NOTE: a STRING holding a JSON array — the real quirk, pinned below. + "test_patch": "diff --git a/astropy/modeling/tests/test_separable.py ...", + "FAIL_TO_PASS": "[\"astropy/modeling/tests/test_separable.py::test_separable\"]", + "PASS_TO_PASS": "[\"astropy/modeling/tests/test_separable.py::test_coord_matrix\"]", + }) + } + + /// The answer is deliberately DIFFERENT from the input here. An identity-function fixture + /// (which the real `sample_0` nearly is) cannot detect an oracle leak at all — the answer + /// string is already legitimately present as the input. + fn crux_row() -> Value { + json!({ + "id": "sample_0", + "code": "def f(nums):\n return len(nums)", + "input": "[1, 1, 3]", + "output": "3", + }) + } + + /// what this catches: the interface silently failing its own outlier test. A shape that fits + /// repo-patching but forces the reasoning tier into a fake workspace would still COMPILE and + /// still pass a SWE-only test — this asserts both extremes project cleanly and land on + /// genuinely different deliverables and oracles, which is the whole claim of the design. + #[test] + fn the_two_extremes_both_project_without_forcing_either_one() { + let swe = SweAdapter.project("swe-bench-lite", &swe_row()).unwrap(); + assert_eq!(swe.id, "astropy__astropy-12907"); + assert!(matches!(swe.deliverable, Deliverable::RepoPatch { .. })); + let Oracle::RepoTests { + fail_to_pass, + pass_to_pass, + .. + } = &swe.oracle + else { + panic!("a SWE task is graded by repo tests"); + }; + assert_eq!(fail_to_pass.len(), 1, "the array-in-a-string must decode"); + assert_eq!(pass_to_pass.len(), 1, "and regressions must be carried too"); + + let crux = ExecutionReasoningAdapter + .project("cruxeval", &crux_row()) + .unwrap(); + assert_eq!(crux.deliverable, Deliverable::Answer, "NO code is written"); + assert_eq!( + crux.oracle, + Oracle::ExactAnswer { + expected: "3".to_string() + } + ); + // The statement must pose the task even though the dataset carries no prose at all. + assert!(crux.statement.contains("def f(nums)") && crux.statement.contains("f([1, 1, 3])")); + // And the ANSWER must never appear in what the citizen reads — a statement built by + // string-formatting every field of the row would measure reading, not reasoning. + let Oracle::ExactAnswer { expected } = &crux.oracle else { + unreachable!() + }; + assert!( + !crux.statement.contains(&format!("= {expected}")) + && !crux.statement.contains(&format!("is {expected}")), + "the oracle leaked into the statement: {}", + crux.statement + ); + } + + /// what this catches: the single most expensive failure this module can have — an import + /// that drops rows it cannot read. A suite that silently projects 280 of 300 rows produces a + /// pass RATE over a denominator nobody chose, which is not comparable to any published + /// number and reads exactly like a real score. + #[test] + fn an_unreadable_row_aborts_the_import_instead_of_shrinking_the_suite() { + let mut broken = swe_row(); + broken.as_object_mut().unwrap().remove("problem_statement"); + let err = project_suite("swe-bench-lite", &[swe_row(), broken]).unwrap_err(); + assert!( + err.contains("problem_statement") && err.contains("astropy__astropy-12907"), + "the refusal must name the field AND the row: {err}" + ); + + // Same rule for a SWE instance with nothing to resolve. + let mut no_target = swe_row(); + no_target.as_object_mut().unwrap()["FAIL_TO_PASS"] = json!("[]"); + let err = project_suite("swe-bench-lite", &[no_target]).unwrap_err(); + assert!(err.contains("FAIL_TO_PASS"), "{err}"); + } + + /// what this catches: a fetched-but-unadapted suite looking runnable. `benchmark/fetch` can + /// stage rows for any HF suite; that is NOT the same as being able to pose them. The refusal + /// has to say which suites ARE adapted, or the caller cannot tell a typo from a gap. + #[test] + fn a_fetchable_but_unadapted_suite_says_so_and_lists_what_is_adapted() { + let err = project_suite("apps", &[]).unwrap_err(); + assert!(err.contains("no SuiteAdapter"), "{err}"); + assert!( + err.contains("swe-bench-lite") && err.contains("cruxeval"), + "the refusal must enumerate the adapted suites: {err}" + ); + } + + /// what this catches: the two program suites drifting into two adapters. They differ ONLY in + /// which field poses the task, and bigcodebench's `instruct_prompt` must win over its + /// `complete_prompt` — otherwise the citizen is handed a half-written function and we + /// measure completion instead of the described task. + #[test] + fn both_program_suites_share_one_adapter_and_prefer_the_instruction_form() { + let evalplus = json!({ + "task_id": "HumanEval/0", "entry_point": "has_close_elements", + "prompt": "from typing import List\n\ndef has_close_elements(...):", + "test": "def check(candidate): ...", + }); + let big = json!({ + "task_id": "BigCodeBench/0", "entry_point": "task_func", + "instruct_prompt": "Calculates the average of the sums of absolute differences.", + "complete_prompt": "import itertools\ndef task_func(...):", + "test": "import unittest ...", + }); + let tasks = project_suite("evalplus", &[evalplus]) + .unwrap() + .into_iter() + .chain(project_suite("bigcodebench", &[big]).unwrap()) + .collect::>(); + + assert_eq!(tasks.len(), 2); + assert!(tasks[0].statement.contains("has_close_elements")); + assert_eq!( + tasks[1].statement, + "Calculates the average of the sums of absolute differences.", + "the instruction form must pose the task, not the code-completion form" + ); + let Deliverable::Program { preamble, .. } = &tasks[1].deliverable else { + panic!("a program task carries the source the citizen starts from"); + }; + assert!( + preamble.contains("import itertools"), + "and the completion form survives as the PREAMBLE: {preamble}" + ); + for t in &tasks { + assert!(matches!(t.oracle, Oracle::TestProgram { .. })); + } + } +} diff --git a/core/continuum-core/src/cognition/mod.rs b/core/continuum-core/src/cognition/mod.rs index 7f68ad623..f93b8d7dc 100644 --- a/core/continuum-core/src/cognition/mod.rs +++ b/core/continuum-core/src/cognition/mod.rs @@ -31,6 +31,7 @@ pub mod act_observe; pub mod adaptive_throughput; pub mod audit; pub mod bench_round; +pub mod bench_task; pub mod round_readiness; pub mod benchmark; pub mod benchmark_humaneval; diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 12b529db0..336deabf7 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3683,6 +3683,17 @@ pub struct BenchmarkFetchResult { /// True when `rows` and `declared_tasks` agree. False is not fatal — datasets are revised /// upstream — but a rate published over a disagreeing denominator is not comparable. pub denominator_matches: bool, + /// How many rows actually PROJECT into posable tasks through this suite's `SuiteAdapter`. + /// `None` = the suite has no adapter yet: its rows are staged but cannot be posed to a + /// citizen. Staged-but-unposable is the honest middle state, and reporting it as a distinct + /// value is what keeps a fetched suite from LOOKING runnable. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub tasks: Option, + /// Present only when projection is unavailable or failed, saying which of those it was. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[ts(optional)] + pub adapter_note: Option, } #[derive(Default)] @@ -3777,6 +3788,14 @@ impl ActionCommand for BenchmarkFetch { computed over this is NOT comparable until the denominator is reconciled" ); } + // Project through the suite's adapter in the same breath as the fetch, so "staged" and + // "posable" can never drift apart in the operator's head. A projection failure is + // reported, NOT swallowed and NOT fatal — the rows are legitimately on disk either way. + let (tasks, adapter_note) = + match crate::cognition::bench_task::project_suite(spec.name, &rows) { + Ok(t) => (Some(t.len()), None), + Err(e) => (None, Some(e)), + }; Ok(BenchmarkFetchResult { benchmark: spec.name.to_string(), dataset: dataset.to_string(), @@ -3785,6 +3804,8 @@ impl ActionCommand for BenchmarkFetch { rows: rows.len(), declared_tasks: spec.tasks, denominator_matches, + tasks, + adapter_note, }) } } From 971de3ebb86b4d880e4ad074996978c5ee405013 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 07:49:23 -0500 Subject: [PATCH 77/80] =?UTF-8?q?feat(benchmark):=20the=20parse=20stops=20?= =?UTF-8?q?at=20the=20foreign=20edge=20=E2=80=94=20typed=20classes=20past?= =?UTF-8?q?=20it,=20answer=20key=20unbroadcastable=20(#370/#445)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel, today: *"parsing is a temporary solution to something that can emit well formed events and constant non string matchy airc events with headers."* Correct, and it sharpens where the boundary belongs. The HF row parse is legitimate ONLY because HuggingFace is a foreign, unversioned source we do not control — an edge always costs a parse. The defect is letting that parse LEAK PAST the edge, and it was about to: `BenchTask` had no wire identity, so every downstream consumer (card, room, grader) would have re-sniffed shapes. Same string-matching disease, one layer in. So the row is parsed exactly once, and everything after travels as a DECLARED class with a schema version, reusing `events::declare_event_class` rather than minting a parallel envelope — subscribers filter daemon-side on the channel instead of matching strings, and `on_unknown_schema` defaults to Fail so a shape nobody knows is never silently decoded. AND THE DISCIPLINE PAID IMMEDIATELY, in a way string-matched routing never could: benchmark:task.posed broadcast, ByRoomId — statement + deliverable; what a citizen may see benchmark:task.oracle NOT broadcast, Local — the held-out answer key The answer key gets its own class, and `broadcast: false` means a citizen has no channel on which to receive it. Held-out-ness stops being a convention someone must remember at each publish site and becomes a property of the transport. `BenchTask::split()` consumes self so no whole-task value lingers for a caller to publish by accident, and `PosedTask` has no field that could carry an oracle — a leak is a compile error, not a review catch. That is the failure this guards: if the oracle ever rode a broadcast class, every score after it would be worthless and NOTHING in a log would look wrong. Live-verified on build cea89e286 before this commit — all five staged suites project every row: swe-bench-lite 300/300, swe-bench-verified 500/500, evalplus 164/164, bigcodebench 1140/1140, cruxeval 800/800. 2,904 tasks, four row families, three adapters, zero drops. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_task.rs | 155 +++++++++++++++++- .../benchmark/BenchmarkFetchResult.ts | 13 +- protocol/typescript/benchmark/Deliverable.ts | 8 + protocol/typescript/benchmark/Oracle.ts | 7 + protocol/typescript/benchmark/PosedTask.ts | 8 + protocol/typescript/benchmark/TaskOracle.ts | 7 + 6 files changed, 195 insertions(+), 3 deletions(-) create mode 100644 protocol/typescript/benchmark/Deliverable.ts create mode 100644 protocol/typescript/benchmark/Oracle.ts create mode 100644 protocol/typescript/benchmark/PosedTask.ts create mode 100644 protocol/typescript/benchmark/TaskOracle.ts diff --git a/core/continuum-core/src/cognition/bench_task.rs b/core/continuum-core/src/cognition/bench_task.rs index e1f96873a..4c5280e60 100644 --- a/core/continuum-core/src/cognition/bench_task.rs +++ b/core/continuum-core/src/cognition/bench_task.rs @@ -39,12 +39,16 @@ //! a capability zero. That is the same confusion [[an-absence-is-an-unfinished-measurement]] //! names, moved one layer earlier. +use schemars::JsonSchema; +use serde::{Deserialize, Serialize}; use serde_json::Value; +use ts_rs::TS; /// What the citizen is asked to produce. Distinct from HOW it is scored ([`Oracle`]) because the /// two vary independently: two suites can both want a written function and grade it completely /// differently. -#[derive(Debug, Clone, PartialEq, Eq)] +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, TS, JsonSchema)] +#[ts(export, export_to = "../../../protocol/typescript/benchmark/Deliverable.ts")] pub enum Deliverable { /// Patch an existing repository at a pinned commit. The workspace IS that repo, and the /// deliverable is a diff against it. @@ -63,7 +67,8 @@ pub enum Deliverable { /// The held-out scoring oracle. NEVER rendered into a citizen's prompt — it is the answer key, /// and a suite whose oracle leaks into the statement is measuring recall, not capability. -#[derive(Debug, Clone, PartialEq, Eq)] +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, TS, JsonSchema)] +#[ts(export, export_to = "../../../protocol/typescript/benchmark/Oracle.ts")] pub enum Oracle { /// Apply a test patch, then run named tests. Resolved iff every `fail_to_pass` passes AND /// no `pass_to_pass` regresses — the second half is what makes a plausible-but-breaking @@ -94,6 +99,117 @@ pub struct BenchTask { pub oracle: Oracle, } +// --------------------------------------------------------------------------- +// The wire: two DECLARED classes, so nothing downstream ever re-parses (#445) +// --------------------------------------------------------------------------- +// +// Joel, 2026-08-19: *"parsing is a temporary solution to something that can emit well formed +// events and constant non string matchy airc events with headers."* +// +// Right. The parse above is legitimate ONLY because HuggingFace is a foreign, unversioned +// source we do not control — that is the edge, and an edge always costs a parse. The defect +// would be letting the parse LEAK past it: if a card, a room, or a grader re-sniffed row shapes, +// the string-matching disease would just move one layer in. So the row is parsed exactly once, +// here, and everything after it travels as a declared class with a schema version, filtered +// daemon-side by channel rather than matched by string. +// +// AND THE DISCIPLINE PAYS IMMEDIATELY, in a way string-matching never could have: +// the ANSWER KEY GETS ITS OWN CLASS. `benchmark:task.posed` is broadcast per-room — citizens +// subscribe and receive the task. `benchmark:task.oracle` is `Local` and NOT broadcast, so it +// never crosses airc and a citizen has no channel on which to receive it. Held-out-ness stops +// being a convention somebody has to remember and becomes a property of the transport. + +/// Wire schema version for both benchmark task classes. Subscribers fail loud on a mismatch +/// (`on_unknown_schema` defaults to `Fail`) — never silently decode a shape they don't know. +pub const BENCH_TASK_SCHEMA_VERSION: &str = "1.0.0"; + +/// The task AS POSED — everything a citizen may see, and nothing else. +pub const EVENT_TASK_POSED: &str = "benchmark:task.posed"; +/// The held-out answer key. Grader-only by TRANSPORT, not by convention. +pub const EVENT_TASK_ORACLE: &str = "benchmark:task.oracle"; + +/// Citizen-visible half of a [`BenchTask`]. Structurally cannot carry the oracle — the field +/// does not exist on this type, so "did we leak the answer?" is a compile-time question. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, TS, JsonSchema)] +#[ts(export, export_to = "../../../protocol/typescript/benchmark/PosedTask.ts")] +pub struct PosedTask { + pub id: String, + pub suite: String, + pub statement: String, + pub deliverable: Deliverable, +} + +/// Grader-only half. Rides a non-broadcast class; `id` + `suite` rejoin it to its [`PosedTask`]. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, TS, JsonSchema)] +#[ts(export, export_to = "../../../protocol/typescript/benchmark/TaskOracle.ts")] +pub struct TaskOracle { + pub id: String, + pub suite: String, + pub oracle: Oracle, +} + +impl BenchTask { + /// Split into the two wire payloads. Consuming `self` is deliberate: there is then no + /// lingering whole-task value for a caller to publish by accident. + pub fn split(self) -> (PosedTask, TaskOracle) { + ( + PosedTask { + id: self.id.clone(), + suite: self.suite.clone(), + statement: self.statement, + deliverable: self.deliverable, + }, + TaskOracle { + id: self.id, + suite: self.suite, + oracle: self.oracle, + }, + ) + } +} + +/// Register both classes. Idempotent, same as `declare_contract_event_classes`. +pub fn declare_bench_task_event_classes() -> Result { + use crate::events::{declare_event_class, EventClassChannelStrategy, EventClassConfig}; + + // Posed: per-room, because a bench task belongs to its run room and subscribers must be + // able to filter on the channel WITHOUT decoding the payload first. + declare_event_class( + EVENT_TASK_POSED, + &EventClassConfig { + broadcast: true, + channel: Some(EventClassChannelStrategy::ByRoomId), + schema_version: BENCH_TASK_SCHEMA_VERSION.to_string(), + on_unknown_schema: None, + description: Some( + "A benchmark task as posed to a citizen — statement + deliverable, never the \ + oracle (#370/#445)" + .to_string(), + ), + }, + ) + .map_err(|e| format!("failed to declare '{EVENT_TASK_POSED}': {e}"))?; + + // Oracle: LOCAL and NOT broadcast. This single line is what makes held-out-ness structural. + declare_event_class( + EVENT_TASK_ORACLE, + &EventClassConfig { + broadcast: false, + channel: Some(EventClassChannelStrategy::Local), + schema_version: BENCH_TASK_SCHEMA_VERSION.to_string(), + on_unknown_schema: None, + description: Some( + "The held-out answer key for a benchmark task. NEVER broadcast — grading is \ + in-process, and a citizen has no channel to receive this on (#370/#445)" + .to_string(), + ), + }, + ) + .map_err(|e| format!("failed to declare '{EVENT_TASK_ORACLE}': {e}"))?; + + Ok(2) +} + /// Projects one suite's raw rows into [`BenchTask`]s. /// /// Polymorphism rather than a match arm (the OpenCV `cv::Algorithm` shape CLAUDE.md prescribes): @@ -379,6 +495,41 @@ mod tests { assert!(err.contains("FAIL_TO_PASS"), "{err}"); } + /// what this catches: THE leak that matters. If the oracle ever rides a broadcast class, a + /// citizen can receive the answer key, every score after that is worthless, and NOTHING in a + /// log would look wrong. Pinning `broadcast: false` on the oracle class makes held-out-ness a + /// property of the TRANSPORT rather than a convention someone has to remember at each publish + /// site — which is the whole reason typed classes beat string-matched routing here. + #[test] + fn the_answer_key_can_never_be_broadcast_while_the_task_always_is() { + use crate::events::lookup_event_class; + declare_bench_task_event_classes().expect("declare must succeed"); + + let posed = lookup_event_class(EVENT_TASK_POSED).expect("posed class must be registered"); + assert!(posed.broadcast, "citizens must be able to receive the task"); + assert_eq!(posed.schema_version, BENCH_TASK_SCHEMA_VERSION); + + let oracle = lookup_event_class(EVENT_TASK_ORACLE).expect("oracle class registered"); + assert!( + !oracle.broadcast, + "the ANSWER KEY must never cross airc — this is the leak that silently invalidates \ + every score downstream" + ); + + // And the split must actually separate them: the posed half has no field that COULD + // carry an oracle, so a leak is a compile error rather than a review catch. + let (posed_task, key) = SweAdapter + .project("swe-bench-lite", &swe_row()) + .unwrap() + .split(); + assert_eq!(posed_task.id, key.id, "the two halves rejoin on id"); + let posed_json = serde_json::to_string(&posed_task).unwrap(); + assert!( + !posed_json.contains("test_patch") && !posed_json.contains("FAIL_TO_PASS"), + "the posed half leaked oracle content: {posed_json}" + ); + } + /// what this catches: a fetched-but-unadapted suite looking runnable. `benchmark/fetch` can /// stage rows for any HF suite; that is NOT the same as being able to pose them. The refusal /// has to say which suites ARE adapted, or the caller cannot tell a typo from a gap. diff --git a/protocol/typescript/benchmark/BenchmarkFetchResult.ts b/protocol/typescript/benchmark/BenchmarkFetchResult.ts index 88005731c..17075ce4b 100644 --- a/protocol/typescript/benchmark/BenchmarkFetchResult.ts +++ b/protocol/typescript/benchmark/BenchmarkFetchResult.ts @@ -15,4 +15,15 @@ declared_tasks: number, * True when `rows` and `declared_tasks` agree. False is not fatal — datasets are revised * upstream — but a rate published over a disagreeing denominator is not comparable. */ -denominator_matches: boolean, }; +denominator_matches: boolean, +/** + * How many rows actually PROJECT into posable tasks through this suite's `SuiteAdapter`. + * `None` = the suite has no adapter yet: its rows are staged but cannot be posed to a + * citizen. Staged-but-unposable is the honest middle state, and reporting it as a distinct + * value is what keeps a fetched suite from LOOKING runnable. + */ +tasks?: number, +/** + * Present only when projection is unavailable or failed, saying which of those it was. + */ +adapter_note?: string, }; diff --git a/protocol/typescript/benchmark/Deliverable.ts b/protocol/typescript/benchmark/Deliverable.ts new file mode 100644 index 000000000..87b807359 --- /dev/null +++ b/protocol/typescript/benchmark/Deliverable.ts @@ -0,0 +1,8 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * What the citizen is asked to produce. Distinct from HOW it is scored ([`Oracle`]) because the + * two vary independently: two suites can both want a written function and grade it completely + * differently. + */ +export type Deliverable = { "RepoPatch": { repo: string, base_commit: string, } } | { "Program": { entry_point: string, preamble: string, } } | "Answer"; diff --git a/protocol/typescript/benchmark/Oracle.ts b/protocol/typescript/benchmark/Oracle.ts new file mode 100644 index 000000000..9074568e0 --- /dev/null +++ b/protocol/typescript/benchmark/Oracle.ts @@ -0,0 +1,7 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. + +/** + * The held-out scoring oracle. NEVER rendered into a citizen's prompt — it is the answer key, + * and a suite whose oracle leaks into the statement is measuring recall, not capability. + */ +export type Oracle = { "RepoTests": { test_patch: string, fail_to_pass: Array, pass_to_pass: Array, } } | { "TestProgram": { source: string, } } | { "ExactAnswer": { expected: string, } }; diff --git a/protocol/typescript/benchmark/PosedTask.ts b/protocol/typescript/benchmark/PosedTask.ts new file mode 100644 index 000000000..c91657ac9 --- /dev/null +++ b/protocol/typescript/benchmark/PosedTask.ts @@ -0,0 +1,8 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { Deliverable } from "./Deliverable"; + +/** + * Citizen-visible half of a [`BenchTask`]. Structurally cannot carry the oracle — the field + * does not exist on this type, so "did we leak the answer?" is a compile-time question. + */ +export type PosedTask = { id: string, suite: string, statement: string, deliverable: Deliverable, }; diff --git a/protocol/typescript/benchmark/TaskOracle.ts b/protocol/typescript/benchmark/TaskOracle.ts new file mode 100644 index 000000000..f62f91f00 --- /dev/null +++ b/protocol/typescript/benchmark/TaskOracle.ts @@ -0,0 +1,7 @@ +// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually. +import type { Oracle } from "./Oracle"; + +/** + * Grader-only half. Rides a non-broadcast class; `id` + `suite` rejoin it to its [`PosedTask`]. + */ +export type TaskOracle = { id: string, suite: string, oracle: Oracle, }; From 2ccada5f350649483d2e966a4ac64bb9d1f960ab Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 08:55:06 -0500 Subject: [PATCH 78/80] =?UTF-8?q?fix(benchmark):=20staging=20held=20two=20?= =?UTF-8?q?copies=20of=20a=20dataset=20to=20learn=20one=20number=20?= =?UTF-8?q?=E2=80=94=20and=20the=20row=20cap=20lied=20(#370)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TWO DEFECTS, BOTH MINE, both found by Joel from the symptom (an OOM under a concurrent full-suite `cargo test`) rather than by me reading the instrument that was already reporting it. 1. `benchmark/fetch` called `project_suite` purely to REPORT a count. That allocated a complete second copy of the suite — every statement, patch, and test body cloned — while the parsed `Vec` was still held. The memleak tracker printed `benchmark/fetch:+138MB` in my own reboot output and I read past it. Projecting was the right call (staged-but-unposable has to be visible); holding two copies to learn one integer was not. `count_projectable` projects, validates, and DROPS each task, so peak stays one dataset at any suite size — same adapter, same abort-on-first-bad-row semantics. The rows are then released before the result is built, since `fetch_hf_rows` has already written them to the on-disk cache. 2. The `MAX_ROWS` comment PROMISED a suite would "never [be] silently truncated" and nothing enforced it — a suite at or past the 5,000-row bound returned looking complete. That is the exact shape of a partial denominator published as a whole one. It now refuses, names the bound, and says the fix is a deliberate edit that states the real suite size. A suite of exactly MAX_ROWS trips it too; a loud refusal on a boundary is the correct trade against a silently short task list. WHAT THIS IS NOT. Joel's larger point stands and this commit does not answer it: staging is a RESOURCE CONSUMER and it does not lease. `resources::consumer::ResourceConsumer` already defines the contract — `consumer_id` / `footprint` / `reclaim`, where the daemon holds handles to trait-driven consumers and each concern decides HOW to fit (release, partial, defer, or refuse with a named reason). Serving implements it (#79); benchmark staging does not, so it will still allocate without regard for a live call or a warm lane. Removing a gratuitous doubling is not the same as participating in the budget. Tracked as the follow-up, not claimed here. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_task.rs | 46 ++++++++++++++++--- .../continuum-core/src/cognition/swe_bench.rs | 13 ++++++ core/continuum-core/src/commands/benchmark.rs | 16 +++++-- 3 files changed, 66 insertions(+), 9 deletions(-) diff --git a/core/continuum-core/src/cognition/bench_task.rs b/core/continuum-core/src/cognition/bench_task.rs index 4c5280e60..65082998a 100644 --- a/core/continuum-core/src/cognition/bench_task.rs +++ b/core/continuum-core/src/cognition/bench_task.rs @@ -380,9 +380,20 @@ fn adapters() -> Vec> { /// comparable to anyone's published number. pub fn project_suite(suite: &str, rows: &[Value]) -> Result, String> { let all = adapters(); - let adapter = all - .iter() + let adapter = adapter_for(&all, suite)?; + rows.iter() + .map(|r| adapter.project(suite, r)) + .collect::, _>>() +} + +/// The adapter lookup + its refusal, in ONE place so the two entry points cannot drift. +fn adapter_for<'a>( + all: &'a [Box], + suite: &str, +) -> Result<&'a dyn SuiteAdapter, String> { + all.iter() .find(|a| a.serves().contains(&suite)) + .map(|b| b.as_ref()) .ok_or_else(|| { let known: Vec<&str> = all.iter().flat_map(|a| a.serves().iter().copied()).collect(); format!( @@ -390,10 +401,33 @@ pub fn project_suite(suite: &str, rows: &[Value]) -> Result, Stri Adapted suites: {}. Adding one is an impl of SuiteAdapter plus a registry row.", known.join(", ") ) - })?; - rows.iter() - .map(|r| adapter.project(suite, r)) - .collect::, _>>() + }) +} + +/// How many rows project, WITHOUT materializing every task. +/// +/// # Why this exists rather than `project_suite(..).len()` +/// +/// MEASURED DEFECT, mine, 2026-08-19: `benchmark/fetch` called `project_suite` purely to report +/// a COUNT, which allocated a full second copy of the dataset — every statement, patch, and test +/// body cloned out of rows that were still held — on top of the parsed `Vec`. The +/// memleak tracker flagged the result at `benchmark/fetch:+138MB` and I read past it; a +/// concurrent full-suite `cargo test` then OOMed the box. +/// +/// The projection was the right thing to do (staged-but-unposable must be visible); holding two +/// copies to learn one number was not. Each task is projected, validated, and DROPPED here, so +/// peak memory stays one dataset regardless of suite size — and the refusal semantics are +/// identical, because it is the same adapter and the same first-error-aborts rule. +pub fn count_projectable(suite: &str, rows: &[Value]) -> Result { + let all = adapters(); + let adapter = adapter_for(&all, suite)?; + let mut projected = 0usize; + for row in rows { + // The BenchTask is built, validated, and dropped at the end of this iteration. + adapter.project(suite, row)?; + projected += 1; + } + Ok(projected) } #[cfg(test)] diff --git a/core/continuum-core/src/cognition/swe_bench.rs b/core/continuum-core/src/cognition/swe_bench.rs index 286acaf91..6a752feec 100644 --- a/core/continuum-core/src/cognition/swe_bench.rs +++ b/core/continuum-core/src/cognition/swe_bench.rs @@ -499,6 +499,19 @@ pub async fn fetch_hf_rows( id and split name; an empty pull is never treated as an empty suite" )); } + // The cap above PROMISED it would never silently truncate, and until now nothing enforced + // that — a suite at or past the bound returned looking complete, which is precisely how a + // partial denominator gets published as if it were the whole suite. Refuse instead. A suite + // of exactly MAX_ROWS trips this too; a loud refusal on a boundary case is the correct trade + // against a silently short task list, and the fix is one deliberate edit that NAMES the size. + if rows.len() >= MAX_ROWS { + return Err(format!( + "`{dataset}` (config={config}, split={split}) hit the {MAX_ROWS}-row fetch bound. \ + The suite may be larger than what was pulled, so this row count is NOT a \ + trustworthy denominator. Raise MAX_ROWS in swe_bench.rs deliberately, naming the \ + real suite size, rather than publishing a rate over a truncated list." + )); + } let _ = std::fs::create_dir_all(swe_cache_dir()); let _ = std::fs::write(&cache, serde_json::to_vec(&rows).unwrap_or_default()); Ok(rows) diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index 336deabf7..c98ecd362 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3791,17 +3791,27 @@ impl ActionCommand for BenchmarkFetch { // Project through the suite's adapter in the same breath as the fetch, so "staged" and // "posable" can never drift apart in the operator's head. A projection failure is // reported, NOT swallowed and NOT fatal — the rows are legitimately on disk either way. + // + // `count_projectable`, NOT `project_suite(..).len()`: this path only needs the COUNT, and + // materializing every task here held a second full copy of the dataset alongside `rows` + // (measured at +138MB on the memleak tracker, and it OOMed the box under a concurrent + // test run). Same adapter, same abort-on-first-bad-row rule, one dataset in memory. let (tasks, adapter_note) = - match crate::cognition::bench_task::project_suite(spec.name, &rows) { - Ok(t) => (Some(t.len()), None), + match crate::cognition::bench_task::count_projectable(spec.name, &rows) { + Ok(n) => (Some(n), None), Err(e) => (None, Some(e)), }; + // `fetch_hf_rows` already wrote the rows to the on-disk cache, and everything below + // needs only counts. Releasing them here keeps peak RSS at one dataset instead of + // holding the whole suite alive until the handler returns. + let row_count = rows.len(); + drop(rows); Ok(BenchmarkFetchResult { benchmark: spec.name.to_string(), dataset: dataset.to_string(), config, split, - rows: rows.len(), + rows: row_count, declared_tasks: spec.tasks, denominator_matches, tasks, From 4dae25f268364cfd996f5b94da94a48661a4ce78 Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 09:01:56 -0500 Subject: [PATCH 79/80] fix(agent): a roomless solve now DECLARES that it is invisible (#425 remainder) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `p.room.unwrap_or_else(Uuid::nil)`. That was the whole defect: a nil room makes `apply_act` skip receipt radiation entirely, so the run executes normally, lands in NO transcript, produces no room turn, and NOTHING anywhere says so. 13,209 turns went by in that state — 8.7% of all turns, 35% for one citizen — before anyone measured it, because an invisible run and a visible one produce identical logs. A roomless run is still LEGITIMATE and this does not refuse it. A bare `agent/solve` with no activity behind it has no room to radiate into, and inventing one puts receipts on a phantom — live-proven 2026-08-12, it stole the single-room chat projection. The defect was never that the branch exists. It is that the branch was taken in silence. So `RunVisibility::{InRoom(Uuid), Invisible}` replaces the `unwrap_or_else`, with `resolve` as the ONE place the param becomes a decision, `room_id()` for the act pipeline (nil is the shape `apply_act` already keys its skip on), and `warning()` returning the sentence ONLY for the invisible case — a visible run stays quiet, because a warning that fires on the happy path trains everyone to ignore it. The call site emits `agent.solve.roomless` + a tracing::warn once, at the point the branch is taken. An EXPLICIT `Uuid::nil()` resolves to Invisible too, pinned by its own test: a caller passing nil means what omitting it means, and treating them differently would reopen the silent branch through the other door. This is the same failure shape as the two fixed an hour ago (a fetch cap whose comment promised it would never truncate while nothing enforced it; a projection that doubled memory without saying so): the system takes a consequential branch and declines to mention it. An absence is only measurable if something declares it. Pairs with slice 1 (PR #2336) — a CLAIMED bench card now solves in the card's room, so the dispatched path is visible by construction and this covers everything else. 3 tests, green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/commands/agent/solve.rs | 127 +++++++++++++++++- 1 file changed, 126 insertions(+), 1 deletion(-) diff --git a/core/continuum-core/src/commands/agent/solve.rs b/core/continuum-core/src/commands/agent/solve.rs index a7dc954a5..fa7e7aa0e 100644 --- a/core/continuum-core/src/commands/agent/solve.rs +++ b/core/continuum-core/src/commands/agent/solve.rs @@ -74,6 +74,8 @@ pub struct AgentSolveParams { #[serde(default, skip_serializing_if = "Option::is_none")] #[ts(optional, type = "string")] pub room: Option, + // See `RunVisibility` below: the roomless case is now DECLARED rather than defaulted, + // because a silent invisible run is how 13,209 of them accumulated unnoticed (#425). /// Fire-and-poll (#86): when true, the solve is spawned DETACHED — `run` returns a job /// handle NOW (arms empty, `detached: true`) and the REAL result (patch + acts) lands in /// `~/.continuum/progress/agent-solve-.json`. A real agentic drive (N full-generation @@ -178,6 +180,68 @@ pub struct AgentSolveParams { pub attempts: Option, } +/// Whether this run's acts will be PERCEIVABLE, decided once and named. +/// +/// # Why this is a type and not `p.room.unwrap_or_else(Uuid::nil)` +/// +/// It was that `unwrap_or_else` (#425). A nil room makes `apply_act` skip receipt radiation +/// entirely, so the run executes normally and lands in NO transcript — and nothing anywhere +/// said so. 13,209 turns accumulated in that state (8.7% of all turns; 35% for one citizen) +/// before anyone measured it, because an invisible run and a visible one produce identical +/// logs. That is the same defect shape as a fetch cap that silently truncates: the system +/// took a consequential branch and declined to mention it. +/// +/// A roomless run is still LEGITIMATE — a bare `agent/solve` with no activity behind it has +/// no room to radiate into, and inventing one would put receipts on a phantom (live-proven +/// 2026-08-12, it stole the single-room chat projection). So this does not refuse. It +/// DECLARES, so the invisibility is a stated property of the run rather than a silence. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RunVisibility { + /// Acts radiate `persona:act` receipts into this room — perceivable by a human screen + /// and a citizen mind through the one ViewState pipe. + InRoom(Uuid), + /// No room: the work executes and is perceived by nobody, and no curriculum-visible + /// room turn is produced. + Invisible, +} + +impl RunVisibility { + /// The single place the room param becomes a decision. + pub fn resolve(room: Option) -> Self { + match room { + // A caller passing the nil uuid EXPLICITLY means the same thing as omitting it; + // treating them differently would let a nil slip through as "in room", which is + // exactly the silent branch this type exists to close. + Some(r) if !r.is_nil() => RunVisibility::InRoom(r), + _ => RunVisibility::Invisible, + } + } + + /// The uuid the act pipeline expects — nil for the invisible case, which is the shape + /// `apply_act` already keys its skip on. + pub fn room_id(&self) -> Uuid { + match self { + RunVisibility::InRoom(r) => *r, + RunVisibility::Invisible => Uuid::nil(), + } + } + + /// What to say when the run will be invisible. `None` when it is perceivable — a + /// visible run needs no announcement, and warning on the happy path trains people to + /// ignore the warning. + pub fn warning(&self) -> Option<&'static str> { + match self { + RunVisibility::InRoom(_) => None, + RunVisibility::Invisible => Some( + "this run has NO room: its acts execute but radiate no receipts, so nobody \ + — human or citizen — can perceive the work, and it produces no room turn. \ + Pass `room` (benchmark/dispatch supplies its per-run activity room) to make \ + the run perceivable.", + ), + } + } +} + /// What the caller grades when the solve returns. Two genuinely different contracts, /// so it is an enum on the wire, never a magic string ([[strings-to-enums]]). #[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize, TS, JsonSchema)] @@ -1250,7 +1314,20 @@ impl AgentSolve { // honest roomless fallback for a bare `agent/solve` with no activity // behind it; a DISPATCHED run always carries one, and that is what // turns her acts into room receipts instead of invisible work. - let room = p.room.unwrap_or_else(Uuid::nil); + let visibility = RunVisibility::resolve(p.room); + if let Some(why) = visibility.warning() { + // Announce ONCE, at the one place the branch is taken. This is the whole + // fix for #425's remaining half: the roomless state was never wrong, it was + // never SAID, and 13,209 turns went by in it. + crate::probe!( + class = "agent.solve.roomless", + run_id = %p.run_id.clone().unwrap_or_default(), + persona_id = %p.persona_id, + "solve run is INVISIBLE — no room, so no receipts and no room turn (#425)", + ); + tracing::warn!(run_id = ?p.run_id, "agent/solve: {why}"); + } + let room = visibility.room_id(); // The workspace-grounding sentence counters the observed "new project ritual" // (glass-boxed 2026-07-22 via turn capture: her first act on a seeded task was // code/create-workspace("my_stack_project") + a Rust hello-world + git/commit — @@ -1972,6 +2049,54 @@ fn frame_task(task: &str) -> String { #[cfg(test)] mod tests { + mod run_visibility { + use super::super::RunVisibility; + use uuid::Uuid; + + // what this catches: the roomless branch going quiet again. It was + // `p.room.unwrap_or_else(Uuid::nil)` — a consequential branch taken in silence — and + // 13,209 turns (8.7% of all turns; 35% for one citizen) executed invisibly before + // anyone measured it, because an invisible run and a visible one log identically. + // The type exists so the branch has a NAME and the invisible case carries a sentence. + #[test] + fn a_roomless_run_is_declared_invisible_and_says_why() { + let v = RunVisibility::resolve(None); + assert_eq!(v, RunVisibility::Invisible); + assert!(v.room_id().is_nil(), "the act pipeline keys its skip on nil"); + let why = v.warning().expect("an invisible run MUST announce itself"); + assert!( + why.contains("NO room") && why.contains("perceive"), + "the warning must say what is lost, not just that a field was absent: {why}" + ); + assert!( + why.contains("room"), + "and name the param that fixes it: {why}" + ); + } + + // what this catches: an EXPLICIT nil uuid slipping through as "in room". A caller + // passing Uuid::nil() means exactly what omitting it means; treating the two + // differently would reopen the silent branch through the other door. + #[test] + fn an_explicit_nil_room_is_the_same_as_no_room() { + assert_eq!( + RunVisibility::resolve(Some(Uuid::nil())), + RunVisibility::Invisible + ); + } + + // what this catches: warning on the happy path. A visible run needs no announcement, + // and a warning that fires every time trains everyone to ignore it. + #[test] + fn a_run_with_a_real_room_is_visible_and_stays_quiet() { + let room = Uuid::from_u128(7); + let v = RunVisibility::resolve(Some(room)); + assert_eq!(v, RunVisibility::InRoom(room)); + assert_eq!(v.room_id(), room, "the room must survive unchanged"); + assert!(v.warning().is_none(), "a perceivable run must not warn"); + } + } + mod patch_custody { // what this catches: patch custody going back to being a caller courtesy. It WAS // one — the write sat behind `if let Some(capture_dir)` with no else — and the From 0e342d193d9f0fed8c9a5744820fd368beddf86b Mon Sep 17 00:00:00 2001 From: Joel Teply Date: Wed, 19 Aug 2026 09:14:27 -0500 Subject: [PATCH 80/80] =?UTF-8?q?feat(resources):=20benchmark=20staging=20?= =?UTF-8?q?becomes=20the=20FIFTH=20peer=20consumer=20=E2=80=94=20and=20the?= =?UTF-8?q?=20one=20that=20always=20yields=20(#56)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Joel: *"crushing a benchmark while simultaneously in a video call with several persona can ONLY work if the intelligent systems have handles to the interface/trait driven consumers of resources... Conversely, the concerns themselves need to decide how to fit within budget."* Staging allocated a whole dataset with no regard for anyone else and OOMed the box. The earlier `drop(rows)` removed a gratuitous doubling and changed NOTHING structural: a large enough suite still evicts a live call by winning a race against `malloc`. A consumer that does not lease cannot be arbitrated, and an unarbitrated consumer is not a peer — it is a hazard. BOTH HALVES OF THE SEAM, which already existed and staging used neither: ASK `available_for(CONSUMER_ID, Ram)` → `StagingPlan::decide` runs BEFORE the allocation. Refusing with a NAMED shortfall beats an OOM that takes the citizens down with it. GIVE `ResourceConsumer::{footprint, reclaim}` → the governor can now take bytes back. WHY THIS SHAPE IS NEW FOR THE TRAIT, and why it was worth implementing rather than special-casing. Serving (#79), Bevy and Voice are all fat-and-holding: they own expensive residency for as long as they are up, and reclaim means degrading something a human perceives — so they weigh, and sometimes REFUSE. Staging inverts every axis: it holds only during a fetch, it holds nothing when idle, and every row it holds is already on disk. It is the cheapest victim on the box. So it NEVER refuses a RAM ask — not on Pressure, not on Rebalance, not on Shutdown. Yielding costs a re-read and nothing perceivable, which is exactly what makes a benchmark safe to run beside a video call instead of racing it. A test pins all three reasons; if this ever starts refusing, a benchmark can starve a call. The one honest complication: an ask for VRAM is REFUSED rather than answered with freed host bytes. Reporting the wrong kind would corrupt the ledger into believing device memory came back. `StagingResidency` is injectable for the same reason `RenderSurface` is — the reclaim disposition is proven without a process-global area or a live governor. The footprint returns to zero the instant the rows drop (normal path AND under reclaim), because a consumer reporting phantom bytes makes the governor evict a REAL holder to recover memory nobody has — the mirror image of the OOM this exists to prevent. DELIBERATELY ABSENT: a `Streamed` variant. When the budget cannot hold a suite the better answer is to page and project row-by-row rather than refuse — but that needs the pager inverted to a per-page callback and the disk cache moved to JSONL so it can be read incrementally. A variant nothing can construct is a lie about what the system does, so it gets its own slice, at which point `Refuse` narrows to "not even one page fits" and everything else adapts. 6 new tests; 172 green across bench_staging + bench_task + benchmark + all `resources::`; de-hardcode guard green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo --- .../src/cognition/bench_staging.rs | 406 ++++++++++++++++++ core/continuum-core/src/cognition/mod.rs | 1 + core/continuum-core/src/commands/benchmark.rs | 33 ++ core/continuum-core/src/ipc/mod.rs | 12 + 4 files changed, 452 insertions(+) create mode 100644 core/continuum-core/src/cognition/bench_staging.rs diff --git a/core/continuum-core/src/cognition/bench_staging.rs b/core/continuum-core/src/cognition/bench_staging.rs new file mode 100644 index 000000000..6a70cbddb --- /dev/null +++ b/core/continuum-core/src/cognition/bench_staging.rs @@ -0,0 +1,406 @@ +//! Benchmark suite staging as a GOVERNED consumer — it plans against a budget, and it gives +//! bytes back when a peer needs them. +//! +//! # Why this exists (Joel, 2026-08-19) +//! +//! > *"The idea of crushing a benchmark while simultaneously in a video call with several +//! > persona can ONLY work if the intelligent systems have handles to the interface/trait +//! > driven consumers of resources. These independent concerns cannot override or we will +//! > fail. Conversely, the concerns themselves need to decide how to fit within budget much of +//! > the time."* +//! +//! Staging allocated an entire dataset with no regard for anyone else on the box. It OOMed the +//! machine the same day. The reflex fix was a `drop(rows)` — which removes a gratuitous +//! doubling and changes NOTHING structural: a large enough suite still evicts a live call by +//! winning a race against `malloc`. A consumer that does not lease cannot be arbitrated, and an +//! unarbitrated consumer is not a peer, it is a hazard. +//! +//! # The shape, and why it is a genuinely new one for this trait +//! +//! Serving (#79), Bevy, and Voice are all **fat and holding**: they own expensive residency for +//! as long as they are up, and reclaim means degrading something a human can perceive. Staging +//! is the opposite on every axis, which is what makes it worth implementing rather than +//! special-casing: +//! +//! | | serving / bevy / voice | benchmark staging | +//! |---|---|---| +//! | lifetime | as long as the subsystem is up | only during a fetch + projection | +//! | reclaim cost | a tier-down, a frozen avatar, a dropped call | a re-read from a file that is already on disk | +//! | when idle | still holding | holding nothing at all | +//! | under pressure | must weigh refusing | should ALWAYS yield | +//! +//! That last row is the point. Staging is the **ideal reclaim victim**: its entire state is +//! reconstructible from the on-disk cache, so releasing costs latency and nothing else. Encoding +//! that as `Released` (never `Refused`) is what lets a video call take bytes back from a +//! benchmark mid-round instead of the two racing each other into an OOM. +//! +//! # Plan before you allocate — the half the other three do not exercise +//! +//! [`ResourceConsumer`] is the give-back half. The ASK half already exists too and staging was +//! blind to it: [`available_for`](crate::resources::ResourceDaemon::available_for) reports the +//! headroom THIS consumer may plan against — global available minus every other consumer's +//! unmet floor — and its own doc records why it exists (#225: serving planned from +//! reservation-blind `available`, grew its window over the embed lane's floor, and embedding +//! went dead). +//! +//! So [`StagingPlan::decide`] runs BEFORE the allocation, not after the failure. Refusing with a +//! named shortfall is strictly better than an OOM that takes the citizens down with it — Joel's +//! *"otherwise broken json and other things make the system completely degrade"*. +//! +//! # What is deliberately NOT here yet +//! +//! There is an obvious third state: when the budget is too small to hold a suite, STREAM it — +//! page from the HF cursor (or the disk cache) and project row-by-row, so peak stays one page +//! regardless of suite size. That upgrades a refusal into an adaptation, and it is the better +//! system. +//! +//! It is not in this enum, because a variant nothing can construct is a lie about what the +//! system does. Real streaming needs the pager inverted (a per-page callback instead of an +//! accumulate-then-return) and the on-disk cache moved from one JSON array to JSONL so it can +//! be read incrementally. That is a real change and it gets its own slice, at which point +//! `Refuse` narrows to "not even one page fits" and everything else adapts. + +use crate::resources::{ + ConsumerFootprint, ReclaimOutcome, ReclaimReason, ReclaimRequest, ResourceConsumer, + ResourceKind, +}; +use async_trait::async_trait; +use std::sync::atomic::{AtomicU64, Ordering}; + +/// The id staging leases under. Matches the `consumer_id` on its leases and its footprint rows. +pub const CONSUMER_ID: &str = "benchmark-staging"; + +/// How staging will read a suite, decided against the budget BEFORE any allocation. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum StagingPlan { + /// Enough plannable headroom: hold the rows for the projection pass. + Resident { + /// What staging expects to hold, and therefore what it should lease. + bytes: u64, + }, + /// Not enough. Refuse and NAME the shortfall — never allocate hopefully and let the + /// allocator arbitrate, which is how a benchmark takes a live call down with it. + Refuse { + needed: u64, + available: u64, + }, +} + +impl StagingPlan { + /// The rule alone, with the governor and the network taken out of it. + /// + /// `estimated_bytes` is what the suite is expected to occupy; `available` is + /// `available_for(CONSUMER_ID, Ram)`. A zero estimate is treated as Resident with zero + /// bytes — an empty suite legitimately needs nothing, and refusing it would turn a + /// harmless no-op into an error. + pub fn decide(estimated_bytes: u64, available: u64) -> Self { + if estimated_bytes <= available { + StagingPlan::Resident { + bytes: estimated_bytes, + } + } else { + StagingPlan::Refuse { + needed: estimated_bytes, + available, + } + } + } + + /// The operator-facing sentence for a refusal, naming the shortfall AND what changes it. + /// A gate that blocks without saying why relocates the archaeology instead of ending it. + pub fn explain_refusal(&self) -> Option { + match self { + StagingPlan::Resident { .. } => None, + StagingPlan::Refuse { needed, available } => Some(format!( + "benchmark staging needs ~{needed} bytes of RAM but the governor can only plan \ + {available} for `{CONSUMER_ID}` right now — another consumer (a serving lane, \ + a live call) holds the difference. Staging REFUSES rather than allocating \ + hopefully, because winning a race against the allocator here takes the \ + citizens down with it. Retry when the box is quieter, or free a lane." + )), + } + } +} + +/// What staging currently holds, and the lever to let it go. +/// +/// Abstracted as a trait for the same reason [`RenderSurface`](crate::modules::bevy_consumer) +/// is: a unit test proves the reclaim disposition without a process-global staging area or a +/// live governor. +pub trait StagingResidency: Send + Sync { + /// Bytes staging believes it is holding right now. Zero when idle, which is most of the time. + fn held_bytes(&self) -> u64; + /// Release the held rows and return what was freed. Safe by construction: every staged row + /// is already on disk, so this costs a re-read and nothing else. + fn release(&self) -> u64; +} + +/// The default residency: a process-wide byte counter staging updates as it holds and drops. +/// +/// A counter rather than the rows themselves because the rows live in the request that fetched +/// them — this type is the governor's HANDLE onto that, not its owner. `release` is therefore a +/// request to the holder, published as a flag the fetch path checks between pages. +#[derive(Debug, Default)] +pub struct StagingArea { + held: AtomicU64, + /// Set when the governor has asked for bytes back. The fetch path reads this and abandons, + /// which is why release is always honest: nothing keeps holding after the ask. + yielded: AtomicU64, +} + +impl StagingArea { + pub fn new() -> Self { + Self::default() + } + + /// Record that staging now holds `bytes` (called when a suite lands in memory). + pub fn hold(&self, bytes: u64) { + self.held.store(bytes, Ordering::SeqCst); + } + + /// Record that staging let go (called when the rows are dropped — including the normal, + /// non-pressure path, so the footprint returns to zero the moment a fetch completes). + pub fn drop_all(&self) { + self.held.store(0, Ordering::SeqCst); + } + + /// How many times the governor has asked staging to yield. Read by the fetch path so a + /// long staging pass can abandon mid-flight rather than finish and only then release. + pub fn yield_requests(&self) -> u64 { + self.yielded.load(Ordering::SeqCst) + } +} + +impl StagingResidency for StagingArea { + fn held_bytes(&self) -> u64 { + self.held.load(Ordering::SeqCst) + } + + fn release(&self) -> u64 { + self.yielded.fetch_add(1, Ordering::SeqCst); + self.held.swap(0, Ordering::SeqCst) + } +} + +/// So the governor's registered consumer and the fetch path that allocates are looking at the +/// SAME counter. Without this they would each hold their own and the footprint would be fiction. +impl StagingResidency for std::sync::Arc { + fn held_bytes(&self) -> u64 { + (**self).held_bytes() + } + fn release(&self) -> u64 { + (**self).release() + } +} + +/// THE staging area — one per process, because there is one pool of host RAM. +/// +/// A global rather than a threaded-through handle for the same reason the other consumers read +/// process-global subsystem state: the governor holds `Arc` and the +/// command path is a stateless `ActionCommand`; there is no shared owner to thread it through. +/// The `StagingResidency` trait is what keeps the tests off it. +pub fn staging_area() -> std::sync::Arc { + static AREA: std::sync::OnceLock> = std::sync::OnceLock::new(); + AREA.get_or_init(|| std::sync::Arc::new(StagingArea::new())) + .clone() +} + +/// Plan a staging pass against the LIVE governor, or fall back to Resident when no governor is +/// running (a unit test, a CLI invocation before boot). The fallback is deliberate and narrow: +/// with no governor there is no peer to starve, so refusing would block work for nobody's +/// benefit — but it is stated here rather than hidden as an `unwrap_or`. +pub fn plan_against_governor(estimated_bytes: u64) -> StagingPlan { + let Some(daemon) = crate::resources::ResourceDaemon::global() else { + return StagingPlan::Resident { + bytes: estimated_bytes, + }; + }; + let available = daemon.available_for(CONSUMER_ID, ResourceKind::Ram); + StagingPlan::decide(estimated_bytes, available) +} + +/// Staging's face to the governor. +pub struct StagingConsumer { + residency: R, +} + +impl StagingConsumer { + pub fn new(residency: R) -> Self { + Self { residency } + } +} + +#[async_trait] +impl ResourceConsumer for StagingConsumer { + fn consumer_id(&self) -> &str { + CONSUMER_ID + } + + fn footprint(&self) -> Vec { + let bytes = self.residency.held_bytes(); + vec![ConsumerFootprint { + kind: ResourceKind::Ram, + bytes, + detail: if bytes == 0 { + "benchmark staging: idle (holds rows only during a fetch + projection)".into() + } else { + format!("benchmark staging: {bytes} bytes of suite rows, re-readable from the on-disk cache") + }, + }] + } + + async fn reclaim(&self, request: ReclaimRequest) -> ReclaimOutcome { + // Staging NEVER refuses, on ANY reason — Pressure, Rebalance, or Shutdown. Every byte + // it holds is reconstructible from a file that is already on disk, so yielding costs a + // re-read and nothing a human or citizen can perceive. It is the cheapest victim on the + // box and should always be taken before serving tiers down or an avatar freezes. + // + // The only honest complication is the RAM/other-kind case: an ask for VRAM cannot be + // satisfied by dropping host rows, and reporting freed bytes for the wrong kind would + // corrupt the ledger. + if request.kind != ResourceKind::Ram { + return ReclaimOutcome::refused(format!( + "benchmark staging holds only RAM; it cannot free {:?}", + request.kind + )); + } + let freed = self.residency.release(); + crate::probe!( + class = "benchmark.staging.reclaim", + freed_bytes = freed, + target_bytes = request.target_bytes, + reason = ?request.reason, + "benchmark staging yielded to a peer (always — its state is on disk)", + ); + // Reporting `Released` with freed == 0 is correct and NOT a silent zero: staging is + // idle most of the time, and "I hold nothing, take it from someone else" is the true + // answer. The daemon reconciles against the hardware scan either way. + ReclaimOutcome::released(freed) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// what this catches: staging allocating hopefully and letting `malloc` arbitrate. That is + /// what OOMed the box on 2026-08-19 — a benchmark and the rest of the system racing, with + /// the loser being whoever asked second. A plan that REFUSES with a named shortfall is + /// strictly better than an allocation that wins and takes the citizens down. + #[test] + fn a_suite_larger_than_the_budget_refuses_and_names_the_shortfall() { + let plan = StagingPlan::decide(8_000, 5_000); + assert_eq!( + plan, + StagingPlan::Refuse { + needed: 8_000, + available: 5_000 + } + ); + let why = plan.explain_refusal().expect("a refusal must explain itself"); + assert!( + why.contains("8000") && why.contains("5000"), + "the shortfall must be NAMED, not described as 'insufficient': {why}" + ); + assert!( + why.contains(CONSUMER_ID), + "and say which consumer was budgeted: {why}" + ); + } + + /// what this catches: an off-by-one at the boundary refusing a suite that exactly fits, and + /// an empty suite being refused — an empty pull needs nothing and erroring on it would turn + /// a harmless no-op into a failure. + #[test] + fn a_suite_that_exactly_fits_is_resident_and_an_empty_one_always_is() { + assert_eq!( + StagingPlan::decide(5_000, 5_000), + StagingPlan::Resident { bytes: 5_000 } + ); + assert_eq!( + StagingPlan::decide(0, 0), + StagingPlan::Resident { bytes: 0 } + ); + assert!(StagingPlan::decide(5_000, 5_000).explain_refusal().is_none()); + } + + /// what this catches: THE property that makes staging safe to run beside a video call. + /// Serving weighs a tier-down and Bevy refuses outright during a live call, because their + /// bytes are load-bearing for something a human perceives. Staging's are not — every row is + /// on disk — so it must yield on EVERY reason, including Rebalance. If this ever starts + /// refusing, a benchmark can starve a call, which is the exact failure this consumer exists + /// to make impossible. + #[tokio::test] + async fn staging_yields_on_every_reason_because_its_state_is_on_disk() { + for reason in [ + ReclaimReason::Pressure, + ReclaimReason::Rebalance, + ReclaimReason::Shutdown, + ] { + let area = StagingArea::new(); + area.hold(4_096); + let consumer = StagingConsumer::new(area); + let out = consumer + .reclaim(ReclaimRequest { + kind: ResourceKind::Ram, + target_bytes: 1_024, + deadline_ms: 100, + reason, + }) + .await; + assert_eq!(out.freed_bytes, 4_096, "must yield ALL of it on {reason:?}"); + assert_eq!( + out.status, + crate::resources::ReclaimStatus::Released, + "staging must never refuse a RAM ask ({reason:?}) — its bytes are re-readable" + ); + } + } + + /// what this catches: reporting freed bytes for a kind staging cannot free. An ask for VRAM + /// answered with "released N" would corrupt the governor's ledger into believing device + /// memory came back when only host rows were dropped. + #[tokio::test] + async fn an_ask_for_a_kind_staging_cannot_free_is_refused_not_faked() { + let area = StagingArea::new(); + area.hold(4_096); + let consumer = StagingConsumer::new(area); + let out = consumer + .reclaim(ReclaimRequest { + kind: ResourceKind::Vram, + target_bytes: 1_024, + deadline_ms: 100, + reason: ReclaimReason::Pressure, + }) + .await; + assert_eq!(out.freed_bytes, 0); + assert_eq!(out.status, crate::resources::ReclaimStatus::Refused); + assert!(out.detail.unwrap_or_default().contains("RAM")); + } + + /// what this catches: the footprint lying while idle. Staging holds nothing between fetches, + /// and a consumer that reports phantom bytes makes the governor evict a REAL holder to + /// recover memory nobody has. + #[test] + fn an_idle_staging_area_reports_zero_and_says_it_is_idle() { + let consumer = StagingConsumer::new(StagingArea::new()); + let fp = consumer.footprint(); + assert_eq!(fp.len(), 1); + assert_eq!(fp[0].bytes, 0); + assert_eq!(fp[0].kind, ResourceKind::Ram); + assert!(fp[0].detail.contains("idle"), "{}", fp[0].detail); + } + + /// what this catches: a long staging pass finishing its work AFTER being asked to yield. + /// The ask has to be observable mid-flight, or "released" means "released eventually", + /// which is the `Deferred` contract wearing a `Released` label. + #[test] + fn a_yield_request_is_observable_so_a_pass_in_flight_can_abandon() { + let area = StagingArea::new(); + area.hold(1_000); + assert_eq!(area.yield_requests(), 0); + assert_eq!(area.release(), 1_000); + assert_eq!(area.yield_requests(), 1, "the ask must be visible to the holder"); + assert_eq!(area.held_bytes(), 0); + } +} diff --git a/core/continuum-core/src/cognition/mod.rs b/core/continuum-core/src/cognition/mod.rs index f93b8d7dc..0a35c21de 100644 --- a/core/continuum-core/src/cognition/mod.rs +++ b/core/continuum-core/src/cognition/mod.rs @@ -31,6 +31,7 @@ pub mod act_observe; pub mod adaptive_throughput; pub mod audit; pub mod bench_round; +pub mod bench_staging; pub mod bench_task; pub mod round_readiness; pub mod benchmark; diff --git a/core/continuum-core/src/commands/benchmark.rs b/core/continuum-core/src/commands/benchmark.rs index c98ecd362..9de4691ff 100644 --- a/core/continuum-core/src/commands/benchmark.rs +++ b/core/continuum-core/src/commands/benchmark.rs @@ -3764,11 +3764,40 @@ impl ActionCommand for BenchmarkFetch { let config = p.config.unwrap_or_else(|| def_config.to_string()); let split = p.split.unwrap_or_else(|| def_split.to_string()); + // PLAN BEFORE ALLOCATING (#56). Staging is a governed RAM consumer; it asks the + // governor for the headroom it may plan against and refuses with a named shortfall + // rather than allocating hopefully and letting the allocator arbitrate against a live + // call. The estimate is the catalog's declared task count × a per-row budget — coarse, + // and deliberately so: it is a SIZING input, not a measurement, and the footprint + // reported to the governor after the fetch is the honest number. + // + // ~24 KiB/row is derived from the staged suites on disk (SWE rows carry a problem + // statement + two patches; program rows carry a test body), rounded up so the estimate + // errs toward refusing rather than toward an OOM. + const EST_BYTES_PER_ROW: u64 = 24 * 1024; + let estimated = u64::from(spec.tasks) * EST_BYTES_PER_ROW; + let plan = crate::cognition::bench_staging::plan_against_governor(estimated); + if let Some(why) = plan.explain_refusal() { + crate::probe!( + class = "benchmark.staging.refused", + benchmark = spec.name, + estimated_bytes = estimated, + "staging refused: the governor cannot plan this suite's RAM right now", + ); + return Err(CommandError::Denied(why)); + } + let staging = crate::cognition::bench_staging::staging_area(); + let rows = crate::cognition::swe_bench::fetch_hf_rows(dataset, &config, &split) .await .map_err(CommandError::Internal)?; + // Declare what staging is actually holding, so the governor's footprint is the measured + // number rather than the estimate that sized the plan. Released below, in BOTH the + // normal path and under a reclaim — the two must agree or the ledger drifts. + staging.hold(estimated.min(rows.len() as u64 * EST_BYTES_PER_ROW)); + let denominator_matches = rows.len() as u32 == spec.tasks; crate::probe!( class = "benchmark.suite.staged", @@ -3806,6 +3835,10 @@ impl ActionCommand for BenchmarkFetch { // holding the whole suite alive until the handler returns. let row_count = rows.len(); drop(rows); + // The footprint returns to zero the moment the rows are gone. A consumer that keeps + // reporting bytes it no longer holds makes the governor evict a REAL holder to recover + // memory nobody has — the mirror image of the OOM this whole path exists to prevent. + staging.drop_all(); Ok(BenchmarkFetchResult { benchmark: spec.name.to_string(), dataset: dataset.to_string(), diff --git a/core/continuum-core/src/ipc/mod.rs b/core/continuum-core/src/ipc/mod.rs index a0c286ca2..6e82b341c 100644 --- a/core/continuum-core/src/ipc/mod.rs +++ b/core/continuum-core/src/ipc/mod.rs @@ -1754,6 +1754,18 @@ pub fn start_server( crate::media::perception_registry(), ), )); + // Benchmark staging joins as a peer consumer (#56) — and it is the one that should ALWAYS + // lose. It holds host RAM only while a suite is being fetched and projected, and every row + // it holds is already on disk, so yielding costs a re-read and nothing a human or citizen + // can perceive. Where serving weighs a tier-down and bevy/voice REFUSE during a live call, + // staging releases unconditionally: it is the cheapest victim on the box and must be taken + // first. Until this existed, a benchmark and a live call simply raced `malloc` — which is + // exactly how staging OOMed this machine on 2026-08-19. + resource_daemon.add_consumer(Arc::new( + crate::cognition::bench_staging::StagingConsumer::new( + crate::cognition::bench_staging::staging_area(), + ), + )); runtime.register(Arc::new(VoiceModule::new(voice_state))); // Phase 3: CodeModule (wraps file engines and shell sessions per-persona)