Repository navigation

feat: make a run measurable, affordable, and honest about failure - #6

Merged

wimaan3 merged 23 commits into

mainfrom

feat/local-scoring

Aug 27, 2026

Merged

feat: make a run measurable, affordable, and honest about failure#6
wimaan3 merged 23 commits into
mainfrom
feat/local-scoring

Commits on Aug 26, 2026
- feat: grade a run locally against the GAIA reference answers
  wimaan3
  committed
  bb18879
- fix: never memoise a failed GAIA dataset listing
  wimaan3
  committed
  9331e7e
- feat: record how many delegation rounds a task actually took
  wimaan3
  committed
  29ef028
- fix: trim the middle of tool output, not the end
  wimaan3
  committed
  a1b0486
- perf: memoise read-only tool results within a task
  wimaan3
  committed
  644421c
- feat: cap what a run may spend, and what the router may generate
  wimaan3
  committed
  b288ce6
- feat: label each run, and make reasoning effort tunable per role
  wimaan3
  committed
  db70036
- refactor: structure the prompts with XML tags, and stop truncating documents
  wimaan3
  committed
  e50344f
- feat: ground the finalizer and router prompts in real reference answers
  wimaan3
  committed
  5525eb3
- feat: run specialists at medium reasoning effort
  wimaan3
  committed
  1a441c0
- fix: four regressions the first five-task run exposed
  wimaan3
  committed
  4cd86c7
Commits on Aug 27, 2026