Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ef971be
fix(orchestrator): wire the realtime LLM-as-judge/fast-mlsirm observa…
seonghobae Sep 2, 2026
3ca8c94
fix(tests): correct test_model_group_mutations_refresh_audit_events
seonghobae Sep 2, 2026
aede81f
test(admin): inherit executable audit-refresh contract repair
seonghobae Sep 2, 2026
a6c7ae8
fix(orchestrator): pin realtime judge to synthesis eligibility, skip …
claude Sep 3, 2026
f0573b2
fix(orchestrator): meter realtime judge spend, drop excluded agents, …
seonghobae Sep 3, 2026
5dc8e87
fix(orchestrator): pin judge and evidence embedding to the request's …
seonghobae Sep 3, 2026
57a0902
fix(orchestrator): count conduct's judge in the gate, scope the evide…
seonghobae Sep 3, 2026
1149e74
fix(orchestrator): pay for rejected spend, never let evidence cost an…
seonghobae Sep 3, 2026
49e8c0a
Merge branch 'main' into fix/orchestrated-completion-judge-observation
seonghobae Sep 3, 2026
d4e003e
fix(orchestrator): meter discarded schema attempts, key evidence on t…
seonghobae Sep 3, 2026
b9b344a
Merge remote-tracking branch's round-4 follow-up into the round-5 fix
seonghobae Sep 3, 2026
629cfc4
Merge protected main into #1032 without discarding either delta
seonghobae Sep 3, 2026
9c191a1
fix(orchestrator): meter terminal schema-failure spend, keep free emb…
seonghobae Sep 3, 2026
d582882
fix(orchestrator): meter transport-failure spend in structured synthe…
seonghobae Sep 3, 2026
3c215c6
fix(orchestrator): meter routing-evidence embedding spend
seonghobae Sep 3, 2026
3a4f7e5
fix(orchestrator): guard embedding-spend persistence, keep in-memory …
seonghobae Sep 3, 2026
e925139
chore: non-force adopt current protected main
seonghobae Sep 3, 2026
5bd37c0
fix(orchestrator): meter repair-call spend on EffortProfileError too
seonghobae Sep 3, 2026
0e5e63f
Merge remote-tracking branch 'origin/codex/nim-evidence-successor' in…
seonghobae Sep 4, 2026
95b1645
Merge remote-tracking branch 'origin/fix/orchestrated-completion-judg…
seonghobae Sep 4, 2026
2661d0c
Merge remote-tracking branch 'origin/codex/nim-evidence-successor' in…
seonghobae Sep 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,23 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
official Run NIM Anywhere terms, restoring fail-closed live benchmark
execution through 2026-10-05 without treating prototype access as production
pricing or licensing evidence.
- Structured synthesis on the main `orchestrator/free` serving path now feeds
the realtime fast-mlsirm judge, so judged quality — not just transport
success — becomes routing evidence on the path that carries most traffic.
This wires an already-implemented, already-grounded measurement pipeline
into a path that was silently not calling it; it introduces no new
technique. The grounding is the existing record in
[`docs/doctoring/measured-routing-evidence.md`](docs/doctoring/measured-routing-evidence.md)
("Real-time judging before returning answers" and "Multi-layer
simple-structure measurement (fast-mlsirm)": Ong et al., 2024; Chen et al.,
2023; Zheng et al., 2023; Jeon et al., 2021), plus Baker (2001) in
[`docs/papers/README.md`](docs/papers/README.md) for the IRT ability
fitting. The call is observation-only: it never branches the served answer,
stays inside the request's own agent eligibility (explicit model pin, free
pool, ZDR policy, file replicas) minus agents this request already proved
unavailable, is skipped once the request's own spend exhausts the operator
budget, and records its own token usage on the run so budget and spend
analytics see it.
- Workflow workers now preserve the caller message array exactly once, while
the added envelope carries only the subtask and Conductor-style prior-step
access list instead of duplicating the task or source attachments.
Expand Down
Loading
Loading