From 19d1709f46d0c4edde7b0c08d775328fa52cc36b Mon Sep 17 00:00:00 2001 From: jennyf19 Date: Mon, 6 Jul 2026 10:08:40 -0700 Subject: [PATCH] Enhance README with execution and outcome signal insights Expanded the README to explain the importance of execution and outcome signals, their implications for trust in AI systems, and related industry directions. --- agent-signals/README.md | 69 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/agent-signals/README.md b/agent-signals/README.md index 02a3f89..727bac9 100644 --- a/agent-signals/README.md +++ b/agent-signals/README.md @@ -284,6 +284,75 @@ remediation sessions: --- +## Two Capture Paths, Two Reliability Profiles + +Here's something we learned the hard way, and it shaped the whole protocol: +**you will lose execution signals, and that's the reason outcome signals exist.** + +An execution signal is a self-report. It only gets captured if the agent is +still running, the session is still open, and the emit step actually fires +before the developer closes the terminal. In practice, a lot of them never +make it out. In our own production system, we captured thousands of execution +signals — but far fewer than the number of tasks the agents actually completed. +The rest were lost to closed sessions and interrupted runs. + +That loss is not a bug you can fully engineer away. It's the nature of +self-assessment: it depends on the speaker being present and willing to speak. + +The outcome signal is different. It doesn't depend on the agent remembering +anything. The work product — the PR, the commit, the change — is *durable*. +It exists in the system whether or not the CLI stayed open. So an independent +evaluator can grade it later, on its own schedule, with no dependency on the +original session. + +This is the same distinction that shows up everywhere in trustworthy AI: + +- **Honesty** is a property of the *speaker*. It's unverifiable from outside, + and — as we learned — it can be lost entirely if the speaker never gets to + report. +- **Transparency** is a property of the *system*. The artifact is there, + checkable by anyone with access, regardless of what the agent said or + whether it said anything at all. + +You need both signal types precisely *because* self-report is fragile. The +execution signal tells you what the agent believed, when you can get it. The +outcome signal is the durable ground truth you can always recover. The gap +between them is the calibration measurement — and it's trustworthy *because* +one leg of it doesn't depend on the agent's cooperation. + +**Design implication:** don't build your trust story on execution signals +alone. Anchor it to something durable. Treat execution signals as valuable- +but-lossy self-report, and treat outcome signals as the verifiable backbone. +A system that can only self-report is a system you're trusting on faith. A +system where the outcome is independently checkable is a system you can +actually verify — even on the runs where the agent never got to speak. + +## Related Industry Direction + +Frontier model providers are shipping architectures that separate capability +from verification. + +Anthropic's Claude Fable 5 and Mythos 5 (2026) expose the same base model +through two tiers: a generally-available tier (Fable 5) with external safety +classifiers that intercept high-risk requests and route them to a fallback +model, and a restricted tier (Mythos 5) without those classifiers, available +only to vetted partners. The two tiers differ not in capability but in the +safeguards and vetting applied around access. + +We read this as consistent with the premise behind Agent Signals: capability +alone isn't the thing to manage — verification is. Access-tiering addresses +*who* operates with which safeguards. Signal calibration addresses a different, +complementary question: *whether an agent's self-reported reliability holds up +against independent evaluation, over time.* The calibration gap between what an +agent claims about its own work and what an independent evaluation confirms is +a number you can track run over run. + +The through-line with the research cited earlier — OpenAI on honest +self-report, Anthropic on scalable oversight, Microsoft on collaboration — is +that systems designed for verification tend to outperform systems that ask you +to take reliability on faith. Agent Signals is a measurement layer for that +kind of verification. + ## Getting Started ### 1. Add a signals directory to your project