Skip to content

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Governed writes in runx: make the refusal as provable as the write

runx is an open-source runtime for portable skills — a skill is a package (X.yaml + SKILL.md + step code) that any agent can install and run, and every run seals a receipt: a signed, content-addressed record of what ran, with what inputs, producing what step outputs. Source: https://github.com/runxhq/runx.

That receipt is the interesting part, and it changes how you should build a skill that writes somewhere — a CRM field, a ticket, a changelog, an outbox. Most such skills are built so the write is the happy path and the refusal is a silent return. On runx you can do better, because the receipt can prove a refusal happened just as precisely as it proves a write.

This repo is a short walkthrough of the pattern plus a working example you can run in about a minute: example/ — a release-note-gate skill that publishes a changelog line only when it can stand behind it.

The pattern

1. The write is a separate step, guarded by the decision.

- id: publish
  when:
    field: decide.publish
    equals: true

When the decision says no, the writer does not take a different branch — it never runs, and the receipt's step_outputs contains only decide. A reviewer who was not there can check that: absence of the step is the evidence.

2. The decision carries a confidence, not a vibe.

Text is not facts. steps/decide.mjs scores a note and subtracts named signals, then reports which ones fired:

signal penalty fires on
hedge 0.40 "I think", "might", "maybe", "not sure"
unfinished 0.35 "WIP", "TODO", "TBD", "draft"
conditional 0.25 "if", "once", "pending"
question 0.25 the line ends in a question mark

Below min_confidence (default 0.75) the note is held. "I think this maybe fixes the retry bug?" is not a changelog entry, and the skill says so instead of publishing it.

3. What gets written is bound to what it was read from.

Each decision carries the source text and a sha256 of it, and the published entry carries the same digest. Later, with only the receipt and the source, you can re-derive the published line rather than trusting the skill.

4. Confidence and human approval are two different gates.

approval_mode=required withholds a 0.9 note until a human passes approved=true. Clearing the first gate does not clear the second — which is what you want for anything customer-visible.

Running it

git clone https://github.com/automerchlab/runx-governed-writes
cd runx-governed-writes/example

runx harness .

Real output (runx-cli 0.8.2, Linux/WSL):

status: passed   case_count: 4   assertion_error_count: 0
  asserted_note_published
  hedged_note_held
  approval_required_withholds
  approved_note_published

Three runs, and what actually comes back:

runx skill . --json -i note='Added a --json flag to the export command.'
# publish=true  disposition=publish  confidence=0.9
# publish_result: status=published, entry_ref=default:1

runx skill . --json -i note='I think this maybe fixes the retry bug?'
# publish=false disposition=held_low_confidence  signals=[hedge, question]
# no publish_result in the result at all - the step never ran

runx skill . --json -i note='Removed the deprecated v1 endpoints.' -i approval_mode=required
# publish=false disposition=held_awaiting_approval  confidence=0.9
# confident, and still not written

The changelog after those three runs contains exactly one line — the first one:

{"version":1,"note":"Added a --json flag to the export command.","note_digest":"sha256:6b1e66ec…","confidence":0.9}

Why bother

Two reasons, both practical.

A skill that writes confidently wrong values is worse than a skill that writes nothing: the wrong value looks like a fact to everyone downstream. A gate turns "the model was unsure" from an invisible internal state into a recorded, reviewable outcome.

And on runx specifically, refusals become portable evidence. The harness case hedged_note_held asserts the held disposition, and the sealed receipt from a real run shows the same thing. Anyone installing the skill can verify the refusal path works without taking your word for it — which is exactly the property that makes a skill safe to hand to an agent.

Notes on the example's honesty

  • The transport is a local JSONL file bundled with the example, not a governed data store. It stands in for whatever real system you would write to; the part worth copying is above the transport.
  • The scoring numbers are calibration constants chosen so hedged and unfinished text falls below the default threshold. They are not probabilities and nothing here is learned.
  • Extraction is deterministic regex, never an LLM, so every run seals reproducibly — which is what makes the harness cases meaningful.

Links

Built and verified by @automerchlab while building governed-write skills on runx. Issues and corrections welcome.

About

Walkthrough + runnable runx example: make a skill's refusal as provable as its write (confidence gate, evidence binding, human approval).

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages