Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .asdd.example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,9 @@ protected_paths:
# style:
# spelling: british
# banned_chars: ["–", "—"] # en dash, em dash - checked on ADDED lines only
# # Your project's own test/lint/type command. The asdd-preflight.yml workflow RUNS this on every PR as
# # a deterministic, blocking gate (distinct from the model test agent), so a regression the review missed
# # is caught by the real suite. Make it a required check. Unset => the gate is an opt-in no-op.
# preflight: "ruff check . && ruff format --check . && mypy pkg && pytest -n 4"
# exemplars: # merged PRs that exemplify house style, cheaper than prose
# - "https://github.com/OWNER/REPO/pull/123"
Expand Down
9 changes: 9 additions & 0 deletions .github/asdd/audit-export.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,15 @@ yaml_in() {
}

SINK="$(yaml_in sink)"; SINK="${SINK:-none}"
# The same-repo refusal below (and the commit message) need the governed repo's name. CI sets
# GITHUB_REPOSITORY; a host produce or council run does NOT, which would SILENTLY SKIP the same-repo
# refusal. Derive it from the git remote when unset, so the refusal is enforced everywhere - CI and host -
# with no per-caller wiring (the same anti-miss philosophy as exporting from the wrapper).
if [ -z "${GITHUB_REPOSITORY:-}" ]; then
GITHUB_REPOSITORY="$(git -C "$ROOT" config --get remote.origin.url 2>/dev/null \
| sed -E 's#(git@|https?://)[^/:]+[/:]##; s#\.git$##')"
fi

SINK_REPO="$(yaml_in sink_repo)"
SINK_CMD="$(yaml_in sink_command)"

Expand Down
10 changes: 10 additions & 0 deletions .github/asdd/audit-export.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@ out="$(run_with_cfg 'audit:
echo "$out" | grep -q "repository being governed" && echo "ok: refuses the governed repo as sink" \
|| { echo "FAIL: should refuse the governed repo (got: $out)"; fail=1; }

# 2b. On a HOST run (GITHUB_REPOSITORY unset), the governed repo is derived from the git remote, so the
# same-repo refusal holds off-CI too instead of being silently skipped.
d="$TMP/hostrepo"; rm -rf "$d"; mkdir -p "$d/.github/asdd" "$d/cli"
cp "$EX" "$d/.github/asdd/audit-export.sh"; cp "$ROOT/cli/audit.py" "$d/cli/audit.py"
( cd "$d" && git init -q && git remote add origin https://github.com/acme/project.git ) >/dev/null 2>&1
printf 'audit:\n sink: repo\n sink_repo: acme/project\n' > "$d/.asdd.yml"
out="$(env -u GITHUB_REPOSITORY bash "$d/.github/asdd/audit-export.sh" "$L" 2>&1)"
echo "$out" | grep -q "repository being governed" && echo "ok: derives the governed repo from the git remote on a host run" \
|| { echo "FAIL: host-run same-repo refusal not enforced (got: $out)"; fail=1; }

# 3. Unverifiable visibility fails closed (no token -> cannot prove it is private).
out="$(run_with_cfg 'audit:
sink: repo
Expand Down
21 changes: 19 additions & 2 deletions .github/asdd/intake-check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -202,20 +202,37 @@ if [ "$overridden" = "true" ] && [ "${#problems[@]}" -gt 0 ]; then
problems+=("Owner override in effect: the above are advisory only for this PR (owner-override label).")
fi

# Lane hygiene (advisory, never a failure): a `chore` PR (spec-exempt) that ADDS or edits a spec is
# self-contradictory - a change that authors a spec is usually a feature or fix, not a chore, so the lane
# is probably mislabelled. WARN so it surfaces; chore still passes.
warnings=()
if [ "$is_chore" = "true" ] && [ -f "$WORKDIR/changed.txt" ] && \
SPEC_RE="^(${spec_re})$" awk -F'\t' 'BEGIN{re=ENVIRON["SPEC_RE"]} $1 ~ /^[AMR]/ && $NF ~ re {f=1} END{exit f?0:1}' \
"$WORKDIR/changed.txt"; then
warnings+=("Lane is 'chore' (spec-exempt) but this change adds or edits a spec. A change that authors a spec is usually a feature or fix, not a chore; check the lane. Chore still passes intake.")
fi

# Emit intake JSON.
if [ "${#problems[@]}" -gt 0 ]; then
probs_json="$(printf '%s\n' "${problems[@]}" | jq -R . | jq -s 'map(select(length>0))')"
else
probs_json='[]'
fi
if [ "${#warnings[@]}" -gt 0 ]; then
warns_json="$(printf '%s\n' "${warnings[@]}" | jq -R . | jq -s 'map(select(length>0))')"
else
warns_json='[]'
fi
jq -n \
--argjson pr "${pr_number}" --arg head "${head_sha}" \
--argjson disc "$disclosed" --argjson sign "$signed_off" --argjson lane "$laned" \
--argjson flood "$flood_ok" --argjson spec "$spec_ok" --argjson conv "$conv_ok" --argjson ovr "$overridden" --argjson probs "$probs_json" \
--argjson flood "$flood_ok" --argjson spec "$spec_ok" --argjson conv "$conv_ok" --argjson ovr "$overridden" \
--argjson probs "$probs_json" --argjson warns "$warns_json" \
'{schema:"asdd/intake/v0.1", pr_number:$pr, head_sha:$head,
disclosed:$disc, signed_off:$sign, laned:$lane, flood_ok:$flood, spec_ok:$spec, conventions_ok:$conv, override:$ovr,
passed:($ovr or ($disc and $sign and $lane and $flood and $spec and $conv)),
problems:$probs}' > "$OUT"
problems:$probs, warnings:$warns}' > "$OUT"

[ "${#warnings[@]}" -gt 0 ] && printf 'intake WARNING: %s\n' "${warnings[@]}" >&2
echo "intake: disclosed=$disclosed signed_off=$signed_off ($signed/$total signed) laned=$laned flood_ok=$flood_ok spec_ok=$spec_ok conventions_ok=$conv_ok override=$overridden"
[ -s "$OUT" ] || { echo "intake-check: no output" >&2; exit 1; }
24 changes: 24 additions & 0 deletions .github/asdd/intake-check.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,30 @@ case_run "a renamed-to spec counts (R status, new path is \$NF)" \
case_run "a chore PR skips the spec requirement" \
asdd-default.yml 'chore' "$DISC" "$(printf 'M\tsrc/a.py\n')" true true

# Lane hygiene: a `chore` PR is spec-exempt, so authoring a spec inside it is self-contradictory (the
# change is really a feature or fix). The gate still PASSES chore (spec is not required), but it must
# surface a non-failing warning so the mislabelled lane is visible.
fw="$TMP/w"; rm -rf "$fw"; mkdir -p "$fw"
printf '%s' "$DISC" > "$fw/body.md"; printf 'chore' > "$fw/labels.txt"
printf 'feat: a change\n\nSigned-off-by: A Dev <a@example.com>\0' > "$fw/commits.txt"
printf 'A\tdocs/specs/new.md\n' > "$fw/changed.txt"
printf 'pr_number=1\nhead_sha=abc123\nrequire_spec=true\n' > "$fw/meta.env"
( cd "$TREE" && ASDD_CONFIG="asdd-default.yml" bash "$GATE" "$fw" "$fw/out.json" >/dev/null 2>&1 )
if [ "$(jq -r '.passed' "$fw/out.json")" = "true" ] \
&& jq -e '.warnings | map(select(startswith("Lane is "))) | length > 0' "$fw/out.json" >/dev/null; then
echo " ok a chore PR that adds a spec passes but warns the lane is mislabelled"
else
echo " FAIL chore+spec-delta: expected pass with a lane-hygiene warning"; fail=1
fi
# A chore with NO spec delta must carry no such warning (no false positive).
printf 'M\tsrc/a.py\n' > "$fw/changed.txt"
( cd "$TREE" && ASDD_CONFIG="asdd-default.yml" bash "$GATE" "$fw" "$fw/out.json" >/dev/null 2>&1 )
if jq -e '.warnings | map(select(startswith("Lane is "))) | length == 0' "$fw/out.json" >/dev/null; then
echo " ok a chore with no spec delta carries no lane-hygiene warning"
else
echo " FAIL chore-without-spec should not warn"; fail=1
fi

# --- a foreign layout: OpenSpec -----------------------------------------------------------------------
case_run "OpenSpec: adding a proposal satisfies the gate" \
asdd-openspec.yml "$OK_LABELS" "$DISC" "$(printf 'A\topenspec/changes/add-auth/proposal.md\n')" true true
Expand Down
44 changes: 44 additions & 0 deletions .github/asdd/operate/runner-trail.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# The operate runners (test.sh, docsync.sh) leave an audit trail on EVERY exit path, and their export half
# is INERT without a sink credential: record locally, do NOT attempt a push, and do NOT fail the run. A stub
# audit-export.sh here proves invocation vs non-invocation directly, so a regression that drops the
# AUDIT_SINK_TOKEN guard (turning "record on every exit" into "try to export on every exit") is caught.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)" # .github/asdd/operate
ROOT="$(cd "$HERE/../../.." && pwd)"
fail=0
T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT

# A throwaway repo so the runner resolves a STUB audit-export.sh and the real deps it needs.
D="$T/repo"; mkdir -p "$D/cli" "$D/.github/asdd/operate" "$D/recipes"
cp "$ROOT/cli/audit.py" "$ROOT/cli/operate-guard.py" "$D/cli/"
cp "$ROOT/recipes/test-runner.yaml" "$D/recipes/" 2>/dev/null || true
printf '#!/usr/bin/env bash\necho called >> "%s/exp.log"\n' "$T" > "$D/.github/asdd/audit-export.sh"
chmod +x "$D/.github/asdd/audit-export.sh"

for pair in "test.sh:test-runner" "docsync.sh:documentation"; do
runner="${pair%%:*}"; role="${pair##*:}"
cp "$ROOT/cli/templates/operate/$runner" "$D/.github/asdd/operate/$runner"
[ "$runner" = "docsync.sh" ] && cp "$ROOT/recipes/documentation.yaml" "$D/recipes/" 2>/dev/null || true
rm -f "$T/exp.log"

# 1. No sink credential: must record locally, exit 0, and NOT invoke the exporter.
env -u ASDD_MODEL_URL -u ASDD_RUNTIME_TOKEN -u AUDIT_SINK_TOKEN ASDD_ACTIVITY_LOG="$T/a.jsonl" \
bash "$D/.github/asdd/operate/$runner" cx "$T/o.md" >/dev/null 2>&1; rc=$?
if [ "$rc" = 0 ] && [ -s "$T/a.jsonl" ] && grep -q "\"role\":\"$role\"" "$T/a.jsonl" && [ ! -f "$T/exp.log" ]; then
echo " ok $runner: no-token run records locally, no export, no failure"
else
echo " FAIL $runner: no-token path (rc=$rc record=$([ -s "$T/a.jsonl" ] && echo y) export=$([ -f "$T/exp.log" ] && echo attempted))"; fail=1
fi

# 2. With a sink credential (trusted post-merge context): the export half fires.
rm -f "$T/exp.log" "$T/b.jsonl"
env -u ASDD_MODEL_URL -u ASDD_RUNTIME_TOKEN AUDIT_SINK_TOKEN=x ASDD_ACTIVITY_LOG="$T/b.jsonl" \
bash "$D/.github/asdd/operate/$runner" cx "$T/o2.md" >/dev/null 2>&1
[ -f "$T/exp.log" ] && echo " ok $runner: a sink credential triggers the export" \
|| { echo " FAIL $runner: export not attempted with a token"; fail=1; }
done

echo
[ "$fail" = 0 ] && echo "runner-trail self-test: PASS" || echo "runner-trail self-test: FAIL"
exit "$fail"
42 changes: 42 additions & 0 deletions .github/asdd/preflight.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# ASDD - the deterministic preflight gate. Runs the adopter's OWN test/lint/type command
# (conventions.preflight in .asdd.yml) as a real, blocking check, so a regression the model review missed
# is still caught by the actual suite. This is DISTINCT from the model test-runner agent (asdd-test.yml):
# that agent judges with a model post-merge; this runs the deterministic suite and its exit status IS the
# gate. A spec-and-test framework that never runs the tests deterministically is the gap this closes.
#
# The command is the adopter's own trusted config (not untrusted PR text), so running it in a shell is the
# intended interface (they write `ruff ... && pytest ...`). The workflow that calls this holds no secrets,
# so executing the change's tests on a PR cannot exfiltrate; a fork PR still needs the maintainer's
# fork-workflow approval before it runs at all.
#
# Usage: preflight.sh (reads .asdd.yml, runs conventions.preflight, exits with its status)
# Env: ASDD_CONFIG overrides the config path.
set -uo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
CFG="${ASDD_CONFIG:-$ROOT/.asdd.yml}"

# Read conventions.preflight (nested one level under conventions:), no YAML dependency. The value is a
# quoted scalar; a `#` inside it is part of the command, so inline-comment stripping is deliberately NOT
# applied here (unlike the lane/path list readers, whose tokens never contain #).
cmd="$(awk '
/^conventions:/ { inc=1; next }
inc && /^[A-Za-z]/ { inc=0 }
inc && /^[[:space:]]+preflight:[[:space:]]*/ {
line=$0; sub(/^[[:space:]]+preflight:[[:space:]]*/, "", line); print line; exit
}' "$CFG" 2>/dev/null)"
# Strip ONE matching pair of outer quotes (a double-quoted value may legitimately end in a single quote,
# e.g. "python3 -c 'x'", so a blanket strip of both would eat the command's own inner quote).
case "$cmd" in
\"*\") cmd="${cmd#\"}"; cmd="${cmd%\"}" ;;
\'*\') cmd="${cmd#\'}"; cmd="${cmd%\'}" ;;
esac

if [ -z "$cmd" ]; then
echo "asdd preflight: no conventions.preflight configured; nothing to run. Declare your project's own"
echo "test/lint/type command (e.g. \"ruff check . && pytest -n 4\") under conventions: to enable the gate."
exit 0
fi

echo "asdd preflight: running the project's own suite -> $cmd"
bash -c "$cmd" # the command's exit status IS the gate: a non-zero here fails the check and blocks merge.
40 changes: 40 additions & 0 deletions .github/asdd/preflight.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Self-test for the deterministic preflight gate (.github/asdd/preflight.sh): it runs the adopter's own
# conventions.preflight command and its exit status IS the gate - a passing suite passes, a failing suite
# (a deliberately broken test) fails the check, and an unconfigured preflight is a no-op that passes.
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
PF="$HERE/preflight.sh"
T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT
fail=0
ok() { echo " ok $1"; }
bad() { echo " FAIL $1"; fail=1; }

run() { ASDD_CONFIG="$1" bash "$PF" >/dev/null 2>&1; echo "$?"; }

# 1. A passing suite passes the gate.
printf 'conventions:\n preflight: "exit 0"\n' > "$T/pass.yml"
[ "$(run "$T/pass.yml")" = 0 ] && ok "a passing preflight passes the gate" || bad "passing preflight"

# 2. A failing suite fails the gate (a regression is caught deterministically).
printf 'conventions:\n preflight: "exit 7"\n' > "$T/fail.yml"
[ "$(run "$T/fail.yml")" != 0 ] && ok "a failing preflight fails the gate" || bad "failing preflight did not fail"

# 3. A real command: a passing then a deliberately broken python check.
printf "conventions:\n preflight: \"python3 -c 'raise SystemExit(0)'\"\n" > "$T/realok.yml"
[ "$(run "$T/realok.yml")" = 0 ] && ok "a real passing check passes" || bad "real passing check"
printf "conventions:\n preflight: \"python3 -c 'raise SystemExit(1)'\"\n" > "$T/realbad.yml"
[ "$(run "$T/realbad.yml")" != 0 ] && ok "a real broken check fails the gate" || bad "real broken check did not fail"

# 4. No preflight configured: a no-op that passes and says so (opt-in).
printf 'lanes:\n - feature\n' > "$T/none.yml"
out="$(ASDD_CONFIG="$T/none.yml" bash "$PF" 2>&1)"; rc=$?
if [ "$rc" = 0 ] && printf '%s' "$out" | grep -q "nothing to run"; then
ok "unconfigured preflight is an opt-in no-op"
else
bad "unconfigured preflight (rc=$rc)"
fi

echo
[ "$fail" = 0 ] && echo "preflight self-test: PASS" || echo "preflight self-test: FAIL"
exit "$fail"
32 changes: 32 additions & 0 deletions .github/workflows/asdd-preflight.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# ASDD preflight - the deterministic test gate.
#
# Runs the adopter's OWN test/lint/type command (conventions.preflight in .asdd.yml) on every PR, so a
# regression the model review missed is still caught by the real suite. This is DISTINCT from the model
# test-runner agent (asdd-test.yml, which judges post-merge with a model): here the deterministic suite's
# exit status IS the gate. Make it a required check in branch protection to block a red suite from merging.
#
# It holds NO secrets (contents: read only), so executing the change's own tests on a PR cannot exfiltrate;
# a fork PR still needs the maintainer's fork-workflow approval before it runs at all. It is a no-op that
# passes until conventions.preflight is set, so installing it is safe before a project wires its suite.
name: ASDD preflight

on:
pull_request:
types: [opened, synchronize, reopened]

permissions:
contents: read

concurrency:
group: asdd-preflight-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
preflight:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
steps:
- name: Check out the change (the PR merged into its base)
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run the project's deterministic preflight
run: bash .github/asdd/preflight.sh
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,41 @@ draft, so pin a conformance claim to a commit or date.
## [Unreleased]

### Added
- **The audit trail is complete: every agent's records reach the sink, not only the reviewer's.** Before,
only the review exported, so a deployment that turned on `audit.sink` captured the reviewer's decisions
but lost the produce side (the developer/council "coding"), the test and documentation agents, and the
operator agents. Now every path records through one route and exports from a credential-safe point (the
produce wrapper, the runners, the council), the untrusted PR review keeps its record/publish split, and
`doctor` runs a **dynamic completeness check** that enumerates every record-writing path and warns on any
with no export route, so a new agent added later cannot silently drop its trail. The check also covers the
CI workflow surface with a stricter rule: because the sink credential lives in the job, a workflow that
runs a recorder which does not itself export (for example `dev-council.py` wired straight into a workflow
instead of through the exporting `dev-council.sh`) must export in that same job, and cannot lean on a
runner it bypassed. `audit-export.sh` also derives the governed repo from the git remote when
`GITHUB_REPOSITORY` is unset, so its "never export into the repo you govern" refusal holds on host runs,
not only in CI.
- **A deterministic preflight gate runs the project's real test suite.** `conventions.preflight` (the
adopter's own `ruff`/`pytest`/`mypy` command) was declared and surfaced to the agents but never actually
run. The new `asdd-preflight.yml` workflow runs it on every PR through `.github/asdd/preflight.sh`, so a
regression the model review missed is still caught by the real suite; its exit status is the gate. It is
distinct from the model test-runner agent, holds no secrets, and is an opt-in no-op until the command is
set. A spec-and-test framework should run the tests, not only reason about them.
- **`doctor` warns when the git identity is unset.** A contributor whose `user.name` / `user.email` are
not set produces commits that cannot be signed off or attributed, so intake rejects them after the work
is done. The preflight now flags this up front, as a warning, so it is fixed before the first commit.
- **`connect-check --json` for tooling and CI.** The per-role connected/dry-run status is now available as
machine-readable JSON with the same accounting and exit code as the human output, so a setup script or
pipeline can gate on it without scraping text.
- **Intake warns when a `chore` change authors a spec.** The `chore` lane is spec-exempt, so a change that
adds or edits a spec while labelled `chore` is almost certainly a mislabelled feature or fix. Intake now
surfaces this as a non-failing warning (the change still passes) so the lane can be corrected.
- **Editor pointers for a bring-your-own developer.** `init` now writes a thin pointer for the common
coding assistants (`CLAUDE.md` for Claude Code and the Claude app, `.cursor/rules/asdd.mdc` for Cursor;
Codex and other AGENTS.md-convention tools read `AGENTS.md` directly), so a contributor's assistant loads
the contribution constitution with no manual setup. Each pointer references `AGENTS.md` and is skipped if
the file already exists, so an existing rule file is never overwritten. A new guide,
[bring your own developer](docs/guides/bring-your-own-developer.md), covers the non-engineer spec path
and that the operate agents run on open-source Goose.
- **`doctor` and `setup` warn when the reviewer is a heavy reasoning model.** A reasoning model reasons at
length and can exceed a hosted inference window on a real code diff, so the review times out and posts no
lenses while trivial or docs-only diffs still pass and look fine. The preflight and the setup wizard now
Expand Down
Loading
Loading