Improvement proposals from a 3-agent MADP self-review dialogue (Hermes ×3) #5

Origin

Output of a real bounded 3-agent MADP dialogue (madp-self-improve-three-hermes-20260820, three Hermes profiles: proposer gpt-5.5 / challenger deepseek-v4-flash / synthesizer kimi-k3, 3 rounds, all turns evidence-verified, final status READY_FOR_OWNER, validation ok: true). The question put to the dialogue: how should MADP itself be improved?

Each item below survived both a proposal round and an adversarial challenge round, and carries a smallest implementation step + a pass/fail canary.

Proposals (priority order)

1. Hermes message-boundary check. README promises the "final active message" from state.db, but nothing bounds which message is the turn. Smallest step: adapter records (session_id, last message_id) pre-launch; post-launch accept only exactly one new final active assistant message, none later; store before/after ids in evidence. Needs explicit recovery/multi-message exception rules. Canary: fixture with a later assistant message → turn rejected, non-zero exit, no ledger commit.

2. Per-turn adapter CLI version in evidence. README says adapters "depend on the exact installed CLI versions", yet accepted-turn evidence records provider/model/session only. Smallest step: probe --version at launch, record verbatim + hash in turn evidence. Canary: same turn under two stubbed CLI versions → evidence records differ; both validate.

3. Engine-detected adapter capability manifest. Fail before execution when the protocol requires evidence the adapter cannot provide. Hard constraint from review: the manifest must be engine-probed from the CLI, never adapter self-report, hashed into evidence; "unsupported features" needs a controlled vocabulary. Canary: protocol requiring a field the probed adapter lacks → launch refused before any runtime spawn.

4. Standard canary command. Operationalizes the README's own "harmless live smoke and bounded canary" advice. madp canary --adapter A --dialogue DIR --no-push: tiny local protocol, one turn through the real acceptance path, pass/fail report with adapter CLI versions. Canary: run against shipped fakes → exit 0; sabotage one evidence field → non-zero, no commit, no push.

5. evidence_version binding per turn. Turn evidence carries evidence_version: 1 but nothing enforces it against the frozen definition. Smallest step: validator compares turn evidence_version against the version range declared in protocol.json; mismatch → fail-closed. Canary: evidence_version 2 against a definition pinned to 1 → rejected.

6. Run-attempt receipt (redesigned). Crash forensics are real, but a committed per-attempt receipt breaks one-commit-per-accepted-turn. Redesign: write run_attempt.json (turn id, actor, argv hash redacted, pid, timestamps, exit, cleanup, outcome) to the recovery namespace only, never the ledger. Canary: kill a launched process mid-turn → receipt in recovery namespace, ledger unchanged, validate fails closed.

7. Evidence index (redesigned). A committed index would become a stale-able second source of truth ("validator result" can be contradicted by a later validator version). Redesign: on-demand madp report DIR deriving the index from accepted commits; never committed, never read by acceptance logic. Canary: tamper one referenced evidence hash → report flags mismatch while validate still re-checks raw evidence independently.

Explicitly dropped

Committed evidence_index.json; any committed per-attempt receipt; runner-written artifacts readable by the acceptance gate.

Not addressed (flagged as open gaps, no design produced)

• Cost controls — "bounded" currently means turn-count only; no spend cap, no estimate-before-launch, no token cost in evidence.
• Cross-host concurrency — lock semantics beyond one shared filesystem (README already places this outside the engine).

Caveats

All items are argument-level only — no implementation was attempted inside the dialogue. The two concrete Hermes-adapter bugs found while running this dialogue are fixed in PR #4.