Framework primitive: visible-intent before every non-idempotent effect (the framework owns the visibility barrier, not package Lua)

[loning]

The defect class (now live-proven across legs)

In the fkst-packages github-devloop dogfood, the same defect class has now appeared at multiple legs: package code writes or assumes a state marker, then acts before the marker is provably VISIBLE in the durable read model (GitHub, eventually-consistent).

• ready→implementing (fixed package-side in fkst-packages #1095, behaviorally, by routing through a write-confirmed causal outbox + a verifiable hand_off).
• implementing→...: the implement-attempt/implementing marker write lags or fails under batch concurrency, so implement re-reads "no marker" and re-spawns the implementation codex (fkst-packages #1101).
• Live cascade (2026-06-18): 6 harness issues filed in a batch each hit the implementing re-spawn burst → ~12-18 concurrent codex + a flood of GitHub comment writes → GitHub secondary rate-limit (403) → the implementing state-marker comment writes FAILED → the markers never materialized → the issues are stuck showing ready while actively implementing, and are about to force-terminate to blocked at the 45-min ready budget. The visibility failure both hides the state and self-reinforces the burst.

This is the architecture ChatGPT Pro flagged as the #1 highest-leverage gap: the visibility barrier must be a framework primitive, not re-implemented (and re-broken) per leg in package Lua.

Proposed primitive (the framework owns handoff/visibility/idempotency/redrive/recovery; package expresses only the business decision)

Every non-idempotent effect goes through ONE framework API:

declare_intent(edge, generation, effect_kind, effect_key)
wait_until_intent_visible(intent_id)
perform_or_recover_effect(intent_id, effect_key)   -- probe by effect_key first; perform at-most-once
write_result_marker(intent_id, result)
wait_until_result_visible(result_id)
derive_next_transition_from_visible_result(result_id)

Mechanical invariants:

• No non-idempotent external effect (agent spawn, PR create, review post, merge, label mutation, branch create) may execute unless a VISIBLE, current-generation intent marker exists for that exact effect_key.
• If the effect may already have happened, redrive must recover/probe by effect_key before re-attempting (at-most-once).
• Every next-state transition is derived from a VISIBLE result marker — never from an in-process write result, in-memory flag, or direct raise.

Acceptance tests (the class is gone iff these pass)

• Inject stale reads after intent write: no agent respawn, duplicate PR, duplicate review, duplicate merge, or stuck state.
• Crash after intent marker write, before effect: redrive eventually performs the effect once.
• Crash after effect succeeds, before result marker: redrive recovers the already-performed effect and writes the result marker WITHOUT repeating the effect.
• Two workers racing one issue: at most one effect per (issue, edge, generation, effect_key).
• Batch concurrency + induced rate-limit: effects retry without storming; markers eventually materialize; no force-terminate of healthy actively-working issues.
• Disable the primitive → reproduce the implementing respawn/rate-limit cascade; enable → prove the class is gone.

Origin: ChatGPT Pro roadmap (next-step planning, operator-mandated). Package-side instances: fkst-packages #1095 (ready, done), #1101 (implementing), #1103 (all forward legs behaviorally). This issue is the ENGINE primitive that makes the class structurally impossible and lets package Lua express only business decisions — the operator's stated ideal (framework does the common parts well and keeps them stable).

⟦AI:FKST⟧

[loning]

github-devloop intake decision: enable
Service class: expedite

Reason:
Distributed-systems best practice for non-idempotent effects is durable visible intent plus idempotency/outbox-style recovery; this is an implementation request and the class carrier for cited recurring visibility failures, not a mere instance.

[loning]

github-devloop thinking: consensus started

[loning]

github-devloop convergence round 0 - no three-angle consensus; narrowing

Narrowed question: Frame #133 as an engine-owned transactional-outbox/durable-workflow barrier for non-idempotent effects, reusing existing supervisor journal/saga mechanisms where applicable, with only the minimal visible-intent/result contracts (declare_intent, wait_until_intent_visible, effect-key recovery, write_result_marker, visible-result transition) and conformance tests for stale reads, crashes, races, and rate limits.

Angle stances:

• minimal: approve — Established practice for this class is a durable workflow/state-machine boundary using a transactional outbox, idempotency keys, and saga-style recovery before non-idempotent external effects; given the repeated visibility-before-effect failures recorded here, the minimal coherent fix is one engine-owned declare_intent/visibility/result/redrive primitive rather than another Lua-side patch, with approval scoped to that primitive and its conformance tests, not broader package rewrites.
• structural: approve — Established practice for non-idempotent distributed effects is a durable workflow/saga with a transactional outbox, idempotency keys, write-ahead intent, and recovery from the durable source of truth; this proposal follows that practice by making declare_intent, wait_until_intent_visible, perform_or_recover_effect, write_result_marker, and derive_next_transition_from_visible_result engine-owned contracts rather than package Lua conventions, which structurally improves module boundaries, data flow, and parse/trust contracts while addressing the repeated visibility-before-effect defect
• delete: abstain — Established practice for this defect class is a transactional outbox / durable workflow log with idempotency keys and recovery probes before non-idempotent side effects; the recurrence evidence justifies moving the visibility barrier out of package Lua, but the proposal has not proven that a new broad framework surface with declare_intent, wait_until_intent_visible, perform_or_recover_effect, write_result_marker, and wait_until_result_visible is necessary instead of collapsing this into existing engine-owned deterministic mechanisms already suggested by #56 supervisor journal, `Engine enforcement of package saga declarations: graph-scan validates persistence classes and transition tables #49

⟦AI:FKST⟧

[loning]

github-devloop decision: approve
Three-angle verdicts: minimal=approve structural=approve delete=approve

minimal:
Established practice for this class is transactional outbox/durable workflow execution with idempotency keys and visible durable state before non-idempotent effects; the proposal follows that practice and, from the minimal angle, stays scoped to the smallest class-level engine primitive: "declare_intent", "wait_until_intent_visible", effect-key recovery, "write_result_marker", and visible-result transitions, with conformance tests for the cited stale-read, crash, race, and rate-limit failures rather than expanding into package-specific policy.

structural:
Established practice for this problem class is the transactional outbox / durable workflow pattern with idempotency keys and visible persisted state before non-idempotent side effects; the proposal aligns with that by moving declare_intent, wait_until_intent_visible, effect_key recovery, write_result_marker, and visible-result transitions into the engine, which improves structural boundaries by making the engine the source of truth for effect ordering, recovery, and parseable trust contracts instead of scattering visibility logic through package Lua.

delete:
Established practice for non-idempotent distributed effects is a transactional outbox/durable workflow barrier with idempotency keys and recovery-before-retry; the proposal does not justify a novel package-level abstraction, but it does narrow the surface to engine-owned declare_intent, wait_until_intent_visible, effect-key recovery, write_result_marker, and visible-result transition while reusing existing supervisor journal/saga machinery, and the manifest shows this is a recurring class failure rather than a one-off, so the delete angle can approve only this collapsed framework primitive and not a broader Lua-facing effect framework.

⟦AI:FKST⟧

[loning]

CORRECTION to the motivating example

The "batch concurrency → GitHub secondary rate-limit (403) → implementing marker writes failed" cascade in the original description was a MISREAD on my side (the "403" was a nanosecond timestamp in a log filename, and the implementing state marker is simply written at implement-codex COMPLETION, not at start, so issues legitimately show ready during the codex run). There was no rate-limit and no dropped write.

The underlying architectural point still stands and is the real motivation: the same defect CLASS — a next action (re-spawn, transition, effect) depending on a marker that is not yet provably visible in the durable read model — is real and live in a milder form (fkst-packages #1101: the implement codex re-spawns during the implement-attempt marker's visibility lag). The visible-intent / recoverable-effect framework primitive is the right general fix so package code cannot re-implement (and re-break) the visibility barrier per leg. Please disregard the rate-limit specifics; keep the primitive + invariants + acceptance tests.

⟦AI:FKST⟧

[loning]

github-devloop implementation started

Worktree: /Users/auric/.fkst-dogfood/dogfood-rt-substrate.1781804686/worktrees/devloop-ChronoAIProject-fkst-substrate-133-3957080407
Branch: devloop/issue/ChronoAIProject/fkst-substrate/133/ready-consensus-github-devloop-issue-ChronoAIProject-fkst-substrate-133-intake-1652784515-loop-1-3957080407
Head: 142daf9
Base branch: integration
Base head: 64d2857