Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/UNVERIFIED.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,8 +471,10 @@ Each has a runbook mode. None has been run by an operator.
| The whole M1 lifecycle against a real forge | **Mode Q** | Cancel, steer, the watchdog, the push gate and the charge ledger have only ever met a WireMock LLM and a local origin |
| Corporate-only bundle → the failure it produces | Mode R §5 | The documented trap (internal forge works, model API fails) is asserted nowhere; it is the mistake an operator will actually make |
| A private-registry pull | Mode S §4 | Nothing pulls from a private registry in any test. `authFor` and the attachment are unit-tested; the *pull* is not |
| **Codex CLI 0.156.1 in the agent image** (2026-09-23) | none yet | Raised from 0.146.0 so the model list includes the gpt-6 models. Re-checked on 0.156.1: every flag the adapter passes, the API-key login, and the shape of the file it writes (`auth_mode=apikey`). NOT re-checked: the `--json` event stream the usage parser reads, and the device sign-in output. Both need a paid run or a real sign-in, and the first of each proves or breaks them |
| **Codex CLI 0.156.1 in the agent image** (2026-09-23) | none yet | Raised from 0.146.0 so the model list includes the gpt-6 models. Re-checked on 0.156.1: every flag the adapter passes, the API-key login, and the shape of the file it writes (`auth_mode=apikey`). The device sign-in output was then proved by a real sign-in (2026-09-25), and one live `codex exec` on a subscription produced the same `--json` event types and the same five usage buckets. Still NOT re-checked: a multi-turn run with tool calls, which exercises the rest of the parser |
| **Runs pinned to the image their model list came from** (M3.5 part M, 2026-09-23) | none yet | Choosing the pin is unit-tested against given daemon answers, and one real-daemon test pins a LOCAL build by its image id. No test pulls a registry image and pins it by its registry digest, and none runs two workers. So "two workers holding different images under one tag run the same one" is argued from the code, not watched. A local-only image is pinned by an id that exists on one daemon only: on a second worker such a run fails to pull — by design, but unobserved |
| **A build paid by a Codex subscription** (M3.5 part F, 2026-09-25) | none yet | Tests prove the emptied refresh token, one seat per account, seat rotation, the unmetered charge lines and the file the adapter writes. One live `codex exec` accepted a file with its refresh token emptied, three hours after sign-in with the id token expired. No item build has run on a seat yet. NOT observed: two builds using one seat at the same moment (seats are shared by the operator's decision of 2026-09-26; whether the vendor accepts two sessions of one account at once is unknown), a run that outlives the access token (10 days — the seat then needs a new sign-in), whether the vendor's CLI ever tries to refresh mid-run with an empty refresh token, and whether `tokens.account_id` is stable across sign-ins of one account |
| **Run-worker log lines are scrubbed of run secrets** (review of PR #178, 2026-09-25) | none yet | `SecretLogFilter` is unit-tested on a JBoss log record and its console configuration is asserted. Not observed: the filter on the JSON console handler of a running worker. And by design it cannot scrub a unit this process did not start: after a worker restart, the watchdog's log lines about an older unit carry that unit's secrets unscrubbed, as `RunFailures` already states for failure details |
| **OIDC sessions actually renew instead of re-authenticating** | **Mode J check 11** (2026-09-10) | The bug it fixes needs a real browser, a real Keycloak and **fifteen elapsed minutes**. No suite here has any of the three: there are zero WebSocket client tests, and nothing observes a token reaching its `exp`. `OidcSessionsAreRenewedTest` asserts the four `application.yml` files *say* renewal is on — it cannot assert Quarkus *does* it |

**Evidence needed.** An operator pass per mode. These are cheap and the runbooks are written.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,19 @@ EXECUTION-LAYER §3.3 with their date and the CLI version:
5. What a usage-limit refusal looks like on the NDJSON stream and in the exit code, so the pool can tell
`rate_limited` from `rejected`, and which usage buckets a subscription run reports.

**Measured on 2026-09-25** from the first real sign-in made through 5.2's screen, with
`@openai/codex@0.156.1`. Field names, types and token lifetimes only; no value was printed:

| Asked | Answer |
|---|---|
| 1. What a ChatGPT-mode `auth.json` holds | `auth_mode` = `chatgpt`; `OPENAI_API_KEY` null; `tokens.id_token` (JWT, **1 hour**); `tokens.access_token` (JWT, **10 days**); `tokens.refresh_token` (opaque); `tokens.account_id`; `last_refresh`. |
| Does `codex login --with-access-token` take that access token? | **No.** It expects an agent-identity JWT and refuses: "agent identity JWT payload is not valid JSON". 5.6's first plan does not work. |
| Does a file with an EMPTY refresh token work? | **Yes.** A missing `refresh_token` field is refused as malformed; an empty one is accepted ("Logged in using ChatGPT"). A live `codex exec` three hours after sign-in — the id token already expired — answered and exited 0. |
| 2. Does a run rewrite the file? | **Not that run:** the file was byte-identical afterwards. A run near the access token's expiry is not yet measured. |
| Usage on a subscription run | The same five buckets as an API-key run: input, cached input, cache write, output, reasoning. |

Questions 3–5 (refresh-token rotation, a quota-free renewal command, the usage-limit refusal) remain open.

**Nothing is built on an unmeasured answer.** Where one is missing, the design takes the option that is
safe when the guess is wrong: no automatic refresh, and a sign-in that is used until the vendor refuses
it.
Expand All @@ -240,6 +253,9 @@ it.

### 5.5 The lease is on agent activity, not on the item

> **Superseded 2026-09-26.** Seats are shared; there is no lease. The paragraphs below record the lease
> as designed and as reviewed; the decision and its reasons are the notes at the end of this section.

A sign-in serves one agent at a time. The lease must therefore start when the agent container starts and
end when that container is gone — not when the item finishes:

Expand All @@ -258,12 +274,31 @@ takes no sign-in and never replays the agent (`WorkRunWorker.java:91`). Cases th
duplicate command delivery, worker death before release, a failed stop, a late release from an old
lease, dispatch failure before the container exists, and two uploads of the same sign-in.

- **Decided 2026-09-26: no lease. Seats are shared like API keys.** A per-run lease was built and
reviewed twice (PR #178), and each round found another race between workers: a queued command
starting after its lease ran out, a failure result freeing the seat of an agent still running, a
worker that does not own a unit reporting it stopped. Closing them needs a durable start permit —
the same weight as the publication permit. The reason for the lease is gone instead: an agent's copy
of the sign-in has its refresh token emptied (§5.6), so two agents cannot refresh one sign-in and log
each other out. The operator chose to share seats. Several builds may use one seat at once; the
subscription's own limits decide how much they get, and seats rotate least recently used first.
**Not verified:** whether the vendor accepts two sessions of one account at the same moment.
- **One seat per account.** `account_ref` is filled from the measured `tokens.account_id`, an enabled
seat's account is unique per harness (V85), and a seat with no known account is never used. Two seats
on one account would share one subscription's limits while looking like two. Signing in again to an
account that has a seat replaces that seat's file — which is also how a seat whose access token
expired is renewed. Seats stored before this are identified at startup, under an advisory lock so two
orchestrators cannot both do it; older duplicates are switched off and cannot be switched back on
beside the newest.

### 5.6 Injection and refresh — the agent never hands a credential back

- The worker passes the credential kind beside `HarnessInvocation.CREDENTIAL`. `CodexAdapter` pipes an
**access token** into `codex login --with-access-token` on stdin, exactly as it pipes an API key into
`--with-api-key` today (F0 measured both). Then it starts `codex exec`. No credential file is written
into the agent container, and nothing reaches argv or the environment.
- **Revised 2026-09-25 (5.3):** `--with-access-token` refuses a ChatGPT access token, so the first plan —
piping the access token into it — cannot work. Instead the worker hands the adapter the stored sign-in
file **with its refresh token emptied**, and `CodexAdapter` writes that file as the agent's
`auth.json` before `codex exec`. It arrives the same way an API key does today: in the environment
under the neutral credential name, never on argv. The access token and id token go in; the refresh token
does not.
- **The refresh token never leaves the orchestrator.** An access token expires on its own; a refresh
token does not, and an agent that reads one holds the sign-in until a person revokes it. Handing the
agent the short-lived half is therefore not a detail of the plumbing — it is the whole difference
Expand Down Expand Up @@ -315,6 +350,10 @@ translation, a retry-time field on the refusal, durable handling of that result,
credential **version**, and the matching change to the arch guard
`spire-arch/src/test/java/dev/codespire/arch/CredentialRefusalHasNoProducerTest.java`.

**Not built in the first part F change (2026-09-25).** A seat whose quota runs out is reported the way
any provider failure is today (collapsed as described above); nothing marks the seat resting. The first live runs show the operator the real quota
message, which is what this path must then classify.

### 5.9 Risks, stated plainly

- **The sign-in file is a person's account credential,** placed in a container that runs ticket text. A
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ record ExecuteRun(String runId, RepoRef repo, String remoteUri,
List<String> protectedPaths, long maxWallClockSeconds,
String scmCredential, String harnessCredential,
boolean existingBranch, String protectedBranch,
String reasoningEffort) implements RunCommand {
String reasoningEffort, boolean harnessSignIn) implements RunCommand {

// Every call site that predates ADR-040 keeps working and keeps the M0 rule — the
// additive treatment the other wire records take. A run already on the bus reads as
Expand All @@ -93,7 +93,7 @@ public ExecuteRun(String runId, RepoRef repo, String remoteUri,
String scmCredential, String harnessCredential) {
this(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt, harness, model,
agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, false, "", null);
harnessCredential, false, "", null, false);
}

/**
Expand All @@ -108,7 +108,22 @@ public ExecuteRun(String runId, RepoRef repo, String remoteUri,
boolean existingBranch, String protectedBranch) {
this(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt, harness, model,
agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, existingBranch, protectedBranch, null);
harnessCredential, existingBranch, protectedBranch, null, false);
}

/**
* Every caller written before subscriptions existed passes an API key (M3.5 part F). A run already
* on the bus decodes with false, which is what every such run carried.
*/
public ExecuteRun(String runId, RepoRef repo, String remoteUri,
String baseBranch, String baseCommit, String branch,
String prompt, String harness, String model, String agentImage,
List<String> protectedPaths, long maxWallClockSeconds,
String scmCredential, String harnessCredential,
boolean existingBranch, String protectedBranch, String reasoningEffort) {
this(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt, harness, model,
agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, existingBranch, protectedBranch, reasoningEffort, false);
}

public ExecuteRun {
Expand Down Expand Up @@ -170,7 +185,7 @@ public boolean pushesToAnExistingBranch() {
public ExecuteRun onExistingBranch(String destination) {
return new ExecuteRun(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt,
harness, model, agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, true, destination, reasoningEffort);
harnessCredential, true, destination, reasoningEffort, harnessSignIn);
}

/**
Expand All @@ -181,7 +196,18 @@ public ExecuteRun onExistingBranch(String destination) {
public ExecuteRun atEffort(String level) {
return new ExecuteRun(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt,
harness, model, agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, existingBranch, protectedBranch, level);
harnessCredential, existingBranch, protectedBranch, level, harnessSignIn);
}

/**
* The same run, where {@code harnessCredential} is a sealed sign-in file rather than an API key
* (M3.5 part F). The worker needs to know which, because the two are handed to the harness
* differently — and guessing from the bytes is how a key ends up written as a file.
*/
public ExecuteRun paidBySignIn() {
return new ExecuteRun(runId, repo, remoteUri, baseBranch, baseCommit, branch, prompt,
harness, model, agentImage, protectedPaths, maxWallClockSeconds, scmCredential,
harnessCredential, existingBranch, protectedBranch, reasoningEffort, true);
}

@Override
Expand All @@ -198,6 +224,7 @@ public String toString() {
+ ", protectedPaths=" + protectedPaths
+ ", maxWallClockSeconds=" + maxWallClockSeconds
+ ", existingBranch=" + existingBranch
+ ", harnessSignIn=" + harnessSignIn
+ ", protectedBranch=" + protectedBranch
+ ", reasoningEffort=" + reasoningEffort
+ ", promptChars=" + prompt.length()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package dev.codespire.contract.work;

/**
* How a build pays for its model calls (M3.5 part F).
*
* <p>Two values, as names rather than a boolean: a third way to pay is easy to imagine, and a flag called
* "subscription" would have to be rewritten to admit one. Every value written before this existed is an
* API key, because until then that was the only way a run could pay.
*/
public final class PayWith {

/** Per token, with a key from the credential pool. */
public static final String API_KEY = "API_KEY";

/** A signed-in seat: no per-token price, the real token counts still recorded. */
public static final String SUBSCRIPTION = "SUBSCRIPTION";

private PayWith() {
}

/** The value, with blank read as {@link #API_KEY}; anything else is refused rather than guessed. */
public static String normalise(String value) {
if (value == null || value.isBlank()) return API_KEY;
String stripped = value.strip();
if (!stripped.equals(API_KEY) && !stripped.equals(SUBSCRIPTION))
throw new IllegalArgumentException("A build pays with " + API_KEY + " or " + SUBSCRIPTION + ", was: " + value);
return stripped;
}
}
Loading
Loading