Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 22 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,17 @@ The design is fully specified in `docs/` — **treat those files as the source o
| `docs/RESEARCH.md` | Market landscape + the PR-Agent code evaluation that justified greenfield |
| `docs/ROADMAP.md` | Phases P0–P4 with exit criteria |
| `docs/HISTORY.md` | The per-milestone delivery log: what shipped, what each review round found, the traps each one paid for. **Append new milestones there**, then rewrite the Status snapshot below |
| `docs/factory/` | **M0–M3 implemented (PRs #95/#96/#119/#153); M3 final review pending, M4–M6 designed.** The software factory: work item → spec → plan → sandboxed agent runs → branch → PR reviewed by the existing reviewer. PRD (FR-F1..F32), architecture, module reference, execution layer (harness terms quoted with retrieval dates), run topology, autonomy model, product packaging, prior art, M0–M6 build order, and `AGENT-IMAGE-CONTRACT.md` — the published contract any agent image may satisfy, checked by `spire-agent-image verify`. Decisions are ADR-029..ADR-040. ROADMAP's M0 section records what the build taught that the design had wrong |
| `docs/factory/` | **M0–M3.5 implemented (PRs #95/#96/#119, M3 on master 2026-09-16, M3.5 accepted 2026-10-07 in `M35-ACCEPTANCE.md`); M4–M6 designed.** The software factory: work item → spec → plan → sandboxed agent runs → branch → PR reviewed by the existing reviewer. PRD (FR-F1..F32), architecture, module reference, execution layer (harness terms quoted with retrieval dates), run topology, autonomy model, product packaging, prior art, M0–M6 build order, and `AGENT-IMAGE-CONTRACT.md` — the published contract any agent image may satisfy, checked by `spire-agent-image verify`. Decisions are ADR-029..ADR-040. ROADMAP's M0 section records what the build taught that the design had wrong |
| `docs/CICD-AND-PACKAGING.md` | **Parked plan.** No CI exists today; analysis of GitHub Actions + GHCR images + Helm/kustomize/ArgoCD, why Terraform is declined, and why it waits for D10 |
| `docs/D10-AUTH-PLAN.md` | **Planned, not started.** The auth gate: hybrid OIDC, per-service URL prefixes so cookie scoping is real, the spike that must precede code, and the two designs review falsified |

## Status (a snapshot — rewrite it, never append to it)

Measured on **2026-09-16**. Delivery history is in docs/HISTORY.md; the consolidated
[M3 acceptance record](docs/factory/M3-ACCEPTANCE.md) maps all seven verified criteria to their
proving slices and reviews. **M3 is on master**, pushed directly on 2026-09-16 by the operator's
decision rather than merged through PR #153, together with the operator-experience fixes.
**M3.5, "one ticket to a build", is next** (operator-experience specification §2.4), then M4 with
verify as its first slice.
Measured on **2026-10-07**. Delivery history is in docs/HISTORY.md; the
[M3 acceptance record](docs/factory/M3-ACCEPTANCE.md) maps M3's seven verified criteria to their
proving slices and reviews, and the [M3.5 acceptance record](docs/factory/M35-ACCEPTANCE.md) holds
the live proof of "one ticket to a build". **M3 and M3.5 are on master.** **M4 is next, with verify
as its first slice**; until it lands, every item build stops held at `verify / capability_unavailable`.

- **The reviewer (P0–P4) is delivered.** Gateway, orchestrator and review worker communicate over
Kafka, with the React dashboard. Bitbucket Cloud, GitHub and GitLab have measured live reviewer
Expand All @@ -67,7 +66,7 @@ verify as its first slice.
proved on 2026-09-12 by runs 3987682681:1 and 3987682176:1, then re-proved on TEST PR #32 by
run 4003204361:1 on 2026-09-14: automatic push, next review, resolved thread and persisted
verdict, with the other six prior findings UNCHANGED. The TEST PR was closed and its exact
branch deleted; review/run history was retained. The development run worker remains stopped.
branch deleted; review/run history was retained.
- **M3 owns repositories, people and work-item policy.** Explicit repository-role bindings replace
account workspace lookup. People resolve to stable provider IDs and render observed handles;
source actor allowlists remain independent. /fix measures effective push access through the
Expand Down Expand Up @@ -96,7 +95,15 @@ verify as its first slice.
waits until the texts on screen match the binding the gate stores. The detail page shows eight
journey steps with their proof, and `/approvals` redirects to the Needs-you filter. Refusals name
their rule, harness and model are selects, the branch head comes from the forge, and run detail
names the key that paid. Spec and plan tickets, the plan JSON and the pricing stop remain (M3.5).
names the key that paid.
- **M3.5: one ticket to a build is delivered** (accepted 2026-10-07). Build defaults per repository,
pricing completeness with "not billed" per token type, system-composed preparation (no spec or
plan tickets, no plan JSON), the models and thinking levels a harness offers with the run pinned
to that image, and paying with a Codex subscription seat (refresh tokens emptied, seats shared,
one seat per account, run secrets scrubbed from run-worker logs). The live proof on `spire-test`
built one change three times: #37 on a seat at cost 0, #39 on an API key at 51,690 millicents,
same model and the same file. #38 found that Codex cache writes were billed twice as input;
PR #182 counts them as part of input. Human steps: write, label, approve.
- **Measured final validation (2026-09-14):** 4076 Java tests across 452 suites and 30 modules,
zero failures/errors and 1 existing Windows symlink privilege skip. Forced testFast,
testServices and packaging passed sequentially. The full UI passed 742 tests across 93 files,
Expand All @@ -113,9 +120,12 @@ verify as its first slice.

**Limits that remain:**

- **Production VERIFY and LAND remain unavailable.** M4 owns the verifier; M3 does not ship one.
- **No live item-build proof.** TEST PR #32 proves standalone /fix; real local-origin item tests
do not prove an item-linked build against a real forge.
- **Production VERIFY and LAND remain unavailable.** M4 owns the verifier; M3 and M3.5 do not
ship one.
- **No live item-linked push or pull request.** M3.5 proves live item builds against a real
GitHub ticket, but each stops held before publication; TEST PR #32 proves standalone /fix only.
- **The test counts above are from 2026-09-16.** Later PRs were checked per module; the full
suite has not been re-measured since, and builds now run in CI rather than on the host.
- **The automated GitLab run-unit gap is still open.** RunUnitSpec has no network field, so a
run unit cannot reach the e2e stack's GitLab. A live GitHub run does not close it.
- **The two factory images are still not on GHCR.**
Expand Down
20 changes: 20 additions & 0 deletions docs/HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2109,3 +2109,23 @@ lives in `docs/`, the locked decisions in `docs/DECISIONS.md`, and claims no tes
re-run. Still open and moved to M3.5: spec and plan tickets and the plan JSON (part C), incomplete
pricing (part D), Codex subscription sign-in (part F). By the operator's decision the branch was
pushed to master directly; PR #153 was not merged through GitHub.

- **Factory M3.5: one ticket to a build (2026-09-17 to 2026-10-07).**
Parts B (repository build defaults), D (pricing completeness) and C (system-composed
preparation), with part F's Codex sign-in, merged to master on 2026-09-17 after five review rounds
and twelve findings. Part M offered the models a harness runs, with their thinking levels, and
pinned a run to the image its model list came from (PR #167, #168). Part F let a build pay with a
Codex subscription seat (PR #177, #178): the agent gets the sign-in file with every refresh token
emptied, seats are shared like API keys by the operator's decision, one seat per vendor account,
and run-worker logs are scrubbed of run secrets. Part P, the live proof on `spire-test`, ran three
times. Run 1 (#37) paid by a subscription: 74,721 tokens recorded `UNMETERED` at cost 0. Run 2
(#38) paid by an API key, on another model by mistake, and was the first run to report a non-zero
Codex cache write: the adapter had read it as additional to input and billed 14,801 tokens twice,
recording the run about 43% above its cost. The Codex session log settled it (its total is input
plus output), and PR #182 subtracts the cache write from input. Run 3 (#39) repeated the API-key
run on run 1's model: 66,948 tokens, 51,690 millicents, INPUT 15 with no overlap. All three wrote
the same nine-line file and stopped at `verify / capability_unavailable`, the expected M3.5 end.
The first subscription build also produced operator fixes (PR #179–#181): deleting harness keys,
naming a build setup's missing fields, explaining the scanner, refreshing the work item page, and
a two-pane model dialog with narrow price inputs. The human steps were write, label and approve.
Record: [M3.5 acceptance](factory/M35-ACCEPTANCE.md). Next: M4, with verify as its first slice.
12 changes: 8 additions & 4 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sizing, not commitments.

---

## The software factory — M0 and M1 delivered, M2 next (updated 2026-09-04)
## The software factory — M0 to M3.5 delivered, M4 next (updated 2026-10-07)

The next phase extends Code Spire from a reviewer into an automated software factory: a tracker work
item becomes a specification, a plan, sandboxed agent runs, a pushed branch, and a pull request that
Expand All @@ -19,9 +19,13 @@ are **ADR-029..ADR-039** in [`DECISIONS.md`](DECISIONS.md); the design record fo
produced it is
[`superpowers/specs/2026-09-01-software-factory-design.md`](superpowers/specs/2026-09-01-software-factory-design.md).

**M0 (the walking skeleton, PR #95, 2026-09-02) and M1 (the lifecycle, PR #96, 2026-09-03) are
delivered** — see [`factory/ROADMAP.md`](factory/ROADMAP.md) for what each shipped and what the build
taught that the design had wrong, and [`HISTORY.md`](HISTORY.md) for the delivery log. **M2 is next.**
**M0 (the walking skeleton, PR #95, 2026-09-02), M1 (the lifecycle, PR #96, 2026-09-03), M2
(delivery, PR #119), M3 (work items, labels and gates, on master 2026-09-16) and M3.5 (one ticket to a
build, accepted 2026-10-07) are delivered** — see [`factory/ROADMAP.md`](factory/ROADMAP.md) for what
each shipped and what the build taught that the design had wrong, [`HISTORY.md`](HISTORY.md) for the
delivery log, and the [M3](factory/M3-ACCEPTANCE.md) and [M3.5](factory/M35-ACCEPTANCE.md) acceptance
records. **M4 is next, with verify as its first slice.** Until it lands, every item build stops held
at `verify / capability_unavailable`: nothing is pushed and no pull request is opened.
**M0–M2 is a shippable product on its own** — *the reviewer now fixes what it finds* — needing no
tracker, no plan engine and no gates.

Expand Down
2 changes: 1 addition & 1 deletion docs/UNVERIFIED.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ Each has a runbook mode. None has been run by an operator.
| A private-registry pull | Mode S §4 | Nothing pulls from a private registry in any test. `authFor` and the attachment are unit-tested; the *pull* is not |
| **Codex CLI 0.156.1 in the agent image** (2026-09-23) | none yet | Raised from 0.146.0 so the model list includes the gpt-6 models. Re-checked on 0.156.1: every flag the adapter passes, the API-key login, and the shape of the file it writes (`auth_mode=apikey`). The device sign-in output was then proved by a real sign-in (2026-09-25), and one live `codex exec` on a subscription produced the same `--json` event types and the same five usage buckets. A live API-key build with 11 tool calls (work item 38, 2026-09-27) was parsed end to end, and its usage was the first with a non-zero `cache_write_input_tokens`. The Codex session log's own `total_tokens` equals input plus output, so the cache write is part of input, not extra to it. The adapter had read it as extra and billed those tokens twice (that run recorded about 43% above its cost); it now subtracts both parts from input. Still NOT re-checked: a run whose parts exceed input on a real vendor, which the adapter degrades to an unreconciled total |
| **Runs pinned to the image their model list came from** (M3.5 part M, 2026-09-23) | none yet | Choosing the pin is unit-tested against given daemon answers, and one real-daemon test pins a LOCAL build by its image id. No test pulls a registry image and pins it by its registry digest, and none runs two workers. So "two workers holding different images under one tag run the same one" is argued from the code, not watched. A local-only image is pinned by an id that exists on one daemon only: on a second worker such a run fails to pull — by design, but unobserved |
| **A build paid by a Codex subscription** (M3.5 part F, 2026-09-25) | none yet | Tests prove the emptied refresh token, one seat per account, seat rotation, the unmetered charge lines and the file the adapter writes. One live `codex exec` accepted a file with its refresh token emptied, three hours after sign-in with the id token expired. No item build has run on a seat yet. NOT observed: two builds using one seat at the same moment (seats are shared by the operator's decision of 2026-09-26; whether the vendor accepts two sessions of one account at once is unknown), a run that outlives the access token (10 days — the seat then needs a new sign-in), whether the vendor's CLI ever tries to refresh mid-run with an empty refresh token, and whether `tokens.account_id` is stable across sign-ins of one account |
| **A build paid by a Codex subscription** (M3.5 part F, 2026-09-25) | none yet | Tests prove the emptied refresh token, one seat per account, seat rotation, the unmetered charge lines and the file the adapter writes. One live `codex exec` accepted a file with its refresh token emptied, three hours after sign-in with the id token expired. One item build then ran on a seat (work item 37, 2026-09-27, [M3.5 acceptance](factory/M35-ACCEPTANCE.md)): real token counts, recorded `UNMETERED` at cost 0. It reported no cache write, so whether a subscription run reports one is not established. NOT observed: two builds using one seat at the same moment (seats are shared by the operator's decision of 2026-09-26; whether the vendor accepts two sessions of one account at once is unknown), a run that outlives the access token (10 days — the seat then needs a new sign-in), whether the vendor's CLI ever tries to refresh mid-run with an empty refresh token, and whether `tokens.account_id` is stable across sign-ins of one account |
| **Run-worker log lines are scrubbed of run secrets** (review of PR #178, 2026-09-25) | none yet | `SecretLogFilter` is unit-tested on a JBoss log record and its console configuration is asserted. Not observed: the filter on the JSON console handler of a running worker. And by design it cannot scrub a unit this process did not start: after a worker restart, the watchdog's log lines about an older unit carry that unit's secrets unscrubbed, as `RunFailures` already states for failure details |
| **OIDC sessions actually renew instead of re-authenticating** | **Mode J check 11** (2026-09-10) | The bug it fixes needs a real browser, a real Keycloak and **fifteen elapsed minutes**. No suite here has any of the three: there are zero WebSocket client tests, and nothing observes a token reaching its `exp`. `OidcSessionsAreRenewedTest` asserts the four `application.yml` files *say* renewal is on — it cannot assert Quarkus *does* it |

Expand Down
5 changes: 3 additions & 2 deletions docs/factory/M3-ACCEPTANCE.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# M3 acceptance record

All seven acceptance criteria were independently verified before the final cleanup slice.
Slice 9 was accepted without findings in round 16. The final slice retains these decisions;
the pull request stays draft until the operator's final review. This record distinguishes
Slice 9 was accepted without findings in round 16. The final slice retains these decisions.
M3 reached master on 2026-09-16, pushed directly by the operator's decision; PR #153 was not
merged through GitHub. This record distinguishes
accepted control-plane evidence, real local execution, and live-forge evidence.

## Seven verified criteria
Expand Down
Loading
Loading