Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
1907259
fix(noema): add repair-attempt telemetry, structured output, local JS…
seonghobae Sep 2, 2026
4ed1334
fix(noema): match response_format probe-count floor to validate_subst…
seonghobae Sep 2, 2026
f4a8b0e
chore(noema): reconcile repair telemetry with protected main
seonghobae Sep 2, 2026
58052ab
fix(noema): repair live review findings
seonghobae Sep 2, 2026
4c84a4c
ci(pr1672): verify exact Noema remediation
seonghobae Sep 2, 2026
f3e2bed
fix(noema): harden temporary repair replacement semantics
seonghobae Sep 2, 2026
749bdae
fix(noema): normalize generated regression EOF
seonghobae Sep 2, 2026
b26ec64
fix(noema): add exact-head follow-up remediation
seonghobae Sep 2, 2026
8d28205
fix(noema): wire exact-head follow-up repair
seonghobae Sep 2, 2026
f975738
fix(pr1672): add current-head Noema review remediation
seonghobae Sep 2, 2026
3bf448b
ci(pr1672): verify current-head Noema remediation
seonghobae Sep 2, 2026
b648b2d
ci(pr1672): target existing focused Noema tests
seonghobae Sep 2, 2026
d5db588
fix(noema): remove competing branch writer
seonghobae Sep 2, 2026
54103ec
test(pr1672): align fixtures with reviewed timeout contract
seonghobae Sep 2, 2026
5649e4f
fix(noema): add missing-value JSON repair remediation
seonghobae Sep 2, 2026
57e33a0
ci(pr1672): verify fixture reconciliation
seonghobae Sep 2, 2026
abe260e
fix(noema): serialize current-head remediation
seonghobae Sep 2, 2026
b66c47d
ci: materialize PR1672 fixed-timeout repair
seonghobae Sep 2, 2026
fbc0361
ci: retire failed PR1672 deadline repair driver
seonghobae Sep 2, 2026
ce9df6e
ci: retry PR1672 fixed-timeout repair with locked deps
seonghobae Sep 2, 2026
6661049
test(noema): reject heuristic probe allocation
seonghobae Sep 2, 2026
db8329e
fix(noema): stage exhaustive probe allocation repair
seonghobae Sep 2, 2026
97e26bd
ci: execute PR1672 no-heuristic probe repair
seonghobae Sep 2, 2026
1e029a3
ci: trigger PR1672 no-heuristic probe repair
seonghobae Sep 2, 2026
bf547a2
fix(ci): parse PR1672 no-heuristic source fix
seonghobae Sep 2, 2026
40ddb90
ci: retrigger PR1672 current-head owner repair
seonghobae Sep 2, 2026
0d611b7
ci: retrigger PR1672 exhaustive probe repair
seonghobae Sep 2, 2026
e15bba4
chore(noema): reconcile PR1672 with protected main
seonghobae Sep 2, 2026
9bd1496
ci: retrigger PR1672 exhaustive probe repair after restack
seonghobae Sep 2, 2026
6f2c936
chore(noema): retire completed PR1672 repair machinery
seonghobae Sep 2, 2026
6330611
ci: materialize PR 1672 reviewer fixes
seonghobae Sep 2, 2026
62d71a5
ci: repair PR 1672 one-shot writer syntax
seonghobae Sep 2, 2026
d9dc75e
test(noema): prove heuristic review policy fails closed
seonghobae Sep 2, 2026
99a079c
chore(noema): add exact no-heuristics repair driver
seonghobae Sep 2, 2026
e106e82
ci: retire superseded PR 1672 repair workflow
seonghobae Sep 2, 2026
d3ab72c
ci(noema): add no-heuristics source repair
seonghobae Sep 2, 2026
fd3bd51
chore(noema): trigger no-heuristics source repair
seonghobae Sep 2, 2026
b39d4ce
fix(noema): drop unbounded exhaustive repair materializer
seonghobae Sep 2, 2026
a76853b
test(noema): remove impossible exhaustive-evidence RED
seonghobae Sep 2, 2026
f14b0e6
test(noema): require fail-closed evidence policy without quotas
seonghobae Sep 2, 2026
556a179
ci: retire incompatible PR 1672 source-fix workflow
seonghobae Sep 2, 2026
0f10a22
ci: remove incompatible PR 1672 source-fix trigger
seonghobae Sep 2, 2026
b578986
chore(noema): restore fail-closed no-heuristics repair driver
seonghobae Sep 2, 2026
f6468bc
ci(noema): restore fail-closed source repair
seonghobae Sep 2, 2026
afdb8aa
chore(noema): retrigger fail-closed no-heuristics repair
seonghobae Sep 2, 2026
dc9dea6
chore(noema): retire invalid PR1672 repair machinery
seonghobae Sep 2, 2026
608e636
merge(main): reconcile PR #1672 with protected main
seonghobae Sep 2, 2026
ed457ad
test(noema): reject caller-owned fixed model timeout
seonghobae Sep 2, 2026
6f1b4ce
ci(pr1672): materialize no-fixed-timeout owner repair
seonghobae Sep 2, 2026
451a900
test(noema): forbid caller retry and sampling heuristics
seonghobae Sep 2, 2026
a1511e3
fix(noema): expand source-fix to remove caller model heuristics
seonghobae Sep 2, 2026
b5842f3
fix(noema): repair retry-specific source-fix fallout
seonghobae Sep 2, 2026
97d993a
ci(noema): retire unsafe self-modifying repair workflow
seonghobae Sep 2, 2026
f42a9d4
repair(noema): stage single-request causal-owner materializer
seonghobae Sep 2, 2026
ea048c3
repair(noema): run exact-head single-request materializer
seonghobae Sep 2, 2026
3e524d3
fix(ci): preserve successor checks for PR1672 repair
seonghobae Sep 2, 2026
efb7576
merge(main): reconcile PR #1672 Noema repair lane onto current protec…
seonghobae Sep 2, 2026
428b94d
fix(noema): make PR1672 materializer match exact head
seonghobae Sep 2, 2026
63cc6d3
fix(noema): make PR1672 one-shot retirement valid
seonghobae Sep 2, 2026
b606ecc
fix(noema): repair PR1672 exact-head materializer transform
seonghobae Sep 2, 2026
5757529
fix(noema): anchor PR1672 materializer by unique labels
seonghobae Sep 2, 2026
df4fe8e
fix(noema): retire stale repair-retry exception in PR1672
seonghobae Sep 2, 2026
13f7f7d
fix(noema): make PR1672 materializer literal and idempotent
seonghobae Sep 2, 2026
5ea2048
fix(noema): retire stale repair exception in materializer
seonghobae Sep 2, 2026
6c39974
fix(noema): remove stale retry doc token in materializer
seonghobae Sep 2, 2026
6bb1d52
fix(noema): restore valid source-fix workflow indentation
seonghobae Sep 2, 2026
e9f77f0
fix(noema): remove decorated retry tests atomically
seonghobae Sep 2, 2026
e153a21
test(noema): align legacy failures with single-request owner
seonghobae Sep 2, 2026
da83599
fix(noema): materialize single-request stale-test contract
seonghobae Sep 2, 2026
3c3ae6c
ci(noema): run parseable single-request repair
seonghobae Sep 2, 2026
0df1116
fix(noema): normalize single-request repair output
seonghobae Sep 2, 2026
7695037
ci(noema): verify normalized single-request repair
seonghobae Sep 2, 2026
a29cf4c
fix(noema): require workflow-starting credential for source repair
seonghobae Sep 2, 2026
a837cf1
ci(noema): retrigger exact-head owner repair
seonghobae Sep 2, 2026
e3b07f2
merge(main): reconcile Noema single-request repair lane
seonghobae Sep 2, 2026
75dffb7
fix(actions): require workflow-capable token for PR 1672 publisher
seonghobae Sep 2, 2026
96daa27
docs(noema): repair markdownlint findings
seonghobae Sep 2, 2026
97bc3d5
fix(actions): narrow PR 1672 publisher permissions
seonghobae Sep 2, 2026
87a05b7
fix(noema): normalize generated repair outputs
seonghobae Sep 2, 2026
bc2f164
chore(noema): retrigger exact-head owner repair
seonghobae Sep 2, 2026
390173e
fix(noema): retire completed timeout source-fix debris
seonghobae Sep 2, 2026
d389e82
ci(noema): surface exact coverage gaps after owner repair
seonghobae Sep 2, 2026
be22a5e
test(noema): cover lossless repair and invalid model metadata edges
seonghobae Sep 2, 2026
43ae025
ci(noema): validate permanent model-output edge regressions
seonghobae Sep 2, 2026
bf3ddd2
fix(ci): keep PR1672 publish credentials out of git URLs
seonghobae Sep 2, 2026
9a2895b
fix(ci): publish verified PR1672 source without workflow-token depend…
seonghobae Sep 2, 2026
d4199e3
ci(noema): retrigger verified PR1672 materialization
seonghobae Sep 2, 2026
097d652
fix(ci): isolate verified PR1672 source publication from workflow cle…
seonghobae Sep 2, 2026
e5ca5e3
ci(noema): retrigger isolated PR1672 source publication
seonghobae Sep 2, 2026
73ab4a1
fix(ci): route PR1672 publisher through registered source-fix workflow
seonghobae Sep 2, 2026
a7fac6b
fix(noema): delegate repair ownership to orchestrator
github-actions[bot] Sep 2, 2026
db13a2d
merge(main): reconcile verified Noema repair and retire publishers
seonghobae Sep 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions .github/actions/noema-review/two_phase.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,20 +167,16 @@ def prepare_verdict(repo: str, number: int, expected_head: str, path: Path) -> i
changed_files = gate.fetch_changed_files(repo, number)
changed_paths = tuple(file_path for file_path, _status in changed_files)
review_context = gate.build_review_context(repo, number, pull_request, changed_files)
try:
verdict = gate.call_llm(
repo,
number,
pull_request,
diff,
truncated,
expected,
review_context,
changed_paths,
)
except gate.StaleHeadDuringRepairRetryError:
print("Pull request head changed during model repair retry; verdict was not sealed.")
return 0
verdict = gate.call_llm(
repo,
number,
pull_request,
diff,
truncated,
expected,
review_context,
changed_paths,
)
Comment thread
seonghobae marked this conversation as resolved.

_write_envelope(
path,
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## 2026-09-02 — Noema single-request gateway ownership

- Removed the repository-owned 900-second repair deadline and duplicate model repair call from Noema. The GitHub Actions caller now issues one structured-output request while `contextual-orchestrator` owns repair/failover/timeouts.
- Hardened serving-model telemetry against control-character/workflow-command injection and lone-surrogate encoding failures, restored actionable exact changed-line diagnostics, and constrained local trailing-comma repair to complete JSON values.
- Added permanent single-request/no-fixed-timeout regressions and retired obsolete deadline/retry fixtures.

# Changelog

All notable changes to the organization automation repository are documented in
Expand Down
34 changes: 34 additions & 0 deletions docs/doctoring/noema-repair-attempt-telemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Noema single-request review incident and telemetry contract

## Incident

On 2026-09-02, a required Noema review reported only a caller-owned 900-second repair deadline after a malformed structured response. The bound had no owner-specified or measured basis and conflicted with ADR-0003: model inference and repair verdict calls do not carry repository-authored fixed wall-clock deadlines.

```text
initial malformed structured response -> repository repair request -> fixed 900-second abort
```

The later review established a second ownership error: `contextual-orchestrator` already owns structured-output validation and its governed repair/failover. Issuing another repository-side model request duplicated that policy and could turn one gateway failure into two expensive calls.

## Final executable contract

Noema now sends exactly one structured-output request to the configured gateway. GitHub Actions fixes the model alias to `orchestrator/free`; the caller declares no provider, paid fallback, sampling temperature, or fixed inference timeout. `contextual-orchestrator` owns provider discovery, capability routing, structured-output repair, failover, and upstream completion. The repository remains responsible for deterministic local validation and exact-head publication.

Every gateway call emits exactly one passive Actions annotation. Success and failure annotations include caller attempt count, elapsed duration, active phase (`connecting`, `reading`, `decoding`, or `validating`), and a best-effort serving-model identifier. Serving-model text is secret-scrubbed, control-character-normalized, UTF-8 printable, and bounded before it can reach an annotation. Raw model output is never logged.

The local trailing-comma parser remains a deterministic syntax transform only. It may remove a genuine trailing comma after a complete JSON value, but missing-value forms such as `[,]`, `{,}`, `[1,,]`, and `{"a":,}` remain invalid. The transform emits no second attempt-level annotation and never bypasses semantic verdict validation.

Exact changed-line diagnostics include the rejected path/line/side, an unambiguous array position, and a bounded nearest-line hint. This keeps a failed verdict repairable at the gateway without expanding the output contract to one record per changed line.

## Ownership and failure scenes

```text
Noema workflow -> local contextual-orchestrator sidecar -> orchestrator/free -> routed free candidate
-> one returned envelope -> local deterministic validation -> exact-head publication
```

If the gateway cannot produce a valid structured verdict, Noema fails closed after that one caller request. If the PR head moves during model work, the post-call exact-head check discards the stale verdict. If telemetry carries hostile model identifiers, annotation sanitization prevents CR/LF or surrogate data from becoming workflow commands or crashing the runner.

## Verification

The permanent contract test forbids `NOEMA_REPAIR_DEADLINE_SECONDS`, `_repair_wall_clock_deadline`, `NoemaRepairDeadlineExceeded`, `signal.setitimer`, retry-only parameters/recursion, and caller-specified `temperature`. Focused regressions prove one request on success and failure, one annotation per attempt, safe serving-model telemetry, strict missing-value rejection, accepted genuine trailing commas, and preserved exact changed-line diagnostics.
12 changes: 12 additions & 0 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -2613,3 +2613,15 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A
**Expected effect.** No observable change to any current GitHub Actions review run (every current invocation already resolves to `free`). The effect is structural: it is no longer possible for a future workflow edit or manual dispatch override to admit priced-model spend into a required review check without an explicit, reviewed code change to this one `case` statement (and its now-locked-in regression test) first.

**Follow-up.** If the organization later solves free+ZDR routing robustly enough to deliberately widen required-review CI to `orchestrator/auto` (e.g. once a spend ceiling and reviewer-visible cost evidence exist for that path), the change is exactly one `case` arm plus the corresponding assertions in `test_sidecar_pins_the_pool_to_free_for_github_actions` — this entry is the record of *why* it was narrowed, not a permanent prohibition.

## Noema single-request model-control ownership — PR #1672 (2026-09-02)

**Status:** Proposed / exact-head verification required before merge.

**Root cause.** Noema duplicated `contextual-orchestrator` structured-output repair by making a second model request and wrapped that request in an unmeasured 900-second repository wall-clock deadline. This created a self-hosting admission failure: the required review could terminate valid long inference using policy that the gateway already owns.

**Context Map / responsibility boundary.** `.github` owns CI review orchestration, exact-revision evidence, deterministic verdict validation and publication. `contextual-orchestrator` owns provider discovery, capability routing, `orchestrator/free`, structured-output repair/failover and provider completion. No provider/model-specific fallback or caller wall-clock timeout crosses that boundary.

**Action.** Replace recursive caller repair with one structured-output gateway request; remove fixed deadline/signal machinery and sampling temperature; retain exact-head checks before and after model work; sanitize serving-model telemetry; restore exact changed-line diagnostics; retain bounded non-heuristic evidence cardinality and strict local JSON parsing.

**Evidence / acceptance.** Permanent tests forbid retry/deadline/sampling symbols and prove one gateway request, one attempt annotation, control-character-safe telemetry, missing-value rejection, valid trailing-comma normalization, and exact changed-line guidance. Fresh exact-head repository checks/reviews remain the admission authority; predecessor-head evidence is not transferable.
Loading
Loading