Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2083a72
test(strix): reject direct-provider model normalization
seonghobae Sep 2, 2026
10c1ddf
fix(strix): fail closed on direct-provider model routes
seonghobae Sep 2, 2026
7e4b7c1
docs(adr): record Strix free-pool model boundary
seonghobae Sep 2, 2026
57460d0
docs(doctoring): trace Strix direct-route RCA
seonghobae Sep 2, 2026
625f6bc
ci(strix): reconcile canonical free-only routing evidence
seonghobae Sep 2, 2026
8471d31
merge current main into Strix free-only model repair
seonghobae Sep 2, 2026
618bf65
test(strix): forbid repository-authored retry compute
seonghobae Sep 2, 2026
9dd5082
test(strix): forbid hand-selected severity admission threshold
seonghobae Sep 2, 2026
e31d7d2
ci(strix): source-fix retry and severity heuristics
seonghobae Sep 2, 2026
ba60aae
ci(strix): add exact-source retry/severity repair driver
seonghobae Sep 2, 2026
ffc0383
fix(ci): simplify Strix no-heuristics source repair workflow
seonghobae Sep 2, 2026
9246a74
merge current main into Strix no-heuristics repair
seonghobae Sep 2, 2026
cbbab94
docs(strix): reconcile free-only routing evidence
github-actions[bot] Sep 2, 2026
4e401d6
Merge remote-tracking branch 'origin/main' into pr1668
seonghobae Sep 5, 2026
15027c1
fix(strix): complete the no-heuristics retry/severity repair by hand
seonghobae Sep 5, 2026
17c5fb8
test(strix): rewrite the quick-gate self-test for single-governed-req…
seonghobae Sep 5, 2026
305e934
docs(adr): renumber the Strix free-only boundary ADR to 0028
seonghobae Sep 5, 2026
82b19c4
Merge origin/main into fix/strix-free-only-model-normalization-20260902
seonghobae Sep 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/workflows/strix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -928,7 +928,6 @@ jobs:
PNPM_CONFIG_IGNORE_SCRIPTS: "true"
YARN_ENABLE_SCRIPTS: "false"
BUN_CONFIG_IGNORE_SCRIPTS: "true"
STRIX_FAIL_ON_MIN_SEVERITY: MEDIUM
STRIX_DISABLE_PR_SCOPING: ${{ (github.event_name == 'pull_request_target' || github.event.client_payload.pr_number != '') && '0' || '1' }}
# A repository_dispatch executes in this central repository, so its
# github.token cannot read the target repository's PR. Reuse the
Expand All @@ -952,8 +951,8 @@ jobs:
model_behavior_error_signal='(^|[^A-Za-z0-9_])(agents|pydantic_ai|strix)(\.[A-Za-z_][A-Za-z0-9_]*)*\.ModelBehaviorError([^A-Za-z0-9_]|$)'
# Any evidence that a vulnerability was actually reported. Its presence
# forces a hard failure so real findings are NEVER downgraded. Keep the
# severity branch anchored away from identifiers so environment lines
# such as STRIX_FAIL_ON_MIN_SEVERITY do not look like findings.
# Keep the severity marker anchored away from identifiers so unrelated
# environment text does not look like a reported finding.
reported_vulnerability_signal='Vulnerabilities[[:space:]]+[1-9]|(^|[^A-Za-z0-9_])severity[[:space:]]*:'

# Capture the gate exit code plus its console output. The gate returns
Expand Down Expand Up @@ -988,7 +987,7 @@ jobs:
exit "$strix_rc"
fi

# An earlier out-of-scope/below-threshold finding may already have
# An earlier out-of-scope finding may already have
# been exempted by the trusted gate. Classify a later provider
# outage from the tail after the last continuation marker, but keep
# that incomplete later scan non-passing.
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ this file. The format follows Keep a Changelog, and versioned releases follow
Semantic Versioning where the repository publishes a release.

## [Unreleased]
- Strix model normalization now fails closed on every direct-provider model identifier and accepts only the contextual-orchestrator `orchestrator/free` virtual pool spellings. This closes the reusable model-selection bypass without removing `OPENAI_API_KEY` from bootstrap/global discovery; the orchestrator's free-pool credential-source policy remains the candidate-admission authority.
- **Pin `opencode-review-dispatch.yml` off the starved floating `ubuntu-latest` image.**
The 2026-09-01 floating-image fix (see that entry below) pinned `strix.yml`,
`opencode-review.yml`, and `noema-review.yml` -- the three required-check
Expand Down
40 changes: 40 additions & 0 deletions docs/adr/0028-strix-orchestrator-free-only-model-boundary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# ADR-0021: Strix model normalization is an `orchestrator/free` boundary

- Status: Proposed
- Date: 2026-09-02
- Owner: ContextualWisdomLab/.github central Strix review path

## Context

The protected central Strix workflow already provisions `ContextualWisdomLab/contextual-orchestrator`, requests the `orchestrator/free` virtual pool, supplies the five bootstrap credential sources, forces Zero Data Retention for private targets, and sets the workflow's external fallback list to empty. The shared `scripts/ci/strix_quick_gate.sh` nevertheless retained generic normalization for concrete provider model identifiers and historical direct-provider fallback code. That left a reusable execution boundary capable of accepting a provider/model name even though provider choice, free-pool admission, privacy admission, retry/failover, and serving are owned by contextual-orchestrator.

Under the organization no-heuristics and routing-owner contract, a GitHub Actions review path must not acquire a second provider/model router. The model identifier accepted by the Strix gate is therefore not a preference or fallback ranking input; it is an exact virtual-pool capability boundary.

## Decision

`normalize_model` in the Strix model helper accepts only `orchestrator/free` and its provider-qualified spelling `contextual-orchestrator/orchestrator/free`. All provider names, concrete model names, unqualified model names, and historical direct-OpenAI aliases fail closed before provider credentials or provider endpoints can become execution authority.

The five bootstrap secrets remain transport/discovery inputs to contextual-orchestrator. In particular, `OPENAI_API_KEY` is not removed from bootstrap or global discovery. The separate contextual-orchestrator free-pool admission contract decides which discovered credential sources may become `orchestrator/free` candidates; OpenAI-derived models remain excluded there while OpenAI integration may remain available to independently governed non-free/global pools.

For private targets, the central workflow's existing `CONTEXTUAL_ORCHESTRATOR_REQUIRE_ZDR=true` path remains mandatory. This ADR does not weaken or duplicate that policy; it prevents the Strix model selector from bypassing it with a concrete provider route.

## Executable evidence

`tests/test_strix_orchestrator_free_only.py` exercises the production shell helper. It admits the two exact governed virtual-model spellings and rejects direct OpenAI, OpenRouter, NVIDIA NIM, GitHub Models, Vertex/Gemini, and unqualified model identifiers before execution.

The full protected exact-head workflow set remains authoritative. Pending or queued runs are not passing evidence.

## Research and standards basis

This decision does not invent a quality score, routing rank, or threshold. It removes an overlapping router and delegates selection to the separately governed orchestrator. Privacy admission remains evidence-based and fail-closed.

OpenRouter. (2026). *Zero data retention*. https://openrouter.ai/docs/guides/features/zdr

ContextualWisdomLab. (2026, September 2). *ADR-0003: Vendored contextual-orchestrator review sidecar with governed gateway pools*. ContextualWisdomLab/.github.

## Consequences

- Strix cannot use a concrete provider/model identifier as an inference route.
- Provider discovery/failover stays inside contextual-orchestrator.
- `OPENAI_API_KEY` may still be registered and globally discovered; the free-pool candidate boundary, not secret transport, excludes OpenAI-derived candidates from `orchestrator/free`.
- Historical direct-provider fallback helpers become unreachable from the accepted Strix model boundary and should be removed as follow-up dead-code cleanup after exact-head regression evidence is available.
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Strix `orchestrator/free` model-boundary doctoring — 2026-09-02

## Exact failing evidence

Protected `main@23df081c36c93da019c89c474351002afb014daa` already hard-pins the central Strix workflow to `contextual-orchestrator/orchestrator/free`, provisions contextual-orchestrator with `CONTEXTUAL_ORCHESTRATOR_POOL: free`, passes all five bootstrap credentials, and sets `STRIX_FALLBACK_MODELS: ""`. The shared Strix gate nevertheless retained direct-provider normalization and direct-OpenAI/OpenRouter/GitHub Models/Vertex fallback machinery. That created a second provider-routing surface beneath a workflow whose accepted owner contract delegates provider selection and failover to contextual-orchestrator.

## Causal owner

The reusable owner is `ContextualWisdomLab/.github/scripts/ci/strix_model_utils.sh` together with `strix_quick_gate.sh`, not downstream repositories. The first repair is placed at model normalization so a concrete provider identifier cannot cross into any later credential/base/fallback branch.

## Test-first repair

Commit `2083a72dccaa1d96ea423a51af537240fde8a210` adds the regression contract before the production change. It requires exactly the two governed `orchestrator/free` spellings to be admitted and representative direct-provider identifiers to fail closed. Commit `10c1ddf822f1e6336b73a9093a56680fea8f4f54` changes the production normalizer accordingly.

No arbitrary rank, weight, score, threshold, retry preference, or provider order replaces the removed routing surface. The accepted identifier is a categorical authority boundary; contextual-orchestrator owns all downstream model choice.

## Credential and privacy correction retained

This repair intentionally does **not** remove `OPENAI_API_KEY` from central workflow bootstrap. All five credential sources may be registered and globally discovered. The `orchestrator/free` candidate-admission owner remains contextual-orchestrator, where OpenAI-derived candidates are excluded while BYTEZ, NVIDIA NIM primary/subaccount, and OpenRouter sources may be considered subject to explicit free/privacy/capability evidence.

Private-target ZDR remains enforced by the central workflow and sidecar. A direct provider route is rejected before it could bypass that boundary.

## Verification status

Fresh hosted exact-head tests are required before merge. Queued, pending, stale, predecessor-head, or synthetic evidence is non-passing. Historical direct-provider fallback code is now unreachable through the accepted model normalizer but remains cleanup debt until a subsequent exact-head change removes it without losing unrelated Strix scanner behavior.

## Retry and severity decision repair

Fresh protected-main evidence showed that the required Strix workflow still allocated two same-model gate retries plus a second outer three-attempt retry loop with fixed backoff values, while the reusable gate classified retryability through hand-authored provider/error regex families. The same path converted Strix severity labels to numeric ranks and used the repository-selected `MEDIUM` cutoff as a merge admission rule. Neither retry allocation nor the severity cutoff had an identified statistical model, authoritative standard, or executable experimental calibration.

The repair therefore does not substitute different retry counts, backoff constants, severity weights, or cutoffs. The central Strix path executes the governed `orchestrator/free` request once; contextual-orchestrator retains provider discovery/failover authority. Any execution that fails to produce authoritative scan evidence fails closed. Any current vulnerability report artifact also fails closed without a repository-authored severity threshold. Severity labels may remain descriptive evidence, but they are not converted into a local admission score.

**2026-09-05 addendum: applied by hand, not by the one-shot driver.** `main` had moved substantially since this doctoring entry and the `2083a72d`/`10c1ddf8` commits were authored: `run_current_target_scan` had grown a full cross-model fallback loop (`FALLBACK_MODELS_RAW`, per-candidate retry, `PR_FINDINGS_DECISION`-gated blocking, a `severity_rank`/`STRIX_MAX_SEVERITY_RANK` threshold check) that the driver script's own `simple_scan` template did not anticipate matching text for. Reconciled by hand: replaced the entire function with the single-governed-request form the driver always intended, confirmed the fallback-model system was already present at this PR's own base commit (not a later regression), and fixed the resulting ripple in six tests across four files that exercised the now-removed `is_transient_same_model_retry_error`/`github_models_rate_limit_should_skip_same_model_retry`/`run_strix_with_transient_retry` orchestration directly — three were rewritten to assert only that each underlying signal classifier (Caido bootstrap timing, ModelBehaviorError, NVIDIA NIM 404) still feeds `has_detected_infrastructure_error`, and three multi-attempt-specific assertions were removed as testing behavior that no longer exists. `scripts/ci/source_fix_strix_no_heuristic_retry_severity.py` and its companion workflow are removed here, matching the driver's own documented one-shot lifecycle, since the repair they existed to apply is now complete.
10 changes: 10 additions & 0 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -2649,6 +2649,16 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A

**Follow-up.** If the organization later solves free+ZDR routing robustly enough to deliberately widen required-review CI to `orchestrator/auto` (e.g. once a spend ceiling and reviewer-visible cost evidence exist for that path), the change is exactly one `case` arm plus the corresponding assertions in `test_sidecar_pins_the_pool_to_free_for_github_actions` — this entry is the record of *why* it was narrowed, not a permanent prohibition.

## 2026-09-02 Strix direct-provider model-boundary repair

**Live gap.** Protected `main@23df081c36c93da019c89c474351002afb014daa` already provisions the central Strix workflow through contextual-orchestrator with `STRIX_MODEL=contextual-orchestrator/orchestrator/free`, `CONTEXTUAL_ORCHESTRATOR_POOL=free`, private-target ZDR enforcement, all five bootstrap credential variables, and no external fallback list. The reusable Strix model normalizer nevertheless continued accepting concrete provider/model identifiers, leaving a second routing surface below the central workflow.

**Causal owner and repair.** `scripts/ci/strix_model_utils.sh` is the earliest reusable model-admission boundary. Regression commit `2083a72dccaa1d96ea423a51af537240fde8a210` requires direct OpenAI, OpenRouter, NVIDIA NIM, GitHub Models, Vertex/Gemini, and unqualified identifiers to fail closed. Production commit `10c1ddf822f1e6336b73a9093a56680fea8f4f54` admits only `orchestrator/free` and `contextual-orchestrator/orchestrator/free`. No fallback rank, weight, score, threshold, or provider order replaces the removed authority; contextual-orchestrator owns candidate generation, routing, serving, and failover.

**Credential/privacy boundary.** The central workflow must continue supplying `BYTEZ_API_KEY`, `NVIDIA_NIM_API_KEY`, `NVIDIA_NIM_API_KEY_SUB`, `OPENROUTER_API_KEY`, and `OPENAI_API_KEY` as bootstrap/global-discovery inputs. `OPENAI_API_KEY` transport is not a defect. The separate contextual-orchestrator `orchestrator/free` candidate policy excludes OpenAI-derived candidates while permitting the four eligible provider-account sources only when their explicit free/privacy/capability evidence passes. Private-target ZDR remains mandatory.

**Verification.** Hosted exact-head checks and independent review remain authoritative. Pending, queued, stale, predecessor-head, or synthetic evidence is non-passing. Historical direct-provider fallback helpers are now unreachable through the accepted model normalizer and remain dead-code cleanup debt until removed with exact-head regression evidence.

## 2026-09-02 org-queue-sweep investigation: historical conclusion superseded by PR #1821

**Current status (2026-09-04).** The conclusion below was invalidated by live queue evidence. PR #1821 removed the organization-wide Actions-run inventory and cancellation block from `org-queue-sweep` and merged as `11bb6a7871f4d95ab8a3eab616b4264d02327010`. Native per-PR concurrency and the current-head coalescer now own stale-run cancellation; the scheduled sweep retains only missed review, merge, and branch-update recovery. Focused ownership contracts passed 78 tests before merge. This preserves the event-gap recovery described below without paying the repository-wide run-listing and cancellation API cost.
Expand Down
42 changes: 9 additions & 33 deletions scripts/ci/strix_model_utils.sh
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -73,45 +73,21 @@ extract_vertex_model_id() {
normalize_model() {
local model
model="$(trim_whitespace "${1-}")"
if [ -z "$model" ]; then
return 0
fi

if is_vertex_resource_path "$model"; then
local provider
provider="$(sanitize_provider_name "${DEFAULT_PROVIDER:-}")" || {
echo "ERROR: Vertex resource paths require an explicit vertex_ai or vertex_ai_beta provider." >&2
return 2
}
case "$provider" in
vertex_ai | vertex_ai_beta) ;;
*)
echo "ERROR: Vertex resource paths require an explicit vertex_ai or vertex_ai_beta provider." >&2
return 2
;;
esac
printf '%s/%s\n' "$provider" "$(extract_vertex_model_id "$model")"
return 0
fi

local provider="${DEFAULT_PROVIDER:-}"
if [ -z "$provider" ]; then
provider="vertex_ai"
fi
provider="$(sanitize_provider_name "$provider")" || return $?

# Strix is an organization review path. Its model selector is therefore a
# policy boundary, not a generic provider normalizer: all inference must go
# through contextual-orchestrator's fail-closed zero-cost virtual pool.
# Provider/model identifiers would bypass the orchestrator's free-candidate
# source, capability, and private-target ZDR admission contracts, so reject
# them before credentials or provider endpoints can participate in execution.
case "$model" in
projects/* | models/* | publishers/*)
printf '%s\n' "$model"
return 0
;;
*/*)
orchestrator/free | contextual-orchestrator/orchestrator/free)
printf '%s\n' "$model"
return 0
;;
*)
printf '%s/%s\n' "$provider" "$model"
return 0
echo "ERROR: Strix model must be orchestrator/free through contextual-orchestrator; direct provider/model routes are forbidden: '$model'." >&2
return 2
;;
esac
}
Expand Down
Loading
Loading