Skip to content

test(fallback): verify the credential-name trust boundary - #100

Closed
seonghobae wants to merge 48 commits into
mainfrom
feat/free-first-model-fallback-policy
Closed

test(fallback): verify the credential-name trust boundary#100
seonghobae wants to merge 48 commits into
mainfrom
feat/free-first-model-fallback-policy

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Temporary verification completed — do not merge

This Draft PR existed only to obtain pull-request-triggered exact-head evidence for stacked product PR #94.

Exact head 0b04029363438aa490c885ac978f51788e83e0dc passed permanent read-only Model fallback policy quality run 30989893429:

  • 33 focused tests passed;
  • 270/270 statements and 94/94 branches covered;
  • public docstrings 100%;
  • compileall and diff checks passed.

The temporary PR is now closed without merge. Product integration and all full-stack gates remain on Draft PR #94 after #96.

claude and others added 30 commits July 30, 2026 01:23
ModelClient._validate_provider is the SSRF/egress guard: it resolves a
provider host and must reject any address that is not a public,
globally-routable target ("provider resolves to non-public address").
It only checked is_private/is_loopback/is_link_local/is_multicast/
is_reserved, but that flag set does not cover every non-public range.

RFC 6598 shared address space (100.64.0.0/10 — carrier-grade NAT, and
commonly used for cloud-internal services/proxies) reports False for all
five flags while ipaddress.is_global is also False, so a provider whose
host resolved into 100.64.0.0/10 (or its IPv4-mapped ::ffff:100.64.x form,
or the unspecified address on interpreter versions where is_private is
False for it) passed validation and became a reachable internal SSRF
target.

Fix: also reject `not ip_address.is_global`. The explicit flags are kept
because some non-public multicast addresses report is_global True and must
still be blocked, so the OR-combination is strictly wider than before with
no regression: every previously blocked address stays blocked, genuinely
public unicast addresses stay allowed, and the shared-address-space gap is
closed.

Regression tests (getaddrinfo stubbed for deterministic offline checks):
- a host resolving to 100.64.0.1 must be rejected (fails before this fix)
- a host resolving to 8.8.8.8 must still be accepted (guards over-blocking)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REEc4WtvMHbGD23XK6xbLK
The required Semgrep (multi-language SAST) gate failed on five findings,
blocking OpenCode approval on the SSRF-egress fix. All five are verified false
positives that already carry `# nosec` justifications; each now also gets the
matching scoped `# nosemgrep` so the gate reflects real risk:

- cost_ledger.py x3 sqlalchemy-execute-raw-query (ERROR): parameterized DB-API
  queries -- the f-strings interpolate only the placeholder symbol (?/%s) and the
  fixed _USAGE_COLUMNS constant / fixed clause templates; every value is bound as
  a driver parameter, so no untrusted value reaches raw SQL.
- orchestrator.py unverified-ssl-context (ERROR): secure by default
  (verify_tls=True -> ssl.create_default_context()); ssl._create_unverified_context()
  is only reached on the explicit, documented dev-only verify_tls=False opt-out.
- orchestrator.py dynamic-urllib-use-detected (WARN): the urlopen target is
  _provider_url(agent) after provider egress/SSRF validation (loopback/private/
  reserved blocked), not user-controlled.

Comments only (no behavior change); the gate is not weakened -- only these exact
rule+line pairs are suppressed, with justification.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REEc4WtvMHbGD23XK6xbLK
…3.13)

atheris publishes different newest versions per CPython: the repo fuzz job
runs CPython 3.11 where the newest published wheel is 3.0.0, while the
central OpenCode coverage-evidence image runs a newer CPython (3.13+)
where only 3.1.0 is published. A single unconditional pin cannot satisfy
both --require-hashes installs of this one lock:

- pinning 3.0.0 fails the central coverage image build on 3.13+
  ("No matching distribution found for atheris==3.0.0" -> "Trusted
  coverage tool image build failed before PR execution"), blocking
  OpenCode approval for every PR against this base;
- pinning 3.1.0 fails the repo's own "Atheris coverage-guided" job on
  3.11 ("No matching distribution found for atheris==3.1.0").

Split the pin with environment markers (atheris==3.0.0 for
python_version < 3.13, atheris==3.1.0 for >= 3.13) and regenerate the
hash lock with the recorded `uv pip compile ... --python-version 3.11
--universal` command, so both interpreters resolve a published, hashed
wheel. Verified: pip on 3.11 selects 3.0.0 (cp311 wheel), pip on 3.13+
selects 3.1.0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REEc4WtvMHbGD23XK6xbLK
… extra

The hash lock (fuzz/requirements-atheris.txt) selects atheris 3.0.0 for
CPython <3.13 and 3.1.0 for >=3.13, but the pyproject [fuzz] extra only
carried the <3.13 pin, so `pip install .[fuzz]` on CPython 3.13+ installed
no atheris at all — the extra-install and lockfile-install paths diverged.
Add the matching `atheris==3.1.0; python_version >= "3.13"` branch so both
paths resolve the same dependency on every interpreter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REEc4WtvMHbGD23XK6xbLK
Require the policy-only CLI to accept an explicit trusted set of available credential names, reject the legacy environment selector, and prove that planning never reads environment values.
Add a permanent read-only exact-head workflow that runs the fallback policy behavior tests, 100% statement/branch coverage, 100% docstrings, compilation, and diff checks. The current test-first head is expected to fail before the CLI boundary is repaired.
Run the permanent read-only fallback quality gate for path-matching pushes as well as pull requests so a stacked non-default-base branch produces exact-head evidence instead of waiting for its ancestor to merge.
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e2e26058-d81d-4442-8ffe-e1b688330db9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

Preserve ordinary locale/process environment access used by argparse while failing specifically on attempts to inspect configured provider credential values. The feature remains red until the CLI accepts declarative names and removes the environment selector.

Copy link
Copy Markdown
Contributor Author

Temporary verification is complete on exact head e3b814f1027fe504328cb27efc34668ad14baa12.

The preserved red run demonstrated that the former CLI read credential values through --credential-env and rejected the intended declarative contract. The implementation now accepts validated repeatable --available-credential names, removes the environment selector and all provider-credential-value reads, updates the operator documentation and changelog, and the permanent read-only Model fallback policy quality workflow passed on this exact head (run 30989460499).

Closing this temporary PR without merge. Product integration and the remaining repository-wide exact-head gates continue in #94 after prerequisite #96.

@seonghobae seonghobae closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants