Add conformance CI workflow (fault + prod lanes) [DEV-1160] - #17
Conversation
Runs the raindrop-sdk-harness against the Rust conformance driver (conformance/, DEV-1145) on every pull request, mirroring the hardened pattern from python-sdk/raindrop-js/java-sdk: - fault-lane: local capture/fault server only, no credentials beyond the HARNESS_READ_TOKEN org secret; loud fork-safe skip. - prod-lane: real Raindrop ingest + Query API via org secrets (RAINDROP_WRITE_KEY / RAINDROP_QUERY_API_KEY), step-scoped env, loud skip when secrets are unavailable. - Harness pinned to invisible-tools/raindrop-sdk-harness@b162d84a (main @ 2026-07-13); bump procedure documented in the header. - Actions SHA-pinned, persist-credentials: false, concurrency cancel-in-progress, 15min timeouts, continue-on-error while the suite beds in, report JSON artifact + job summary per lane. - Swatinem/rust-cache scoped to the standalone conformance crate keeps registry + target caching sane. DEV-1160 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 522fb32. Configure here.
| pull_request: | ||
|
|
||
| permissions: | ||
| contents: read |
There was a problem hiding this comment.
Artifact upload lacks token permission
Medium Severity
The workflow caps GITHUB_TOKEN at contents: read, but both lanes call actions/upload-artifact under if: always(). Uploading workflow artifacts needs write access to Actions (typically actions: write); with only contents: read, those upload steps often fail with “Resource not accessible by integration,” so report artifacts may never be published even when the runner succeeds.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 522fb32. Configure here.
There was a problem hiding this comment.
Refuted empirically: this PR's own run (29274896649) uploaded both artifacts successfully — conformance-prod-report and conformance-fault-report are live on the run — under exactly this permissions block. actions/upload-artifact v4 authenticates uploads with the runner's ACTIONS_RUNTIME_TOKEN, not the job's GITHUB_TOKEN, so contents: read is sufficient (and is the same block the merged python-sdk/java-sdk/raindrop-js conformance workflows have been uploading with all day). Keeping least-privilege as-is.
New scenarios since the previous pin (wrap-capture-attachments, the three expect.traces span scenarios) are all status: experimental — they report but never gate, so no ratchet changes are needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>


Summary
Adds
.github/workflows/conformance.yml(DEV-1160): per-PR conformance CI running the raindrop-sdk-harness against this repo's Rust conformance driver (conformance/, DEV-1145). Mirrors the hardened workflow already landed in python-sdk, raindrop-js, and java-sdk.Two lanes
cargo build --manifest-path conformance/Cargo.toml, binary atconformance/target/debug/raindrop-conformance-driver), starts the harness's local capture/fault server, and runs the full scenario corpus against it. No production calls; the only secret involved is theHARNESS_READ_TOKENorg secret used to check out the private harness repo, with a loud fork-safe skip when it's unavailable.RAINDROP_WRITE_KEY/RAINDROP_QUERY_API_KEYorg secrets (dedicated test org). Endpoints and credentials are scoped to the single step that needs them; skips loudly when secrets are missing.Hardening (same pattern as the sibling SDKs)
invisible-tools/raindrop-sdk-harness@b162d84a4c25b7203173f99e595de7ba88a57960(main @ 2026-07-13, 27 scenarios); the bump procedure is documented in the workflow header.dtolnay/rust-toolchainstable pin);persist-credentials: falseon every checkout;permissions: contents: read; concurrency group withcancel-in-progress; 15-minute job timeouts.continue-on-error: trueon the runner steps while the suite beds in (non-blocking); each lane uploads a report JSON artifact and writes a full job summary. Ratchet baseline is the existingconformance/failures.txt.Swatinem/rust-cachescoped to the conformance crate (workspaces: conformance) caches the cargo registry + the crate's target dir, keyed on its lockfile, to keep runtime sane.Notes
🤖 Generated with Claude Code
Note
Medium Risk
Prod lane uses org secrets against live Raindrop APIs (scoped to one step), but no SDK/runtime code changes; main risk is CI misconfiguration or secret exposure patterns, mitigated by gating and
persist-credentials: false.Overview
Adds
.github/workflows/conformance.yml, a PR-only workflow that runs the pinned raindrop-sdk-harness against the Rustconformance/driver (cargo build --manifest-path conformance/Cargo.toml).fault-lane builds the harness local server + runner, starts the fault capture server on
127.0.0.1:8787, and runs scenarios with--lane faultandconformance/failures.txtratchet. prod-lane runs--lane prodagainst real ingest/query APIs whenHARNESS_READ_TOKEN,RAINDROP_WRITE_KEY, andRAINDROP_QUERY_API_KEYare set; prod credentials are limited to the runner step.Both jobs use gate steps for fork-safe loud skips, SHA-pinned actions,
persist-credentials: false, concurrency cancel, rust-cache onconformance, job summaries, and report artifacts. Runner steps usecontinue-on-error: trueso conformance is visible but not blocking yet.Reviewed by Cursor Bugbot for commit 7886b2e. Bugbot is set up for automated code reviews on this repo. Configure here.