[P1] Build an immutable signed release, promotion, and rollback pipeline

[seonghobae]

Production gap

Protected main currently has a minimal Rust CI job (format, locked tests, Clippy) but no repository-owned release workflow that proves coverage, packages an immutable image, emits an SBOM, signs artifacts, attaches provenance, deploys an ephemeral production-shaped environment, runs smoke/attack tests, or promotes the exact verified digest. The Kubernetes manifest references a tag rather than an image digest.

Audited protected base: b53dc7a1b8904a16752abbdc04429df893a4e32e.

Required pipeline

Build and test

1. Merge PR build(rust): pin and track Rust 1.98.0 #77 or otherwise pin the exact supported Rust toolchain; no floating stable compiler in local or CI contracts.
2. Preserve PR fix(deploy): require externally provisioned admin secret #72's explicit 100% production line/branch coverage gate and graceful-shutdown evidence after it lands.
3. Run format, locked workspace tests, strict Clippy, rustdoc/doc coverage, fuzz/property regressions, smoke tests, dependency review, OSV, Trivy, Semgrep, CodeQL, secret scanning, license policy, and the Strix attack lane from 서버를 켜고 Strix가 포트를 향해 각종 공격을 할 때 감지해내야 함 (CI) #11 on the exact candidate.
4. Use deterministic/hermetic fixtures for external feeds and integrations.
5. Upload machine-readable test, coverage, security, and attack evidence tied to the candidate SHA.

Artifact integrity

1. Build the release image once from the reviewed commit; do not rebuild separately for staging and production.
2. Pin builder/runtime bases and GitHub Actions by immutable digest/SHA.
3. Generate CycloneDX or SPDX SBOMs for Rust and the final container filesystem.
4. Produce SLSA provenance and keyless Sigstore/Cosign signatures using a narrowly scoped GitHub OIDC identity.
5. Record source SHA, toolchain, dependency lock digest, image digest, SBOM digest, signature bundle, provenance, and test evidence in a release manifest.
6. Verify signatures/provenance before deployment and admission; reject mutable tags as authority.

Promotion and deployment

1. Deploy by digest to an ephemeral production-shaped namespace with the hardened external-secret contract from fix(deploy): require externally provisioned admin secret #72 and filename migration from Rename Kubernetes manifest to wardnet.yaml after external-secret hardening lands #75.
2. Validate Kubernetes Restricted Pod Security controls, NetworkPolicies, resource bounds, readiness/liveness/startup probes, graceful shutdown, migration compatibility, and non-root/read-only filesystem behavior.
3. Run smoke, persistence/restart, auth fail-closed ([P0] Fail closed when management credentials are absent #78), SSRF ([P0] Enforce a fail-closed destination policy for all outbound traffic #79), attack (서버를 켜고 Strix가 포트를 향해 각종 공격을 할 때 감지해내야 함 (CI) #11), and rollback tests against the deployed digest.
4. Promote the same digest through environments with recorded approval; no force-push or branch-protection bypass.
5. Define canary/blue-green criteria, automatic halt, database compatibility window, and rollback/roll-forward procedures.
6. Publish a versioned changelog/release note and supported-upgrade matrix. The repository currently lacks a CHANGELOG authority.

RED → GREEN verification

• Tampered image, SBOM, signature, provenance, source SHA, or workflow identity must fail verification.
• A mutable tag resolving to a different digest must never be promoted.
• Rebuild reproducibility differences are reported and bounded/explained.
• Failed security, coverage, attack, migration, smoke, or approval evidence blocks promotion.
• Rollback exercise restores the previous digest and compatible schema/state within the declared objective.
• Artifact retention makes the full evidence bundle independently retrievable after the workflow run.

Acceptance criteria

• One reviewed SHA maps to one immutable release digest and complete evidence manifest.
• Production promotion verifies signatures, provenance, SBOM, policy gates, and exact-head test evidence.
• No production manifest relies on mutable tags or repository-shipped credentials.
• Rollback/roll-forward is rehearsed and measured, not merely documented.
• Release notes, upgrade compatibility, support window, and deprecation policy are published.
• Current-head independent review and zero unresolved actionable threads remain mandatory.

Dependencies

• fix(deploy): require externally provisioned admin secret #72: external-secret boundary, coverage, and shutdown correctness.
• Rename Kubernetes manifest to wardnet.yaml after external-secret hardening lands #75: post-hardening manifest rename.
• build(rust): pin and track Rust 1.98.0 #77: pinned Rust compiler.
• [P0] Fail closed when management credentials are absent #78, [P0] Enforce a fail-closed destination policy for all outbound traffic #79, [P0] Add an authoritative PostgreSQL control plane with tenant isolation and recoverable migrations #80, [P0] Add a transactional outbox and idempotent leased workers for external effects #81, [P1] Add bounded distributed admission control, trusted client attribution, and overload behavior #83: production behavior that the release lane must exercise.

References

• SLSA Project. (2025). SLSA specification version 1.2. https://slsa.dev/spec/v1.2/
• Sigstore. (n.d.). Cosign documentation. https://docs.sigstore.dev/cosign/
• Kubernetes Authors. (2026). Pod Security Standards. https://kubernetes.io/docs/concepts/security/pod-security-standards/
• Kubernetes Authors. (2026). Good practices for Kubernetes Secrets. https://kubernetes.io/docs/concepts/security/secrets-good-practices/
• National Institute of Standards and Technology. (2022). Secure Software Development Framework (SSDF) version 1.1 (NIST SP 800-218). https://doi.org/10.6028/NIST.SP.800-218