Skip to content

test: add OPA provider to cross-repo integration tests#586

Open
gvauter wants to merge 1 commit into
complytime:mainfrom
gvauter:feat/opa-cross-repo-validation
Open

test: add OPA provider to cross-repo integration tests#586
gvauter wants to merge 1 commit into
complytime:mainfrom
gvauter:feat/opa-cross-repo-validation

Conversation

@gvauter

@gvauter gvauter commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

Add end-to-end OPA provider validation to the cross-repo integration test pipeline. The tests exercise the full complyctl + complyctl-provider-opa workflow using the existing mock registry test data (K8s container security policies).

Changes

  • CI workflow: Add conftest@v0.68.2 install step (OPA provider runtime dependency)
  • Test script: Add OPA provider prerequisite checks, binary install, and 4 new test functions:
    • test_get_opa — verifies OPA policy + complypack pulled from mock registry
    • test_generate_opa — verifies generate succeeds with OPA provider
    • test_scan_opa — scans non-compliant K8s fixture, asserts failures with correct requirement IDs
    • test_scan_opa_compliant — swaps in compliant K8s fixture, asserts all pass
  • Test fixtures: Add test-deployment-bad.yaml (fails both Rego policies) and test-deployment-good.yaml (passes both)
  • Rename: test-opa-bptest-opa-k8s (these policies check K8s container security, not branch protection)
  • Docs: Update TESTING_ENVIRONMENT.md policy ID references

What it tests

The full pipeline end-to-end with real binaries:

  1. complyctl get pulls OPA Gemara policy + complypack from mock OCI registry
  2. complyctl generate resolves policy graph, passes complypack content path to OPA provider via gRPC
  3. OPA provider extracts complypack, loads complytime-mapping.json, matches requirement IDs to Rego namespaces
  4. complyctl scan triggers conftest test against K8s Deployment YAML with real Rego policies
  5. Results flow back through gRPC with correctly resolved requirement IDs

Companion PR

A companion change to install conftest in the complytime-providers cross-repo CI workflow will be opened separately against complytime/complytime-providers.

Ref: complytime/complytime-providers#73

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivy found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@gvauter

gvauter commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

Trivy Findings

The 30 Trivy annotations are expected — they flag the K8s Deployment test fixtures (test-deployment-bad.yaml and test-deployment-good.yaml) for security misconfigurations.

  • test-deployment-bad.yaml is intentionally non-compliant (no securityContext, no resources.limits). This is the fixture the OPA provider is expected to flag as failing.
  • test-deployment-good.yaml passes the two OPA Rego policies we test (has runAsNonRoot: true and resources.limits), but Trivy has a broader set of checks (seccomp, capabilities, read-only root filesystem, etc.) that are not in scope for this test.

These are test fixtures, not deployed infrastructure. The devcontainer's post-create.sh generates a similar fixture inline specifically to avoid this issue (see the comment at line 103: "generated inline to avoid Trivy false positives").

@gvauter gvauter marked this pull request as ready for review June 16, 2026 17:59
@gvauter gvauter requested a review from a team as a code owner June 16, 2026 17:59
@gvauter gvauter force-pushed the feat/opa-cross-repo-validation branch from 1221a94 to d718b28 Compare June 17, 2026 13:23
Comment thread tests/cross-repo/testdata/test-deployment-bad.yaml Fixed

@gxmiranda gxmiranda left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: #586 — test: add OPA provider to cross-repo integration tests

Verdict: COMMENT (no blocking findings)

Clean, well-structured test-only PR. Code follows existing patterns, CI is green, scope is tight, and test coverage is thorough (positive and negative cases for get/generate/scan stages). The .trivyignore is well-scoped with clear rationale comments.

Finding

[MEDIUM] Missing Signed-off-by trailer — Commit d718b28 does not include a Signed-off-by trailer. Constitution "Commit Trailers" section: "All commits MUST include a Signed-off-by trailer." Fix with git commit --amend -s and force-push.

This review was generated by /review-pr (AI-assisted).

@gvauter gvauter force-pushed the feat/opa-cross-repo-validation branch from d718b28 to f311157 Compare June 18, 2026 13:05
Comment thread .trivyignore Outdated
Add end-to-end OPA provider validation to the cross-repo integration
test pipeline. The tests exercise the full complyctl + OPA provider
workflow using the existing mock registry test data (K8s container
security policies).

Changes:
- Add conftest install step to CI workflow
- Add OPA provider binary prerequisite check and install
- Add K8s Deployment test fixtures (non-compliant and compliant)
- Add test_get_opa: verifies OPA policy + complypack pulled
- Add test_generate_opa: verifies generate with OPA provider
- Add test_scan_opa: scans non-compliant fixture, asserts failures
- Add test_scan_opa_compliant: scans compliant fixture, asserts pass
- Rename test-opa-bp to test-opa-k8s (not branch protection)

Ref: complytime/complytime-providers#73
Signed-off-by: George Vauter <gvauter@redhat.com>
@gvauter gvauter force-pushed the feat/opa-cross-repo-validation branch from f311157 to 9abb608 Compare June 18, 2026 15:43
@gvauter

gvauter commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

Thanks for the review @gxmiranda. Signed-off-by trailer has been added.

@trevor-vaughan trevor-vaughan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gxmiranda gxmiranda left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marcusburghardt marcusburghardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants