add qa-infra-automation as a provider via interoperability by slickwarren · Pull Request #552 · rancher/tests

slickwarren · 2026-03-09T19:08:10Z

This PR connects rancher/tests -> rancher/qa-infra-automation by adding it as a client to rancher/tests/interoperability.

Feedback on design is welcome. Overall, it looks like this:

clone rancher/qa-infra-automation (An alternative to this is, we could release versions and put it in interoperability/go.mod)
add qaInfraAutomation to your config (see interoperability/README.md for examples / placeholders)
add a provisioning option to your test (see validation/neuvector/neuvector-hardened-test.go for an example)

in principle, this should only be used as a 'setup' option, when a test needs access to a downstream cluster to do some tests.

leaving the neuvector test in for now to provide an example, but for complexity's sake I can put this in a separate PR upon request.

Copilot

Pull request overview

This PR introduces a new interoperability provider (interoperability/qainfraautomation) that lets rancher/tests orchestrate infrastructure + downstream cluster setup through a local clone of rancher-qa-infra-automation, and adds a NeuVector “hardened custom cluster” validation suite as a concrete usage example.

Changes:

Added a new interoperability/qainfraautomation package (config schema, OpenTofu runner, Ansible runner, provisioning orchestration).
Added a NeuVector hardened test suite + accompanying docs/config example under validation/neuvector.
Updated UI plugin install flow to tolerate watch errors when the app is already deployed; added NeuVector UI extension constant and new chart-install helpers for NeuVector hardened options.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
validation/neuvector/neuvector_hardened_test.go	Adds an example suite that provisions a hardened custom downstream cluster via qainfraautomation and installs NeuVector + UI extension.
validation/neuvector/aws_custom_cluster_config.yaml	Adds an example cattle config for AWS-backed custom cluster provisioning via qainfraautomation.
validation/neuvector/README.md	Documents prerequisites/config for the NeuVector validation tests using qainfraautomation.
interoperability/qainfraautomation/provisioning.go	Implements orchestration for provisioning Rancher/custom/standalone clusters via OpenTofu + Ansible.
interoperability/qainfraautomation/tofu/tofu.go	Adds a thin OpenTofu CLI wrapper used by the provisioning layer.
interoperability/qainfraautomation/ansible/ansible.go	Adds an Ansible CLI wrapper (inventory rendering, ssh-add, playbook execution).
interoperability/qainfraautomation/config/config.go	Defines the qainfraautomation config model loaded from cattle config.
interoperability/qainfraautomation/README.md	Documents qainfraautomation configuration and supported flows.
interoperability/charts/constants.go	Adds a NeuVector UI extension name constant for interoperability tests.
actions/uiplugins/uiplugins.go	Adds fallback verification when the watch for UI plugin install ends with an error.
actions/charts/neuvector_new.go	Adds NeuVector install helpers with hardened/K3s value overrides and chart-state polling.
actions/charts/charts.go	Extends chart payload options with `K3s` and `Hardened` flags (used by NeuVector hardened flow).
interoperability/go.mod	Updates the interoperability module dependency set.
interoperability/go.sum	Updates the interoperability module checksum set.

interoperability/qainfraautomation/ansible/ansible.go

validation/neuvector/neuvector_hardened_test.go

validation/neuvector/aws_custom_cluster_config.yaml

interoperability/qainfraautomation/provisioning.go

interoperability/go.mod

validation/neuvector/README.md

Copilot · 2026-03-09T19:21:56Z

interoperability/qainfraautomation/tofu/tofu.go

+func (c *Client) logOutputs() {
+	out, err := c.run("tofu", []string{"output", "-json"}, c.moduleDir, nil)
+	if err != nil {
+		logrus.Warnf("[tofu] could not retrieve outputs after apply in %s: %v", c.moduleDir, err)
+		return
+	}
+	var outputs map[string]struct {
+		Value any `json:"value"`
+	}
+	if err := json.Unmarshal(out, &outputs); err != nil {
+		logrus.Warnf("[tofu] could not parse outputs JSON after apply in %s: %v", c.moduleDir, err)
+		return
+	}
+	if len(outputs) == 0 {
+		logrus.Infof("[tofu] apply outputs: (none) in %s (workspace=%s)", c.moduleDir, c.workspace)
+		return
+	}
+	for k, v := range outputs {
+		logrus.Infof("[tofu] apply output %q = %v (workspace=%s)", k, v.Value, c.workspace)
+	}
+}


logOutputs() logs every OpenTofu output value at INFO level. Tofu outputs frequently contain sensitive material (kubeconfigs, tokens, keys), so this can leak secrets into CI logs. Prefer logging only output names, or parse the sensitive flag from tofu output -json and redact/skip sensitive values (and/or lower to DEBUG with redaction).

I don't think this is important to add on this PR, but if it becomes an issue we can address it later.

Agreed. We kind of need an entire logging overhaul AND security overhaul, but neither are too important to dedicate our limited time and resources to at the moment.

interoperability/qainfraautomation/provisioning.go

hamistao

Addressing the overall design, I think if we can avoid the need for cloning rancher/qa-infra-automation, we should. I really like your suggestion of adding it to go.mod, and perhaps instead of doing releases we can have automated pull requests to ensure we are constantly using HEAD on the main branch, similarly to what we do with shepherd.

Also, I do think it would be simpler to review having the neuvector example in a separate PR as you said, but I am not requiring you to move that, feel free to leave it here you wish.

Lastly, I know this is not ready for a full review, but I left some comments that hopefully will de helpful.

If you want to talk more about the design we can schedule a meeting some time today to go over it more.

interoperability/qainfraautomation/README.md

interoperability/qainfraautomation/ansible/ansible.go

interoperability/qainfraautomation/tofu/tofu.go

hamistao

I received a request review so dropped in some more comments. But during the review I saw some comments from the previous one are unresolved so I guess I came too early.

actions/charts/neuvector_new.go

actions/uiplugins/uiplugins.go

hamistao · 2026-03-16T12:42:10Z

interoperability/qainfraautomation/ansible/ansible.go

+// RunPlaybook executes ansible-playbook with the given playbook and inventory paths, streaming output via logrus.
+func (c *Client) RunPlaybook(playbookPath, inventoryPath string, extraEnv []string) error {
+	absPlaybook := filepath.Join(c.repoPath, playbookPath)
+	absInventory := filepath.Join(c.repoPath, inventoryPath)


What do you think of embedding the inventory in the client object?

floatingman

Just a few comments. There seems to be inconsistent use of context.TODO() and context.Background() throughout the code. I'd prefer context.Background() if the context is not just a placeholder.

interoperability/qainfraautomation/provisioning.go

actions/charts/neuvector.go

actions/charts/neuvector_new.go

floatingman · 2026-03-16T17:26:21Z

validation/neuvector/neuvector_hardened_test.go

+const (
+	uiPluginChartsRepoName = "rancher-ui-plugins"
+	uiPluginChartsURL      = "https://github.com/rancher/ui-plugin-charts"
+	uiPluginChartsBranch   = "main"


Why is this hardcoded to main? Is it possible to make this a variable so other versions can be tested?

yes, but I think that should be in a separate PR. I think in that case we would want a dedicated test for the UI extension, since its not really being tested here but only used in setup.

…e repo

slickwarren requested a review from Copilot March 9, 2026 19:08

Copilot started reviewing on behalf of slickwarren March 9, 2026 19:11 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

slickwarren force-pushed the cwarren/qainfra-interoperability branch from a06a31c to 86ebff3 Compare March 9, 2026 20:05

slickwarren mentioned this pull request Mar 9, 2026

Add custom cluster hardening support rancher/qa-infra-automation#72

Draft

hamistao requested changes Mar 11, 2026

View reviewed changes

slickwarren mentioned this pull request Mar 11, 2026

introduce go versioning to import to other go projects rancher/qa-infra-automation#74

Merged

slickwarren requested review from Copilot, hamistao and rancher-max and removed request for Copilot March 12, 2026 16:24

Copilot started reviewing on behalf of slickwarren March 12, 2026 16:25 View session

slickwarren marked this pull request as ready for review March 12, 2026 16:25

slickwarren requested a review from a team as a code owner March 12, 2026 16:25

slickwarren force-pushed the cwarren/qainfra-interoperability branch from 8008e8a to 3946043 Compare March 12, 2026 18:39

hamistao requested changes Mar 16, 2026

View reviewed changes

floatingman reviewed Mar 16, 2026

View reviewed changes

floatingman requested a review from lscalabrini01 March 16, 2026 17:36

slickwarren requested a review from hamistao March 17, 2026 18:32

lscalabrini01 previously approved these changes Mar 23, 2026

View reviewed changes

slickwarren dismissed lscalabrini01’s stale review via 138cc39 March 23, 2026 15:02

slickwarren force-pushed the cwarren/qainfra-interoperability branch 9 times, most recently from bb95097 to b154995 Compare March 23, 2026 19:06

slickwarren force-pushed the cwarren/qainfra-interoperability branch 5 times, most recently from de164cd to fd9e5d1 Compare March 24, 2026 20:04

github-actions added 22 commits March 24, 2026 13:04

init commit with copilot -- qainfra provisioning

ea00ba3

adjust for downstream rancher provisioning

049bd35

cleanup changes

1cf43ab

allow generic custom cluster provisioning

b100ca4

cleaning up comments; add harden support for chart

756918a

addressing copilot PR review

83d05dc

use versioning for qa-infra-automation instead of directly cloning th…

4fbd036

…e repo

addressing comments on formatting

7baa602

addressing cleanup inconsistencies + formatting

305230c

fix ui wait

e739ffd

go version bump

ef86449

bump gomod

ebc6cd4

adjust e2e to not depend on corral

ba89af8

use a new dockerfile to allow using tofu

c9bc9ee

fix path

a1dc07b

fix path

400a13e

rm build

37a5ad3

remove volume dependency

1c563a7

eval ssh

4fd3f14

revert ssh-add

059727b

trying to fix ssh agent

2ab6918

fix cleanup

3e0672a

slickwarren force-pushed the cwarren/qainfra-interoperability branch from fd9e5d1 to 3e0672a Compare March 24, 2026 20:04

github-actions added 2 commits March 24, 2026 13:32

trying to remove privatekeypath

ce069a7

update to use latest chart again

6e84a65

Conversation

slickwarren commented Mar 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

slickwarren Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

rancher-max Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hamistao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hamistao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hamistao Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

floatingman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

floatingman Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

slickwarren Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants