Skip to content

OSAC-3273, OSAC-3280: Volume reconciler, controller, and feedback controller - #223

Closed
akshaynadkarni wants to merge 13 commits into
osac-project:mainfrom
akshaynadkarni:feat/OSAC-2872-volume-controller
Closed

OSAC-3273, OSAC-3280: Volume reconciler, controller, and feedback controller#223
akshaynadkarni wants to merge 13 commits into
osac-project:mainfrom
akshaynadkarni:feat/OSAC-2872-volume-controller

Conversation

@akshaynadkarni

@akshaynadkarni akshaynadkarni commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the Volume lifecycle controllers (PR 2 of 3 for the storage control
plane), building on the data model from
#201.

  • fulfillment-service: Volume reconciler watches for Volume events via gRPC
    streaming, creates/patches/deletes Volume CRs on the hub cluster. Maps proto
    VolumeAccessMode enum to CRD typed strings. Selects a hub randomly (same as
    ComputeInstance, since volumes have no parent resource to inherit from).
  • osac-operator: Volume resource controller with VendorProvisioner interface
    (nil for this PR, real vendor CSI integration in a follow-up). Feedback
    controller syncs CR status back to the fulfillment-service
    (Progressing->CREATING, Ready->AVAILABLE, Failed->FAILED,
    Deleting->DELETING) plus vendorVolumeID, backend, and protocol.
  • Helm: RBAC for volumes resource, OSAC_VOLUME_NAMESPACE env var,
    hub-access clusterrole updated.

Why

#201 established the Volume
data model (proto, DB, server, CRD types). This PR adds the controllers that
drive the provisioning lifecycle end-to-end: the fulfillment-service creates
Volume CRs on the hub, the operator controller calls the vendor to provision,
and the feedback controller syncs the result back. The VendorProvisioner is nil
for now (volumes stay in Progressing), with the real vendor CSI client planned
for a follow-up PR once the CSI driver integration PRs land.

What's included

fulfillment-service

  • internal/controllers/volume/ (reconciler function, tests, mocks)
  • internal/kubernetes/labels/kubernetes_labels.go (VolumeUuid label)
  • internal/cmd/service/start/controller/start_controller_cmd.go (wiring)

osac-operator

  • internal/controller/volume_controller.go (VendorProvisioner interface,
    provisioning lifecycle, management-state skip, status retry with NotFound
    handling on delete)
  • internal/controller/volume_feedback_controller.go (feedback.Bridge,
    phase-to-state mapping, vendor field sync with explicit protocol enum
    conversion via crdProtocolToProto)
  • internal/controller/volume_mock_provisioner.go (test mock with
    deterministic IDs and call counting)
  • internal/controller/volume_names.go (labels, finalizers, namespace default)
  • cmd/main.go (setupVolumeControllers under ctrlFlags.Storage)
  • charts/operator/templates/ (deployment, clusterrole, hub-access)

Docs

  • AGENTS.md Resources Managed: Volume added
  • README.md CRD list: Volume added
  • architecture-patterns.md Resource Hierarchy: Storage Resources section

Dependencies

  • CSI driver integration:
    #141 (merged),
    #94 (open)

Testing

# fulfillment-service: 87 suites pass (27 Volume-specific)
cd fulfillment-service && go run github.com/onsi/ginkgo/v2/ginkgo run -r internal

# osac-operator: all tests pass, 73% coverage
cd osac-operator && make test

# lint: 0 issues on both components
cd osac-operator && make lint
cd fulfillment-service && uv run dev.py lint go

# manifests unchanged
cd osac-operator && make manifests generate && git diff --exit-code

Ticket

Feature: OSAC-2872
(Storage Control Plane)

Epics:

Tasks covered in this PR:

  • OSAC-3276: Volume reconciler
    (fulfillment-service to hub Volume CR)
  • OSAC-3282: Volume resource
    controller
  • OSAC-3283: Volume feedback
    controller

Userflow: OSAC-3371
(hub PVC create happy path)


Signed-off-by: akshaynadkarni 25892229+akshaynadkarni@users.noreply.github.com
Assisted-by: Cursor/Claude

Summary by CodeRabbit

  • New Features
    • Added end-to-end volume lifecycle management, including provisioning, deletion, status tracking, finalizers, and retry handling.
    • Added synchronization of volume status and vendor details between clusters and the fulfillment service.
    • Added namespace-aware volume filtering and startup configuration.
    • Updated access permissions for volume resources and status endpoints.
  • Documentation
    • Documented volumes as supported block-storage resources.
  • Tests
    • Added comprehensive coverage for provisioning, deletion, synchronization, validation, and failure scenarios.

@openshift-ci-robot

openshift-ci-robot commented Aug 10, 2026

Copy link
Copy Markdown

@akshaynadkarni: This pull request references OSAC-2872 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the feature to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Adds the Volume lifecycle controllers (PR 2 of 3 for the storage control plane). Stacked on #201 (Volume API + CRD, data model).

  • fulfillment-service: Volume reconciler watches for Volume events, creates/patches/deletes Volume CRs on the hub cluster, maps proto VolumeAccessMode enum to CRD typed strings
  • osac-operator: Volume resource controller with VendorProvisioner interface (nil for this PR, vendor CSI integration in PR OSAC-1733: Merge fulfillment-service and osac-operator into osac mono-repo #3), feedback controller syncs CR status back to fulfillment-service (phase-to-state: Progressing->CREATING, Ready->AVAILABLE, Failed->FAILED, Deleting->DELETING, plus vendorVolumeID/backend/protocol)
  • Helm: RBAC for volumes resource, OSAC_VOLUME_NAMESPACE env var, hub-access clusterrole

Why

PR #201 established the Volume data model (proto, DB, server, CRD types). This PR adds the controllers that drive the provisioning lifecycle end-to-end: fulfillment-service creates Volume CRs on the hub, the operator controller calls the vendor to provision, and the feedback controller syncs the result back. The VendorProvisioner is nil for now (volumes stay in Progressing), with the real vendor CSI client planned for PR #3 after Roy's CSI driver PRs land.

Testing

# fulfillment-service: 87 suites pass (27 Volume-specific)
ginkgo run -r internal

# osac-operator: 653 tests pass, 0 failures, 73.1% coverage
make test

# lint: 0 issues on both components
make lint  # osac-operator
uv run dev.py lint go  # fulfillment-service

# manifests unchanged
make manifests generate && git diff --exit-code

Pre-merge ToDos

  1. PR OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 must merge first (Volume proto, CRD types)
  2. After OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 merges and BSR publishes, bump buf.gen.yaml in osac-operator to the new BSR version and regenerate
  3. Rebase on main after OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 merges

Related PRs

Ticket

OSAC-2872


Signed-off-by: akshaynadkarni 25892229+akshaynadkarni@users.noreply.github.com
Assisted-by: Cursor/Claude

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Added end-to-end Volume lifecycle support. Fulfillment reconciliation creates hub resources, the operator provisions vendor volumes, and feedback synchronization updates the fulfillment service. Startup wiring, RBAC, mocks, tests, labels, and documentation were added.

Changes

Volume lifecycle

Layer / File(s) Summary
Fulfillment volume reconciliation
fulfillment-service/internal/controllers/volume/..., fulfillment-service/internal/cmd/service/start/controller/start_controller_cmd.go
The fulfillment service validates volume data, selects hubs, creates or deletes hub Volume resources, manages finalizers, maps specifications, updates status, and starts the reconciler.
Operator provisioning controller
osac-operator/internal/controller/volume_controller.go, osac-operator/internal/controller/volume_names.go, osac-operator/cmd/main.go, osac-operator/config/rbac/role.yaml, osac-operator/charts/operator/templates/*
The operator manages Volume resources, calls the vendor provisioner, records lifecycle status, handles finalizers, filters namespaces, registers controllers, and grants required permissions.
Operator feedback synchronization
osac-operator/internal/controller/volume_feedback_controller.go, osac-operator/internal/controller/volume_feedback_controller_test.go, fulfillment-service/internal/controllers/volume/*_mock.go
The feedback controller maps Volume status and deletion phases to the private gRPC API, synchronizes vendor fields, signals deletion, and avoids redundant updates.
Validation and supporting updates
fulfillment-service/internal/controllers/volume/*_test.go, osac-operator/internal/controller/*volume*_test.go, osac-operator/README.md, osac-operator/AGENTS.md, fulfillment-service/internal/kubernetes/labels/kubernetes_labels.go
Tests cover reconciliation, provisioning, deletion, feedback synchronization, mocks, labels, and documentation. Existing fixtures now use pointer-valued FabricManager fields.

Estimated code review effort: 5 (Critical) | ~120 minutes

Mergeability Score: ⚪ Minimal · up to 515ad

The change adds volume lifecycle behavior, while the only remaining issue is a localized test robustness gap where setup errors are ignored. It has no direct production impact and presents no actionable merge-blocking risk beyond normal review.

Sequence Diagram(s)

sequenceDiagram
  participant FulfillmentVolumeReconciler
  participant HubKubernetesAPI
  participant OperatorVolumeReconciler
  participant VendorProvisioner
  participant VolumeFeedbackReconciler
  participant FulfillmentVolumesAPI
  FulfillmentVolumeReconciler->>HubKubernetesAPI: Create or patch Volume
  OperatorVolumeReconciler->>VendorProvisioner: CreateVolume
  VendorProvisioner-->>OperatorVolumeReconciler: Return vendor metadata
  OperatorVolumeReconciler->>HubKubernetesAPI: Persist status
  VolumeFeedbackReconciler->>HubKubernetesAPI: Read Volume status
  VolumeFeedbackReconciler->>FulfillmentVolumesAPI: Update remote status
Loading

Possibly related PRs

Suggested labels: requires-manual-review

Suggested reviewers: rgolangh

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 76.92% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the Volume reconciler, controller, and feedback controller added by the pull request.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
No-Hardcoded-Secrets ✅ Passed The complete PR diff adds no API keys, tokens, passwords, private keys, credential-bearing URLs, or base64/hex secret blobs; matches are only auth-related imports and non-secret test/config text.
No-Weak-Crypto ✅ Passed The PR diff adds no MD5, SHA-1, DES, RC4, Blowfish, ECB, custom crypto, or variable-time secret comparison; crypto imports remain TLS/RSA/SHA-256 only.
No-Injection-Vectors ✅ Passed Diff scans found no listed unsafe APIs or SQL concatenation; command execution uses exec.Command with separate arguments and no shell=True.
Container-Privileges ✅ Passed The PR adds no privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, allowPrivilegeEscalation, or root settings; deployment security contexts remain restrictive.
No-Sensitive-Data-In-Logs ✅ Passed Added logs contain reconciliation metadata and generic errors only; review found no passwords, tokens, API keys, PII, session IDs, customer payloads, or internal hostnames.
Ai-Attribution ✅ Passed AI use is declared in the PR and commits; all 17 OSAC-2872 commits have Assisted-by trailers, and none has an AI Co-Authored-By trailer.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@akshaynadkarni

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously requested changes Aug 10, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 15

🧹 Nitpick comments (11)
fulfillment-service/internal/controllers/volume/volume_reconciler_function.go (1)

270-274: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Route the Delete error through HandleK8sWriteError for consistency.

Create at Line 196 and Patch at Line 209 both convert terminal Kubernetes write errors into a FAILED status through controllers.HandleK8sWriteError. Delete returns the raw error. A terminal rejection on delete therefore retries forever and never reaches the volume status.

♻️ Proposed change
 	if object.GetDeletionTimestamp() == nil {
 		err = t.hubClient.Delete(ctx, object)
 		if err != nil {
-			return
+			return controllers.HandleK8sWriteError(ctx, t.r.logger, err, t.setFailed)
 		}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@fulfillment-service/internal/controllers/volume/volume_reconciler_function.go`
around lines 270 - 274, Update the Delete error path in the volume reconciler to
pass failures from t.hubClient.Delete through controllers.HandleK8sWriteError,
matching the existing Create and Patch handling and ensuring terminal write
errors update volume status instead of being returned raw.
fulfillment-service/internal/controllers/volume/volume_reconciler_function_test.go (1)

626-700: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Add coverage for the update branch that patches an existing CR.

This spec covers the Create failure path only. The Patch branch in volume_reconciler_function.go Lines 204-217 and the duplicate-CR error in getKubeObject Lines 350-356 have no spec. A fake client seeded with an existing CR that carries the labels.VolumeUuid label would cover the Patch branch cheaply.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@fulfillment-service/internal/controllers/volume/volume_reconciler_function_test.go`
around lines 626 - 700, Extend the “Kubernetes validation error handling” spec
to seed the fake client with an existing CR labeled by labels.VolumeUuid for the
test volume, then exercise f.run through the existing-object path so the Patch
branch in getKubeObject/volume reconciliation is covered. Assert the Invalid
error from patching produces the same FAILED state and validation message, and
add coverage for the duplicate-CR error returned by getKubeObject.
fulfillment-service/proto/private/osac/private/v1/volume_type.proto (1)

124-153: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

VolumeStatus.hub and backend are documented as private-only, but nothing enforces that.

The comments say backend and protocol are "Visible only through the private API". This file is under proto/private, so that holds by placement, not by a rule. If a public Volume type is added later and copies this message, the constraint is lost. Consider recording the restriction in the shared rendering/masking metadata instead of a comment only.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/proto/private/osac/private/v1/volume_type.proto` around
lines 124 - 153, Update the shared rendering/masking metadata for VolumeStatus
fields backend, protocol, and hub to explicitly mark them as private-only,
rather than relying solely on comments or the proto/private package location.
Use the existing metadata mechanism and preserve the current field definitions
and behavior.
fulfillment-service/internal/servers/private_volumes_server_test.go (2)

73-122: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Three near-identical volume factories.

createVolume equals createVolumeWithName("test-volume"). createStandaloneVolume equals createVolumeWithName("standalone-volume") with a different size. Collapse them into one helper that takes the name and the size.

♻️ Proposed consolidation
-		createVolume := func() *privatev1.Volume {
-			response, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
-				Object: privatev1.Volume_builder{
-					Metadata: privatev1.Metadata_builder{
-						Name: "test-volume",
-					}.Build(),
-					Spec: privatev1.VolumeSpec_builder{
-						StorageTier: "gold",
-						SizeGib:     100,
-						AccessMode:  privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
-					}.Build(),
-				}.Build(),
-			}.Build())
-			Expect(err).ToNot(HaveOccurred())
-			return response.GetObject()
-		}
-
-		createVolumeWithName := func(name string) *privatev1.Volume {
+		createVolumeWithNameAndSize := func(name string, sizeGib int64) *privatev1.Volume {
 			response, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
 				Object: privatev1.Volume_builder{
 					Metadata: privatev1.Metadata_builder{
 						Name: name,
 					}.Build(),
 					Spec: privatev1.VolumeSpec_builder{
 						StorageTier: "gold",
-						SizeGib:     100,
+						SizeGib:     sizeGib,
 						AccessMode:  privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
 					}.Build(),
 				}.Build(),
 			}.Build())
 			Expect(err).ToNot(HaveOccurred())
 			return response.GetObject()
 		}
 
-		createStandaloneVolume := func() *privatev1.Volume {
-			response, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
-				Object: privatev1.Volume_builder{
-					Metadata: privatev1.Metadata_builder{
-						Name: "standalone-volume",
-					}.Build(),
-					Spec: privatev1.VolumeSpec_builder{
-						StorageTier: "gold",
-						SizeGib:     50,
-						AccessMode:  privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
-					}.Build(),
-				}.Build(),
-			}.Build())
-			Expect(err).ToNot(HaveOccurred())
-			return response.GetObject()
+		createVolume := func() *privatev1.Volume {
+			return createVolumeWithNameAndSize("test-volume", 100)
 		}
+
+		createVolumeWithName := func(name string) *privatev1.Volume {
+			return createVolumeWithNameAndSize(name, 100)
+		}
+
+		createStandaloneVolume := func() *privatev1.Volume {
+			return createVolumeWithNameAndSize("standalone-volume", 50)
+		}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/internal/servers/private_volumes_server_test.go` around
lines 73 - 122, Consolidate the three local volume factory closures into a
single helper that accepts both the volume name and size. Update createVolume,
createVolumeWithName, and createStandaloneVolume call sites to use this helper
with their respective values, while preserving the shared storage tier, access
mode, request construction, error assertion, and returned object behavior.

235-263: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Update coverage misses the immutability guarantee and the lock path.

VolumeSpec is documented as immutable after creation. No test attempts a spec.* field-mask update to confirm the server rejects it. No test sets lock: true to confirm optimistic locking rejects a stale metadata.version.

Add both cases. A retrieved learning asks for comprehensive Update branch coverage on private servers in this package.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/internal/servers/private_volumes_server_test.go` around
lines 235 - 263, Add Update tests alongside the existing partial field-mask test
in the private volume server tests: attempt a spec.* field-mask update and
assert the server rejects it without mutating the immutable VolumeSpec, then set
lock: true with a stale metadata.version and assert optimistic locking rejects
the update. Reuse the existing createVolume and server.Update setup and verify
the expected error outcomes.

Source: Learnings

fulfillment-service/proto/private/osac/private/v1/volumes_service.proto (1)

31-43: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The filter example uses a magic enum number.

this.status.state == 2 requires the reader to look up that 2 is VOLUME_STATE_AVAILABLE. If the CEL evaluator accepts enum value names, use the name in the example. If it does not, name the state in the prose so the example stays readable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/proto/private/osac/private/v1/volumes_service.proto`
around lines 31 - 43, Update the filter example near the filter field to use the
readable enum name VOLUME_STATE_AVAILABLE if CEL supports enum value names;
otherwise keep the numeric comparison and explicitly identify 2 as
VOLUME_STATE_AVAILABLE in the surrounding prose.
osac-operator/api/v1alpha1/volume_types.go (1)

39-44: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicated kubebuilder enum markers produce redundant allOf blocks in both generated CRDs. VolumeAccessMode and VolumeProtocol each carry a +kubebuilder:validation:Enum marker on the named type and again on every field that uses the type. controller-gen emits both, so each generated property contains an allOf with two identical enum lists. The schema behaves correctly, but it is larger and harder to read, and the two copies can drift when a value is added to only one marker.

  • osac-operator/api/v1alpha1/volume_types.go#L39-L44: remove the field-level +kubebuilder:validation:Enum=ReadWriteOnce;ReadOnlyMany;ReadWriteMany;ReadWriteOncePod on AccessMode (line 42) and the matching field-level marker on Protocol (line 152); keep the markers on the VolumeAccessMode type (line 86) and the VolumeProtocol type (line 97), then run make manifests.
  • osac-operator/charts/operator-crds/templates/osac.openshift.io_volumes.yaml#L66-L78: regenerate so accessMode (lines 66-78) and protocol (lines 213-221) each carry a single flat enum instead of allOf.
  • osac-operator/config/crd/bases/osac.openshift.io_volumes.yaml#L65-L76: regenerate so accessMode (lines 65-76) and protocol (lines 212-219) each carry a single flat enum instead of allOf.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/api/v1alpha1/volume_types.go` around lines 39 - 44, Remove the
field-level Enum markers from AccessMode and Protocol in
osac-operator/api/v1alpha1/volume_types.go, while retaining the Enum markers on
the VolumeAccessMode and VolumeProtocol type definitions; then run make
manifests. Regenerate
osac-operator/charts/operator-crds/templates/osac.openshift.io_volumes.yaml
(accessMode lines 66-78 and protocol lines 213-221) and
osac-operator/config/crd/bases/osac.openshift.io_volumes.yaml (accessMode lines
65-76 and protocol lines 212-219) so each property has one flat enum rather than
a duplicated allOf.
osac-operator/api/v1alpha1/volume_names.go (1)

36-38: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Update VolumeCleanupFinalizer to describe only the constant’s future use.

VolumeCleanupFinalizer is declared but not added to any ClusterOrder in this PR. If cleanup logic remains pending, keep the constant but rewrite lines 36-37 to say it is intended for future use or the follow-up behavior instead of asserting it currently blocks cluster deletion.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/api/v1alpha1/volume_names.go` around lines 36 - 38, Update the
comment for VolumeCleanupFinalizer to describe it as reserved or intended for
future cleanup behavior, without claiming it is currently added to ClusterOrder
or blocks cluster deletion. Keep the constant value unchanged.
osac-operator/internal/controller/volume_mock_provisioner.go (1)

25-44: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Move the mock out of the production build.

The type carries a "test-only" comment, but the file is volume_mock_provisioner.go, not _test.go. The mock therefore compiles into the operator binary and any caller can wire it as a real VendorProvisioner. Rename the file to volume_mock_provisioner_test.go, or move it to a testing subpackage if other packages need it.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/internal/controller/volume_mock_provisioner.go` around lines 25
- 44, Move MockVendorProvisioner and NewMockVendorProvisioner out of the
production build by renaming volume_mock_provisioner.go to
volume_mock_provisioner_test.go; if external packages require the mock, place it
in a dedicated testing subpackage instead. Preserve its existing test behavior
and VendorProvisioner compatibility.
osac-operator/internal/controller/volume_controller.go (1)

250-254: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Use the embedded client for consistency.

handleUpdate calls r.Update, but this path calls r.mgr.GetLocalManager().GetClient().Update. The constructor assigns that same client to the embedded Client, so the behavior matches. Use r.Update here to keep one access path.

♻️ Proposed refactor
 	if controllerutil.RemoveFinalizer(vol, v1alpha1.VolumeFinalizer) {
-		if err := r.mgr.GetLocalManager().GetClient().Update(ctx, vol); err != nil {
+		if err := r.Update(ctx, vol); err != nil {
 			return ctrl.Result{}, err
 		}
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/internal/controller/volume_controller.go` around lines 250 -
254, Update the finalizer-removal path in handleUpdate to call the controller’s
embedded r.Update method instead of r.mgr.GetLocalManager().GetClient().Update,
preserving the existing context, volume object, and error-return behavior.
osac-operator/internal/controller/volume_feedback_controller_test.go (1)

408-461: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Guard the assertion reads with the same mutex.

Update and Signal write m.updates and m.signals on the gRPC server goroutine under m.mu. The specs read those slices directly with no lock. The RPC completes before the read, so the values are correct, but go test -race can still report the access as unsynchronized. Add small accessors that take the lock.

Separately, Signal appends the ID before it checks signalErr, so a failed signal is still recorded. Move the error check first if a test ever asserts on that.

♻️ Proposed refactor
+func (m *mockVolumesServer) updateCount() int {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	return len(m.updates)
+}
+
+func (m *mockVolumesServer) updateAt(i int) *privatev1.Volume {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	return m.updates[i]
+}
+
+func (m *mockVolumesServer) signalIDs() []string {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	return append([]string(nil), m.signals...)
+}
+
 func (m *mockVolumesServer) Signal(_ context.Context, req *privatev1.VolumesSignalRequest) (*privatev1.VolumesSignalResponse, error) {
 	m.mu.Lock()
 	defer m.mu.Unlock()
 
-	m.signals = append(m.signals, req.GetId())
-
 	if m.signalErr != nil {
 		return nil, m.signalErr
 	}
+	m.signals = append(m.signals, req.GetId())
 	return &privatev1.VolumesSignalResponse{}, nil
 }

Note: the test at line 289 expects one signal after a successful call, and the test at line 298 does not assert signals, so this reordering is safe.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/internal/controller/volume_feedback_controller_test.go` around
lines 408 - 461, Protect test assertions by adding mutex-guarded accessors for
the mockVolumesServer updates and signals slices, and use them instead of direct
reads. In Signal, check signalErr before appending the request ID so failed
signals are not recorded; preserve recording for successful calls.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@fulfillment-service/internal/controllers/volume/volume_reconciler_function.go`:
- Around line 304-307: Update the hub-selection logic in the reconciler function
to guard against a nil response before reading response.Items, returning the
existing “no hubs available” error when the response or its Items field is nil
or empty. Only call rand.IntN and select a hub after this validation.

In
`@fulfillment-service/internal/database/migrations/94_create_volumes_tables_test.go`:
- Around line 131-165: Update the immutability test around its existing id,
name, and tenant assertions to cover the project column as well: execute an
update of volumes.project and assert it fails with the same Z0001 error code.
Keep project in the test name only if this assertion is added.

In
`@fulfillment-service/internal/database/migrations/94_create_volumes_tables.up.sql`:
- Around line 71-75: Update the comment above trigger check_immutable_columns to
list name alongside id, tenant, and project, matching the columns passed to the
trigger and the enforced immutability behavior.

In `@fulfillment-service/internal/servers/private_volumes_server_test.go`:
- Around line 324-413: Move the five Create validation cases from the direct
server tests around server.Create into an integration test that invokes the full
gRPC interceptor chain. Add declarative buf.validate rules in the Volume schema
for required spec and non-zero/required access_mode, while preserving the
existing storage_tier, size_gib, and metadata.name constraints; do not add
duplicate validation to server handlers.

In `@fulfillment-service/internal/servers/private_volumes_server.go`:
- Around line 126-129: Update the Create flow in the private volumes server to
always replace the volume’s existing status with a new empty
privatev1.VolumeStatus, rather than only initializing it when nil. Then set the
fresh status state to VOLUME_STATE_CREATING, ensuring caller-provided
operational fields are discarded before generic.Create persists the volume.

In `@osac-operator/api/v1alpha1/volume_types.go`:
- Around line 33-37: Bound the requested volume size in the SizeGiB validation
markers using the platform’s maximum supported capacity, preventing overflow
during byte conversion. Apply the identical maximum to the size_gib field in the
volume_type.proto schema so API and fulfillment-service validation remain
consistent.
- Line 24: Align the pvcRef validation in the Volume spec with the status
contract: move the immutability rule from the pvcRef field to the Volume
struct-level validation, using optionalOldSelf when needed so API-created
volumes cannot later receive a PVC reference. If late CSI-driver population is
intentionally supported, preserve that exception and document it instead.

In `@osac-operator/cmd/main.go`:
- Around line 479-501: Resolve an empty volumeNamespace to
defaultVolumeNamespace once in setupVolumeControllers before constructing either
reconciler, so both NewVolumeFeedbackReconciler and NewVolumeReconciler receive
the same namespace; update osac-operator/cmd/main.go lines 479-501. No direct
change is required in
osac-operator/internal/controller/volume_feedback_controller.go lines 92-102
because the resolved value fixes its constructor input.

In `@osac-operator/internal/controller/volume_controller_test.go`:
- Around line 92-106: Rename the test case around the first reconcile in
volume_controller_test.go to state that it sets the phase to Ready, matching the
VolumePhaseReady assertion and its inline comment.

In `@osac-operator/internal/controller/volume_controller.go`:
- Around line 140-151: The Reconcile status update must tolerate the object
disappearing after handleDelete removes the finalizer. In
osac-operator/internal/controller/volume_controller.go:140-151, wrap
updateStatusWithRetry in client.IgnoreNotFound (or skip the write when deletion
completed). In
osac-operator/internal/controller/volume_controller_test.go:217-244, ensure the
test reaches the post-delete status write and retain the
Expect(err).ToNot(HaveOccurred()) assertion as the regression guard.
- Around line 200-205: Update the provisioning error path in the volume
reconciliation handler to return a delayed requeue result instead of an empty
result with nil error. Preserve the existing failure status and condition
updates, and use the controller’s established backoff duration if one is
available so transient vendor failures retry while the condition remains
visible.
- Around line 297-315: Update setVendorProvisionedCondition to delegate
condition insertion and updates to apimeta.SetStatusCondition, so unchanged
statuses still refresh Reason and Message while LastTransitionTime changes only
on status transitions. Preserve the existing VendorProvisioned condition type
and input fields.

In `@osac-operator/internal/controller/volume_feedback_controller_test.go`:
- Around line 80-98: Close the gRPC client connection during AfterEach by
retaining the connection created in BeforeEach and calling its close method
during teardown. Declare the connection and error separately to avoid shadowing
the outer connection variable, and guard the close when no connection was
created.
- Around line 183-201: In the volume feedback controller tests, replace the
thinking-out-loud comments in “should not overwrite remote fields when CR fields
are empty” with a concise conclusion explaining that no update is expected. In
the vendor-field propagation test, assert updated.GetStatus().GetProtocol()
equals privatev1.StorageProtocol_STORAGE_PROTOCOL_NFS to verify the protocol
mapping.

In `@osac-operator/internal/controller/volume_feedback_controller.go`:
- Around line 163-165: Replace the StorageProtocol_value lookup in
volume_feedback_controller.go:163-165 with an explicit switch mapping each
v1alpha1.VolumeProtocol value to its corresponding privatev1.StorageProtocol_*
constant. Update volume_feedback_controller_test.go:183-201 to assert the
propagated GetProtocol() value in the vendor-field test, covering the mapping
for CRD protocols such as Block and NFS.

---

Nitpick comments:
In
`@fulfillment-service/internal/controllers/volume/volume_reconciler_function_test.go`:
- Around line 626-700: Extend the “Kubernetes validation error handling” spec to
seed the fake client with an existing CR labeled by labels.VolumeUuid for the
test volume, then exercise f.run through the existing-object path so the Patch
branch in getKubeObject/volume reconciliation is covered. Assert the Invalid
error from patching produces the same FAILED state and validation message, and
add coverage for the duplicate-CR error returned by getKubeObject.

In
`@fulfillment-service/internal/controllers/volume/volume_reconciler_function.go`:
- Around line 270-274: Update the Delete error path in the volume reconciler to
pass failures from t.hubClient.Delete through controllers.HandleK8sWriteError,
matching the existing Create and Patch handling and ensuring terminal write
errors update volume status instead of being returned raw.

In `@fulfillment-service/internal/servers/private_volumes_server_test.go`:
- Around line 73-122: Consolidate the three local volume factory closures into a
single helper that accepts both the volume name and size. Update createVolume,
createVolumeWithName, and createStandaloneVolume call sites to use this helper
with their respective values, while preserving the shared storage tier, access
mode, request construction, error assertion, and returned object behavior.
- Around line 235-263: Add Update tests alongside the existing partial
field-mask test in the private volume server tests: attempt a spec.* field-mask
update and assert the server rejects it without mutating the immutable
VolumeSpec, then set lock: true with a stale metadata.version and assert
optimistic locking rejects the update. Reuse the existing createVolume and
server.Update setup and verify the expected error outcomes.

In `@fulfillment-service/proto/private/osac/private/v1/volume_type.proto`:
- Around line 124-153: Update the shared rendering/masking metadata for
VolumeStatus fields backend, protocol, and hub to explicitly mark them as
private-only, rather than relying solely on comments or the proto/private
package location. Use the existing metadata mechanism and preserve the current
field definitions and behavior.

In `@fulfillment-service/proto/private/osac/private/v1/volumes_service.proto`:
- Around line 31-43: Update the filter example near the filter field to use the
readable enum name VOLUME_STATE_AVAILABLE if CEL supports enum value names;
otherwise keep the numeric comparison and explicitly identify 2 as
VOLUME_STATE_AVAILABLE in the surrounding prose.

In `@osac-operator/api/v1alpha1/volume_names.go`:
- Around line 36-38: Update the comment for VolumeCleanupFinalizer to describe
it as reserved or intended for future cleanup behavior, without claiming it is
currently added to ClusterOrder or blocks cluster deletion. Keep the constant
value unchanged.

In `@osac-operator/api/v1alpha1/volume_types.go`:
- Around line 39-44: Remove the field-level Enum markers from AccessMode and
Protocol in osac-operator/api/v1alpha1/volume_types.go, while retaining the Enum
markers on the VolumeAccessMode and VolumeProtocol type definitions; then run
make manifests. Regenerate
osac-operator/charts/operator-crds/templates/osac.openshift.io_volumes.yaml
(accessMode lines 66-78 and protocol lines 213-221) and
osac-operator/config/crd/bases/osac.openshift.io_volumes.yaml (accessMode lines
65-76 and protocol lines 212-219) so each property has one flat enum rather than
a duplicated allOf.

In `@osac-operator/internal/controller/volume_controller.go`:
- Around line 250-254: Update the finalizer-removal path in handleUpdate to call
the controller’s embedded r.Update method instead of
r.mgr.GetLocalManager().GetClient().Update, preserving the existing context,
volume object, and error-return behavior.

In `@osac-operator/internal/controller/volume_feedback_controller_test.go`:
- Around line 408-461: Protect test assertions by adding mutex-guarded accessors
for the mockVolumesServer updates and signals slices, and use them instead of
direct reads. In Signal, check signalErr before appending the request ID so
failed signals are not recorded; preserve recording for successful calls.

In `@osac-operator/internal/controller/volume_mock_provisioner.go`:
- Around line 25-44: Move MockVendorProvisioner and NewMockVendorProvisioner out
of the production build by renaming volume_mock_provisioner.go to
volume_mock_provisioner_test.go; if external packages require the mock, place it
in a dedicated testing subpackage instead. Preserve its existing test behavior
and VendorProvisioner compatibility.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d1996f7-0a57-4cf6-8228-b79b6918df86

📥 Commits

Reviewing files that changed from the base of the PR and between 58e38f5 and 9c3afdd.

⛔ Files ignored due to path filters (21)
  • fulfillment-service/internal/api/osac/private/v1/event_type.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/event_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/storage_common_type.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/storage_common_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/storage_tier_type.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/storage_tier_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/volume_type.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/volume_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/volumes_service.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/volumes_service.pb.gw.go is excluded by !**/*.pb.gw.go
  • fulfillment-service/internal/api/osac/private/v1/volumes_service_grpc.pb.go is excluded by !**/*.pb.go
  • fulfillment-service/internal/api/osac/private/v1/volumes_service_protoopaque.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/storage_common_type.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/storage_common_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/storage_tier_type.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/storage_tier_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/volume_type.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/volume_type_protoopaque.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/volumes_service.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/volumes_service_grpc.pb.go is excluded by !**/*.pb.go
  • osac-operator/internal/api/osac/private/v1/volumes_service_protoopaque.pb.go is excluded by !**/*.pb.go
📒 Files selected for processing (37)
  • fulfillment-service/internal/cmd/service/start/controller/start_controller_cmd.go
  • fulfillment-service/internal/cmd/service/start/grpcserver/start_grpc_server_cmd.go
  • fulfillment-service/internal/cmd/service/start/restgateway/start_rest_gateway_cmd.go
  • fulfillment-service/internal/controllers/volume/hubs_client_mock.go
  • fulfillment-service/internal/controllers/volume/volume_reconciler_function.go
  • fulfillment-service/internal/controllers/volume/volume_reconciler_function_test.go
  • fulfillment-service/internal/controllers/volume/volume_suite_test.go
  • fulfillment-service/internal/controllers/volume/volumes_client_mock.go
  • fulfillment-service/internal/database/migrations.sha256
  • fulfillment-service/internal/database/migrations/94_create_volumes_tables.up.sql
  • fulfillment-service/internal/database/migrations/94_create_volumes_tables_test.go
  • fulfillment-service/internal/kubernetes/labels/kubernetes_labels.go
  • fulfillment-service/internal/rendering/tables/osac.private.v1.Volume.yaml
  • fulfillment-service/internal/servers/private_volumes_server.go
  • fulfillment-service/internal/servers/private_volumes_server_test.go
  • fulfillment-service/proto/private/osac/private/v1/event_type.proto
  • fulfillment-service/proto/private/osac/private/v1/storage_common_type.proto
  • fulfillment-service/proto/private/osac/private/v1/storage_tier_type.proto
  • fulfillment-service/proto/private/osac/private/v1/volume_type.proto
  • fulfillment-service/proto/private/osac/private/v1/volumes_service.proto
  • osac-operator/api/v1alpha1/groupversion_info.go
  • osac-operator/api/v1alpha1/volume_names.go
  • osac-operator/api/v1alpha1/volume_types.go
  • osac-operator/api/v1alpha1/zz_generated.deepcopy.go
  • osac-operator/charts/operator-crds/templates/osac.openshift.io_volumes.yaml
  • osac-operator/charts/operator/templates/clusterrole.yaml
  • osac-operator/charts/operator/templates/deployment.yaml
  • osac-operator/charts/operator/templates/hub-access-clusterrole.yaml
  • osac-operator/cmd/main.go
  • osac-operator/config/crd/bases/osac.openshift.io_volumes.yaml
  • osac-operator/config/rbac/role.yaml
  • osac-operator/internal/controller/volume_controller.go
  • osac-operator/internal/controller/volume_controller_test.go
  • osac-operator/internal/controller/volume_feedback_controller.go
  • osac-operator/internal/controller/volume_feedback_controller_test.go
  • osac-operator/internal/controller/volume_mock_provisioner.go
  • osac-operator/internal/controller/volume_names.go

Comment thread fulfillment-service/internal/controllers/volume/volume_reconciler_function.go Outdated
Comment on lines +131 to +165
It("Enforces immutability of id, name, tenant, and project", func(ctx context.Context) {
err := tool.Migrate(ctx, 94)
Expect(err).ToNot(HaveOccurred())

_, err = conn.Exec(ctx,
`insert into volumes (id, name, tenant, data) values ($1, $2, $3, $4)`,
"immutable-id", "immutable-vol", "system", `{}`,
)
Expect(err).ToNot(HaveOccurred())

var pgErr *pgconn.PgError

_, err = conn.Exec(ctx,
`update volumes set id = $1 where id = $2`,
"changed-id", "immutable-id",
)
Expect(err).To(HaveOccurred())
Expect(errors.As(err, &pgErr)).To(BeTrue())
Expect(pgErr.Code).To(Equal("Z0001"))

_, err = conn.Exec(ctx,
`update volumes set name = $1 where id = $2`,
"changed-name", "immutable-id",
)
Expect(err).To(HaveOccurred())
Expect(errors.As(err, &pgErr)).To(BeTrue())
Expect(pgErr.Code).To(Equal("Z0001"))

_, err = conn.Exec(ctx,
`update volumes set tenant = $1 where id = $2`,
"other-tenant", "immutable-id",
)
Expect(err).To(HaveOccurred())
Expect(errors.As(err, &pgErr)).To(BeTrue())
Expect(pgErr.Code).To(Equal("Z0001"))

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Test the stated project immutability.

The test name includes project, but the test only updates id, name, and tenant. Add a failing update volumes set project = ... assertion if the migration makes project immutable. Otherwise, remove project from the test name.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@fulfillment-service/internal/database/migrations/94_create_volumes_tables_test.go`
around lines 131 - 165, Update the immutability test around its existing id,
name, and tenant assertions to cover the project column as well: execute an
update of volumes.project and assert it fails with the same Z0001 error code.
Keep project in the test name only if this assertion is added.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is on code from the base PR #201. Will address in that PR's review cycle.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshaynadkarni, acknowledged. This test originates in base PR #201. I will leave the change for that review cycle.


🧠 Learnings used
Learnt from: akshaynadkarni
Repo: osac-project/osac PR: 201
File: fulfillment-service/internal/database/migrations/94_create_volumes_tables_test.go:35-46
Timestamp: 2026-08-07T20:01:15.978Z
Learning: In Go migration tests under fulfillment-service/internal/database/migrations, do not flag SQL constructed with fmt.Sprintf and pgx.Identifier.Sanitize() as SQL injection when the table-name identifier comes from a hardcoded test constant and is not externally controlled. This established pattern is used in files such as 73_create_nat_gateways_tables_test.go and 62_create_storage_backends_tables_test.go.

You are interacting with an AI system.

Comment on lines +71 to +75
-- Enforce immutability of id, tenant, and project columns at the database level.
create trigger check_immutable_columns
before update on volumes
for each row
execute function check_immutable_columns('id', 'name', 'tenant', 'project');

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

The comment omits name, which the trigger does enforce.

Line 71 lists "id, tenant, and project". Line 75 passes 'id', 'name', 'tenant', 'project'. The test in 94_create_volumes_tables_test.go confirms that a name update raises Z0001. Fix the comment.

📝 Proposed comment fix
--- Enforce immutability of id, tenant, and project columns at the database level.
+-- Enforce immutability of the id, name, tenant, and project columns at the database level.
 create trigger check_immutable_columns
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
-- Enforce immutability of id, tenant, and project columns at the database level.
create trigger check_immutable_columns
before update on volumes
for each row
execute function check_immutable_columns('id', 'name', 'tenant', 'project');
-- Enforce immutability of the id, name, tenant, and project columns at the database level.
create trigger check_immutable_columns
before update on volumes
for each row
execute function check_immutable_columns('id', 'name', 'tenant', 'project');
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@fulfillment-service/internal/database/migrations/94_create_volumes_tables.up.sql`
around lines 71 - 75, Update the comment above trigger check_immutable_columns
to list name alongside id, tenant, and project, matching the columns passed to
the trigger and the enforced immutability behavior.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is on code from the base PR #201. Will address in that PR's review cycle.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshaynadkarni, understood. This comment applies to the migration introduced in base PR #201. I will leave it for that PR’s review cycle.

You are interacting with an AI system.

Comment on lines +324 to +413
Describe("Validation", func() {
It("Create without metadata.name fails", func() {
_, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
Object: privatev1.Volume_builder{
Spec: privatev1.VolumeSpec_builder{
StorageTier: "gold",
SizeGib: 100,
AccessMode: privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
}.Build(),
}.Build(),
}.Build())
Expect(err).To(HaveOccurred())
st, ok := status.FromError(err)
Expect(ok).To(BeTrue())
Expect(st.Code()).To(Equal(codes.InvalidArgument))
Expect(st.Message()).To(ContainSubstring("metadata.name"))
})

It("Create without storage_tier fails", func() {
_, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
Object: privatev1.Volume_builder{
Metadata: privatev1.Metadata_builder{
Name: "test-volume",
}.Build(),
Spec: privatev1.VolumeSpec_builder{
SizeGib: 100,
AccessMode: privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
}.Build(),
}.Build(),
}.Build())
Expect(err).To(HaveOccurred())
st, ok := status.FromError(err)
Expect(ok).To(BeTrue())
Expect(st.Code()).To(Equal(codes.InvalidArgument))
Expect(st.Message()).To(ContainSubstring("storage_tier"))
})

It("Create without size_gib fails", func() {
_, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
Object: privatev1.Volume_builder{
Metadata: privatev1.Metadata_builder{
Name: "test-volume",
}.Build(),
Spec: privatev1.VolumeSpec_builder{
StorageTier: "gold",
AccessMode: privatev1.VolumeAccessMode_VOLUME_ACCESS_MODE_READ_WRITE_ONCE,
}.Build(),
}.Build(),
}.Build())
Expect(err).To(HaveOccurred())
st, ok := status.FromError(err)
Expect(ok).To(BeTrue())
Expect(st.Code()).To(Equal(codes.InvalidArgument))
Expect(st.Message()).To(ContainSubstring("size_gib"))
})

It("Create without access_mode fails", func() {
_, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
Object: privatev1.Volume_builder{
Metadata: privatev1.Metadata_builder{
Name: "test-volume",
}.Build(),
Spec: privatev1.VolumeSpec_builder{
StorageTier: "gold",
SizeGib: 100,
}.Build(),
}.Build(),
}.Build())
Expect(err).To(HaveOccurred())
st, ok := status.FromError(err)
Expect(ok).To(BeTrue())
Expect(st.Code()).To(Equal(codes.InvalidArgument))
Expect(st.Message()).To(ContainSubstring("access_mode"))
})

It("Create without spec fails", func() {
_, err := server.Create(ctx, privatev1.VolumesCreateRequest_builder{
Object: privatev1.Volume_builder{
Metadata: privatev1.Metadata_builder{
Name: "test-volume",
}.Build(),
}.Build(),
}.Build())
Expect(err).To(HaveOccurred())
st, ok := status.FromError(err)
Expect(ok).To(BeTrue())
Expect(st.Code()).To(Equal(codes.InvalidArgument))
Expect(st.Message()).To(ContainSubstring("spec"))
})
})

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

These validation tests call the server directly, so the protovalidate interceptor never runs.

All five cases assert codes.InvalidArgument for constraints declared with buf.validate in volume_type.proto. server.Create is invoked directly here, which bypasses the gRPC interceptor chain that applies those rules.

Two of the cases cannot pass even with the interceptor. access_mode is only guarded by enum.defined_only, which accepts the zero value, and Volume.spec carries no required rule. See the comment on fulfillment-service/proto/private/osac/private/v1/volume_type.proto lines 101-122.

Move these cases to an integration test that exercises the full gRPC chain, and add the missing declarative rules.

Based on learnings: "treat validation expressible with buf.validate annotations as enforced by the protovalidate gRPC interceptor. Do not duplicate that validation in server handlers or direct server unit tests; test negative declarative Create-request cases in integration tests that exercise the complete gRPC chain".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/internal/servers/private_volumes_server_test.go` around
lines 324 - 413, Move the five Create validation cases from the direct server
tests around server.Create into an integration test that invokes the full gRPC
interceptor chain. Add declarative buf.validate rules in the Volume schema for
required spec and non-zero/required access_mode, while preserving the existing
storage_tier, size_gib, and metadata.name constraints; do not add duplicate
validation to server handlers.

Source: Learnings

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is on code from the base PR #201, not this controller PR. Will address there.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshaynadkarni, acknowledged. This validation issue belongs to base PR #201. I will treat it as out of scope for this controller PR.


🧠 Learnings used
Learnt from: ajamias
Repo: osac-project/osac PR: 40
File: fulfillment-service/internal/servers/private_baremetal_instance_types_server_test.go:59-73
Timestamp: 2026-07-31T18:16:29.969Z
Learning: In fulfillment-service Go tests, treat validation expressible with buf.validate annotations as enforced by the protovalidate gRPC interceptor. Do not duplicate that validation in server handlers or direct server unit tests; test negative declarative Create-request cases in integration tests that exercise the complete gRPC chain, consistent with fulfillment-service/AGENTS.md.

You are interacting with an AI system.

Comment on lines +126 to +129
if vol.GetStatus() == nil {
vol.SetStatus(&privatev1.VolumeStatus{})
}
vol.GetStatus().SetState(privatev1.VolumeState_VOLUME_STATE_CREATING)

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Discard caller-provided status during Create.

This code preserves caller-provided vendor_volume_id, backend, protocol, hub, and PV/PVC references. It only overwrites state. generic.Create can then persist unverified operational status data.

Create a fresh VolumeStatus before setting VOLUME_STATE_CREATING.

Proposed fix
-	if vol.GetStatus() == nil {
-		vol.SetStatus(&privatev1.VolumeStatus{})
-	}
-	vol.GetStatus().SetState(privatev1.VolumeState_VOLUME_STATE_CREATING)
+	status := &privatev1.VolumeStatus{}
+	status.SetState(privatev1.VolumeState_VOLUME_STATE_CREATING)
+	vol.SetStatus(status)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if vol.GetStatus() == nil {
vol.SetStatus(&privatev1.VolumeStatus{})
}
vol.GetStatus().SetState(privatev1.VolumeState_VOLUME_STATE_CREATING)
status := &privatev1.VolumeStatus{}
status.SetState(privatev1.VolumeState_VOLUME_STATE_CREATING)
vol.SetStatus(status)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fulfillment-service/internal/servers/private_volumes_server.go` around lines
126 - 129, Update the Create flow in the private volumes server to always
replace the volume’s existing status with a new empty privatev1.VolumeStatus,
rather than only initializing it when nil. Then set the fresh status state to
VOLUME_STATE_CREATING, ensuring caller-provided operational fields are discarded
before generic.Create persists the volume.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is on code from the base PR #201, not this controller PR. Will address there.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshaynadkarni Understood. This finding belongs to base PR #201, not this controller PR. I will leave this comment open for validation against the fix in #201. No separate follow-up issue is needed while #201 tracks the change.

You are interacting with an AI system.

Comment on lines +200 to +205
if err != nil {
log.Error(err, "vendor provisioning failed")
vol.Status.Phase = v1alpha1.VolumePhaseFailed
setVendorProvisionedCondition(&vol.Status.Conditions, metav1.ConditionFalse, "ProvisioningFailed", err.Error())
return ctrl.Result{}, nil
}

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Failed provisioning never retries.

The handler returns nil, so controller-runtime does not requeue. No further event arrives for this CR, because nothing else changes it. A transient vendor outage therefore strands the volume in Failed forever. The condition is still visible if you requeue with a delay.

🐛 Proposed fix: requeue after a backoff delay
 	if err != nil {
 		log.Error(err, "vendor provisioning failed")
 		vol.Status.Phase = v1alpha1.VolumePhaseFailed
 		setVendorProvisionedCondition(&vol.Status.Conditions, metav1.ConditionFalse, "ProvisioningFailed", err.Error())
-		return ctrl.Result{}, nil
+		return ctrl.Result{RequeueAfter: vendorProvisioningRetryInterval}, nil
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@osac-operator/internal/controller/volume_controller.go` around lines 200 -
205, Update the provisioning error path in the volume reconciliation handler to
return a delayed requeue result instead of an empty result with nil error.
Preserve the existing failure status and condition updates, and use the
controller’s established backoff duration if one is available so transient
vendor failures retry while the condition remains visible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f0a6f21. Added an explicit guard so Failed volumes skip provisioning. Re-provisioning is triggered by Signal or periodic sync after the admin fixes the vendor issue.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- current revision ---'
git rev-parse --short HEAD

printf '%s\n' '--- volume controller outline ---'
ast-grep outline osac-operator/internal/controller/volume_controller.go --items all --type function,method --view expanded

printf '%s\n' '--- provisioning and phase guards ---'
rg -n -C 8 'VolumePhase(Failed|Ready)|handleProvisioning|CreateVolume' \
  osac-operator/internal/controller/volume_controller.go

printf '%s\n' '--- volume Signal and periodic sync references ---'
rg -n -i -C 5 'signal|periodic|sync' osac-operator \
  -g '*.go' -g '*.yaml' -g '*.yml'

Length of output: 201148


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- f0a6f211 diff ---'
git show --format=fuller --stat --oneline f0a6f211
git show --format= --unified=30 f0a6f211 -- osac-operator/internal/controller/volume_controller.go

printf '%s\n' '--- current volume controller, relevant range ---'
sed -n '145,235p' osac-operator/internal/controller/volume_controller.go

printf '%s\n' '--- Volume-specific signal and sync source references (exclude generated files) ---'
rg -n -i -C 4 'VolumesSignal|Volume.*Signal|Signal.*Volume|periodic.{0,80}(volume|sync)|(volume|sync).{0,80}periodic' \
  osac-operator/internal/controller \
  fulfillment-service/internal \
  -g '*.go' -g '!**/*.pb.go' -g '!**/*mock*.go'

printf '%s\n' '--- volume controller tests for failed phase and re-provisioning ---'
rg -n -C 6 'Failed|fail|Signal|re.?provision|CreateCallCount' \
  osac-operator/internal/controller/volume_controller_test.go

Length of output: 30529


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- f0a6f211 changes outside VolumeReconciler ---'
git show --format= --unified=35 f0a6f211 -- \
  fulfillment-service/internal/controllers/volume/volume_reconciler_function.go \
  osac-operator/internal/controller/volume_feedback_controller.go \
  osac-operator/internal/controller/volume_feedback_controller_test.go

printf '%s\n' '--- fulfillment Volume reconciler lifecycle mapping ---'
rg -n -C 8 'Signal|VOLUME_STATE_(FAILED|PENDING|CREATING)|VolumeState|Failed|Progressing|Patch|Update' \
  fulfillment-service/internal/controllers/volume/volume_reconciler_function.go

printf '%s\n' '--- feedback reconciler status-to-remote mapping ---'
sed -n '45,180p' osac-operator/internal/controller/volume_feedback_controller.go

printf '%s\n' '--- all Volume periodic runnable registrations ---'
rg -n -i -C 3 'volume.*(runnable|periodic|ticker|resync)|((runnable|periodic|ticker|resync).*volume)' \
  osac-operator/cmd osac-operator/internal/controller \
  -g '*.go' -g '!**/*.pb.go'

Length of output: 23022


@akshaynadkarni, I verified f0a6f211. The finding remains unresolved.

handleUpdate now returns early when vol.Status.Phase == VolumePhaseFailed. Volumes.Signal causes fulfillment-service reconciliation, but that reconciler only creates or patches the hub Volume specification. It does not reset Volume.Status.Phase to Progressing.

I also found no Volume periodic-sync runnable. Therefore, after a vendor error, a Volume stays Failed and every later reconcile skips CreateVolume.

Please reset the hub CR phase to Progressing through the intended retry action, or return a delayed requeue for retryable vendor errors.

You are interacting with an AI system.

Comment thread osac-operator/internal/controller/volume_controller.go
Comment thread osac-operator/internal/controller/volume_feedback_controller_test.go Outdated
Comment thread osac-operator/internal/controller/volume_feedback_controller.go
akshaynadkarni added a commit to akshaynadkarni/osac that referenced this pull request Aug 10, 2026
Fix five issues identified by CodeRabbit on PR osac-project#223:

1. (Critical) Protocol enum lookup: CRD uses "Block"/"NFS" but the proto
   map is keyed by "STORAGE_PROTOCOL_BLOCK". Replaced the map lookup with
   an explicit switch in syncVolumeVendorFields. Added test coverage for
   both Block and NFS protocol mapping.

2. (Major) Feedback controller namespace default: NewVolumeFeedbackReconciler
   now defaults to "osac-volume" when the namespace parameter is empty,
   matching NewVolumeReconciler's behavior. Previously the feedback
   controller would watch "" (matching nothing) when OSAC_VOLUME_NAMESPACE
   was unset.

3. (Major) Status update after delete: after handleDelete removes the
   finalizer, the API server may delete the object before updateStatusWithRetry
   runs. Now ignores NotFound during deletion since the status was already
   persisted in the finalizer-removal Update call.

4. (Major) Failed provisioning retry: added an explicit guard so Failed
   volumes are not re-provisioned on every reconcile (avoids spamming the
   vendor API). Re-provisioning is triggered by Signal or periodic sync
   after the admin fixes the vendor issue.

5. (Minor) Nil-check on hub list response before selecting a hub.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
akshaynadkarni added a commit to akshaynadkarni/osac that referenced this pull request Aug 10, 2026
Fix five issues identified by CodeRabbit on PR osac-project#223:

1. (Critical) Protocol enum lookup: CRD uses "Block"/"NFS" but the proto
   map is keyed by "STORAGE_PROTOCOL_BLOCK". Replaced the map lookup with
   an explicit switch in syncVolumeVendorFields. Added test coverage for
   both Block and NFS protocol mapping.

2. (Major) Feedback controller namespace default: NewVolumeFeedbackReconciler
   now defaults to "osac-volume" when the namespace parameter is empty,
   matching NewVolumeReconciler's behavior. Previously the feedback
   controller would watch "" (matching nothing) when OSAC_VOLUME_NAMESPACE
   was unset.

3. (Major) Status update after delete: after handleDelete removes the
   finalizer, the API server may delete the object before updateStatusWithRetry
   runs. Now ignores NotFound during deletion since the status was already
   persisted in the finalizer-removal Update call.

4. (Major) Failed provisioning retry: added an explicit guard so Failed
   volumes are not re-provisioned on every reconcile (avoids spamming the
   vendor API). Re-provisioning is triggered by Signal or periodic sync
   after the admin fixes the vendor issue.

5. (Minor) Nil-check on hub list response before selecting a hub.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
@akshaynadkarni
akshaynadkarni force-pushed the feat/OSAC-2872-volume-controller branch from f0a6f21 to 126d2dc Compare August 10, 2026 02:29

@zszabo-rh zszabo-rh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

Comment on lines +150 to +153
default:
log := ctrllog.FromContext(ctx)
log.Info("Unknown phase, will ignore it", "phase", obj.Status.Phase)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would leave remote.Status.State unchanged, maybe it should return an error so the controller re-queues?
(same pattern in all feedback controllers, so this is a generic issue. we can ignore for now, just might consider handling as a follow-up)

@akshaynadkarni akshaynadkarni Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The default case is a defensive guard for phases that don't exist yet. Returning an error would requeue indefinitely for a phase the controller genuinely doesn't know how to handle, so logging and leaving the state unchanged is the safer default. We can revisit this across all feedback controllers as a follow-up if we want consistent error-or-log behavior.

@akshaynadkarni akshaynadkarni changed the title OSAC-2872: add Volume reconciler, controller, and feedback controller OSAC-3273, OSAC-3280: Volume reconciler, controller, and feedback controller Aug 10, 2026
@openshift-ci-robot

openshift-ci-robot commented Aug 10, 2026

Copy link
Copy Markdown

@akshaynadkarni: This pull request references OSAC-3273 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.0.0" version, but no target version was set.

This pull request references OSAC-3280 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Adds the Volume lifecycle controllers (PR 2 of 3 for the storage control plane).
Stacked on #201: review that PR
first, then review only the commits unique to this branch.

  • fulfillment-service: Volume reconciler watches for Volume events via gRPC
    streaming, creates/patches/deletes Volume CRs on the hub cluster. Maps proto
    VolumeAccessMode enum to CRD typed strings. Selects a hub randomly (same as
    ComputeInstance, since volumes have no parent resource to inherit from).
  • osac-operator: Volume resource controller with VendorProvisioner interface
    (nil for this PR, vendor CSI integration in PR OSAC-1733: Merge fulfillment-service and osac-operator into osac mono-repo #3). Feedback controller syncs
    CR status back to the fulfillment-service (Progressing->CREATING,
    Ready->AVAILABLE, Failed->FAILED, Deleting->DELETING) plus vendorVolumeID,
    backend, and protocol.
  • Helm: RBAC for volumes resource, OSAC_VOLUME_NAMESPACE env var,
    hub-access clusterrole updated.

Why

PR #201 established the Volume
data model (proto, DB, server, CRD types). This PR adds the controllers that
drive the provisioning lifecycle end-to-end: the fulfillment-service creates
Volume CRs on the hub, the operator controller calls the vendor to provision,
and the feedback controller syncs the result back. The VendorProvisioner is nil
for now (volumes stay in Progressing), with the real vendor CSI client planned
for PR #3 after Roy's CSI driver PRs land.

What's included

fulfillment-service

  • internal/controllers/volume/ (reconciler function, tests, mocks)
  • internal/kubernetes/labels/kubernetes_labels.go (VolumeUuid label)
  • internal/cmd/service/start/controller/start_controller_cmd.go (wiring)

osac-operator

  • internal/controller/volume_controller.go (VendorProvisioner interface,
    provisioning lifecycle, management-state skip, status retry with
    NotFound handling on delete)
  • internal/controller/volume_feedback_controller.go (feedback.Bridge,
    phase-to-state mapping, vendor field sync with explicit protocol enum
    conversion via crdProtocolToProto)
  • internal/controller/volume_mock_provisioner.go (test mock with
    deterministic IDs and call counting)
  • internal/controller/volume_names.go (defaultVolumeNamespace)
  • cmd/main.go (setupVolumeControllers under ctrlFlags.Storage)
  • charts/operator/templates/ (deployment, clusterrole, hub-access)

Testing

# fulfillment-service: 87 suites pass (27 Volume-specific)
cd fulfillment-service && go run github.com/onsi/ginkgo/v2/ginkgo run -r internal

# osac-operator: 653 tests pass, 0 failures, 73% coverage
cd osac-operator && make test

# lint: 0 issues on both components
cd osac-operator && make lint
cd fulfillment-service && uv run dev.py lint go

# manifests unchanged
cd osac-operator && make manifests generate && git diff --exit-code

Pre-merge ToDos

  1. PR OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 must merge first (Volume proto, CRD types)
  2. After OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 merges and BSR publishes, bump buf.gen.yaml in osac-operator
    to the new BSR version and regenerate
  3. Rebase on main after OSAC-3273, OSAC-3280: Volume private API, DB migration, and CRD types #201 merges

Related PRs

Ticket

Feature: OSAC-2872 (Storage Control Plane)

Epics:

Userflow: OSAC-3371 (hub PVC create happy path)


Signed-off-by: akshaynadkarni 25892229+akshaynadkarni@users.noreply.github.com
Assisted-by: Cursor/Claude

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

akshaynadkarni added a commit to akshaynadkarni/osac that referenced this pull request Aug 11, 2026
Fix five issues identified by CodeRabbit on PR osac-project#223:

1. (Critical) Protocol enum lookup: CRD uses "Block"/"NFS" but the proto
   map is keyed by "STORAGE_PROTOCOL_BLOCK". Replaced the map lookup with
   an explicit switch in syncVolumeVendorFields. Added test coverage for
   both Block and NFS protocol mapping.

2. (Major) Feedback controller namespace default: NewVolumeFeedbackReconciler
   now defaults to "osac-volume" when the namespace parameter is empty,
   matching NewVolumeReconciler's behavior. Previously the feedback
   controller would watch "" (matching nothing) when OSAC_VOLUME_NAMESPACE
   was unset.

3. (Major) Status update after delete: after handleDelete removes the
   finalizer, the API server may delete the object before updateStatusWithRetry
   runs. Now ignores NotFound during deletion since the status was already
   persisted in the finalizer-removal Update call.

4. (Major) Failed provisioning retry: added an explicit guard so Failed
   volumes are not re-provisioned on every reconcile (avoids spamming the
   vendor API). Re-provisioning is triggered by Signal or periodic sync
   after the admin fixes the vendor issue.

5. (Minor) Nil-check on hub list response before selecting a hub.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
@akshaynadkarni
akshaynadkarni force-pushed the feat/OSAC-2872-volume-controller branch from 126d2dc to 7589a7d Compare August 11, 2026 19:31
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 11, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 7:32 PM UTC · Ended 7:42 PM UTC

Commit: 7589a7d · View workflow run →

@akshaynadkarni
akshaynadkarni marked this pull request as ready for review August 11, 2026 19:42
func (r *VolumeReconciler) handleProvisioning(ctx context.Context, vol *v1alpha1.Volume) (ctrl.Result, error) {
log := ctrllog.FromContext(ctx)

resp, err := r.VendorProvisioner.CreateVolume(ctx, VendorCreateVolumeRequest{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the CreateVolume request needs to say which storageBackend to use and not the tier. Meaning that the tier resolution already executed by the osac volume creation, at the volume-api level.

The vendor provisioner would route the request to the right backend.

The Volume reconciler watches for Volume events via gRPC streaming and
creates corresponding Volume CRs on the hub cluster. It follows the
same three-struct pattern (FunctionBuilder/function/task) as NATGateway
and other hub-based reconcilers.

Key behaviors:
- Adds the fulfillment-controller finalizer before any K8s work
- Selects a hub randomly from available hubs (volumes are independent
  resources with no parent to inherit from, same as ComputeInstance)
- Creates Volume CRs with the volume-uuid label for lookup
- Converts the proto VolumeAccessMode enum to the CRD typed string
- On delete, removes the CR from the hub and cleans up the finalizer
- Handles decommissioned hubs by removing the finalizer gracefully

Tests cover buildSpec (all access mode enum values, PVC reference),
setDefaults, validateTenant, finalizer lifecycle, delete paths
(missing hub, missing CR, active deletion, decommissioned hub),
selectHub (existing hub, random selection, no hubs), and K8s
validation error handling.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
…sioner

Volume controller reconciles Volume CRs on the hub cluster through the
vendor provisioning lifecycle (Progressing -> Ready or Failed). Unlike
networking and compute controllers that use AAP, the Volume controller
calls a VendorProvisioner interface directly because storage provisioning
is a synchronous vendor CSI gRPC call. For this PR the implementation is
a MockVendorProvisioner; the real vendor CSI client is wired in PR osac-project#3.

The feedback controller syncs Volume CR status back to the fulfillment-
service using the shared feedback.Bridge. Phase-to-state mapping:
Progressing->CREATING, Ready->AVAILABLE, Failed->FAILED, Deleting->DELETING.
It also syncs vendorVolumeID, backend, and protocol so the fulfillment-
service inventory reflects the actual storage array state.

Generated gRPC client from local fulfillment-service proto (BSR version
will be bumped after PR osac-project#1 merges and publishes the Volume proto).

Tests cover the full lifecycle: finalizer addition, provisioning success
and failure, idempotent re-reconcile after Ready, management-state skip,
deletion with vendor deprovisioning, and not-found handling.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Register the Volume resource controller and feedback controller under
the Storage controller flag (ctrlFlags.Storage). The VendorProvisioner
is nil until the real vendor CSI client is integrated in a follow-up PR;
the controller sets phase to Progressing and skips provisioning.

Helm chart changes:
- deployment.yaml: add OSAC_VOLUME_NAMESPACE env var (metadata.namespace)
- clusterrole.yaml: add volumes, volumes/finalizers, volumes/status
- hub-access-clusterrole.yaml: add volumes, volumes/status so the
  fulfillment-service controller can manage Volume CRs on the hub

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Tests cover the full feedback lifecycle using an in-process gRPC mock
server (bufconn pattern, same as VirtualNetwork feedback tests):

Phase-to-state mapping:
- Progressing -> CREATING, Ready -> AVAILABLE, Failed -> FAILED

Vendor field syncing:
- vendorVolumeID, backend, protocol copied to fulfillment-service
- Empty CR fields do not overwrite existing remote values

Deletion handling:
- Deleting phase maps to DELETING state
- Failed phase during deletion maps to FAILED
- Last-finalizer removal triggers Signal RPC
- Signal failure does not block finalizer removal
- NotFound remote record during deletion removes finalizer gracefully

Idempotency:
- No Update RPC when remote state already matches CR

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Fix five issues identified by CodeRabbit on PR osac-project#223:

1. (Critical) Protocol enum lookup: CRD uses "Block"/"NFS" but the proto
   map is keyed by "STORAGE_PROTOCOL_BLOCK". Replaced the map lookup with
   an explicit switch in syncVolumeVendorFields. Added test coverage for
   both Block and NFS protocol mapping.

2. (Major) Feedback controller namespace default: NewVolumeFeedbackReconciler
   now defaults to "osac-volume" when the namespace parameter is empty,
   matching NewVolumeReconciler's behavior. Previously the feedback
   controller would watch "" (matching nothing) when OSAC_VOLUME_NAMESPACE
   was unset.

3. (Major) Status update after delete: after handleDelete removes the
   finalizer, the API server may delete the object before updateStatusWithRetry
   runs. Now ignores NotFound during deletion since the status was already
   persisted in the finalizer-removal Update call.

4. (Major) Failed provisioning retry: added an explicit guard so Failed
   volumes are not re-provisioned on every reconcile (avoids spamming the
   vendor API). Re-provisioning is triggered by Signal or periodic sync
   after the admin fixes the vendor issue.

5. (Minor) Nil-check on hub list response before selecting a hub.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Move Volume label, finalizer, and namespace constants from api/v1alpha1/
to internal/controller/volume_names.go using the existing pattern
(unexported vars with fmt.Sprintf and osacPrefix). All references are
within the controller package so no external consumers are affected.

Doc updates:
- AGENTS.md Resources Managed: add Volume (vol)
- README.md CRD list: add Volume with VendorProvisioner description
- architecture-patterns.md Resource Hierarchy: add Storage Resources

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
MockVendorProvisioner is only used in tests. Move from
volume_mock_provisioner.go to volume_mock_provisioner_test.go so it is
not compiled into the production binary.

Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
- Rename misleading test: "should set phase to Progressing on first
  reconcile" correctly reflects Ready, not Progressing, since the mock
  provisioner succeeds immediately
- Close gRPC client connection in AfterEach of feedback controller
  tests to prevent goroutine leaks across specs
- Add ReadOnlyMany access mode test in volume reconciler to cover the
  explicit switch case in protoAccessModeToCRD
- Add test for handleDelete when DeleteVolume returns an error: verifies
  the finalizer is kept and Phase is Deleting when vendor fails
- Replace r.mgr.GetLocalManager().GetClient().Update with r.Update in
  handleDelete for consistency with all other controllers
- Rename oldStatus to oldstatus in volume_controller.go to match the
  convention used across all 11 other resource controllers
- Replace thinking-out-loud comment in feedback controller test with
  a concise explanation of why no Update RPC is sent

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Remove management-state annotation check: volumes are always provisioned
by the vendor CSI driver and have no manual management path, unlike AAP-
provisioned resources where the annotation makes sense.

Use Backend instead of StorageTier in VendorCreateVolumeRequest: tier
resolution happens in the fulfillment-service at CreateVolume time
(OSAC-3277). By the time the Volume CR reaches the operator, the backend
is already resolved and stored in vol.Status.Backend. The vendor
provisioner routes by backend name, not tier name.

Type AccessMode as VolumeAccessMode instead of string in
VendorCreateVolumeRequest, matching the CRD type and removing the
unsafe string conversion at the call site.

Add comment on the nil VendorProvisioner guard explaining it is
temporary until the real vendor CSI client is wired.

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Volume provisioning (vendor CSI) has different operational dependencies
from tenant storage management (StorageClass lifecycle, ClusterOrder
storage). Bundling both under OSAC_ENABLE_STORAGE_CONTROLLER prevents
deploying one without the other and establishes a bad precedent for
future storage types (file, object) that will each need independent
enable flags.

Add a dedicated OSAC_ENABLE_VOLUME_CONTROLLER / --enable-volume-controller
flag that gates setupVolumeControllers independently of the storage
controller. Both default to true when no flags are set.

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
Replace updateStatusWithRetry with a direct r.Status().Update() call,
matching the pattern used by all other resource controllers. The retry
wrapper introduced a deletion race where re-fetching the object after
finalizer removal returned NotFound, requiring a novel guard that no
other controller needs. Write conflicts on status are handled by
controller-runtime requeue, which is sufficient.

Assert reconcile errors in setup loops so a silent failure cannot put
the test in an unexpected state before the actual assertion.

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
The error "client is mandatory" was misleading since the field being
checked is a grpc.ClientConn, not a client. Changed to "connection is
mandatory" to match the field name and SetConnection() method.

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 13, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 9:07 PM UTC · Completed 9:25 PM UTC

Commit: af83279 · View workflow run →

…exity

main() exceeded gocyclo's limit of 30 after adding the separate
OSAC_ENABLE_VOLUME_CONTROLLER if block. Extract all controller setup
into a dedicated setupControllers() function, reducing main()'s branch
count and making the enable-flag logic easier to follow.

Assisted-by: Cursor/Claude
Signed-off-by: akshaynadkarni <25892229+akshaynadkarni@users.noreply.github.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:54 AM UTC · Completed 3:12 AM UTC

Commit: fa76883 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.


Note: The following inline comments could not be posted on the diff (GitHub returned 422) and are included here instead:

  • fulfillment-service/internal/controllers/volume/volume_reconciler_function.go:165: [medium] error-handling-gap

validateTenant() returns a hard error when the tenant is empty, but the error propagates to the reconciler loop which retries indefinitely. Unlike K8s validation errors that go through HandleK8sWriteError and call setFailed(), a missing tenant will never set the volume status to FAILED, leaving the user with no visibility into why the volume is stuck.

Suggested fix: Call t.setFailed(err) before returning the error from validateTenant failure, similar to how HandleK8sWriteError handles validation errors.

  • osac-operator/internal/controller/volume_controller.go (file-level): Line 558 · [medium] missing-management-state-check

The VolumeReconciler.Reconcile method does not check the osac.openshift.io/management-state annotation before reconciling. Per AGENTS.md, all resource controllers (except tenant_controller.go) check this annotation and skip reconciliation when set to Unmanaged. Without this check, operators cannot halt Volume reconciliation for debugging or emergency intervention.

Suggested fix: Add the management-state annotation check after the Get call, consistent with other controllers (e.g., computeinstance_controller.go:145).

  • osac-operator/internal/controller/volume_controller.go (file-level): Line 580 · [medium] pattern-inconsistency

Direct r.Status().Update(ctx, vol) instead of the updateStatusWithRetry helper used by all other resource controllers. Other controllers wrap status updates in retry.RetryOnConflict to handle optimistic locking conflicts. The direct call is susceptible to conflict errors under concurrent modification.

Suggested fix: Add an updateStatusWithRetry method matching the pattern in other controllers (e.g., computeinstance_controller.go:173-186).

  • fulfillment-service/internal/controllers/volume/volume_reconciler_function.go:306: [low] edge-case

selectHub accesses response.Items via rand.IntN. If a Hub in the Items slice has an empty ID, the reconciler assigns an empty hubId and likely fails with a confusing error. Low-probability edge case that applies equally to ComputeInstance.

  • osac-operator/internal/controller/volume_names.go:28: [low] naming-convention

Finalizer uses -finalizer suffix (osac.openshift.io/volume-finalizer). Other resources are inconsistent: NATGateway uses -finalizer, ComputeInstance uses just the resource name. Volume follows the majority pattern.

@fullsend-ai-review fullsend-ai-review Bot removed the requires-manual-review Review requires human judgment label Aug 14, 2026

resp, err := r.VendorProvisioner.CreateVolume(ctx, VendorCreateVolumeRequest{
Name: vol.Name,
Backend: vol.Status.Backend,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might getting this wrong but I think vol.Status.Backend is always empty on first provisioning. Status.Backend originates from the CreateVolume response (at least for now), not from any input. Neither does the fulfillment-service reconciler Status fields.

I saw the struct comment claiming OSAC-3277 will pre-populate Backend via tier resolution before the CR is created, but it's not listed as a dependency of this PR and nothing in this diff implements that path.

Again, maybe I missed something, or this is something that will be covered in the third PR so just raising it here so it doesn’t get overlooked.

@akshaynadkarni akshaynadkarni Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline. It's true that backend will be empty on first provisioning for now. However, I am putting up a PR to tackle tier -> backend resolution. So this will change soon.

@akshaynadkarni

Copy link
Copy Markdown
Contributor Author

Closing this PR in favor of smaller, focused PRs split by component boundary:

All code from this PR is preserved in the split PRs (final file states copied, all review fixes from CodeRabbit, fullsend, Roy, and Zoltan are included). Each PR is independently mergeable against main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved enhancement New feature or request go Pull requests that update go code jira/valid-reference storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants