Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: S3 Object Lock support (grid + tenant + per-bucket sub-resources)
Summary
Adds first-class support for S3 Object Lock across the SDK:
/grid/compliance-global(read/update thecomplianceEnabled,legacyComplianceEnabled,createLegacyComplianceBucketsflags).
(
allowComplianceMode,maxRetentionDays,maxRetentionYears).endpoints (
region,usage,object-lock,notification,policy,cors,compliance) instead of always listing all buckets and filteringclient-side.
Changes
New: grid-wide S3 Object Lock (
/grid/compliance-global)models.S3ObjectLock—ComplianceEnabled,LegacyComplianceEnabled,CreateLegacyComplianceBuckets(all*bool+omitemptyso partial PUTsand partial GET responses round-trip cleanly).
services.S3ObjectLockServiceInterfacewithGet(ctx)andUpdate(ctx, *S3ObjectLock).GridClientasgridClient.S3ObjectLock().testing.MockS3ObjectLockServicewithGetFunc/UpdateFuncand acompile-time interface conformance check.
Tenant policy: S3 Object Lock fields
Added to
models.TenantPolicy:AllowComplianceMode *bool— whether the tenant may use S3 Object Lock incompliance mode (requires grid-wide S3 Object Lock to be enabled).
MaxRetentionDays *int,MaxRetentionYears *int— per-tenant caps on theretention period a bucket may specify.
nilmeans no cap.Bucket sub-resources (
/org/containers/{name}/...)BucketServiceInterfacenow exposes typed methods that hit the dedicatedper-bucket endpoints, matching the current StorageGRID API surface:
GetBucketUsage→/usageGetRegion→/region(returnsstring)GetObjectLock/UpdateObjectLock→/object-lockGetNotification/UpdateNotification→/notificationGetPolicy/UpdatePolicy→/policyGetCors/UpdateCors→/corsGetCompliance/UpdateCompliance→/compliance(legacy compliance)New wrapper models in
models/buckets.go:BucketUsage(objectCount,dataBytes)BucketCorsConfiguration(corsXML, nil ⇒ disable)BucketNotificationConfiguration(notificationXML, nil ⇒ disable)BucketPolicyConfiguration(policy, nil ⇒ disable)BucketPolicy,BucketPolicyStatement— typed S3 policy document(
Id,Version,Statement[]withEffect,Action/NotAction,Resource/NotResource,Condition,Principal/NotPrincipal).Polymorphic fields that the S3 policy language allows as either a string
or a list (and the
"*"-or-object Principal) remaininterface{}; theConditionblock is typed asmap[string]map[string]interface{}matchingcondition_type → key → value.Bucket model fixes
BucketS3ObjectLockDefaultRetentionSettings.Modenow documents that bothcomplianceandgovernanceare valid, instead of "must be compliance".Mocks
MockBucketServiceextended with*Funcfields and pass-through defaultimplementations for every new method. The existing compile-time interface
check still passes.
Example
examples/grid/s3-object-lock/— new end-to-end example. By default it onlyreads grid-wide settings. With
STORAGEGRID_APPLY=trueand tenantcredentials it:
/grid/compliance-global.complianceEnabledgrid-wide.allowComplianceMode=true,maxRetentionYears=10).compliance retention period.
/org/containers/{name}/object-locksub-resource.examples/tenant/bucket-operations/main.gowas migrated off the deprecatedGetUsagetoGetBucketUsage+GetRegion.Docs
README.md: feature bullets for grid-wide S3 Object Lock and the newbucket sub-resources, an API-coverage row for
/grid/compliance-global, anAPI-coverage row for the bucket sub-resources, and
MockS3ObjectLockServiceadded to the available-mocks list.
examples/README.md: pruned stale links to non-existent examples and addedthe new
grid/s3-object-lockentry.Deprecations
BucketServiceInterface.GetUsage(ctx, name) (*BucketStats, error)is nowmarked
// Deprecated:. It still works (it fetches the entire tenantusage payload from
/org/usageand filters by name) but new code shoulduse the per-bucket endpoint:
GetBucketUsagereturns the slimmermodels.BucketUsage(ObjectCount,DataBytes); other fields previously stitched in from/org/usage(
Region,VersioningEnabled,Encryption, …) are available via thematching per-bucket sub-resources.
The deprecated method will be removed in a future release.
Compatibility
authentication flow.
GetUsageis annotated asdeprecated).
Testing
go build ./...andgo vet ./...pass.examples/grid/s3-object-lockbuilds standalone (go mod init+go mod edit -replace+go build).MockS3ObjectLockServiceandMockBucketServiceis enforced at compile time viavar _assertions.References
https://docs.netapp.com/us-en/storagegrid/ilm/managing-objects-with-s3-object-lock.html