Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,21 @@ S3 and GCS also auto-create the bucket if it does not exist.

`AddLocalTestsCloudFiles` registers `CloudFilesService` (concrete) as a singleton **in addition to** the `ICloudFilesService` interface alias, so test classes can inject `CloudFilesService` directly and call `Cleanup()` in teardown. Storage root defaults to `Path.GetTempPath()/SW.CloudFiles.LocalTests/{BucketName}` via `LocalTestsCloudFilesOptions.ResolvedStoragePath`. Metadata is persisted as sidecar `.meta.json` files alongside each stored blob. `GetSignedUrl` returns the same `file://` URI as `GetUrl`. `OpenWrite` throws `NotImplementedException`.

## Known Issues / DI Lifetime Audit (2026-08-25)

A production incident on 2026-08-25 (invoice-attachment and shipment-label uploads hanging ~1 minute then failing) traced back to the S3 provider building a brand-new `AmazonS3Client` on every DI resolution (`ICloudFilesService` was `AddTransient`) with no explicit timeout, so any latency to the storage endpoint compounded through the AWS SDK's default ~100s timeout and several backoff retries. Fixed in 8.1.11 (S3 provider): `ICloudFilesService` is now `AddSingleton`, and `S3CloudFilesOptions.TimeoutSeconds`/`MaxErrorRetry` (defaults 15/2) are configurable via the `CloudFiles` config section or the `AddS3CloudFiles(options => ...)` delegate.

Prompted by that incident, the other three network-backed providers were audited for the same class of bug (fresh client construction on every resolution instead of reusing a pooled, thread-safe SDK client):

- **Azure (`SW.CloudFiles.AS`) — same bug, and worse. Fixed alongside S3.** `ICloudFilesService` was also `AddTransient`, and `CloudFilesService`'s constructor called `cloudFilesOptions.CreateClient()` itself rather than using the `BlobContainerClient` the DI extension registered as a singleton — that singleton registration was dead code, never actually injected anywhere. Worse than the S3 case: in the non-managed-identity (shared-key) auth path, `CreateClient()` does a **synchronous, blocking `GetBlobContainers()` list call** plus an existence check — a real network round-trip, not just a fresh TCP/TLS handshake — on every single construction. Fixed: `CloudFilesService` now takes `BlobContainerClient` as a constructor parameter (reusing the singleton), and `ICloudFilesService` is registered `AddSingleton`.
- **Oracle (`SW.CloudFiles.OC`) — milder version, fixed alongside S3.** Was `AddScoped` (so only once per HTTP request rather than per resolution, unlike S3/Azure's `AddTransient`), but the constructor still built a fresh `ObjectStorageClient` + `UploadManager` per request scope, including a `ConfigFileAuthenticationDetailsProvider` re-reading the PEM/config file from disk each time. Fixed: registered `AddSingleton` instead, so the client is built once for the process lifetime.
- **Google Cloud (`SW.CloudFiles.GC`) — already correct, no change needed.** `ICloudFilesService` is `AddScoped`, but the actual `StorageClient`/`UrlSigner` doing the I/O are `AddSingleton` and injected in — no repeated client construction regardless of the wrapper service's lifetime. This is the reference-correct pattern the other providers now follow.
- **LocalTests** — file-based, no network client involved, not applicable.

None of Azure/GCS/Oracle are referenced by any Traxis service (only the S3 provider is, via `AddS3CloudFiles()` in all 8 backend microservices) — this was pure library-quality debt with no live-incident urgency, unlike the S3 fix.

Note: `SW.CloudFiles.AS.UnitTest` and `SW.CloudFiles.OC.UnitTest` fail locally regardless of these changes (confirmed by reverting and re-running) — they appear to be integration tests requiring live cloud credentials, consistent with CI's `run-tests: 'false'` in `nuget-publish.yml`. Not a regression from this audit.

## Key Dependencies

- `SimplyWorks.PrimitiveTypes` v8.1.3 — shared interface and base types (all providers)
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,11 +327,20 @@ All four providers support `GetSignedUrl(key, expiry)`:
### S3-Compatible Storage
- Works with AWS S3, DigitalOcean Spaces, MinIO, and any S3-compatible service.
- Bucket is created automatically if it does not exist.
- The underlying `AmazonS3Client` is built once and reused for the life of the process (`ICloudFilesService` is registered as a singleton) rather than per call.
- `TimeoutSeconds` (default `15`) and `MaxErrorRetry` (default `2`) on `S3CloudFilesOptions` bound the client's per-request timeout and automatic retry count, so a genuine outage on the storage endpoint fails fast instead of hanging on the AWS SDK's much longer defaults:
```csharp
services.AddS3CloudFiles(o => {
o.TimeoutSeconds = 20;
o.MaxErrorRetry = 1;
});
```

### Azure Blob Storage
- Container is created automatically if it does not exist.
- **Managed Identity**: Set `Managed = true`. Optionally set `ManagedIdentityClientId` for a user-assigned identity; omit it to use the system-assigned identity or ambient `DefaultAzureCredential`.
- **Public URL override**: If you connect via private link (`ServiceUrl`) but need public-facing URLs, set `PublicServiceUrl` to the standard public endpoint (e.g. `https://account.blob.core.windows.net`).
- The underlying `BlobContainerClient` is built once and reused for the life of the process (`ICloudFilesService` is registered as a singleton) rather than per call.

```csharp
// Managed Identity example
Expand All @@ -352,6 +361,7 @@ services.AddAsCloudFiles(o => {
### Oracle Cloud Storage
- OCI credentials (`UserId`, `TenantId`, `FingerPrint`, `RSAKey`) are written to temporary files on startup and used to authenticate via `ConfigFileAuthenticationDetailsProvider`.
- `GetSignedUrl` creates a Pre-Authenticated Request (PAR) with read-only access.
- The underlying `ObjectStorageClient` is built once and reused for the life of the process (`ICloudFilesService` is registered as a singleton) rather than per request.

### Local Filesystem (Testing / Local Development Only)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public static IServiceCollection AddAsCloudFiles(this IServiceCollection service
serviceCollection.AddSingleton(blobContainerClient);
serviceCollection.AddSingleton(cloudFilesOptions);
serviceCollection.AddSingleton<CloudFilesOptions>(cloudFilesOptions);
serviceCollection.AddTransient<ICloudFilesService, CloudFilesService>();
serviceCollection.AddSingleton<ICloudFilesService, CloudFilesService>();
return serviceCollection;
}
}
4 changes: 2 additions & 2 deletions SW.CloudFiles.AS/CloudFilesService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
namespace SW.CloudFiles.AS;

/// <summary>Azure Blob Storage implementation of <see cref="ICloudFilesService"/>.</summary>
public class CloudFilesService(AzureCloudFilesOptions cloudFilesOptions) : IDisposable, ICloudFilesService
public class CloudFilesService(AzureCloudFilesOptions cloudFilesOptions, BlobContainerClient blobContainerClient) : IDisposable, ICloudFilesService
{
private readonly AzureCloudFilesOptions cloudFilesOptions = cloudFilesOptions;
private readonly BlobContainerClient blobContainerClient = cloudFilesOptions.CreateClient();
private readonly BlobContainerClient blobContainerClient = blobContainerClient;

/// <inheritdoc/>
public async Task<RemoteBlob> WriteAsync(Stream inputStream, WriteFileSettings settings)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ public static IServiceCollection AddOracleCloudFiles(this IServiceCollection ser
if (!cloudFilesOptions.DisableAutoLifecycle)
EnsureLifecycleRules(cloudFilesOptions);

serviceCollection.AddScoped<ICloudFilesService, CloudFilesService>();
serviceCollection.AddSingleton<ICloudFilesService, CloudFilesService>();
serviceCollection.AddSingleton(cloudFilesOptions);
serviceCollection.AddSingleton<CloudFilesOptions>(cloudFilesOptions);
return serviceCollection;
Expand Down
Loading