diff --git a/AGENTS.md b/AGENTS.md index 8cf1b27..d145286 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -60,9 +60,21 @@ whenever a target framework, a backend package or the `buildTransitive` assets c - Never swallow caller cancellation as a successful exec/stop. Startup is a total five-minute managed budget; exec defaults to one minute, stop to 15 seconds and cleanup to 30 seconds. Synchronous WSLC native calls cannot be safely interrupted; never advertise hard zero-hang guarantees. -- Reuse is explicit, named and incompatible with auto-remove. Docker follows Testcontainers; - WSLC reuse is session-scoped. Explicit deletion bypasses reuse retention. Dispose containers before - managed networks or built images. CLI diagnostics must not echo environment/build output. +- Disposal cleanup mirrors Testcontainers (`WithCleanUp`, default `true`): disposing removes the + container, and Docker also removes anonymous volumes; named `WithVolumeMount` volumes are retained. + `WithCleanUp(false)` stops and retains the container. Auto-remove is off by default, is independent + of cleanup, and wins once the init process exits. +- WSLC named volumes are session-scoped but persist in the session's storage VHD. Named volumes on both + backends are retained by default; `ContainerCleanup.CleanUpVolumesOnExit` (or + `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true`) deletes the volumes the backend created at process/session + termination, after removing the containers that used them. + `WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides the shared switch; + `IContainerBackend.CleanUpAsync` is the explicit path. Never delete the shared image store or host + bind mounts. +- Reuse is explicit, named, incompatible with auto-remove, and implies cleanup off. Docker follows + Testcontainers; WSLC reuse is session-scoped. Explicit deletion bypasses reuse retention. Dispose + containers before managed networks or built images. CLI diagnostics must not echo environment/build + output. - Restart an exited WSLC reusable container through the CLI **before** reopening its SDK event handle. Calling native `Start()` after attaching that handle fails with "Process handle has already been attached". - Docker pre-start resource archives run inside Testcontainers' synchronous `Created` event, before init diff --git a/README.md b/README.md index 9581e06..a1a72be 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,9 @@ It requires no Docker installation. The Docker backend is built on Both backends expose file copy/read, one-shot exit waits, managed bridge networks/aliases and Dockerfile builds through the same contract. Startup has a five-minute total budget; exec defaults to one minute, stop to 15 seconds and cleanup to 30 seconds, all configurable. Cancellation is -propagated; synchronous native WSLC calls cannot be forcibly interrupted. See +propagated; synchronous native WSLC calls cannot be forcibly interrupted. Disposal follows +Testcontainers' `WithCleanUp`: containers are removed by default, `.WithCleanUp(false)` retains them +for inspection, and `.WithAutoRemove()` is opted into separately. See [Capabilities and Cancellation](docs/wiki/Backend-Capabilities.md) for APIs, ownership and limits. > **Experimental.** This project is an experiment in running throwaway containers through Microsoft @@ -41,7 +43,10 @@ propagated; synchronous native WSLC calls cannot be forcibly interrupted. See > by default** (`StorageMode.Shared`): sessions reuse a stable image store > (`%LOCALAPPDATA%\Purview\WslContainers\images`) so images are pulled once, not per session; > `StorageMode.PerSession` provides isolation. Move the store with `PURVIEW_CONTAINERS_STORAGE_PATH` or -> `WslContainerRuntimeOptions.StoragePath`. Runnable samples live in `samples/getting-started`. +> `WslContainerRuntimeOptions.StoragePath`. Named volumes are retained by default; set +> `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true` (or `ContainerCleanup.CleanUpVolumesOnExit`) to delete the +> volumes the library created when the process or session ends. Runnable samples live in +> `samples/getting-started`. ## Prerequisites diff --git a/docs/wiki/Backend-Capabilities.md b/docs/wiki/Backend-Capabilities.md index 2d5b165..3fbcd12 100644 --- a/docs/wiki/Backend-Capabilities.md +++ b/docs/wiki/Backend-Capabilities.md @@ -20,6 +20,7 @@ do not preserve an image's default command. | Managed bridge networks and aliases | `CreateNetworkAsync`, `WithNetwork`, `WithNetworkAliases` | CLI network and container creation; alias DNS | | Dockerfile builds | `BuildImageAsync` | CLI build in the owning SDK session/store | | Reuse | `WithName(...).WithReuse()` | Same configuration/name within the owning session | +| Disposal cleanup | `WithCleanUp(bool)` (default `true`) | Stop then force-delete; `false` stops and retains the container | | Explicit deletion | `DeleteAsync(ct)` | Force-delete, including reusable containers | Use absolute Linux destinations for copy/mapping operations. A file source names the destination @@ -111,12 +112,28 @@ Use unique names for independently owned resources. ## Reuse and platform limits -Reuse is opt-in, requires a stable explicit name, and cannot be combined with auto-remove. -`DisposeAsync` no longer promises deletion for a reusable container. Call `DeleteAsync(ct)` to -remove it deliberately. +Reuse is opt-in, requires a stable explicit name, cannot be combined with auto-remove, and implies +cleanup off (mirroring Testcontainers). `WithReuse().WithCleanUp(true)` is rejected at build time. +`DisposeAsync` no longer promises deletion for a reusable container; it stops and retains it. Call +`DeleteAsync(ct)` to remove it deliberately. Delete before disposing the wrapper; to remove a retained container later, reopen it with matching reuse settings and delete through that wrapper. +Disposal cleanup is on by default (`WithCleanUp(true)`). It removes the container, and the Docker +backend also removes the container's anonymous volumes (`RemoveVolumes = true`, as Testcontainers +does). Named `WithVolumeMount` volumes are shared data and are retained on both backends; WSLC +named volumes are recorded in the session's storage VHD and do not disappear automatically when the +session ends. `WithCleanUp(false)` stops the container and keeps it (and its volumes) for inspection, +exactly like Testcontainers' `WithCleanUp(false)`; remove it with `DeleteAsync` before disposal or +with the backend's tooling. + +WSLC named volumes are retained across sessions by default, matching Docker. A test run that must +leave no volume state behind can set `ContainerCleanup.CleanUpVolumesOnExit = true` +(or `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true`). Session termination then deletes the containers it +recorded and their named volumes; the shared image store and host bind mounts are never deleted. +`WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides the shared switch for one runtime, and +`IContainerBackend.CleanUpAsync()` performs the same cleanup explicitly. + Docker delegates reuse to Testcontainers: it survives wrapper/process lifetimes and opts out of normal resource-reaper cleanup. WSLC reuse lasts only as long as its owning session, checks the requested configuration fingerprint before reopening a named container, and is not cross-process diff --git a/docs/wiki/Backends.md b/docs/wiki/Backends.md index c8f3c0e..a5d4eba 100644 --- a/docs/wiki/Backends.md +++ b/docs/wiki/Backends.md @@ -38,6 +38,8 @@ Service modules (`Purview.Containers.PostgreSql`, `Redis`, …) are backend-neut | **How containers run** | the managed SDK, with `wslc` CLI fallback for SDK-missing features | the Docker Engine API via Testcontainers | | **Images** | a shared store (`%LOCALAPPDATA%\Purview\WslContainers\images`) reused across runs; the location is configurable | the daemon's own image store | | **Leak protection** | session disposal plus a process-exit hook | the Testcontainers resource reaper (Ryuk) | +| **Disposal cleanup** | `WithCleanUp` (default `true`) stops and deletes the container | `WithCleanUp` (default `true`) delegates to Testcontainers, which also removes anonymous volumes | +| **Volume cleanup** | opt-in via `ContainerCleanup.CleanUpVolumesOnExit` / `PURVIEW_CONTAINERS_CLEAN_VOLUMES`; `CleanUpVolumesOnTerminate` overrides per runtime | opt-in via the same shared switch; removes volumes the backend created | | **Check the host** | `wsl --version`, `wslc version` | `docker info` | | **Typical fit** | local Windows development without Docker Desktop, fastest cold start | CI runners, non-Windows hosts, teams already running Docker | @@ -104,9 +106,16 @@ ContainerBackends.Use(new WslContainerBackend(WslContainerRuntimeOptions.Default `StorageMode.Shared` (the default) keeps a stable, warm image store at `StoragePath` (or the default above when it is unset); `StorageMode.PerSession` gives each session a throwaway store under `%LOCALAPPDATA%\Purview\WslContainers\sessions\{name}`. The full `WslContainerRuntimeOptions` set (CPU, -memory, GPU, session name, `StoragePath`, `StorageMode`, timeout) is honoured on both a platform-neutral -`net10.0` consumer — where the facade forwards it to the Windows build at run time — and a Windows target -framework. The pre-rename `WSL_CONTAINERS_STORAGE_PATH` is still honoured as a fallback. +memory, GPU, session name, `StoragePath`, `StorageMode`, timeout, `CleanUpVolumesOnTerminate`) is honoured +on both a platform-neutral `net10.0` consumer — where the facade forwards it to the Windows build at run +time — and a Windows target framework. The pre-rename `WSL_CONTAINERS_STORAGE_PATH` is still honoured as a +fallback. + +WSLC named volumes are retained by default, matching Docker. Set `ContainerCleanup.CleanUpVolumesOnExit = true` +(or `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true`) to delete the volumes the library created when the process +exits or the session terminates; `WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides the shared +switch for one runtime, and `CleanUpAsync` performs the cleanup explicitly. Containers are deleted first; +the shared image store and host bind mounts are untouched. ## Switch between them without touching test code: @@ -461,6 +470,8 @@ release its image-store VHD. | Reuse | session-scoped, explicit name | Testcontainers reuse | | Log tailing | polled until the container stops | polled until the container stops | | Leak protection | session disposal + process-exit hook | Testcontainers resource reaper (Ryuk) | +| Disposal cleanup | `WithCleanUp`, default `true` | `WithCleanUp`, default `true`, anonymous volumes removed | +| Volume cleanup | opt-in via `ContainerCleanup.CleanUpVolumesOnExit` (session override available) | opt-in via the same shared switch | | Image store | shared WSLC store, warm across runs | the daemon's store | | Lifetime cost | one process-wide session, cheap start | first pull per image, per daemon | diff --git a/docs/wiki/Lifecycle.md b/docs/wiki/Lifecycle.md index 8674d7a..ca18db6 100644 --- a/docs/wiki/Lifecycle.md +++ b/docs/wiki/Lifecycle.md @@ -25,12 +25,25 @@ WSLC `ContainerState`: `Invalid → Created → Running → Exited → Deleted`. | `StartAsync` | Creates the session if needed, ensures image (per pull policy), `CreateContainer`, then `Container.Start()`. A missing init binary throws `ArgumentException 0x80070057` with the OCI message → surfaced as `WslContainerStartupException`. | | `StopAsync` | Signal the init process with SIGTERM, allow 10 seconds to exit, then send SIGKILL and wait up to 5 seconds. Managed polling avoids the native blocking `Container.Stop` call and does not hold the session gate during the grace period. **Idempotent.** | | `DeleteAsync` | `Container.Delete(Force)`. Double-delete throws `0x80010108` (`RPC_E_DISCONNECTED`) → swallowed. | -| `DisposeAsync` | Normally stop (if running), delete and release handles. Reusable containers are retained. **Idempotent.** | +| `DisposeAsync` | Normally stop (if running), delete and release handles. Reusable containers and `WithCleanUp(false)` containers are stopped and retained. **Idempotent.** | ## Cleanup guarantees - Use `await using` so caller failures always reach `DisposeAsync`. The runtime process-exit hook owns session cleanup; it is not a substitute for disposing each container. +- `EnableCleanUp` defaults to `true`, mirroring Testcontainers' `WithCleanUp`: `DisposeAsync` stops and + removes the container. Docker also removes the container's anonymous volumes; named + `WithVolumeMount` volumes are shared data and are retained (WSLC volumes belong to the session). + Set `.WithCleanUp(false)` to stop and retain the container (and its volumes) for inspection, then + remove it deliberately with `DeleteAsync` before disposal (or with the backend's own tooling). + `WithReuse()` implies cleanup off; `WithReuse().WithCleanUp(true)` is rejected at build time. +- WSLC named volumes live on the session and are retained by default, matching Docker. Set + `ContainerCleanup.CleanUpVolumesOnExit = true`, or `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true` before the + first container is created, to delete every volume the session recorded when it terminates. That is the + session-wide, test-run switch: it removes the session's containers first and then those volumes, while + leaving the shared image store intact. `WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides + the shared switch (`null` inherits it), and `IContainerBackend.CleanUpAsync()` performs the cleanup + explicitly. - `EnableAutoRemove` defaults to `false`: the library owns deletion via `DisposeAsync`. WSLC auto-removal would delete one-shot containers before their state/output could be inspected. Opt in with `.WithAutoRemove()` when desired. - Partial startup failure (image pull / create / start / wait): the container is torn down via the same `DisposeAsync` path. - Cancellation: a cancelled `StartAsync`/wait triggers the same teardown. diff --git a/docs/wiki/Testing.md b/docs/wiki/Testing.md index caf71b0..93157ab 100644 --- a/docs/wiki/Testing.md +++ b/docs/wiki/Testing.md @@ -79,6 +79,27 @@ reaper and the 4 GB WSLC session VM from being stampeded on a cold run. In Visual Studio, untick **Run Tests in Parallel** (or set *Maximum Parallel Test Projects* to 1) before running the WSLC integration suites. +## Cleaning up named volumes + +Named volumes are retained by default on both backends, matching Docker/Testcontainers. A suite that must +leave no volume state behind can turn on the shared switch before the first container is created: + +```bash +PURVIEW_CONTAINERS_CLEAN_VOLUMES=true dotnet test ... +``` + +```powershell +$env:PURVIEW_CONTAINERS_CLEAN_VOLUMES = "true" +dotnet test ... +``` + +The active backend then deletes the containers it recorded and the named volumes it created when the +process exits or its session terminates (including from the process-exit hook). Volumes that already +existed are never deleted, and the shared image store and host bind mounts are untouched. Set +`ContainerCleanup.CleanUpVolumesOnExit` in code instead if you prefer, or call +`ContainerBackends.CleanUpAsync()` / `IContainerBackend.CleanUpAsync()` for an explicit cleanup. On WSLC, +`WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` can override the shared switch for one runtime. + ## Running the WSLC suites in CI The WSLC half used to be manual-only because no hosted runner was assumed to have WSL Containers. That is diff --git a/package.json b/package.json index 6365b69..63a5358 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "containers", - "version": "1.0.0-prerelease.10", + "version": "1.0.0-prerelease.11", "description": "Testcontainers-style library for .NET that runs throwaway Linux containers for integration testing, on Microsoft WSL Containers or Docker.", "license": "MIT", "author": { diff --git a/skills/purview-containers-lifecycle/SKILL.md b/skills/purview-containers-lifecycle/SKILL.md index 747387a..7856b45 100644 --- a/skills/purview-containers-lifecycle/SKILL.md +++ b/skills/purview-containers-lifecycle/SKILL.md @@ -44,9 +44,22 @@ These are the ones that matter when a test throws: - Use `await using` so failures run `DisposeAsync`. The process-exit hook owns session cleanup, not a per-container disposal registry. +- `EnableCleanUp` defaults to **`true`** (Testcontainers' `WithCleanUp`): `DisposeAsync` stops and + deletes the container; Docker also removes its anonymous volumes. Named `WithVolumeMount` volumes + are shared data and survive (WSLC volumes live on the session). `.WithCleanUp(false)` stops the + container and keeps it — and its volumes — for inspection; remove it with `DeleteAsync` before + disposal, or with the backend's own tooling. - `EnableAutoRemove` defaults to **`false`**: the library owns deletion. WSLC auto-removal would delete a one-shot container before its state or output could be inspected — precisely when you need - them. Opt in with `.WithAutoRemove()`. + them. Opt in with `.WithAutoRemove()`. It is independent of cleanup and wins when the init process + exits. +- `WithReuse()` implies cleanup off, and `WithReuse().WithCleanUp(true)` is rejected at build time. +- Named volumes are **retained by default**, matching Docker. Set + `ContainerCleanup.CleanUpVolumesOnExit = true`, or `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true` before the + first container is created, to delete the volumes the active backend created when the process exits or + its session terminates. `ContainerBackends.CleanUpAsync()` / `IContainerBackend.CleanUpAsync()` does it + explicitly; pre-existing volumes, the shared image store and host bind mounts are never touched. On WSLC, + `WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides the shared switch. - A **partial startup failure** — image pull, create, start or wait — tears down through the same `DisposeAsync` path. So does a **cancelled** `StartAsync` or wait. - Builder-produced containers serialize lifecycle operations; disposal cancels an in-flight startup @@ -155,7 +168,7 @@ Readiness exhaustion throws `ContainerTimeoutException`. Live log tails need a c Native synchronous WSLC calls cannot be interrupted safely. Managed waits and CLI processes are bounded, but do not claim hard zero-hang guarantees for a blocked native runtime. `Wait.ForExitCode(expected)` handles one-shot jobs; `WaitForExitAsync(ct)` returns the actual code. -`WithName(...).WithReuse()` retains a named container on disposal; `DeleteAsync(ct)` explicitly removes +`WithName(...).WithReuse()` retains a named container on disposal, with cleanup implied off; `DeleteAsync(ct)` explicitly removes it. WSLC reuse ends with the owning session; Docker follows Testcontainers reuse semantics. ## Related skills diff --git a/src/src/Core/ContainerBackends.cs b/src/src/Core/ContainerBackends.cs index 24f2395..f2ed25e 100644 --- a/src/src/Core/ContainerBackends.cs +++ b/src/src/Core/ContainerBackends.cs @@ -188,6 +188,22 @@ public static void Reset() } } + /// + /// Best-effort cleanup of resources owned by every registered backend, such as the named volumes they + /// created. Each backend's is called in registration order. + /// Disposing the containers that use those resources first is the caller's responsibility; a backend may + /// force-remove its own retained containers when is + /// enabled. + /// + public static async Task CleanUpAsync(CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + foreach (var backend in All) + { + await backend.CleanUpAsync(cancellationToken).ConfigureAwait(false); + } + } + static async Task ResolveCoreAsync( IContainerBackend? pinned, ContainerBackendSelection selection, diff --git a/src/src/Core/ContainerBuilder.cs b/src/src/Core/ContainerBuilder.cs index 615b842..ced4946 100644 --- a/src/src/Core/ContainerBuilder.cs +++ b/src/src/Core/ContainerBuilder.cs @@ -32,6 +32,7 @@ public abstract class ContainerBuilder bool _privileged; bool _enableGPU; bool _enableAutoRemove; + bool _enableCleanUp = true; PullPolicy _pullPolicy = PullPolicy.Missing; TimeSpan _startupTimeout = TimeSpan.FromMinutes(5); readonly List _waitStrategies = []; @@ -208,6 +209,8 @@ public TBuilder WithNetworkAliases(params string[] aliases) public TBuilder WithReuse(bool reuse = true) { _reuse = reuse; + // Mirrors Testcontainers: reuse owns the lifecycle, so it switches cleanup off. + _enableCleanUp = !reuse; return (TBuilder)this; } @@ -340,6 +343,18 @@ public TBuilder WithAutoRemove(bool enable = true) return (TBuilder)this; } + /// + /// Controls whether disposal removes the container, mirroring Testcontainers' WithCleanUp. Defaults + /// to true. Set false to stop and retain the container (and its volumes) for inspection; + /// remove it with before disposal, or with the backend's own tooling + /// afterwards. Independent of . + /// + public TBuilder WithCleanUp(bool cleanUp = true) + { + _enableCleanUp = cleanUp; + return (TBuilder)this; + } + /// Sets the image pull policy. Defaults to . public TBuilder WithPullPolicy(PullPolicy pullPolicy) { @@ -422,6 +437,7 @@ protected virtual TConfiguration BuildConfiguration() Privileged = _privileged, EnableGPU = _enableGPU, EnableAutoRemove = _enableAutoRemove, + EnableCleanUp = _enableCleanUp, PullPolicy = _pullPolicy, StartupTimeout = _startupTimeout, WaitStrategies = _waitStrategies.ToArray(), @@ -483,6 +499,11 @@ var timeout in new[] ); } + if (configuration.Reuse && configuration.EnableCleanUp) + { + throw new ContainerConfigurationException("Reuse cannot be used in conjunction with WithCleanUp(true)."); + } + if (configuration.NetworkAliases.Count > 0 && string.IsNullOrWhiteSpace(configuration.Network)) { throw new ContainerConfigurationException("Network aliases require WithNetwork(...)."); diff --git a/src/src/Core/ContainerCleanup.cs b/src/src/Core/ContainerCleanup.cs new file mode 100644 index 0000000..9f01015 --- /dev/null +++ b/src/src/Core/ContainerCleanup.cs @@ -0,0 +1,42 @@ +namespace Purview.Containers; + +/// +/// Process-wide cleanup defaults shared by every backend. +/// +/// +/// Backends record the named volumes they create. When is enabled, a +/// backend deletes those volumes (and, where needed, the containers using them) when the process exits or +/// its session terminates. Volumes that already existed before the library used them are never deleted, +/// and neither the shared image store nor host bind mounts are touched. +/// +public static class ContainerCleanup +{ + /// + /// Environment variable that enables . Accepts 1 or + /// true. It is read when this type is first used; set it before creating containers. + /// + public const string CleanUpVolumesEnvironmentVariable = "PURVIEW_CONTAINERS_CLEAN_VOLUMES"; + + static bool CleanUpVolumesOnExitDefault = IsEnabled( + Environment.GetEnvironmentVariable(CleanUpVolumesEnvironmentVariable) + ); + + /// + /// Gets or sets whether backends delete the named volumes they created when the process exits or its + /// session terminates. Defaults to the value of . + /// + /// + /// Set it before creating containers: backends record volume ownership as containers are created. + /// Resources that already existed are not owned and are left alone. The cleanup is best-effort and is + /// also exposed explicitly through and each backend's + /// . + /// + public static bool CleanUpVolumesOnExit + { + get => Volatile.Read(ref CleanUpVolumesOnExitDefault); + set => Volatile.Write(ref CleanUpVolumesOnExitDefault, value); + } + + static bool IsEnabled(string? value) => + value is not null && (value == "1" || (bool.TryParse(value, out var enabled) && enabled)); +} diff --git a/src/src/Core/ContainerConfiguration.cs b/src/src/Core/ContainerConfiguration.cs index 9d3158d..f9044ff 100644 --- a/src/src/Core/ContainerConfiguration.cs +++ b/src/src/Core/ContainerConfiguration.cs @@ -53,6 +53,9 @@ public record ContainerConfiguration : IContainerConfiguration /// public bool EnableAutoRemove { get; init; } + /// + public bool EnableCleanUp { get; init; } = true; + /// public PullPolicy PullPolicy { get; init; } = PullPolicy.Missing; @@ -125,6 +128,7 @@ protected virtual bool PrintMembers(StringBuilder builder) builder.Append(", ").Append(nameof(Privileged)).Append(" = ").Append(Privileged); builder.Append(", ").Append(nameof(EnableGPU)).Append(" = ").Append(EnableGPU); builder.Append(", ").Append(nameof(EnableAutoRemove)).Append(" = ").Append(EnableAutoRemove); + builder.Append(", ").Append(nameof(EnableCleanUp)).Append(" = ").Append(EnableCleanUp); builder.Append(", ").Append(nameof(PullPolicy)).Append(" = ").Append(PullPolicy); builder.Append(", ").Append(nameof(StartupTimeout)).Append(" = ").Append(StartupTimeout); builder.Append(", ").Append(nameof(WaitStrategies)).Append(" = ").Append(WaitStrategies); diff --git a/src/src/Core/IContainerBackend.cs b/src/src/Core/IContainerBackend.cs index 3301e4c..636856f 100644 --- a/src/src/Core/IContainerBackend.cs +++ b/src/src/Core/IContainerBackend.cs @@ -27,4 +27,10 @@ Task BuildImageAsync( ImageBuildConfiguration configuration, CancellationToken cancellationToken = default ) => throw new Runtime.ContainerNotSupportedException("This backend does not support image builds."); + + /// + /// Best-effort removal of resources this backend owns that outlive individual containers, such as the + /// named volumes it created. The default does nothing. Dispose containers before calling it. + /// + Task CleanUpAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; } diff --git a/src/src/Core/IContainerConfiguration.cs b/src/src/Core/IContainerConfiguration.cs index 6564e48..d8b9ba6 100644 --- a/src/src/Core/IContainerConfiguration.cs +++ b/src/src/Core/IContainerConfiguration.cs @@ -50,6 +50,14 @@ public interface IContainerConfiguration /// Automatically remove the container when it stops. bool EnableAutoRemove { get; } + /// + /// Remove the container when it is disposed, mirroring Testcontainers' WithCleanUp. Defaults to + /// true. When false, disposal stops and retains the container (and its volumes) so its + /// state and output can be inspected. Remove a retained container with DeleteAsync before disposal, + /// or with the backend's own tooling. Independent of . + /// + bool EnableCleanUp => true; + /// Image pull policy. Defaults to . PullPolicy PullPolicy { get; } diff --git a/src/src/Core/Sdk/README.md b/src/src/Core/Sdk/README.md index 225c250..4be0403 100644 --- a/src/src/Core/Sdk/README.md +++ b/src/src/Core/Sdk/README.md @@ -20,7 +20,7 @@ separate package. | --- | --- | | Container contract | `IContainer`, `IContainerConfiguration`, `ContainerConfiguration`, `ContainerBuilder`, `ContainerBase` | | Connection strings | `IConnectionStringProvider`, `IConnectionStringProvider`, `ContainerConnectionStringProvider`, `ConnectionMode` | -| Backends | `IContainerBackend`, `ContainerBackendInfo`, `ContainerBackends`, `ContainerBackendUnavailableException` | +| Backends | `IContainerBackend`, `ContainerBackendInfo`, `ContainerBackends`, `ContainerBackendUnavailableException`, `ContainerCleanup` | | Readiness | `Purview.Containers.Waiting`: `Wait`, `IWaitStrategy`, TCP/HTTP/command/log strategies | | Configuration model | `PortBinding`, `BindMount`, `NamedVolume`, `RegistryCredentials`, `Image`, `PullPolicy` | | Diagnostics | `Secret`, `SecretRedactor`, `ContainerState`, `ExecResult`, `ContainerLogEntry` | @@ -30,9 +30,14 @@ separate package. The shared contract includes entrypoint/user/labels, exec environment overrides, pre-start `WithResourceMapping`, runtime `CopyAsync`/`ReadFileAsync`, `Wait.ForExitCode`/`WaitForExitAsync`, -`IContainerBackend.CreateNetworkAsync`/`BuildImageAsync`, and explicit `DeleteAsync`. +`IContainerBackend.CreateNetworkAsync`/`BuildImageAsync`/`CleanUpAsync`, and explicit `DeleteAsync`. Networks/images return an `IContainerResource`; dispose attached containers first. -Reuse requires `WithName(...).WithReuse()` and changes disposal ownership. +Disposal cleanup follows Testcontainers: `WithCleanUp(true)` (the default) removes the container, +`WithCleanUp(false)` stops and retains it, and the Docker backend also removes anonymous volumes. +Named volumes are owned per backend: set `ContainerCleanup.CleanUpVolumesOnExit` (or +`PURVIEW_CONTAINERS_CLEAN_VOLUMES=true`) before creating containers to delete the volumes the library +created, or call `ContainerBackends.CleanUpAsync()` explicitly. Pre-existing volumes are never deleted. +Reuse requires `WithName(...).WithReuse()`, implies cleanup off, and changes disposal ownership. Defaults: total startup 5 minutes; exec/exit wait 1 minute; stop 15 seconds; cleanup 30 seconds; copy/read/log snapshots 1 minute. Fluent timeout overrides and cancellation tokens are supported. diff --git a/src/src/Docker/DockerContainer.cs b/src/src/Docker/DockerContainer.cs index a57934d..57f36ca 100644 --- a/src/src/Docker/DockerContainer.cs +++ b/src/src/Docker/DockerContainer.cs @@ -17,20 +17,24 @@ public sealed class DockerContainer : IContainer { readonly IContainerConfiguration _configuration; readonly Func> _create; + readonly DockerVolumeCleanup _cleanup; TcContainer? _handle; int _disposed; bool _deleted; + ContainerState? _retainedState; readonly CancellationTokenSource _lifetime = new(); Task? _exitTask; internal DockerContainer( IContainerConfiguration configuration, Func> create, - string name + string name, + DockerVolumeCleanup cleanup ) { _configuration = configuration; _create = create; + _cleanup = cleanup; Name = name; } @@ -45,10 +49,14 @@ string name /// public ContainerState State => - _deleted ? ContainerState.Deleted - : _handle is null ? ContainerState.Created - : _exitTask?.IsCompletedSuccessfully == true ? ContainerState.Exited - : ToState(Handle.State); + _deleted + ? ContainerState.Deleted + : _retainedState + ?? ( + _handle is null ? ContainerState.Created + : _exitTask?.IsCompletedSuccessfully == true ? ContainerState.Exited + : ToState(Handle.State) + ); /// public string Image => _configuration.Image; @@ -72,6 +80,11 @@ public async Task StartAsync(CancellationToken cancellationToken = default) { _handle ??= await _create(deadline.Token).ConfigureAwait(false); await Handle.StartAsync(deadline.Token).ConfigureAwait(false); + if (!_configuration.Reuse) + { + _cleanup.RegisterContainer(Handle.Id); + } + _exitTask = ObserveExitAsync(); _ = _exitTask.ContinueWith( static task => _ = task.Exception, @@ -292,9 +305,28 @@ public async ValueTask DisposeAsync() { await _lifetime.CancelAsync().ConfigureAwait(false); if (_handle is not null) + { + var stateBeforeDispose = State; await Handle.DisposeAsync().AsTask().WaitAsync(_configuration.CleanupTimeout).ConfigureAwait(false); - if (!_configuration.Reuse) - _deleted = true; + if (_configuration.Reuse || !_configuration.EnableCleanUp) + { + if (!_configuration.Reuse && _configuration.EnableAutoRemove) + { + // Docker's --rm removes the container as it stops, so nothing is retained. + _deleted = true; + } + else + { + // Testcontainers stops a retained container; it is not removed, so state stays observable. + _retainedState = + stateBeforeDispose == ContainerState.Running ? ContainerState.Exited : stateBeforeDispose; + } + } + else + { + _deleted = true; + } + } } finally { diff --git a/src/src/Docker/DockerContainerBackend.cs b/src/src/Docker/DockerContainerBackend.cs index eafcfb8..303f482 100644 --- a/src/src/Docker/DockerContainerBackend.cs +++ b/src/src/Docker/DockerContainerBackend.cs @@ -13,6 +13,7 @@ namespace Purview.Containers.Docker; public sealed class DockerContainerBackend : IContainerBackend, IContainerBackendPreference { readonly Lazy _wslDockerHost = new(WslDockerHost.Detect); + readonly DockerVolumeCleanup _cleanup = new(); /// Stable backend identifier. public string Name => "docker"; @@ -31,9 +32,17 @@ public IContainer CreateContainer(IContainerConfiguration configuration) { ArgumentNullException.ThrowIfNull(configuration); var name = ContainerName.Generate(configuration); - return new DockerContainer(configuration, token => CreateHandleAsync(configuration, name, token), name); + return new DockerContainer( + configuration, + token => CreateHandleAsync(configuration, name, token), + name, + _cleanup + ); } + /// + public Task CleanUpAsync(CancellationToken cancellationToken = default) => _cleanup.CleanUpAsync(cancellationToken); + /// public async Task GetInfoAsync(CancellationToken cancellationToken = default) { @@ -169,6 +178,7 @@ await client } } cancellationToken.ThrowIfCancellationRequested(); + await _cleanup.EnsureOwnedAsync(configuration.NamedVolumes, cancellationToken).ConfigureAwait(false); var container = CreateBuilder(configuration, name).Build(); if (configuration.ResourceMappings.Count > 0) { @@ -192,6 +202,7 @@ TcContainerBuilder CreateBuilder(IContainerConfiguration configuration, string n .WithName(name) .WithAutoRemove(configuration.EnableAutoRemove) .WithReuse(configuration.Reuse) + .WithCleanUp(configuration.EnableCleanUp) .WithImagePullPolicy( (configuration.RegistryCredentials is null ? configuration.PullPolicy : Images.PullPolicy.Never) switch { diff --git a/src/src/Docker/DockerVolumeCleanup.cs b/src/src/Docker/DockerVolumeCleanup.cs new file mode 100644 index 0000000..65585bd --- /dev/null +++ b/src/src/Docker/DockerVolumeCleanup.cs @@ -0,0 +1,172 @@ +using System.Collections.Concurrent; +using System.Net; +using Docker.DotNet.Models; +using DotNet.Testcontainers.Configurations; +using Purview.Containers.Mounts; + +namespace Purview.Containers.Docker; + +/// +/// Owns the named Docker volumes this backend created and, when cleanup is enabled, removes them (and the +/// containers that used them) on process exit or on an explicit backend cleanup. +/// +sealed class DockerVolumeCleanup +{ + const string OwnerLabel = "purview.containers.owner"; + const string CleanupLabel = "purview.containers.cleanup"; + const string OwnerValue = "volume"; + + static readonly ConcurrentDictionary Trackers = new(); + static int HookRegistered; + + readonly ConcurrentDictionary _volumes = new(StringComparer.Ordinal); + readonly ConcurrentDictionary _containers = new(StringComparer.Ordinal); + + /// + /// Creates the named volume when it does not already exist, so only volumes created by this backend are + /// ever deleted. An existing volume is left unowned. + /// + internal async Task EnsureOwnedAsync(IReadOnlyList volumes, CancellationToken cancellationToken) + { + if (!ContainerCleanup.CleanUpVolumesOnExit || volumes.Count == 0) + { + return; + } + + Trackers[this] = 0; + EnsureProcessExitHook(); + using var client = CreateClient(); + foreach (var volume in volumes) + { + cancellationToken.ThrowIfCancellationRequested(); + try + { + await client + .Volumes.CreateAsync( + new VolumesCreateParameters + { + Name = volume.Name, + Labels = new Dictionary(StringComparer.Ordinal) + { + [OwnerLabel] = OwnerValue, + [CleanupLabel] = "true", + }, + }, + cancellationToken + ) + .ConfigureAwait(false); + + // Docker returns an existing volume unchanged, so only a volume carrying our labels is ours. + var created = await client.Volumes.InspectAsync(volume.Name, cancellationToken).ConfigureAwait(false); + if ( + created.Labels is { } labels + && labels.TryGetValue(OwnerLabel, out var owner) + && string.Equals(owner, OwnerValue, StringComparison.Ordinal) + && labels.TryGetValue(CleanupLabel, out var cleanup) + && string.Equals(cleanup, "true", StringComparison.Ordinal) + ) + { + _volumes[volume.Name] = 0; + } + } + catch (global::Docker.DotNet.DockerApiException) + { + // Let container creation report the underlying driver or permission problem. + } + } + } + + /// Records a container so cleanup can force-remove it before deleting its volumes. + internal void RegisterContainer(string id) + { + if (!ContainerCleanup.CleanUpVolumesOnExit || string.IsNullOrEmpty(id)) + { + return; + } + + Trackers[this] = 0; + EnsureProcessExitHook(); + _containers[id] = 0; + } + + internal async Task CleanUpAsync(CancellationToken cancellationToken = default) + { + if (_volumes.IsEmpty && _containers.IsEmpty) + { + return; + } + + using var client = CreateClient(); + foreach (var id in _containers.Keys) + { + try + { + await client + .Containers.RemoveContainerAsync( + id, + new ContainerRemoveParameters { Force = true, RemoveVolumes = false }, + cancellationToken + ) + .ConfigureAwait(false); + } + catch (global::Docker.DotNet.DockerApiException exception) + when (exception.StatusCode == HttpStatusCode.NotFound) + { + // Already removed by Testcontainers, the caller, or the resource reaper. + } + catch (global::Docker.DotNet.DockerApiException) + { + // Best-effort: a volume still in use is reported by the volume removal below. + } + } + + _containers.Clear(); + foreach (var name in _volumes.Keys) + { + try + { + await client.Volumes.RemoveAsync(name, force: true, cancellationToken).ConfigureAwait(false); + } + catch (global::Docker.DotNet.DockerApiException) + { + // Best-effort cleanup; never fail the caller for a volume that is already gone or in use. + } + } + + _volumes.Clear(); + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage( + "Design", + "CA1031:Do not catch general exception types", + Justification = "Process-exit cleanup is best-effort; the process is already terminating." + )] + static void EnsureProcessExitHook() + { + if (Interlocked.Exchange(ref HookRegistered, 1) == 1) + { + return; + } + + AppDomain.CurrentDomain.ProcessExit += (_, _) => + { + if (!ContainerCleanup.CleanUpVolumesOnExit) + { + return; + } + + using CancellationTokenSource deadline = new(TimeSpan.FromSeconds(30)); + foreach (var tracker in Trackers.Keys) + { + try + { + tracker.CleanUpAsync(deadline.Token).GetAwaiter().GetResult(); + } + catch { } + } + }; + } + + static global::Docker.DotNet.DockerClient CreateClient() => + TestcontainersSettings.OS.DockerEndpointAuthConfig.GetDockerClientBuilder(Guid.NewGuid()).Build(); +} diff --git a/src/src/Docker/Sdk/README.md b/src/src/Docker/Sdk/README.md index 050c04a..0e5d18a 100644 --- a/src/src/Docker/Sdk/README.md +++ b/src/src/Docker/Sdk/README.md @@ -61,6 +61,11 @@ PURVIEW_CONTAINERS_BACKEND=docker # or "auto" (the default) to probe WSLC fi Reuse follows Testcontainers semantics and disables normal reaper cleanup; `DeleteAsync` removes retained containers explicitly. See [capabilities and ownership](https://github.com/purview-dev/containers/blob/main/docs/wiki/Backend-Capabilities.md). +- Named volumes created through `WithVolumeMount` are tracked by the backend. Set + `ContainerCleanup.CleanUpVolumesOnExit` (or `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true`) before creating + containers to force-remove those containers and the volumes the backend created when the process exits; + `CleanUpAsync` does it explicitly. Volumes that already existed are never deleted. The shared image + store and bind mounts are untouched. - Log tailing polls the accumulated output until the container stops, because the Testcontainers API exposes no positioned log stream. diff --git a/src/src/Wsl/IContainerRuntime.cs b/src/src/Wsl/IContainerRuntime.cs index 5fbbb7a..c1ca112 100644 --- a/src/src/Wsl/IContainerRuntime.cs +++ b/src/src/Wsl/IContainerRuntime.cs @@ -8,4 +8,10 @@ public interface IContainerRuntime : IAsyncDisposable /// Returns the (shared, lazily started) session, creating it on first use. Task GetSessionAsync(CancellationToken cancellationToken = default); + + /// + /// Best-effort cleanup of the resources the current session owns, such as named volumes. Does nothing + /// when no session has been created. The default does nothing. + /// + Task CleanUpAsync(CancellationToken cancellationToken = default) => Task.CompletedTask; } diff --git a/src/src/Wsl/Sdk/README.md b/src/src/Wsl/Sdk/README.md index dd6e776..0168972 100644 --- a/src/src/Wsl/Sdk/README.md +++ b/src/src/Wsl/Sdk/README.md @@ -80,7 +80,7 @@ invalid images, ports and mounts fail during configuration rather than at pull t | `WithPortBinding(ushort, bool assignRandomHostPort)`, `WithPortBinding(ushort, ushort hostPort, ...)`, `WithPortBinding(PortBinding)` | Host port mappings; random host ports use native `windowsPort=0` allocation. | | `WithBindMount(string hostPath, string containerPath, bool readOnly = false)` | Bind-mount a Windows directory. | | `WithVolumeMount(string name, string containerPath, bool readOnly = false)` | Named volume on the session VHD. | -| `WithNetworkMode`, `WithPrivileged`, `WithGPU`, `WithAutoRemove`, `WithPullPolicy`, `WithStartupTimeout`, `WithWaitStrategy`, `WithRegistryCredentials`, `WithRuntime` | Runtime behaviour and readiness. | +| `WithNetworkMode`, `WithPrivileged`, `WithGPU`, `WithAutoRemove`, `WithCleanUp`, `WithPullPolicy`, `WithStartupTimeout`, `WithWaitStrategy`, `WithRegistryCredentials`, `WithRuntime` | Runtime behaviour and readiness. | ## Container API @@ -99,7 +99,8 @@ mode combined with CLI-only features and multi-line environment values. See `IContainer` (and `WslContainer`) exposes `StartAsync`, `StopAsync`, `ExecAsync`, `GetMappedPublicPort`, `GetMappedPublicPorts`, `GetLogsAsync(LogOutput?)` and a tailing `IAsyncEnumerable` `GetLogsAsync(CancellationToken)`. `Id`, `Name`, `State` and `Image` describe the running container. -`DisposeAsync` normally stops and deletes the container; reusable containers are retained. It is +`DisposeAsync` normally stops and deletes the container; reusable containers and containers built with +`WithCleanUp(false)` are stopped and retained. It is idempotent and never terminates the shared session. Stopping signals the init process with SIGTERM, waits up to 10 seconds, then escalates to SIGKILL with a further 5-second exit wait. The grace period uses managed polling rather than the native @@ -148,8 +149,18 @@ ContainerBackends.Use(new WslContainerBackend(WslContainerRuntimeOptions.Default ``` The full `WslContainerRuntimeOptions` set (CPU, memory, GPU, session name, `StoragePath`, `StorageMode`, -timeout) is honoured on both a platform-neutral (`net10.0`) consumer — where the facade forwards it to the -Windows build at run time — and a Windows target framework, where the Windows build applies it directly. +timeout, `CleanUpVolumesOnTerminate`) is honoured on both a platform-neutral (`net10.0`) consumer — where the +facade forwards it to the Windows build at run time — and a Windows target framework, where the Windows build +applies it directly. + +Named volumes live on the session and are **retained by default**, matching Docker/Testcontainers. Set +`ContainerCleanup.CleanUpVolumesOnExit = true`, or set `PURVIEW_CONTAINERS_CLEAN_VOLUMES=true` before the +first container is created, to delete every volume the session recorded when it terminates. The +WSLC-specific `WslContainerRuntimeOptions.CleanUpVolumesOnTerminate` overrides that shared switch with +`true`/`false`; leave it `null` to use the shared value. `IContainerBackend.CleanUpAsync()` (and +`ContainerBackends.CleanUpAsync()`) performs the same cleanup explicitly. That makes a test run leave no +session volume state behind; the shared image store and host bind mounts are never touched. Containers are +deleted first, then the volumes. The Microsoft types (`Session`, `Container`, `Process`, …) stay behind the public interfaces; the only escape hatch is the opt-in accessor for `Inspect()` and raw handles. diff --git a/src/src/Wsl/WslContainer.cs b/src/src/Wsl/WslContainer.cs index 03766e0..cd66c6d 100644 --- a/src/src/Wsl/WslContainer.cs +++ b/src/src/Wsl/WslContainer.cs @@ -439,14 +439,20 @@ await _session } catch { } - try + _state = + Configuration.EnableAutoRemove && !Configuration.EnableCleanUp + ? ContainerState.Deleted + : MapState(_handle.State); + if (Configuration.EnableCleanUp) { - _state = MapState(_handle.State); - await _session - .DeleteContainerAsync(_handle, DeleteContainerOption.Force, cleanup.Token) - .ConfigureAwait(false); + try + { + await _session + .DeleteContainerAsync(_handle, DeleteContainerOption.Force, cleanup.Token) + .ConfigureAwait(false); + } + catch { } } - catch { } try { diff --git a/src/src/Wsl/WslContainerBackend.Facade.cs b/src/src/Wsl/WslContainerBackend.Facade.cs index bd44e2c..45e9cfc 100644 --- a/src/src/Wsl/WslContainerBackend.Facade.cs +++ b/src/src/Wsl/WslContainerBackend.Facade.cs @@ -49,6 +49,13 @@ public IContainer CreateContainer(IContainerConfiguration configuration) return Resolve().CreateContainer(configuration); } + /// + public Task CleanUpAsync(CancellationToken cancellationToken = default) + { + var backend = WslPayload.TryCreateBackend(); + return backend is null ? Task.CompletedTask : backend.CleanUpAsync(cancellationToken); + } + /// public async Task GetInfoAsync(CancellationToken cancellationToken = default) { diff --git a/src/src/Wsl/WslContainerBackend.cs b/src/src/Wsl/WslContainerBackend.cs index 9c226e2..77c18b6 100644 --- a/src/src/Wsl/WslContainerBackend.cs +++ b/src/src/Wsl/WslContainerBackend.cs @@ -52,7 +52,8 @@ internal static WslContainerBackend Create( string? storagePath, int storageMode, long? sessionTimeoutTicks, - bool disableProcessExitCleanup + bool disableProcessExitCleanup, + bool? cleanUpVolumesOnTerminate ) { WslContainerRuntimeOptions options = new() @@ -65,6 +66,7 @@ bool disableProcessExitCleanup StorageMode = (StorageMode)storageMode, SessionTimeout = sessionTimeoutTicks is { } ticks ? TimeSpan.FromTicks(ticks) : null, DisableProcessExitCleanup = disableProcessExitCleanup, + CleanUpVolumesOnTerminate = cleanUpVolumesOnTerminate, }; return new WslContainerBackend(options); } @@ -109,6 +111,9 @@ public async Task GetInfoAsync(CancellationToken cancellat ); } + /// + public Task CleanUpAsync(CancellationToken cancellationToken = default) => Runtime.CleanUpAsync(cancellationToken); + /// public async Task CreateNetworkAsync( NetworkConfiguration? configuration = null, diff --git a/src/src/Wsl/WslContainerRuntime.Facade.cs b/src/src/Wsl/WslContainerRuntime.Facade.cs index a7c9b0b..266fd43 100644 --- a/src/src/Wsl/WslContainerRuntime.Facade.cs +++ b/src/src/Wsl/WslContainerRuntime.Facade.cs @@ -25,6 +25,12 @@ public sealed class WslContainerRuntime : IContainerRuntime /// The pre-rename storage path variable, still honoured as a fallback. public const string LegacyStoragePathEnvironmentVariable = "WSL_CONTAINERS_STORAGE_PATH"; + /// + /// Environment variable that turns on for every + /// backend. Kept here for discoverability; it is the same shared setting. + /// + public const string CleanUpVolumesEnvironmentVariable = ContainerCleanup.CleanUpVolumesEnvironmentVariable; + static readonly Lazy InstanceHolder = new(() => new WslContainerRuntime()); /// The default process-wide runtime. @@ -69,6 +75,13 @@ public Task GetSessionAsync(CancellationToken cancellationTok + "(net10.0-windows10.0.19041.0 or later). Use the backend-neutral container API instead." ); + /// + public Task CleanUpAsync(CancellationToken cancellationToken = default) + { + var backend = WslPayload.TryCreateBackend(); + return backend is null ? Task.CompletedTask : backend.CleanUpAsync(cancellationToken); + } + /// public ValueTask DisposeAsync() => ValueTask.CompletedTask; } diff --git a/src/src/Wsl/WslContainerRuntime.cs b/src/src/Wsl/WslContainerRuntime.cs index 4612796..a27c37a 100644 --- a/src/src/Wsl/WslContainerRuntime.cs +++ b/src/src/Wsl/WslContainerRuntime.cs @@ -27,6 +27,12 @@ public sealed class WslContainerRuntime : IContainerRuntime /// The pre-rename storage path variable, still honoured as a fallback. public const string LegacyStoragePathEnvironmentVariable = "WSL_CONTAINERS_STORAGE_PATH"; + /// + /// Environment variable that turns on for every + /// backend. Kept here for discoverability; it is the same shared setting. + /// + public const string CleanUpVolumesEnvironmentVariable = ContainerCleanup.CleanUpVolumesEnvironmentVariable; + static readonly Lazy InstanceHolder = new(() => new WslContainerRuntime()); readonly WslContainerRuntimeOptions _options; @@ -144,6 +150,14 @@ async Task EnsureStoreAsync(WslContainerSession session, Ca } } + /// + public Task CleanUpAsync(CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + _session?.CleanUpRetainedResources(force: true); + return Task.CompletedTask; + } + /// public async ValueTask DisposeAsync() { diff --git a/src/src/Wsl/WslContainerRuntimeOptions.cs b/src/src/Wsl/WslContainerRuntimeOptions.cs index 7f7c562..429dfee 100644 --- a/src/src/Wsl/WslContainerRuntimeOptions.cs +++ b/src/src/Wsl/WslContainerRuntimeOptions.cs @@ -33,6 +33,19 @@ public sealed record WslContainerRuntimeOptions /// Session idle timeout. public TimeSpan? SessionTimeout { get; init; } + /// + /// Overrides for this runtime. true deletes the + /// named session volumes at termination, false retains them, and null uses the shared + /// setting. Defaults to null. + /// + /// + /// Enable this - or set PURVIEW_CONTAINERS_CLEAN_VOLUMES=true before the first container is + /// created - for test runs that must leave no volume state behind. Only volumes recorded for this + /// session are deleted; the shared image store is never touched. Session termination already removes + /// the session's containers, so this option adds the volume deletion that Testcontainers does not do. + /// + public bool? CleanUpVolumesOnTerminate { get; init; } + /// Disables the process-exit cleanup hook (mainly for testing). public bool DisableProcessExitCleanup { get; init; } diff --git a/src/src/Wsl/WslContainerSession.cs b/src/src/Wsl/WslContainerSession.cs index eba83ce..8d89c36 100644 --- a/src/src/Wsl/WslContainerSession.cs +++ b/src/src/Wsl/WslContainerSession.cs @@ -21,8 +21,12 @@ sealed class WslContainerSession : IContainerSession readonly Session _session; readonly SemaphoreSlim _lifecycleGate = new(1, 1); readonly ConcurrentDictionary> _activePulls = new(StringComparer.Ordinal); + readonly ConcurrentDictionary _containerNames = new(StringComparer.Ordinal); + readonly ConcurrentDictionary _namedVolumes = new(StringComparer.Ordinal); + readonly Lock _cleanupSync = new(); readonly string _storagePath; readonly bool _deleteStorageOnTerminate; + readonly bool _cleanUpVolumesOnTerminate; int _started; int _terminated; @@ -36,6 +40,7 @@ public WslContainerSession( Name = name; _storagePath = storagePath; _deleteStorageOnTerminate = deleteStorageOnTerminate; + _cleanUpVolumesOnTerminate = options.CleanUpVolumesOnTerminate ?? ContainerCleanup.CleanUpVolumesOnExit; SessionSettings settings = new(name, storagePath) { CpuCount = options.CPUCount, @@ -146,6 +151,7 @@ public async Task TerminateAsync(CancellationToken cancellationToken = default) return; } + CleanUpRetainedResources(force: false); try { _session.Terminate(); @@ -171,6 +177,7 @@ internal void TerminateBestEffort() return; } + CleanUpRetainedResources(force: false); try { _session.Terminate(); @@ -216,6 +223,14 @@ CancellationToken cancellationToken try { ThrowIfTerminated(); + if (!string.IsNullOrWhiteSpace(settings.Name)) + _containerNames[settings.Name] = 0; + if (settings.NamedVolumes is { } volumes) + { + foreach (var volume in volumes) + _namedVolumes[volume.Name] = 0; + } + return _session.CreateContainer(settings); } finally @@ -237,6 +252,10 @@ CancellationToken cancellationToken try { ThrowIfTerminated(); + _containerNames[name] = 0; + foreach (var volume in configuration.NamedVolumes) + _namedVolumes[volume.Name] = 0; + var hash = WslCreateArguments.ReuseHash(configuration); if (configuration.Reuse) { @@ -598,6 +617,73 @@ CancellationToken cancellationToken } } + /// + /// Best-effort removal of the containers and named volumes this session created. Containers are deleted + /// first because a volume still attached to a container cannot be removed. + /// + /// + /// When true, cleanup runs even if automatic volume cleanup is disabled. Used by the explicit + /// path. + /// + internal void CleanUpRetainedResources(bool force) + { + if (!force && !_cleanUpVolumesOnTerminate) + { + return; + } + + if (Volatile.Read(ref _started) == 0) + { + return; + } + + lock (_cleanupSync) + { + foreach (var containerName in _containerNames.Keys) + { + try + { + var container = _session.OpenContainer(containerName, ProcessOutputMode.Event); + try + { + container.Delete(DeleteContainerOption.Force); + } + finally + { + container.Dispose(); + } + } + catch { } + } + + foreach (var volumeName in _namedVolumes.Keys) + { + DeleteVolumeBestEffort(volumeName); + } + + _containerNames.Clear(); + _namedVolumes.Clear(); + } + } + + void DeleteVolumeBestEffort(string name) + { + for (var attempt = 0; attempt < 3; attempt++) + { + try + { + _session.DeleteVhdVolume(name); + return; + } + catch + { + // The container handle can still be detaching the volume; retry briefly. + if (attempt < 2) + Thread.Sleep(50); + } + } + } + void TryDeleteStorage() { if (!_deleteStorageOnTerminate) diff --git a/src/src/Wsl/WslPayload.cs b/src/src/Wsl/WslPayload.cs index 59c77e0..5c3ac9e 100644 --- a/src/src/Wsl/WslPayload.cs +++ b/src/src/Wsl/WslPayload.cs @@ -44,6 +44,7 @@ static class WslPayload typeof(int), typeof(long?), typeof(bool), + typeof(bool?), ]; static bool DefaultAttempted; static IContainerBackend? DefaultBackend; @@ -139,6 +140,7 @@ static class WslPayload (int)options.StorageMode, options.SessionTimeout?.Ticks, options.DisableProcessExitCleanup, + options.CleanUpVolumesOnTerminate, ]; if (factory?.Invoke(null, arguments) is IContainerBackend backend) diff --git a/src/tests/Docker.IntegrationTests/DockerContainerTests.cs b/src/tests/Docker.IntegrationTests/DockerContainerTests.cs index f96ae59..3329d87 100644 --- a/src/tests/Docker.IntegrationTests/DockerContainerTests.cs +++ b/src/tests/Docker.IntegrationTests/DockerContainerTests.cs @@ -1,4 +1,6 @@ +using Docker.DotNet.Models; using Purview.Containers.Waiting; +using TcSettings = DotNet.Testcontainers.Configurations.TestcontainersSettings; namespace Purview.Containers.Docker; @@ -181,6 +183,148 @@ public async Task DisposeAsync_IsIdempotent() await container.DisposeAsync(); await container.DisposeAsync(); - await Assert.That(container.State).IsNotEqualTo(ContainerState.Running); + await Assert.That(container.State).IsEqualTo(ContainerState.Deleted); + } + + [Test] + public async Task DisposeAsync_WithCleanUpDisabled_RetainsTheStoppedContainer(CancellationToken cancellationToken) + { + // Arrange + await SkipIfUnavailableAsync(); + var container = new ContainerBuilder() + .WithBackend(DockerContainerBackend.Create()) + .WithImage(Image) + .WithCommand("/bin/sh", "-c", "echo ready && sleep 60") + .WithCleanUp(false) + .Build(); + await container.StartAsync(cancellationToken); + var id = container.Id; + + try + { + // Act + await container.DisposeAsync(); + + // Assert: Testcontainers stops the container but leaves it on the daemon. + await Assert.That(container.State).IsEqualTo(ContainerState.Exited); + using var client = TcSettings.OS.DockerEndpointAuthConfig.GetDockerClientBuilder(Guid.NewGuid()).Build(); + var inspect = await client.Containers.InspectContainerAsync(id, cancellationToken); + await Assert.That(inspect.State?.Status).IsEqualTo("exited"); + } + finally + { + // Remove the retained container explicitly so the daemon is left clean. + using var client = TcSettings.OS.DockerEndpointAuthConfig.GetDockerClientBuilder(Guid.NewGuid()).Build(); + try + { + await client.Containers.RemoveContainerAsync( + id, + new ContainerRemoveParameters { Force = true, RemoveVolumes = true }, + cancellationToken + ); + } + catch (global::Docker.DotNet.DockerApiException exception) + when (exception.StatusCode == System.Net.HttpStatusCode.NotFound) { } + } + } + + [Test] + [NotInParallel] + public async Task CleanUpAsync_RemovesNamedVolumesTheBackendCreated(CancellationToken cancellationToken) + { + // Arrange + await SkipIfUnavailableAsync(); + var previous = ContainerCleanup.CleanUpVolumesOnExit; + var volumeName = "purview-cleanup-" + Guid.NewGuid().ToString("N")[..8]; + var backend = DockerContainerBackend.Create(); + ContainerCleanup.CleanUpVolumesOnExit = true; + string? containerId = null; + try + { + var container = new ContainerBuilder(backend) + .WithImage(Image) + .WithVolumeMount(volumeName, "/data") + .WithCommand("/bin/sh", "-c", "echo data > /data/file.txt; sleep 60") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test -f /data/file.txt")) + .Build(); + await container.StartAsync(cancellationToken); + containerId = container.Id; + + using (var client = CreateClient()) + { + var volume = await client.Volumes.InspectAsync(volumeName, cancellationToken); + await Assert.That(volume.Labels.ContainsKey("purview.containers.owner")).IsTrue(); + } + + // Act + await backend.CleanUpAsync(cancellationToken); + + // Assert + using var verify = CreateClient(); + await Assert.That(await VolumeExistsAsync(verify, volumeName, cancellationToken)).IsFalse(); + await Assert.That(await ContainerExistsAsync(verify, containerId, cancellationToken)).IsFalse(); + } + finally + { + ContainerCleanup.CleanUpVolumesOnExit = previous; + using var client = CreateClient(); + if (containerId is not null) + { + try + { + await client.Containers.RemoveContainerAsync( + containerId, + new ContainerRemoveParameters { Force = true, RemoveVolumes = false }, + CancellationToken.None + ); + } + catch (global::Docker.DotNet.DockerApiException) { } + } + + try + { + await client.Volumes.RemoveAsync(volumeName, force: true, CancellationToken.None); + } + catch (global::Docker.DotNet.DockerApiException) { } + } + } + + static global::Docker.DotNet.DockerClient CreateClient() => + TcSettings.OS.DockerEndpointAuthConfig.GetDockerClientBuilder(Guid.NewGuid()).Build(); + + static async Task VolumeExistsAsync( + global::Docker.DotNet.DockerClient client, + string name, + CancellationToken cancellationToken + ) + { + try + { + await client.Volumes.InspectAsync(name, cancellationToken); + return true; + } + catch (global::Docker.DotNet.DockerApiException exception) + when (exception.StatusCode == System.Net.HttpStatusCode.NotFound) + { + return false; + } + } + + static async Task ContainerExistsAsync( + global::Docker.DotNet.DockerClient client, + string id, + CancellationToken cancellationToken + ) + { + try + { + await client.Containers.InspectContainerAsync(id, cancellationToken); + return true; + } + catch (global::Docker.DotNet.DockerApiException exception) + when (exception.StatusCode == System.Net.HttpStatusCode.NotFound) + { + return false; + } } } diff --git a/src/tests/Wsl.IntegrationTests/CleanupTests.cs b/src/tests/Wsl.IntegrationTests/CleanupTests.cs index 79a0230..d540a22 100644 --- a/src/tests/Wsl.IntegrationTests/CleanupTests.cs +++ b/src/tests/Wsl.IntegrationTests/CleanupTests.cs @@ -61,4 +61,25 @@ public async Task ImmediateExit_ThenDispose_Works() await container.StartAsync(); await container.DisposeAsync(); } + + [Test] + public async Task Dispose_WithCleanUpDisabled_RetainsTheStoppedContainer() + { + await WslcTest.SkipIfUnavailableAsync(); + + var name = "cleanup-" + Guid.NewGuid().ToString("N")[..8]; + var container = new ContainerBuilder() + .WithImage("alpine:latest") + .WithName(name) + .WithCommand("/bin/sleep", "300") + .WithCleanUp(false) + .Build(); + + await container.StartAsync(); + await container.DisposeAsync(); + + // The wrapper is disposed, but the container is retained until the shared session is terminated + // when the test process exits. + await Assert.That(container.State).IsEqualTo(ContainerState.Exited); + } } diff --git a/src/tests/Wsl.IntegrationTests/VolumeTests.cs b/src/tests/Wsl.IntegrationTests/VolumeTests.cs index 5857f2f..f53e3ca 100644 --- a/src/tests/Wsl.IntegrationTests/VolumeTests.cs +++ b/src/tests/Wsl.IntegrationTests/VolumeTests.cs @@ -74,4 +74,155 @@ public async Task NamedVolume_IsSharedBetweenContainers(CancellationToken cancel await Assert.That(logs.Any(text => text.Contains("shared-data", StringComparison.Ordinal))).IsTrue(); } + + [Test] + [Arguments(false)] + [Arguments(true)] + public async Task SessionCleanup_DeletesNamedVolumesOnTerminate(bool useCli, CancellationToken cancellationToken) + { + await WslcTest.SkipIfUnavailableAsync(); + + var storagePath = Path.Combine(Path.GetTempPath(), "wslc-session-cleanup-" + Guid.NewGuid().ToString("N")[..8]); + var volume = "wslc-clean-" + Guid.NewGuid().ToString("N")[..6]; + try + { + // The writer is deliberately not disposed: session cleanup owns the container and its volume. + await using (var runtime = CreateRuntime(storagePath, cleanUpVolumes: true)) + { + var writerBuilder = new ContainerBuilder(new WslContainerBackend(runtime)) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "echo cleanup-data > /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test -f /data/file.txt")); + if (useCli) + { + // A label forces the CLI creation path, which records volumes separately. + writerBuilder = writerBuilder.WithLabel("purview.test", "session-cleanup"); + } + + var writer = writerBuilder.Build(); + await writer.StartAsync(cancellationToken); + } + + await using (var runtime = CreateRuntime(storagePath, cleanUpVolumes: true)) + { + await using var reader = new ContainerBuilder(new WslContainerBackend(runtime)) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "test ! -f /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test ! -f /data/file.txt")) + .Build(); + await reader.StartAsync(cancellationToken); + } + } + finally + { + DeleteStorageBestEffort(storagePath); + } + } + + [Test] + public async Task SessionCleanup_Disabled_RetainsNamedVolumesAcrossRuntimes(CancellationToken cancellationToken) + { + await WslcTest.SkipIfUnavailableAsync(); + + var storagePath = Path.Combine(Path.GetTempPath(), "wslc-session-retain-" + Guid.NewGuid().ToString("N")[..8]); + var volume = "wslc-retain-" + Guid.NewGuid().ToString("N")[..6]; + try + { + await using (var runtime = CreateRuntime(storagePath, cleanUpVolumes: false)) + { + var writer = new ContainerBuilder(new WslContainerBackend(runtime)) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "echo retained-data > /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test -f /data/file.txt")) + .Build(); + await writer.StartAsync(cancellationToken); + } + + await using (var runtime = CreateRuntime(storagePath, cleanUpVolumes: false)) + { + await using var reader = new ContainerBuilder(new WslContainerBackend(runtime)) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "test -f /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test -f /data/file.txt")) + .Build(); + await reader.StartAsync(cancellationToken); + } + } + finally + { + DeleteStorageBestEffort(storagePath); + } + } + + [Test] + public async Task ExplicitBackendCleanup_DeletesVolumesEvenWhenAutomaticCleanupIsOff( + CancellationToken cancellationToken + ) + { + await WslcTest.SkipIfUnavailableAsync(); + + var storagePath = Path.Combine( + Path.GetTempPath(), + "wslc-explicit-cleanup-" + Guid.NewGuid().ToString("N")[..8] + ); + var volume = "wslc-explicit-" + Guid.NewGuid().ToString("N")[..6]; + try + { + await using var runtime = CreateRuntime(storagePath, cleanUpVolumes: false); + WslContainerBackend backend = new(runtime); + var writer = new ContainerBuilder(backend) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "echo explicit-data > /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test -f /data/file.txt")) + .Build(); + await writer.StartAsync(cancellationToken); + + // The generic explicit cleanup does not depend on the automatic session switch. + await backend.CleanUpAsync(cancellationToken); + await writer.DisposeAsync(); + + await using var reader = new ContainerBuilder(backend) + .WithImage("alpine:latest") + .WithVolumeMount(volume, "/data") + .WithCommand("/bin/sh", "-c", "test ! -f /data/file.txt; sleep 300") + .WithWaitStrategy(Wait.ForCommand("/bin/sh", "-c", "test ! -f /data/file.txt")) + .Build(); + await reader.StartAsync(cancellationToken); + } + finally + { + DeleteStorageBestEffort(storagePath); + } + } + + static WslContainerRuntime CreateRuntime(string storagePath, bool? cleanUpVolumes) => + new( + new WslContainerRuntimeOptions + { + StoragePath = storagePath, + CleanUpVolumesOnTerminate = cleanUpVolumes, + DisableProcessExitCleanup = true, + } + ); + + static void DeleteStorageBestEffort(string path) + { + try + { + Directory.Delete(path, recursive: true); + } + catch (IOException) + { + // The session manager may still hold the store VHD briefly. + } + catch (UnauthorizedAccessException) + { + // Same. + } + } } diff --git a/src/tests/Wsl.UnitTests/ContainerBuilderTests.cs b/src/tests/Wsl.UnitTests/ContainerBuilderTests.cs index db31206..ef45a7d 100644 --- a/src/tests/Wsl.UnitTests/ContainerBuilderTests.cs +++ b/src/tests/Wsl.UnitTests/ContainerBuilderTests.cs @@ -78,6 +78,63 @@ await Assert .Throws(); } + [Test] + public async Task BuildConfiguration_CleanUpEnabledByDefault() + { + // Arrange / Act + var configuration = new TestBuilder().WithImage("alpine").BuildConfig(); + + // Assert + await Assert.That(configuration.EnableCleanUp).IsTrue(); + } + + [Test] + public async Task WithCleanUp_DisablesContainerRemovalOnDispose() + { + // Arrange / Act + var configuration = new TestBuilder().WithImage("alpine").WithCleanUp(false).BuildConfig(); + + // Assert + await Assert.That(configuration.EnableCleanUp).IsFalse(); + } + + [Test] + public async Task WithReuse_DisablesCleanUp() + { + // Arrange / Act + var configuration = new TestBuilder().WithImage("alpine").WithName("retained").WithReuse().BuildConfig(); + + // Assert + await Assert.That(configuration.Reuse).IsTrue(); + await Assert.That(configuration.EnableCleanUp).IsFalse(); + } + + [Test] + public async Task Build_ReuseWithCleanUp_Throws() + { + // Arrange + var builder = new TestBuilder().WithImage("alpine").WithName("retained").WithReuse().WithCleanUp(); + + // Act / Assert + await Assert.That(() => builder.Build()).Throws(); + } + + [Test] + public async Task Build_ReuseWithExplicitCleanUpDisabled_IsAllowed() + { + // Arrange / Act + var configuration = new TestBuilder() + .WithImage("alpine") + .WithName("retained") + .WithReuse() + .WithCleanUp(false) + .BuildConfig(); + + // Assert + await Assert.That(configuration.Reuse).IsTrue(); + await Assert.That(configuration.EnableCleanUp).IsFalse(); + } + [Test] public async Task BuildConfiguration_PopulatesAllCommonFields() {