[BUG] Docker Hub image references are not normalized to one canonical form #275

Summary

ImageReferenceParser produces different registry/repository pairs for spellings that all name the same Docker Hub image: nginx, docker.io/nginx, docker.io/library/nginx, index.docker.io/library/nginx and registry-1.docker.io/library/nginx. Because the image catalog compares registry and repository exactly, one image ends up as several catalog entries, so a shared base image is not recognized as shared, an own (discovered) image is not linked to the containers that run it, and Docker Scout refuses images whose registry is not spelled docker.io. Every Docker Hub reference must be normalized to one canonical form before it reaches the catalog.

Area

• docs/areas/image-catalog.md (added in [Docs] Describe the current behavior by area #263), Image references; also affects docs/areas/base-images.md (shared base images), docs/areas/vulnerabilities.md (Docker Scout, Trivy target) and docs/areas/registries.md.
• Security area: Database and migrations (the fix needs a data correction of existing catalog rows; see Implementation notes).

Current behavior

src/DockerUpdateGuard/Images/ImageReferenceParser.cs:130-146:

if (segments.Length == 1)
{
    registry = "docker.io";
    repository = $"library/{segments[0].ToLowerInvariant()}";
}
else if (LooksLikeRegistry(segments[0]))
{
    registry = segments[0].ToLowerInvariant();
    repository = string.Join('/', segments.Skip(1)).ToLowerInvariant();
}

• The library prefix is only added for a single-segment reference; the Docker Hub host aliases are kept as separate registries.
• ImageCatalogRepository (src/DockerUpdateGuard.Data/Repositories/ImageCatalogRepository.cs:124-125) looks up a registry repository by exact Registry == registry && Repository == repository, so each spelling creates its own RegistryRepositories / ImageVersions rows.
• Registry access already treats the aliases alike (src/DockerUpdateGuard/DockerHub/DockerHubClient.cs:358-363 accepts docker.io, index.docker.io, registry-1.docker.io, …; SplitRepository at :371-381 adds library for a single segment), so tag lookups succeed and the duplicates go unnoticed.
• Writers that parse references: runtime containers (src/DockerUpdateGuard/Images/RuntimeContainerScanOrchestrator.cs:564-582), manual registration (src/DockerUpdateGuard/Images/ImageRegistrationService.cs:61-67), base-image labels (src/DockerUpdateGuard/Images/RegistryBaseImageResolver.cs:94, stored via ImageScanOrchestrator.cs:590), and OciRegistryClient.cs:971 / DockerHubClient.cs:747 for label parsing. Discovery writes docker.io + namespace/name directly (DockerHubClient.cs:312-326), which is already canonical.

Consequences:

1. Shared base images split: image A has the label org.opencontainers.image.base.name=docker.io/nginx:1.27, image B has nginx:1.27. /base-images shows two entries (docker.io/nginx:1.27, docker.io/library/nginx:1.27), each with one observed image, instead of one entry used by two images (grouping key in src/DockerUpdateGuard.Data/Queries/SharedBaseImageQueryService.cs:108-113 is built from the stored registry/repository).
2. Own images not linked: discovery stores docker.io/myuser/app; a container started from index.docker.io/myuser/app:1.0 is stored under registry index.docker.io. LoadOwnRepositoryKeysAsync (RuntimeContainerScanOrchestrator.cs:1155-1167) compares registry/repository, so the container is not recognized as running an own image (no "Own image …" derived-runtime summary, no link on /my-images).
3. Docker Scout unsupported: src/DockerUpdateGuard/Vulnerabilities/DockerScoutVulnerabilityProvider.cs:389 returns unsupported unless the registry equals docker.io, so an image version stored under index.docker.io is never scanned by Docker Scout. TrivyVulnerabilityProvider.GetScanTarget (:183-187) likewise only strips the registry for docker.io.
4. Duplicate vulnerability work: each catalog duplicate is a separate live image version and is scanned separately.

Reproduction: run two containers, nginx:1.27 and docker.io/nginx:1.27 (or index.docker.io/library/nginx:1.27), and run a runtime scan. The container list shows two different image references and two catalog entries for the same image.

Expected behavior

• Docker Hub references have exactly one canonical form: registry docker.io, repository with the library/ namespace for official images (a repository without a namespace), everything lowercase except the tag (tags are case-sensitive and keep their case, as today).
• The registry hosts docker.io, index.docker.io and registry-1.docker.io (any case) are Docker Hub and are normalized to docker.io. (hub.docker.com and docker.com are web hosts, not registry hosts in a reference; they are not added to the parser.)
• After normalization, every spelling in the table above yields docker.io/library/nginx (respectively docker.io/foo/bar), so all of them resolve to the same catalog entries.
• The canonical text form stays registry/repository:tag[@digest]; parsing the canonical form gives back the same parts (round-trip guarantee).
• Mirror-wrapped references keep working: docker.io/ghcr.io/owner/app:1 → ghcr.io/owner/app:1, and index.docker.io/ghcr.io/owner/app:1 behaves the same.
• References of other registries (including localhost:5000/app, ghcr.io/owner/app) are unchanged.
• Existing duplicate catalog entries are reconciled once so that tracked state (observed images, tag selections) points at the canonical entries (see Implementation notes).

Acceptance criteria

• ImageReferenceParser.Parse returns registry docker.io and repository library/nginx for nginx, library/nginx, docker.io/nginx, docker.io/library/nginx, index.docker.io/nginx, index.docker.io/library/nginx, registry-1.docker.io/library/nginx and DOCKER.IO/Nginx, with tag and digest unchanged.
• docker.io/myuser/app, index.docker.io/myuser/app and myuser/app all yield docker.io / myuser/app.
• Mirror unwrapping works after an aliased Docker Hub host (index.docker.io/ghcr.io/owner/app:1 → ghcr.io / owner/app).
• Non-Docker-Hub registries and localhost references are unchanged; the tag keeps its case.
• Format(Parse(x)) is stable: parsing the formatted canonical form gives the same parts.
• Unit tests in src/Tests/DockerUpdateGuard.Tests/ImageReferenceParserTests.cs, e.g. ImageReferenceParserParseDockerIoSingleSegmentAddsLibraryNamespace, ImageReferenceParserParseIndexDockerIoNormalizesRegistry, ImageReferenceParserParseRegistryOneDockerIoNormalizesRegistry, ImageReferenceParserParseWrappedRegistryBehindDockerHubAliasUnwraps, ImageReferenceParserFormatCanonicalReferenceRoundTrips (data-driven tests with [DataRow] are fine).
• A test proves that a runtime container started from index.docker.io/myuser/app:1 is linked to the discovered image docker.io/myuser/app (RuntimeContainerScanOrchestratorTests).
• A test proves that base images named docker.io/nginx:1.27 and nginx:1.27 by two observed images produce one /base-images entry with an observed image count of 2 (SharedBaseImageQueryServiceTests or an orchestrator test).
• The one-time reconciliation of existing data is covered by tests (non-canonical observed image current version and tag selection are re-pointed to the canonical entry; running it twice changes nothing).
• The area document docs/areas/image-catalog.md describes the new behavior and its Known gaps entry for this issue is removed; docs/areas/vulnerabilities.md / docs/areas/base-images.md are updated where they describe the old behavior.

Implementation notes

• Parser: after splitting the segments, map a first segment equal (case-insensitive) to docker.io, index.docker.io or registry-1.docker.io to docker.io; when the registry is docker.io and the remaining repository has a single segment, prefix library/. Do this before NormalizeWrappedRegistry (which only checks docker.io), so unwrapping keeps working for the aliases. A small IsDockerHubRegistryHost helper in the parser keeps the alias list in one place; DockerHubClient.IsSupportedDockerHubHost can stay as it is (it also accepts web hosts).
• ImageCatalogRepository must stay the single entry point for catalog writes and keeps comparing exactly (guarantee in .squad/project.md); normalization belongs in the parser, which every writer already uses. Discovery already writes canonical values.
• Existing data: duplicate rows exist in installations that ran containers or registered images with a non-canonical spelling. Options:
  1. Full merge — re-point every foreign key (container snapshots, image relationships, update findings incl. recommended version, tag candidates, vulnerability findings, tag selections, observed images) to the canonical rows and delete the duplicates. Collides with several unique indexes (relationships per child/base/depth/type, active vulnerability finding per version/advisory/package, image version per repository/tag/digest) and rewrites history; high risk.
  2. Do nothing — new scans write canonical entries; but manual observed images never re-parse their reference (their current version never moves), so they would stay non-canonical forever, and tag selections keyed by the old registry repository would silently stop applying.
  3. Recommended: targeted reconciliation of tracked state. A one-time, idempotent step re-points only the rows that represent current tracked state: ObservedImages.CurrentImageVersionId (get-or-create the canonical image version via IImageCatalogRepository, copying publication time and metadata) and RuntimeContainerTagSelections.RegistryRepositoryId (drop the non-canonical selection if one for the same container already exists on the canonical repository). Historical rows (snapshots, findings, relationships, scan runs) stay on the old entries; they leave the live image version set with the next runtime and observed-image scans, the full vulnerability refresh deactivates their findings and retention removes old snapshots. Catalog entries are never deleted (unchanged rule).
     Implement the step in C# (e.g. a CatalogCanonicalizationService run once at startup after DatabaseMigrator.MigrateAsync, guarded so it is cheap when nothing is left to do) rather than as raw SQL in a migration, because it must use the parser's rules and IImageCatalogRepository, and it is unit-testable with SQLite/InMemory. If a schema change is needed after all, name the migration per AGENTS.md (next is Update8).
• Containers re-parse their reference on every runtime scan, and base-image relationships are rebuilt on the next observed-image scan, so no extra work is needed for them.
• Do not change: the tag's case, the digest normalization, the @digest handling, the integration clients' ExternalOperationResult contract.
• No configuration change; no README change.

Related

• F-046 (P7) — ImageReferenceParser is practically untested with a single test case #10 (ImageReferenceParser untested) — the new tests cover part of it.
• [Docs] Describe the current behavior by area #263 (area documents).

Generated by Claude Code