Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Production container builds now use digest-pinned Docker base images while retaining readable Maven/Temurin tags, preventing upstream tag movement from silently changing reviewed build inputs.
- Durable `POST /api/etl/jobs` submissions now return RFC 9110 `202 Accepted`, a stable pending-job representation, `Location` status-monitor metadata, and explicit replay metadata without changing the synchronous `/api/etl/process` contract. The incomplete intake controller is fail-closed and requires explicit `xtrmetl.etl.jobs.intake-enabled=true` operator opt-in until worker execution and terminal payload clearing are implemented.
- Concurrent requests using the same authenticated-principal-scoped semantic idempotency key now return immediate RFC 9457 `409 etl_idempotency_request_in_progress` responses through PostgreSQL `pg_try_advisory_xact_lock`; retries after completion still replay the committed response.
- `POST /api/etl/process` now supports optional authenticated-principal-scoped `Idempotency-Key` retries with atomic target writes, durable response replay, payload-conflict rejection, and explicit replay response metadata.
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM maven:3.9.13-eclipse-temurin-25 AS build
FROM maven:3.9.13-eclipse-temurin-25@sha256:ade3c87e3cdfbe04932afa16b31814cbf60b0122d21d78a76530684a1eeb7cc2 AS build

WORKDIR /workspace

Expand All @@ -18,10 +18,10 @@ RUN mvn -B -DskipTests -pl "${SERVICE}" -am package \
&& mkdir -p /out \
&& cp "${SERVICE}/target/${SERVICE}-"*.jar /out/app.jar

FROM eclipse-temurin:25-jre
FROM eclipse-temurin:25-jre@sha256:681c543d6f36c50f45e9b5226930a46203dcfa351d3670e9d0bdf0dabae53539

WORKDIR /app
COPY --from=build --chown=65532:65532 /out/app.jar /app/app.jar

USER 65532:65532
ENTRYPOINT ["java", "-jar", "/app/app.jar"]
ENTRYPOINT ["java", "-jar", "/app/app.jar"]
67 changes: 67 additions & 0 deletions docs/doctoring/docker-base-image-digest-pinning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Docker base-image digest pinning

Status: active PR; this document does not describe protected `develop` truth until the owning change is integrated.

## Purpose

mightyETL keeps an explicit Maven or Eclipse Temurin tag in each external production `Dockerfile` `FROM` instruction, but a **mutable tag** is an update channel rather than an immutable build identity. An upstream registry can move a tag to different bytes without any mightyETL commit. For audited builds, the readable tag therefore remains for semantic review while a SHA-256 digest binds the selected image content.

The production rule is:

```text
image:explicit-tag@sha256:64-lowercase-hex-characters
```

`latest` and tag-only external base images are rejected by a repository test.

## Image identity boundary

Docker Official Images publish multi-architecture tags. mightyETL binds the declared tag to its **multi-platform index digest**, not to one platform-specific layer or manifest digest. The index is the reviewed registry object from which Docker selects the matching platform manifest. Build and release provenance still need to record the actual build platform and final artifact digest; this source control does not make a cross-platform image byte-identical by itself.

Dated implementation evidence for this PR, revalidated before the production edit:

- `maven:3.9.13-eclipse-temurin-25` — `sha256:ade3c87e3cdfbe04932afa16b31814cbf60b0122d21d78a76530684a1eeb7cc2`;
- `eclipse-temurin:25-jre` — `sha256:681c543d6f36c50f45e9b5226930a46203dcfa351d3670e9d0bdf0dabae53539`.

These values are dated external-input evidence, not timeless architecture. A later reviewed image update is expected to change them.

## Reviewed update procedure

When a base image must change:

1. Resolve the exact intended tag from the official registry again; never copy a stale digest from an old PR, issue, cache, or log.
2. Verify that the digest is the current multi-platform index digest for that exact tag.
3. Review the semantic version change, upstream security posture, compatibility, supported platforms, and relevant licensing/NOTICE implications.
4. Update the human-readable tag and digest together when the intended version changes. If a mutable tag has moved without a desired semantic-version change, the digest change is still an explicit reviewed supply-chain change rather than an invisible refresh.
5. Run the Dockerfile policy test plus the full applicable CI, dependency, SBOM, SAST, security, packaging, and future release-provenance gates on the resulting exact source.
6. Preserve the resulting source SHA, image identity, build platform, final artifact digest, SBOM, and attestation together when #165 implements release provenance.

Digest pinning intentionally stops an upstream tag movement from silently applying a security fix. Base-image vulnerability remediation therefore requires a reviewed digest/tag update; digest pinning is not a substitute for dependency or image vulnerability monitoring.

## Failure and rollback

If a pinned image fails compatibility or operational acceptance, **rollback** means selecting a previously known image identity only after checking that the older digest does not knowingly reintroduce a remediated vulnerability or unsupported runtime. When rollback would restore a known security defect, use a forward update to another supported digest instead. Never remove the digest merely to regain automatic tag movement.

A registry lookup failure during an update is fail-closed: retain the currently reviewed digest and defer the update rather than guessing an image identity. A digest mismatch between documentation, Dockerfile, registry evidence, or produced provenance is an RCA trigger.

## Security and acquisition limits

This control narrows one supply-chain input. It does **not** by itself prove:

- byte-for-byte reproducibility of the final JAR or container image;
- the final image or JAR digest, SBOM, or provenance attestation;
- completeness of Maven vulnerability resolution;
- licensing or NOTICE rights;
- literal-source GitHub Actions execution;
- non-vacuous repository-wide production coverage;
- runtime deployment acceptance or release readiness.

Issue #165 owns the broader reproducible release/provenance boundary. Build-context secret exclusion is separately owned by #213/#214, and legacy runtime bootstrap/artifact cleanup is separately owned by #168/#169. These controls complement one another and must not be collapsed into one green check.

## References — APA 7

Docker. (2026). *Building best practices*. https://docs.docker.com/build/building/best-practices/

Docker. (2026). *Dockerfile reference*. https://docs.docker.com/reference/dockerfile/

Docker. (2026). *Image digests*. https://docs.docker.com/dhi/core-concepts/digests/
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package com.xtrmetl.etl.config;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.junit.jupiter.api.Test;

/**
* Guards the production Dockerfile against mutable external base-image references.
*
* <p>Human-readable tags communicate the intended Maven/JDK/JRE line, while a full SHA-256 image
* digest binds the exact immutable multi-platform image index selected for an audited build. A tag
* without a digest can move upstream without any mightyETL source change and therefore cannot be
* treated as reproducible build input.</p>
*/
class RepositoryDockerBaseImagePolicyTest {

private static final Pattern FROM_INSTRUCTION = Pattern.compile(
"(?im)^\\s*FROM\\s+(?:--platform=\\S+\\s+)?(\\S+)(?:\\s+AS\\s+\\S+)?\\s*$"
);
private static final Pattern TAG_AND_SHA256 = Pattern.compile(
"^[^@\\s]+:[^@\\s]+@sha256:[0-9a-f]{64}$"
);

@Test
void everyExternalDockerBaseImageKeepsReadableTagAndPinsFullSha256Digest() throws IOException {
Path repositoryRoot = findRepositoryRoot(Path.of("").toAbsolutePath().normalize());
assertNotNull(repositoryRoot, "repository root containing Dockerfile and pom.xml must be discoverable");

String dockerfile = Files.readString(repositoryRoot.resolve("Dockerfile"));
List<String> imageReferences = externalBaseImageReferences(dockerfile);
assertFalse(imageReferences.isEmpty(), "production Dockerfile must declare at least one external base image");

for (String imageReference : imageReferences) {
assertTrue(
TAG_AND_SHA256.matcher(imageReference).matches(),
() -> "external Docker base image must use readable tag plus full lowercase sha256 digest: "
+ imageReference
);
assertFalse(
imageReference.toLowerCase(Locale.ROOT).contains(":latest@"),
() -> "external Docker base image must not use latest tag: " + imageReference
);
}
}

@Test
void digestPinningHasSourceBackedSupplyChainDoctoringAndChangelog() throws IOException {
Path repositoryRoot = findRepositoryRoot(Path.of("").toAbsolutePath().normalize());
assertNotNull(repositoryRoot, "repository root containing Dockerfile and pom.xml must be discoverable");

Path doctoringPath = repositoryRoot.resolve("docs/doctoring/docker-base-image-digest-pinning.md");
assertTrue(Files.isRegularFile(doctoringPath), "Docker base-image digest policy must have source-backed doctoring");

String doctoring = Files.readString(doctoringPath);
assertTrue(doctoring.contains("mutable tag"), "doctoring must explain mutable tag risk");
assertTrue(doctoring.contains("multi-platform index digest"), "doctoring must distinguish the index digest boundary");
assertTrue(doctoring.contains("Dockerfile"), "doctoring must bind the decision to the production Dockerfile");
assertTrue(doctoring.contains("rollback"), "doctoring must document rollback/update recovery");
assertTrue(doctoring.contains("APA 7"), "doctoring must identify its reference format");

String changelog = Files.readString(repositoryRoot.resolve("CHANGELOG.md"));
assertTrue(
changelog.contains("digest-pinned Docker base images"),
"CHANGELOG must expose the supply-chain input-integrity change"
);
}

private static List<String> externalBaseImageReferences(String dockerfile) {
List<String> imageReferences = new ArrayList<>();
Matcher matcher = FROM_INSTRUCTION.matcher(dockerfile);
while (matcher.find()) {
imageReferences.add(matcher.group(1));
}
return imageReferences;
}

private static Path findRepositoryRoot(Path start) {
Path candidate = start;
while (candidate != null) {
if (Files.isRegularFile(candidate.resolve("Dockerfile"))
&& Files.isRegularFile(candidate.resolve("pom.xml"))) {
return candidate;
}
candidate = candidate.getParent();
}
return null;
}
}
Loading