superseded: add conditional durable-job status validation

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- Owner-scoped durable-job status responses now emit deterministic weak SHA-256 `ETag` validators; ordinary and wildcard `If-None-Match` requests return an empty RFC 9110 `304 Not Modified` response only after authenticated owner-safe lookup, while `Cache-Control: no-store` remains unchanged.
 - Active durable-job status responses now emit an RFC 9110 `Retry-After` delay for `PENDING` and `RUNNING` states, derived from the bounded worker fixed-delay configuration with upward whole-second rounding; terminal states omit the advisory.
 - The durable job pagination index now uses PostgreSQL `CREATE INDEX CONCURRENTLY` with a migration-local Flyway `executeInTransaction=false` companion configuration, preserving production writers and documenting invalid-index recovery and concurrent rollback.
 - Durable job operators can now list only their own jobs through bounded newest-first keyset pagination with canonical opaque cursors, deterministic timestamp-plus-UUID ordering, `Cache-Control: no-store`, and RFC 8288 next-page links without offset drift or cross-tenant existence leakage.
@@ -31,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- Deterministic ordinary, wildcard, changed-state, changed-failure-code, and unrelated-response conditional polling tests, complete controller Javadoc, privacy and rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-polling.md`.
 - Controller-scoped polling advice, deterministic active/terminal lifecycle tests, sub-second rounding coverage, rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-polling.md`.
 - Owner-scoped durable job list models and HTTP contract, strict cursor and page-limit validation, one-extra-row next-page detection, the descriptive `etl_job_owner_pagination_index`, deterministic tenant-isolation and equal-timestamp tests, migration rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-intake.md`.
 - PostgreSQL `FOR UPDATE SKIP LOCKED` durable-job claiming, per-process and per-claim lease fencing, expiry reclaim, bounded attempts, exact-live-lease transitions, terminal payload clearing, stable failure codes, and finite-cardinality worker metrics.
@@ -67,6 +69,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 
+- Conditional status validators are SHA-256 digests of only the complete owner-authorized operator-safe representation; payloads, raw principals, idempotency keys, internal hashes, leases, SQL, and exception text remain excluded, and wildcard evaluation occurs only after owner-safe lookup.
 - Job-list cursors contain only ordering keys, never authority or sensitive values; every page query independently binds the hashed authenticated principal, malformed and non-canonical cursors fail before database access, and list responses exclude payloads, principals, keys, hashes, lease identifiers, SQL, and exception text.
 - Durable-worker metrics and ordinary logs exclude payloads, raw principals, raw idempotency keys, hashes, job and lease identifiers, SQL, exception messages, and unbounded exception labels.
 - Retained payload or response-ledger identity conflicts fail closed with `etl_job_integrity_failure`; an expired or superseded lease rolls back target, ledger, and terminal-state effects.
@@ -273,4 +276,4 @@ This changelog will be updated:
 
 **Changelog Version**: 1.0  
 **Last Updated**: 2026-08-05  
-**Maintained By**: Development Team
+**Maintained By**: Development Team

docs/etl/durable-job-polling.md

@@ -1,18 +1,24 @@
-# Durable job polling advisory
+# Durable job polling contract
 
 ## Purpose
 
-Authenticated clients poll `GET /api/etl/jobs/{job_record_id}` while a durable ETL job is active. Without a machine-readable cadence, independently implemented clients tend to use arbitrary tight loops that waste control-plane capacity and produce inconsistent operator behavior.
+Authenticated clients poll `GET /api/etl/jobs/{job_record_id}` while a durable ETL job is active. Without a machine-readable cadence, independently implemented clients tend to use arbitrary tight loops that waste control-plane capacity and produce inconsistent operator behavior. Without a representation validator, every unchanged poll also retransmits the complete JSON status body.
 
-mightyETL therefore emits the RFC 9110 `Retry-After` response field only when the owner-scoped status representation is `PENDING` or `RUNNING`. The field is advisory: it does not grant authority, alter the job state machine, extend a worker lease, or replace client-side exponential backoff and jitter.
+mightyETL therefore provides two complementary RFC 9110 mechanisms:
+
+- `Retry-After` is emitted only while the owner-scoped status is `PENDING` or `RUNNING`, giving clients a bounded minimum polling delay.
+- A weak `ETag` is emitted on every successful owner-scoped status response, allowing an authenticated client to send `If-None-Match` and receive an empty `304 Not Modified` response when the complete operator-visible representation is unchanged.
+
+Neither mechanism grants authority, alters the job state machine, extends a worker lease, authorizes shared-cache storage, or replaces client-side exponential backoff and jitter.
 
 ## Wire contract
 
-An active job returns the existing operator-safe status representation, `Cache-Control: no-store`, and a whole-second polling delay:
+An active job returns the existing operator-safe status representation, `Cache-Control: no-store`, a weak entity tag, and a whole-second polling delay:
 
 ```http
 HTTP/1.1 200 OK
 Cache-Control: no-store
+ETag: W/"86b79e..."
 Retry-After: 5
 Content-Type: application/json
 
@@ -25,7 +31,25 @@ Content-Type: application/json
 }
 ```
 
-`SUCCEEDED` and `FAILED` are terminal and omit `Retry-After`. Submission responses, list responses, problem responses, and unrelated controllers are unchanged.
+The client can explicitly validate that representation on its next owner-authorized request:
+
+```http
+GET /api/etl/jobs/cf4f083f-8c90-4f34-a8b6-b53761de44ef HTTP/1.1
+Authorization: Bearer <credential>
+If-None-Match: W/"86b79e..."
+```
+
+When the complete representation is unchanged, ordinary entity tags use Spring MVC's RFC 9110 weak comparison and return no representation body:
+
+```http
+HTTP/1.1 304 Not Modified
+Cache-Control: no-store
+ETag: W/"86b79e..."
+```
+
+The RFC wildcard form `If-None-Match: *` is evaluated explicitly after the same authenticated owner-safe lookup. This narrow compatibility handling is required because Spring Framework 6.2.19 does not recognize the wildcard as a matching safe-method validator, even though RFC 9110 requires it to match any selected current representation.
+
+When any represented value changes, including lifecycle state, attempt count, failure code, or update timestamp, the request returns `200 OK`, the current JSON representation, and a new weak tag. `SUCCEEDED` and `FAILED` are terminal and omit `Retry-After` while retaining their validator. Submission responses, list responses, problem responses, and unrelated controllers do not receive a status entity tag.
 
 ## Cadence derivation
 
@@ -41,32 +65,54 @@ Fractional seconds are rounded upward, so one millisecond through one second adv
 
 The worker schedule remains a local execution setting rather than a completion promise. Queue depth, target latency, retries, process restarts, and lease recovery can make a job remain active for multiple polling intervals.
 
+## Entity-tag derivation
+
+The controller first authenticates the caller, parses the opaque UUID, and performs the existing owner-scoped lookup. Only the resulting operator-safe `EtlJobStatusResponse` is eligible for a validator. Every response field is converted to canonical text and then SHA-256 hashed. The hexadecimal digest becomes a weak HTTP entity tag.
+
+Every non-null value receives a value marker and decimal length prefix before its canonical text. A dedicated null marker is distinct from the value marker and a zero-length value, preserving the wire distinction between an omitted nullable field and an explicitly empty string. These markers and length prefixes prevent adjacent field values from creating ambiguous input. Hashing keeps even the operator-safe values out of the header. The tag changes when any represented field changes and is deterministic across replicas for the same representation. It is intentionally weak because the contract validates semantic status equivalence rather than byte-for-byte transfer encoding.
+
+`Cache-Control: no-store` remains authoritative: intermediaries and clients must not persist the tenant-scoped representation for reuse. The validator supports an explicit authenticated conditional request and does not turn the status endpoint into a publicly cacheable resource.
+
 ## Security and privacy boundary
 
-The polling field contains only a bounded integer. It exposes no request payload, authenticated principal, idempotency key, internal hash, job or lease identifier, SQL, exception message, target identity, or queue depth.
+The polling delay contains only a bounded integer. The entity tag contains only a one-way digest of the complete operator-safe status representation. Neither header contains the request payload, raw authenticated principal, idempotency key, internal principal hash, lease identifier, SQL, exception message, target identity, or queue depth.
 
-The response still requires authenticated owner scope. A malformed, absent, or foreign-owned identifier returns the same owner-safe not-found problem and does not receive an active-job polling advisory. The advice is scoped to `EtlJobController` and modifies only `EtlJobStatusResponse` bodies.
+The response still requires authenticated owner scope. A malformed, absent, or foreign-owned identifier returns the same owner-safe not-found problem and receives neither a status validator nor an active-job polling advisory. Conditional evaluation occurs after that lookup, so `If-None-Match`, including `*`, cannot become a cross-principal existence oracle.
+
+The polling advice is scoped to `EtlJobController` and modifies only `EtlJobStatusResponse` bodies. The entity tag is created only by the controller's single-job status method. Submission, list, problem, and unrelated responses remain outside both mechanisms.
 
 ## Client guidance
 
 Clients should:
 
-1. treat `Retry-After` as the minimum delay before the next status request;
-2. add bounded jitter when many jobs are polled concurrently;
-3. apply a larger local backoff after transport failures or `429`/`503` responses;
-4. stop polling when the returned state is `SUCCEEDED` or `FAILED`;
-5. retain their own overall timeout and operator escalation policy.
+1. retain the most recent status `ETag` only in process memory when permitted by their own security policy;
+2. send that value in `If-None-Match` on the next authenticated status request;
+3. treat `304 Not Modified` as confirmation that the previously received representation is still current, not as a new representation;
+4. treat `Retry-After` on a `200` active-status representation as the minimum delay before the next request;
+5. retain the last applicable local cadence when a bodyless `304` does not repeat `Retry-After`;
+6. add bounded jitter when many jobs are polled concurrently;
+7. apply a larger local backoff after transport failures or `429`/`503` responses;
+8. stop polling when the returned state is `SUCCEEDED` or `FAILED`;
+9. retain their own overall timeout and operator escalation policy.
+
+Clients must not interpret either header as a guarantee that the job will finish within one interval or as permission to access another principal's resource.
 
-Clients must not interpret the field as a guarantee that the job will finish within one interval.
+## Compatibility and rollback
 
-## Rollback
+This slice adds no database object, migration, persisted state, API body field, shared cache, or mutation precondition. Existing clients that do not send `If-None-Match` continue receiving the same `200` JSON representation plus ignorable response headers. Conditional clients must tolerate a `200` response with a replacement tag whenever the representation changes or the validator is unavailable after rollback.
 
-The slice adds no database object, migration, persisted state, or API body field. Rolling back the application removes the advisory header while preserving submission, list, status, worker, and lease behavior. Older clients already operate without the header; newer clients must tolerate its absence because HTTP response fields are optional unless a separate client contract makes them mandatory.
+Rolling back the application removes the entity-tag builder and conditional response behavior together with the advisory header if the preceding slice is also rolled back. Submission, list, status body, worker, lease, authorization, and persistence behavior remain intact. No database rollback is required.
 
 ## Standards basis
 
 RFC 9110 Section 10.2.3 defines `Retry-After` as either an HTTP-date or a non-negative decimal number of seconds indicating how long a user agent ought to wait before a follow-up request. mightyETL uses the bounded `delay-seconds` form because it is deterministic, timezone-independent, and directly derived from the configured worker cadence.
 
-### Reference
+RFC 9110 Sections 8.8 and 13.1.2 define entity tags and require weak comparison for `If-None-Match`. For GET and HEAD, a false `If-None-Match` precondition produces `304 Not Modified`. Spring MVC documents that an `ETag` supplied through `ResponseEntity` participates in ordinary conditional request processing and yields an empty `304` response when unchanged. The Spring Framework 6.2.19 primary source also establishes the exact wildcard limitation compensated by the controller.
+
+### References
 
 Fielding, R., Nottingham, M., & Reschke, J. (2022). *HTTP semantics* (RFC 9110). RFC Editor. https://www.rfc-editor.org/rfc/rfc9110
+
+Spring Framework. (2025). *ServletWebRequest.java* (Version 6.2.19) [Source code]. GitHub. https://github.com/spring-projects/spring-framework/blob/v6.2.19/spring-web/src/main/java/org/springframework/web/context/request/ServletWebRequest.java
+
+Spring Framework. (2026). *HTTP caching: Spring Web MVC*. https://docs.spring.io/spring-framework/reference/6.2/web/webmvc/mvc-caching.html

etl-service/src/main/java/com/xtrmetl/etl/controller/EtlJobController.java

@@ -9,11 +9,13 @@
 import com.xtrmetl.etl.job.EtlJobSubmission;
 import com.xtrmetl.etl.service.EtlRequestError;
 import com.xtrmetl.etl.service.EtlRequestException;
+import com.xtrmetl.etl.service.Sha256Digest;
 import io.micrometer.observation.annotation.Observed;
 import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
 import org.springframework.dao.DataAccessException;
 import org.springframework.http.CacheControl;
 import org.springframework.http.HttpHeaders;
+import org.springframework.http.HttpStatus;
 import org.springframework.http.MediaType;
 import org.springframework.http.ResponseEntity;
 import org.springframework.lang.Nullable;
@@ -45,7 +47,9 @@
  *
  * <p>Success and covered failure responses use {@code Cache-Control: no-store}. Malformed, absent,
  * and foreign-owned job identifiers use the same owner-safe not-found classification so the status
- * endpoint does not become a cross-principal existence oracle.</p>
+ * endpoint does not become a cross-principal existence oracle. Successful status responses also
+ * carry a weak entity tag so an authenticated client can explicitly validate an unchanged
+ * representation without authorizing shared-cache persistence.</p>
  */
 @ConditionalOnBooleanProperty(
         prefix = "xtrmetl.etl.jobs",
@@ -177,14 +181,25 @@ public ResponseEntity<EtlJobPageResponse> list(
     /**
      * Returns one status resource only within the authenticated principal namespace.
      *
+     * <p>The response includes a weak {@code ETag} derived from every operator-visible status
+     * field. Spring MVC applies RFC 9110 weak comparison to ordinary {@code If-None-Match} entity
+     * tags. The controller handles the RFC wildcard after the existing owner-safe lookup because
+     * Spring Framework 6.2.19 does not treat {@code If-None-Match: *} as a match for safe methods.
+     * Either matching form returns {@code 304 Not Modified} with no representation body.
+     * Authentication and owner-safe lookup still occur before validation, and
+     * {@code Cache-Control: no-store} remains in force.</p>
+     *
      * @param jobRecordIdText opaque durable job identifier text
+     * @param ifNoneMatch optional conditional request field
      * @param principal authenticated principal namespace
-     * @return operator-safe status representation
+     * @return operator-safe status representation or an empty not-modified response
      */
     @GetMapping("/{jobRecordId}")
     @Observed(name = "etl.jobs.status", contextualName = "etl-job-status")
     public ResponseEntity<EtlJobStatusResponse> status(
             @PathVariable("jobRecordId") String jobRecordIdText,
+            @RequestHeader(value = HttpHeaders.IF_NONE_MATCH, required = false)
+            @Nullable String ifNoneMatch,
             @Nullable Principal principal
     ) {
         if (principal == null) {
@@ -200,9 +215,59 @@ public ResponseEntity<EtlJobStatusResponse> status(
         } catch (RuntimeException exception) {
             throw new EtlUnexpectedException(exception);
         }
+        EtlJobStatusResponse responseBody = EtlJobStatusResponse.from(snapshot);
+        String entityTag = statusEntityTag(responseBody);
+        if ("*".equals(Objects.toString(ifNoneMatch, "").trim())) {
+            return ResponseEntity.status(HttpStatus.NOT_MODIFIED)
+                    .cacheControl(CacheControl.noStore())
+                    .eTag(entityTag)
+                    .build();
+        }
         return ResponseEntity.ok()
                 .cacheControl(CacheControl.noStore())
-                .body(EtlJobStatusResponse.from(snapshot));
+                .eTag(entityTag)
+                .body(responseBody);
+    }
+
+    /**
+     * Builds an opaque weak validator from the complete status representation.
+     *
+     * <p>Each non-null field is marked and length-prefixed before SHA-256 hashing, while a
+     * dedicated marker represents {@code null}. This prevents adjacent-value ambiguity and keeps
+     * an omitted nullable JSON field distinct from an explicitly empty string. The digest prevents
+     * the response header from exposing even the operator-safe values themselves. Payloads,
+     * principals, idempotency keys, lease identifiers, SQL text, and exception text are not part of
+     * the response model and therefore cannot enter this tag.</p>
+     *
+     * @param responseBody complete owner-authorized status representation
+     * @return syntactically valid weak HTTP entity tag
+     */
+    private static String statusEntityTag(EtlJobStatusResponse responseBody) {
+        EtlJobStatusResponse requiredResponse = Objects.requireNonNull(
+                responseBody,
+                "responseBody must not be null"
+        );
+        String canonicalRepresentation = canonicalField(requiredResponse.jobRecordId())
+                + canonicalField(requiredResponse.jobStatus())
+                + canonicalField(requiredResponse.attemptCount())
+                + canonicalField(requiredResponse.failureCode())
+                + canonicalField(requiredResponse.createdAt())
+                + canonicalField(requiredResponse.updatedAt());
+        return "W/\"" + Sha256Digest.digest(canonicalRepresentation) + "\"";
+    }
+
+    /**
+     * Encodes one possibly-null value without delimiter or null/empty ambiguity.
+     *
+     * @param value representation field value
+     * @return a null marker or a value marker followed by length-prefixed canonical text
+     */
+    private static String canonicalField(@Nullable Object value) {
+        if (value == null) {
+            return "N;";
+        }
+        String canonicalValue = value.toString();
+        return "V" + canonicalValue.length() + ":" + canonicalValue;
     }
 
     private static UUID parseJobRecordId(String jobRecordIdText) {