diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8a6e5b44..c5e12622 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
+- Owner-scoped durable-job status responses now emit deterministic weak SHA-256 `ETag` validators; ordinary and wildcard `If-None-Match` requests return an empty RFC 9110 `304 Not Modified` response only after authenticated owner-safe lookup, while `Cache-Control: no-store` remains unchanged.
- Active durable-job status responses now emit an RFC 9110 `Retry-After` delay for `PENDING` and `RUNNING` states only when local worker execution is enabled, derived from the bounded worker fixed-delay configuration with upward whole-second rounding; terminal states and intake-only maintenance mode omit the advisory.
- Durable job operators can now list only their own jobs through bounded newest-first keyset pagination with canonical opaque cursors, deterministic timestamp-plus-UUID ordering, `Cache-Control: no-store`, and RFC 8288 next-page links without offset drift or cross-tenant existence leakage.
- The durable job pagination index uses PostgreSQL `CREATE INDEX CONCURRENTLY` with migration-local Flyway `executeInTransaction=false`, preserving production writers while documenting invalid-index recovery and concurrent rollback.
@@ -37,6 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- Deterministic ordinary, wildcard, changed-state, changed-failure-code, null-versus-empty, and unrelated-response conditional polling tests, complete controller Javadoc, privacy and rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-polling.md`.
- Controller-scoped polling advice, deterministic active/terminal lifecycle tests, disabled-worker fail-closed behavior, sub-second rounding coverage, rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-polling.md`.
- Owner-scoped durable job list models and HTTP contract, strict cursor and page-limit validation, one-extra-row next-page detection, the descriptive `etl_job_owner_pagination_index`, deterministic tenant-isolation and equal-timestamp tests, migration rollback guidance, and APA 7th standards evidence in `docs/etl/durable-job-intake.md`.
- Test-first doctoring for external-wait progress, root-cause analysis, realistic remediation feasibility, exact post-action verification, source-actionable pull-request classification, invalid-stack isolation, and read-only dependency leases in `docs/doctoring/hourly-opencode-nonblocking-progress-evidence.md`.
diff --git a/docs/etl/durable-job-polling.md b/docs/etl/durable-job-polling.md
index d2e72c6b..9e4ac83f 100644
--- a/docs/etl/durable-job-polling.md
+++ b/docs/etl/durable-job-polling.md
@@ -1,23 +1,24 @@
-# Durable job polling advisory
+# Durable job polling contract
## Purpose
-Authenticated clients poll `GET /api/etl/jobs/{job_record_id}` while a durable ETL job is active. Without a machine-readable cadence, independently implemented clients tend to use arbitrary tight loops that waste control-plane capacity and produce inconsistent operator behavior.
+Authenticated clients poll `GET /api/etl/jobs/{job_record_id}` while a durable ETL job is active. Without a machine-readable cadence, independently implemented clients tend to use arbitrary tight loops that waste control-plane capacity and produce inconsistent operator behavior. Without a representation validator, every unchanged poll also retransmits the complete JSON status body.
-mightyETL emits the RFC 9110 `Retry-After` response field only when both conditions hold:
+mightyETL therefore provides two complementary RFC 9110 mechanisms:
-1. the owner-scoped status representation is `PENDING` or `RUNNING`; and
-2. durable-job execution is explicitly enabled.
+- `Retry-After` is emitted only when the owner-scoped status is `PENDING` or `RUNNING` and durable-job execution is explicitly enabled, giving clients a bounded minimum polling delay.
+- A weak `ETag` is emitted on every successful owner-scoped status response, allowing an authenticated client to send `If-None-Match` and receive an empty `304 Not Modified` response when the complete operator-visible representation is unchanged.
-The field is advisory: it does not grant authority, alter the job state machine, extend a worker lease, or replace client-side exponential backoff and jitter. Intake-only maintenance mode does not emit a processing cadence because no local worker is available to advance accepted jobs.
+Neither mechanism grants authority, alters the job state machine, extends a worker lease, authorizes shared-cache storage, or replaces client-side exponential backoff and jitter. Intake-only maintenance mode omits the processing cadence because no local worker is available to advance accepted jobs.
## Wire contract
-An active job with the worker enabled returns the existing operator-safe status representation, `Cache-Control: no-store`, and a whole-second polling delay:
+An active job with the worker enabled returns the existing operator-safe status representation, `Cache-Control: no-store`, a weak entity tag, and a whole-second polling delay:
```http
HTTP/1.1 200 OK
Cache-Control: no-store
+ETag: W/"86b79e..."
Retry-After: 5
Content-Type: application/json
@@ -30,7 +31,25 @@ Content-Type: application/json
}
```
-`SUCCEEDED` and `FAILED` are terminal and omit `Retry-After`. An active job also omits the field when `mightyetl.etl.jobs.worker.enabled=false`, preventing a maintenance-mode intake deployment from advertising a cadence that cannot advance work. Submission responses, list responses, problem responses, and unrelated controllers are unchanged.
+The client can explicitly validate that representation on its next owner-authorized request:
+
+```http
+GET /api/etl/jobs/cf4f083f-8c90-4f34-a8b6-b53761de44ef HTTP/1.1
+Authorization: Bearer Success and covered failure responses use {@code Cache-Control: no-store}. Malformed, absent,
* and foreign-owned job identifiers use the same owner-safe not-found classification so the status
- * endpoint does not become a cross-principal existence oracle.
The response includes a weak {@code ETag} derived from every operator-visible status + * field. Spring MVC applies RFC 9110 weak comparison to ordinary {@code If-None-Match} entity + * tags. The controller handles the RFC wildcard after the existing owner-safe lookup because + * Spring Framework 6.2.19 does not treat {@code If-None-Match: *} as a match for safe methods. + * Either matching form returns {@code 304 Not Modified} with no representation body. + * Authentication and owner-safe lookup still occur before validation, and + * {@code Cache-Control: no-store} remains in force.
+ * * @param jobRecordIdText opaque durable job identifier text + * @param ifNoneMatch optional conditional request field * @param principal authenticated principal namespace - * @return operator-safe status representation + * @return operator-safe status representation or an empty not-modified response */ @GetMapping("/{jobRecordId}") @Observed(name = "etl.jobs.status", contextualName = "etl-job-status") public ResponseEntityEach non-null field is marked and length-prefixed before SHA-256 hashing, while a + * dedicated marker represents {@code null}. This prevents adjacent-value ambiguity and keeps + * an omitted nullable JSON field distinct from an explicitly empty string. The digest prevents + * the response header from exposing even the operator-safe values themselves. Payloads, + * principals, idempotency keys, lease identifiers, SQL text, and exception text are not part of + * the response model and therefore cannot enter this tag.
+ * + * @param responseBody complete owner-authorized status representation + * @return syntactically valid weak HTTP entity tag + */ + private static String statusEntityTag(EtlJobStatusResponse responseBody) { + EtlJobStatusResponse requiredResponse = Objects.requireNonNull( + responseBody, + "responseBody must not be null" + ); + String canonicalRepresentation = canonicalField(requiredResponse.jobRecordId()) + + canonicalField(requiredResponse.jobStatus()) + + canonicalField(requiredResponse.attemptCount()) + + canonicalField(requiredResponse.failureCode()) + + canonicalField(requiredResponse.createdAt()) + + canonicalField(requiredResponse.updatedAt()); + return "W/\"" + Sha256Digest.digest(canonicalRepresentation) + "\""; + } + + /** + * Encodes one possibly-null value without delimiter or null/empty ambiguity. + * + * @param value representation field value + * @return a null marker or a value marker followed by length-prefixed canonical text + */ + private static String canonicalField(@Nullable Object value) { + if (value == null) { + return "N;"; + } + String canonicalValue = value.toString(); + return "V" + canonicalValue.length() + ":" + canonicalValue; } private static UUID parseJobRecordId(String jobRecordIdText) { diff --git a/etl-service/src/test/java/com/xtrmetl/etl/job/EtlJobConditionalStatusTest.java b/etl-service/src/test/java/com/xtrmetl/etl/job/EtlJobConditionalStatusTest.java new file mode 100644 index 00000000..1525d3ac --- /dev/null +++ b/etl-service/src/test/java/com/xtrmetl/etl/job/EtlJobConditionalStatusTest.java @@ -0,0 +1,215 @@ +package com.xtrmetl.etl.job; + +import com.xtrmetl.etl.controller.EtlApiProblemHandler; +import com.xtrmetl.etl.controller.EtlJobController; +import com.xtrmetl.etl.service.EtlRequestError; +import com.xtrmetl.etl.service.EtlRequestException; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.http.HttpHeaders; +import org.springframework.http.MediaType; +import org.springframework.test.web.servlet.MockMvc; +import org.springframework.test.web.servlet.MvcResult; +import org.springframework.test.web.servlet.setup.MockMvcBuilders; + +import java.security.Principal; +import java.time.Instant; +import java.util.UUID; + +import static org.hamcrest.Matchers.equalTo; +import static org.hamcrest.Matchers.not; +import static org.hamcrest.Matchers.startsWith; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.header; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +/** + * Defines conditional-request behavior for owner-scoped durable-job status polling. + */ +class EtlJobConditionalStatusTest { + + private static final String JOBS_PATH = "/api/etl/jobs"; + private static final UUID JOB_RECORD_ID = UUID.fromString( + "cf4f083f-8c90-4f34-a8b6-b53761de44ef" + ); + private static final Principal PRINCIPAL = () -> "tenant_alpha"; + private static final Instant CREATED_AT = Instant.parse("2026-08-05T01:00:00Z"); + private static final Instant UPDATED_AT = Instant.parse("2026-08-05T01:00:05Z"); + + private EtlJobService etlJobService; + private MockMvc mockMvc; + + @BeforeEach + void setUp() { + etlJobService = mock(EtlJobService.class); + mockMvc = MockMvcBuilders + .standaloneSetup(new EtlJobController(etlJobService)) + .setControllerAdvice(new EtlApiProblemHandler()) + .build(); + } + + @Test + void unchangedStatusUsesAWeakEntityTagForAnEmptyNotModifiedResponse() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenReturn(snapshot(EtlJobStatus.PENDING, 0, null, UPDATED_AT)); + + MvcResult initialResult = mockMvc.perform(statusRequest()) + .andExpect(status().isOk()) + .andExpect(header().string(HttpHeaders.CACHE_CONTROL, "no-store")) + .andExpect(header().string(HttpHeaders.ETAG, startsWith("W/\""))) + .andReturn(); + String entityTag = initialResult.getResponse().getHeader(HttpHeaders.ETAG); + assertNotNull(entityTag); + assertTrue(entityTag.endsWith("\"")); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, entityTag)) + .andExpect(status().isNotModified()) + .andExpect(header().string(HttpHeaders.CACHE_CONTROL, "no-store")) + .andExpect(header().string(HttpHeaders.ETAG, entityTag)) + .andExpect(content().string("")); + + verify(etlJobService, times(2)).findOwned(JOB_RECORD_ID, "tenant_alpha"); + } + + @Test + void changedStatusInvalidatesThePriorEntityTag() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenReturn( + snapshot(EtlJobStatus.PENDING, 0, null, UPDATED_AT), + snapshot(EtlJobStatus.RUNNING, 1, null, UPDATED_AT.plusSeconds(5)) + ); + + String priorEntityTag = mockMvc.perform(statusRequest()) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getHeader(HttpHeaders.ETAG); + assertNotNull(priorEntityTag); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, priorEntityTag)) + .andExpect(status().isOk()) + .andExpect(header().string(HttpHeaders.ETAG, not(equalTo(priorEntityTag)))) + .andExpect(jsonPath("$.jobStatus").value("RUNNING")) + .andExpect(jsonPath("$.attemptCount").value(1)); + } + + @Test + void changedFailureCodeInvalidatesTheTagEvenAtTheSameTimestamp() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenReturn( + snapshot(EtlJobStatus.FAILED, 3, "etl_source_failure", UPDATED_AT), + snapshot(EtlJobStatus.FAILED, 3, "etl_target_failure", UPDATED_AT) + ); + + String priorEntityTag = mockMvc.perform(statusRequest()) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getHeader(HttpHeaders.ETAG); + assertNotNull(priorEntityTag); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, priorEntityTag)) + .andExpect(status().isOk()) + .andExpect(header().string(HttpHeaders.ETAG, not(equalTo(priorEntityTag)))) + .andExpect(jsonPath("$.failureCode").value("etl_target_failure")); + } + + @Test + void nullAndEmptyFailureCodesHaveDifferentRepresentationValidators() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenReturn( + snapshot(EtlJobStatus.FAILED, 3, null, UPDATED_AT), + snapshot(EtlJobStatus.FAILED, 3, "", UPDATED_AT) + ); + + String priorEntityTag = mockMvc.perform(statusRequest()) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.failureCode").doesNotExist()) + .andReturn() + .getResponse() + .getHeader(HttpHeaders.ETAG); + assertNotNull(priorEntityTag); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, priorEntityTag)) + .andExpect(status().isOk()) + .andExpect(header().string(HttpHeaders.ETAG, not(equalTo(priorEntityTag)))) + .andExpect(jsonPath("$.failureCode").value("")); + } + + @Test + void wildcardIfNoneMatchRecognizesTheExistingOwnerScopedRepresentation() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenReturn(snapshot(EtlJobStatus.SUCCEEDED, 1, null, UPDATED_AT)); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, "*")) + .andExpect(status().isNotModified()) + .andExpect(header().string(HttpHeaders.ETAG, startsWith("W/\""))) + .andExpect(content().string("")); + + verify(etlJobService).findOwned(JOB_RECORD_ID, "tenant_alpha"); + } + + @Test + void wildcardDoesNotBypassTheOwnerSafeNotFoundBoundary() throws Exception { + when(etlJobService.findOwned(JOB_RECORD_ID, "tenant_alpha")) + .thenThrow(new EtlRequestException(EtlRequestError.JOB_NOT_FOUND)); + + mockMvc.perform(statusRequest().header(HttpHeaders.IF_NONE_MATCH, "*")) + .andExpect(status().isNotFound()) + .andExpect(header().string(HttpHeaders.CACHE_CONTROL, "no-store")) + .andExpect(header().doesNotExist(HttpHeaders.ETAG)) + .andExpect(jsonPath("$.errorCode").value("etl_job_not_found")); + + verify(etlJobService).findOwned(JOB_RECORD_ID, "tenant_alpha"); + } + + @Test + void submissionResponsesDoNotReceiveAStatusEntityTag() throws Exception { + String requestPayload = "[{\"id\":\"record_alpha\"}]"; + String idempotencyKey = "\"550e8400-e29b-41d4-a716-446655440000\""; + when(etlJobService.submit(requestPayload, idempotencyKey, "tenant_alpha")) + .thenReturn(new EtlJobSubmission( + JOB_RECORD_ID, + EtlJobStatus.PENDING, + false + )); + + mockMvc.perform(post(JOBS_PATH) + .principal(PRINCIPAL) + .header("Idempotency-Key", idempotencyKey) + .contentType(MediaType.APPLICATION_JSON) + .content(requestPayload)) + .andExpect(status().isAccepted()) + .andExpect(header().doesNotExist(HttpHeaders.ETAG)); + } + + private static org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder + statusRequest() { + return get(JOBS_PATH + "/" + JOB_RECORD_ID).principal(PRINCIPAL); + } + + private static EtlJobSnapshot snapshot( + EtlJobStatus jobStatus, + int attemptCount, + String failureCode, + Instant updatedAt + ) { + return new EtlJobSnapshot( + JOB_RECORD_ID, + jobStatus, + attemptCount, + failureCode, + CREATED_AT, + updatedAt + ); + } +}